iSoftStone Ad Copy Bing Ad Evaluation Project Review

Bing Ad Evaluator Project Purpose

The iSoftStone Ad Copy project is a Microsoft Bing analog of the Google ads rating project for Appen called the Arrow project (Ads Assessor at Lionbridge). An evaluator gets queries and your job is to evaluate how satisfied users would be with the ads proposed by Bing search engine. The purpose of the project is to help Bing to decide which ads are the most helpful and the most relevant to a
searcher. This, of course, improves the overall quality of ads that Bing presents to its users. It sounds like a daunting task, and it can be confusing at first.

iSoftstone is another vendor and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification for the iSoftStone Ad Copy and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done on the HRS platform and you have to complete a qualification for each task. 

Rating Logic In the iSoftStone Ad Copy Project

There are however basically only five steps when it comes to determining if the ad
is going to make the user happy by satisfying their needs.

  1. Consider what the user had in their mind when making the query, or their intent when they typed the query. You need to take into consideration that
    people also sometimes misspell something when searching and in most cases you
    will understand what they meant.
  2. Do some quick research to understand the meaning of the query unless you
    understand it immediately.
  3. When presented with the ads, determine whether the Ad covers any of the
    user intent and how well it satisfies them
  4. The ad is then given a rating based on a seven-point scale to indicate how
    useful the ad would be to users who type in the query.
  5. Sometimes you need to explain additional information on a rating and explain

If all that still sounds tough, there are some tips that can help as you perform the
rating job:

 When considering the ad, make sure you are reviewing the content that is displayed. No, you don’t have time to read over the content of an entire website, but a quick glance at the top-fold of any website will tell you right away if the ad is relevant or not.

 Often you will need to research possible intents with a query but you should never base your rating on how the ad is treated in the search engines.

 The rating you provide is based on the query which is the keyword or string of keywords that a user types into the search engine. You should consider if it’s a perfect match or not. Ask if the query is broader than the ad or if the query is narrower than the ad.

 Don’t base your rating on a particular item like the title, description, or the
display URL in the ad.

 Examples are in the guidelines and the guidelines should always be handy in case you get stuck on something and aren’t sure what to do. Of course,
the queries in the guidelines aren’t exhaustive but they do give you a basic idea on many types of ads that you will run into.

Bing Ad Evaluation project Qualification process

The qualification process for getting into the Ad Copy project for online ad
evaluator is just like most other applications for search engine rating projects.
When you apply at isoftstone for the iSoftStone Ad Copy project and are selected to go
through the training process, you are sent an email telling you that you’ve been
added to the training group on the system with a login.
It is important to understand what you need to do in order to pass training:

  1. The most important thing that cannot be stressed enough is to read
    through the guidelines as many times as you need to fully understand
  2. Once you feel confident enough to try out what you know, there is a
    system that gives you what is called training hits where you can start to judge some ads. Hits are refreshed every so often on the system and you need to aim for 70% correct accuracy because this is the requirement to continue on and qualify for the project. If you’re not getting 70% then go back to the guidelines and read them again.
  3. If you are confirmed by the trainer to be abler to achieve 70% accuracy or
    better, then you are added to the qualification group to take the exam.


Once you pass the training the company will issue you a contract and you can begin working.

type of tasks and rating scale

When doing the job, the most important thing that we mentioned before was that you should understand the query and the intent of the user when they were typing it into the search engine. So you must evaluate it by first understanding the query meaning and if not then you must do quick research to understand it before you can determine the query intent. To determine the intent of the query,
you need to consider why the user entered the query and what the results the user would have liked to see. There are three basic intents; Navigation,
Information, and Transactional like websites with a product or service that can be purchased.

The next thing, of course, would be to evaluate the level of specifying and ask yourself; do the query and the ad match perfectly or not. With that said, you
should always disregard marketing terms like “cheap”, “best”, “highest quality”,
“superior”, “great”, and “greatest”. The meaning of these words is arbitrary because what may seem “cheap” to one user may not be cheap to another.
The guidelines go over everything from how to understand the ad, how to best research the ad, and then give you the correct options for rating it. Ads are given
a rating from perfect to bad and plenty of examples are given in the guidelines
that cover just about everything you are likely to face.

Where to apply & requirements

If you are interested in applying for the job, then iSoftstone is currently hiring for
ad evaluators and you can do apply here:

The requirements for getting this job should be a good computer running a
Windows operating system (sorry MAC users) with a high-speed internet connection. You should have some strong computer skills for this job and strong communication skills when you are evaluating ad copy and have to provide additional information. You also need to make sure you are fluent in the language that you applied for and live in the country where you applied as well. The current
job openings want a native proficiency in English and must reside permanently in
the USA.

If given a contract, you are an independent contractor and can usually get 10-25
hours a week if you stay in the accuracy range you need to be in. Currently, the company doesn’t pay for the training and instead says that it’s a ‘free training’ to make it seem more attractive. However, the time involved in getting ready for this does take some hours that should be paid by the company. It could be that training is not paid because many people fail the exam. So make sure you read the guidelines like they were the bible.

As always, if you have any questions, then please send me an email or leave a
comment and I will always reply right away and help you with anything you might


  1. Currently working as internet assessor at LB. As I want to work as an ad evaluatior at ISS and I applied. Now they sent me guidelines for preparation. I fear I may lose LB as if I take this contract from ISS. Please suggest me whether I proceed with their exam and contract if successful.

  2. Do you know the task amount of this project in Vietnam? I read that the tasks of this project are available 24/7, however I saw the message No tasks available for the last few hours. Is there any shortage of tasks due to quality or something else?

Leave a Reply