iSoftStone Ad Copy/Bing Ad Evaluation Project

Bing Ad Evaluator Project Purpose

The Ad Copy project is Microsoft Bing analogue of Google ads rating project for Appen called the Arrow project (Ads Assessor at Lionbridge). An evaluator gets queries and your job is to evaluate how satisfied users would be with the ads proposed by Bing search engine. The purpose of the project is to help Bing to decide which ads are the most helpful and the most relevant to a
searcher. This, of course, improves the overall quality of ads that Bing presents
to their users. It sounds like a daunting task, and it can be confusing at first.

iSoftstone is another vendor and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done in the HRS platform and you have to complete a qualification for each task. 

Rating Logic

There are however basically only five steps when it comes to determining if the ad
is going to make the user happy by satisfying their needs.

  1. Consider what the user had in their mind when making the query, or their
    intent when they typed the query. You need to take into consideration that
    people also sometimes misspell something when searching and in most cases you
    will understand what they meant.
  2. Do some quick research to understand the meaning of the query unless you
    understand it immediately.
  3. When presented with the ads, determine whether the Ad covers any of the
    user intent and how well it satisfies them
  4. The ad is then given a rating based on a seven-point scale to indicate how
    useful the ad would be to users who type in the query.
  5. Sometimes you need to explain additional information on a rating and explain

If all that still sounds tough, there are some tips that can help as you perform the
rating job:

 When considering the ad, make sure you are reviewing the content that is
displayed. No, you don’t have time to read over the content of an entire
website, but a quick glance at the top-fold of any website will tell you right
away if the ad is relevant or not.

 Often you will need to research possible intents with a query but you
should never base your rating on how the ad is treated in the search

 The rating you provide is based on the query which is the keyword or string
of keywords that a user types into the search engine. You should consider if
it’s a perfect match or not. Ask if the query is broader than the ad or if the
query is narrower than the ad.

 Don’t base your rating on a particular item like the title, description, or the
display URL in the ad.

 Examples are in the guidelines and the guidelines should always be handy
in case you get stuck on something and aren’t sure what to do. Of course
the queries in the guidelines aren’t exhaustive but they do give you a basic
idea on many types of ads that you will run into.

Bing Ad Evaluation project Qualification process

The qualification process for getting into the Ad Copy project for online ad
evaluator are just like most other applications for search engine rating projects.
When you apply at isoftstone for the ad copy project and are selected to go
through the training process, you are sent an email telling you that you’ve been
added to the training group on the system with a login.
It is important to understand what you need to do in order to pass training:

  1. The most important thing that cannot be stressed enough is to read
    through the guidelines as many times as you need to fully understand
  2. Once you feel confident enough to try out what you know, there is a
    system that gives you what are called training hits where you can start to
    judge some ads. Hits are refreshed every so often on the system and you
    need to aim for 70% correct accuracy because this is the requirement to
    continue on and qualify for the project. If you’re not getting 70% then go
    back to the guidelines and read them again.
  3. If you are confirmed by the trainer to be abler to achieve 70% accuracy or
    better, then you are added to the qualification group to take the exam.


Once you pass the training the company will issue you a contract and you can
begin working.

type of tasks and rating scale

When doing the job, the most important thing that we mentioned before was
that you should understand the query and the intent of the user when they were
typing it into the search engine. So you must evaluate it by first understanding
the query meaning and if not then you must do quick research to understand it
before you can determine the query intent. To determine the intent of the query,
you need to consider why the user entered the query and what the results the
user would have liked to see. There are three basic intents; Navigation,
Information, and Transactional like websites with a product or service that can be

The next thing of course would be to evaluate the level of specify and ask
yourself; do the query and the ad match perfectly or not. With that said, you
should always disregard marketing terms like “cheap”, “best”, “highest quality”,
“superior”, “great”, and “greatest”. The meaning of these words is arbitrary
because what may seem “cheap” to one user may not be cheap to another.
The guidelines go over everything from how to understand the ad, how to best
research the ad, and then give you the correct options for rating it. Ads are given
a rating from perfect to bad and plenty of examples are given in the guidelines
that cover just about everything you are likely to face.

Where to apply & requirements

If you are interested in applying for the job, then iSoftstone is currently hiring for
ad evaluators and you can do apply here:

The requirements for getting this job should be a good computer running a
Windows operating system (sorry MAC users) with a high speed internet
connection. You should have some strong computer skills for this job and strong
communication skills when you are evaluating ad copy and have to provide
additional information. You also need to make sure you are fluent in the language
that you applied for and live in the country where you applied as well. The current
job openings want a native proficiency in English and must reside permanently in
the USA.

If given a contract, you are an independent contractor and can usually get 10-25
hours a week if you stay in the accuracy range you need to be in. Currently, the
company doesn’t pay for the training and instead says that it’s a ‘free training’ to
make it seem more attractive. However, the time involved in getting ready for this
does take some hours that should be paid by the company. It could be that
training is not paid because many people fail the exam. So make sure you read
the guidelines like they were the bible.

As always, if you have any questions, then please send me an email or leave a
comment and I will always reply right away and help you with anything you might


One thought on “iSoftStone Ad Copy/Bing Ad Evaluation Project

  1. Currently working as internet assessor at LB. As I want to work as an ad evaluatior at ISS and I applied. Now they sent me guidelines for preparation. I fear I may lose LB as if I take this contract from ISS. Please suggest me whether I proceed with their exam and contract if successful.

Leave a Reply