Ipsoot Project Review From Appen Connect – Pearl and Falcon PT

Ipsoot project 

There is a new project that we’re going to go over today called Ipsoot Project (also known as Pearl and Falcon PT) where you are shown a random post on Facebook, and then you have to decide what topic that post fits under. It is basically a categorizing system job which aims to put the posts that are made into the correct categories of broad topics. 

Ipsoot project qualification 

The qualification process for this project is straight-forward and not difficult provided candidates study and understand project guidelines. The qualification consists of 22 questions, candidates will be asked to apply categorization and assign relevant topic to randomly selected images. 

Ipsoot project Rating process

The posts you receive are random and in no order and appear on the left, including the ad and page information under it such as the page name, who the author is, etc. You will also notice on the top left a Post ID and a Job ID number. 

экрана 2019-08-05 в 20.44.33

The objective when working on the Ipsoot Project is to look at the post and spend 30-60 seconds making sure that you understand it. Of course, it is important to consider the text in the ad along with the media. Basically, you want to think about how a photo might be understood or interpreted in combination with the text in the ad. 

It’s important to pay special attention to everything that is in the ad, including the photo and any text that may be overlaid on the photo as well as any text appearing above the photo. 

If the ad is text-only it’s important to understand that text and including any emojis that might be used in the post to understand what the ad is trying to accomplish and how it should be grouped. 

Sometimes you will see a link in the ad, and at this point, you should click on the link and quickly scan the content of the page it takes you to and consider if the ad itself or the text relates to that landing page. 

Sometimes an ad will contain a video, and you should watch that video for up to 60 seconds. However, if the video is too long then watch the first 10 to 15 seconds, and then click through five spots and watch another 5 to 10 seconds. You should also consider any accompanying text that is inside of the video. 

On the right-hand of the ad, you will see a box that says ‘To which topic(s) is this post related. Select all that apply.’ You should select every topic to which the post is related on the other hand if the post isn’t related to any of the topics then select the no topic checkbox underneath the box of topics.  

Underneath of this, you will see a panel that says ‘Additional Attributes’ and you should select every attribute which applies to the ad. Of course, if there aren’t any attributes that apply to the ad then you should select the no attribute checkbox. And finally, the last thing to do is to click submit which will serve you up another random ad to check. 

Next, to the submit button, you’ll see some skip buttons. You will use these if you encounter any error that prevents you from evaluating the post and they are as follows:

Skip: Tool Error – You should use this if for some reason the tool doesn’t load. This can mean that the ad area will appear blank, or you may not see all the buttons or functions. 

Skip Other Language – the majority of the time you’ll receive ads in the language which you are assigned to so if you work in the United States your ADS will be either English or Spanish depending on what type of position that you apply for and if your multilingual. However, if you do not understand the language in the ad then you should do nothing else but click on Skip: Other Language button.

That’s pretty much it for the tool and it’s quite straight forward. 

The biggest challenge to this type of job is trying to understand Topic Relevance. To give an example, if it would make sense to see the post in an Instagram feed about a certain topic, then the post is probably related to the topic itself. 

However, it is not sufficient for a post to merely mention the topic. While it’s true that some posts explicitly mention the topic, some do not address the topic as it would show up in a search result. 

A good example of this would be an ad about a school competition winner who is pictured with his parents, his sister, and a dog. Simply looking at the ad isn’t going to help you unless you take a look at the text. So you wouldn’t tag this as “animals and pets” simply because there is a picture of a boy and his dog in the photo. However, if that same photo mentioned something about ‘man’s best friend’ you could tag it as animals and pets.

One of the questions that you may have is ‘What if the post is about more than 1 topic?’ Well, you should pay particular attention to a post that can be related to multiple topics because many posts can be coded with more than one category. 

We’ll give you some examples. 

For instance, let’s say that you have a post about a new economic policy. In this case, you would tag ‘politics’ + ‘business finance and economics’.

If a post was about someone using a new fitness routine to improve their health, then you would tag ‘health and medical’ + ‘fitness and workouts’.

A post about a politicians stance and welfare programs you would select ‘politics + social issues’ 

If the post is about the best restaurant three friends found while traveling through Italy then you would tag it ‘food and drink + travel and leisure + relationships friends and family’ 

As you can see, it’s quite possible many posts can have more than 1 topic. 

There are a few other rules when it comes to tagging topics about a post when it comes to regard for quality, professionalism, or the size of the audience for the post. 

We’ll show you some examples of how a topic doesn’t change regardless of these attributes. 

If the post is a professional video of a Justin Timberlake concert then you would tag this under ‘music and audio’.

But let’s say that the post is of a band performing in a bar. You would still tag it ‘music and audio’. Even though that band may not be as popular as Justin Timberlake.

Even if it is a post of a cell phone video of a 3-year old singing in a car, you would still tag this is as ‘music and audio’.

However, it is important to pay special attention to whether the post is related to any entity such as unique people, places or things. For example

The post is about President Trump – You would tag ‘politics’

If the post was about a McDonald’s restaurant, then you would tag ‘Food and drink’

And if the post was about The Beatles, then you would tag ‘Music and audio’

However, you should be careful to not assume that every post that mentions an entity is it about that entity itself. For instance, a video about the Olympics might show a logo of its sponsor, McDonald’s, but that is not enough to consider the post to be related to McDonald’s itself and you wouldn’t tag the post ‘food and drink’. 

When the author of the post is a well-known individual like Barack Obama, and he’s posted something saying that the Chicago Bulls are the best team ever, you would tag this as politics because it’s just a personal opinion by an individual strongly associated with politics. 

If you use Facebook, you probably understand that memes are something that you see quite often and its important to understand how to tag a meme. Memes often used content from one topic to make a statement about another topic. In the case of memes, you would actually tag both topics. For example, if you see a meme of a TV character looking sad with the caption ‘how I feel when my pet walks away from me’ you would tag this TV & Movies + Animals and Pets.

There’s going to be a lot of times when you’re going to have to do research because you’ll find names or phrases in a post that you do not understand. You should always do a search to find out if you’re not sure when referencing a topic. 

It’s also a good idea to identify some of the definitions in the guidelines. 

Topics describe the subject of the content. It is important to determine if the content in a Facebook feed is relevant to the topic. This means that it’s not sufficient to call something ‘related’ to a piece of content that merely mentions the topic. There are some examples of topics and how they should be grouped. And although the guidelines go over each one and give inclusions and exclusions, it is not an exhaustive list but merely examples.


If you are working on the Ipsoot Project then we would like to hear from you and your experiences. If you need help with any information on this page, then email me and I’ll get back to you as soon as possible. 


Exam Preparation Tool For Appen Arrow and Lionbridge Exams


An Exam Preparation Tool has just been created to help you pass these difficult exams you are given when you apply for ads evaluation jobs, you have to pass a qualification exam that is quite difficult. Each company has a name for their ads assessor project. Appen Ads Assessor project is called Arrow if you work for Lionbridge you work on the Ads Assessor project.

Despite all the fancy names, the exam and rating process are the same for these companies because the customer is one, only vendors are different. The biggest problem with this exam is that it is difficult to pass this qualification with only one attempt. In fact, even Appen and Lionbridge explain in a welcome mail that there is a high chance of failing it. With that said, you’re going to need all the help you can get. With the Exam Preparation Tool you can be sure you’re ready.

Keep in mind that unlike Yukon project/Personalized Assessor and Maps Analyst (Shasta) projects’ qualification, the Ads Assessor/Arrow project qualification allows only one attempt to pass the exam.


If you think this is an exaggeration, then just search for comments on passing the exam and see what you will find. You’ll find in a lot of cases people who claim that they studied for several days before the exam began. With that said, it’s just not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort in the one-week time frame they give you to study the guide. The guide itself is designed in a way that assesses your ability to understand and apply rating concepts and the rules based on instructions. There is a lot of material to go over, and it can be a bit confusing. But once again, you’re not going to get anywhere near passing if you don’t start going through the guide.


The tool will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include: Search Ads Evaluation, User to Keywords, User Interests and Close Match Evaluation tasks. The tool has 10 Search Ads Evaluation tasks, 7 User to Keywords tasks, 5 User Interests Evaluation tasks and and 5 Close Match Evaluation tasks, these tasks are the main exam tasks types which cover basically all rating concepts, which will help you to develop rating skills which will be tested during a real exam.


You might be wondering how using this tool can help you pass the exam. The way that it is designed will show you exactly your problem area that you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to disclaim that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from most typical exam tasks, and is designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam.


Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains 1 Search Ads Evaluation task, 1 User to Keywords task, 1 User Interests Evaluation task, and 1 Close Match Evaluation tasks so you can see how it works. Once again, the demo is 100% free and you’re completely anonymous.

Exam Simulation Full Version

The tool has 10 Search Ads Evaluation tasks, 7 User to Keywords tasks, 5 User Interests Evaluation tasks and and 5 Close Match Evaluation tasks, these tasks are the main exam tasks types which cover basically all rating concepts, will allow to develop rating skills which will be tested during a real exam.



iSoftStone Ad Copy Bing Ad Evaluation Project Review

Bing Ad Evaluator Project Purpose

The iSoftStone Ad Copy project is a Microsoft Bing analog of the Google ads rating project for Appen called the Arrow project (Ads Assessor at Lionbridge). An evaluator gets queries and your job is to evaluate how satisfied users would be with the ads proposed by Bing search engine. The purpose of the project is to help Bing to decide which ads are the most helpful and the most relevant to a
searcher. This, of course, improves the overall quality of ads that Bing presents to its users. It sounds like a daunting task, and it can be confusing at first.

iSoftstone is another vendor and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification for the iSoftStone Ad Copy and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done on the HRS platform and you have to complete a qualification for each task. 

Rating Logic In the iSoftStone Ad Copy Project

There are however basically only five steps when it comes to determining if the ad
is going to make the user happy by satisfying their needs.

  1. Consider what the user had in their mind when making the query, or their intent when they typed the query. You need to take into consideration that
    people also sometimes misspell something when searching and in most cases you
    will understand what they meant.
  2. Do some quick research to understand the meaning of the query unless you
    understand it immediately.
  3. When presented with the ads, determine whether the Ad covers any of the
    user intent and how well it satisfies them
  4. The ad is then given a rating based on a seven-point scale to indicate how
    useful the ad would be to users who type in the query.
  5. Sometimes you need to explain additional information on a rating and explain

If all that still sounds tough, there are some tips that can help as you perform the
rating job:

 When considering the ad, make sure you are reviewing the content that is displayed. No, you don’t have time to read over the content of an entire website, but a quick glance at the top-fold of any website will tell you right away if the ad is relevant or not.

 Often you will need to research possible intents with a query but you should never base your rating on how the ad is treated in the search engines.

 The rating you provide is based on the query which is the keyword or string of keywords that a user types into the search engine. You should consider if it’s a perfect match or not. Ask if the query is broader than the ad or if the query is narrower than the ad.

 Don’t base your rating on a particular item like the title, description, or the
display URL in the ad.

 Examples are in the guidelines and the guidelines should always be handy in case you get stuck on something and aren’t sure what to do. Of course,
the queries in the guidelines aren’t exhaustive but they do give you a basic idea on many types of ads that you will run into.

Bing Ad Evaluation project Qualification process

The qualification process for getting into the Ad Copy project for online ad
evaluator is just like most other applications for search engine rating projects.
When you apply at isoftstone for the iSoftStone Ad Copy project and are selected to go
through the training process, you are sent an email telling you that you’ve been
added to the training group on the system with a login.
It is important to understand what you need to do in order to pass training:

  1. The most important thing that cannot be stressed enough is to read
    through the guidelines as many times as you need to fully understand
  2. Once you feel confident enough to try out what you know, there is a
    system that gives you what is called training hits where you can start to judge some ads. Hits are refreshed every so often on the system and you need to aim for 70% correct accuracy because this is the requirement to continue on and qualify for the project. If you’re not getting 70% then go back to the guidelines and read them again.
  3. If you are confirmed by the trainer to be abler to achieve 70% accuracy or
    better, then you are added to the qualification group to take the exam.


Once you pass the training the company will issue you a contract and you can begin working.

type of tasks and rating scale

When doing the job, the most important thing that we mentioned before was that you should understand the query and the intent of the user when they were typing it into the search engine. So you must evaluate it by first understanding the query meaning and if not then you must do quick research to understand it before you can determine the query intent. To determine the intent of the query,
you need to consider why the user entered the query and what the results the user would have liked to see. There are three basic intents; Navigation,
Information, and Transactional like websites with a product or service that can be purchased.

The next thing, of course, would be to evaluate the level of specifying and ask yourself; do the query and the ad match perfectly or not. With that said, you
should always disregard marketing terms like “cheap”, “best”, “highest quality”,
“superior”, “great”, and “greatest”. The meaning of these words is arbitrary because what may seem “cheap” to one user may not be cheap to another.
The guidelines go over everything from how to understand the ad, how to best research the ad, and then give you the correct options for rating it. Ads are given
a rating from perfect to bad and plenty of examples are given in the guidelines
that cover just about everything you are likely to face.

Where to apply & requirements

If you are interested in applying for the job, then iSoftstone is currently hiring for
ad evaluators and you can do apply here: https://issworld.isoftstone.com/JobDescription/tabid/104/jobid/9630/title/Onlin

The requirements for getting this job should be a good computer running a
Windows operating system (sorry MAC users) with a high-speed internet connection. You should have some strong computer skills for this job and strong communication skills when you are evaluating ad copy and have to provide additional information. You also need to make sure you are fluent in the language that you applied for and live in the country where you applied as well. The current
job openings want a native proficiency in English and must reside permanently in
the USA.

If given a contract, you are an independent contractor and can usually get 10-25
hours a week if you stay in the accuracy range you need to be in. Currently, the company doesn’t pay for the training and instead says that it’s a ‘free training’ to make it seem more attractive. However, the time involved in getting ready for this does take some hours that should be paid by the company. It could be that training is not paid because many people fail the exam. So make sure you read the guidelines like they were the bible.

As always, if you have any questions, then please send me an email or leave a
comment and I will always reply right away and help you with anything you might

Appen Arrow Ads/Lionbridge Ads Rating Evaluator Project Review

Appen Arrow and Lionbridge Ads rating projects overview

The Appen Arrow Ads project and Lionbridge Ads rating are similar search engine evaluation project that is using the same Rater Hub platform that the other two large companies Appen and Lionbridge are using to rate search engine results depending on a query or intent of the user. Basically, you are doing the same thing as other search engine evaluators, but the difference is that you are judging the quality of ads and advertising-related data as they are calling it. The feedback that you provide is supposed to help the client monitor and improve their advertising as well as functionality.  

So what you are doing with this certain project is taking a look at different ads on web pages, and using the Rater Hub tool, you answer some questions on your findings. It sounds easy but answering these questions, and the answers that you provide are coming from guidelines that need to be closely followed. It’s not simply about providing your own feedback but providing the feedback based on the guidelines themselves and specific metrics.  

Appen Arrow and Ads Evaluation program purpose 

As a human evaluator, you are not approving or disapproving the ads you evaluate, and you are not deciding directly which ads get shown or don’t get shown. In fact, even though you are providing scores and feedback about individual advertisements and advertisers, your evaluations don’t generally have any direct effect on those advertisements or advertisers. Sometimes the ads you rate are from a long time ago. Sometimes they are in a different form than they originally appeared. Sometimes they were never shown by Google at all, in any form. Basically what you are doing is collecting data to help train the algorithms that will eventually assess the ads being shown on their own. Without human interaction, this would not be possible. So this job is very important to the engineers who are collecting the data from you.  

This is the main reason why it’s very difficult to pass the assessment part of this project. Your judgments are used to determine whether ads are getting better or worse over time, to see whether particular kinds of ads are working well or not, to see what kinds of ads cause problems, and to see what kind of impact a particular change to our systems will have. When Google engineers design new systems or propose changes to existing systems, the human evaluation data you provide is used to judge whether those proposals succeed or fail. This data is fed to computers that train automatic algorithms.  

This is also a big reason that a lot of raters end up being terminated or losing their contracts. Accuracy and quality of your ratings are very important. You should always check the Ads Guidelines if you aren’t sure about something that you are looking at. Even some raters that have been doing this job for a while still refer to the guidelines.  

Appen Arrow and Lionbridge Ads rating program requirements 

Getting on this project can be difficult. There is a long assessment required. Its mentioned early on that you will perform all work and including the assessment tasks on a non-shared and non-public computer. That’s pretty straight forward. You’ve got to have your own computer with a high-speed internet connection. And according to them you can’t take your laptop to the coffee shop and work from there. So it’s a very good possibility that your home IP address is recorded in the RaterHub, and you need to use this same connection to perform all work. Anything else is strictly prohibited. 

Another thing to mention is that you’re going to need to use either Firefox or Chrome web browser but that should hinder anyone since those seem to be the main web browsers being used by the masses. This is because the Rater Hub tool will not work on other browsers.  

If you do pass the assessment, then you may be offered a contract position. Usually, the contract hours are part-time 20 hours a week, but Aztec states 19 hours, Monday through Sunday. Once you complete the assessment, the results are usually sent within 48 business hours after your deadline to complete the assessment has passed, not 48 hours after you submit the assessment.  

As with all projects, we can’t stress enough the importance of one thing. Thoroughly read the guide they send you and have it open and ready while you are taking the tests. Make sure the minute you get the link to review the guide, that going over it is the first thing you do. Give yourself enough time to digest everything in it at least 1-2 times before taking the tests. The guide itself has to be accessed directly through EWOQ which is part of the RaterHub portal. You’ve got to be logged in using the Gmail address you used when you signed up at Appen/Lionbridge. This Gmail account should be a dedicated email account.  

Since you are not able to download the guide, you should bookmark the page with the guidelines so that you can access it quickly and easily. We said that it cannot be mentioned enough and it’s very important that you read and understand the guidelines completely before starting or you will fail the test.  

Appen Arrow and Lionbridge Ads program payment rates 

Wages on each contractual hour depend on the country and can vary between $4 to $15 per hour. For more information on wages read here:

Agents will invoice vendors for rendered services once per calendar month. Fees to independent agents are paid (via direct deposit) within 30 days of receipt of a valid invoice. The payment is made on a monthly basis via direct deposit into your bank account. Appen projects are paid now by Payoneer. If you do not have a Payoneer account yet you can get yours here: 

Appen Arrow and Lionbridge Ads program exam

The first set of tasks are multiple choice questions (in total 45 questions) that come straight from the Search Ad instructions in the Ads Guide itself. After you have completed the comprehension questions, you will be shown a set of assessment tasks (in total 190 tasks of different types: search ads evaluation, user interest, key words, etc) that asks you to rate just the first section of a Search Ads task. Once this is done you’ll get a set that asks you to rate just the second section. Finally, you’ll get a set of complete Search Ads tasks, User to Keyword, Close Match Evaluation to rate.

The typical exam tasks are a query and an ad creative generated by a search engine with respect to the user query. You as a rater will need to apply rating rule to determine how promising or unpromising is the ad with respect to user intent. Once you evaluate the ad then you will be asked to evaluate a landing page associated with the ad and to determine how satisfactory is the page and website content in terms of meeting the user intent needs.

Get Expert Personalized Assistance with Any Project Qualification

Lionbridge Ads rating project/Appen Arrow Exam Tips

Additionally, I would like to give general tips on how to proceed with Aztec/Ads Rating Project qualification:

As usual, before proceeding with rating you need to make sure you clearly understand user intent. If not, use Google search to research the query and establish the dominant intent, and the secondary intent if one exists. Sometimes it’s hard to understand what the user intent is without doing the search yourself;

Search Ads Evaluation Tasks

Select the appropriate query flag if required. For instance, if the query is in a foreign language, or it is clearly a porn related query, make sure you select the appropriate flag. Remember that when you are doing ratings, the language should be in your contracted regional language. So if you are working in English, and you get a page in another language, then flag it appropriately.

Examine the ad creative and answer the question of how promising the ad is. You should refer to a specific section of instructions, and not evaluate the ad creative based on your own perception or personal likes or dislikes. Pay close attention to the wording of the ad and the URL. In many cases, the wording of the ad may indicate the ad deserves a porn flag or has reference to another search engine.

Check the query and ad creative for the navigational flag. If the query is looking for a specific page or website and the ad suggests the website, then the ad and the query should be given a navigational flag.

There are a few scenarios in which you should pay special attention to. The first one is when the ad offers to repeat the search in another search engine. In this case, the appropriate rating for ads and LP (Landing Page) is somewhat unpromising and doesn’t satisfy the query because the action won’t bring the user closer to his goal. Google wants to provide the user with the quickest possible and satisfying solution for the user intent and query. Sending a user to another search engine defeats that purpose entirely.

Select the ad’s creative rating, or how promising the ad is in relation to user intent. Once you’ve completed the first part of the task you will then be directed to the Landing Page section.

Visit the landing page and examine the page based on the guideline rules of how satisfactory the page is based on the query or intent. Once again, you should not use your own judgment but instead, learn how to apply the guidelines rules. In many cases research on the website is required, for instance, a quick product search will show you more information about the page, and help you understand if it meets user intent based on the query.

Make sure to pay special attention to the user location. If you see that the user is looking for something in a specific location, such as a restaurant in a specific location, and you get a restaurant in a completely different location, you should apply a negative rating. In this case, it’s a good idea to refer to the guidelines and review typical user location scenarios.

User to Keywords/User Interest Tasks/Close Match Evaluation

Before proceeding to these types of tasks make sure you read and understand instructions because the logic is different comparing to the approach required for search ads evaluation tasks. You will be asked to rater how much is interesting for a user the suggested page/url based on examination of a user’s current web interest.

Once you have completed all the assessment tasks, including the survey, you will temporarily run out of tasks. This is because your answers will be checked before you will be allowed to move forward. Either you will pass this part or not. You will not see the Acquire Tasks button at this time. Remember that you need to wait 48 hours after the assessment deadline and not 48 hours after you submit. The assessment will be complete after you submit all the tasks available. 


Appen Arrow vs. Lionbridge Ads rating project  

Many people ask me which project is better and which one should I join. In my opinion, I think that Lionbridge is definitely better in terms of offering a better payment rate compared to Appen. It’s also important to note that the exam qualification and training are paid by Lionbridge where at Appen it’s not. So take this into consideration when you apply for both of these companies. Three days of unpaid training is a lot for people to deal with and really when you think about it, no one wants to do an exam for three days, and then fail it having put all the time and effort into it to get a job.  

As always, if you have any questions or thoughts leave a comment or shoot me an email.  

Wishing you great success in your work at home career!

Step-by-Step Guide How to Pass Appen Yukon Project Exam and Lionbridge Internet Assessor, Internet Safety Evaluator Exams

Looking to pass aPPEN Yukon project/lIONBRIDGE iNTERNET Assessor & Internet Safety Evaluator Exams? THEN YOU HAVE COME TO THE RIGHT PLACE FOR all THE INFORMATION THAT YOU NEED
(Updated Jan 2020) 

Before you can start working on various tasks, you must pass the qualification exam. The qualification exam you will take covers two types of rating tasks which will make up 70% of all tasks in the project: Needs Met and Page Quality.

Once your candidature is approved for certification, you will get an exam email with detailed instructions. Within seven days you have to study 160 pages of the general guideline which consists of rating rules and concepts. The guide explains you how to do the rating of search results and assessment of quality of web pages. You should follow these rules and not use your personal judgment. What they want is for you to represent a typical user in your country. Before you proceed with the exam you need to study the guideline at least twice paying particular attention to practical task examples. Keep in mind that it’s 160 pages long. As I always advise, don’t wait till the last minute.

Note: At the stage of examination, Lionbridge provides candidates with very helpful exam preparation support. You will get access as an exam guest to the Lionbridge learning portal where you can listen to webinars and even try to do a few practical tasks before proceeding with actual exam.

Important update: Lionbridge is very selective in terms of recruiting new raters, up on revising exam results they will check  your activity on their exam preparation portal, with this in mind, I strongly recommend you go through all the webinars  on exam preparation portal before proceeding with the exam itself. 

There are three parts in the exam: you can proceed with the next part only if you pass the previous one. Once you complete any part of exam, within 3-10 minutes you will receive an email with information about results.

Google updates the way the exam is done from time to time. The main feature that has been changed is that you must rate pages from your smartphone (send to device tasks). If you do not have smartphone, they say you cannot apply for this job. This is due to the fact that the number of mobile users is constantly growing and the company wants to assess websites from the perspective of on-the-go mobile users.

Note: There is a recommended schedule that will give you plenty of time so that you’re not rushing to complete the exam at the last minute:

  • Day 1 – Study the Guidelines – Make sure to setup your smartphone so that you are ready to take the final portion of the exam.
  • Day 2 – Start the Theory part of the exam making sure you are taking time to CAREFULLY read each question. Some of them can be tricky and that’s meant to make sure you fully understand the concepts.
  • Day 3 – Go back and study the guidelines once again starting with the Page Quality section.
  • Day 3/4 You can start the second part of the exam which is the Page Quality section if you feel comfortable on day 3. You may want to study this entire day, and you’ve got plenty of time if you decide this to take the exam on day 4.
  • Day 5 – Read the Needs Met section of the guidelines again and make sure to take your time. You’ve still got two more days to finish. And as it has been mentioned before, have your guidelines open in case you need to refer back to them.
  • Days 5/6/7 – Get started on the last part of the exam ‘Needs Met Rating’
Internet Assessor and Yukon project exam part 1 (theory)

In this part of the exam you will be assessed on how well you understand the theoretical part of rating concepts, as well as on your ability to interpret questions and search for information in the general guidelines. It is an open book exam, so you can use the general guidelines to find answers to the test questions. This part has 32 question sections; each section has 4 parts, so in fact you have 132 true/false questions to answer. Read questions carefully, some of questions can be tricky in terms of their wording. You can consult with the guideline book while passing the exam and it’s strongly recommended to use this opportunity.

Note: Do not leave incomplete tasks more than for 1 hour, otherwise they will expire and you will fail the exam automatically. If you need to take a break, complete tasks and press “Submit and Stop rating”.

In most cases an hour would be enough to complete the theoretical part of the exam. The following message will be shown to you once you have submitted the last page of questions:

No tasks are currently available. Please try again later. 

Within a few minutes a result notification email will come to your gmail, and in the case you are successful with part 1, the next part will be available in a few minutes minutes.

Lionbridge Internet Assessor and Appen Yukon project exam part 2 (Page Quality)

In this part of the exam you will be asked to assess 7 web pages for their quality. My advice for this part is to check all quality aspects, and fill in the online form very carefully so you can give a correct overall quality score (from lowest to highest) to a page. You should expect 2-3 low quality web pages, 2 high quality web pages and 2-3 medium quality web pages. However this is not a rule, sometimes candidates get 4-5 pages of high or medium quality pages and only 1 or 2 of low quality pages to rate. Keep this in mind. In order to give correct ratings, examine web pages for ETA (Expertise, Authority and Trustworthiness) criteria very carefully.

Also, keep in mind that in this part of exam you need to give a PQ rating mainly to a web page and not to a website, although some quality aspects of a website have to be taken into consideration when assigning overall PQ rating, in particular in case of YMYL (Your Money Or Your Life) content which Google takes very serious.

Do not forget to write useful comments where required, otherwise the system may show an error.

You don’t need to finish this part at once. If you want, you can take a rest. The tasks are still available when you sign in. Again, if you pass this part, the company will send you an e-mail saying that you can proceed with part 3.

Note: The url/web page may not open on your smartphone, in this case do not mark these as “Did Not Load”. During the exam you should not expect to give a website this flag. If you do so, your answer will be incorrect. To solve this issue try to open link using incognito mode on your browser or/and give rating using other sources of information (website reputation, etc).

Снимок экрана 2016-01-10 в 17.25.24
Page Quality Task
Internet Assessor and Yukon project exam part 3 (needs met)

This is probably the most difficult part of the exam, and many candidates fail the exam at this stage. There are 7 tasks, and each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from “Fails to Met” to “Fully Met”.

Internet Assessor and Yukon project exam Passing score

In order to pass this part you have to get an overall score of at least 80%.


Note: while doing Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3, keep in mind that all links have to be opened on your phone because Google analyses web pages from the point of view of mobile users. If you do not have a smartphone, you may use any android emulator for this purpose, in some cases it is even more convenient than using a smartphone for opening links.In order to open the link on your mobile phone, click the  link on the result block on your desktop and then go to www.raterhub.com/url  on your mobile browser  (Google Chrome) to visit the link. You must be logged in your gmail account in order to have access to raterhub platform on your phone. The same rule works if you use android emulator.

Here is an official Leapforce (now Appen) video instruction on how to open links on mobile. Please note that this video was created by Leapforce and uploaded on Youtube by someone.

Note: Some candidates when accessing the link (www.raterhub.com/url ), got informed that Gmail account was unauthorized to visit the link. Most likely reason for this error is the fact of being logged on multiple Gmail accounts on a device. The solution for this problem is to go to android setting -> users & accounts -> add project associated Gmail account, then restart your device. This setting should fix the issue. 

Lionbridge Internet Assessor and Appen Yukon project exam part 3 (Needs Met) Expert Tips and recommendations 

Additionally I would like to give general tips on how to proceed with Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3:

1. Make sure that you fully understand the query and user intent. If you do not fully understand user intent, make additional research on the Internet.

2. When you are evaluating landing pages in the exam, don’t place too much importance on the Page Quality slider. The focus needs to be on Needs Met Ratings. Do not worry about using flags in the exam such as the ‘Upsetting-Offensive Content’ flags.

3. One of the most important things is to absolutely know when to use Fully Meets. Fully Meets ratings are very rare compared to Highly Meets ratings. Avoid being too generous on landing pages. If you have any doubts at all, you should not be rating a page ‘Fully Meets’.

4. Determine what type of query (user intent) is:

o Single entity (business) query with local or non local or both intent/to visit in person/navigate to a website

o Business chain query or query which is looking for business within one category (e.g. car repair service near me)

o Simple know query (where the user requests a specific piece of information, do not assign the Fully Meets rating to any particular result, the correct answer must be present in an SCRB).Examples of simple know queries:paris weather, delta airlines customer service number, current time in Amsterdam)

o Information query seeking a long answer/piece of information (what cases stress)

Types of SCRBs (Special Content Result Blocks): ny weather (simple know query), how old is universe (simple know query), does god exist (long answer seeking), berlin airport (single entity), burger king amsterdam (chain), coffee shops amsterdam (categorical).

o URL/website query

5. Consider the following aspects while giving ratings to result blocks: user location, for local intent queries far away results are useless or slightly relevant. This aspect is very important for the following types of queries: tire shops near me, where to eat near me, Walmart near me, etc. When you get a task with this type of query do use in built map for researching user location/view port and results.

6. Make sure you give correct ratings to special content result blocks for single entity and business category queries. In case of business chain/category query, the result block with a list of businesses is always better than just result for a single business within one category.

7. Consider expertise and trustworthiness of search results for financial, legal, and health related queries; low expertise and trustworthiness automatically reduce needs met rating of the results.

6. Think of dominant, possible common, and minor interpretations for the query and take this aspect into account while giving ratings to a result block. Minor interpretation results should get a low usefulness (Needs Met) rating.

8. Take into account how well the web page/website is optimized for mobile users.

9. Once you have analyzed and considered all listed above aspects, you should give a rating to result blocks one by one.

Note: In some cases there might be some results (links) which are not accessible from outside U.S. (usually websites of well known retailers such as Target, Walmart, etc. They ban access from foreign IPs to prevent hacker attacks). Appen, Raterlabs or Lionbridge do not take into account this factor and therefore rating such results as fails to met may lead to overall fail.

To make sure you do not miss anything before proceeding with part 3, follow pre-exam instructions to pass this part. There will be a link to a video. Watch it and follow the instructions.

Examination Needs Met Task

Note: In part 3 of the exam (Needs Met), most of results blocks have Needs Met and E-A-T (previously PQ) sliders. Main focus should be on providing correct Needs Met rating, which in some cases might be affected by E-T-A (PQ) rating when it comes to YMYL queries and results. Leapforce/Google re-named PQ to E-T-A in part 3 to make it clear that the most important aspects in PQ are expertise, trustworthiness and authority. In general it is similar to PQ concept. While giving E-T-A rating you should evaluate MC under these 3 criteria. In most cases you may do it fast at a first glance approach, but be sure to provide good evaluation for E-T-A when you have YMYL query.

Note: If the user location map does not show user location correctly, and you have a blank page, hit the black arrow button. The task will be closed, but you will see “Incomplete tasks” on the evaluation platform. Click continue and come back to your task with the updated user location field. Remember that it is very important to take into account the user location while giving ratings to block results. Also keep in mind that there is no need to finish the task in one sitting. You can take a break if you need to.

Once you complete 7 tasks you will get this message:

No tasks are currently available. Please try again later.

Yukon (Appen) exam results and Internet Assessor (Lionbridge) exam results

You will get overall exam results on the exam due date from Appen and exam due date plus one day  if your vendor is Lionbridge. Appen will ask you to sign a few electronic forms and in 2-4 days you will be given access to live tasks. At Lionbridge you will have to complete educational course with no pass/no fail quiz before you are granted full access to the tasks.

US YUKON Raterlabs and Lionbridge raters

Once you pass the exam you will then go through the employment process which might take up to 3 weeks. You will be asked to complete an online version of the IRS Form 8850 (Pre-Screening Notice and Certification Request for the Work Opportunity Credit), and a tax credit questionnaire from a vendor payroll provider. After you have completed these two forms, you will be presented with a formal employment offer letter. After acknowledging your offer, you will still need to complete the required activities in a timely manner. Try not to postpone the completion of these important steps: Providing bank details for ACH payments on Paycom website. Providing your I9 Employee Section 1 and I9 Employment Eligibility Documents (section 2) must be completed by specified deadline. It is very important that candidates complete all required steps as soon as possible to avoid termination of your employment.

NEW!!! Welcome to Search Engine Evaluation Exam (Yukon, Internet Assessor) Simulation Tool!
What is The Exam Simulation Tool? 

I have created a special tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

How can this tool can help you pass the exam?

The way the tool is designed will show you exactly your problem area and what you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to mention that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from the most typical exam tasks, and is also designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam. 

How do I get my hands on the exam simulation tool?

Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains a simplified Needs Met and one Page Quality task so you can see how it works. Once again, the demo is 100% free and remain completely anonymous.

Exam Simulation Pro

The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met)  of all the major query types. These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

Explore the tool here: https://examsimulation.passleapforceexam.com/

Yukon and Internet Assessor Expert Support

If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.

Payoneer Payment method for Appen 

Leapforce/Appen/Raterlabs candidate? If you get a job at one of these places, you’re going to need Payoneer account because that is the only payment option available for these companies. Simply sign up for Payoneer under my referral link and you’ll get the simulation tool subscription fee back as Payoneer subscription 25$ bonus once your account is loaded with 1000$! Normally it takes 1-2 months to make 1000$ for EU/USA/Canada raters and 2-4 months for raters from another countries.

The following posts are recommended for reading:

Common Issues of New Raters: Appen Yukon and Lionbridge Internet Safety Evaluator Google Rating Projects


You Failed Appen Yukon Project Lionbridge Internet Assessor Exam Now What?


Exam Simulation Tool For Appen Yukon Project Exam/Lionbridge Internet Assessor & Internet Safety Evaluator Exams

What is The Exam Simulation Tool? 

I have created an exam simulation tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

How can this tool can help you pass the exam?

The way the tool is designed will show you exactly your problem area and what you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to mention that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from the most typical exam tasks, and is also designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam. 

How do I get my hands on the exam simulation tool?

Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains a simplified Needs Met and one Page Quality task so you can see how it works. Once again, the demo is 100% free and remain completely anonymous.

Exam Simulation Pro

The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met)  of all the major query types. These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

Explore the tool here: https://examsimulation.passleapforceexam.com/

Step-by-Step Guide How to Pass Appen Yukon Project Exam and Lionbridge Internet Assessor, Internet Safety Evaluator Exams

Common Issues of New Raters: Appen Yukon and Lionbridge Internet Safety Evaluator Google Rating Projects

In this post I will discuss some common issues that new raters face. If you have other issues, feel free to share by leaving a comment below.

What kind of tasks to expect while providing search engine evaluation service for Appen/Lionbridge? 

If you get a position as a search engine rater with Appen or Lionbridge (Internet Assessor), you may encounter many different tasks. Some of these tasks are your normal rating tasks, and some are side by side, and you may even run into all kinds of experimental tasks. Some of the tasks are easier than others, like experimental tasks. Each task has a different set of instructions, so it’s best to make sure you know exactly what you are doing, and read very carefully each of the task instructions.

Yukon Project Side By Side tasks

Side by Side rating tasks are very interesting and have everything to do with Google algorithm changes. The human role or ‘rating’ in Side By Side tasks is to determine which side, left or right, is better than the other. In this task you are shown blocks on the left and right side. Each block has a slider from Fails to Meet to Fully Meets, and you should choose which one is best for each block depending on the user intent and query. The same guidelines apply for the blocks as they would for a page rating. Meaning that if block meets user intent, it should get the appropriate rating. Once you have chosen the ratings for each block, you will then be asked to choose on a scale which side is better than the other. A comment may be required to give your own opinion on why you’ve chosen which side, and why it is the best over the other side. It’s very important that you base all your ratings on user intent and taking their location in mind. You will also have a quality review on Side by Side tasks  once in two months.

Yukon Project Page Quality tasks

This is the main rating task that a rater is hired to rate. You will be given a website URL along with the first 5 questions which check things like foreign language, which if you’re doing English/USA ratings, would need to be in English to be understood from your location. Pages that don’t load need to be checked, as well as flagging for porn.

The second question is asking you to describe the purpose of the page. The purpose of the page and the user intent are entirely different aspects. Make sure you’re checking the website out to see if it matches user intent, and by looking at the website, you should be able to understand what its main purpose is. If the page is malicious, harmful, or just lacks its purpose for what it is searched for, then you should mark that here. Pages that are simply existing to show ads, without any help to the user, should be flagged. And lastly, it is important to make sure that you understand if the page is a YMYL (Your money or your life) webpage. Understanding this is important because it will change the overall rating of the website.

Once you have completed this, you will then rate the different aspects of the page including its main content quality, and the amount of content. Expertise, authority, and trustworthiness, who is responsible for the website, the supplementary content, page design, layout, and website care and maintenance are all graded on a scale from lowest to highest. The last thing to do is to give the website its overall rating. Usually these types of tasks are allowed 9 minutes to complete. A quality review is done each month to see how you rate. So it’s important to make sure that you are taking the time to evaluate each website properly.

Appen Yukon and Lionbridge Internet Assessor Needs Met tasks

This type of tasks you have already saw while doing part 3 of the qualification exam. Each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from ‘Fails to Met’ to ‘Fully Met’

Other Experimental tasks

Sometimes during rating of side by side, you will see experimental tasks pop up. In most cases experimental tasks are standard page quality or need met tasks, but sometimes you will get new types of tasks. Experimental tasks have a wide range of instructions. It’s important to read the instructions for each task as they will be different each time. Experimental tasks can be very simple, or take a few minutes to complete. They are not always available, and each task can be completely different than the other.

Task Comments 

 A wide variety of tasks require comments; others do not require comments but give you the  option to leave them. While it may seem like comments just put time pressure on you and end up disappearing into a black hole. Your comments help Google analysts understand how you interpret queries and the logic behind your ratings. 

Additionally, while block-level comments are not required, they can make Google aware of different experiences raters may have had with a result. 

The more information we have the better, so please make sure your comments provide an  appropriate amount of information. Here are some basic guidelines for common types of required  comments: 

  • SXS comments always need to include what you think the user intent is, which results are  good/bad and why that is, and what led you to prefer one side over another. 
  • PQ comments should not just repeat grid contents (“All the characteristics in the PQ grid are  high, so the overall page quality is High too”) – rather, they should explain why a page’s reputation/EAT/MC/SC/etc. are good/bad and why, giving a more step-by-step walkthrough of why you gave a particular PQ rating. 
Appen Yukon Project quality feedback & Lionbridge internet assessor monthly quality review

New raters should not do live tasks immediately after getting access to the task portal. It is highly advised that you hold off on rating live tasks if you do not want to be restricted for quality in the next month. Typically tasks which are available for new raters are quality review tasks on which raters are accessed for their quality. Instead, new raters should focus on doing simulation tasks and self-education. Time spent doing this will really pay off, and you wont be restricted for quality in the future. Also, by doing a certain number of simulation tasks correctly you will upgrade your status in the system, and will be allowed to work more hours.

During your participation in the project your service will be evaluated for its quality by means of quality review tasks. In the beginning of every month, a certain number of tasks are uploaded into the system. Normally there are 10-12 tasks of Needs Met or Side by Side, and once in 3 months PQ tasks to be released as quality review tasks. These tasks are quality review tasks up on which your quality is measured every month, in the end of the month you will get your quality score feedback. In case of raters get substandard quality for several consecutive months, most likely such raters will be removed from the rating program.

If your ratings often stand out of general range of ratings provided by other raters in the project, most likely your account will be flagged by the system automatically and your work will be checked manually for quality .

Read more about this topic here:

How to Work at Lionbridge and Appen For a Long Time and Not Get Fired?

recording and invoicing Working time Yukon and Lionbridge internet assessor project

Time management (recording and invoicing working time) of the work you complete can become a problem if you aren’t managing how much time you are spending on tasks. First of all, you should not sacrifice quality for time. In the beginning you may spend more time than the average estimated time (AET) that Appen or Lionbridge allocate to a specific  task, but make sure that in the invoice you put average estimated time that they give you. If you spend less time than average estimated time per a task or set of tasks, be informed that your invoice will be returned for correction, Appen and Lionbridge will ask you to put actual time spent, that is why there is no sense to complete tasks faster than average estimated time. Take your time, do everything correctly and watch your time.

the Yukon Project Extension

Do not forget to log on to the Yukon extension each time you work in the system, otherwise your time spent on doing tasks will not be paid.

According to the vendor policy: any time spent logging on/off, reading, studying, and break times are not billable. Any tasks that are not submitted or are released are also not billable. Please make sure you are not including any of this time on your invoice.

Also keep in mind that consecutive invoices submitted with productivity below 100% may result in removal from the project or you may lose your contract with Appen.

Productivity and Average Estimated Time (AET) 

When it comes to the time spent working on a task, the listed AET is meant to represent the  average amount of time that a task is expected to take. In general, it is best to complete tasks near the given AET, but we understand that some tasks may take a bit longer than others. If more time is needed to complete a task, we encourage you to take it within reason. The overall goal is to complete tasks within 10% of  the given AET. When productivity falls below 90%, a Low Productivity Notice is typically sent out. 

Appen Yukon project Invoice Returned for Missing or Incorrect Data

Lots of new raters receive the following emails from vendors: 

Your invoice has been rejected for missing or incorrect data. Please make the necessary corrections or provide specific details on the reason for each discrepancy.

Time Discrepancy: For the following line item(s) there is a discrepancy between the hours you have submitted and the activity and work recorded on our servers.

Reported: x hours, xx mins
Unaccounted Variance: -xx mins

What raters should do in this situation?

In order to get invoice approved, raters have to deduct xx unaccounted variance minutes from the invoice, in the future make sure only AET is entered into invoice and the Yukon extension has to be on during active rating.

Payment method for Appen, Lionbridge and Raterlabs
HOW to add Already existing Payoneer account at appen connect personal project page
How to earn additional 25$  by signing up to Payoneer via RefeRral Link

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?

Appen Yukon get locked or restricted for Productivity 

Working on some tasks much faster than the AET followed by holding on to a task for much longer than the AET and billing for the full amount on your invoice

Raters are allowed to bill for the AET of a task OR the actual time spent working, whichever is less of the two amounts. It has been discovered that some raters working some tasks much faster than the AET followed by holding on to a task for much longer than the AET and billing for the full amount on invoice, which is a violation of official policy and will not be tolerated.

For example: Rater works 5 tasks, each with an AET of 5 minutes. The first 4 tasks rater completes in 1 minute each, and the 5th task rater holds on to for 30 minutes.

If this pattern continues, invoices will be rejected and raters will be removed from the rating program. Be sure that all invoices are in compliance with policy.

Appen Yukon & Lionbridge project multi-accounts

According to Appen Yukon project policy you can only have one account per IP address, while Lionbridge allows to have one account per household. Those pretty much mean the same thing. There can’t be two accounts coming from the same internet service and this requirement come from Google system requirements.  

When it comes to Google rating projects or Facebook related projects, Appen and Lionbridge can find duplicate accounts very fast, and as a result, you will end up losing your accounts and they will be terminated. You certainly don’t want that to happen.  

Be informed that it is prohibited to have multiple accounts, without special arrangement. Appen and Lionbridge can find duplicate accounts very fast, and as a result, you will lose both accounts.

There are a few things you can do to get around this problem, and arrange them in way that Leapforce, Lionbridge, and Appen won’t be able to detect.  

Multiple Accounts – Is There a Safe Solution? 

IP  address change and network sharing while working in Appen Yukon and Lionbridge projects

In some cases people may share their network, while working as search engine evaluator, this case leads to IP address conflict and removal from the project. Make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.

When rating tasks on raterhub platform you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place and in a country of  your project locale.

Non-completion of a minimum required number of tasks

This issue is very common among new Appen Yukon project agents and may lead to termination of contract. This happens when new agents show no activity in the first month of service. In particular, this can happen if new agents enter the project in the second half of month, and for whatever reasons, they did not complete a minimum required number of tasks. In order to avoid this issue, make sure to show the minimum required activity in the raterhub system.

Yukon and Lionbridge project tasks availability and new rater time restriction

In general, Appen and Lionbridge guarantee at least 2 hours of work per day but task availability may depend on project locale. There might be some days when tasks are available for more than 4 hours. Some locales, like the US project provides raters with 24 hrs/7 days tasks availability.

For the rest market  task availability may be limited. You should understand that the system of task distribution is based on a ‘first come first served’ basis and you need to be ready to catch your portion of tasks when they are uploaded into the system. In order to identify time slots when task availability is the highest please look at task completion graph, usually tasks are uploaded every day into the system at the same time slots.

Lionbridge does not provide raters with a task availability instrument, so raters have to identify time slots when tasks are uploaded into the system by themselves.

Appen also has a system that restricts access to tasks for new raters, and those who are undergoing their quality review. To lift this restriction, you have to complete a certain number of simulation tasks or wait till the next quality review, if your quality score is good, the restriction will be lifted automatically.

Appen Yukon and Lionbridge raters get locked for quality audit or restricted for quality

If you receive an email notifying you that your account has been placed under quality review, this is a good time to identify your weak points and improve your rating skills. Below you will find some tips on how to deal with a “Locked for quality audit” situation.

  1. To speed up the process and to get feedback on your mistakes as quickly as possible, I recommend writing to your vendor and ask why exactly your account has been placed under review. Sometimes the automatic system locks raters accounts without specific reasons. Usually such cases are being resolved within 1-2 days.
  1. While waiting for detailed feedback from your vendor, proceed with doing simulation tasks in order to improve your rating skills. In most cases the issue is related to Side by Side tasks. Make sure that you understand how to deal with this type of task and that you provide well written comments where required. In many cases, raters are placed under quality review due to poorly written comments.
  1. Once you receive the feedback email with your mistakes/quality issues, read it carefully and reply back with an explanation on how you will improve your weak points.
  1. Once your status is moved to “Restricted for quality”, keep in mind that your service is under inspection, thus making sure that you do your best to provide the best possible service.
Appen yukon raters hold status restricted for quality after receiving meets expectations quality feedback

Sometimes some raters do not get their restricted status changed to unrestricted with getting a meets expectations or above quality score. This happens because they were restricted for side by side or PQ tasks quality issues and feedback was released for needs met tasks. In order to lift up all restrictions a rater will have to wait for the next quality review cycle for type of tasks a rater was initially restricted for. If you get meets expectations or an above quality score your account will be unrestricted again.

Have you ever wondered how many hours raters work?

Knowing these statistics can be useful for raters in terms of estimating  possible productivity .

The survey is completely anonymous and once completed will show the overall statistics.

Thanks for participating!

How Raters help to improve Google Search enginE

uTest Software Testing Platform. Can You Make Decent Money Online?

Getting started at uTest

utestIf you would like to obtain a job as a tester, you are probably wondering what sorts of skills does it take to become one. I would definitely recommend starting with some basic tutorial on “how to test”, you can find those on official uTest Youtube channel, and there are couple of good books for beginners: “Software Testing” by Ron Patton; “Agile Testing: A Practical Guide for Testers and Agile Teams” by Lisa Crispin and many others. But to become a good tester these kind of skills you need to have for starters:

– Great English skills (written);
– Analytical Thinking;
– Attention to Detail;
– Thinking Outside the Box;
– Persistence.

Tasks availability

There are no restrictions on amount of work, but it really depends on the platform and your rating as a tester. There are projects that I get invited daily. The project really depends on the hardware you have. Mostly there it’s a web or app based projects. For instance – some hotel website, or booking service. There is so much to test, starting with a web forms, ending with validation of every input field. They are easy for beginners to start finding bugs right away. I worked from US, and got what the platform offered me for the region, but I do know that project can be different depending on country you are working from.

Types of tasks

Usually is’s a whole functionality testing of some elements of the site. For instance: the customer wants a tester to be precise in Log-in page. So the tester focuses on those bugs which occur only on a login page, not considering the other parts of the website. Sometimes there are localization task. Let say you know Chinese, and customer requires the testers to check the spelling and grammatical errors of specific language.Most of the tasks are ad-hoc (exploratory testing) for functional issues. (App crashes, buttons not working, etc).

How to become a successful rater at uTest and make decent money

But there is no deep inside mentioning of a rating system and how it works. Let me explain why is it so important on uTest. This will be a more “in depth” article on earning on uTest. There is this system where it takes your results (amount of submitted bugs) and amount of rejected bugs and counts the successful percentage rate.

Based on this rate there are classifications of testers: Rated; Proven; Bronze; Silver; Gold

And there are different Testing Types, for each of which there is a rating: Functional; Usability; Load; Security; Localization; Automation

But don’t worry, most of the tests are functional and Usability, and polishing skills on these Testing Types can set you with a good additional income for each month.Getting more in those ratings, as I mentioned earlier it is the key filter for uTest to invite you to the project. To become a Gold rated testers it takes quite some time and dedication. Not to mention the quality of your bug reports should be on very high level.

Once you get a couple of rejections it is really hard to achieve the Gold rating. The highest rating gets you not only the more amount of offers, but you also get more paid, and those rare and exclusive projects available only for best testers out there.

Apart from this rating there is also a very cool and interesting opportunity on becoming a “Favorite tester”. Once you have basic rating of Silver or Gold you might end up becoming the one.

Let me explain simply how it works. There is certain person which manages a test cycle a TTL or Testing Team Lead. This person is responsible for other testers to follow the rules and code of conduct of Utest platform, and the project in particular. Sometimes there are even NDA’s which are needed to be signed. TTL is responsible for all this and more.

Let’s assume you have submitted 10 bugs in 1 test cycle, and all of those issues are either very valuable or exceptionally valuable bugs for the customer, Testing Team Lead can easily forward your account to the customer, so that they can give you a “Favorite Tester” status.

What does it mean? Well, you get to be automatically invited to each testing cycle this customer has and many more. Some customers always slot a test case for a favorite tester, which leads to more earnings.

I’ll go into more detail with the favorite tester part and test cases. Why is its important?

How much really you can make of uTest?

Once you become a favorite tester, usually it’s a big project with a big company, and the pay is really good. Some test cases can be up to 50$ per/case.

Once you start doing those cases, simultaneously it is easy and very practical to look for the bugs, (the inconsistencies between an actual and expected results). It really pops in the eye, once you get going through a test case.In that moment when you have found an issue and reported it to the system, apart from getting those 50$ for the test case itself, you can earn a lot more for reporting bugs additionally.

Talking about rates paid for bugs filled… Well, it absolutely depends on the person, the amount of details he/she is giving, the amount of devices the person holds, and the overall work ethics a freelancer sticks up to.

Bugs may vary from 3$ for “Somewhat valuable” to 30-40$ (per bug) for “Exceptionally valuable” issue, depending on the customer.

For sure you are willing to know the real numbers, and the sacramental question needs to be answered.  “How much really you can make of Utest”? I will repeat myself stating that everything depends from the person. But, you can crank out a fair 1000-1500$ per/month of income depending on your dedication.


Everything is done via PayPal and Payoneer. Sometimes the bugs are not approved by customer for as long as 35 days. After that period, they are approved automatically by the system.

Use this link for applying for signing up to Payoneer and we will both benefit by earning 25$, once your account is loaded with payments.



There are many factors on becoming successful on this platform. Speed is one of them.

Sometimes it’s critical to start testing immediately, as other (hungry) testers will beat you to that critical issues, while you’re yawning.

The same rule is applied for the test cases.

Test Cases are usually limited, and not everyone can claim them, so it’s lethal to be fast.

To make the numbers I mentioned earlier, it really takes time to pump your account to a certain rating level. Usually it will take around month or two to get going, but hey, Rome wasn’t built in a day either.Once you become an experienced tester and acquainted with a platform like a “Pro”, you actually might get invited easily on becoming a Testing Team Lead.

Who are Testing Team Leads?

The roles of this person are really dependent of the customer you are working for.

My roles were:

Writing test cases;

Participating in meetings with other TTL’s, Project Managers and Software Delivery Managers;

Create and control a test cycle;

Check and forward bugs for the customer (yes, TTL has the power to approve or decline bugs).

The income story is completely different, as the TTL starts working on per hour rate, and once again it really depends from the customer on the rate. (It is near to a very entry level junior position in software testing working for an onsite company in the States).

The benefit of becoming TTL on one project is that very rapidly you become noticed by others, and easily can get invited to manage some other cycle.In addition, TTL’s are still allowed to do the functional testing as other participants, though some customers forbid this.

In conclusion, now that I completely moved to an onsite job I can say with a share amount of confidence – Utest is a great place to start. It takes that will power, but sure does pay off. Good luck fellow freelancers!