How to Pass Appen (Yukon) and Lionbridge (Internet Assessor & Internet Safety Evaluator) Exams?

How to pass Appen Yukon exam? How to pass Internet Assessor & Internet Safety Lionbridge exam? How to pass Yukon Raterlabs exam? Exam Guidelines 2019

Before you can start working on various tasks, you must pass the qualification exam. The qualification exam you will take covers two types of rating tasks which will make up 70% of all tasks in the project: Needs Met and Page Quality.

Once your candidature is approved for certification, you will get an exam email with detailed instructions. Within seven days you have to study 160 pages of the general guideline which consists of rating rules and concepts. The guide explains you how to do the rating of search results and assessment of quality of web pages. You should follow these rules and not use your personal judgment. What they want is for you to represent a typical user in your country. Before you proceed with the exam you need to study the guideline at least twice paying particular attention to practical task examples. Keep in mind that it’s 160 pages long. As I always advise, don’t wait till the last minute.

Note: At the stage of examination, Lionbridge provides candidates with very helpful exam preparation support. You will get access as an exam guest to the Lionbridge learning portal where you can listen to webinars and even try to do a few practical tasks before proceeding with actual exam.

Important update: Lionbridge is very selective in terms of recruiting new raters, up on revising exam results they will check  your activity on their exam preparation portal, with this in mind, I strongly recommend you go through all the webinars  on exam preparation portal before proceeding with the exam itself. 

There are three parts in the exam: you can proceed with the next part only if you pass the previous one. Once you complete any part of exam, within 3-10 minutes you will receive an email with information about results.

Google updates the way the exam is done from time to time. The main feature that has been changed is that you must rate pages from your smartphone (send to device tasks). If you do not have smartphone, they say you cannot apply for this job. This is due to the fact that the number of mobile users is constantly growing and the company wants to assess websites from the perspective of on-the-go mobile users.

Note: There is a recommended schedule that will give you plenty of time so that you’re not rushing to complete the exam at the last minute:

  • Day 1 – Study the Guidelines – Make sure to setup your smartphone so that you are ready to take the final portion of the exam.
  • Day 2 – Start the Theory part of the exam making sure you are taking time to CAREFULLY read each question. Some of them can be tricky and that’s meant to make sure you fully understand the concepts.
  • Day 3 – Go back and study the guidelines once again starting with the Page Quality section.
  • Day 3/4 You can start the second part of the exam which is the Page Quality section if you feel comfortable on day 3. You may want to study this entire day, and you’ve got plenty of time if you decide this to take the exam on day 4.
  • Day 5 – Read the Needs Met section of the guidelines again and make sure to take your time. You’ve still got two more days to finish. And as it has been mentioned before, have your guidelines open in case you need to refer back to them.
  • Days 5/6/7 – Get started on the last part of the exam ‘Needs Met Rating’
Internet Assessor and Yukon project exam part 1 (theory)

In this part of the exam you will be assessed on how well you understand the theoretical part of rating concepts, as well as on your ability to interpret questions and search for information in the general guidelines. It is an open book exam, so you can use the general guidelines to find answers to the test questions. This part has 32 question sections; each section has 4 parts, so in fact you have 132 true/false questions to answer. Read questions carefully, some of questions can be tricky in terms of their wording. You can consult with the guideline book while passing the exam and it’s strongly recommended to use this opportunity.

Note: Do not leave incomplete tasks more than for 1 hour, otherwise they will expire and you will fail the exam automatically. If you need to take a break, complete tasks and press “Submit and Stop rating”.

In most cases an hour would be enough to complete the theoretical part of the exam. The following message will be shown to you once you have submitted the last page of questions:

No tasks are currently available. Please try again later. 

Within a few minutes a result notification email will come to your gmail, and in the case you are successful with part 1, the next part will be available in a few minutes minutes.

Lionbridge Internet Assessor and Appen Yukon project exam part 2 (Page Quality)

In this part of the exam you will be asked to assess 7 web pages for their quality. My advice for this part is to check all quality aspects, and fill in the online form very carefully so you can give a correct overall quality score (from lowest to highest) to a page. You should expect 2-3 low quality web pages, 2 high quality web pages and 2-3 medium quality web pages. However this is not a rule, sometimes candidates get 4-5 pages of high or medium quality pages and only 1 or 2 of low quality pages to rate. Keep this in mind. In order to give correct ratings, examine web pages for ETA (Expertise, Authority and Trustworthiness) criteria very carefully.

Also, keep in mind that in this part of exam you need to give a PQ rating mainly to a web page and not to a website, although some quality aspects of a website have to be taken into consideration when assigning overall PQ rating, in particular in case of YMYL (Your Money Or Your Life) content which Google takes very serious.

Do not forget to write useful comments where required, otherwise the system may show an error.

You don’t need to finish this part at once. If you want, you can take a rest. The tasks are still available when you sign in. Again, if you pass this part, the company will send you an e-mail saying that you can proceed with part 3.

Note: The url/web page may not open on your smartphone, in this case do not mark these as “Did Not Load”. During the exam you should not expect to give a website this flag. If you do so, your answer will be incorrect. To solve this issue try to open link using incognito mode on your browser or/and give rating using other sources of information (website reputation, etc).

Снимок экрана 2016-01-10 в 17.25.24
Page Quality Task
Internet Assessor and Yukon project exam part 3 (needs met)

This is probably the most difficult part of the exam, and many candidates fail the exam at this stage. There are 7 tasks, and each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from “Fails to Met” to “Fully Met”.

Internet Assessor and Yukon project exam Passing score

In order to pass this part you have to get an overall score of at least 80%.

YUKON EXAM PART 3 (NEEDS MET) SEND TO DEVICE – OPENING LINKS ON MOBILE & MULTIPLE Accounts LOGIN ISSUE

Note: while doing Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3, keep in mind that all links have to be opened on your phone because Google analyses web pages from the point of view of mobile users. If you do not have a smartphone, you may use any android emulator for this purpose, in some cases it is even more convenient than using a smartphone for opening links.In order to open the link on your mobile phone, click the  link on the result block on your desktop and then go to www.raterhub.com/url  on your mobile browser  (Google Chrome) to visit the link. You must be logged in your gmail account in order to have access to raterhub platform on your phone. The same rule works if you use android emulator.

Here is an official Leapforce (now Appen) video instruction on how to open links on mobile. Please note that this video was created by Leapforce and uploaded on Youtube by someone.

Note: Some candidates when accessing the link (www.raterhub.com/url ), got informed that Gmail account was unauthorized to visit the link. Most likely reason for this error is the fact of being logged on multiple Gmail accounts on a device. The solution for this problem is to go to android setting -> users & accounts -> add project associated Gmail account, then restart your device. This setting should fix the issue. 

Lionbridge Internet Assessor and Appen Yukon project exam part 3 (Needs Met) Expert Tips and recommendations 

Additionally I would like to give general tips on how to proceed with Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3:

1. Make sure that you fully understand the query and user intent. If you do not fully understand user intent, make additional research on the Internet.

2. When you are evaluating landing pages in the exam, don’t place too much importance on the Page Quality slider. The focus needs to be on Needs Met Ratings. Do not worry about using flags in the exam such as the ‘Upsetting-Offensive Content’ flags.

3. One of the most important things is to absolutely know when to use Fully Meets. Fully Meets ratings are very rare compared to Highly Meets ratings. Avoid being too generous on landing pages. If you have any doubts at all, you should not be rating a page ‘Fully Meets’.

4. Determine what type of query (user intent) is:

o Single entity (business) query with local or non local or both intent/to visit in person/navigate to a website

o Business chain query or query which is looking for business within one category (e.g. car repair service near me)

o Simple know query (where the user requests a specific piece of information, do not assign the Fully Meets rating to any particular result, the correct answer must be present in an SCRB).Examples of simple know queries:paris weather, delta airlines customer service number, current time in Amsterdam)

o Information query seeking a long answer/piece of information (what cases stress)

Types of SCRBs (Special Content Result Blocks): ny weather (simple know query), how old is universe (simple know query), does god exist (long answer seeking), berlin airport (single entity), burger king amsterdam (chain), coffee shops amsterdam (categorical).

o URL/website query

5. Consider the following aspects while giving ratings to result blocks: user location, for local intent queries far away results are useless or slightly relevant. This aspect is very important for the following types of queries: tire shops near me, where to eat near me, Walmart near me, etc. When you get a task with this type of query do use in built map for researching user location/view port and results.

6. Make sure you give correct ratings to special content result blocks for single entity and business category queries. In case of business chain/category query, the result block with a list of businesses is always better than just result for a single business within one category.

7. Consider expertise and trustworthiness of search results for financial, legal, and health related queries; low expertise and trustworthiness automatically reduce needs met rating of the results.

6. Think of dominant, possible common, and minor interpretations for the query and take this aspect into account while giving ratings to a result block. Minor interpretation results should get a low usefulness (Needs Met) rating.

8. Take into account how well the web page/website is optimized for mobile users.

9. Once you have analyzed and considered all listed above aspects, you should give a rating to result blocks one by one.

Note: In some cases there might be some results (links) which are not accessible from outside U.S. (usually websites of well known retailers such as Target, Walmart, etc. They ban access from foreign IPs to prevent hacker attacks). Appen, Raterlabs or Lionbridge do not take into account this factor and therefore rating such results as fails to met may lead to overall fail.

To make sure you do not miss anything before proceeding with part 3, follow pre-exam instructions to pass this part. There will be a link to a video. Watch it and follow the instructions.

Снимок
Examination Needs Met Task

Note: In part 3 of the exam (Needs Met), most of results blocks have Needs Met and E-A-T (previously PQ) sliders. Main focus should be on providing correct Needs Met rating, which in some cases might be affected by E-T-A (PQ) rating when it comes to YMYL queries and results. Leapforce/Google re-named PQ to E-T-A in part 3 to make it clear that the most important aspects in PQ are expertise, trustworthiness and authority. In general it is similar to PQ concept. While giving E-T-A rating you should evaluate MC under these 3 criteria. In most cases you may do it fast at a first glance approach, but be sure to provide good evaluation for E-T-A when you have YMYL query.

Note: If the user location map does not show user location correctly, and you have a blank page, hit the black arrow button. The task will be closed, but you will see “Incomplete tasks” on the evaluation platform. Click continue and come back to your task with the updated user location field. Remember that it is very important to take into account the user location while giving ratings to block results. Also keep in mind that there is no need to finish the task in one sitting. You can take a break if you need to.

Once you complete 7 tasks you will get this message:

No tasks are currently available. Please try again later.

Yukon (Appen) exam results and Internet Assessor (Lionbridge) exam results

You will get overall exam results on the exam due date from Appen and exam due date plus one day  if your vendor is Lionbridge. Appen will ask you to sign a few electronic forms and in 2-4 days you will be given access to live tasks. At Lionbridge you will have to complete educational course with no pass/no fail quiz before you are granted full access to the tasks.

US YUKON Raterlabs and Lionbridge raters

Once you pass the exam you will then go through the employment process which might take up to 3 weeks. You will be asked to complete an online version of the IRS Form 8850 (Pre-Screening Notice and Certification Request for the Work Opportunity Credit), and a tax credit questionnaire from a vendor payroll provider. After you have completed these two forms, you will be presented with a formal employment offer letter. After acknowledging your offer, you will still need to complete the required activities in a timely manner. Try not to postpone the completion of these important steps: Providing bank details for ACH payments on Paycom website. Providing your I9 Employee Section 1 and I9 Employment Eligibility Documents (section 2) must be completed by specified deadline. It is very important that candidates complete all required steps as soon as possible to avoid termination of your employment.

 
NEW!!! Welcome to Search Engine Evaluation Exam (Yukon, Internet Assessor) Simulation Tool!
What is Yukon exam simulation tool? 

I have created a special tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

How using this tool can help you pass Yukon (Appen) and Internet Assessor (Lionbridge) exam?

You might be wondering how using this tool can help you pass the exam. The way that it is designed will show you exactly your problem area that you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to disclaim that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from most typical exam tasks, and is designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam!

Explore the tool here: examsimulation.passleapforceexam.com

Yukon and Internet Assessor Expert Support

If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.

Payoneer Payment method for Appen 

Leapforce/Appen/Raterlabs candidate? If you get a job at one of these places, you’re going to need Payoneer account because that is the only payment option available for these companies. Simply sign up for Payoneer under my referral link and you’ll get the simulation tool subscription fee back as Payoneer subscription 25$ bonus once your account is loaded with 1000$! Normally it takes 1-2 months to make 1000$ for EU/USA/Canada raters and 2-4 months for raters from another countries.

The following posts are recommended for reading:

Common Issues of New Raters: Appen Yukon and Lionbridge (Internet Safety Evaluator) Google Rating Projects

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest

You Failed Appen Yukon project Lionbridge Internet Assessor Exam – Now What?

What Should You Do If You Fail Appen Yukon or Lionbridge Internet Assessor Exam Twice and How to Re-Apply?

Advertisements

Exam Preparation Tool To Help Candidates To Crack Appen (Yukon) and Lionbridge (Internet Assessor & Internet Safety Evaluator) Exams

Why candidates fail Appen Yukon and Lionbridge exam?

When you apply for search engine evaluator jobs from Appen and  Lionbridge, you have to pass qualification exam that is quite difficult. Each company has a name for their search engine project. Appen/Raterlabs is called Yukon, if you work for Lionbridge you work on the Internet Assessor or Social Media Evaluator.

Despite all the fancy names, the exam  is the same for these companies. The biggest problem with this exam is that it is very hard. In fact, even Lionbridge and Appen explain in a welcome mail that there is high chance of failing it. With that said, you’re going to need all the help you can get.

Studying the guidelines and doing the exam simulation practice are essential in success in passing the real Yukon project exam

If you think this is an exaggeration, then just search for comments on passing the exam simulation and see what you will find. You’ll find in a lot of cases people who claim that they studied for several days before the exam began. With that said, it’s just not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort in the one-week time frame they give you to study the guide. The guide itself is designed in a way that assesses your ability to understand and apply rating concepts and the rules based on the hefty guide. There is a lot of material to go over, and it can be a bit confusing. But once again, you’re not going to get anywhere near passing if you don’t start going through the guide.

What is exam simulation tool? 

I have created a special tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Content Quality tasks, and 5 Search Results Satisfaction  tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

How using this tool can help you pass the exam?

You might be wondering how using this tool can help you pass the exam. The way that it is designed will show you exactly your problem area that you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to disclaim that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from most typical exam tasks, and is designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam!

Welcome to Search Engine Evaluation Exam Simulation Tool!

Explore the tool here: https://examsimulation.passleapforceexam.com/

How to Pass Appen Raterlabs (Yukon) and Lionbridge (Internet Assessor) Exams ? (Exam Guidelines 2019)

Appen Arrow and Lionbridge Ads Rating (Ads Evaluator) Project Review

deep-links-within-mobile-app-engagement-search-adsAppen Arrow and Lionbridge Ads rating projects overview 

The Arrow Ads project and Lionbridge Ads rating are similar search engine evaluation project that is using the same Rater Hub platform that the other two large companies Appen and Lionbridge are using to rate search engine results depending on a query or intent of the user. Basically, you are doing the same thing as other search engine evaluators, but the difference is that you are judging the quality of ads and advertising-related data as they are calling it. Feedback that you provide is supposed to help the client monitor and improve their advertising as well as functionality.  

So what you are doing with this certain project is taking a look at different ads on web pages, and using the Rater Hub tool, you answer some questions on your findings. It sounds easy but answering these questions, and the answers that you provide are coming from guidelines that need to be closely followed. It’s not simply about providing your own feedback, but providing the feedback based on the guidelines themselves and specific metrics.  

Appen Arrow and Ads Evaluation program purpose 

As a human evaluator, you are not approving or disapproving the ads you evaluate, and you are not deciding directly which ads get shown or don’t get shown. In fact, even though you are providing scores and feedback about individual advertisements and advertisers, your evaluations don’t generally have any direct effect on those advertisements or advertisers. Sometimes the ads you rate are from a long time ago. Sometimes they are in a different form than they originally appeared. Sometimes they were never shown by Google at all, in any form. Basically what you are doing is collecting data to help train the algorithms that will eventually assess the ads being shown on their own. Without human interaction this would not be possible. So this job is very important to the engineers who are collecting the data from you.  

This is the main reason why it’s very difficult to pass the assessment part of this project. Your judgments are used to determine whether ads are getting better or worse over time, to see whether particular kinds of ads are working well or not, to see what kinds of ads cause problems, and to see what kind of impact a particular change to our systems will have. When Google engineers design new systems or propose changes to existing systems, the human evaluation data you provide is used to judge whether those proposals succeed or fail. This data is fed to computers that train the automatic algorithms.  

This is also a big reason that a lot of raters end up being terminated or losing their contracts. Accuracy and quality of your ratings are very important. You should always check the Ads Guidelines if you aren’t sure about something that you are looking at. Even some raters that have been doing this job for a while still refer to the guidelines.  

Appen Arrow and Lionbridge Ads rating program requirements 

Getting on this project can be difficult. There is a long assessment required. Its mentioned early on that you will perform all work and including the assessment tasks on a non-shared and non-public computer. That’s pretty straight forward. You’ve got to have your own computer with a high speed internet connection. And according to them you can’t take your laptop to the coffee shop and work from there. So it’s a very good possibility that your home IP address is recorded in the RaterHub, and you need to use this same connection to perform all work. Anything else is strictly prohibited. 

Another thing to mention is that you’re going to need to use either Firefox or Chrome web browser but that should hinder anyone since those seem to be the main web browsers being used by the masses. This is because the Rater Hub tool will not work on other browsers.  

If you do pass the assessment, then you may be offered a contract position. Usually the contract hours are part-time 20 hours a week, but Aztec states 19 hours, Monday through Sunday. Once you complete the assessment, the results are usually sent within 48 business hours after your deadline to complete the assessment has passed, not 48 hours after you submit the assessment.  

As with all projects we can’t stress enough the important of one thing. Thoroughly read the guide they send you and have it open and ready while you are taking the tests. Make sure the minute you get the link to review the guide, that going over it is the first thing you do. Give yourself enough time to digest everything in it at least 1-2 times before taking the tests. The guide itself has to be accessed directly through EWOQ which is part of the RaterHub portal. You’ve got to be logged in using the Gmail address you used when you signed up at Appen/Lionbridge. This Gmail account should be a dedicated email account.  

Since you are not able to download the guide, you should bookmark the page with the guidelines so that you can access it quickly and easily. We said that it cannot be mentioned enough and it’s very important that you read and understand the guidelines completely before starting or you will fail the test.  

Appen Arrow and Lionbridge Ads program payment rates 

Wages on each contractual hour depend on the country and can vary between $4 to $15 per hour. For more information on wages read here:

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Leapforce, Lionbridge and Appen When Combining Several Projects?

Agents will invoice vendors for rendered services once per calendar month. Fees to independent agents are paid (via direct deposit) within 30 days of receipt of a valid invoice. The payment is made on monthly basis via direct deposit into your bank account. Appen projects are paid now by Payoneer. If you do not have a Payoneer account yet you can get yours here: 

http://tracking.payoneer.com/SHRq 

Note: By using this link for applying for a Payoneer account/card we will both benefit by earning 25$. 

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest

Appen Arrow and Lionbridge Ads program exam

The first set of tasks are multiple choice questions (in total 45 questions) that come straight from the Search Ad instructions in the Ads Guide itself. After you have completed the comprehension questions, you will be shown a set of assessment tasks (in total 190 tasks of different types: search ads evaluation, user interest, key words, etc) that asks you to rate just the first section of a Search Ads task. Once this is done you’ll get a set that asks you to rate just the second section. Finally, you’ll get a set of complete Search Ads tasks, User to Keyword, Close Match Evaluation to rate.

The typical exam tasks is a query and an ad creative generated by a search engine in respect to the user query. You as a rater will need to apply rating rule to determine how much promising or unpromising is the ad in respect to user intent. Once you evaluate the ad then you will be asked to evaluate a landing page associated with the ad and to determine how satisfactory is the page and website content in terms of meeting the user intent needs.

Lionbridge Ads rating project/Appen Arrow Exam Tips

Additionally I would like to give general tips on how to proceed with Aztec/Ads Rating Project qualification:

As usual before proceeding with rating you need to make sure you clearly understand user intent. If not, use Google search to research the query and establish the dominant intent, and the secondary intent if one exists. Sometimes it’s hard to understand what the user intent is without doing the search yourself;

Search Ads Evaluation Tasks

Select appropriate query flag if required. For instance, if the query is in foreign language, or it is clearly a porn related query, make sure you select the appropriate flag. Remember that when you are doing ratings, the language should be in your contracted regional language. So if you are working in English, and you get a page in another language, then flag it appropriately.

Examine the ad creative and answer the question of how promising the ad is. You should refer to a specific section of instructions, and not evaluate the ad creative based on your own perception or personal likes or dislikes. Pay close attention to wording of the ad and the URL. In many cases, wording of the ad may indicate the ad deserves a porn flag, or has reference to another search engine.

Check the query and ad creative for the navigational flag. If the query is looking for a specific page or website and the ad suggests the website, then the ad and the query should be given a navigational flag.

There are a few scenarios in which you should pay special attention to. The first one is when the ad offers to repeat the search in another search engine. In this case, the appropriate rating for ads and LP (Landing Page) are somewhat unpromising, and don’t satisfy the query because the action won’t bring the user closer to his goal. Google wants to provide the user with the quickest possible and satisfying solution for the user intent and query. Sending a user to another search engine defeats that purpose entirely.

Select the ads creative rating, or how promising the ad is in relation to user intent. Once you’ve completed the first part of the task you will then be directed to the Landing Page section.

Visit the landing page and examine the page based on the guideline rules of how satisfactory the page is based on the query or intent. Once again, you should not use your own judgement but instead learn how to apply the guidelines rules. In many cases research on the website is required, for instance a quick product search will show you more information about the page, and help you understand if it meets user intent based on the query.

Make sure to pay special attention to the user location. If you see that the user is looking for something in a specific location, such as a restaurant in a specific location, and you get a restaurant in a completely different location, you should apply a negative rating. In this case it’s a good idea to refer to the guidelines and review typical user location scenarios.

User to Keywords/User Interest Tasks/Close Match Evaluation

Before proceeding to these types of tasks make sure you read and understand instructions, because the logic is different comparing to approach required for search ads evaluation tasks. You will be asked to rater how much is interesting for a user the suggested page/url based on examination of a user current web interest.

Once you have completed all assessment tasks, including the survey, you will temporarily run out of tasks. This is because your answers will be checked before you will be allowed to move forward. Either you will pass this part or not. You will not see the Acquire Tasks button at this time. Remember that you need to wait 48 hours after the assessment deadline and not 48 hours after you submit. The assessment will be complete after you submit all tasks available.  

Keep in mind that unlike Yukon project/Personalized Assessor and Maps Analyst (Shasta) projects’ qualifications, the Ads Assessor/Arrow project qualification allows only one attempt to pass the exam. In most cases you will not be allowed to attempt the exam for the second time. 

Arrow and Ads Evaluator Expert Support Available

If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.

Appen Arrow vs. Lionbridge Ads rating project  

Many people ask me which project is better and which one should I join. In my opinion I think that Lionbridge is definitely better in terms of offering a better payment rate compared to Appen. It’s also important to note that the exam qualification and training are paid by Lionbridge where at Appen it’s not. So take this into consideration when you apply for both of these companies. Three days of unpaid training is a lot for people to deal with and really when you think about it, no one wants to do an exam for three days, and then fail it having put all the time and effort into it to get a job.  

As always, if you have any questions or thoughts leave a comment or shoot me an email.  

Wishing you great success in your work at home career!

Common Issues of New Raters (Appen Arrow and Lionbridge Ads Rating Projects)

How To Apply for Search Engine Rater and Social Media Evaluation Jobs at Appen and Lionbridge

Seacrh Engine evaluation jobs Application process – Where to start?

Sometimes finding these types of jobs online takes a bit of research. Most of the big companies that offer these types of jobs don’t usually advertise in common places or online job boards. You usually have to go directly to the company and apply on their website. I have seen that Appen Global advertises their jobs on Flexjobs.com, but that website charges a monthly fee. I never quite understood how a company can charge you to find a job, but they are the exception.

Finding these jobs and applying for them are different for each company. Going to each individual website is the best way, as long as you know where to go, and how to apply. With that being said, this article, and the videos we have provided, will show you how to find the work from home jobs.

One of the other important things to consider when looking for these types of jobs is your location. Some of the contracts that are offered through companies like Appen and Lionbridge require you to live in the demographic area that the project requires itself. The reason for this is because the data from different regions will be different depending on location and the language.

Before you begin to apply for these jobs, or really any job for that matter, it’s probably a good idea to go over your resume and make sure it’s up to date, and that it includes only the important information that is necessary. It might also be to your benefit to have your resume checked by a professional. The resume experts at Resumewriters.com is an awesome service that I have used in the past, and having your resume edited or corrected by a professional is a sure fire way to get noticed and get you the job you want. Many of our readers have had very good results with them.

To visit their website Click Here: Resumewriters.com

Another important note is that if you have worked previously on other social media projects with different companies, don’t mention this on your CV/resume. The two most important factors about this are that you do not need previous experience for these jobs. And second, if you have worked on these types of projects before you will most certainly be looked over and turned down for similar projects. The reason for this is uncertain, but we have seen other people turned down because of this.

You can work on two different projects for more than one company. However, you need to use a different IP address for one or the other. So if you work for Appen and want to work on another project with Lionbridge, you will likely need a different IP address. Appen seems to be the only company that records an IP address and this is shown in the backend or the portal of the work dashboard. This is why it’s a good idea to use a separate IP address for another project. It might also be a good idea to use a different email between companies when applying for and working on these jobs.

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Lionbridge and Appen When Combining Several Projects?

Some advice when working on different projects so that you don’t overwhelm yourself is to get some experience working on one project with a company before trying to get another contract with a separate company. Most of these contracts are part time. So its beneficial to have more than one project at a time if you can handle it.

A really important thing to note here is that you can only have one contract per IP address when it comes to these companies. So if more than one person is working for any of these companies, it is important to have a separate IP address. You can usually order another IP address from your internet service provider for about $4-5 per month extra. Check with your ISP to find out if you require more than 1 IP address per household. You can read more about this in FAQ

Application process at Lionbridge – Ads Assessor, Internet Safety Evaluator, Ads Assessor 

Lionbridge search rater project is called the Google rating project (ads assessor, internet safety evaluator, ads assessor) which is the same as Yukon at Appen (Raterlabs). They also have the positions of Personalized Social Media Assessor, Personalized Internet Assessor, and Personalized Web Assessor.

Application process at Appen – Search Engine Evalutor, Rater, Internet Analyst 

At Appen/Raterlabs you would apply for search engine evaluator position first, and then based on project availability, you would get invites for Yukon, Nile, (Facebook social media evaluator) Truckee or Shasta projects. You can work on more than one project at a time.

Appen Global

Many of my readers will be interested in how to get invited to other projects at Appen Global. Appen Global offers a lot of different types of projects that you are able to work on. Unfortunately, is not likely that you would be contracted to start on projects such as the Thunderbird music project from the beginning. You would be more likely to start on a larger project first such FalconInstaAds or the Arrow project. Once you have started working at Appen, and you are on a Falcon project, like any of the social media projects that they offer, you will likely get an invitation to start on other projects as well. The more you work, and the better your performance is, the more offers you will get from Appen.

Those projects include:

Arrow (ads assessor);

Yukon (search engine evaluator);

Nile/Truckee Facebook/Instagram ads assessor projects (social media evaluator).

application process video instructions -Lionbridge, Appen, Appen Global

We thought it would be a good idea to create some quick walk-through videos to show you how to apply for these jobs and the different companies. Some of these:

Lionbridge, go to http://www.thesmartcrowd.com/workers/job-opportunities/country/russia/#jobs-list

Appen, go  to https://connect.appen.com/qrp/public/jobs/list

Appen Global go here: https://talent-appen.icims.com/jobs/intro

How long does take to hear back from Lionbridge and Appen?

It depends on many factors, sometimes immediately, and sometimes it can take 3-4 months.

It might be that at the moment of your application submission, quota of raters for your country was reached and therefore Appen or Lionbridge does not hire new rater at the moment. As soon as they fire unsuccessful raters, they continue hiring new people to maintain the required number of raters in your country.

iSoftStone Ad Copy/Bing Ad Evaluation Project

Bing Ad Evaluator Project Purpose

The Ad Copy project is Microsoft Bing analogue of Google ads rating project for Appen called the Arrow project (Ads Assessor at Lionbridge). An evaluator gets queries and your job is to evaluate how satisfied users would be with the ads proposed by Bing search engine. The purpose of the project is to help Bing to decide which ads are the most helpful and the most relevant to a
searcher. This, of course, improves the overall quality of ads that Bing presents
to their users. It sounds like a daunting task, and it can be confusing at first.

iSoftstone is another vendor and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done in the HRS platform and you have to complete a qualification for each task. 

Rating Logic

There are however basically only five steps when it comes to determining if the ad
is going to make the user happy by satisfying their needs.

  1. Consider what the user had in their mind when making the query, or their
    intent when they typed the query. You need to take into consideration that
    people also sometimes misspell something when searching and in most cases you
    will understand what they meant.
  2. Do some quick research to understand the meaning of the query unless you
    understand it immediately.
  3. When presented with the ads, determine whether the Ad covers any of the
    user intent and how well it satisfies them
  4. The ad is then given a rating based on a seven-point scale to indicate how
    useful the ad would be to users who type in the query.
  5. Sometimes you need to explain additional information on a rating and explain
    it.

If all that still sounds tough, there are some tips that can help as you perform the
rating job:

 When considering the ad, make sure you are reviewing the content that is
displayed. No, you don’t have time to read over the content of an entire
website, but a quick glance at the top-fold of any website will tell you right
away if the ad is relevant or not.

 Often you will need to research possible intents with a query but you
should never base your rating on how the ad is treated in the search
engines.

 The rating you provide is based on the query which is the keyword or string
of keywords that a user types into the search engine. You should consider if
it’s a perfect match or not. Ask if the query is broader than the ad or if the
query is narrower than the ad.

 Don’t base your rating on a particular item like the title, description, or the
display URL in the ad.

 Examples are in the guidelines and the guidelines should always be handy
in case you get stuck on something and aren’t sure what to do. Of course
the queries in the guidelines aren’t exhaustive but they do give you a basic
idea on many types of ads that you will run into.

Bing Ad Evaluation project Qualification process

The qualification process for getting into the Ad Copy project for online ad
evaluator are just like most other applications for search engine rating projects.
When you apply at isoftstone for the ad copy project and are selected to go
through the training process, you are sent an email telling you that you’ve been
added to the training group on the system with a login.
It is important to understand what you need to do in order to pass training:

  1. The most important thing that cannot be stressed enough is to read
    through the guidelines as many times as you need to fully understand
    them.
  2. Once you feel confident enough to try out what you know, there is a
    system that gives you what are called training hits where you can start to
    judge some ads. Hits are refreshed every so often on the system and you
    need to aim for 70% correct accuracy because this is the requirement to
    continue on and qualify for the project. If you’re not getting 70% then go
    back to the guidelines and read them again.
  3. If you are confirmed by the trainer to be abler to achieve 70% accuracy or
    better, then you are added to the qualification group to take the exam.

IF YOU FAIL THE EXAM MORE THAN TWICE THEN YOU WILL NOT BE ABLE TO
CONTINUE ON…SO MAKE SURE YOU ARE READY!

Once you pass the training the company will issue you a contract and you can
begin working.

type of tasks and rating scale

When doing the job, the most important thing that we mentioned before was
that you should understand the query and the intent of the user when they were
typing it into the search engine. So you must evaluate it by first understanding
the query meaning and if not then you must do quick research to understand it
before you can determine the query intent. To determine the intent of the query,
you need to consider why the user entered the query and what the results the
user would have liked to see. There are three basic intents; Navigation,
Information, and Transactional like websites with a product or service that can be
purchased.

The next thing of course would be to evaluate the level of specify and ask
yourself; do the query and the ad match perfectly or not. With that said, you
should always disregard marketing terms like “cheap”, “best”, “highest quality”,
“superior”, “great”, and “greatest”. The meaning of these words is arbitrary
because what may seem “cheap” to one user may not be cheap to another.
The guidelines go over everything from how to understand the ad, how to best
research the ad, and then give you the correct options for rating it. Ads are given
a rating from perfect to bad and plenty of examples are given in the guidelines
that cover just about everything you are likely to face.

Where to apply & requirements

If you are interested in applying for the job, then iSoftstone is currently hiring for
ad evaluators and you can do apply here: https://issworld.isoftstone.com/JobDescription/tabid/104/jobid/9630/title/Onlin
e%20Ad%20Evaluator%20-%20United%20States/Default.aspx?language=en-US

The requirements for getting this job should be a good computer running a
Windows operating system (sorry MAC users) with a high speed internet
connection. You should have some strong computer skills for this job and strong
communication skills when you are evaluating ad copy and have to provide
additional information. You also need to make sure you are fluent in the language
that you applied for and live in the country where you applied as well. The current
job openings want a native proficiency in English and must reside permanently in
the USA.

If given a contract, you are an independent contractor and can usually get 10-25
hours a week if you stay in the accuracy range you need to be in. Currently, the
company doesn’t pay for the training and instead says that it’s a ‘free training’ to
make it seem more attractive. However, the time involved in getting ready for this
does take some hours that should be paid by the company. It could be that
training is not paid because many people fail the exam. So make sure you read
the guidelines like they were the bible.

As always, if you have any questions, then please send me an email or leave a
comment and I will always reply right away and help you with anything you might
need.

Lionbridge & Appen Connect Platform – List of Projects and Code Names

As you may know Appen acquired Leapforce in 2017, since then the company has decided to close it is own project management platform known as Appen Global and move all projects to Appen connect. The group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform.

Taking into account the fact that all projects have code names and little description is provided, many candidates might be lost in terms what project to go for. 

Bellow you will find a list of most popular projects and links to more info about each project:

Appen connect
Project code nameDescription Reference for more info
YukonGoogle Search Engine Evaluation Project Click here
ArrowGoogle Ads Evaluation Project Click here
ShastaMaps Evaluation Project

Provide relevance and accuracy ratings for one of the world's largest and most popular mapping applications.
The job has an ABOVE average volume of work.
Click here
Echo Apple iTunes Music Search Results Evaluation Project

Improving the relevance of search results for music-related content.
Raters must have access to an iOS device capable of running iOS 12.x. and higher. The job has an average volume of work.
Click here
CableApple TV Search Result Evaluation.

Search Relevance task focused on improving the relevance of search results, based on the most likely intent of user searches. Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo
IndexApple App Store Search Results Evaluation Project

Search Relevance task focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo
project
TruckeeInstagram Ads Evaluation Project

Help improve ads quality for one of the world’s largest image sharing sites.
Required: An active Facebook and Instagram account – The Instagram account must be connected to your Facebook account. You also need a smartphone with Android or iOS. The job has an Above Average work volume.
Click here
NileFacebook Ads Evaluation Project

Help improve ads quality for one of the world’s largest social networking platforms.
Rate is $11.00 per hour (USA) with an Above Average work volume. Active Facebook account required.
Click here
RattleThe goal of this project is to improve the relevance of search results for one of the world’s largest search engines. The job has an ABOVE average volume of work. Click here
ParadiseThe goal of this project is to improve the accuracy of search results for businesses and entities on one of the world’s leading search engines. The job has an ABOVE average volume of work. Click here
Mohawk *Help improve ad relevancy on a popular social media platform.
No info
Prospect MI * Facebook Falcon Projects. Help improve ad relevancy on a popular social media platform.
Click here
Zeya *In this task, you will be asked to research and evaluate claims stated in various pieces of content (e.g., articles, links, photos, or videos).
No info
Copper * The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads. Average volume of work.
Click here
Alkali * The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
Average volume of work.
No info
TigrisThe project consists of rating tasks focusing on improving the relevancy of advertisement results from a major social media platform. Consultants will rate advertisements to help the client understand which advertisements (also known as ads) people want to see, and why. The daily task of the consultant consists of rating a batch of ads in a client-owned rating tool. No info

*group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform. Newly added projects such as Mohalwk, Prospect MI, Zeya, Copper and Alkali are most likely related to small Facebook testing projects. If you have been participating in one of the above listed projects, feel free to share your experience with us via leave a comment option!

Lionbridge
Lionbridge project code nameDescription Reference for more info
Internet Assessor, Internet Safety Evaluator, Personalized Internet Assessor Google Search Engine Evaluation Project Click here
Multimedia JudgeBing Search Engine Evaluation project Click here
Personalized Ads Assessor Google Ads Evaluation Project Click here
Maps Analyst Maps Evaluation Project Click here
ISoftstone
iSoftStone project code namesDescription Reference for more info
Online Ad EvaluatorBing Ad Evaluation project Click here
Search Engine EvaluatorBing Search Engine Evaluation project Click here

How to Work at Appen and Lionbridge from Abroad without Any Risk – VPN & Teamviewer Free Solution/How to Apply, Qualify and Work from Abroad for Any Appen Projects of Any Locale (Provided You Have a Work Permit and Language Expertise Required for a Specific Country/Project)

You need to login to view this content. Please . Not a Member? Join Us

Common Issues of New Raters: Appen Yukon and Lionbridge (Internet Safety Evaluator) Google Rating Projects

In this post I will discuss some common issues that new raters face. If you have other issues, feel free to share by leaving a comment below.

What kind of tasks to expect while providing search engine evaluation service for Appen/Lionbridge? 

If you get a position as a search engine rater with Appen or Lionbridge (Internet Assessor), you may encounter many different tasks. Some of these tasks are your normal rating tasks, and some are side by side, and you may even run into all kinds of experimental tasks. Some of the tasks are easier than others, like experimental tasks. Each task has a different set of instructions, so it’s best to make sure you know exactly what you are doing, and read very carefully each of the task instructions.

Yukon Project Side By Side tasks

Side by Side rating tasks are very interesting and have everything to do with Google algorithm changes. The human role or ‘rating’ in Side By Side tasks is to determine which side, left or right, is better than the other. In this task you are shown blocks on the left and right side. Each block has a slider from Fails to Meet to Fully Meets, and you should choose which one is best for each block depending on the user intent and query. The same guidelines apply for the blocks as they would for a page rating. Meaning that if block meets user intent, it should get the appropriate rating. Once you have chosen the ratings for each block, you will then be asked to choose on a scale which side is better than the other. A comment may be required to give your own opinion on why you’ve chosen which side, and why it is the best over the other side. It’s very important that you base all your ratings on user intent and taking their location in mind. You will also have a quality review on Side by Side tasks  once in two months.

Yukon Project Page Quality tasks

This is the main rating task that a rater is hired to rate. You will be given a website URL along with the first 5 questions which check things like foreign language, which if you’re doing English/USA ratings, would need to be in English to be understood from your location. Pages that don’t load need to be checked, as well as flagging for porn.

The second question is asking you to describe the purpose of the page. The purpose of the page and the user intent are entirely different aspects. Make sure you’re checking the website out to see if it matches user intent, and by looking at the website, you should be able to understand what its main purpose is. If the page is malicious, harmful, or just lacks its purpose for what it is searched for, then you should mark that here. Pages that are simply existing to show ads, without any help to the user, should be flagged. And lastly, it is important to make sure that you understand if the page is a YMYL (Your money or your life) webpage. Understanding this is important because it will change the overall rating of the website.

Once you have completed this, you will then rate the different aspects of the page including its main content quality, and the amount of content. Expertise, authority, and trustworthiness, who is responsible for the website, the supplementary content, page design, layout, and website care and maintenance are all graded on a scale from lowest to highest. The last thing to do is to give the website its overall rating. Usually these types of tasks are allowed 9 minutes to complete. A quality review is done each month to see how you rate. So it’s important to make sure that you are taking the time to evaluate each website properly.

Appen Yukon and Lionbridge Internet Assessor Needs Met tasks

This type of tasks you have already saw while doing part 3 of the qualification exam. Each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from ‘Fails to Met’ to ‘Fully Met’

Other Experimental tasks

Sometimes during rating of side by side, you will see experimental tasks pop up. In most cases experimental tasks are standard page quality or need met tasks, but sometimes you will get new types of tasks. Experimental tasks have a wide range of instructions. It’s important to read the instructions for each task as they will be different each time. Experimental tasks can be very simple, or take a few minutes to complete. They are not always available, and each task can be completely different than the other.

Appen Yukon Project quality feedback & Lionbridge internet assessor monthly quality review

New raters should not do live tasks immediately after getting access to the task portal. It is highly advised that you hold off on rating live tasks if you do not want to be restricted for quality in the next month. Typically tasks which are available for new raters are quality review tasks on which raters are accessed for their quality. Instead, new raters should focus on doing simulation tasks and self-education. Time spent doing this will really pay off, and you wont be restricted for quality in the future. Also, by doing a certain number of simulation tasks correctly you will upgrade your status in the system, and will be allowed to work more hours.

During your participation in the project your service will be evaluated for its quality by means of quality review tasks. In the beginning of every month, a certain number of tasks are uploaded into the system. Normally there are 10-12 tasks of Needs Met or Side by Side, and once in 3 months PQ tasks to be released as quality review tasks. These tasks are quality review tasks up on which your quality is measured every month, in the end of the month you will get your quality score feedback. In case of raters get substandard quality for several consecutive months, most likely such raters will be removed from the rating program.

If your ratings often stand out of general range of ratings provided by other raters in the project, most likely your account will be flagged by the system automatically and your work will be checked manually for quality .

Read more about this topic here:

How to Work at Lionbridge and Appen For a Long Time and Not Get Fired?

recording and invoicing Working time Yukon and Lionbridge internet assessor project

Time management (recording and invoicing working time) of the work you complete can become a problem if you aren’t managing how much time you are spending on tasks. First of all, you should not sacrifice quality for time. In the beginning you may spend more time than the average estimated time (AET) that Appen or Lionbridge allocate to a specific  task, but make sure that in the invoice you put average estimated time that they give you. If you spend less time than average estimated time per a task or set of tasks, be informed that your invoice will be returned for correction, Appen and Lionbridge will ask you to put actual time spent, that is why there is no sense to complete tasks faster than average estimated time. Take your time, do everything correctly and watch your time.

the Yukon Project Extension

Do not forget to log on to the Yukon extension each time you work in the system, otherwise your time spent on doing tasks will not be paid.

According to the vendor policy: any time spent logging on/off, reading, studying, and break times are not billable. Any tasks that are not submitted or are released are also not billable. Please make sure you are not including any of this time on your invoice.

Also keep in mind that consecutive invoices submitted with productivity below 100% may result in removal from the project or you may lose your contract with Appen.

Appen Yukon project Invoice Returned for Missing or Incorrect Data

Lots of new raters receive the following emails from vendor: 

Your invoice has been rejected for missing or incorrect data. Please make the necessary corrections or provide specific details on the reason for each discrepancy.

Time Discrepancy: For the following line item(s) there is a discrepancy between the hours you have submitted and the activity and work recorded on our servers.

Reported: x hours, xx mins
Unaccounted Variance: -xx mins

What raters should do in this situation?

In order to get invoice approved, raters have to deduct xx unaccounted variance minutes from the invoice, in the future make sure only AET is entered into invoice and the Yukon extension has to be on during active rating.

Payment method for Appen, Lionbridge and Raterlabs
HOW to add Already existing Payoneer account at appen connect personal project page
How to earn additional 25$  by signing up to Payoneer via RefeRral Link

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?

Appen Yukon get locked or restricted for Productivity 

Working on some tasks much faster than the AET followed by holding on to a task for much longer than the AET and billing for the full amount on your invoice

Raters are allowed to bill for the AET of a task OR the actual time spent working, whichever is less of the two amounts. It has been discovered that some raters working some tasks much faster than the AET followed by holding on to a task for much longer than the AET and billing for the full amount on invoice, which is a violation of official policy and will not be tolerated.

For example: Rater works 5 tasks, each with an AET of 5 minutes. The first 4 tasks rater completes in 1 minute each, and the 5th task rater holds on to for 30 minutes.

If this pattern continues, invoices will be rejected and raters will be removed from the rating program. Be sure that all invoices are in compliance with policy.

Appen Yukon & Lionbridge project multi-accounts

According to Appen Yukon project policy you can only have one account per IP address, while Lionbridge allows to have one account per household. Those pretty much mean the same thing. There can’t be two accounts coming from the same internet service and this requirement come from Google system requirements.  

When it comes to Google rating projects or Facebook related projects, Appen and Lionbridge can find duplicate accounts very fast, and as a result, you will end up losing your accounts and they will be terminated. You certainly don’t want that to happen.  

Be informed that it is prohibited to have multiple accounts, without special arrangement. Appen and Lionbridge can find duplicate accounts very fast, and as a result, you will lose both accounts.

There are a few things you can do to get around this problem, and arrange them in way that Leapforce, Lionbridge, and Appen won’t be able to detect.  

Multiple Accounts – Is There a Safe Solution? 

IP  address change and network sharing while working in Appen Yukon and Lionbridge projects

In some cases people may share their network, while working as search engine evaluator, this case leads to IP address conflict and removal from the project. Make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.

When rating tasks on raterhub platform you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place and in a country of  your project locale.

Non-completion of a minimum required number of tasks

This issue is very common among new Appen Yukon project agents and may lead to termination of contract. This happens when new agents show no activity in the first month of service. In particular, this can happen if new agents enter the project in the second half of month, and for whatever reasons, they did not complete a minimum required number of tasks. In order to avoid this issue, make sure to show the minimum required activity in the raterhub system.

Yukon and Lionbridge project tasks availability and new rater time restriction

In general, Appen and Lionbridge guarantee at least 2 hours of work per day but task availability may depend on project locale. There might be some days when tasks are available for more than 4 hours. Some locales, like the US project provides raters with 24 hrs/7 days tasks availability.

For the rest market  task availability may be limited. You should understand that the system of task distribution is based on a ‘first come first served’ basis and you need to be ready to catch your portion of tasks when they are uploaded into the system. In order to identify time slots when task availability is the highest please look at task completion graph, usually tasks are uploaded every day into the system at the same time slots.

Lionbridge does not provide raters with a task availability instrument, so raters have to identify time slots when tasks are uploaded into the system by themselves.

Appen also has a system that restricts access to tasks for new raters, and those who are undergoing their quality review. To lift this restriction, you have to complete a certain number of simulation tasks or wait till the next quality review, if your quality score is good, the restriction will be lifted automatically.

Appen Yukon and Lionbridge raters get locked for quality audit or restricted for quality

If you receive an email notifying you that your account has been placed under quality review, this is a good time to identify your weak points and improve your rating skills. Below you will find some tips on how to deal with a “Locked for quality audit” situation.

  1. To speed up the process and to get feedback on your mistakes as quickly as possible, I recommend writing to your vendor and ask why exactly your account has been placed under review. Sometimes the automatic system locks raters accounts without specific reasons. Usually such cases are being resolved within 1-2 days.
  1. While waiting for detailed feedback from your vendor, proceed with doing simulation tasks in order to improve your rating skills. In most cases the issue is related to Side by Side tasks. Make sure that you understand how to deal with this type of task and that you provide well written comments where required. In many cases, raters are placed under quality review due to poorly written comments.
  1. Once you receive the feedback email with your mistakes/quality issues, read it carefully and reply back with an explanation on how you will improve your weak points.
  1. Once your status is moved to “Restricted for quality”, keep in mind that your service is under inspection, thus making sure that you do your best to provide the best possible service.
Appen yukon raters hold status restricted for quality after receiving meets expectations quality feedback

Sometimes some raters do not get their restricted status changed to unrestricted with getting a meets expectations or above quality score. This happens because they were restricted for side by side or PQ tasks quality issues and feedback was released for needs met tasks. In order to lift up all restrictions a rater will have to wait for the next quality review cycle for type of tasks a rater was initially restricted for. If you get meets expectations or an above quality score your account will be unrestricted again.

Have you ever wondered how many hours raters work?

Knowing these statistics can be useful for raters in terms of estimating  possible productivity .

The survey is completely anonymous and once completed will show the overall statistics.

Thanks for participating!

How Raters help to improve Google Search enginE

uTest Software Testing Platform – How to Become a Successful Tester and Make Decent Money Online?

Getting started at uTest

utestIf you would like to obtain a job as a tester, you are probably wondering what sorts of skills does it take to become one. I would definitely recommend starting with some basic tutorial on “how to test”, you can find those on official uTest Youtube channel, and there are couple of good books for beginners: “Software Testing” by Ron Patton; “Agile Testing: A Practical Guide for Testers and Agile Teams” by Lisa Crispin and many others. But to become a good tester these kind of skills you need to have for starters:

– Great English skills (written);
– Analytical Thinking;
– Attention to Detail;
– Thinking Outside the Box;
– Persistence.

Tasks availability

There are no restrictions on amount of work, but it really depends on the platform and your rating as a tester. There are projects that I get invited daily. The project really depends on the hardware you have. Mostly there it’s a web or app based projects. For instance – some hotel website, or booking service. There is so much to test, starting with a web forms, ending with validation of every input field. They are easy for beginners to start finding bugs right away. I worked from US, and got what the platform offered me for the region, but I do know that project can be different depending on country you are working from.

Types of tasks

Usually is’s a whole functionality testing of some elements of the site. For instance: the customer wants a tester to be precise in Log-in page. So the tester focuses on those bugs which occur only on a login page, not considering the other parts of the website. Sometimes there are localization task. Let say you know Chinese, and customer requires the testers to check the spelling and grammatical errors of specific language.Most of the tasks are ad-hoc (exploratory testing) for functional issues. (App crashes, buttons not working, etc).

How to become a successful rater at uTest and make decent money

But there is no deep inside mentioning of a rating system and how it works. Let me explain why is it so important on uTest. This will be a more “in depth” article on earning on uTest. There is this system where it takes your results (amount of submitted bugs) and amount of rejected bugs and counts the successful percentage rate.

Based on this rate there are classifications of testers: Rated; Proven; Bronze; Silver; Gold

And there are different Testing Types, for each of which there is a rating: Functional; Usability; Load; Security; Localization; Automation

But don’t worry, most of the tests are functional and Usability, and polishing skills on these Testing Types can set you with a good additional income for each month.Getting more in those ratings, as I mentioned earlier it is the key filter for uTest to invite you to the project. To become a Gold rated testers it takes quite some time and dedication. Not to mention the quality of your bug reports should be on very high level.

Once you get a couple of rejections it is really hard to achieve the Gold rating. The highest rating gets you not only the more amount of offers, but you also get more paid, and those rare and exclusive projects available only for best testers out there.

Apart from this rating there is also a very cool and interesting opportunity on becoming a “Favorite tester”. Once you have basic rating of Silver or Gold you might end up becoming the one.

Let me explain simply how it works. There is certain person which manages a test cycle a TTL or Testing Team Lead. This person is responsible for other testers to follow the rules and code of conduct of Utest platform, and the project in particular. Sometimes there are even NDA’s which are needed to be signed. TTL is responsible for all this and more.

Let’s assume you have submitted 10 bugs in 1 test cycle, and all of those issues are either very valuable or exceptionally valuable bugs for the customer, Testing Team Lead can easily forward your account to the customer, so that they can give you a “Favorite Tester” status.

What does it mean? Well, you get to be automatically invited to each testing cycle this customer has and many more. Some customers always slot a test case for a favorite tester, which leads to more earnings.

I’ll go into more detail with the favorite tester part and test cases. Why is its important?

How much really you can make of uTest?

Once you become a favorite tester, usually it’s a big project with a big company, and the pay is really good. Some test cases can be up to 50$ per/case.

Once you start doing those cases, simultaneously it is easy and very practical to look for the bugs, (the inconsistencies between an actual and expected results). It really pops in the eye, once you get going through a test case.In that moment when you have found an issue and reported it to the system, apart from getting those 50$ for the test case itself, you can earn a lot more for reporting bugs additionally.

Talking about rates paid for bugs filled… Well, it absolutely depends on the person, the amount of details he/she is giving, the amount of devices the person holds, and the overall work ethics a freelancer sticks up to.

Bugs may vary from 3$ for “Somewhat valuable” to 30-40$ (per bug) for “Exceptionally valuable” issue, depending on the customer.

For sure you are willing to know the real numbers, and the sacramental question needs to be answered.  “How much really you can make of Utest”? I will repeat myself stating that everything depends from the person. But, you can crank out a fair 1000-1500$ per/month of income depending on your dedication.

Payment

Everything is done via PayPal and Payoneer. Sometimes the bugs are not approved by customer for as long as 35 days. After that period, they are approved automatically by the system.

Use this link for applying for signing up to Payoneer and we will both benefit by earning 25$, once your account is loaded with payments.

%d1%81%d0%bd%d0%b8%d0%bc%d0%be%d0%ba-%d1%8d%d0%ba%d1%80%d0%b0%d0%bd%d0%b0-2016-12-04-%d0%b2-19-31-58

Productivity 

There are many factors on becoming successful on this platform. Speed is one of them.

Sometimes it’s critical to start testing immediately, as other (hungry) testers will beat you to that critical issues, while you’re yawning.

The same rule is applied for the test cases.

Test Cases are usually limited, and not everyone can claim them, so it’s lethal to be fast.

To make the numbers I mentioned earlier, it really takes time to pump your account to a certain rating level. Usually it will take around month or two to get going, but hey, Rome wasn’t built in a day either.Once you become an experienced tester and acquainted with a platform like a “Pro”, you actually might get invited easily on becoming a Testing Team Lead.

Who are Testing Team Leads?

The roles of this person are really dependent of the customer you are working for.

My roles were:

Writing test cases;

Participating in meetings with other TTL’s, Project Managers and Software Delivery Managers;

Create and control a test cycle;

Check and forward bugs for the customer (yes, TTL has the power to approve or decline bugs).

The income story is completely different, as the TTL starts working on per hour rate, and once again it really depends from the customer on the rate. (It is near to a very entry level junior position in software testing working for an onsite company in the States).

The benefit of becoming TTL on one project is that very rapidly you become noticed by others, and easily can get invited to manage some other cycle.In addition, TTL’s are still allowed to do the functional testing as other participants, though some customers forbid this.

In conclusion, now that I completely moved to an onsite job I can say with a share amount of confidence – Utest is a great place to start. It takes that will power, but sure does pay off. Good luck fellow freelancers!

Bing Search Engine Rating Projects vs Google Search Engine Evaluation Project

Bing Search Engine Rating Project (Side by Side Judge, Multimedia Judge) vs Google projects (Yukon/Internet Safety/Internet Assessor)

When getting into jobs as a search engine evaluator, you will find that one company will produce ratings for one search engine, such as Google, while another company will offer jobs as a rater in Bing which is Microsoft’s Search Engine. Like Google, Microsoft also needs research and pages rated just like Google.

As far as the job is concerned, it is basically the same job except you are rating results in different search engines that turn up different results. The pay generally varies depending what company you work for with some reporting earnings in the $12 to $15 dollar per hour range.

Google projects (Yukon/Internet Safety/Internet Assessor)

Some of these jobs are changing and they are being called different things such as ‘internet analyst’ which puts a twist on the tasks you are given. Previously when working as a search engine rater, you were given results that were based on generic search results. However, as time went by, Google started asking raters for permission to use their personal data. Basically, if you were surfing for stereo’s or smartphones, and you started doing this job, you’d get back results based on your surfing habits as linked to your Gmail address that you use.

The job that Raterlabs offers is a Google Search Engine and Google Playstore project where the raters are now called ‘Internet Analyst’ where the tasks you get are based solely on your personal Google use. It’s a part-time position that currently looks to be for the USA region only, but this does change from time to time. The requirements and skills to get this job are nothing that most people don’t already have if they are good at searching or surfing the internet. With this job they want raters to possess a computer with high speed internet, and also an Android phone with version 4.1 or higher, or an iPhone version 4 or higher. The job most likely involves doing ratings on both the computer and the smartphone. Sometimes tasks are available on only one device or the other. So it’s good to have them both because you will likely get smartphone tasks. It is also noted that in the description that they want raters who use Google Play often. There is a possibility that tasks involving Google Play will be available. This job’s testing and qualification is no different from the previous exams for search engine related jobs. There are three parts to the exam leading us to believe it’s the same exact material for the regular rater. The job is basically the same after all, the only difference is now its personalized to you. It is not known what the pay is for this job as that information is confidential and you sign a non-disclosure agreement about not discussing your pay. But it’s likely in the $12 dollar per hour range if its Raterlabs.

Appen Global also offers this type of position even though they also own Leapforce and likely are using them as a source of experience and also taking advantage of the workers who were already employed by Leapforce along with their experience. Appen calls their position Web Search Evaluator which likely has the same test. It is unknown if Appen supplies raters for both Google and Bing but it’s possible that they do both. All the requirements and testing are the same for Appen since they are providing raters for Googles Search Engine. But it is rumored that the pay for this position is much lower than you would get if you worked at Lionbridge or Raterlabs. This is because Appen is a much larger company with a much larger talent pool to take people from. This allows them to pay much lower wages. So in this case it’s actually better to get these types of jobs at other companies like Leapforce/Raterlabs, and Lionbridge. It’s interesting to hear that Appen is doing some interviews from Skype now.

Speaking of Lionbridge, we are getting to them next. This company seems to have a good reputation for paying people better than the other companies for the same type of job. They also offer opportunities for the Search Engine Rater but they likely offer rating jobs for both Google and Bing. Microsoft does use Lionbridge to hire human search evaluators, but Microsoft calls you ‘judges’ instead of raters. Lionbridge was known to pay raters $14 an hour for these positions although that wage has probably fluctuated over time.

Miscrosoft’s HRS project (Multimedia Judge, Side by Side Judge)

You’re probably wondering what the difference is in rating for Google or the Bing search engine. Bing has a project called Human Relevance System or HRS and it’s the system that they use to have human raters judge search results with. The HRS project is similar to the Quality Rater program that Google employs. However, it’s interesting to know that Microsoft’s version has been around since 2004 since MSN Search began generating its own results.

Not much has been written about Miscrosoft’s HRS project, but leaked copies of the guidelines are floating around that go over the process of rating or ‘judging’ websites. The difference between Googles rating guidelines and the HRS is that HRS uses what is called a rating matrix. This matrix combines the searcher intent with how well the returned document satisfies the searchers intent.

You can see the five rating options that judges can use from perfect to bad. This is the absolute heart of how you would rate returned search results. This might actually be a little easier to understand for some raters by judging just the intent and how well the intent is satisfied.

But there is one other aspect of the HRS system and the guidelines go over it in a pretty detailed manner. This is called Freshness and basically you will look at the content to see how up to date it is. Some information can be relevant but be a couple years old when some things have changed and the information isn’t updated or fresh enough to contain those changes. The Freshness system is based on three categories; Fresh Not Important, Very Likely Fresh, and Most Likely Fresh. The Fresh Not Important will be for things where the facts will never change no matter how fresh the article or the page is, while the other two will be based on the articles ability to receive fresh content.

When it comes to the difference between Google and Bing as far as their rating systems, they aren’t really all that different from each other. But if you will be working as a search engine rater, and you are looking for stable tasks, you might want to try and get Google search rating jobs since there is more volume when it comes to this search engine over Bing. Still, both jobs are decent, but it really depends on who you are employed by. Appen seems to be one of those companies that will pay people as little money as possible to make more money themselves. This was their whole point of buying Leaforce; to squeeze out the competition. So this company will pay less. Lionbridge or Raterlabs will be the places to try and get these jobs.

When it comes down to Bing, it’s not a very big project in terms of the number of raters who are hired for this as compared to the raters that are hired for Google. That shouldn’t be a big surprise because we all know that Google has a much higher search volume and a lot more users. The judging platform for HRS is a bit outdated, not convenient to use, and it only works on Internet Explorer of course. Also, the test has a part in where you have to judge the search results for relevance of around 100 items and they give you very little time to do this. Many candidates often fail the test and are unable to pass the exam.

iSoftstone is another vendor and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done in the HRS platform and you have to complete a qualification for each task. There is a guideline for each one of the tasks, but you don’t need to worry because everything is explained in videos and emails. Tasks are categorized as hits or pages and for each hit or webpage that is shown you get 0.01 penny to 0.20 cents US. If you do tasks/hits quickly, then you can make easily $10-$15 dollars US per hour, and some people even more. You might be tempted to go through some of the tasks as quickly as possible because the more you complete, the more money you obviously make. However, sometimes during these tasks you are sometimes tested so you have to be very careful. Too many mistakes and they will remove you from the project and you won’t receive any more hits. ISoftstone is a bit easier to work for in that they don’t have a huge amount of work compared to Appen/Raterlabs.

Basically, what this whole thing boils down to is the individual. Some people might like working for Bing because the workload is lower, but you’re going to make less money because of this. If you need more stable work and higher than average work, then working on a project that involves Google will probably be a better alternative.

Do you work for any of these companies? We would like it if you shared your experience here in the comments. And as always, if you have any questions, let me know by email or in the comments and I will answer them as soon as possible.