Looking to pass Appen Yukon project exam or Lionbridge Internet Assessor & Internet Safety evaluator exams? Then you have come to the right place for all the information that you need
(Updated Jan 2020)
Before you can start working on various tasks, you must pass the qualification exam. The qualification exam you will take covers two types of rating tasks which will make up 70% of all tasks in the project: Needs Met and Page Quality.
Once your candidature is approved for certification, you will get an exam email with detailed instructions. Within seven days you have to study 160 pages of the general guideline which consists of rating rules and concepts. The guide explains to you how to do the rating of search results and assessment of the quality of web pages. You should follow these rules and not use your personal judgment. What they want is for you to represent a typical user in your country. Before you proceed with the exam you need to study the guideline at least twice paying particular attention to practical task examples. Keep in mind that it’s 160 pages long. As I always advise, don’t wait till the last minute.
Note: At the stage of examination, Lionbridge provides candidates with very helpful exam preparation support. You will get access as an exam guest to the Lionbridge learning portal where you can listen to webinars and even try to do a few practical tasks before proceeding with the actual exam.
Important update: Lionbridge is very selective in terms of recruiting new raters, up on revising exam results they will check your activity on their exam preparation portal, with this in mind, I strongly recommend you go through all the webinars on exam preparation portal before proceeding with the exam itself.
There are three parts in the exam: you can proceed with the next part only if you pass the previous one. Once you complete any part of the exam, within 3-10 minutes you will receive an email with information about results.
Google updates the way the exam is done from time to time. The main feature that has been changed is that you must rate pages from your smartphone (Send to Device tasks). If you do not have a smartphone, they say you cannot apply for this job. This is due to the fact that the number of mobile users is constantly growing and the company wants to assess websites from the perspective of on-the-go mobile users.
Note: There is a recommended schedule that will give you plenty of time so that you’re not rushing to complete the exam at the last minute:
- Day 1 – Study the Guidelines – Make sure to set up your smartphone so that you are ready to take the final portion of the exam.
- Day 2 – Start the Theory part of the exam making sure you are taking time to CAREFULLY read each question. Some of them can be tricky and that’s meant to make sure you fully understand the concepts.
- Day 3 – Go back and study the guidelines once again starting with the Page Quality section.
- Day 3/4 You can start the second part of the exam which is the Page Quality section if you feel comfortable on day 3. You may want to study this entire day, and you’ve got plenty of time if you decide this to take the exam on day 4.
- Day 5 – Read the Needs Met section of the guidelines again and make sure to take your time. You’ve still got two more days to finish. And as it has been mentioned before, have your guidelines open in case you need to refer back to them.
- Days 5/6/7 – Get started on the last part of the exam ‘Needs Met Rating’
Yukon project and Internet Assessor (Rater) exam theory part 1
In this part of the exam, you will be assessed on how well you understand the theoretical part of rating concepts, as well as on your ability to interpret questions and search for information in the general guidelines. It is an open book exam, so you can use the general guidelines to find answers to the test questions. This part has 32 question sections; each section has 4 parts, so in fact, you have 132 true/false questions to answer. Read questions carefully, some of the questions can be tricky in terms of their wording. You can consult with the guideline book while passing the exam and it’s strongly recommended to use this opportunity.
There is one part of this exam that is important! The last page of the exam should only have 1 question with the 4 statements. If you do not see this short last page, and instead see ‘No available tasks’ then you should inform the company right away. The reason for this is because if you do not pass this portion of the test, you will not be able to advance to parts 2 and 3 of the exam. If you do happen to fail the exam, there may be an opportunity to take the exam a second time.
Each task has a 1.5-hour limit before it expires and so it is advised that you don’t let it expire as it will affect your score. Make sure that if you need to step away from that you hit submit to complete the task before exiting. This is important because any work is done will not be saved until you click Submit.
You can only submit a task if you have filled out all the required sections. This means that no partial work will be saved. To exit, you can either click ‘Sign Out’ or simply close the window. To reopen the exam, you follow the same steps as when starting. To find that information, look in Section III: Accessing the Qualification Exam. It is recommended that you finish this part of the exam in one day.
Note: do not leave incomplete tasks more than for 1,5 hours, otherwise, they will expire and you will fail the exam automatically. If you need to take a break, complete tasks and press “Submit and Stop rating”.
Once you have reached the end of part one, the tool should automatically take you back to the homepage. If the homepage says ‘No available tasks’ at the top, then you have finished the exam. It is important to understand that if you release any tasks during qualification, it will be marked as a WRONG ANSWER! So whatever you do, do not release tasks.
Once you have finished taking Part 1 of the test, it may take some time before you are able to access Part 2 of the test. The one important thing to remember is that you might not get an email saying that Part 2 is ready for you. Instead, you should log into RaterHub after a few hours, or even wait until the next day to see if you are able to acquire new tasks.
If you see nothing in RaterHub after 24 hours, then you should email the company and let them know.
In most cases, an hour would be enough to complete the theoretical part of the exam. The following message will be shown to you once you have submitted the last page of questions:
No tasks are currently available. Please try again later.
Lionbridge Internet Assessor and Appen Yukon project exam part 2 (Page Quality)
In this part of the exam, you will be asked to assess 7 web pages for their quality. My advice for this part is to check all quality aspects, and fill in the online form very carefully so you can give a correct overall quality score (from lowest to highest) to a page. You should expect 2-3 low-quality web pages, 2 high-quality web pages, and 2-3 medium quality web pages. However this is not a rule, sometimes candidates get 4-5 pages of high or medium quality pages and only 1 or 2 of low-quality pages to rate. Keep this in mind. In order to give correct ratings, examine web pages for ETA (Expertise, Authority, and Trustworthiness) criteria very carefully.
The goal of Page Quality is to evaluate how well the page achieves its intended purpose and is not dependent on a query or the user location, bellow you will find general tips on how to approach part 2:
- Your Money Or Your Life (YMYL) pages should have a lot of formal Expertise, Authority, and Trustworthiness or EAT which can also come in the form of ‘life experience’ from non -formal pages.
- High-quality pages should represent the time, effort, skill, and expertise of their creators. Pages that are malicious, harmful, deceptive, and seem to be lacking any purpose should get the lowest rating.
- You don’t need to finish this part at once. If you want, you can take a rest. The tasks are still available when you sign in. Again, if you pass this part, the company will send you an e-mail saying that you can proceed with part 3.
- Also, keep in mind that in this part of the exam you need to give a PQ rating mainly to a web page and not to a website, although some quality aspects of a website have to be taken into consideration when assigning overall PQ rating, in particular in case of YMYL (Your Money Or Your Life) content which Google takes very seriously.
- Do not forget to write useful comments where required, otherwise, the system may show an error.
Note: The URL/web page may not open on your smartphone, in this case, do not mark these as “Did Not Load”. During the exam, you should not expect to give a website this flag. If you do so, your answer will be incorrect. To solve this issue try to open links using incognito mode on your browser or/and give rating using other sources of information (website reputation, etc).
Internet Assessor and Yukon project exam part 3 (needs met)
This is probably the most difficult part of the exam, and many candidates fail the exam at this stage. There are 7 tasks, and each task contains 8-12 result blocks. In each task, you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from “Fails to Met” to “Fully Met”.
Internet Assessor and Yukon project exam passing score
In order to pass this part, you have to get an overall score of at least 80%.
Send to Device opening links on mobile & multiple accounts login issue
Note: while doing Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3, keep in mind that all links have to be opened on your phone because of Google analyses web pages from the point of view of mobile users. If you do not have a smartphone, you may use any android emulator for this purpose, in some cases, it is even more convenient than using a smartphone for opening links. In order to open the link on your mobile phone, click the link on the result block on your desktop and then go to www.raterhub.com/url on your mobile browser (Google Chrome) to visit the link. You must be logged in your Gmail account in order to have access to raterhub platform on your phone. The same rule works if you use an android emulator.
Note: Some candidates when accessing the link (www.raterhub.com/url ), got informed that the Gmail account was unauthorized to visit the link. The most likely reason for this error is the fact of being logged on multiple Gmail accounts on a device. The solution for this problem is to go to android setting -> users & accounts -> add project associated Gmail account, then restart your device. This setting should fix the issue.
Lionbridge Internet Assessor and Appen Yukon project exam part 3 (Needs Met) tips and recommendations
It should be well known by now that the Yukon Project qualification exam isn’t easy, and you should study the General Guidelines inside and out before starting the exam. There are some tips and tricks that you can use to maximize your chances of passing the exam. The number one thing to remember is that you can always refer to the guidelines as you would a book in an open test. Use this to your advantage as you will likely refer to the guidelines while you are performing tasks, even after you already have the job.
Note: Make sure to mark important concepts from the Guidelines so that you can refer back to them if you get stuck. Even if you feel that your answer is correct, you should refer to the guidelines to double-check.
Note: Keep in mind that if you start the exam, incompleted exam tasks expire after one hour. If you need to take a break, make sure to select and click ‘Submit and Stop Rating’ to avoid the task from expiring.
Additionally, I would like to give general tips on how to proceed with Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3:
- Make sure that you fully understand the query and user intent. If you do not fully understand user intent, make additional research on the Internet.
- When you are evaluating landing pages in the exam, don’t place too much importance on the Page Quality slider. The focus needs to be on Needs Met Ratings. Do not worry about using flags in the exam such as the ‘Upsetting-Offensive Content’ flags.
- One of the most important things is to absolutely know when to use Fully Meets. Fully Meets ratings are very rare compared to Highly Meets ratings. Avoid being too generous on landing pages. If you have any doubts at all, you should not be rating a page ‘Fully Meets’.
- Determine what type of query (user intent) is: single entity (business) query with local or non-local or both intent/to visit in person/navigate to a website; business chain query or query which is looking for business within one category (e.g. car repair service near me); simple know query (where the user requests a specific piece of information, do not assign the Fully Meets rating to any particular result, the correct answer must be present in an SCRB). Examples of simple know queries: Paris weather, delta airlines customer service number, current time in Amsterdam); Information query seeking a long answer/piece of information (what cases stress); URL/website query.
- Consider the following aspects while giving ratings to result blocks: user location, for local intent, queries far away results are useless or slightly relevant. This aspect is very important for the following types of queries: tire shops near me, where to eat near me, Walmart near me, etc. When you get a task with this type of query do use an inbuilt map for researching user location/viewport and results.
- Make sure you give correct ratings to special content result blocks for a single entity and business category queries. In the case of a business chain/category query, the result block with a list of businesses is always better than just results for a single business within one category.
- Consider expertise and trustworthiness of search results for financial, legal, and health-related queries; low expertise and trustworthiness automatically reduce needs met rating of the results.
- Think of dominant, possible common, and minor interpretations for the query and take this aspect into account while giving ratings to a result block. Minor interpretation results should get a low usefulness (Needs Met) rating.
- Take into account how well the web page/website is optimized for mobile users.
- Once you have analyzed and considered all listed above aspects, you should give a rating to result blocks one by one.
Types of SCRBs (Special Content Result Blocks): ny weather (simple know query), how old is universe (simple know query), does god exist (long answer seeking), berlin airport (single entity), burger king amsterdam (chain), coffee shops amsterdam (categorical).
Note: In some cases, there might be some results (links) that are not accessible from outside the U.S. (usually websites of well-known retailers such as Target, Walmart, etc. They ban access from foreign IPs to prevent hacker attacks). Appen, Raterlabs, or Lionbridge do not take into account this factor, and therefore rating such results as fails to met may lead to overall fail.
To make sure you do not miss anything before proceeding with part 3, follow pre-exam instructions to pass this part. There will be a link to a video. Watch it and follow the instructions.
Note: In part 3 of the exam (Needs Met), most of the result blocks have Needs Met and E-A-T (previously PQ) sliders. The main focus should be on providing correct Needs Met rating, which in some cases might be affected by E-T-A (PQ) rating when it comes to YMYL queries and results. Leapforce/Google re-named PQ to E-T-A in part 3 to make it clear that the most important aspects in PQ are expertise, trustworthiness, and authority. In general, it is similar to the PQ concept. While giving E-T-A rating you should evaluate MC under these 3 criteria. In most cases, you may do it fast at a first glance approach, but be sure to provide a good evaluation for E-T-A when you have a YMYL query.
Note: If the user location map does not show user location correctly, and you have a blank page, hit the black arrow button. The task will be closed, but you will see “Incomplete tasks” on the evaluation platform. Click to continue and come back to your task with the updated user location field. Remember that it is very important to take into account the user location while giving ratings to block results. Also, keep in mind that there is no need to finish the task in one sitting. You can take a break if you need to.
Once you complete 7 tasks you will get this message:
No tasks are currently available. Please try again later.
Yukon (Appen) exam results and Internet Assessor (Lionbridge) exam results
You will get overall exam results on the exam due date from Appen and exam due date plus one day if your vendor is Lionbridge. Appen will ask you to sign a few electronic forms and in 2-4 days you will be given access to live tasks. At Lionbridge you will have to complete an educational course with no pass/no fail quiz before you are granted full access to the tasks.
US Yukon Raterlabs and Lionbridge raters
Once you pass the exam you will then go through the employment process which might take up to 3 weeks. You will be asked to complete an online version of the IRS Form 8850 (Pre-Screening Notice and Certification Request for the Work Opportunity Credit), and a tax credit questionnaire from a vendor payroll provider. After you have completed these two forms, you will be presented with a formal employment offer letter. After acknowledging your offer, you will still need to complete the required activities in a timely manner. Try not to postpone the completion of these important steps: Providing bank details for ACH payments on the Paycom website. Providing your I9 Employee Section 1 and I9 Employment Eligibility Documents (section 2) must be completed by the specified deadline. It is very important that candidates complete all required steps as soon as possible to avoid the termination of your employment.
NEW!!! Welcome to the Search Engine Evaluation Exam (Yukon, Internet Assessor) Simulation Tool!
What is The Exam Simulation Tool?
I have created a special tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/task types and skills which will be tested during a real exam.
How can this tool help you pass the exam?
The way the tool is designed will show you exactly your problem area and what you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes, and know exactly what you need to go back and study in the guide. Of course, we need to mention that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand-designed from the most typical exam tasks and is also designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam.
How do I get my hands on the exam simulation tool?
Exam Simulation Demo
If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains a simplified Needs Met and one Page Quality task so you can see how it works. Once again, the demo is 100% free and remain completely anonymous.
Exam Simulation Pro
The tool has 5 Page Quality tasks and 5 Search Results Satisfaction (Needs Met) of all the major query types. These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/task types and skills which will be tested during a real exam.
Explore the tool here: https://examsimulation.passleapforceexam.com/
Yukon and Internet Assessor Expert Support
If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.
The following posts are recommended for reading:
Common Issues of New Raters: Appen Yukon and Lionbridge Internet Safety Evaluator Google Rating Projects