Payoneer as a payment method for Appen, Raterlabs, Leapforce and Lionbridge

Payoneer – what is Payoneer and how it works? 

downloadPayoneer is a payment processing company like PayPal, or Skrill, but in many cases PayPal or Skrill might be more difficult to deal with depending on which country you live in. One of the most important to think about when working online is how you are going to get paid. If you don’t have a tool that can bring you your payments fast, secure, and in a guaranteed way, you will not be able to get paid for the work you do. Having an easy way to get paid will help you focus on your work without being worried about your payments.

Before I started working with Payoneer I tried PayPal, Skrill and others, but I had a lot of cases where I faced difficulties with these companies. It wasn’t only me, but also the companies that I worked with had problems with them, so this caused my payments to be delayed. Now I  receive payments directly from Payoneer Mass Payout partner, and I also use the U.S. payment service to withdraw my funds from PayPal. Recently, I also started to use the new service – withdrawal to local bank or direct deposit as its more well known.

Benefits of pre-paid card:

  • Non US residents can open a bank account in a US bank and get the prepaid master card in USD currency.
  • Possibility to purchase goods in US online retailers for non US residents such Newegg, Amazon, etc.
  • You can withdraw funds anywhere in the world
  • Fair service and conversion fees
  • Possibility to withdraw money in USD currency from PayPal to your Payoneer prepaid card without currency exchage in PayPal system

Payoneer and Appen 

Appen Global was using one of the largest payment processors ‘ADP’ but has decided to use Payoneer for making the majority of their payments. It is much easier to get the account setup and have this beforehand. As an independent contractor, you are always responsible for doing your own taxes, so Payoneer will not withhold taxes. You must do this on your own. Payoneer will make you go through some approval process but if you are working for Appen Global, you will be approved as there is no other way to accept payments. Even if you may not use it for Appen (if you do not into any project) you can use it send and receive payments for majority of services/websites as it is becoming an alternative for PayPal.

Payoneer now has a sign up program that’s a good incentive to join, lets benefit together!

%d1%81%d0%bd%d0%b8%d0%bc%d0%be%d0%ba-%d1%8d%d0%ba%d1%80%d0%b0%d0%bd%d0%b0-2016-12-04-%d0%b2-19-31-58As Payoneer is now the only way to get paid by Appen, and if you do not have a Payoneer account yet I invite you to apply for Payoneer account via my referral link:

Use this link for applying for a Payoneer account/card and we will both benefit by earning 25$, once your account is loaded with payments.

How To Add Payoneer For US raters at Leapfoce/Raterlabs and Lionbridge (Direct Deposit)

Payoneer offers receiving accounts via the Global Payment Service, which enable you to receive local bank transfers from companies and marketplaces in the US, Europe, UK, Japan and China directly to your Payoneer account.

US Lionbridge independent agents and Raterlabs and Leapforce part time workers may use Payoneer account for receiving payment from Leapforce and Lionbridge. Only US raters can use this method of payment  since only ACH (US local bank) transfers in USD can be accepted, and Leapforce and Lionbridge send payments to US raters via ACH (direct deposit), while international raters revive their payment via wire transfer.

Where to find your Payoneer bank details to provide Leapforce or Lionbridge raters with?

1.Go to your account page->Receive->Global Payment Service


2.You will then see the bank information page here including name of bank, your count number, and the bank routing number which you will need to give to Leapforce or Lionbridge when setting up direct deposit:




Join Payoneer and be awarded!

Are you a US search engine candidate or a US rater working on one or several projects? You have not got a Payoneer account yet? Join Payoneer today via my VIP referral link. Use your Payoneer account for receiving payments from your employees and/or customers. Receive payments at the amount of 5000$ within 3 months after registration and be awarded with 250$! Contact me for VIP referral link via



How to pass the Leapforce (Raterlabs), Appen (Inca) and Lionbridge exams? (Exam Guidelines 2018)

How to crack Leapforce exam? How to pass Yukon exam? How to pass Lionbridge exam? How to pass Inca exam?

Before you can start working on various tasks, you must pass the qualification exam. The qualification exam you will take covers two types of rating tasks which will make up 70% of all tasks in the project: Needs Met and Page Quality.

Once your candidature is approved for certification, you will get an exam email with detailed instructions. Within seven days you have to study 160 pages of the General Guideline which consists of rating rules and concepts which teaches you how to do the rating of search results and assessment of quality of web pages. You should follow these rules and not use your personal judgment. What they want is for you to represent a typical user in your country. Before you proceed with the exam you need to study the Guideline at least twice paying particular attention to practical task examples. Keep in mind that it’s 160 pages long. As I always advise, don’t wait till the last minute.

Note: At the stage of examination, Lionbridge provides candidates with very good exam preparation support. You will get access as an exam guest to the Lionbridge learning portal where you can listen to webinars and even try to do a few practical tasks before the exam!

Important update: Lionbridge is very selective in terms of recruiting new raters, up on revising exam results they will check  your activity on their exam preparation portal, with this in mind, I strongly recommend you go through all the webinars and tasks simulation on their exam preparation portal before proceeding with the exam itself. Make sure that you get a pass for Needs Met simulation tasks. You may do them several times until getting a pass.

There are three parts in the exam: you can proceed with the next part only if you pass the previous one. Once you complete any part of exam, within 3-10 minutes you will receive an email with information about results.

Google updates the way the exam is done from time to time. The main feature that has been changed recently is that you must rate pages from your smartphone (Send to Device tasks). If you do not have smartphone, they say you cannot apply for this job. This is due to the fact that the number of mobile users is constantly growing and the company wants to assess websites from the perspective of on-the-go mobile users.

Note:There is a recommended schedule that will give you plenty of time so that you’re not rushing to complete the exam at the last minute:

  • Day 1 – Study the Guidelines – Make sure to setup your smartphone so that you are ready to take the final portion of the exam.
  • Day 2 – Start the Theory part of the exam making sure you are taking time to CAREFULLY read each question. Some of them can be tricky and that’s meant to make sure you fully understand the concepts.
  • Day 3 – Go back and study the guidelines once again starting with the Page Quality section.
  • Day 3/4 You can start the second part of the exam which is the Page Quality section if you feel comfortable on day 3. You may want to study this entire day, and you’ve got plenty of time if you decide this to take the exam on day 4.
  • Day 5 – Read the Needs Met section of the guidelines again and make sure to take your time. You’ve still got two more days to finish. And as it has been mentioned before, have your guidelines open in case you need to refer back to them.
  • Days 5/6/7 – Get started on the last part of the exam ‘Needs Met Rating’

Part one Theory (Leapforce exam part one)

In this part of the exam you will be assessed on how well you understand the theoretical part of rating concepts, as well as on your ability to interpret questions and search for information in the General Guidelines. It is an open book exam, so you can use the General Guidelines to find answers to the test questions. This part has 32 question sections; each section has 4 parts, so in fact you have 132 true/false questions to answer. Read questions carefully, some of questions can be tricky in terms of their wording. You can consult with the guideline book while passing the exam and it’s strongly recommended to use this opportunity.

Note: Do not leave incomplete tasks more than for 1 hour, otherwise they will expire and you will fail the exam automatically. If you need to take a break,complete tasks and press “Submit and Stop rating”.

In most cases an hour would be enough to complete the theoretical part of Leapforce exam. The following message will be shown to you once you have submitted the last page of questions:

No tasks are currently available. Please try again later. 

Within 10 minutes a result notification will come to your gmail, and in the case you are successful with part 1, the next part will be available in 10 minutes.

Part two Page Quality (Leapforce exam part two)

In this part of the exam you will be asked to assess 7 web pages for their quality. My advice for this part is to check all quality aspects, and fill in the online form very carefully so you can give a correct overall quality score (from lowest to highest) to a page. You should expect 2-3 low quality web pages, 2 high quality web pages and 2-3 medium quality web pages. Keep this in mind. In order to give correct ratings, examine web pages for ETA (Expertise, Authority and Trustworthiness) criteria very carefully.

Also, keep in mind that in this part of exam you need to give a PQ rating mainly to a web page and not to a website, although some quality aspects of a website have to be taken into consideration when assigning overall PQ rating, in particular in case of YMYL (Your Money Or Your Life) content which Google takes very serious.

Do not forget to write useful comments where required, otherwise the system may show an error.

You don’t need to finish this part at once. If you want, you can take a rest. The tasks are still available when you sign in. Again,  if you pass this part, the company will send you an e-mail saying that you can proceed with part 3.

Note: The url/web page may not open on your PC, in this case do not mark these as “Did Not Load”. During the exam you should not expect to give a website this flag. If you do so, your answer will be incorrect. To solve this issue try to open link using incognito mode on your browser or/and give rating using other sources of information (website reputation, etc).

Снимок экрана 2016-01-10 в 17.25.24
Page Quality Task

Part three Send to device (Leapforce exam part three)

This is probably the most difficult part of the exam, and many candidates fail the exam at this stage. There are 7 tasks, and each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from “Fails to Meet” to “Fully Meets”. Lionbridge  and Leapforce exam passing score: in order to pass this part you have to get an overall score of at least 80%.

Note: while doing Leapforce and Lionbridge exam part 3, keep in mind that all links have to be opened on your phone because Google analyses web pages from the point of view of mobile users. If you do not have a smartphone, you may use any android emulator for this purpose, in some cases it is even more convenient than using a smartphone for opening links.In order to open the link on your mobile phone, click the  link on the result block on your desktop and then go to  on your mobile browser  (Google Chrome) to visit the link. You must be logged in your gmail account in order to have access to raterhub platform on your phone. The same rule works if you use android emulator.

When you are evaluating landing pages in the exam, don’t place too much importance on the Page Quality slider. The focus needs to be on Needs Met Ratings. Do not worry about using flags in the exam such as the ‘Hard to Use’ flags.

One of the most important things is to absolutely know when to use Fully Meets. Fully Meets ratings are very rare compared to Highly Meets ratings. Avoid being too generous on landing pages. If you have any doubts at all, you should not be rating a page ‘Fully Meets’.

Additionally I would like to give general tips on how to proceed with Leapforce (Lionbridge) exam part 3:

1. Make sure that you fully understand the query and user intent. If you do not fully understand user intent, make additional research on the Internet.

2. Determine what type of query (user intent) is:

o Single entity (business) query with local or non local or both intent/to visit in person/navigate to a website

o Business chain query or query which is looking for business within one category (e.g. car repair service near me)

o Simple know query (where the user requests a specific piece of information, do not assign the Fully Meets rating to any particular result, the correct answer must be present in an SCRB).Examples of simple know queries:paris weather, delta airlines customer service number, current time in Amsterdam)

o Information query seeking a long answer/piece of information (what cases stress)

Types of SCRBs (Special Content Result Blocks): ny weather (simple know query), how old is universe (simple know query), does god exist (long answer seeking), berlin airport (single entity), burger king amsterdam (chain), coffee shops amsterdam (categorical).

o URL/website query

3. Consider the following aspects while giving ratings to result blocks: user location, for local intent queries far away results are useless or slightly relevant. This aspect is very important for the following types of queries: tire shops near me, where to eat near me, Walmart near me, etc. When you get a task with this type of query do use in built map for researching user location/view port and results.

4. Make sure you give correct ratings to special content result blocks for single entity and business category queries. In case of business chain/category query, the result block with a list of businesses is always better than just result for a single business within one category.

5. Consider expertise and trustworthiness of search results for financial, legal, and health related queries; low expertise and trustworthiness automatically reduce needs met rating of the results.

6. Think of dominant, possible common, and minor interpretations for the query and take this aspect into account while giving ratings to a result block. Minor interpretation results should get a low usefulness (Needs Met) rating.

7. Take into account how well the web page/website is optimized for mobile users.

8. Once you have analyzed and considered all listed above aspects, you should give a rating to result blocks one by one.

Note: In some cases there might be some results (links) which are not accessible from outside U.S. (usually websites of well known retailers such as Target, Walmart, etc. They ban access from foreign IPs to prevent hacker attacks). Raterlabs or Lionbridge do not take into account this factor and therefore rating such results as fails to met may lead to overall fail.

To make sure you do not miss anything before proceeding with part 3, follow pre-exam instructions to pass this part. There will be a link to a video. Watch it and follow the instructions.

Examination Need Met Task

Note: In part 3 of the exam (Needs Met), most of results blocks have Needs Met and E-A-T (previously PQ) sliders. Main focus should be on providing correct Needs Met rating, which in some cases might be affected by E-T-A (PQ) rating when it comes to YMYL queries and results. Leapforce/Google re-named PQ to E-T-A in part 3 to make it clear that the most important aspects in PQ are expertise, trustworthiness and authority. In general it is similar to PQ concept. While giving E-T-A rating you should evaluate MC under these 3 criteria. In most cases you may do it fast at a first glance approach, but be sure to provide good evaluation for E-T-A when you have YMYL query.

Note: If the user location map does not show user location correctly, and you have a blank page, hit the black arrow button. The task will be closed, but you will see “Incomplete tasks” on the evaluation platform. Click continue and come back to your task with the updated user location field. Remember that it is very important to take into account the user location while giving ratings to block results. Also keep in mind that there is no need to finish the task in one sitting. You can take a break if you need to.

Once you complete 7 tasks you will get this message:

No tasks are currently available. Please try again later.

Leapforce exam results and Lionbriage exam results

You will get overall exam results on the exam due date from Leapforce and exam due date plus one day  if your vendor is Lionbridge. At Leapforce you will be  asked to sign a few electronic forms and in 2-4 days you will be given access to live tasks. At Lionbridge you will have to complete educational course with no pass/no fail quiz before you are granted full access to the tasks.

US Raterlabs and Lionbridge raters

Once you pass the exam you will then go through the employment process which might take up to 3 weeks. You will be asked to complete an online version of the IRS Form 8850 (Pre-Screening Notice and Certification Request for the Work Opportunity Credit), and a tax credit questionnaire from a vendor payroll provider. After you have completed these two forms, you will be presented with a formal employment offer letter. After acknowledging your offer, you will still need to complete the required activities in a timely manner. Try not to postpone the completion of these important steps: Providing bank details for ACH payments on Paycom website. Providing your I9 Employee Section 1 and I9 Employment Eligibility Documents (section 2) must be completed by specified deadline. It is very important that candidates complete all required steps as soon as possible to avoid termination of your employment.

The following posts are recommended for reading:





Taxation for Independent Contractor’s Pros and Cons

If you are working as a Search Engine Rater or Social Media evaluator for any company that doesn’t hire you directly, then you are known as an independent contractor. Regardless of whether you are an independent contractor, a freelancer, or consultant, you will be considered as ‘self-employed’. What this also means is that you are responsible for doing your own taxes. There can be some advantages to this and also some drawbacks.

But first some of the pro’s.

If you’ve been working for companies like Appen Global, Leapforce, or Lionbridge, you know that you are basically your own boss. It’s very rare that you have to work with other people in these jobs or have much interaction at all. This can be a good thing or a bad thing. Keep in mind that the people you are working for are not your supervisors. You’re a contract worker which simply means that you provide your services in exchange for a set wage or price. You are largely in control of when, how, and where the work is done. And you can do the work when you want to as long as it gets done.

Another advantage of being an independent contractor is that you will make more money than if you were simply an employee. The reason for this is because companies are willing to pay more for an independent contractor because there aren’t any expensive long term commitments and the company doesn’t have to pay extra money for employment benefits.

The other great thing about being an independent contractor and working on your own is that you get to deduct more business expenses than a typical employee can claim. Think about everything you need to do your job, your computer, your desk, your chair, even the internet connection you are paying for can all be deducted as an expense at the end of the year. This basically means free office equipment.

One of the first things that you will want to do is come up with a plan to save enough money to pay your taxes at the end of the year. Working as a freelancer means that no one is doing your taxes. Your pay is given to you all in one lump sum with no taxes taken out. It is important to understand the basic tax laws. If you are not sure what to do, it is advised to talk to a tax specialist in order to help you save enough tax out of your paycheck.

But there are also some cons of working as an independent contractor.

One of the big things is that contractors aren’t eligible for any type of health or unemployment benefits because you are basically self-employed. So if you need health insurance, you would have to find something on your own that you could pay per month. Now you see why being an employee has its benefits when it comes to taking care of taxes and health insurance. You can still do all this stuff on your own, and it’s not impossible to get your own health insurance, but all that stuff takes time to keep track of.

It should be noted that if you are a US Yukon project rater then you’re a part time employee and your taxes are done for you. There still are other projects that are done on contract basis.

Knowing your taxes and how much you need to deduct is the most important part. As long as you keep enough money to cover taxes then you won’t have to pay anything. The biggest problem here is not withholding the right amount or not enough taxes. A single person in the USA will pay more taxes than a married person will. So it’s important to know exactly what you should be deducting from your pay.

For the USA you must take out self-employment taxes which are Social Security and Medicare contributions.

Some things that can help you:

There are some tax calculators online that can be used to accurately deduct the correct amount of money in taxes, including the self-employment taxes. That money should be kept in a separate bank account always so that if anything happens with the main bank account, that money is safe. So create a savings account and put that money into the account and don’t touch it until the end of the year when it’s time to do taxes. It is important to remember that it’s better to save too much than not enough.

When it comes time to do your taxes, it might be good to hire someone that can put together everything for you including your business deductions such as your office equipment or computer that you use to do your work on. There are some limits of course on what you can deduct. Find out exactly what you should have paid and make out a check and send it to the tax government in which you live. In the USA it’s the Internal Revenue Service.

As an independent contractor you won’t pay less or more taxes than if you were an employee. The only difference is that you have to do all the paperwork yourself. However, you are able to take more physical deductions than an employee would because you provide yourself with all the tools you need to do your job on your own.

In closing, it shouldn’t scare you to do your own taxes. It’s not any more difficult than doing your taxes as if you were an employee, and you might actually get the tools you need to do your job for free.

Appen Typeahead Project

Appen Typeahead Project 

This article goes over the guidelines of the project at Appen called Typeahead e2e. This project is somewhat the same as being a search engine rater and rating queries and results, but you are rating queries for Facebook and their website search engine. They may be trying to get this search engine as tuned as Google, except it’s based on their platform. Facebook is considered a family website, and they are trying to make sure that none of the bad stuff gets through. The main purpose of the job you perform is to detect low quality suggestions, and the ratings tasks are mostly based on your own point of view, which is unusual for a project of this type.  

What is Typeahead e2e?  

Requirements and payment rates 

E2E stands for ‘end to end’ but that seems irrelevant at this point. In order to work on this project, you must have a Facebook account. The project has a work volume from 1 to 4 hours per day. This is the usual amount of time on Appen projects. The pay range for this project depends on where you live, but it can range from $6 an hour to $14 an hour. This project is exclusive of Appen and runs in many markets. 


Qualification process is straight-forward, before proceeding with actual exam you will be given a chance to do a practice test which consists of 20 tasks and covers all possible rating scenarios. Once you complete the practice set you will receive feedback and recommendations on your ratings. The next step will be actual exam which also consists of 20 tasks. If you pass this exam successfully you will be moved to production. 

The process 

Typeahead are queries that are given with suggestions when you type something in the search box on Facebook.  

The search Typeahead provides suggestions, and it’s your job to see if the suggestions are relevant, or if suggestions are bad. There are a lot of different things that can make them bad. You might recall that Google at one point had a big problem many years back when Google ads were coming up with some violent and hateful content that caused them to lose some advertising with big names. It wasn’t until this happened that Google wanted a real human to look at what ads were being displayed, and what websites were providing as far as search content. The same thing is happening at Facebook and they want to make sure that people are not getting bad results with violent, offensive, hateful, or pornographic content. That is where the human rater comes in, and it’s your job to determine if the results are good or bad, or if they should be flagged.  

As mentioned previously, the workflow of this project is to determine the quality of suggestions in the search bar drop down. There are some things that the guidelines go over. 

You are presented with a list of categories when doing the search. You make a decision based on what the results are. You would follow these labels and determine what is acceptable and what is offensive/porny (cute new word that Appen or Facebook apparently made up), junky, or acceptable.  

It is important to remember that you are rating the quality of the suggestion, and not the quality of the query. Quality doesn’t have anything to do with the query. The query should be looked at to determine if it’s in a foreign language other than the language you are contracted to work in.  

Furthermore, a suggestion should be rated 1 if it is offensive, violent or porny, such as “hot girls”, “sexy videos”, etc. The suggestion, at first glance, needs to be obviously offensive or seeking offensive material.  

You need to remember that you are not rating the results generated by the Facebook search engine but the suggestion itself. Many common names may generate pornographic results (ex.“mary ann”) but should not be rated as offensive. If it’s junky, then it is because the suggestion is unrecognizable and, most importantly, you think the suggestion is junky/gibberish. Junky entails the suggestion is difficult to understand or is entirely nonsense. If you rate the suggestion as junky, you would then choose a category to support your decision.  

A suggestion gets a 3 – Acceptable if it is not junky. This is considered a good suggestion which doesn’t contain foreign language or anything offensive. It is spelled correctly and free of grammatical errors. If a suggestion is rated acceptable, then it must receive an additional rating to indicate whether it is personally recognizable to you. This is a weird task, because you wonder how it can be acceptable if it’s not recognizable. They clear that up by saying ‘The suggestion is UNRECOGNIZABLE based on your own point of view but still makes sense’. They say that you can use Google/Bing to determine if the suggestion is acceptable or junky. But once again, the rating is based on your own personal view.  

The job itself doesn’t look very hard, and the work flow should go pretty fast since you only have to choose from 4 different categories, and then choose a label from one of those 4 categories. There are a lot of examples in the guidelines that show you what to do in certain cases in case you are unsure of what to select.  

It is interesting to know what this will end up doing to the Facebook pages that rely on certain words that are brought up. Some searches may end up disappearing from the suggestions at all.  

As always, if you have any questions about this project, or you are working on this project, leave us a comment and tell us how it’s going.


Multiple Accounts – Is There a Safe Solution? 

Multiple accounts – is there a safe solution? 

There might be a reason that you would want to have two accounts, and some contractors do use more than two accounts. There might even be a possibility that you may want to work on this job while someone else in your household is also working for the same company or on one project but at different vendors (Leapforce and Lionbridge).  

However, according to the Leapforce policy you can only have one account per IP address, while Lionbridge allows to have one account per household. Those pretty much mean the same thing. There can’t be two accounts coming from the same internet service and this requirement come from Google system requirements.  

When it comes to Google rating projects or Facebook related projects, Leapforce, Appen and Lionbridge can find duplicate accounts very fast, and as a result, you will end up losing your accounts and they will be terminated. You certainly don’t want that to happen.  

Luckily, there are a few things you can do to get around this problem, and arrange them in way that Leapforce, Lionbridge, and Appen won’t be able to detect.  

Note: One thing before we go on that should be totally understood. There isn’t anything illegal about what you are doing, but if there are two of you that want to work for the same company, or if you want to work on multiple accounts from different vendors, you need to do things the right way. And let’s face it, most of these jobs are part time and give very little hours with most projects allowing a maximum of 20 hours per week. This restriction is due to the fact that vendors do not want pay for insurance and extra expenses which are associated with full time employment.

So how can you work on multiple accounts without violating the company’s rules?  

  1.  You will need two different laptops/PCs and smartphones/one set of devices per account. The reason for this is because of the devices MAC Address which can be detected per device;
  2. Use different internet or mobile networks which guarantees different IP addresses per network. Make sure that each PC is connected only to its designated network. Your internet service provider can provide you with one or more IP addresses, or use SIM card (cellular data connection) for this purpose. SIM card internet has dynamic IP addresses for each SIM card;
  3. Do not use your Wi-Fi network to connect your devices because it will increase the chance of your device accidently connecting to another network by accident and as a result, may lead to an IP address conflict with your client; 
  4. In the project where smartphone devices are required, you will need a separate smartphone for each account; Smartphones must be connected to Internet via sim card only. If you use your WiFi network for your mobile devices, they will all connect to the same IP address, and you will be flagged as duplicate account on the same IP address. For this instance, it might be a good idea to switch off WiFi so that you don’t run the risk of this problem;  
  5. Don’t use the same Gmail addresses for the accounts. This is already a given, but we thought that we should mention it. As an example, if you use one Gmail account for Appen, create another account exclusively for Lionbridge and make sure you reply using only those emails;  
  6. Once you have everything setup, and you’ve checked your IP addresses, only then should you try to apply for other accounts at other vendors. 
  7. You will need to apply on behalf of two different persons as normal.

Thanks to this approach, vendors’ detection system will see your accounts as two separate accounts with different IP addresses which are registered on different people, it means all contract conditions are met.

You also might be interested in this article:



Appen Acquires Leapforce in Multimillion Dollar Deal: What Are Potential Risks for Current Leapforce/Raterlabs Contractors?

About the Deal

In what might not come as a surprise to many, Appen, the language and search data service provider has made a move to acquire its US-based competitor ‘Leapforce’ for an estimated $105.3 million dollars. Leapforce was founded in 2008, headquartered in Pleasanton, California, and specializes in search relevance through a highly automated and proprietary end-to-end technology platform, which of course is Raterhub; the tool used by search engine raters to rate search engine results. It is a move that will make the company a leading provider in search data services. Mark Brayan, Appen company CEO said that it gives Appen the capability to serve their customers better. This deal comes after Mr. Brayan and Leapforce CEO, Daren Jackson, met a couple of years ago and remained in touch and formed a friendship. About six months ago, Mr. Jackson reached out to discuss a potential deal as both acknowledged at some point that the businesses would be better working together instead of against each other.

The Acquisition is subject to customary conditions precedent and is expected to complete on or around 7 December 2017. Appen’s existing customer base consists of, amongst others, a number of large global multi-national technology companies. The projects awarded by these companies, or the ongoing services which Appen may provide to these companies, can generate large amounts of revenue from that one client. The same is true for existing customers of Leapforce. However, and not surprisingly enough, Leapforce generates 99% of its revenue from only two customers, in the approximate proportion of 71% revenue from one customer and 28% revenue from the other customer. We’re guessing that these are both Google, and Facebook.

This brings up questions such as the acquisition of those contracts, or how Google feels about this merger of the two companies. The existing CEO and CTO of Leapforce, Daren Jackson, will join the Appen management team and will continue to manage the Leapforce business going forward. Leapforce is heavily dependent on the CEO who has the key relationship with its major customer and is key to the operations and service delivery of the Leapforce business. This of course means Google. Leapforce has only about 21 full-time staff, all of whom will be joining Appen.

What Are Potential Risks for Current Leapforce/Raterlabs Contractors?

The biggest question about this merger is finding out what exactly what will happen with Leapforce independent contractors. One of the risks that was mentioned in the acquisition merger was as follows:

“Some of the Appen Group’s search relevance and data analytics services are crowdsourced to, and often performed by, independent contractors. This is also the case for the Leapforce business. The independent contractors performing these services are retained pursuant to written agreements with a member of the Appen Group that commonly specify the individual’s status as an independent contractor, confirm the individuals are not employees of the employing company, and require the individuals to indemnify the employing company in the event the individual incorrectly represented their status to the employing company.

So if you are a Leapforce independent contractor, then you might have to sign new agreements with Appen after a certain time. And for the time being it looks like your job is secure, but its only as secure as it was before. Leapforce can at any time choose not to renew your contract for whatever reasons they see fit.

Or there could be a good chance that Leapforce independent contractors could have their contracts not renewed because Appen has a huge crowdsource pool from which to draw people from. Why do we think this? It was also mentioned in the merger agreement the following as seen as a risk factor for the merger:

“Notwithstanding the foregoing express contractual language, from time to time in the United States individuals retained by a member of the Appen Group as independent contractors may file claims for unemployment with the applicable state unemployment agencies claiming employee status with a member of the Appen Group and seeking unemployment benefits. Leapforce, being a US-based business, enhances this risk. Unemployment benefits are, from time to time, awarded by the US state unemployment agencies, which may result in nominal charges or increases to the employer’s unemployment tax accounts with the various states in which these individuals perform services and in which the member of the Appen Group does not have existing employees. The Appen Group is also subject to the usual risks posed to businesses that employ crowdsourcing, including claims relating to employee classification, claims to benefits, wage and hour claims and other employment claims.”

Of course this is true for any company when they acquire another business. But to be honest, in some markets Appen is known for paying miserable wages as compared to Leapforce and Lionbridge for the same search engine evaluation job. They have even lowered wages in some countries and pay far less than Leapforce did. There really isn’t any regard for the independent contractor in this situation.

And while its certain that all the internal employees were taken care of, as they will also go to Appen with their CEO, it is uncertain what the discussion was about the current raters. In our experience, independent contractors with both companies are simply a number or statistic. As an independent contractor for either of these companies, you are nameless, and you are expendable. This is the sad truth about being a contractor and working for these companies, along with the fact that you’re not really an employee for any company, and you have no benefits at all unless you are an employee.

And outside of this merger, it’s easy to see the behavior that this company takes against the people that help make them money. In some markets, Appen Global reduced their hourly rate for a project known as Aztec. In fact, some raters were hired with a higher starting hourly wage than others. And as mentioned before, in some countries Appen only pays HALF of what a Leapforce rater is paid.

It will be interesting to see what happens with the current independent contractors who work for Leapforce and whether they retain their contracts. One thing that is uncertain is if Appen will be calling the shots on the current Leapforce raters, or if Appen will take over and get rid of the ones that are paid more so they can take from their talent pool those who are paid far less for the same job. Regardless of the outcome, we feel that there might be a lot of raters who will end up losing their jobs in the mid or long term. After all, Appen Global doesn’t really care about the little guys who actually do the work. They seem simply set on cornering the market by acquiring their competitor, and then paying people lower wages for the same job.

If you are currently a Leapforce search engine rater or Raterlabs rater, leave us a comment and let us know what you think about the merger, what has been shared with you. We’d like to hear from you.



What Should You Do If You Fail Leapforce or Lionbridge Exam Twice and How to Re-Apply?

If you fail the exam the second time, Leapforce will not allow you to retake it a third time. In this case I would advise that you apply to another vendor like Lionbridge (or vice versa). 

Option 1: re-apply from your own PC 

Before proceeding with application you will need to clean your devices from the data left from unsuccessful exams.

  1. Sign out all Gmails associated with your unsuccessful attempts (make a complete sign out on a laptop, PC and your smartphone) and to be on the safe side, re-install your chrome browser on your PC;
  2. A new IP address will be required. You may use a residential VPN to change your IP address. Do not use normal VPN if you are going to apply for Lionbridge, use only a residential VPN. It is VERY easy to know if you are using a regular VPN. As an option you may change your Internet provider or if you have a dynamic IP address then no action is required at this stage;
  3. Create new gmail;
  4. Proceed with application as normal.Once you have started the application and qualification process, I recommend you keep the same IP address during the whole process. If you need to, later on you may change your IP as many times you want.

Note: I would like to point out that Lionbridge is very strict in terms of IP address. The vendor prefers to deal with applicants who have a residential IP address, but it depends on your country. 

Option 2: re-application on behalf of another person the same vendor is possible

Use someone else’s PC to apply and move the account to your PC once you complete the qualification process. 

Note: Before proceeding with registration, make sure that you use a new Gmail account which previously was not used for application process.

Note: It is very important that when you filling out the application and answering the application questions, you must indicate that you have never applied/worked the search engine evaluator position before with another company. Answering yes to this question will automatically disqualify you.  

Lionbridge Unsuccessful Application

There might be many reasons for the rejection of your application. These are some of the reasons: 

It might be possible that your CV does not meet Lionbridge standards. Lionbridge is very selective in terms of quality of CVs of applicants in India and other developing countries.Make sure that you CV looks professional and accurate and there are no gaps in your experience.  

Your IP address might be suspicious, or does not meet Lionbridge requirement. If you have a dynamic IP address (an IP which changes each time you connect to the Internet) – you will be rejected for sure. Make sure that you have static IP address. One way to check is to ask your internet service provider.  

Someone is using the same IP address for a rating project (in the case of sharing an Internet network connection), make sure that your IP address is unique. 

And finally, try to arrange new IP addresses, update your CV, create new Gmails, and apply one more time. If possible you might need to find a person you can trust, like a friend or family member, and apply on behalf of them. It could be a possibility that your name is blacklisted in the vendor database.  

Unable to enter IP address into the Lionbridge exam

It can be possible that the vendor was unable to enter your IP into exam because you have dynamic IP address. It might be very risky to have this type of IP address because it means that many people have the same shared IP address. Some internet service providers put a group of customers on the same IP address behind a NAT. This could be a problem and may lead to getting flagged for multiple accounts being used on the same platform such as raterhub.  

Common Issues of New Raters (Appen Aztec and Lionbridge Ads Rating Projects)


After you have passed either the Lionbridge Ads or the Aztec ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a chance that you will still see some English tasks come through once in a while.  

Time keeping and quality 

Time keeping is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Timesheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.  

One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.  

What are the quality review tasks? 

Blinds sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. 

Reasons for losing your service as an Ads Quality Rater 

It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.  

Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.  

Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.  

IP address conflict/Network sharing

In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.


Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by differenet vendors), and as a result, you will lose both accounts.

The Final Word 

If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too. 

Aztec Ads and Lionbridge Ads Rating Projects Review

deep-links-within-mobile-app-engagement-search-adsAztec Ads and Lionbridge Ads rating projects overview 

The Aztec Ads project and Lionbridge Ads rating are similar search engine evaluation project that is using the same Rater Hub platform that the other two large companies Leapforce and Lionbridge are using to rate search engine results depending on a query or intent of the user. Basically, you are doing the same thing as other search engine evaluators, but the difference is that you are judging the quality of ads and advertising-related data as they are calling it. Feedback that you provide is supposed to help the client monitor and improve their advertising as well as functionality.  

So what you are doing with this certain project is taking a look at different ads on web pages, and using the Rater Hub tool, you answer some questions on your findings. It sounds easy but answering these questions, and the answers that you provide are coming from guidelines that need to be closely followed. It’s not simply about providing your own feedback, but providing the feedback based on the guidelines themselves and specific metrics.  

What is the program purpose? 

As a human evaluator, you are not approving or disapproving the ads you evaluate, and you are not deciding directly which ads get shown or don’t get shown. In fact, even though you are providing scores and feedback about individual advertisements and advertisers, your evaluations don’t generally have any direct effect on those advertisements or advertisers. Sometimes the ads you rate are from a long time ago. Sometimes they are in a different form than they originally appeared. Sometimes they were never shown by Google at all, in any form. Basically what you are doing is collecting data to help train the algorithms that will eventually assess the ads being shown on their own. Without human interaction this would not be possible. So this job is very important to the engineers who are collecting the data from you.  

This is the main reason why it’s very difficult to pass the assessment part of this project. Your judgments are used to determine whether ads are getting better or worse over time, to see whether particular kinds of ads are working well or not, to see what kinds of ads cause problems, and to see what kind of impact a particular change to our systems will have. When Google engineers design new systems or propose changes to existing systems, the human evaluation data you provide is used to judge whether those proposals succeed or fail. This data is fed to computers that train the automatic algorithms.  

This is also a big reason that a lot of raters end up being terminated or losing their contracts. Accuracy and quality of your ratings are very important. You should always check the Ads Guidelines if you aren’t sure about something that you are looking at. Even some raters that have been doing this job for a while still refer to the guidelines.  

Appen Aztec  and Lionbridge Ads rating program requirements 

Getting on this project can be difficult. There is a long assessment required. Its mentioned early on that you will perform all work and including the assessment tasks on a non-shared and non-public computer. That’s pretty straight forward. You’ve got to have your own computer with a high speed internet connection. And according to them you can’t take your laptop to the coffee shop and work from there. So it’s a very good possibility that your home IP address is recorded in the RaterHub, and you need to use this same connection to perform all work. Anything else is strictly prohibited. 

Another thing to mention is that you’re going to need to use either Firefox or Chrome web browser but that should hinder anyone since those seem to be the main web browsers being used by the masses. This is because the Rater Hub tool will not work on other browsers.  

If you do pass the assessment, then you may be offered a contract position. Usually the contract hours are part-time 20 hours a week, but Aztec states 19 hours, Monday through Sunday. Once you complete the assessment, the results are usually sent within 48 business hours after your deadline to complete the assessment has passed, not 48 hours after you submit the assessment.  

As with all projects we can’t stress enough the important of one thing. Thoroughly read the guide they send you and have it open and ready while you are taking the tests. Make sure the minute you get the link to review the guide, that going over it is the first thing you do. Give yourself enough time to digest everything in it at least 1-2 times before taking the tests. The guide itself has to be accessed directly through EWOQ which is part of the RaterHub portal. You’ve got to be logged in using the Gmail address you used when you signed up at Appen/Lionbridge. This Gmail account should be a dedicated email account.  

Since you are not able to download the guide, you should bookmark the page with the guidelines so that you can access it quickly and easily. We said that it cannot be mentioned enough and it’s very important that you read and understand the guidelines completely before starting or you will fail the test.  

What are Aztec and Lionbridge Ads program payment rates? 

Wages on each contractual hour depend on the country and can vary between $4 to $15 per hour. For more information on wages read here: HOW MUCH CAN YOU EARN WITH LEAPFORCE /LIONBRIDGE/APPEN  

Agents will invoice vendors for rendered services once per calendar month. Fees to independent agents are paid (via direct deposit) within 30 days of receipt of a valid invoice. The payment is made on monthly basis via direct deposit into your bank account. Appen projects are paid now by Payoneer. If you do not have a Payoneer account yet you can get yours here: 

Note: By using this link for applying for a Payoneer account/card we will both benefit by earning 25$. 


Aztec and Lionbridge Ads program Assessment  

The first set of tasks are multiple choice questions (in total 45 questions) that come straight from the Search Ad instructions in the Ads Guide itself. After you have completed the comprehension questions, you will be shown a set of assessment tasks (in total 190 tasks) that asks you to rate just the first section of a Search Ads task. Once this is done you’ll get a set that asks you to rate just the second section. Finally, you’ll get a set of complete Search Ad tasks to rate. All of these tasks will provide you with comments as soon as you submit your responses. 

The typical exam tasks is a query and an ad creative generated by a search engine in respect to the user query. You as a rater will need to apply rating rule to determine how much promising or unpromising is the ad in respect to user intent. Once you evaluate the ad then you will be asked to evaluate a landing page associated with the ad and to determine how satisfactory is the page and website content in terms of meeting the user intent needs.

Ads rating project/Aztec Exam Tips

Additionally I would like to give general tips on how to proceed with Aztec/Ads Rating Project qualification:

As usual before proceeding with rating you need to make sure you clearly understand user intent. If not, use Google search to research the query and establish the dominant intent, and the secondary intent if one exists. Sometimes it’s hard to understand what the user intent is without doing the search yoursel;

Select appropriate query flag if required. For instance, if the query is in foreign language, or it is clearly a porn related query, make sure you select the appropriate flag. Remember that when you are doing ratings, the language should be in your contracted regional language. So if you are working in English, and you get a page in another language, then flag it appropriately.

Examine the ad creative and answer the question of how promising the ad is. You should refer to a specific section of instructions, and not evaluate the ad creative based on your own perception or personal likes or dislikes. Pay close attention to wording of the ad and the URL. In many cases, wording of the ad may indicate the ad deserves a porn flag, or has reference to another search engine.

Check the query and ad creative for the navigational flag. If the query is looking for a specific page or website and the ad suggests the website, then the ad and the query should be given a navigational flag.

There are a few scenarios in which you should pay special attention to. The first one is when the ad offers to repeat the search in another search engine. In this case, the appropriate rating for ads and LP (Landing Page) are somewhat unpromising, and don’t satisfy the query because the action won’t bring the user closer to his goal. Google wants to provide the user with the quickest possible and satisfying solution for the user intent and query. Sending a user to another search engine defeats that purpose entirely.

Select the ads creative rating, or how promising the ad is in relation to user intent. Once you’ve completed the first part of the task you will then be directed to the Landing Page section.

Visit the landing page and examine the page based on the guideline rules of how satisfactory the page is based on the query or intent. Once again, you should not use your own judgement but instead learn how to apply the guidelines rules. In many cases research on the website is required, for instance a quick product search will show you more information about the page, and help you understand if it meets user intent based on the query.

Make sure to pay special attention to the user location. If you see that the user is looking for something in a specific location, such as a restaurant in a specific location, and you get a restaurant in a completely different location, you should apply a negative rating. In this case it’s a good idea to refer to the guidelines and review typical user location scenarios.

Once you have completed all assessment tasks, including the survey, you will temporarily run out of tasks. This is because your answers will be checked before you will be allowed to move forward. Either you will pass this part or not. You will not see the Acquire Tasks button at this time. Remember that you need to wait 48 hours after the assessment deadline and not 48 hours after you submit. The assessment will be complete after you submit all tasks available.  

Appen Aztec vs. Lionbridge Ads rating project  

Many people ask me which project is better and which one should I join. In my opinion I think that Lionbridge is definitely better in terms of offering a better payment rate compared to Appen. It’s also important to note that the exam qualification and training are paid by Lionbridge where at Appen it’s not. So take this into consideration when you apply for both of these companies. Three days of unpaid training is a lot for people to deal with and really when you think about it, no one wants to do an exam for three days, and then fail it having put all the time and effort into it to get a job.  


As always, if you have any questions or thoughts leave a comment or shoot me an email.  

Wishing you great success in your work at home career!

How To Apply for Search Engine Rater And Social Media Evaluation Jobs

Sometimes finding these types of jobs online takes a bit of research. Most of the big companies that offer these types of jobs don’t usually advertise in common places or online job boards. You usually have to go directly to the company and apply on their website. I have seen that Appen Global advertises their jobs on, but that website charges a monthly fee. I never quite understood how a company can charge you to find a job, but they are the exception.

Finding these jobs and applying for them are different for each company. Going to each individual website is the best way, as long as you know where to go, and how to apply. With that being said, this article, and the videos we have provided, will show you how to find the work from home jobs.

One of the other important things to consider when looking for these types of jobs is your location. Some of the contracts that are offered through companies like Appen, Lionbridge, and Leapforce require you to live in the demographic area that the project requires itself. The reason for this is because the data from different regions will be different depending on location and the language.

Before you begin to apply for these jobs, or really any job for that matter, it’s probably a good idea to go over your resume and make sure it’s up to date, and that it includes only the important information that is necessary. It might also be to your benefit to have your resume checked by a professional. The resume experts at is an awesome service that I have used in the past, and having your resume edited or corrected by a professional is a sure fire way to get noticed and get you the job you want. Many of our readers have had very good results with them.

To visit their website Click Here:

Another important note is that if you have worked previously on other social media projects with different companies, don’t mention this on your CV/resume. The two most important factors about this are that you do not need previous experience for these jobs. And second, if you have worked on these types of projects before you will most certainly be looked over and turned down for similar projects. The reason for this is uncertain, but we have seen other people turned down because of this.

You can work on two different projects for more than one company. However, you need to use a different IP address for one or the other. So if you work for Leapforce and want to work on another project with Lionbridge, you will likely need a different IP address. Leapforce seems to be the only company that records an IP address and this is shown in the backend or the portal of the work dashboard. This is why it’s a good idea to use a separate IP address for another project. It might also be a good idea to use a different email between companies when applying for and working on these jobs.

Some advice when working on different projects so that you don’t overwhelm yourself is to get some experience working on one project with a company before trying to get another contract with a separate company. Most of these contracts are part time. So its beneficial to have more than one project at a time if you can handle it.

A really important thing to note here is that you can only have one contract per IP address when it comes to these companies. So if more than one person is working for any of these companies, it is important to have a separate IP address. You can usually order another IP address from your internet service provider for about $4-5 per month extra. Check with your ISP to find out if you require more than 1 IP address per household. You can read more about this in FAQ


Lionbridge search rater project is called the Google rating project which is the same as Yukon at Leapforce and raterlabs. They also have the positions of Personalized Social Media Assessor, Personalized Internet Assessor, and Personalized Web Assessor.


At Leapforce/Raterlabs you would apply for Search engine evaluator position first, and then based on project availability, you would get invites for Yukon, Nile, (Facebook social media evaluator) Truckee or Shasta projects. You can work on more than one project at a time.

Appen Global

Many of my readers will be interested in how to get invited to other projects at Appen Global. Appen Global offers a lot of different types of projects that you are able to work on. Unfortunately, is not likely that you would be contracted to start on projects such as the Thunderbird music project from the beginning. You would be more likely to start on a larger project first such FalconInstaAds or the Aztec project. Once you have started working at Appen, and you are on a Falcon project, like any of the social media projects that they offer, you will likely get an invitation to start on other projects as well. The more you work, and the better your performance is, the more offers you will get from Appen.

Those projects include:

Aztec (Web search evaluator/Search engine evalutor)

Facebook/Instagram Ads assessor projects (Social media Evalutor)

How to Apply

We thought it would be a good idea to create some quick walk-through videos to show you how to apply for these jobs and the different companies. Some of these

Lionbridge: go to website


Leapforce: go to website


Appen Global go here:



Hermes – Netflix subtitles project

So what is Hermes? Hermes is a system devised by Netflix to test the capabilities of prospective freelance translators wanting to work remotely for them. Netflix is hoping to solve the problem of generating high-quality subtitles and translations by creating Hermes. Hermes was also likely created because other vendors weren’t paying their translators well enough for high quality. The Hermes project is all about high quality. Netflix doesn’t want to waste their time for incorrect or bad subtitles or translation.

Anyone with multi-language capability can become a translator for subtitling, but your quality has to be nearly perfect in order to be considered. So the first thing you will need to do is pass the test. It can be difficult depending on your language ability and how good your typing skills are. Translating can be a tough job unless you are absolutely fluent in the language you are translating for. It might be a good time to mention here that you don’t need to be bilingual to get a subtitle job with Netflix. There will always be a need to have subtitles in your own language, such as English, for those who are deaf and rely on subtitles to understand a movie. But, those types of subtitle jobs will be harder to get because many people can transcribe, which is simply writing down what you hear in that language using the same language. Timing becomes a huge factor as well. You don’t want to input subtitles too early or too late when the dialog is spoken. Can it be a tough job? Yes, but again it depends on your abilities.

So what is the process to get started? The first thing you need to do is register with the site here:

Once you have completed the application forms, you will be sent a text message with what is called your unique H-number and a unique URL. Make sure you don’t lose this number because you are going to need it to log into Hermes platform. You will perform the test on your computer. You want to be sure you have a good broadband connection because you’re going to be streaming video content, and when you are writing subtitles, this is important. They recommend no less than a 20Mbit speed. Start by opening the link on your computer and login with the phone number, and your H number. You will then be taken to the Hermes admission test. This is what you will see:


There are 5 different tests or phases as they are called. It will take you about 90 minutes to complete all 5 phases.

Phase 1 gives you a time limit of 10 minutes and this test starts out by testing your understanding of English phrases. It has 15 multiple choice questions, and you need to select the correct word to complete a sentence, select the correct meaning of a word or a phrase, or choose the sentence with the correct grammatical structure.

Phase 2 has a 10-minute time limit and also has 15 multiple choice questions as in the first phase. The only difference in this phase is that you are asked to select the correct answer from a list of options. This is done in the target testing language.

Phase 3 has a 15-minute time limit. You will be asked 10 different questions based on 10 short video clips. Choices for answers will be in the target language you applied for. Each clip can be replayed as many times as you need but keep in mind your time limit.

Phase 4 is a technical phase moving away from the translation part of the job and focuses on technical aspects of subtitling. It also has 10 video clips as the previous phase. This part of the phase tests your working knowledge of timing, punctuation, and line breaks.

Phase 5 The last phase has a 40-minute time limit per clip in which there are two clips provided. You are asked to watch the clips and translate the dialogue into your target language. So if the clip is in English, and your target language is German, you will translate English into German. The original language subtitles will be provided on the left side of the dashboard. This will help with your translation that you will enter on the right hand side.

Once the 5 phases are completed, the phases will disappear, and a banner will show the status as completed. This is the same place the results will show up and you may have to wait 10 business days to get the results. The phases are graded by an actual administrator, so it can take some time to get the results.

There are some things to take into consideration before doing this test. There are some languages that are more in demand that others. For example, as I previously mentioned, English subtitling from English source will be more difficult to get than if you speak two languages. If you are not proficient in the target language you choose, you will not likely pass some of the phases. The languages that are more in demand than others are Korean, Arabic, Japanese, Swedish, and Danish languages. Keep in mind that there is likely higher score thresholds for languages such as English, Spanish, or Portuguese. The type of pay can range anywhere from $1 per minute to in some cases $6 per minute if you are proficient and the language you are targeting is a more desired language.

Having a bad internet connection is going to work against you. So make sure you check your speed and make sure you can get at least 20Mbit download speed. You may also need to invest in subtitling software as I have read its required for this job. So you are going to have some other out of pocket expenses.

In closing, doing subtitling or translation can be profitable through Netflix. But if you are applying for this type of job, you should probably already have some experience, and the correct software needed for this type of job. If you already have experience and everything that is required, then you will likely find that Hermes is a good fit for you.

As always, we welcome comments and questions, and don’t forget to hit the forum for discussion on other projects.

Useful links:

Maps Analyst Projects at Lionbridge and Leapforce

Apple-Maps-IconWe’ve got some new information to share about a position called the “Maps Analyst” at Lionbridge (Codenamed: Shasta project at Leapforce). It should be mentioned that the recruitment process for this project normally lasts a very limited period of time (at Lionbridge), and usually 1-2 months during different months depending on the country. However, the Shasta project could be available at any time. 

 The project involves evaluating online mapping, and routing related tasks in order to improve content and accuracy of map service: Apple maps. Raters are required to conduct research, and use online tools to investigate the geographic accuracy of tasks. Work is provided to you as a list of tasks through an online work platform, which is somewhat like raterhub. Most of tasks will be in the language related to the position that you applied for. For example, if you applied for the French role, then you will be rating in French etc. You will also be required to provide comments/feedback in English. 

Payment and working hours

This project is paid on a per task basis. You need to keep track of the amount of tasks, as well as the type of tasks completed on a day-to-day basis. This information is used to generate your invoice on the billing dates. The payment rate per tasks depends on your market, including the complexity of each task, and payment may vary from, $0.22 to $1.00 per task. Since you are working in a freelance independent capacity, you can manage your working time as you like and work whatever times you want. But this is a part-time job, and the number of hours you can accrue is set at a maximum of 20 per week. You must not exceed the maximum of 20 hours per week unless authorized. Lionbridge expects new Map Quality Analysts to be active and to participate each week. 

 Project computability 

This project is fully compatible with any other projects at other vendors. You are not allowed to work on another project at Lionbridge, although you are allowed to qualify on another projects if your vendor is Leapforce. 

Maps Analyst/Shasta project qualification 

In my opinion the qualification process for this particular project is quite difficult if you compare it with other projects. First, you need to study and understand 200 pages of guidelines and instructions within a limited period of time, and then complete a 2 stage exam, which consists of theoretical and practical parts, within 10 days. As with any project, you should read the Guidelines as many times as you can before you start working. There is a lot of information in the guidelines so you may want to read them multiple times. You should also use them while working, and refer to them to ensure you are applying the correct rating. There is no chance to pass this exam without getting a profound understanding of the rating rules and concepts. 

Exam Preparation Approach (Maps Analyst and Shasta project exam)

Lionbridge has very good exam support so candidates should use this to their advantage as much as possible. It is highly recommended to use the below approach in order to increase your chances of successfully passing the exam: 

  1. Read the Guidelines thoroughly and understand them
  2. Read/watch the content on the Pre-Exam Support Portal
  3. Complete the Sample Exam Questions

Completing sample exam tasks will help you understand what kind of exam tasks to expect. A typical maps analyst task is a query; address, chain/organization, point of interest, and a list of results generated by the map service based on user location, user view port, and type of query you will need to provide:

  •  Relevance rating (taking into account distance/user intent issue) 
  • Keep in mind that distance ratings are variable ratings which depend on query type (type of business), user location, area, etc. So each case should be evaluated individually in terms of proximity and distance, that is why there are no specific criteria for distance
  •  Name accuracy (you always need to check name accuracy on official website of organization) 
  •  Address accuracy (check all components of address on official website/UPS ZIP code look up website) 
  •  Pin accuracy (apply rules as per guidelines) 
  • The pin location, name and address accuracy do not affect the relevance rating. If the result meets the requirements for a navigational rating then you should go for it. Relevance and the pin location, name and address accuracy are independent ratings

Finally, you will provide comments if your relevance rating is below good. This just a general overview of typical task/algorithm however it gives a good understanding what kind of tasks to expect in this project. 

Maps Analyst Exam Task

 It is very important that you thoroughly research the query, and the result in every task you are rating. In order to establish the correct rating, you need to understand the query, and you also need to verify that the information returned in a result is correct and accurate including the name, address, location, existence, popularity etc. In order to determine if the information is true, use official sources wherever possible. Include links in your comments whenever you find out that the task returned incorrect data. To determine popularity, you will need to check reviews on reputable sites to decide on the most accurate rating to apply. You do not need to include a comment for every rating, but comments are mandatory for any rating below Good for relevance or below Correct/Perfect for data accuracy. 

  • Reference Guidelines – If you are demoting because of a guideline intricacy, it is useful to reference the guideline/rule that influenced your decision.
  • Be Concise – Don’t waste time writing elaborate comments, be specific and efficient with your comments.
  • Comment in English Only- Your comments will sometimes be reviewed or used by those who are not very familiar with your locale or its language.

As mentioned before you will be given 7 days to complete the exam. The theoretical part will consist of 20 questions which are entirely based on the guidelines. I think this part won’t create any problem because answers to most of questions can be found in the guidelines, average estimated time for completing this part of the exam is 1 – 2 hours. 

 Part 2 is the most difficult part of the qualification, some of the tasks are really difficult, and will require a high level of attention, the ability to apply rating concepts, and analytical thinking. In total you will be asked to rate 20 tasks, but in fact there more tasks because some of them are complex tasks which have a list of results that have to be rated. You will be given almost all types of results to rate (chain, address, transit, POI etc.) So be careful to research and examine each case step-by-step. Keep in mind that only one attempt to pass this exam is allowed. If you need any tips or further information, do not hesitate to contact me.

 Maps Analyst Exam results 

Once you complete this exam within 2-3 days you should expect exam results and a welcome email containing information about access to the supplier and project portal. You also will be given a login and password to the rating tool where you will do your production. 

Why do people fail Maps Analyst Shasta project qualification exams?

Lack of effort and sufficient preparation

The most common reason that people fail the Maps Analyst/Shasta project exam, is that some candidates think that they may get lucky and pass the exam without proper studying, or you may think you know enough about the internet or have good search skills. Be aware that all parts of exam are designed in a way to assess your ability to understand and apply rating concepts and rules based on a 200 page guide. Its not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort within the one week they give you to study for the exam. If you don’t at least study the guide, or go over it a couple times, you will most likely fail the theoretical part (part 1) of the exam.

Poor time management

One of the common reasons for unsuccessful attempt to pass the exam is poor time management. People do exam preparation and taking all 2 parts within 1-2 days. This strategy in many cases leads to exam failure, because the exam requires lots of concentration and attention to details…Thus plan your exam carefully so you have enough time for studying the material and taking the exam without rushing. Basically, don’t wait till the last minute.

Inappropriate understanding and/or implementation of rating rules and concepts

Have you failed the Leapforce or Lionbridge exams after carefully studying for them? Don’t worry, a lot of people have. Even people with experience in Search Engine Optimization, or even experience in taking exams have failed this test. In fact it is the first thing that Leapforce mentions before you get the exam. That it is very difficult and many fail. If you have failed, it is most likely you misunderstood the rating concepts and/or applied them in a wrong way; e.g. used personal judgment/common sense, or did not pay attention to details where it was required (e.g. user location for local queries).


When you start working, the process will be more smooth because you have already gained a conceptual understanding of the work during testing, plus most of tasks will be in your local language. It is recommended that 2 minutes be taken to rate each task in order to complete your research accurately. Please note this is an average. Some tasks may require more time, while others may require less time. Tasks can be made available any day of the week, at any time of the day. There is no specific time when tasks are launched. If you don’t see tasks, then you will need to check more frequently to see if new tasks have been added to your queue.