Appen Connect Wouri Project Review And Information

Wouri project review

There’s another project to work on over at Appen called the Appen Connect Wouri Project. We’ll go over the project a little and briefly touch on the main points, what type of job it is and what requirements there are. We will also touch on the grading process and give a few examples. 

The goal of the Appen Connect Wouri Project is to understand the user’s intention as represented by the query, and then determine how good the result is and if it satisfies the user’s needs. Users are using Safari and Spotlight to search for different information needs. Some examples include wanting to find out knowledge or information on something or somebody, wanting to know if there is a certain store nearby, the latest news related to a topic, or since they are MAC users, getting an application on iTunes. This project is a bit like the average search engine evaluator, but it does have a different grading system. However, it uses some of the same ideas from those programs on search engine results. 

A desktop or laptop computer is recommended to work on the Appen Connect Wouri Project over a mobile phone and all graders should be checking search results with Safari. Safari can be used on Windows operating systems, but it should be mentioned that there is no support from Apple when using Safari on MS Windows. You might be better off with a MAC computer, but it isn’t required. 

To participate in the Appen Connect Wouri Project you have to pass an initial assessment and achieve a minimum score of 80%. If you don’t pass the first time, you’ll get another attempt. After you pass the initial assessment you will be given a final assessment in which you must also pass with a minimum score of 80%. 

Wouri Project rating Logic

When looking at a user query, there are three steps that you will take in the grading process. 

  • The first thing is understanding the query. You should make sure it’s not too vague, inappropriate, or in a foreign language. 
  • The second thing is understanding the result. Determine if it is GEO or Time Sensitive. There is also the option to choose both or choose neither GEO or Time Sensitive. 
  • Once you understand the query and the result, you will then decide on a relevance rating for the result based on the query intent. 

It is important to note that Appen expects local graders working on the Appen Connect Wouri Project to judge the search results of the same locale. What that means is you must in the same location as the project requires. In this case, you may need to configure your computer and your search engine to be in the language of the project that you are working on. 

A web relevant search is one that satisfies the user’s information need. Of course, a search result can be more or less relevant based on how well or how completely it satisfies the users’ needs. It is important, and we cannot stress this enough, that you have the guidelines open and you have thoroughly read them. 

In step number one, you need to determine what it is that the user is searching for and decide if it’s valid. You should read the query, and review the third party results and think about the possible user intent that it represents. This is called ‘Understanding the Query.

For example, the query ‘Samsung’ could have many different user intents such as shopping for Samsung products, getting Samsung technical support, or the user may be looking for the Samsung homepage. However, if the user simply types in Samsung, then the query can be too vague. 

If you have no idea what the user might be looking for, then the next step would be to go to a popular search engine and study the term. Some common terms could have some special meaning. However, if it doesn’t have some special meaning, or it has too many special meanings with each one having a small portion of intent, then you would select Query is Too Vague. 

You might also run into a situation where a query is inappropriate and it encourages illegal behavior, includes profane words, using derogatory language, racial slurs or hate speech. It also includes graphic violence or the query might like to retrieve grisly content. You are not expected to do anything about these types of queries except to mark it as inappropriate.  

When in doubt about any type of query, you should check results from other search engines. It’s absolutely crucial to understand what the user is searching for, and you may need to check results from other search engines. The good thing about this project is that top results from other search engines are provided for you. There is a small link that says ‘Web Search Link’ that will automatically do this for you and show you the top results in order to better help you understand the query. 

It is VITAL to point out that opening the webpages is a MUST when you are working on this project. If the results match the first result of the other search engine, then the result is likely an excellent match. Make sure to check the results that are returned by the Web Search Link. Each result has a clickable URL, a small description, and a photo where applicable. 

We touched a little on GEO and Time Sensitive in the beginning, and since you might not understand what that is, we will go over it to help you understand it better.

If a query is about a store or some location that exists in other locations aside from the searcher, let’s say a Starbucks, or a query like ‘movie theatres near me’ then you know its geo sensitive meaning that it matters where and how close the result to the searcher is. If the searcher is from California and looking for a Starbucks and the first result is the homepage or another location, then it doesn’t satisfy the user’s needs. 

Anything that deals with a query that involves some kind of time frame, such as ‘Buy Superbowl Tickets’, age of Harrison Ford, or checking on the weather are results that should be marked as ‘Time Sensitive’. 

It is important to understand that a query can be both Geo sensitive and Time sensitive, and a good example is a query such as ‘tickets to the local basketball game’, local movie showtimes, etc. 

Once you understand the query and you judge the results, then you will assign a relevant grade for each result. There are 9-grade levels:

● Excellent

● Good

● Fair

● Bad

● Unavailable

● Query In Foreign Language

● Result Page In Foreign Language

● Vague

● Inappropriate

There are many examples in the guidelines that go over difficult queries in order to help you evaluate what results would satisfy the informational need for the searcher. It is a must that you go over the guidelines and read them and understand them so you know how to grade something. 

The guidelines go over when you should choose an excellent result, and the biggest point to keep in mind here is to believe that the result will be everything that the user will need, and they will not have to continue searching. If you believe that the result isn’t good enough, and you would also do an additional search, then you might choose ‘Good’ or ‘Fair’ as the grade. 

A ‘Good Result is one that the user wants to see above the fold on the device screen. It is typically the first 4-5 results. However, it is not the best possible result, which would be excellent, but it still provides good information. The guidelines have many examples of what should be chosen as a ‘Good Result’. 

When choosing a ‘Fair Result’ for the query, the result would only be somewhat relevant, but not be of interest to all users. An example of this would be a query for a celebrity name, but the result is an obscure person that has the same name. 

Choosing  ‘Bad Result’ would pertain to a result that does not satisfy user intent and is something that should not be shown to users at all. An example of this would be something that we previously mentioned before if a user was looking for a Starbucks Coffee and lives in San Francisco, but the map result is for a store in San Diego. There are other examples in the guidelines. 

It should be straight-forward of when you should choose ‘Foreign Language’. If the given query is in a different language from the specified one, or the target language that you are assigned to work, then you would mark this as “Query in Foreign Language”. If you are not sure if the query is in a foreign language, then just search that query to understand the word or words. 

Sometimes in the given search results for a query, the URL will be inaccessible, or website link will not be available or a dead link. If this happens then you would grade it ‘Unavailable’ This also includes landing pages that take longer than 10 seconds to load, or you may see that the domain is up for sale by being sent to a redirection page. This grade also pertains to when video content is unavailable or blocked. 

There are some other specific result types that the guidelines go over including ‘Ambiguous Queries’ or when the interpretation is much more popular than others. For example, the query is ‘Golden Retriever’ but the result is a song titled the same. This should be graded as fair because the dominant interpretation would be for the dog breed itself and not a song title. There are some instanced when multiple interpretations can be as equally popular though. In order to assign something to the excellent category, it’s a good idea to take into consideration what geo-location the seacher is looking from. Example:

Query is “UM football”, and the result is the official website for the University of Miami football team. If user’s location is in San Francisco, grade the result as Good, since UM could refer to either U of Miami or U of

Michigan and we don’t know which is best. But if the user location is in Miami, grade as Excellent.

As always, if you have any questions about the program, please send an email or reply in the comments. If you are working on this program, then we would love to hear about your experience in the comments. 


List of Projects And Code Names For Lionbridge & Appen Connect

If you are looking for a list of projects out there from companies like Lionbridge and Appen, we have them. As you may know Appen acquired Leapforce in 2017, since then the company has decided to close it is own project management platform known as Appen Global and move all projects to Appen connect. The group of projects known as Falcon (mainly Facebook-related projects) now is no longer actual, these projects also have been moved to Appen connect platform.

Taking into account the fact that all projects have code names and little description is provided, many candidates might be lost in terms what project to go for. 

Bellow you will find a list of most popular projects and links to more info about each project:

Appen connect
Project code nameDescription Reference for more info
YukonGoogle Search Engine Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
ArrowGoogle Ads Evaluation ProjectClick here

Exam Simulation Tool

Get Personalized Assistance or Advise
ShastaMaps Evaluation Project

Provide relevance and accuracy ratings for one of the world's largest and most popular mapping applications.
The job has an ABOVE average volume of work.
Click here

Get Personalized Assistance or Advise
Echo Apple iTunes Music Search Results Evaluation Project

Improving the relevance of search results for music-related content.
Raters must have access to an iOS device capable of running iOS 12.x. and higher. The job has an average volume of work.
Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
CableApple TV Search Result Evaluation.

Search Relevance task focused on improving the relevance of search results, based on the most likely intent of user searches. Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo

Get Personalized Assistance or Advise
JavaApple Store Search Relevance project focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4
Similar to Echo
SharpApple Search Relevance task focused on improving the relevance of search results within different media categories.
Raters must have access to an iOS device capable of running iOS 12.1.4
Similar to Echo
IndexApple App Store Search Results Evaluation Project

Search Relevance task focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo
Wouri Apple Safari Search Result Evaluation project. The goal of the project is to understand the user intention as represented by the query and determine how well or how completely the result satisfies the user's need. Click here
Pecos/Ipsoot (also known as Pearl and Falcon PT)Post categorization on Facebook. You have to decide what topic that post fits under. It is basically a categorizing system job which aims to put the posts that are made into the correct categories of broad topics. Click here
TruckeeInstagram Ads Evaluation Project

Help improve ads quality for one of the world’s largest image sharing sites.
Required: An active Facebook and Instagram account – The Instagram account must be connected to your Facebook account. You also need a smartphone with Android or iOS. The job has an Above Average work volume.
Click here
NileFacebook Ads Evaluation Project

Help improve ads quality for one of the world’s largest social networking platforms.
Active Facebook account required.
Click here
RattleThe goal of this project is to improve the relevance of search results for one of the world’s largest search engines. Click here
ParadiseThe goal of this project is to improve the accuracy of search results for businesses and entities on one of the world’s leading search engines. Click here
Mohawk Help improve ad relevancy on a popular social media platform.
Similar to FB
Falcon projects
Prospect MI Facebook Falcon Projects. Help improve ad relevancy on a popular social media platform.
Click here
Zeya Facebook Falcon Projects. In this task, you will be asked to research and evaluate claims stated in various pieces of content (e.g., articles, links, photos, or videos). 20 hrs per week, work is done via Facebook rating tool.
Click here
Copper The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
Click here
Alkali The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
Average volume of work.
No info
TigrisThe project consists of rating tasks focusing on improving the relevancy of advertisement results from a major social media platform. Consultants will rate advertisements to help the client understand which advertisements (also known as ads) people want to see, and why. The daily task of the consultant consists of rating a batch of ads in a client-owned rating tool. FB project?
CarmelUtterance annotation and evaluation.Microsoft Bing project aimed to learn search engine to recognize voice and human speech. No info

*group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform. Newly added projects such as Mohalwk, Prospect MI, Zeya, Copper and Alkali are most likely related to small Facebook testing projects. If you have been participating in one of the above listed projects, feel free to share your experience with us via leave a comment option!

Lionbridge project code nameDescription Reference for more info
Internet Assessor, Personalized Internet Assessor, RaterGoogle Search Engine Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
Internet Safety EvaluatorGoogle Youtube Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
Multimedia JudgeBing Search Engine Evaluation project Click here
Personalized Ads Assessor Google Ads Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
Online Map Quality AnalystMaps Evaluation Project Click here

Get Personalized Assistance or Advise
iSoftStone project code namesDescription Reference for more info
Online Ad EvaluatorBing Ad Evaluation project Click here
Search Engine EvaluatorBing Search Engine Evaluation project Click here

Ipsoot Project Review From Appen Connect – Pearl and Falcon PT

Ipsoot project 

There is a new project that we’re going to go over today called Ipsoot Project (also known as Pearl and Falcon PT) where you are shown a random post on Facebook, and then you have to decide what topic that post fits under. It is basically a categorizing system job which aims to put the posts that are made into the correct categories of broad topics. 

Ipsoot project qualification 

The qualification process for this project is straight-forward and not difficult provided candidates study and understand project guidelines. The qualification consists of 22 questions, candidates will be asked to apply categorization and assign relevant topic to randomly selected images. 

Ipsoot project Rating process

The posts you receive are random and in no order and appear on the left, including the ad and page information under it such as the page name, who the author is, etc. You will also notice on the top left a Post ID and a Job ID number. 

экрана 2019-08-05 в 20.44.33

The objective when working on the Ipsoot Project is to look at the post and spend 30-60 seconds making sure that you understand it. Of course, it is important to consider the text in the ad along with the media. Basically, you want to think about how a photo might be understood or interpreted in combination with the text in the ad. 

It’s important to pay special attention to everything that is in the ad, including the photo and any text that may be overlaid on the photo as well as any text appearing above the photo. 

If the ad is text-only it’s important to understand that text and including any emojis that might be used in the post to understand what the ad is trying to accomplish and how it should be grouped. 

Sometimes you will see a link in the ad, and at this point, you should click on the link and quickly scan the content of the page it takes you to and consider if the ad itself or the text relates to that landing page. 

Sometimes an ad will contain a video, and you should watch that video for up to 60 seconds. However, if the video is too long then watch the first 10 to 15 seconds, and then click through five spots and watch another 5 to 10 seconds. You should also consider any accompanying text that is inside of the video. 

On the right-hand of the ad, you will see a box that says ‘To which topic(s) is this post related. Select all that apply.’ You should select every topic to which the post is related on the other hand if the post isn’t related to any of the topics then select the no topic checkbox underneath the box of topics.  

Underneath of this, you will see a panel that says ‘Additional Attributes’ and you should select every attribute which applies to the ad. Of course, if there aren’t any attributes that apply to the ad then you should select the no attribute checkbox. And finally, the last thing to do is to click submit which will serve you up another random ad to check. 

Next, to the submit button, you’ll see some skip buttons. You will use these if you encounter any error that prevents you from evaluating the post and they are as follows:

Skip: Tool Error – You should use this if for some reason the tool doesn’t load. This can mean that the ad area will appear blank, or you may not see all the buttons or functions. 

Skip Other Language – the majority of the time you’ll receive ads in the language which you are assigned to so if you work in the United States your ADS will be either English or Spanish depending on what type of position that you apply for and if your multilingual. However, if you do not understand the language in the ad then you should do nothing else but click on Skip: Other Language button.

That’s pretty much it for the tool and it’s quite straight forward. 

The biggest challenge to this type of job is trying to understand Topic Relevance. To give an example, if it would make sense to see the post in an Instagram feed about a certain topic, then the post is probably related to the topic itself. 

However, it is not sufficient for a post to merely mention the topic. While it’s true that some posts explicitly mention the topic, some do not address the topic as it would show up in a search result. 

A good example of this would be an ad about a school competition winner who is pictured with his parents, his sister, and a dog. Simply looking at the ad isn’t going to help you unless you take a look at the text. So you wouldn’t tag this as “animals and pets” simply because there is a picture of a boy and his dog in the photo. However, if that same photo mentioned something about ‘man’s best friend’ you could tag it as animals and pets.

One of the questions that you may have is ‘What if the post is about more than 1 topic?’ Well, you should pay particular attention to a post that can be related to multiple topics because many posts can be coded with more than one category. 

We’ll give you some examples. 

For instance, let’s say that you have a post about a new economic policy. In this case, you would tag ‘politics’ + ‘business finance and economics’.

If a post was about someone using a new fitness routine to improve their health, then you would tag ‘health and medical’ + ‘fitness and workouts’.

A post about a politicians stance and welfare programs you would select ‘politics + social issues’ 

If the post is about the best restaurant three friends found while traveling through Italy then you would tag it ‘food and drink + travel and leisure + relationships friends and family’ 

As you can see, it’s quite possible many posts can have more than 1 topic. 

There are a few other rules when it comes to tagging topics about a post when it comes to regard for quality, professionalism, or the size of the audience for the post. 

We’ll show you some examples of how a topic doesn’t change regardless of these attributes. 

If the post is a professional video of a Justin Timberlake concert then you would tag this under ‘music and audio’.

But let’s say that the post is of a band performing in a bar. You would still tag it ‘music and audio’. Even though that band may not be as popular as Justin Timberlake.

Even if it is a post of a cell phone video of a 3-year old singing in a car, you would still tag this is as ‘music and audio’.

However, it is important to pay special attention to whether the post is related to any entity such as unique people, places or things. For example

The post is about President Trump – You would tag ‘politics’

If the post was about a McDonald’s restaurant, then you would tag ‘Food and drink’

And if the post was about The Beatles, then you would tag ‘Music and audio’

However, you should be careful to not assume that every post that mentions an entity is it about that entity itself. For instance, a video about the Olympics might show a logo of its sponsor, McDonald’s, but that is not enough to consider the post to be related to McDonald’s itself and you wouldn’t tag the post ‘food and drink’. 

When the author of the post is a well-known individual like Barack Obama, and he’s posted something saying that the Chicago Bulls are the best team ever, you would tag this as politics because it’s just a personal opinion by an individual strongly associated with politics. 

If you use Facebook, you probably understand that memes are something that you see quite often and its important to understand how to tag a meme. Memes often used content from one topic to make a statement about another topic. In the case of memes, you would actually tag both topics. For example, if you see a meme of a TV character looking sad with the caption ‘how I feel when my pet walks away from me’ you would tag this TV & Movies + Animals and Pets.

There’s going to be a lot of times when you’re going to have to do research because you’ll find names or phrases in a post that you do not understand. You should always do a search to find out if you’re not sure when referencing a topic. 

It’s also a good idea to identify some of the definitions in the guidelines. 

Topics describe the subject of the content. It is important to determine if the content in a Facebook feed is relevant to the topic. This means that it’s not sufficient to call something ‘related’ to a piece of content that merely mentions the topic. There are some examples of topics and how they should be grouped. And although the guidelines go over each one and give inclusions and exclusions, it is not an exhaustive list but merely examples.

If you are working on the Ipsoot Project then we would like to hear from you and your experiences. If you need help with any information on this page, then email me and I’ll get back to you as soon as possible. 

Appen Arrow Ads/Lionbridge Ads Rating Evaluator Project Review

Appen Arrow and Lionbridge Ads rating projects overview

The Appen Arrow Ads project and Lionbridge Ads rating are similar search engine evaluation project that is using the same Rater Hub platform that the other two large companies Appen and Lionbridge are using to rate search engine results depending on a query or intent of the user. Basically, you are doing the same thing as other search engine evaluators, but the difference is that you are judging the quality of ads and advertising-related data as they are calling it. The feedback that you provide is supposed to help the client monitor and improve their advertising as well as functionality.  

So what you are doing with this certain project is taking a look at different ads on web pages, and using the Rater Hub tool, you answer some questions on your findings. It sounds easy but answering these questions, and the answers that you provide are coming from guidelines that need to be closely followed. It’s not simply about providing your own feedback but providing the feedback based on the guidelines themselves and specific metrics.  

Appen Arrow and Ads Evaluation program purpose 

As a human evaluator, you are not approving or disapproving the ads you evaluate, and you are not deciding directly which ads get shown or don’t get shown. In fact, even though you are providing scores and feedback about individual advertisements and advertisers, your evaluations don’t generally have any direct effect on those advertisements or advertisers. Sometimes the ads you rate are from a long time ago. Sometimes they are in a different form than they originally appeared. Sometimes they were never shown by Google at all, in any form. Basically what you are doing is collecting data to help train the algorithms that will eventually assess the ads being shown on their own. Without human interaction, this would not be possible. So this job is very important to the engineers who are collecting the data from you.  

This is the main reason why it’s very difficult to pass the assessment part of this project. Your judgments are used to determine whether ads are getting better or worse over time, to see whether particular kinds of ads are working well or not, to see what kinds of ads cause problems, and to see what kind of impact a particular change to our systems will have. When Google engineers design new systems or propose changes to existing systems, the human evaluation data you provide is used to judge whether those proposals succeed or fail. This data is fed to computers that train automatic algorithms.  

This is also a big reason that a lot of raters end up being terminated or losing their contracts. Accuracy and quality of your ratings are very important. You should always check the Ads Guidelines if you aren’t sure about something that you are looking at. Even some raters that have been doing this job for a while still refer to the guidelines.  

Appen Arrow and Lionbridge Ads rating program requirements 

Getting on this project can be difficult. There is a long assessment required. Its mentioned early on that you will perform all work and including the assessment tasks on a non-shared and non-public computer. That’s pretty straight forward. You’ve got to have your own computer with a high-speed internet connection. And according to them you can’t take your laptop to the coffee shop and work from there. So it’s a very good possibility that your home IP address is recorded in the RaterHub, and you need to use this same connection to perform all work. Anything else is strictly prohibited. 

Another thing to mention is that you’re going to need to use either Firefox or Chrome web browser but that should hinder anyone since those seem to be the main web browsers being used by the masses. This is because the Rater Hub tool will not work on other browsers.  

If you do pass the assessment, then you may be offered a contract position. Usually, the contract hours are part-time 20 hours a week, but Aztec states 19 hours, Monday through Sunday. Once you complete the assessment, the results are usually sent within 48 business hours after your deadline to complete the assessment has passed, not 48 hours after you submit the assessment.  

As with all projects, we can’t stress enough the importance of one thing. Thoroughly read the guide they send you and have it open and ready while you are taking the tests. Make sure the minute you get the link to review the guide, that going over it is the first thing you do. Give yourself enough time to digest everything in it at least 1-2 times before taking the tests. The guide itself has to be accessed directly through EWOQ which is part of the RaterHub portal. You’ve got to be logged in using the Gmail address you used when you signed up at Appen/Lionbridge. This Gmail account should be a dedicated email account.  

Since you are not able to download the guide, you should bookmark the page with the guidelines so that you can access it quickly and easily. We said that it cannot be mentioned enough and it’s very important that you read and understand the guidelines completely before starting or you will fail the test.  

Appen Arrow and Lionbridge Ads program payment rates 

Wages on each contractual hour depend on the country and can vary between $4 to $15 per hour. For more information on wages read here:

Agents will invoice vendors for rendered services once per calendar month. Fees to independent agents are paid (via direct deposit) within 30 days of receipt of a valid invoice. The payment is made on a monthly basis via direct deposit into your bank account. Appen projects are paid now by Payoneer. If you do not have a Payoneer account yet you can get yours here:

Note: By using this link for applying for a Payoneer account/card we will both benefit by earning 25$.
Appen Arrow and Lionbridge Ads program exam

The first set of tasks are multiple choice questions (in total 45 questions) that come straight from the Search Ad instructions in the Ads Guide itself. After you have completed the comprehension questions, you will be shown a set of assessment tasks (in total 190 tasks of different types: search ads evaluation, user interest, key words, etc) that asks you to rate just the first section of a Search Ads task. Once this is done you’ll get a set that asks you to rate just the second section. Finally, you’ll get a set of complete Search Ads tasks, User to Keyword, Close Match Evaluation to rate.

The typical exam tasks are a query and an ad creative generated by a search engine with respect to the user query. You as a rater will need to apply rating rule to determine how promising or unpromising is the ad with respect to user intent. Once you evaluate the ad then you will be asked to evaluate a landing page associated with the ad and to determine how satisfactory is the page and website content in terms of meeting the user intent needs.

Lionbridge Ads rating project/Appen Arrow Exam Tips

Additionally, I would like to give general tips on how to proceed with Aztec/Ads Rating Project qualification:

As usual, before proceeding with rating you need to make sure you clearly understand user intent. If not, use Google search to research the query and establish the dominant intent, and the secondary intent if one exists. Sometimes it’s hard to understand what the user intent is without doing the search yourself;

Search Ads Evaluation Tasks

Select the appropriate query flag if required. For instance, if the query is in a foreign language, or it is clearly a porn related query, make sure you select the appropriate flag. Remember that when you are doing ratings, the language should be in your contracted regional language. So if you are working in English, and you get a page in another language, then flag it appropriately.

Examine the ad creative and answer the question of how promising the ad is. You should refer to a specific section of instructions, and not evaluate the ad creative based on your own perception or personal likes or dislikes. Pay close attention to the wording of the ad and the URL. In many cases, the wording of the ad may indicate the ad deserves a porn flag or has reference to another search engine.

Check the query and ad creative for the navigational flag. If the query is looking for a specific page or website and the ad suggests the website, then the ad and the query should be given a navigational flag.

There are a few scenarios in which you should pay special attention to. The first one is when the ad offers to repeat the search in another search engine. In this case, the appropriate rating for ads and LP (Landing Page) is somewhat unpromising and doesn’t satisfy the query because the action won’t bring the user closer to his goal. Google wants to provide the user with the quickest possible and satisfying solution for the user intent and query. Sending a user to another search engine defeats that purpose entirely.

Select the ad’s creative rating, or how promising the ad is in relation to user intent. Once you’ve completed the first part of the task you will then be directed to the Landing Page section.

Visit the landing page and examine the page based on the guideline rules of how satisfactory the page is based on the query or intent. Once again, you should not use your own judgment but instead, learn how to apply the guidelines rules. In many cases research on the website is required, for instance, a quick product search will show you more information about the page, and help you understand if it meets user intent based on the query.

Make sure to pay special attention to the user location. If you see that the user is looking for something in a specific location, such as a restaurant in a specific location, and you get a restaurant in a completely different location, you should apply a negative rating. In this case, it’s a good idea to refer to the guidelines and review typical user location scenarios.

User to Keywords/User Interest Tasks/Close Match Evaluation

Before proceeding to these types of tasks make sure you read and understand instructions because the logic is different comparing to the approach required for search ads evaluation tasks. You will be asked to rater how much is interesting for a user the suggested page/url based on examination of a user’s current web interest.

Once you have completed all the assessment tasks, including the survey, you will temporarily run out of tasks. This is because your answers will be checked before you will be allowed to move forward. Either you will pass this part or not. You will not see the Acquire Tasks button at this time. Remember that you need to wait 48 hours after the assessment deadline and not 48 hours after you submit. The assessment will be complete after you submit all the tasks available. 


Appen Arrow vs. Lionbridge Ads rating project  

Many people ask me which project is better and which one should I join. In my opinion, I think that Lionbridge is definitely better in terms of offering a better payment rate compared to Appen. It’s also important to note that the exam qualification and training are paid by Lionbridge where at Appen it’s not. So take this into consideration when you apply for both of these companies. Three days of unpaid training is a lot for people to deal with and really when you think about it, no one wants to do an exam for three days, and then fail it having put all the time and effort into it to get a job.  

As always, if you have any questions or thoughts leave a comment or shoot me an email.  

Wishing you great success in your work at home career!

Appen Connect Paradise Project Falcon Megataxo

Text-AnalyticsParadise project overview

The Appen Connect Paradise Project Falcon Megataxo is a new project from Appen. As you may know Appen acquired Leapforce in 2017, since then the company has decided to close it is own project management platform known as Appen Global and move all projects to Appen connect. The group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform.

Appen usually has a lot of new and different projects, and a new project has recently come out called Paradise (previously known as Megataxon) which is an Facebook ad review job. This is an interesting project because most of the Falcon Facebook projects usually are rating social network stuff like actual videos from pages or the actual content on Facebook or Instagram. This project focuses on the types of ads that you would see on Facebook. The objective on the project is to decide if the core purpose of the ad is presenting a product, service, or advertising content such as getting you to go to a Facebookpage and subscribing to that pages feed.

Rating process

Once you understand what the ad is about, you categorize the ad by using a tool that is specially designed by and for Facebook.For each ad there are basically 3 steps to putting each one in the proper category. For the first step, you have to choose the core purpose of the ad and decide if it presents a product, a service, or content.

Along with the tool, you get the project's guidelines that go over all the categories, and it does a very good job of explaining the categories and how you should categorize them. As previously mentioned, the object of the job is to categorize ads as you see them, and you are asked to spend no more than 60 seconds on each ad. This is likely due to the sheer volume of ads that are currently cycling on Facebook. However, there is no minimum view time to apply the tags. With that said, if you are on this project it might take some time to get used to deciding what ad should be categorized where, but after time it is likely that you will pick up speed. It should be noted, however, that if you rush through the ads and categorize them wrong, you could be taken off the project. Once again, they want you to spend no more than 1 minute on each ad with no minimum view time, but you also shouldn’t spend more than a few minutes on each ad. Doing this will surely get the attention of a project manager asking you if there is a problem.

There are times when you can automatically skip the ad if you run into errors, the ad won’t load, the ad is not in the target language for your region, for example the ad is not in English, or there are other issues where you immediately see that the ad is illegal, offensive, or needs to be immediately escalated such as in a case of child exploitation, child nudity, hate speech, acts of terrorism, etc. If you see this bad stuff you have the ability to escalate the Job ID to a manager who handles escalations. If you have to skip the ad for any of these reasons, you just indicate the correct option at the bottom of the Facebook tool.

After you select the purpose of the ad, the next step is to choose a top-level category for the ad. Some of the top level categories have some sub-categories and you will see these appear below the top level categories if there are sub-categories available. This is an extra categorization for some top-level categories. If you select a top level and see a sub-level category appear, you simply choose the best one that fits.

Defining the purpose of the ad simply asks you to choose whether the ad is a Product, a Service, or Content which is defined as the sharing of information that doesn’t provide a specific physical product, service that the reader can buy. Content includes all digital content, articles about an experience with a product, photos of products with no specific call to action, or if the ad uses the ‘Learn More’ button which more than likely leads to content.

Choosing the purpose of the ad should become second nature and you should simply ask yourself ‘What is the core purpose of this ad?’ Ads might reference many different products, services, or content, but your goal is to consider and categorize what is most important for the reader.

The tool itself isn’t difficult and is fairly straightforward. Under the ad, you simply choose the appropriate categories by clicking the radio buttons. Each one of the radio buttons opens with different categories or sub-categories. After you have selected the categories, then on the bottom under that is the submit button. You want to be sure that you have chosen the most correct categories for each ad. Also at the bottom next to the “Submit’ button are the other buttons that we previously mentioned. Those are that the ad has some error, the ad won’t load in the tool, or the ad is not in the English language. You will also see the ‘Other Issue’ button here where you would report an ad that contains something that it shouldn’t that may be offensive or illegal. In this case, the ad would go to one of the managers of the project where its escalated and they take action to remove the ad altogether. You should understand that you have to be careful using this and that the ad should really be offensive and not just something you disagree with or don’t like.

One of the things that is not mentioned in the guidelines is that you might see the ad appear more than once during the week or the month. You should try to remember what you have categorized this ad as and not change the results on the next day if you see the ad again. It’s important that Appen collects precise data. An example of this could be categorizing an ad as something different one day, as opposed to the day before. This will surely get the attention of a product manager. Try to fully understand the categories and email your project manager if you have a question. This will ensure you are able to stay on the project if there is something you don’t understand. Admittedly, there are a lot of products and services on Facebook and categorizing them all correctly might be confusing at first.

The guidelines are really concise and fully detailed and go over the majority of products and services in order to help you categorize them. The guidelines are also very helpful when it comes to deciding what is content or not, and discusses what should fall into those categories.

In conclusion, the Megataxon Falcon project looks to be a very easy job. It might be interesting for some people who want to make a difference in what ads are seen on Facebook, however, after 4 hours of doing this job, it might become tedious on a day to day basis. But for those who don’t mind doing this kind of job, it’s definitely not something incredibly vague or difficult such as some search engine rating jobs.

If you are working on this project, we would like to hear from you and your experience with it. And if you have any questions, you can email me or leave a comment and I will reply as quickly as possible.

As Payoneer is now the only way to get paid by Appen, and if you do not have a Payoneer account yet I invite you to apply for Payoneer account via referral link:

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?

Common Issues New Raters Face – Appen Arrow/Lionbridge Ads Evaluator

Support_636x350-600x330Arrow & Lionbridge Ads Evaluator projects workflow 

Want to know what the common issues new raters face? After you have passed either the Lionbridge Ads or the Arrow ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a high chance that you will still see some English tasks come through. In fact, 70% of all tasks will be in English language.  

RECORDING AND INVOICING WORKING TIME Appen Arrow and Lionbridge Ads Assessor project

Time management is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Times sheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.  

One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.  

Appen Arrow blind test and Lionbridge Ads Assessor quality review tasks

Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. In total during  a month working cycle there are will be about 150 blind test tasks of different types of tasks, raters are expected to obtain at least 70% of correct answers in the pool of blind test task, raters with lower quality score will be removed from the program.

Reasons for losing your service as an Ads Quality Rater 

It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.  

Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.  

Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.  

IP address conflict/Internet Network sharing

In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.


Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.

No tasks are available, all task options are not available anymore except the QR code

This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.

Lionbridge (Ads rating project) vs Appen (Arrow)


In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.

Appen has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen(Leapforce) has again invented their own metrics to pay less to its raters.

Working hours and flexibility

At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.

At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.


Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.

Appen(Leapfroce) pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?

Payments rates

In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.

More info about payment rates is available here:

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Lionbridge and Appen When Combining Several Projects?

General Support

The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of  a monthly practice test and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.

Optional Agreements

It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.

The Final Word 

If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too. 

Appen Arrow invoice correction required due to low RHP

what is mystical RHP and is this practice legal?

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater. 

Appen has productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters.

In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RPH metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer (Google) understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen invented their own metrics to pay less to its raters and make more money by invoicing actual time spent on doing task by all raters to Google. Good business, in fact. In this respect Lionbridge plays a fair game and most of raters get their time sheets approved without any issue.  

The example of the email is bellow:


Dear Search Engine Evaluator,

Our records show a significant discrepancy between the hours you’ve written in your invoice and the amount of work completed. You over-billed more than xx hours for the week of x/x/201X – X/X/201X.

Please check our Chat feature, review our project information page and Intl FAQ.
Do not write your missed hours in your invoice time card as the entry will be considered billable.

In the meantime, these hours will not be approved until clarification has been received.

Please let us know if there’s ever an issue outside your control that is impacting your overall productivity such as tool errors.

The productivity rate/ rate per hour (RPH) is calculated by looking at the average of team who worked on a particular task. When you have a low RPH it negatively impacts the RPH average of the entire team and project. It is our expectation that raters meet the average rate per hour on a consistent basis. Listed below are some tips to try if you have not done so already, to hopefully push your rate per hour to the average:

• Time Management: Refrain from multi-tasking while working on EWOQ. Watching videos or listening to music not associated to a task may impact your focus and performance.
• Timing Devices:

In order to get a better feel for how fast or slow you are working, you might consider using a desktop timer, Toggl, or stopwatch. Always exit EWOQ when taking a break, no matter how short. Do not count the time spent on breaks towards your billable hours.


A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.  

What raters can do in this situation?  

      1. Before making any invoice correction, ask Appen for detailed clarification about RHP: what is the RPH ratio for all raters; what is your own RHP ratio, what is recommended RPH ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks,  ask why customer does not indicate recommended time for completion for many types of tasks.  Be ready that they will not provide any info, because there is no official RHP approved by customer (Google).
      2. Point out that fact there is neither any policy regarding RHP can be found on Appen connect project platform nor customer  (Google) does not provide any recommended  time for most of tasks.
      3. Notify Google about the issue by sending email to the ads rating project admin (Google Ads Eval: to make them aware that this is some kind of issue that really needs to be investigated. The message should be the following: actual time spent on doing live tasks was not approved due to RHP, what is the RHP ratio for all raters; what is your own RHP ratio, what is recommended RHP ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks,  ask why Google does not indicate recommended time for completion for many types of task. FINALLY  ask them – is it legal that Appen cuts hours based on mystical RHP which is not trasparant?

Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.  

If you have questions or comments then leave them below.  

Appen Nile Project for Facebook Review and Qualifications

Appen Nile project – Ads Evaluation for Facebook

Appen offers more than just rating projects for Google. Currently there is a project called Nile where, as a rater, you can review ads based on your likes, your interests, or your web viewing habits.

You can change these settings in Facebook to adjust what ads you get ‘served’ when doing the ad ratings. The ads are not based on companies that you have already manually liked, but ads that are similar. When starting the project, you get 20 ads to rate. These can be anything from video ads, to just plain text ads or ads with a photo. 

Before you are accepted to the Nile project, you are asked for your Facebook profile URL. You must have at least 50 friends on this Facebook profile. Appen simply checks this and nothing else about your profile. You will hear back from Appen within a week if you are accepted into the program. In order to stay in the Nile program, you must work no less than 5 hours per week.

When rating you are asked ‘How much do you feel that this ad is ‘relevant, useful, entertaining, if the ad is offensive, or misleading. You can rate each one of these on a scale from 1-3 using radio buttons.

You are then asked ‘How much do you want to see this in your news feed’ and are shown a scale from 1-5 from ‘I definitely do not want to see this in my feed’ to ‘I definitely want to see this in my feed. Once you have done this, there is an explanation that asks you ‘In your own words, why do you feel this way about the ad.

It is not known if the comments are actually looked at in the ads or not. You don’t need to create a huge explanation. A simple reason why you do or do not like the ad is good enough. When you are finished rating the 20 different ads, you are then asked to wait for one hour to complete what is called a ‘Bakeoff’ task. The Bakeoff task determines which ads you liked and which ads you didn’t. It asks you to choose out of 6 ads in a side-by-side comparison, which one you would like to see in your feed over the other. This task should be completed exactly one hour after you’ve rated the ads. Currently you are allowed to only work one hour per day on Nile. The hour that you wait for the Bakeoff tasks is not paid. So you charge 1 hour for the Nile ratings and the Bakeoff task which takes you less than a minute to do since all you are doing is clicking which ad over the other you want to see.

You must work at least 1 hour per day to Nile is an interesting project and will probably go full time, but currently there is just one hour allowed per day. That’s an extra 30 plus hours per month and totally worth the time. It also pays the same hourly wage as the Google rating program.

Appen Nile Project Qualification

Appen has introduced qualification exam for Nile project. The Nile project exam consists of 30 questions designed in a way to assess your understanding of basic concepts of how to evaluate ads and search results. Again before taking this exam you will be given an access to specific guidelines which you need to study carefully before proceeding with the exam itself. Do not underestimate the complexity of the Nile qualification exam; some of the questions are tricky and confusing in its wording, so without preparation and thoughtful approach you have a very low chance to pass this exam. One attempt of taking this exam is allowed. Once the exam is passed (you will get result immediately after completing the test), you will be offered to sign a contract.

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest

Appen Connect Echo Music Search Project Review


Have you heard of the Echo Music Search project from Appen Connect? I would like to tell you about this new recently launched project at Appen. I started on the project a month ago, and so far I think this project is one of the most interesting projects because it has to do with evaluation of music at iTunes! Yes, Apple also decided to work on improving of search relevancy of their services, and I think we can expect lots of new Apple projects being added at Appen Connect. 

This new project is code named Echo. The nature of the work is to evaluate an iTunes user query, and the relevance of the resulting content based on the specific artist, album, song, or playlist. Like the Yukon project, this music search project has more or less the same evaluation scale and rating logic: (Fully Met vs Navigational, Highly Met vs Excellent etc.) However, some rating rules are specific and unique for this project. 

So far I have been working on this project for about 1 month, and I really enjoy it. Besides making decent money, I expanded my favorite song list significantly. The working process is easy going and fairly smooth at around the 1-month mark, and not much intelligent effort is required. But before achieving this level of work flow, raters will have to carefully read and understand the 20 page guidelines. In order to develop a deep understanding of rating concepts, and deliver a high quality level of service, you need to thoroughly understand these guidelines. As with any Appen  project, in order to be successful in the project, high quality output is required. As for the technical process, Apple has developed for its raters a very convenient tool called ‘baseline’. This tool is an integrated tool on a website from where raters perform their evaluation. This tool is convenient and user friendly, and simple to learn to use.

Many of my readers will be interested in how to be invited to this project. Appen offers a lot of different types of projects that you are able to work on. Unfortunately, is not likely that you would be contracted to start on the Echo music project from the beginning if you are hired by Appen. You would be more likely to start on a larger project first such FalconInstaAds or the Arrow project. Once you have started working at Appen, and you are on Facebook project, like any of the social media projects that they offer, you will likely get an invitation to come aboard and start on other projects including the Echo Music Search Project.

Project Requirements:

  • You must hold an Apple ID, or willing to obtain one
  • You must own an Apple Device.
  • You must be available 5 days a week and for 
  • Max 20 hours per week
Echo project Qualification

Prior to starting production, Appen requires that candidates go through Echo Qualification which is a 30 question quiz, real tasks which have to be evaluated in terms of query vs result, in which you will be given a music search query, and the result generated by iTunes. The passing score for this test is 70%.

If you are not confident with going through the qualification quiz you may request my assistance via contact me channels.

Production and payment

Payment is made per hour of work and pay rate is different for each market.

  • This is a long term project, your daily limit is up to 4 hours a day, 5 days a week (includes weekends). 
  • Expected RPH is 60 tasks per hour. But because this is the initial wave of the project, Appen wants raters to value Quality over RPH until the project progresses. 

As per Appen project requirement raters are allowed to work 20 hours per week max, or a max of 30 hours in total if raters work on other Appen projects. Raters cannot exceed 30 hours per week working on 2 different projects at Appen. On average you can make about $390-$405 per week at the average rate of $13.50 per hour. If you are interested in working on several projects at once, then this article is a must read: 

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Lionbridge and Appen When Combining Several Projects?

In fact there are lots of different cool and legitimate projects and opportunities to make money online. I will continue sharing my experience with my readers, I’ve got an upcoming article on another project called – Maps Analyst, which also is going under Apple Map service, there is another cool translation and localization project at Netflix, available worldwide in many countries, so continue reading my blog to get valuable tips and insights!

Welcome to Echo Exam Simulation Tool
Why do candidates fail the Echo project exam?

When you apply for ads evaluation jobs, you have to pass an Echo project qualification exam. The biggest problem with the Echo Qualification Exam is that it is very difficult to pass with one or even two attempts because no feedback is provided in case of an unsuccessful attempt. In fact, even Appen and Lionbridge explain in a welcome mail that there is a high chance of failing it. With that said, you’re going to need all the help you can get.

Useful links on the topic:

Studying the guidelines and doing the exam simulation practice are essential in passing the actual exam

If you think this is an exaggeration, then just search for comments on passing the exam and see what you will find. You’ll find in a lot of cases people who claim that they studied for several days before the exam began. With that said, it’s just not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort in the one-week time frame they give you to study the guide. The guide itself is designed in a way that assesses your ability to understand and apply rating concepts and the rules based on the hefty guide. There is a lot of material to go over, and it can be a bit confusing. But once again, you’re not going to get anywhere near passing if you don’t start going through the guide.

What is the Echo Exam Simulation Tool?

The tool will help anyone who is getting ready or considering taking the Echo project qualification quiz. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include the most tricky rating scenarios which are tested in the real Echo project exam quiz.

The Echo exam simulation tool has 17 tasks + 4 demo tasks, the same number as the real exam has, these tasks are the main exam tasks types which cover basically all rating concepts, and which will help you to develop rating skills which will be tested during a real exam.

You might be wondering how using this tool can help you pass the exam. The way that it is designed will show you exactly your problem area that you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to disclaim that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from most typical exam tasks, and is designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam.

How to Proceed With The Exam Simulation Tool?
Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains 4 typical Echo quiz tasks so you can see how it works. Once again, the demo is 100% free and you’re completely anonymous.

Exam Simulation Full Version

The tool has 17 tasks, the same number of tasks as the real exam. These tasks are the main exam tasks types which basically cover all rating concepts, and will allow you to develop rating skills which will be tested during a real exam.

Explore the tool here:

​Appen UCI Zeya – Falcon Post Interest Tagging Project

Appen has several projects within the Falcon Code Name and the latest one I have come across is what is called Post Interest Tagging. It appears that the client is FB and the job doesn’t appear to be very difficult. Although I can personally attest that the quiz they give you to be contracted into the project is tricky and if you don’t go over the short 1 page guidelines and the examples they give you, you will likely fail the quiz.

Post Interest Tagging is a job that has to be done by a human because you are checking the relevance of each post and then deciding on if the post is relevant for the term that you get or not. For example: If you get a post asking you if it has something to do with cars, but it’s a post about trucks, you need to decide if that post would be relevant for people who are interested in cars, but not about trucks. This is tricky though and its most likely that this post would be answered as yes it does have something to do with cars as trucks are automotive. This is where the quiz is tricky and you’ve got to be careful when doing it. You only get one chance to do this quiz, so again, take your time and use the internet.

The quiz is 25 questions and if you miss more than 1-2 questions, you will likely not be offered a position as the management team goes for those who score the very highest on the quiz. The quiz is not timed, but you do have a deadline of when to finish it. The biggest and best advice here that I can give is to use the internet to check and see if the post in question has anything to do with the subject. You always get a subject like ‘Baby Clothes’ and if you aren’t sure what a toddler jumper is, then its best to check the internet and take your time when you are doing the quiz.

Post Interest Tagging gives you 4 hours per day Monday through Friday with no weekend work making it the perfect job for those who don’t want to work on the weekends. You’ve got to put in 4 hours per day though as this is the minimum. Now for this job, the only thing you are doing is answering yes or no to the post. You have a spot for a comment, but it is not necessary to comment on each post. It is important to pay attention to what you are tagging. This can be a very tedious and monotonous job and you would likely tag a couple wrong, and it’s also very likely that you would be removed from the project if you start missing too many. There is also an option to choose ‘Not sure’ but you don’t want to use this option very often either. In some cases, there might be a category that is really obscure and you wouldn’t know how to categorize this. Then you would use this option.

As I mentioned before, this job can get a little boring, but if you don’t mind doing such a job for 4 hours a day, then it can be a perfect at home job for you. As always, if you have any questions about this project, or need some help or advice, please leave a comment or send your question via email.

Appen Rattle Project – Falcon Video Recommendations.

online-social-medi_2734204bThe Appen Rattle Project is here. Appen now has a new hot opportunity that anyone with a verified Facebook account can work on. It’s a very unique project that involves watching live video or live video feeds and rating these videos based on their content and whether or not they are useful to you or not. And while it sounds like a very easy job, it can be tricky if you don’t understand what Appen Hill is looking for when it comes to data collection.

Application process and project set up

The process to apply for the job is the same process that you would use normally for any other project that Appen offers. Once you apply for the position, if you are selected, then you will start to receive instructions on qualifying. Yes you need to qualify for this particular job, but it’s nothing too difficult.

How To Apply for Search Engine Rater and Social Media Evaluation Jobs at Appen and Lionbridge


The process will take 2-3 days in which you will be able to review the project guidelines, any additional materials concerning the job, including the presentation, and then completing a ‘Qualification quiz’. It’s very important that you use the same email and I recommend creating a new gmail account just for this purpose. The reason is that the emails that you will get are time sensitive, and having a new email will ensure that they don’t end up in spam or junk. Also it’s very important that you use the email address you have on file with your Appen application. So if you have already applied for Appen, use this same email.

After you have applied for the position, you will get an automated email from Appen saying that you will be contacted for the Rattle Video Recommendations Project. Once you have been contacted, a 6 step process begins. Applying for the project and making sure that your email is verified is the first part.

FACEBOOK ID Requirements

The second part of the process will be setting up the Facebook Profile. You must use your Facebook account and it must be at least 1 year old and have no less than 50-55 friends. Your Facebook account has to be 100% truthful using your name that is on your Appen application. You should not have more than 1 Facebook account as this goes against Facebook TOS. Age group, location, name, and gender must be accurate.

Facebook Rating Tool 

Your Facebook ID will be used to set up your access in the Facebook tool you will use to complete daily assignments. Once set up, you’ll be given a batch of videos/pages which you will then evaluate based on the project guidelines. Through the tool, Appen Project Managers will have access to those batches, your ratings and the comments you provide. Appen does not have access to your personal FB information such as your account, messages, posts or network of friends. There will be no changes to your actual News Feed. Everything will be the same except that you will have access to what is called the FRT Tool.

Very soon you will be getting the Rattle Video Recommendations Project Guidelines. You should take some time to read through this carefully to understand exactly what you will be doing as a video rater, and most importantly, that you give the correct ratings and data. While it is an easy job, it can be a little tricky to understand how you should comment, and how those comments should reflect the numbers rating that you have given. Otherwise, you could find yourself moved off the project quickly.

The next step will be getting your Sharepoint Access where you will be able to download all materials, including the guidelines, for the project. Take the time to get familiar with Sharepoint. This email will come from Appen-Connect Support.

Rattle (Falcon Video recommendation) project qualification quiz

The last step is receiving the instructions on completing the Falcon News Feed Qualification Quiz via Appen Connect. Once you complete the online module, you will need to complete the quiz to qualify. Quiz results are instant after each question. IMPORTANT! You only get one chance to pass this quiz. They will not give a second chance to pass it under any circumstances. Qualification should take around 5 hours (Review of all emails and project documents, review of guidelines and PowerPoint presentation, open forum attendance and quiz).

Once you have completed the Quiz and successfully passed it, you will then be contacted by a Project Manager 1-2 days after the quiz. It is not necessary to email them.

Rattle Video  Recommendation project production 

Production work starts as soon as you have successfully completed the entire process. It is VERY important to once again mention that the deadlines for the process are very strict.

The Falcon Video Recommendations Project requires that you work no less than 5 days per week and one of those days must include a weekend. You can split the days as you like as long as one weekend day is worked. Each batch consists of 28-30 videos in which you are given a total time of 2.25 hours to complete. Time is automatically kept track of in the rating tool. You cannot go over the 2.25 hours per 30 video batch. If you stay on track, this comes out to about 4:00 minutes per video. There is a task at the end called the ‘bakeoff task’ which takes about 5 minutes to complete.

Further details concerning how the system actually works will be laid out in the project guidelines.

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?