List of Projects And Code Names For Lionbridge & Appen Connect

If you are looking for a list of projects out there from companies like Lionbridge and Appen, we have them. As you may know Appen acquired Leapforce in 2017, since then the company has decided to close it is own project management platform known as Appen Global and move all projects to Appen connect. The group of projects known as Falcon (mainly Facebook-related projects) now is no longer actual, these projects also have been moved to Appen connect platform.

Taking into account the fact that all projects have code names and little description is provided, many candidates might be lost in terms what project to go for. 

Bellow you will find a list of most popular projects and links to more info about each project:

Appen connect
Project code nameDescription Reference for more info
YukonGoogle Search Engine Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
ArrowGoogle Ads Evaluation ProjectClick here

Exam Simulation Tool

Get Personalized Assistance or Advise
ShastaMaps Evaluation Project

Provide relevance and accuracy ratings for one of the world's largest and most popular mapping applications.
The job has an ABOVE average volume of work.
Click here

Get Personalized Assistance or Advise
Echo Apple iTunes Music Search Results Evaluation Project

Improving the relevance of search results for music-related content.
Raters must have access to an iOS device capable of running iOS 12.x. and higher. The job has an average volume of work.
Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
CableApple TV Search Result Evaluation.

Search Relevance task focused on improving the relevance of search results, based on the most likely intent of user searches. Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo

Get Personalized Assistance or Advise
JavaApple Store Search Relevance project focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4
Similar to Echo
SharpApple Search Relevance task focused on improving the relevance of search results within different media categories.
Raters must have access to an iOS device capable of running iOS 12.1.4
Similar to Echo
IndexApple App Store Search Results Evaluation Project

Search Relevance task focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo
Wouri Apple Safari Search Result Evaluation project. The goal of the project is to understand the user intention as represented by the query and determine how well or how completely the result satisfies the user's need. Click here
Pecos/Ipsoot (also known as Pearl and Falcon PT)Post categorization on Facebook. You have to decide what topic that post fits under. It is basically a categorizing system job which aims to put the posts that are made into the correct categories of broad topics. Click here
TruckeeInstagram Ads Evaluation Project

Help improve ads quality for one of the world’s largest image sharing sites.
Required: An active Facebook and Instagram account – The Instagram account must be connected to your Facebook account. You also need a smartphone with Android or iOS. The job has an Above Average work volume.
Click here
NileFacebook Ads Evaluation Project

Help improve ads quality for one of the world’s largest social networking platforms.
Active Facebook account required.
Click here
TahoeThere is a Facebook project called Tahoe that uses a team that checks events that come up in Facebook results. The main mission of this program is to bring people together through real-world activities by showing real events.Click here

Get Personalized Assistance or Advise
RattleThe goal of this project is to improve the relevance of search results for one of the world’s largest search engines. Click here
ParadiseThe goal of this project is to improve the accuracy of search results for businesses and entities on one of the world’s leading search engines. Click here
Mohawk Help improve ad relevancy on a popular social media platform.
Similar to FB
Falcon projects
Prospect MI Facebook Falcon Projects. Help improve ad relevancy on a popular social media platform.
Click here
Zeya Facebook Falcon Projects. In this task, you will be asked to research and evaluate claims stated in various pieces of content (e.g., articles, links, photos, or videos). 20 hrs per week, work is done via Facebook rating tool.
Click here
Copper The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
Click here
Alkali The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
Average volume of work.
No info
TigrisThe project consists of rating tasks focusing on improving the relevancy of advertisement results from a major social media platform. Consultants will rate advertisements to help the client understand which advertisements (also known as ads) people want to see, and why. The daily task of the consultant consists of rating a batch of ads in a client-owned rating tool. FB project?
CarmelUtterance annotation and evaluation.Microsoft Bing project aimed to learn search engine to recognize voice and human speech. No info

*group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform. Newly added projects such as Mohalwk, Prospect MI, Zeya, Copper and Alkali are most likely related to small Facebook testing projects. If you have been participating in one of the above listed projects, feel free to share your experience with us via leave a comment option!

Lionbridge
Lionbridge project code nameDescription Reference for more info
Internet Assessor, Personalized Internet Assessor, RaterGoogle Search Engine Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
Internet Safety EvaluatorGoogle Youtube Evaluation Project Click here

Exam Simulation Tool

Get Personalized Assistance or Advise
Multimedia JudgeBing Search Engine Evaluation project Click here
Personalized Ads Assessor Google Ads Evaluation Project Click here


Exam Simulation Tool

Get Personalized Assistance or Advise
Online Map Quality AnalystMaps Evaluation Project Click here

Get Personalized Assistance or Advise
ISoftstone
iSoftStone project code namesDescription Reference for more info
Online Ad EvaluatorBing Ad Evaluation project Click here
Search Engine EvaluatorBing Search Engine Evaluation project Click here
Advertisements

Step-by-Step Guide How to Pass Appen Yukon Project Exam and Lionbridge Internet Assessor, Internet Safety Evaluator Exams

Looking to pass aPPEN Yukon project/lIONBRIDGE iNTERNET Assessor & Internet Safety Evaluator Exams? THEN YOU HAVE COME TO THE RIGHT PLACE FOR all THE INFORMATION THAT YOU NEED
(Updated Jan 2020) 

Before you can start working on various tasks, you must pass the qualification exam. The qualification exam you will take covers two types of rating tasks which will make up 70% of all tasks in the project: Needs Met and Page Quality.

Once your candidature is approved for certification, you will get an exam email with detailed instructions. Within seven days you have to study 160 pages of the general guideline which consists of rating rules and concepts. The guide explains you how to do the rating of search results and assessment of quality of web pages. You should follow these rules and not use your personal judgment. What they want is for you to represent a typical user in your country. Before you proceed with the exam you need to study the guideline at least twice paying particular attention to practical task examples. Keep in mind that it’s 160 pages long. As I always advise, don’t wait till the last minute.

Note: At the stage of examination, Lionbridge provides candidates with very helpful exam preparation support. You will get access as an exam guest to the Lionbridge learning portal where you can listen to webinars and even try to do a few practical tasks before proceeding with actual exam.

Important update: Lionbridge is very selective in terms of recruiting new raters, up on revising exam results they will check  your activity on their exam preparation portal, with this in mind, I strongly recommend you go through all the webinars  on exam preparation portal before proceeding with the exam itself. 

There are three parts in the exam: you can proceed with the next part only if you pass the previous one. Once you complete any part of exam, within 3-10 minutes you will receive an email with information about results.

Google updates the way the exam is done from time to time. The main feature that has been changed is that you must rate pages from your smartphone (send to device tasks). If you do not have smartphone, they say you cannot apply for this job. This is due to the fact that the number of mobile users is constantly growing and the company wants to assess websites from the perspective of on-the-go mobile users.

Note: There is a recommended schedule that will give you plenty of time so that you’re not rushing to complete the exam at the last minute:

  • Day 1 – Study the Guidelines – Make sure to setup your smartphone so that you are ready to take the final portion of the exam.
  • Day 2 – Start the Theory part of the exam making sure you are taking time to CAREFULLY read each question. Some of them can be tricky and that’s meant to make sure you fully understand the concepts.
  • Day 3 – Go back and study the guidelines once again starting with the Page Quality section.
  • Day 3/4 You can start the second part of the exam which is the Page Quality section if you feel comfortable on day 3. You may want to study this entire day, and you’ve got plenty of time if you decide this to take the exam on day 4.
  • Day 5 – Read the Needs Met section of the guidelines again and make sure to take your time. You’ve still got two more days to finish. And as it has been mentioned before, have your guidelines open in case you need to refer back to them.
  • Days 5/6/7 – Get started on the last part of the exam ‘Needs Met Rating’
Internet Assessor and Yukon project exam part 1 (theory)

In this part of the exam you will be assessed on how well you understand the theoretical part of rating concepts, as well as on your ability to interpret questions and search for information in the general guidelines. It is an open book exam, so you can use the general guidelines to find answers to the test questions. This part has 32 question sections; each section has 4 parts, so in fact you have 132 true/false questions to answer. Read questions carefully, some of questions can be tricky in terms of their wording. You can consult with the guideline book while passing the exam and it’s strongly recommended to use this opportunity.

Note: Do not leave incomplete tasks more than for 1 hour, otherwise they will expire and you will fail the exam automatically. If you need to take a break, complete tasks and press “Submit and Stop rating”.

In most cases an hour would be enough to complete the theoretical part of the exam. The following message will be shown to you once you have submitted the last page of questions:

No tasks are currently available. Please try again later. 

Within a few minutes a result notification email will come to your gmail, and in the case you are successful with part 1, the next part will be available in a few minutes minutes.

Lionbridge Internet Assessor and Appen Yukon project exam part 2 (Page Quality)

In this part of the exam you will be asked to assess 7 web pages for their quality. My advice for this part is to check all quality aspects, and fill in the online form very carefully so you can give a correct overall quality score (from lowest to highest) to a page. You should expect 2-3 low quality web pages, 2 high quality web pages and 2-3 medium quality web pages. However this is not a rule, sometimes candidates get 4-5 pages of high or medium quality pages and only 1 or 2 of low quality pages to rate. Keep this in mind. In order to give correct ratings, examine web pages for ETA (Expertise, Authority and Trustworthiness) criteria very carefully.

Also, keep in mind that in this part of exam you need to give a PQ rating mainly to a web page and not to a website, although some quality aspects of a website have to be taken into consideration when assigning overall PQ rating, in particular in case of YMYL (Your Money Or Your Life) content which Google takes very serious.

Do not forget to write useful comments where required, otherwise the system may show an error.

You don’t need to finish this part at once. If you want, you can take a rest. The tasks are still available when you sign in. Again, if you pass this part, the company will send you an e-mail saying that you can proceed with part 3.

Note: The url/web page may not open on your smartphone, in this case do not mark these as “Did Not Load”. During the exam you should not expect to give a website this flag. If you do so, your answer will be incorrect. To solve this issue try to open link using incognito mode on your browser or/and give rating using other sources of information (website reputation, etc).

Снимок экрана 2016-01-10 в 17.25.24
Page Quality Task
Internet Assessor and Yukon project exam part 3 (needs met)

This is probably the most difficult part of the exam, and many candidates fail the exam at this stage. There are 7 tasks, and each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from “Fails to Met” to “Fully Met”.

Internet Assessor and Yukon project exam Passing score

In order to pass this part you have to get an overall score of at least 80%.

YUKON EXAM PART 3 (NEEDS MET) SEND TO DEVICE – OPENING LINKS ON MOBILE & MULTIPLE Accounts LOGIN ISSUE

Note: while doing Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3, keep in mind that all links have to be opened on your phone because Google analyses web pages from the point of view of mobile users. If you do not have a smartphone, you may use any android emulator for this purpose, in some cases it is even more convenient than using a smartphone for opening links.In order to open the link on your mobile phone, click the  link on the result block on your desktop and then go to www.raterhub.com/url  on your mobile browser  (Google Chrome) to visit the link. You must be logged in your gmail account in order to have access to raterhub platform on your phone. The same rule works if you use android emulator.

Here is an official Leapforce (now Appen) video instruction on how to open links on mobile. Please note that this video was created by Leapforce and uploaded on Youtube by someone.

Note: Some candidates when accessing the link (www.raterhub.com/url ), got informed that Gmail account was unauthorized to visit the link. Most likely reason for this error is the fact of being logged on multiple Gmail accounts on a device. The solution for this problem is to go to android setting -> users & accounts -> add project associated Gmail account, then restart your device. This setting should fix the issue. 

Lionbridge Internet Assessor and Appen Yukon project exam part 3 (Needs Met) Expert Tips and recommendations 

Additionally I would like to give general tips on how to proceed with Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3:

1. Make sure that you fully understand the query and user intent. If you do not fully understand user intent, make additional research on the Internet.

2. When you are evaluating landing pages in the exam, don’t place too much importance on the Page Quality slider. The focus needs to be on Needs Met Ratings. Do not worry about using flags in the exam such as the ‘Upsetting-Offensive Content’ flags.

3. One of the most important things is to absolutely know when to use Fully Meets. Fully Meets ratings are very rare compared to Highly Meets ratings. Avoid being too generous on landing pages. If you have any doubts at all, you should not be rating a page ‘Fully Meets’.

4. Determine what type of query (user intent) is:

o Single entity (business) query with local or non local or both intent/to visit in person/navigate to a website

o Business chain query or query which is looking for business within one category (e.g. car repair service near me)

o Simple know query (where the user requests a specific piece of information, do not assign the Fully Meets rating to any particular result, the correct answer must be present in an SCRB).Examples of simple know queries:paris weather, delta airlines customer service number, current time in Amsterdam)

o Information query seeking a long answer/piece of information (what cases stress)

Types of SCRBs (Special Content Result Blocks): ny weather (simple know query), how old is universe (simple know query), does god exist (long answer seeking), berlin airport (single entity), burger king amsterdam (chain), coffee shops amsterdam (categorical).

o URL/website query

5. Consider the following aspects while giving ratings to result blocks: user location, for local intent queries far away results are useless or slightly relevant. This aspect is very important for the following types of queries: tire shops near me, where to eat near me, Walmart near me, etc. When you get a task with this type of query do use in built map for researching user location/view port and results.

6. Make sure you give correct ratings to special content result blocks for single entity and business category queries. In case of business chain/category query, the result block with a list of businesses is always better than just result for a single business within one category.

7. Consider expertise and trustworthiness of search results for financial, legal, and health related queries; low expertise and trustworthiness automatically reduce needs met rating of the results.

6. Think of dominant, possible common, and minor interpretations for the query and take this aspect into account while giving ratings to a result block. Minor interpretation results should get a low usefulness (Needs Met) rating.

8. Take into account how well the web page/website is optimized for mobile users.

9. Once you have analyzed and considered all listed above aspects, you should give a rating to result blocks one by one.

Note: In some cases there might be some results (links) which are not accessible from outside U.S. (usually websites of well known retailers such as Target, Walmart, etc. They ban access from foreign IPs to prevent hacker attacks). Appen, Raterlabs or Lionbridge do not take into account this factor and therefore rating such results as fails to met may lead to overall fail.

To make sure you do not miss anything before proceeding with part 3, follow pre-exam instructions to pass this part. There will be a link to a video. Watch it and follow the instructions.

Снимок
Examination Needs Met Task

Note: In part 3 of the exam (Needs Met), most of results blocks have Needs Met and E-A-T (previously PQ) sliders. Main focus should be on providing correct Needs Met rating, which in some cases might be affected by E-T-A (PQ) rating when it comes to YMYL queries and results. Leapforce/Google re-named PQ to E-T-A in part 3 to make it clear that the most important aspects in PQ are expertise, trustworthiness and authority. In general it is similar to PQ concept. While giving E-T-A rating you should evaluate MC under these 3 criteria. In most cases you may do it fast at a first glance approach, but be sure to provide good evaluation for E-T-A when you have YMYL query.

Note: If the user location map does not show user location correctly, and you have a blank page, hit the black arrow button. The task will be closed, but you will see “Incomplete tasks” on the evaluation platform. Click continue and come back to your task with the updated user location field. Remember that it is very important to take into account the user location while giving ratings to block results. Also keep in mind that there is no need to finish the task in one sitting. You can take a break if you need to.

Once you complete 7 tasks you will get this message:

No tasks are currently available. Please try again later.

Yukon (Appen) exam results and Internet Assessor (Lionbridge) exam results

You will get overall exam results on the exam due date from Appen and exam due date plus one day  if your vendor is Lionbridge. Appen will ask you to sign a few electronic forms and in 2-4 days you will be given access to live tasks. At Lionbridge you will have to complete educational course with no pass/no fail quiz before you are granted full access to the tasks.

US YUKON Raterlabs and Lionbridge raters

Once you pass the exam you will then go through the employment process which might take up to 3 weeks. You will be asked to complete an online version of the IRS Form 8850 (Pre-Screening Notice and Certification Request for the Work Opportunity Credit), and a tax credit questionnaire from a vendor payroll provider. After you have completed these two forms, you will be presented with a formal employment offer letter. After acknowledging your offer, you will still need to complete the required activities in a timely manner. Try not to postpone the completion of these important steps: Providing bank details for ACH payments on Paycom website. Providing your I9 Employee Section 1 and I9 Employment Eligibility Documents (section 2) must be completed by specified deadline. It is very important that candidates complete all required steps as soon as possible to avoid termination of your employment.

 
NEW!!! Welcome to Search Engine Evaluation Exam (Yukon, Internet Assessor) Simulation Tool!
What is The Exam Simulation Tool? 

I have created a special tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

How can this tool can help you pass the exam?

The way the tool is designed will show you exactly your problem area and what you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to mention that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from the most typical exam tasks, and is also designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam. 

How do I get my hands on the exam simulation tool?

Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains a simplified Needs Met and one Page Quality task so you can see how it works. Once again, the demo is 100% free and remain completely anonymous.

Exam Simulation Pro

The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met)  of all the major query types. These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

Explore the tool here: https://examsimulation.passleapforceexam.com/

Yukon and Internet Assessor Expert Support

If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.

Payoneer Payment method for Appen 

Leapforce/Appen/Raterlabs candidate? If you get a job at one of these places, you’re going to need Payoneer account because that is the only payment option available for these companies. Simply sign up for Payoneer under my referral link and you’ll get the simulation tool subscription fee back as Payoneer subscription 25$ bonus once your account is loaded with 1000$! Normally it takes 1-2 months to make 1000$ for EU/USA/Canada raters and 2-4 months for raters from another countries.

The following posts are recommended for reading:

Common Issues of New Raters: Appen Yukon and Lionbridge Internet Safety Evaluator Google Rating Projects

https://passleapforceexam.com/2017/06/25/payoneer-as-a-payment-method-for-appen-raterlabs-utest-and-lionbridge/

You Failed Appen Yukon Project Lionbridge Internet Assessor Exam Now What?

https://passleapforceexam.com/2017/11/11/what-should-you-do-if-you-fail-leapforce-or-lionbridge-the-exam-twice-and-how-to-re-apply/

 

Exam Simulation Tool For Appen Yukon Project Exam/Lionbridge Internet Assessor & Internet Safety Evaluator Exams

What is The Exam Simulation Tool? 

I have created an exam simulation tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

How can this tool can help you pass the exam?

The way the tool is designed will show you exactly your problem area and what you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to mention that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from the most typical exam tasks, and is also designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam. 

How do I get my hands on the exam simulation tool?

Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains a simplified Needs Met and one Page Quality task so you can see how it works. Once again, the demo is 100% free and remain completely anonymous.

Exam Simulation Pro

The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met)  of all the major query types. These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.

Explore the tool here: https://examsimulation.passleapforceexam.com/

Step-by-Step Guide How to Pass Appen Yukon Project Exam and Lionbridge Internet Assessor, Internet Safety Evaluator Exams

Apply for Search Engine Rater and Social Media Evaluation Jobs at Appen/Lionbridge

Want to Apply For Search Engine Rater and Social Media Evaluations jobs at Lionbridge and Appen?

Sometimes finding these types of jobs online takes a bit of research. Most of the big companies that offer these types of jobs don’t usually advertise in common places or online job boards. You usually have to go directly to the company and apply on their website. I have seen that Appen Global advertises their jobs on Flexjobs.com, but that website charges a monthly fee. I never quite understood how a company can charge you to find a job, but they are the exception.

Finding these jobs and applying for them are different for each company. Going to each individual website is the best way, as long as you know where to go, and how to apply. With that being said, this article, and the videos we have provided, will show you how to find the work from home jobs.

One of the other important things to consider when looking for these types of jobs is your location. Some of the contracts that are offered through companies like Appen and Lionbridge require you to live in the demographic area that the project requires itself. The reason for this is because the data from different regions will be different depending on location and the language.

Before you begin to apply for these jobs, or really any job for that matter, it’s probably a good idea to go over your resume and make sure it’s up to date, and that it includes only the important information that is necessary. It might also be to your benefit to have your resume checked by a professional. The resume experts at Resumewriters.com is an awesome service that I have used in the past, and having your resume edited or corrected by a professional is a sure fire way to get noticed and get you the job you want. Many of our readers have had very good results with them.

To visit their website Click Here: Resumewriters.com

Another important note is that if you have worked previously on other social media projects with different companies, don’t mention this on your CV/resume. The two most important factors about this are that you do not need previous experience for these jobs. And second, if you have worked on these types of projects before you will most certainly be looked over and turned down for similar projects. The reason for this is uncertain, but we have seen other people turned down because of this.

You can work on two different projects for more than one company. However, you need to use a different IP address for one or the other. So if you work for Appen and want to work on another project with Lionbridge, you will likely need a different IP address. Appen seems to be the only company that records an IP address and this is shown in the backend or the portal of the work dashboard. This is why it’s a good idea to use a separate IP address for another project. It might also be a good idea to use a different email between companies when applying for and working on these jobs.

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Lionbridge and Appen When Combining Several Projects?

Some advice when working on different projects so that you don’t overwhelm yourself is to get some experience working on one project with a company before trying to get another contract with a separate company. Most of these contracts are part time. So its beneficial to have more than one project at a time if you can handle it.

A really important thing to note here is that you can only have one contract per IP address when it comes to these companies. So if more than one person is working for any of these companies, it is important to have a separate IP address. You can usually order another IP address from your internet service provider for about $4-5 per month extra. Check with your ISP to find out if you require more than 1 IP address per household. You can read more about this in FAQ

Application process at Lionbridge – Ads Assessor, Internet Safety Evaluator, Ads Assessor 

Lionbridge search rater project is called the Google rating project (ads assessor, internet safety evaluator, ads assessor) which is the same as Yukon at Appen (Raterlabs). They also have the positions of Personalized Social Media Assessor, Personalized Internet Assessor, and Personalized Web Assessor.

Application process at Appen – Search Engine Evalutor, Rater, Internet Analyst 

At Appen/Raterlabs you would apply for search engine evaluator position first, and then based on project availability, you would get invites for Yukon, Nile, (Facebook social media evaluator) Truckee or Shasta projects. You can work on more than one project at a time.

Appen Global

Many of my readers will be interested in how to get invited to other projects at Appen Global. Appen Global offers a lot of different types of projects that you are able to work on. Unfortunately, is not likely that you would be contracted to start on projects such as the Thunderbird music project from the beginning. You would be more likely to start on a larger project first such FalconInstaAds or the Arrow project. Once you have started working at Appen, and you are on a Falcon project, like any of the social media projects that they offer, you will likely get an invitation to start on other projects as well. The more you work, and the better your performance is, the more offers you will get from Appen.

Those projects include:

Arrow (ads assessor);

Yukon (search engine evaluator);

Nile/Truckee Facebook/Instagram ads assessor projects (social media evaluator).

application process video instructions -Lionbridge, Appen, Appen Global

We thought it would be a good idea to create some quick walk-through videos to show you how to apply for these jobs and the different companies. Some of these:

Lionbridge, go to http://www.thesmartcrowd.com/workers/job-opportunities/country/russia/#jobs-list

Appen, go  to https://connect.appen.com/qrp/public/jobs/list

Appen Global go here: https://talent-appen.icims.com/jobs/intro

How long does take to hear back from Lionbridge and Appen?

It depends on many factors, sometimes immediately, and sometimes it can take 3-4 months.

It might be that at the moment of your application submission, quota of raters for your country was reached and therefore Appen or Lionbridge does not hire new rater at the moment. As soon as they fire unsuccessful raters, they continue hiring new people to maintain the required number of raters in your country.

Appen Connect Tahoe Event Realness 3.0 Facebook Project Review

Tahoe Event Realness 3.0 Facebook Project Review

There is a Facebook project called Tahoe that uses a team that checks events that come up in Facebook results. The main mission of this program is to bring people together through real-world activities by showing real events. This is done by working with tools and resources that allow users to discover interesting real events that they might be interested in attending. 

TAHOE PROJECT QUALIFICATION PROCESS

To qualify for this project, you must take a quiz that has 60 questions. 53 of the questions are practical, and 7 are theoretical. The exam looks like it might be difficult to pass, and you must have a score of 90% in order to pass it. Be sure that you follow the guidelines and the instructions in order to make sure you are giving the correct ratings. If you fail the exam, it can be retaken. However, if you fail the exam the second time, chances are they will not give you another opportunity. Get Expert Personalized Assistance with Any Project Qualification

Tahoe project rating process and logic

Improving the discovery of events is done by making a distinction between real events that are happening in the real world at a specific time, and Non-Real Events which are obviously the opposite and not happening in the real world. The job of the individual working on this team is to label events as either Real or Non-Real events by reading the event and basic information. If needed, the Facebook page is opened, and the page is rated either a real or non-real event. This data will be used to help train an algorithm to be more accurate when it comes to events that are shown on Facebook. Non-real events are not removed from Facebook, so this work is important to help people find real events that are happening in their area. 

So what is a real event? 

When looking at an event, the main question is to ask if the event is an activity in the real world, and if it is, is there sufficient information for the user to attend the event like date and time, and directions or a phone number to reach someone. Chances are if it has all this information on an event, then it’s a real event and can be tagged as such. 

The location for the real event should have the city, state, or a venue where the event is listed and usually found in the description box. 

Time is important of course and there are some rules as far as the time is concerned.

Events that last up to 3 days from start to end, such as weekend festivals, are acceptable. Events that have a duration of more than 4 days are only considered an event if there is enough activity within each day to justify having one event created for the entire duration such as a retreat event. 

When looking at these event ads, it is important to watch for phrases such as “time to be determined at a later date” which tells you that the event creator doesn’t have a set date, therefore the event should be tagged as Non-real. 

Reoccurring events and long-term exhibitions need to have event pages for each day those events are going on. Some examples of this are a movie theatre that has midnight showings every Friday. This needs an event page for every Friday. An art exhibit that is being displayed at a studio for a month needs an event page for every day it is being displayed. This also goes for several performances on the same day. Each performance is considered a separate event and should have a new page. 

The activity of the event doesn’t need to be explicitly stated in the description. Keep in mind that the main intent of the event post does not need to be about the activity itself, such as a page selling tickets to an event. This is still considered a real event because its selling tickets to a real event. 

The key here is to watch out for events that closely resemble an advertisement rather than a real activity for the user. This includes things like promoting sales, tour packages, causes, etc. These are not considered real events. 

What is considered a Non-Real event? 

Depending on the location, many events will be encountered that initially seem like they are real events, but in reality, they do not satisfy the event’s realness requirements. When evaluating these types of posts, remember to always use the main frame of thinking as with every post: Is this an event I can attend, and if it is, is there enough information for me to attend the event?

Below are some examples of some common Non-events that will be encountered. 

Promotional events make up the majority of Non-Real cases that the rater will encounter during this experiment. Most of the time these events will more closely resemble advertisements more than actual activities or events such as a company using the event creation tool to advertise a discount or a sale. Travel agencies creating an event for the purpose of selling some tour package. However, if it is an experience type of event or an educational event such as an ecological walk, even if its called a tour, if its still a real event that holds interest for locals then it is considered real. One key thing to note is that these posts are usually made by non-profit organizations, government agencies, or educators and are a good indicator that they are real. 

Watch out for insufficient times such as an event that has a start and end time that doesn’t align with the actual activity for the event, although these events are very rare. 

Time details to be determined is another factor in determining if something is Non-real. If there is no strict date, then its nothing you can plan for, and therefore not a real event. 

If you see a post that has a schedule of events, it needs to be tagged as non-real as every event should have a single post of its own. 

Call to action events are created for the purpose of receiving donations, votes, or any other type of contribution from a user without the need for in-person activity. With that said, if the event is a fundraiser that occurs at a real-world location, this would be acceptable as Real. 

If the event post is gathering support to see a certain performer or artist in a specific city or country, then the post should be considered non-real because there is no confirmed real-world event. 

Online events are considered Non-Real because they do not happen in the real world and that includes live streams, radio, and TV shows. Watch out for memes, just for fun, and spam that pretends to show real events happening in the world, but upon inspection, you realize that it’s evident that it can’t be real such as bands performing in unrealistic situations, or events that are centered around popular internet memes. 

Another thing that event creators like to do is use the event creation tool to announce a large-scale celebration or holiday. If it’s just some announcement that doesn’t lead to a specific activity, then it gets labeled Non-real. 

Below are some more examples:

Jobs should be rejected if you do not understand the language that the post is in. Any post with solicitation of Child Exploitation or Child Nudity should be escalated immediately with the JOB ID to the manager. If you are uncomfortable reviewing the content, then the job can be skipped after it is escalated. Canceled events are still marked real as long as they fulfill the Real event requirements. 

The language of the experiment should be present in the event title or description. If these are missing then you can immediately reject the job. Make sure that time details listed for an event make sense for the actual activity, and also make sure that the city and state, or venue name is listed somewhere in the event info. 

If you work on this project or know someone that does, we would like to hear from you in the comments about your experience on this project. If you have any more questions, then shoot me an email or just leave a comment and I will answer as quickly as possible. 

Sykes Search Quality Evaluator – New Vendor For Google Search Engine Evaluation Project (US only)

Sykes Search Quality Evaluator

There’s a new job position that is available through Sykes for a Search Quality Evaluator working from home part-time.

It isn’t known exactly what vendor this job could be for since Sykes is an outsourcing provider, but it could be Sykes, Appen or even Lionbridge. Another thing to note is that this job was posted on December 21, 2019 and is still listed. So, if you are looking for a job as a Search Quality Evaluator (known as Google Search Engine Evaluation project: Yukon, Internet Assessor, Rater) this could be your big work at home opportunity.

Judging from the job description about searching for online topics, and how frustrating it can be to get bad results, this is most certainly the same type of job as Search Engine Evaluator and you would be responsible for performing simple web searches on specific topics and then rating the results that are returned within the time frame they give you. A Windows PC is required for this work at home job, and no MACS are being accepted in this project. You must also have an activated smartphone with Wi-Fi capability. Applicants need to have strong computer skills or they will not be considered.  

The time you want to work on this project is flexible and you can work on your own time as you wish. There is a schedule provided after training, although it is uncertain what this training might be since there is no previous training for these types of jobs. Only the guide is supplied to you and you must learn on your own. After this training period you must work a minimum of 20 hours per week. 

One thing to keep in mind for this project is that you would be reviewing offensive and mature content. If this type of stuff offends you, then you may want to skip this one. 

Also, if you live in these states, Sykes is unable to hire you: Alaska, Arizona, California, Colorado, District of Columbia, Hawaii, Illinois, Iowa, Maine, Massachusetts, Montana, New Hampshire, New Mexico, New York, North Dakota, Ohio, Oregon, Rhode Island, South Dakota, Vermont, Washington and West Virginia.

You may apply for Search Quality Evaluator role here:https://jobs.sykes.com/en-US/search?keywords=Search%20Quality%20Evaluator&location=

There is no mention of what the pay is at this time. If you happen to be working on this program, we’d like to hear from you about the training procedure and what the rate of pay might be. As always, if you have questions about this project, send me an email or leave a comment and I will reply as quickly as possible. 

Appen and Lionbridge projects – Hourly Rates Across Different Markets

Hourly rates across different countries: Lionbridge, Appen
Country

Vendor/Project
USAEUTurkeyBrazilJapanRussia
Appen
Echo, Index, Sharp, Cable 12$-13,5$ 14$ 7$
Arrow (Google) 10$ 14$ was 8$/now 5$ was 8$/now 5$ 15$-17$ was 8$/now 5$
Truckee (Insta) was 13,5$/now 10$ 14$ was 9,45$/now 7$ was 9,45$/now 7$ was 8$/now 5$
Nile (Facebook) was 13,5$/now 10$ 14$ was 9,45$/now 7$ was 9,45$/now 7$ 15$
Shasta per task per task per task per task per task per task
Yukon (Google) was 12,5$ (net)/now 10$ (net) 14$ was 10,5$/now 5$ was 10,5$/now 7$ 14$ was 8$/now 5$
Lionbridge
Internet Assessor, Rater, Internet Safety Evaluator (Google) 14,5$ (gross)
14$ 10$ 12$ 7,3$
Ads Assessor (Google) 13,5$ 14$ 10$ 12$ 7,3$
Maps Analyst (Apple) per task per task
Why Appen decreased hourly rates almost in all markets after acquiring Leapforce?

Feel free to share your information regarding payment rates in your specific country, based on your input I will update information on this post.

Internet Safety Evaluator Project For Lionbridge Review

Internet Safety Evaluator project review

This project is about a year old and was created and started when Google decided to exclude some YouTube video evaluation tasks from its main search engine evaluation projects: You are probably familiar with these projects: Yukon (Appen/Raterlabs) and Internet Assessor/Rater at Lionbridge. The Internet Safety Evaluator project picks up where search engine evaluation leaves off and the job has you looking at video search results to improve the search result quality for content. As an Internet Safety Evaluator, you are also required to provide feedback and analysis on video content found in the search results and provide ratings on their relevance with the search terms that were used. 

So where does the Internet Safety Evaluator part come in? Well, that’s the most important part and aspect of this role. While reviewing content, you will flag potential upsetting or offensive material to make the internet safer and more secure for all users. You will be able to express your opinions in comments about the video concerning the content and the quality of content, and at the same time, you will be protecting users from viewing unsuitable material. 

So what do you need to have to qualify for the position of Internet Safety Evaluator? Appen and Lionbridge are looking for dynamic and creative people who have a strong interest in improving online search results. One of the main things that you should feel comfortable about is working on sensitive or adult content. Candidates must use a smartphone for this position, and either an Android or iPhone device is suitable. The Internet Safety Evaluator should also have some familiarity with social media and Google products. Be prepared to be flexible concerning working hours and you should have the ability to understand and follow the established guidelines. 

The Internet Safety Evaluator position only allows you to have 20-hours max work time per week. 

The qualification process is identical to the Appen/Yukon Project or the Internet Assessor for Lionbridge. Read more about how to pass that exam here:

Also, don’t forget to check out the really cool tool that we’ve created for helping to pass these tough exams. See Our Exam Simulation Tool Here: https://examsimulation.passleapforceexam.com

Rating Process And Quality Metrics

As an Internet Safety Rater working on the project, you will have access to a variety of different task types. This is important because it helps you become more familiar with the core concepts of rating and it will help you build experience and knowledge concerning the program. During the beginning, the number of tasks you will receive will be low until you gain enough experience in the program. Once you feel that you have a good understanding of the program and become familiar with all the task types, you will start to see a switch in the task types that you receive. But keep in mind, this will only happen if the managers feel you have a strong grasp on the concept of what you are doing. From that point forward, you will only be required to focus on evaluating more video type tasks from an Internet Safety Rater perspective. You will also see a higher number of tasks given to you. 

As an Internet Safety Rater, you cannot work more than 1 hour per day and this limitation can last several months. The purpose of this limitation is to evaluate the overall quality of ratings that you are providing. During this time you will be given standard Yukon/Internet Assessor tasks (Needs Met and Side by Side and other types of tasks), additionally, you will be evaluated on your quality by means of standard Needs Met and Side by Side quality review tasks. 

If you want to know more about this process, read here:

The quality requirements are high and you need to achieve and keep a quality level higher than 65% or above. This is the very minimum and work is monitored on a regular basis. Feedback will be shared with you so that you can learn from your mistakes. If an Internet Safety Evaluator constantly performs below that minimum quality level or performs significantly lower than the minimum level, that person will be removed from the program. This is why it is very important to use the months long limitations and make sure to learn as much as possible. 

The things closely monitored to evaluation an Internet Safety Evaluator performance is the Needs Met, Page Quality (E.A.T.) and SXS rating. However, other things such as rating accuracy and the quality of the comments that you leave are another thing that is closely looked at. 

Once the Internet Safety Evaluator has shown a stable level of quality, the probation period ends and you will get a notification that the Internet Safety Evaluation transition period ended and all restrictions are then lifted and you see a higher volume of tasks. This is when you will start receiving ONLY video evaluation tasks and you’ll get a higher volume of them. There will be no more review tasks after the completion of the transition period. 

Internet Safety Evaluator probation and transition period

The Internet Safety Evaluator probation and transition period length all depends on the quality performance of the candidate who should demonstrate positive quality review scores. This period usually takes about 3-4 months but could possibly last up to 8 months depending on the performance of the individual. 

In terms of the complexity of this project and how easy it is to participate, the Internet Safety Evaluator project seems to be easier in particular after the transition period has ended. Working on video tasks is a lot easier than doing the Needs Met and Side By Side tasks. However, the big drawback to the project is online being able to work 1-2 hours per day during the transition period. 

You can work on other projects at the same time to make up for these small hours. You can provide similar rating services to other providers. However, due to project restrictions and customer requirements, you cannot work on Appen Yukon and the Lionbridge Internet Assessor at the same time due to an IP address conflict. 

If you want more information on other unrelated projects, you can find that information here:

If you have any questions about the Internet Safety Evaluator position or any other questions, let me know in the comments or simply send me an email and I will reply. If you are working as an Internet Safety Evaluator we’d like to hear about your experience. 

Preferred Raters Program For Lionbridge/Raterlabs

Preferred Raters Program

If you’ve landed here, you might have heard about the Preferred Raters Program…and It looks like there is finally some great benefits coming out for Lionbridge/Raterlabs for raters who achieve high scores and have consistently high-quality ratings and comments for all task types. This is for raters who work on the Yukon/Raterlabs project. Through this program, agents are identified and rewarded based on high-quality review scores and high-quality work they do on the Yukon project. 

It sounds really great, but it is highly selective and prestigious, and only available to a limited amount of raters. The requirements to get into this program are very high, as well as inclusion and retention…meaning it will be tough to get in and also stay there. Participants are held to strict standards of quality and professionalism and the quality team reviews candidates quarterly to determine if they qualify for the program for admission or if you’re already in the program if you stay or someone comes along and takes your place. 


So, what about the rewards and are they really worth it? 

If you do get into the Preferred Raters Program, you’ll get a boost in pay rate. You’ll go from $10 to $15.00 dollars for Raterlabs and from $14 to $18 at Lionbridge. Not a bad little pay raise at all even if it is temporary. 

As another added benefit, your username will be in green in chatrooms showing that you’re a highly skilled and very experienced rater. 

How Do I Qualify? 

As we mentioned earlier, raters are regularly reviewed to see who meets the requirements of the Preferred Agent Program. 

Note: Sending them emails might be the quickest way to NOT get into the program. It is not recommended to email them about your status in the program. 

Only if you have met the program’s requirements will you hear from the project’s Quality Team. If you don’t hear from them, then it may mean that you need to step up your quality or performance, or there just may be too many candidates and not enough room to fill them all. We do believe it might be the opposite since requirements are so high for the program. 

The most important thing to focus on is quality review scores for all task types. Having high scores in this area will get you looked at for eligibility for the Preferred Raters Program. But it’s not all as there are several other additional criteria used to determine an agent’s final acceptance into the program and we’ll go over all the things they use to select someone. 

Primary Selection Criteria

The very first thing to point out is only agents who have worked on the Yukon project for at least one year are eligible. 

And agent needs to complete enough of the following tasks to receive the minimum Quality Review feedback:

Needs Met: 6 sets of feedback

Side by Side: 3 sets of feedback

Page Quality: 2 sets of feedback

The quality reviews all have to be extremely positive for all the task types, not just one, and high adherence and demonstrated comprehension of the principles detailed in all the guidelines and project instructions. 

Comments are also a part of this and Preferred Raters will show excellent commenting skills for all tasks. 

An agent will have a consistent history of good interactions with the community that should always be positive. This includes all areas of communication with admins in chat, social posts, and emails. 

While we’re on the subject, we shouldn’t have to tell you that candidates that have a history of unprofessional, nasty, or obnoxious behavior in chat or other social channels won’t be considered for this program. 

Agents in the program will always meet the minimum hours’ requirement and complete all the tasks in a responsible manner and not releasing them in an excessive or malicious way because maybe the agent doesn’t want to rate the produced results. 

Did we mention that it’s hard to get into the Preferred Raters Program? We should mention it again and also state that it’s not an easy thing to achieve or maintain. But if you give hard work, perseverance, and show a very thorough understanding of the concepts from the General Guidelines, then chances are good that you’ll make it as a candidate. 

I’ve made it as a candidate, now how do I keep my status?

We mentioned that the Preferred Raters Program is considered prestigious, highly selective, and it is limited to a small number of raters. That just means it’s going to be a lot more difficult for you to get in and stay there. But after you are in, a rater must maintain the high-quality standards across all task types and have high-quality reviews after acceptance. 

If a rater fails to maintain high standards of quality, then they are put into a probational period and given the opportunity to improve. That’s good news since if you are in this program, you’ve worked hard to get where you are. However, if an agent continues to fail meeting the program criteria, they will lose their preferred status and the rewards associated. Also, once you are in the program you can be removed at any time at the Quality Teams’ discretion even without a probational period. 

So what if you are put on probation?

If a rater fails to meet requirements they are placed on probation. This probation is actually two more reviews and if your reviews are bad, then you’re out. One other thing:  If you are placed on probation and receive a substandard or unacceptable review score in ANY review type during your probationary period, your Preferred Agent status will be immediately removed.

So what triggers probation? 

When an agent gets three bad scores within the most recent 6-month period below the Preferred Agent standard, this will trigger a probationary period. Also, if the majority of your recent scores are all bad and not meeting the high standard, you are placed on probation. The bottom line is that your scores should be primarily Exceeds Expectations or higher. Meets expectations do not cut the cake. While they are respectable, they are not high enough scores to be in the program. Multiple scores of Meets Expectations can result in the loss of your Preferred Agent status. 

Another thing that should be mentioned is that two Substandard and Unacceptable review scores within a recent year’s time period will trigger a probation period. Any additional scores of the same type will result in the immediate removal from the program. Preferred Raters cannot have such low scores. 

The most important requirement of a Preferred Agent is that they consistently provide exceptionally high-quality work and if your scores fluctuate, it might suggest to the Quality Team that you are unable to maintain the high standards of the program, and you will be placed on probation. While it is a disheartening thing to be placed on probation, this does mean that you are being given the chance to improve your quality of rating. Remember that Meets Expectations is not good enough. 

Some other things that will provoke the Quality Team are poor commenting habits, excessive releasing of tasks, and complaints about unprofessional behavior in chat or other social channels could lead to probation or it could just see you leaving the program altogether. Remember that we said they have total discretion and can release you from the program at any time for any reason. 

If you are put on probation then it’s lucky you and you’ll have to use the opportunity to correct the problems and retain your Preferred Agent status. 

As we touched on a little earlier, Quality Teams review agents on a quarterly basis and if there is room to promote new agents, they will do so. If the program is full and all those raters are keeping their high standards up, then there won’t be any new raters accepted into the program. When a rater is removed from a program not meeting high-quality standards, then another rater who is consistently doing this is taken into the Preferred Agent program. 

You might be wondering if one low score will put you on probation. The answer is no and the Quality Agents recognize that sometimes bad things happen. However, if you continue to get low scores, then you will be put on probation until you fix them and they only give you two reviews after this and you must score high on BOTH reviews, not just one. 

Just so we are sure we covered this, while you are in the program, three low scores will get you noticed by the system which will trigger a probation period. 

A rater must meet high-quality standards in all areas and all tasks, not just one area. Because the program is considered prestigious, it is limited to a certain amount of raters. That said, if you are in the program and have some problem in one area, it will show and you will likely end up on probation. 

You can earn the status back if you lose it by demonstrating that the quality of your work meets the program’s qualification standards on a consistent basis. Keep in mind that just because a rater has been in the program before, preference will not be given over them when determining new rater eligibility. There are only two times you can be put on probation meaning that the third time and you are out of the program. 

We hope that we were able to answer your questions about the Preferred Agent program. Are you in this program? Tell us your experience in the comments and if you need help you can always reach us by email. 

Appen Connect Wouri Project Review And Information

Wouri project review

There’s another project to work on over at Appen called the Appen Connect Wouri Project. We’ll go over the project a little and briefly touch on the main points, what type of job it is and what requirements there are. We will also touch on the grading process and give a few examples. 

The goal of the Appen Connect Wouri Project is to understand the user’s intention as represented by the query, and then determine how good the result is and if it satisfies the user’s needs. Users are using Safari and Spotlight to search for different information needs. Some examples include wanting to find out knowledge or information on something or somebody, wanting to know if there is a certain store nearby, the latest news related to a topic, or since they are MAC users, getting an application on iTunes. This project is a bit like the average search engine evaluator, but it does have a different grading system. However, it uses some of the same ideas from those programs on search engine results. 

A desktop or laptop computer is recommended to work on the Appen Connect Wouri Project over a mobile phone and all graders should be checking search results with Safari. Safari can be used on Windows operating systems, but it should be mentioned that there is no support from Apple when using Safari on MS Windows. You might be better off with a MAC computer, but it isn’t required. 

To participate in the Appen Connect Wouri Project you have to pass an initial assessment and achieve a minimum score of 80%. If you don’t pass the first time, you’ll get another attempt. After you pass the initial assessment you will be given a final assessment in which you must also pass with a minimum score of 80%. 

Wouri Project rating Logic

When looking at a user query, there are three steps that you will take in the grading process. 

  • The first thing is understanding the query. You should make sure it’s not too vague, inappropriate, or in a foreign language. 
  • The second thing is understanding the result. Determine if it is GEO or Time Sensitive. There is also the option to choose both or choose neither GEO or Time Sensitive. 
  • Once you understand the query and the result, you will then decide on a relevance rating for the result based on the query intent. 

It is important to note that Appen expects local graders working on the Appen Connect Wouri Project to judge the search results of the same locale. What that means is you must in the same location as the project requires. In this case, you may need to configure your computer and your search engine to be in the language of the project that you are working on. 

A web relevant search is one that satisfies the user’s information need. Of course, a search result can be more or less relevant based on how well or how completely it satisfies the users’ needs. It is important, and we cannot stress this enough, that you have the guidelines open and you have thoroughly read them. 

In step number one, you need to determine what it is that the user is searching for and decide if it’s valid. You should read the query, and review the third party results and think about the possible user intent that it represents. This is called ‘Understanding the Query.

For example, the query ‘Samsung’ could have many different user intents such as shopping for Samsung products, getting Samsung technical support, or the user may be looking for the Samsung homepage. However, if the user simply types in Samsung, then the query can be too vague. 

If you have no idea what the user might be looking for, then the next step would be to go to a popular search engine and study the term. Some common terms could have some special meaning. However, if it doesn’t have some special meaning, or it has too many special meanings with each one having a small portion of intent, then you would select Query is Too Vague. 

You might also run into a situation where a query is inappropriate and it encourages illegal behavior, includes profane words, using derogatory language, racial slurs or hate speech. It also includes graphic violence or the query might like to retrieve grisly content. You are not expected to do anything about these types of queries except to mark it as inappropriate.  

When in doubt about any type of query, you should check results from other search engines. It’s absolutely crucial to understand what the user is searching for, and you may need to check results from other search engines. The good thing about this project is that top results from other search engines are provided for you. There is a small link that says ‘Web Search Link’ that will automatically do this for you and show you the top results in order to better help you understand the query. 

It is VITAL to point out that opening the webpages is a MUST when you are working on this project. If the results match the first result of the other search engine, then the result is likely an excellent match. Make sure to check the results that are returned by the Web Search Link. Each result has a clickable URL, a small description, and a photo where applicable. 

We touched a little on GEO and Time Sensitive in the beginning, and since you might not understand what that is, we will go over it to help you understand it better.

If a query is about a store or some location that exists in other locations aside from the searcher, let’s say a Starbucks, or a query like ‘movie theatres near me’ then you know its geo sensitive meaning that it matters where and how close the result to the searcher is. If the searcher is from California and looking for a Starbucks and the first result is the homepage or another location, then it doesn’t satisfy the user’s needs. 

Anything that deals with a query that involves some kind of time frame, such as ‘Buy Superbowl Tickets’, age of Harrison Ford, or checking on the weather are results that should be marked as ‘Time Sensitive’. 

It is important to understand that a query can be both Geo sensitive and Time sensitive, and a good example is a query such as ‘tickets to the local basketball game’, local movie showtimes, etc. 

Once you understand the query and you judge the results, then you will assign a relevant grade for each result. There are 9-grade levels:

● Excellent

● Good

● Fair

● Bad

● Unavailable

● Query In Foreign Language

● Result Page In Foreign Language

● Vague

● Inappropriate

There are many examples in the guidelines that go over difficult queries in order to help you evaluate what results would satisfy the informational need for the searcher. It is a must that you go over the guidelines and read them and understand them so you know how to grade something. 

The guidelines go over when you should choose an excellent result, and the biggest point to keep in mind here is to believe that the result will be everything that the user will need, and they will not have to continue searching. If you believe that the result isn’t good enough, and you would also do an additional search, then you might choose ‘Good’ or ‘Fair’ as the grade. 

A ‘Good Result is one that the user wants to see above the fold on the device screen. It is typically the first 4-5 results. However, it is not the best possible result, which would be excellent, but it still provides good information. The guidelines have many examples of what should be chosen as a ‘Good Result’. 

When choosing a ‘Fair Result’ for the query, the result would only be somewhat relevant, but not be of interest to all users. An example of this would be a query for a celebrity name, but the result is an obscure person that has the same name. 

Choosing  ‘Bad Result’ would pertain to a result that does not satisfy user intent and is something that should not be shown to users at all. An example of this would be something that we previously mentioned before if a user was looking for a Starbucks Coffee and lives in San Francisco, but the map result is for a store in San Diego. There are other examples in the guidelines. 

It should be straight-forward of when you should choose ‘Foreign Language’. If the given query is in a different language from the specified one, or the target language that you are assigned to work, then you would mark this as “Query in Foreign Language”. If you are not sure if the query is in a foreign language, then just search that query to understand the word or words. 

Sometimes in the given search results for a query, the URL will be inaccessible, or website link will not be available or a dead link. If this happens then you would grade it ‘Unavailable’ This also includes landing pages that take longer than 10 seconds to load, or you may see that the domain is up for sale by being sent to a redirection page. This grade also pertains to when video content is unavailable or blocked. 

There are some other specific result types that the guidelines go over including ‘Ambiguous Queries’ or when the interpretation is much more popular than others. For example, the query is ‘Golden Retriever’ but the result is a song titled the same. This should be graded as fair because the dominant interpretation would be for the dog breed itself and not a song title. There are some instanced when multiple interpretations can be as equally popular though. In order to assign something to the excellent category, it’s a good idea to take into consideration what geo-location the seacher is looking from. Example:

Query is “UM football”, and the result is the official website for the University of Miami football team. If user’s location is in San Francisco, grade the result as Good, since UM could refer to either U of Miami or U of

Michigan and we don’t know which is best. But if the user location is in Miami, grade as Excellent.

As always, if you have any questions about the program, please send an email or reply in the comments. If you are working on this program, then we would love to hear about your experience in the comments. 

Timesheet Disapproval On Lionbridge Ads Rating Project

Ads Rating Project massive timesheet disapproval: on purpose or unprofessional approach? 

It appears that lately there are a lot of problems with the Lionbridge payment systems that has resulted in a timesheet disapproval and it’s not actually a new thing. According to comments of my readers and discussions on https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/  (2018), https://www.reddit.com/r/WorkOnline/comments/cvol9y/lionbridge_scammers_2/ (2019, August),https://www.reddit.com/r/WorkOnline/comments/d95mwc/lionbridge_scammingasking_to_remove_hours_again/ 

The problem with the timesheet disaproval started to happen in May-June 2018. It just seems that now it’s becoming worse and there are more and more raters complaining that the time sheets that are being submitted are being returned. Lionbridge is asking people to carve a lot of hours from their time sheets and it’s obvious that there is a big problem somewhere, but the company isn’t acknowledging that there is an issue, or perhaps they are just doing this to save themselves a lot of money and get away with it. And the raters who are affected by this timesheet disapproval payment problem aren’t being asked to take an hour or two off, in fact, in some cases, people have been asked to cut 30-40 hours from their time sheets, some of them are asked to remove all hours from July timesheet. This is huge and it’s a lot of money for someone. In fact, it appears that it’s happening to a lot of people as one rater who has been working for the company says that he has 6 friends who all got the same notifications. In one case, one of our readers PrestigiousExplorer says that after working one year for Lionbridge the same thing has now happened to him and the company wants him to deduct 68 hours!  

The problem on their end which results in the timesheet disaproval appears to be that the hours that are being submitted don’t match their data. What it means is that the amount of tasks completed isn’t matching up with what your submitted amount is. But here’s the thing; if suddenly the Lionbridge payment department starts seeing a lot of discrepancies in time sheets and the data that they have, shouldn’t they realize that there is a problem somewhere? Now suddenly here you have a bunch of people being asked to shave a lot of hours off their time sheets, and even people who have been working for the company for a long time and then suddenly are having issues. It seems that Lionbridge doesn’t care about any of this and are sticking to their guns and won’t budge on time sheets. If you were subject to a timesheet disapproval then you might be stuck with it.  

The big question is does no one at Lionbridge see this as a problem? Or is there something else going on?  

As you may know raterhub automatically records the time and the number of tasks completed by each rater on the platform. It is very unlikely that the problem takes place there, however many long-term raters remember that raterhub used to have built-in task counter which was very helpful in terms of recording completed tasks, but for some reason this useful tool was removed from the raterhub functionality.

It looks like the Lionbridge system is just an Excel spreadsheet which corresponds timesheet data with data from raterhub (time recorded/the number of tasks completed) and the problem is there. Many raters worked lots of hours in July and got an email with 0 hours/0 tasks recorded by the system. Lionbridge seems to be very unprofessional in the handling of this problem. It undermines the project and the company’s reputation.  

But the fact that no one has caught the problem with timesheet disapproval or made any statement about this timesheet disaproval problem since it started happening in July leads one to believe in some kind of conspiracy here. Some people are calling Lionbridge scammers because of what is going on. If you’re suddenly asking a lot of raters to cut hours off their paychecks, there’s an obvious problem somewhere.

It does appear the company had the same kind of problem with timesheet disaproval in May in which they sent out incorrect data. It’s quite possible that this is the same issue, but instead of trying to figure out what is going on, it looks like the raters are getting the short end of the deal.

A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.  

Real timesheet correction required due to low productivity vs. Lionbridge error 

In some case raters might be asked to correct heir timesheet due to low productivity and it is OK, keep in mind that your average productivity rate  should be 1.5 – 2 minutes/per task.  

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater. 

Appen and Lionbridge have productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen and Lionbridge have again invented their own metrics to pay less to its raters. 

What raters can do in this situation?  
  1. To calculate the number of tasks completed weekly and entered into time sheet vs. Lionbridge records, if you have a huge discrepancy (like most of raters have) this is a clear indication of wrong data sent by Lionbridge. 
  2. Send an email to Lionbridge explaining the discrepancies.  
  3. If necessary as a proof of working hours/number of tasks a browsing history can be sent, it can be found, filtered by dates and website here: https://myactivity.google.com/myactivity
  4. Additionally I recommend sending an email directly to the Ads rating project admin (Google Ads Eval: ads_eval@google.com) to make them aware that this is some kind of issue that really needs to be resolved. 
  5. Do not correct time if you clearly identified discrepancies until Lionbridge responds to your email and provides you with correct data 

Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.  

If you have questions or comments then leave them below.  

Ipsoot Project Review From Appen Connect – Pearl and Falcon PT

Ipsoot project 

There is a new project that we’re going to go over today called Ipsoot Project (also known as Pearl and Falcon PT) where you are shown a random post on Facebook, and then you have to decide what topic that post fits under. It is basically a categorizing system job which aims to put the posts that are made into the correct categories of broad topics. 

Ipsoot project qualification 

The qualification process for this project is straight-forward and not difficult provided candidates study and understand project guidelines. The qualification consists of 22 questions, candidates will be asked to apply categorization and assign relevant topic to randomly selected images. 

Ipsoot project Rating process

The posts you receive are random and in no order and appear on the left, including the ad and page information under it such as the page name, who the author is, etc. You will also notice on the top left a Post ID and a Job ID number. 

экрана 2019-08-05 в 20.44.33

The objective when working on the Ipsoot Project is to look at the post and spend 30-60 seconds making sure that you understand it. Of course, it is important to consider the text in the ad along with the media. Basically, you want to think about how a photo might be understood or interpreted in combination with the text in the ad. 

It’s important to pay special attention to everything that is in the ad, including the photo and any text that may be overlaid on the photo as well as any text appearing above the photo. 

If the ad is text-only it’s important to understand that text and including any emojis that might be used in the post to understand what the ad is trying to accomplish and how it should be grouped. 

Sometimes you will see a link in the ad, and at this point, you should click on the link and quickly scan the content of the page it takes you to and consider if the ad itself or the text relates to that landing page. 

Sometimes an ad will contain a video, and you should watch that video for up to 60 seconds. However, if the video is too long then watch the first 10 to 15 seconds, and then click through five spots and watch another 5 to 10 seconds. You should also consider any accompanying text that is inside of the video. 

On the right-hand of the ad, you will see a box that says ‘To which topic(s) is this post related. Select all that apply.’ You should select every topic to which the post is related on the other hand if the post isn’t related to any of the topics then select the no topic checkbox underneath the box of topics.  

Underneath of this, you will see a panel that says ‘Additional Attributes’ and you should select every attribute which applies to the ad. Of course, if there aren’t any attributes that apply to the ad then you should select the no attribute checkbox. And finally, the last thing to do is to click submit which will serve you up another random ad to check. 

Next, to the submit button, you’ll see some skip buttons. You will use these if you encounter any error that prevents you from evaluating the post and they are as follows:

Skip: Tool Error – You should use this if for some reason the tool doesn’t load. This can mean that the ad area will appear blank, or you may not see all the buttons or functions. 

Skip Other Language – the majority of the time you’ll receive ads in the language which you are assigned to so if you work in the United States your ADS will be either English or Spanish depending on what type of position that you apply for and if your multilingual. However, if you do not understand the language in the ad then you should do nothing else but click on Skip: Other Language button.

That’s pretty much it for the tool and it’s quite straight forward. 

The biggest challenge to this type of job is trying to understand Topic Relevance. To give an example, if it would make sense to see the post in an Instagram feed about a certain topic, then the post is probably related to the topic itself. 

However, it is not sufficient for a post to merely mention the topic. While it’s true that some posts explicitly mention the topic, some do not address the topic as it would show up in a search result. 

A good example of this would be an ad about a school competition winner who is pictured with his parents, his sister, and a dog. Simply looking at the ad isn’t going to help you unless you take a look at the text. So you wouldn’t tag this as “animals and pets” simply because there is a picture of a boy and his dog in the photo. However, if that same photo mentioned something about ‘man’s best friend’ you could tag it as animals and pets.

One of the questions that you may have is ‘What if the post is about more than 1 topic?’ Well, you should pay particular attention to a post that can be related to multiple topics because many posts can be coded with more than one category. 

We’ll give you some examples. 

For instance, let’s say that you have a post about a new economic policy. In this case, you would tag ‘politics’ + ‘business finance and economics’.

If a post was about someone using a new fitness routine to improve their health, then you would tag ‘health and medical’ + ‘fitness and workouts’.

A post about a politicians stance and welfare programs you would select ‘politics + social issues’ 

If the post is about the best restaurant three friends found while traveling through Italy then you would tag it ‘food and drink + travel and leisure + relationships friends and family’ 

As you can see, it’s quite possible many posts can have more than 1 topic. 

There are a few other rules when it comes to tagging topics about a post when it comes to regard for quality, professionalism, or the size of the audience for the post. 

We’ll show you some examples of how a topic doesn’t change regardless of these attributes. 

If the post is a professional video of a Justin Timberlake concert then you would tag this under ‘music and audio’.

But let’s say that the post is of a band performing in a bar. You would still tag it ‘music and audio’. Even though that band may not be as popular as Justin Timberlake.

Even if it is a post of a cell phone video of a 3-year old singing in a car, you would still tag this is as ‘music and audio’.

However, it is important to pay special attention to whether the post is related to any entity such as unique people, places or things. For example

The post is about President Trump – You would tag ‘politics’

If the post was about a McDonald’s restaurant, then you would tag ‘Food and drink’

And if the post was about The Beatles, then you would tag ‘Music and audio’

However, you should be careful to not assume that every post that mentions an entity is it about that entity itself. For instance, a video about the Olympics might show a logo of its sponsor, McDonald’s, but that is not enough to consider the post to be related to McDonald’s itself and you wouldn’t tag the post ‘food and drink’. 

When the author of the post is a well-known individual like Barack Obama, and he’s posted something saying that the Chicago Bulls are the best team ever, you would tag this as politics because it’s just a personal opinion by an individual strongly associated with politics. 

If you use Facebook, you probably understand that memes are something that you see quite often and its important to understand how to tag a meme. Memes often used content from one topic to make a statement about another topic. In the case of memes, you would actually tag both topics. For example, if you see a meme of a TV character looking sad with the caption ‘how I feel when my pet walks away from me’ you would tag this TV & Movies + Animals and Pets.

There’s going to be a lot of times when you’re going to have to do research because you’ll find names or phrases in a post that you do not understand. You should always do a search to find out if you’re not sure when referencing a topic. 

It’s also a good idea to identify some of the definitions in the guidelines. 

Topics describe the subject of the content. It is important to determine if the content in a Facebook feed is relevant to the topic. This means that it’s not sufficient to call something ‘related’ to a piece of content that merely mentions the topic. There are some examples of topics and how they should be grouped. And although the guidelines go over each one and give inclusions and exclusions, it is not an exhaustive list but merely examples.

https://passleapforceexam.com/2019/03/30/appen-connect-platform-list-of-projects-and-code-names/

If you are working on the Ipsoot Project then we would like to hear from you and your experiences. If you need help with any information on this page, then email me and I’ll get back to you as soon as possible. 

Exam Preparation Tool For Appen Arrow and Lionbridge Exams

WHY CANDIDATES FAIL ARROW/ADS ASSESSOR EXAM?

An Exam Preparation Tool has just been created to help you pass these difficult exams you are given when you apply for ads evaluation jobs, you have to pass a qualification exam that is quite difficult. Each company has a name for their ads assessor project. Appen Ads Assessor project is called Arrow if you work for Lionbridge you work on the Ads Assessor project.

Despite all the fancy names, the exam and rating process are the same for these companies because the customer is one, only vendors are different. The biggest problem with this exam is that it is difficult to pass this qualification with only one attempt. In fact, even Appen and Lionbridge explain in a welcome mail that there is a high chance of failing it. With that said, you’re going to need all the help you can get. With the Exam Preparation Tool you can be sure you’re ready.

Keep in mind that unlike Yukon project/Personalized Assessor and Maps Analyst (Shasta) projects’ qualification, the Ads Assessor/Arrow project qualification allows only one attempt to pass the exam.

STUDYING THE GUIDELINES AND DOING THE EXAM SIMULATION PRACTICE ARE ESSENTIAL IN SUCCESS IN PASSING THE ACTUAL EXAM

If you think this is an exaggeration, then just search for comments on passing the exam and see what you will find. You’ll find in a lot of cases people who claim that they studied for several days before the exam began. With that said, it’s just not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort in the one-week time frame they give you to study the guide. The guide itself is designed in a way that assesses your ability to understand and apply rating concepts and the rules based on instructions. There is a lot of material to go over, and it can be a bit confusing. But once again, you’re not going to get anywhere near passing if you don’t start going through the guide.

WHAT IS ADS EVALUATION EXAM SIMULATION TOOL?

The tool will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include: Search Ads Evaluation, User to Keywords, User Interests and Close Match Evaluation tasks. The tool has 10 Search Ads Evaluation tasks, 7 User to Keywords tasks, 5 User Interests Evaluation tasks and and 5 Close Match Evaluation tasks, these tasks are the main exam tasks types which cover basically all rating concepts, which will help you to develop rating skills which will be tested during a real exam.

HOW USING THIS TOOL CAN HELP YOU PASS THE EXAM?

You might be wondering how using this tool can help you pass the exam. The way that it is designed will show you exactly your problem area that you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to disclaim that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from most typical exam tasks, and is designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam.

HOW TO PROCEED WITH EXAM SIMULATION TOOL?

Exam Simulation Demo

If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains 1 Search Ads Evaluation task, 1 User to Keywords task, 1 User Interests Evaluation task, and 1 Close Match Evaluation tasks so you can see how it works. Once again, the demo is 100% free and you’re completely anonymous.

Exam Simulation Full Version

The tool has 10 Search Ads Evaluation tasks, 7 User to Keywords tasks, 5 User Interests Evaluation tasks and and 5 Close Match Evaluation tasks, these tasks are the main exam tasks types which cover basically all rating concepts, will allow to develop rating skills which will be tested during a real exam.

EXPLORE ADS ASSESSOR/ARROW EXAM SIMULATION TOOL HERE:

https://adsevaluatorexamsimulation.passleapforceexam.com/