Lionbridge Ads Rating Project Massive Timesheet Disapproval: on Purpose or Unprofessional Approach? 

Ads Rating Project massive timesheet disapproval: on purpose or unprofessional approach? 

It appears that lately there are a lot of problems with the Lionbridge payment systems and it’s not actually a new thing. According to comments of my readers and discussions on https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/ , it the problem started to happen in May-June 2018. It just seems that now it’s becoming worse and there are more and more raters complaining that the time sheets that are being submitted are being returned. Lionbridge is asking people to carve a lot of hours from their time sheets and it’s obvious that there is a big problem somewhere, but the company isn’t acknowledging that there is an issue, or perhaps they are just doing this to save themselves a lot of money and get away with it. And the raters who are affected by this Lionbridge payment problem aren’t being asked to take an hour or two off, in fact, in some cases, people have been asked to cut 30-40 hours from their time sheets, some of them are asked to remove all hours from July timesheet. This is huge and it’s a lot of money for someone. In fact, it appears that it’s happening to a lot of people as one rater who has been working for the company says that he has 6 friends who all got the same notifications. In one case, one of our readers PrestigiousExplorer says that after working one year for Lionbridge the same thing has now happened to him and the company wants him to deduct 68 hours!  

The problem on their ends appears to be that the hours that are being submitted don’t match their data. What it means is that the amount of tasks completed isn’t matching up with what your submitted amount is. But here’s the thing; if suddenly the Lionbridge payment department starts seeing a lot of discrepancies in time sheets and the data that they have, shouldn’t they realize that there is a problem somewhere? Now suddenly here you have a bunch of people being asked to shave a lot of hours off their time sheets, and even people who have been working for the company for a long time and then suddenly are having issues. It seems that Lionbridge doesn’t care about any of this and are sticking to their guns and won’t budge on time sheets.  

The big question is does no one at Lionbridge see this as a problem? Or is there something else going on?  

As you may know raterhub automatically records the time and the number of tasks completed by each rater on the platform. It is very unlikely that the problem takes place there, however many long-term raters remember that raterhub used to have built-in task counter which was very helpful in terms of recording completed tasks, but for some reason this useful tool was removed from the raterhub functionality.

More than likely this is some kind of problem some database issue, it looks like the Lionbridge system is just an Excel spreadsheet which corresponds with data from raterhub (time recorded/the number of tasks completed) with their internal timesheet database and the problem is likely there. Many raters worked lots of hours in July and got an email with 0 hours/0 tasks recorded by the system. Lionbridge seems to be very unprofessional in the handling of this problem. It undermines the project and the company’s reputation.  

But the fact that no one has caught the problem or made any statement about it since it started happening in July leads one to believe in some kind of conspiracy here. Some people are calling Lionbridge scammers because of what is going on. If you’re suddenly asking a lot of raters to cut hours off their paychecks, there’s an obvious problem somewhere.

It does appear the company had the same kind of problem in May in which they sent out incorrect data. It’s quite possible that this is the same issue, but instead of trying to figure out what is going on, it looks like the raters are getting the short end of the deal.

A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.  

Real timesheet correction required due to low productivity vs. Lionbridge error 

In some case raters might be asked to correct heir timesheet due to low productivity and it is OK, keep in mind that your average productivity per a week should be 2 minutes/per tasks.  

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater. 

Appen and Lionbridge have productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH matric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen and Lionbridge have again invented their own metrics to pay less to its raters. 

What raters can do in this situation?  

  1. To calculate the number of tasks completed weekly and entered into time sheet vs. Lionbridge records, if you have a huge discrepancy (like most of raters have) this is a clear indication of wrong data sent by Lionbridge. 
  2. Send an email to Lionbridge explaining the discrepancies.  
  3. Additionally I recommend sending an email directly to the Ads rating project admin (Google Ads Eval: ads_eval@google.com) to make them aware that this is some kind of issue that really needs to be resolved. 
  4. Do not correct time if you clearly identified discrepancies until lionbridge responds to your email and provides you with correct data 

 Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.  

If you have questions or comments then leave them below.  

Advertisements

Common Issues of New Raters (Arrow Leapforce and Lionbridge Ads Rating Projects)

Support_636x350-600x330Workflow 

After you have passed either the Lionbridge Ads or the Arrow ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a chance that you will still see some English tasks come through once in a while.  

Time keeping and quality 

Time keeping is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Timesheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.  

One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.  

What are the quality review tasks? 

Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. 

Reasons for losing your service as an Ads Quality Rater 

It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.  

Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.  

Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.  

IP address conflict/Network sharing

In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.

Multi-accounts

Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.

No tasks are available, all task options are not available anymore except the QR code

This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.

Lionbridge (Ads rating project) vs Appen (Aztec)&Leapforce(Arrow)

Productivity 

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.

Appen(Leapforce) has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH matric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen(Leapforce) has again invented their own metrics to pay less to its raters.

Working hours and flexibility

At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.

At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.

Payments

Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.

Appen(Leapfroce) pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.

Payoneer As a Payment Method for Appen, Raterlabs, Leapforce, Lionbridge and Utest

Payments rates

In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.

More info about payment rates is available here:

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Leapforce, Lionbridge and Appen When Combining Several Projects?

General Support

The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of monthly practice sets and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.

Optional Agreements

It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.

The Final Word 

If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too. 

Ads Rating Project massive timesheet disapproval: on purpose or unprofessional approach? 

It appears that lately there are a lot of problems with the Lionbridge payment systems and it’s not actually a new thing. According to comments of my readers and discussions on https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/ , it the problem started to happen in May-June 2018. It just seems that now it’s becoming worse and there are more and more raters complaining that the time sheets that are being submitted are being returned. Lionbridge is asking people to carve a lot of hours from their time sheets and it’s obvious that there is a big problem somewhere, but the company isn’t acknowledging that there is an issue, or perhaps they are just doing this to save themselves a lot of money and get away with it. And the raters who are affected by this Lionbridge payment problem aren’t being asked to take an hour or two off, in fact, in some cases, people have been asked to cut 30-40 hours from their time sheets, some of them are asked to remove all hours from July timesheet. This is huge and it’s a lot of money for someone. In fact, it appears that it’s happening to a lot of people as one rater who has been working for the company says that he has 6 friends who all got the same notifications. In one case, one of our readers PrestigiousExplorer says that after working one year for Lionbridge the same thing has now happened to him and the company wants him to deduct 68 hours!  

The problem on their ends appears to be that the hours that are being submitted don’t match their data. What it means is that the amount of tasks completed isn’t matching up with what your submitted amount is. But here’s the thing; if suddenly the Lionbridge payment department starts seeing a lot of discrepancies in time sheets and the data that they have, shouldn’t they realize that there is a problem somewhere? Now suddenly here you have a bunch of people being asked to shave a lot of hours off their time sheets, and even people who have been working for the company for a long time and then suddenly are having issues. It seems that Lionbridge doesn’t care about any of this and are sticking to their guns and won’t budge on time sheets.  

The big question is does no one at Lionbridge see this as a problem? Or is there something else going on?  

As you may know raterhub automatically records the time and the number of tasks completed by each rater on the platform. It is very unlikely that the problem takes place there, however many long-term raters remember that raterhub used to have built-in task counter which was very helpful in terms of recording completed tasks, but for some reason this useful tool was removed from the raterhub functionality.

More than likely this is some kind of problem some database issue, it looks like the Lionbridge system is just an Excel spreadsheet which corresponds with data from raterhub (time recorded/the number of tasks completed) with their internal timesheet database and the problem is likely there. Many raters worked lots of hours in August and got an email with 0 hours/0 tasks recorded by the system. Lionbridge seems to be very unprofessional in the handling of this problem. It undermines the project and the company’s reputation.  

But the fact that no one has caught the problem or made any statement about it since it started happening in July leads one to believe in some kind of conspiracy here. Some people are calling Lionbridge scammers because of what is going on. If you’re suddenly asking a lot of raters to cut hours off their paychecks, there’s an obvious problem somewhere.

It does appear the company had the same kind of problem in May in which they sent out incorrect data. It’s quite possible that this is the same issue, but instead of trying to figure out what is going on, it looks like the raters are getting the short end of the deal.

A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.  

Real timesheet correction required due to low productivity vs. Lionbridge error 

In some case raters might be asked to correct heir timesheet due to low productivity and it is OK, keep in mind that your average productivity per a week should be 2 minutes/per tasks.  

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater. 

Appen and Lionbridge have productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH matric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen and Lionbridge have again invented their own metrics to pay less to its raters. 

What raters can do in this situation?  

    1. To calculate the number of tasks completed weekly and entered into time sheet vs. Lionbridge records, if you have a huge discrepancy (like most of raters have) this is a clear indication of wrong data sent by Lionbridge. 
    1. Send an email to Lionbridge explaining the discrepancies.  
    1. Additionally I recommend sending an email directly to the Ads rating project admin (Google Ads Eval: ads_eval@google.com) to make them aware that this is some kind of issue that really needs to be resolved. 
  1. Do not correct time if you clearly identified discrepancies until lionbridge responds to your email and provides you with correct data 

 Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.  

If you have questions or comments then leave them below.  

Arrow Leapforce(Appen Aztec) and Lionbridge Ads Rating Projects Review

deep-links-within-mobile-app-engagement-search-adsArrow Leapforce (Appen Aztec) and Lionbridge Ads rating projects overview 

The Arrow Ads project and Lionbridge Ads rating are similar search engine evaluation project that is using the same Rater Hub platform that the other two large companies Leapforce and Lionbridge are using to rate search engine results depending on a query or intent of the user. Basically, you are doing the same thing as other search engine evaluators, but the difference is that you are judging the quality of ads and advertising-related data as they are calling it. Feedback that you provide is supposed to help the client monitor and improve their advertising as well as functionality.  

So what you are doing with this certain project is taking a look at different ads on web pages, and using the Rater Hub tool, you answer some questions on your findings. It sounds easy but answering these questions, and the answers that you provide are coming from guidelines that need to be closely followed. It’s not simply about providing your own feedback, but providing the feedback based on the guidelines themselves and specific metrics.  

What is the program purpose? 

As a human evaluator, you are not approving or disapproving the ads you evaluate, and you are not deciding directly which ads get shown or don’t get shown. In fact, even though you are providing scores and feedback about individual advertisements and advertisers, your evaluations don’t generally have any direct effect on those advertisements or advertisers. Sometimes the ads you rate are from a long time ago. Sometimes they are in a different form than they originally appeared. Sometimes they were never shown by Google at all, in any form. Basically what you are doing is collecting data to help train the algorithms that will eventually assess the ads being shown on their own. Without human interaction this would not be possible. So this job is very important to the engineers who are collecting the data from you.  

This is the main reason why it’s very difficult to pass the assessment part of this project. Your judgments are used to determine whether ads are getting better or worse over time, to see whether particular kinds of ads are working well or not, to see what kinds of ads cause problems, and to see what kind of impact a particular change to our systems will have. When Google engineers design new systems or propose changes to existing systems, the human evaluation data you provide is used to judge whether those proposals succeed or fail. This data is fed to computers that train the automatic algorithms.  

This is also a big reason that a lot of raters end up being terminated or losing their contracts. Accuracy and quality of your ratings are very important. You should always check the Ads Guidelines if you aren’t sure about something that you are looking at. Even some raters that have been doing this job for a while still refer to the guidelines.  

Arrow Leapforce (Appen Aztec)  and Lionbridge Ads rating program requirements 

Getting on this project can be difficult. There is a long assessment required. Its mentioned early on that you will perform all work and including the assessment tasks on a non-shared and non-public computer. That’s pretty straight forward. You’ve got to have your own computer with a high speed internet connection. And according to them you can’t take your laptop to the coffee shop and work from there. So it’s a very good possibility that your home IP address is recorded in the RaterHub, and you need to use this same connection to perform all work. Anything else is strictly prohibited. 

Another thing to mention is that you’re going to need to use either Firefox or Chrome web browser but that should hinder anyone since those seem to be the main web browsers being used by the masses. This is because the Rater Hub tool will not work on other browsers.  

If you do pass the assessment, then you may be offered a contract position. Usually the contract hours are part-time 20 hours a week, but Aztec states 19 hours, Monday through Sunday. Once you complete the assessment, the results are usually sent within 48 business hours after your deadline to complete the assessment has passed, not 48 hours after you submit the assessment.  

As with all projects we can’t stress enough the important of one thing. Thoroughly read the guide they send you and have it open and ready while you are taking the tests. Make sure the minute you get the link to review the guide, that going over it is the first thing you do. Give yourself enough time to digest everything in it at least 1-2 times before taking the tests. The guide itself has to be accessed directly through EWOQ which is part of the RaterHub portal. You’ve got to be logged in using the Gmail address you used when you signed up at Appen/Lionbridge. This Gmail account should be a dedicated email account.  

Since you are not able to download the guide, you should bookmark the page with the guidelines so that you can access it quickly and easily. We said that it cannot be mentioned enough and it’s very important that you read and understand the guidelines completely before starting or you will fail the test.  

What are Arrow Leapforce (Appen Aztec) and Lionbridge Ads program payment rates? 

Wages on each contractual hour depend on the country and can vary between $4 to $15 per hour. For more information on wages read here: HOW MUCH CAN YOU EARN WITH LEAPFORCE /LIONBRIDGE/APPEN  

Agents will invoice vendors for rendered services once per calendar month. Fees to independent agents are paid (via direct deposit) within 30 days of receipt of a valid invoice. The payment is made on monthly basis via direct deposit into your bank account. Appen projects are paid now by Payoneer. If you do not have a Payoneer account yet you can get yours here: 

http://tracking.payoneer.com/SHRq 

Note: By using this link for applying for a Payoneer account/card we will both benefit by earning 25$. 

Please read here for more info here:

Payoneer As a Payment Method for Appen, Raterlabs, Leapforce, Lionbridge and Utest

Arrow Leapforce (Aztec Appen) and Lionbridge Ads program Assessment  

The first set of tasks are multiple choice questions (in total 45 questions) that come straight from the Search Ad instructions in the Ads Guide itself. After you have completed the comprehension questions, you will be shown a set of assessment tasks (in total 190 tasks of different types: search ads evaluation, user interest, key words, etc) that asks you to rate just the first section of a Search Ads task. Once this is done you’ll get a set that asks you to rate just the second section. Finally, you’ll get a set of complete Search Ads tasks, User to Keyword, Close Match Evaluations to rate.

The typical exam tasks is a query and an ad creative generated by a search engine in respect to the user query. You as a rater will need to apply rating rule to determine how much promising or unpromising is the ad in respect to user intent. Once you evaluate the ad then you will be asked to evaluate a landing page associated with the ad and to determine how satisfactory is the page and website content in terms of meeting the user intent needs.

Ads rating project/Arrow Exam Tips

Additionally I would like to give general tips on how to proceed with Aztec/Ads Rating Project qualification:

As usual before proceeding with rating you need to make sure you clearly understand user intent. If not, use Google search to research the query and establish the dominant intent, and the secondary intent if one exists. Sometimes it’s hard to understand what the user intent is without doing the search yourself;

Search Ads Evaluation Tasks

Select appropriate query flag if required. For instance, if the query is in foreign language, or it is clearly a porn related query, make sure you select the appropriate flag. Remember that when you are doing ratings, the language should be in your contracted regional language. So if you are working in English, and you get a page in another language, then flag it appropriately.

Examine the ad creative and answer the question of how promising the ad is. You should refer to a specific section of instructions, and not evaluate the ad creative based on your own perception or personal likes or dislikes. Pay close attention to wording of the ad and the URL. In many cases, wording of the ad may indicate the ad deserves a porn flag, or has reference to another search engine.

Check the query and ad creative for the navigational flag. If the query is looking for a specific page or website and the ad suggests the website, then the ad and the query should be given a navigational flag.

There are a few scenarios in which you should pay special attention to. The first one is when the ad offers to repeat the search in another search engine. In this case, the appropriate rating for ads and LP (Landing Page) are somewhat unpromising, and don’t satisfy the query because the action won’t bring the user closer to his goal. Google wants to provide the user with the quickest possible and satisfying solution for the user intent and query. Sending a user to another search engine defeats that purpose entirely.

Select the ads creative rating, or how promising the ad is in relation to user intent. Once you’ve completed the first part of the task you will then be directed to the Landing Page section.

Visit the landing page and examine the page based on the guideline rules of how satisfactory the page is based on the query or intent. Once again, you should not use your own judgement but instead learn how to apply the guidelines rules. In many cases research on the website is required, for instance a quick product search will show you more information about the page, and help you understand if it meets user intent based on the query.

Make sure to pay special attention to the user location. If you see that the user is looking for something in a specific location, such as a restaurant in a specific location, and you get a restaurant in a completely different location, you should apply a negative rating. In this case it’s a good idea to refer to the guidelines and review typical user location scenarios.

User to Keywords/User Interest Tasks/Close Match Evalution

Before proceeding to these types of tasks make sure you read and understand instructions, because the logic is different comparing to approach required for search ads evaluation tasks. You will be asked to rater how much is interesting for a user the suggested page/url based on examination of a user current web interest.

Once you have completed all assessment tasks, including the survey, you will temporarily run out of tasks. This is because your answers will be checked before you will be allowed to move forward. Either you will pass this part or not. You will not see the Acquire Tasks button at this time. Remember that you need to wait 48 hours after the assessment deadline and not 48 hours after you submit. The assessment will be complete after you submit all tasks available.  

Keep in mind that unlike Yukon project/Personalized Assessor and MapsAnalyst (Shasta) projects’ qualifications, the Ads Assessor/Aztec project allows only one attempt to pass the exam. You will not be given to attempt the exam for the second time. 

If you are not very confident you are welcome to contact me for expert tips and recommendations via contact email.

Appen Aztec/Arrow Leapforce vs. Lionbridge Ads rating project  

Many people ask me which project is better and which one should I join. In my opinion I think that Lionbridge is definitely better in terms of offering a better payment rate compared to Appen. It’s also important to note that the exam qualification and training are paid by Lionbridge where at Appen it’s not. So take this into consideration when you apply for both of these companies. Three days of unpaid training is a lot for people to deal with and really when you think about it, no one wants to do an exam for three days, and then fail it having put all the time and effort into it to get a job.  

As always, if you have any questions or thoughts leave a comment or shoot me an email.  

Wishing you great success in your work at home career!

Common Issues of New Raters (Appen Aztec and Lionbridge Ads Rating Projects)