Common Issues of New Raters (Appen Aztec and Lionbridge Ads Rating Projects)

Support_636x350-600x330Workflow 

After you have passed either the Lionbridge Ads or the Aztec ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a chance that you will still see some English tasks come through once in a while.  

Time keeping and quality 

Time keeping is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Timesheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.  

One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.  

What are the quality review tasks? 

Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. 

Reasons for losing your service as an Ads Quality Rater 

It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.  

Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.  

Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.  

IP address conflict/Network sharing

In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.

Multi-accounts

Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.

No tasks are available, all task options are not available anymore except the QR code

This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.

Lionbridge (Ads rating project) vs Appen (Aztec)

Productivity 

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.

Appen has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH matric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen has again invented their own metrics to pay less to its raters.

Working hours and flexibility

At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.

At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.

Payments

Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.

Appen pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.

Payoneer As a Payment Method for Appen, Raterlabs, Leapforce and Lionbridge

Payments rates

In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.

More info about payment rates is available here:

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Leapforce, Lionbridge and Appen When Combining Several Projects?

General Support

The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of monthly practice sets and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.

Optional Agreements

It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.

The Final Word 

If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too. 

Ads Rating Project massive timesheet disapproval: on purpose or unprofessional approach? 

It appears that lately there are a lot of problems with the Lionbridge payment systems and it’s not actually a new thing. According to comments of my readers and discussions on https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/ , it the problem started to happen in May-June 2018. It just seems that now it’s becoming worse and there are more and more raters complaining that the time sheets that are being submitted are being returned. Lionbridge is asking people to carve a lot of hours from their time sheets and it’s obvious that there is a big problem somewhere, but the company isn’t acknowledging that there is an issue, or perhaps they are just doing this to save themselves a lot of money and get away with it. And the raters who are affected by this Lionbridge payment problem aren’t being asked to take an hour or two off, in fact, in some cases, people have been asked to cut 30-40 hours from their time sheets, some of them are asked to remove all hours from July timesheet. This is huge and it’s a lot of money for someone. In fact, it appears that it’s happening to a lot of people as one rater who has been working for the company says that he has 6 friends who all got the same notifications. In one case, one of our readers PrestigiousExplorer says that after working one year for Lionbridge the same thing has now happened to him and the company wants him to deduct 68 hours!  

The problem on their ends appears to be that the hours that are being submitted don’t match their data. What it means is that the amount of tasks completed isn’t matching up with what your submitted amount is. But here’s the thing; if suddenly the Lionbridge payment department starts seeing a lot of discrepancies in time sheets and the data that they have, shouldn’t they realize that there is a problem somewhere? Now suddenly here you have a bunch of people being asked to shave a lot of hours off their time sheets, and even people who have been working for the company for a long time and then suddenly are having issues. It seems that Lionbridge doesn’t care about any of this and are sticking to their guns and won’t budge on time sheets.  

The big question is does no one at Lionbridge see this as a problem? Or is there something else going on?  

As you may know raterhub automatically records the time and the number of tasks completed by each rater on the platform. It is very unlikely that the problem takes place there, however many long-term raters remember that raterhub used to have built-in task counter which was very helpful in terms of recording completed tasks, but for some reason this useful tool was removed from the raterhub functionality.

More than likely this is some kind of problem some database issue, it looks like the Lionbridge system is just an Excel spreadsheet which corresponds with data from raterhub (time recorded/the number of tasks completed) with their internal timesheet database and the problem is likely there. Many raters worked lots of hours in August and got an email with 0 hours/0 tasks recorded by the system. Lionbridge seems to be very unprofessional in the handling of this problem. It undermines the project and the company’s reputation.  

But the fact that no one has caught the problem or made any statement about it since it started happening in July leads one to believe in some kind of conspiracy here. Some people are calling Lionbridge scammers because of what is going on. If you’re suddenly asking a lot of raters to cut hours off their paychecks, there’s an obvious problem somewhere.

It does appear the company had the same kind of problem in May in which they sent out incorrect data. It’s quite possible that this is the same issue, but instead of trying to figure out what is going on, it looks like the raters are getting the short end of the deal.

A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.  

Real timesheet correction required due to low productivity vs. Lionbridge error 

In some case raters might be asked to correct heir timesheet due to low productivity and it is OK, keep in mind that your average productivity per a week should be 2 minutes/per tasks.  

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater. 

Appen and Lionbridge have productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH matric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen and Lionbridge have again invented their own metrics to pay less to its raters. 

What raters can do in this situation?  

  1. To calculate the number of tasks completed weekly and entered into time sheet vs. Lionbridge records, if you have a huge discrepancy (like most of raters have) this is a clear indication of wrong data sent by Lionbridge. 
  2. Send an email to Lionbridge explaining the discrepancies.  
  3. Additionally I recommend sending an email directly to the Ads rating project admin (Google Ads Eval: ads_eval@google.com) to make them aware that this is some kind of issue that really needs to be resolved. 
  4. Do not correct time if you clearly identified discrepancies until lionbridge responds to your email and provides you with correct data 

 Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.  

If you have questions or comments then leave them below.  

Advertisements

30 thoughts on “Common Issues of New Raters (Appen Aztec and Lionbridge Ads Rating Projects)

  1. Hello Tim

    After two months into working with LB, my account has been blocked and I got an email explains that my quality has been poor(around sixty). So I went on to check my feedback in the portal. There are 6 tasks that were reviewed and given feedback to.

    However, about half of the feedback are wrong. I think the auditors failed to check accuracy of the news, clicking further into the links. So I sent an email explaining the possible mistakes of them and indicate which tasks that I can agree with. After one week that my account has been blocked, I got another email instructs me to complete additional learning path but nothing related to my questions.

    Is this audit final and not disputable? Should I continuing asking for explanation or just accept this?
    What if this quality assessment stands, how much more errors can I make?

    NB. I have been always met the required hours, but I missed about 1-2 hours from maximum the week before the blockage.

    Please help. Thanks you.

  2. I have been working on ads rating project for a year, never had a task shortage before but since last and I’m facing a task shortage Tuesday. However I still got the 3 options marked on my EOWQ. Is anyone experiencing the same issue with shortage of tasks? I am from USA.

    1. Hello,

      As far as I know all markets are experiencing the same issue (shortage of tasks in Ads rating/Aztec projects).
      I have an assumption that it has to do with updating raterhub tool. Vendors do not provide any info regarding this issue, the only response I got from them is to check raterhub for tasks on a regular basis. I hope the volume of tasks will increase in the near future.

  3. Hello,

    I got such email from Appen:

    Our records show a significant discrepancy between the hours you’ve submitted and estimated hours based on RPH as shown below.

    Week

    Estimated Hours Based on RPH

    Total Rater Hours Submitted

    04/02 – 04/08

    19

    29

    When you have a low RPH it negatively impacts the RPH average of the entire roster and project. It is the project requirement raters meet the average rate per hour on a consistent basis. The average rate per hour varies according to each task and should be made available to you in the judging interface or guidelines. In the meantime, these hours will not be approved until clarification has been received. Please amend your hours if needed.

    *************************************************************************************************************************

    Why do they send this email and how do they calculate RHP? I had lost of time consuming tasks (like video evaluation) and I am sure that I entered correct and reasonable time…moreover there is no any indication for the recommended time for many tasks…Do you think Appen cuts my hours to pay me less on purpose?

  4. Hey Tim,

    I have a question regarding the Lionbridge payment.

    “Non-US raters will receive their first payment only on the second month”. What about the next payments after the first one ? Will they be again after 60 days or switched to 30? Thanks.

    1. Hello,

      After the first payment, the following up payment will be paid on a monthly basis, you may find payment schedule on your project portal.

  5. Hi Tim! Thanks for the website: your work is a lifesaver!

    I recently completed the LF Yukon evaluation and when the exam due date came I received no email or any contact from them.

    At around the same time as the exam was due (same day, some minutes after), I got an email from Appen inviting me to join/qualify for Aztec. I know LF belongs to Appen now. Was this invitation their way of letting me know I had failed the exam? I had applied for positions at Appen so long ago, I had forgotten about it.

    Normally having two job opportunities on the table wouldn’t bother me at all but you wrote Yukon and Aztec are incompatible and that is the source of my worries.

    1. I have information that some of candidates have not received exam results on time I assume this is due to transition of the qualification process from Leapforce administration to Appen. I recommend you to write to agent support and ask for an exam retake.

      1. Hi Tim! Thanks for the reply.

        I got the results later and.. I passed! Unfortunately I got an email stating they had no use for me for the time being, so there’s that. I suppose they don’t call you after sending that, do they?

        Any idea why I might have received that email?

        Cheers

      2. Hello, try to apply for the the same or similar role at anther vendor (Lionbridge).

  6. Hello,

    Today I have got an email from Lionbridge where they ask me to cut 40 hrs from my July timesheet/invoice. I am really confused about this request as I always keep track of time I spend on doing live tasks in Ads Rating project, my average monthly productivity is 2 minutes. Does anyone experience the same issue? I also noticed that the number of tasks I completed each week in July does not correspond to the number of tasks I entered into the timesheet, is it a system glitch? I remember the had the same issue in may that time they sent out emails that they sent out incorrect data, and now the same issue???

    1. Hi everyone here

      I have the same issue again and this does not make me happy at all.

      I am still having problems with my timesheet and this is getting bigger and more worrying.

      Since the May timesheet incident, I’ve been very careful logging time; I log time for ‘live’ tasks only, I disregard time spent on reading task guidelines, I make sure not to let any tasks expire, etc. Yet I am still getting emails from Lionbridge – regularly – warning me that I have overstated my work time.

      Fyi, I had to remove 6.5 hours from my May timesheet, 12 hours from my June timesheet. Few days ago, I was told I had overstated by 20 hours for the first week of this month (I was given permission to work up to 30 hours due to large volume of tasks). And the list goes on..

      There is no explanation from Lionbridge as to why this is happening and it’s driving me crazy.

      Do you have any idea what’s going on? Do you think perhaps I’m spending too much time on each task?

      If I can’t solve this problem, I might quit and apply to Appen..its too stressful.

      Thanks

  7. Hello,

    Thank you for your comments. I am informed about this issue with massive disapproval of time sheets in Ads rating project as I am getting lots of personal emails from raters from different markets… What is in common is the fact that the correction data they send (weekly number of tasks/hours) do not correspond with weekly number of tasks/hours entered by raters into their timesheet. Based on this information I may assume that they have a system glitch and my recommendation is to compare your timesheet data vs their records for discrepancy (in particular the number of tasks completed weekly vs. lionbridge records). Once you do that, write email to Lionbridge about the issue. I think if they get lots of emails about the issue they will finally review all their records and fix their stupid system.

    1. THEY ARE REALLY BECOMING SCAMMERS. I tried to make a post before, but it was deleted. Here is my message:

      I’m a web search evaluator from Lionbridge and in the past 2 months I’ve been receiving emails from the homeworker supporter team asking for us raters to update the timesheet accordingly the time registered in their system and remove the hours as stated. The problem here is that, most of us raters have contact with each other and we know exactly the time we spent rating tasks and we all are receiving the same email asking to uptade the timesheet. Most of us are honest about the time we spend and record rating live tasks and it’s pretty IMPOSSIBLE for us to have ENORMOUS amounts of discrepancies in the timesheet. We worked an enormous amount of times and they are asking us to remove almost half of the hours we worked throughout the month. We don’t even record the time we spend reading the guidelines or tests, we register the time we spend rating live tasks only. I’m almost sure they are being dishonest this time. Emailing them won’t resolve, they will always email tag back never admitting they are wrong. Why did they delete the timer and rating history to keep track of our tasks? They’ve been doing the same thing to so many people lately and the only explanation I can find is that they are trying to deceive or steal us, or maybe their systems are glitched or bugged, but they don’t want to fix it in order to profit on our backs.

      1. I assume there is a glitch in the system (raterhub), what we can do in this situation to write to Lionbridge about the issue and submit the issue report/complain directly to Google rating team via raterhub platform->report a problem option

  8. I’ve been working with Lionbridge for almost 1 year and the same thing happened to me, Lionbridge has asked me to take 68 hours from 06/25/2018 to 08/05/2018 and I’m outraged! I sent 3 emails to them and they didn´t respond to me until now. I don´t know how to proceed. I found this another link where several people are complaining about the same issue in comments section, check it out:

    https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/

    I think that we need to discuss in both websites and take an action together…

    1. Hi guy I sent email directly to the project admin team at Google via raterhub, I hope they will take actions on the issue.

      Hello,

      I would like to infom you about the issue related to the invoicing process at Lionbridge, it looks like that they have a system glitch in terms of recording time/the number of tasks completed by raters. In my case they asked me to cut hours and number of tasks completed, thier data is wrong, because the number of tasks and hours donot even closely correspond to actual number of tasks and hours I completed and entereted into the timesheet.

      Based on the thread on Reddit I assume this issue is massive and take place across all markets.https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/

      Please take actions on this unprofessional approch which undermine project reputation.

  9. I think this issue started from May when they had a issue with gmail logon on ratehub, then many raters reported that they could not login the platform and they could see someone’s gmail on their page, I assume that this cased the issue in SQL database and as a result the system records a wrong rater time/the number of tasks, and it can be clearly seen from the fact of huge discrepancies…I wander why Lionbridge still cannot/or does not want to see this problem. I recommend to connect the project admin team directly, i have their gmail from the old version of raterhub:ads_eval@google.com

  10. I’ve emailed EWOQ now [ads_eval@google.com], for two reasons: because of this timesheet issue, and because I’m not getting into EWOQ to work, and an automated message has been directed to my email where they say we should not contact them in case of issues in the timesheet, as I show below in the excerpt of this email:

    Contact your contract administrator when you need an individualized response, or a specific action taken on your account.
    As a contract worker, your contract administrator manages and is responsible for all work, timesheet, and billing issues. Contact them if you have questions about:
    timesheets
    billing
    time off/leaves of absence
    any other contract-related question.
    So, I believe that our only option be connect the project admin team directly through this email: homeworkers.bal@lionbridge.com

    1. The reason I recommend to send email directly to project admin is to make them aware of the massive issue.

      1. Update on the issue: many raters who indicated discrepancies in Lionbridge emails with request to cut hours, got all their invoices approved! Lionbridge acknowledged that the issue was on their side.

  11. Hello Dear Allen,
    I failed the Appen Aztec Ads Assessment Project Task, something got wrong and trouble about QR code etc. and appen regret me to inform this project.

    Do you know that Appen give a second chance me for retaking exam?

    sincerely

  12. Hello,

    First of all, I would like to congratulate and thank you for your work with tips and information on projects and crowdsourcing companies; His training is good, but in practice, a lot of things go dark. So I would very much like you to help me to clear some doubts.

    1 – Recently, aiming to level up I started to do the tasks of the simulator. At first I was not worried about the number of errors – and I was wrong because the simulators and quizzes are based on situations, places, and American research, which makes it difficult to understand who is from another country – And I noticed that parallel to this I started to receive feedbacks, in the beginning within expectations, but then I was below expectations.
    The question is: Does the quality review and feedback they send us by email are based on simulators or the real tasks of the Rather Hub ??
    1.1 – Can I not do the tasks of the simulators to avoid being poorly evaluated?

    2 – Are Invoices divided into 30 days of work or month?
    2.1 – The conversion of the currency is made by the quotation of which day?

    3 – How is the calculation of the number of hours that Rater works? Is the total number of hours worked and divided by the number of days added, or is the number of exact hours worked in the day observed?

    4 – Should I note the time worked manually or is there an allowed app that facilitates this count?

    Thank you very much

    1. Hello,

      Answers to your question you will find here:https://passleapforceexam.com/2016/01/10/common-mistakesof-new-raters/

      New raters do live tasks immediately after getting access to the task portal. It is highly advised that you hold off on these if you do not want to be restricted for quality in the next month. Typically tasks which are available for new raters are quality tasks on which raters are accessed for their quality. Instead, new agents should focus on doing simulation tasks and self-education. Time spent doing this will really pay off, and you wont be restricted for quality in the future. Also, by doing a certain number of simulation tasks correctly you will upgrade your status in the system, and will be allowed to work more hours.

      During your participation in the project your service will be evaluated on quality by means of quality review tasks.In the beginning of each month, a certain number of tasks are uploaded into the system. These tasks are quality performance tasks up on which your quality is measured each month,in the end of the month you will get your quality score feedback.

      If your ratings often falls out of general range of ratings provided by another raters in the project, most likely your account will be flagged by the system automatically and your work will be checked for quality manually.

  13. Hi Tim,

    Thanks for your blog, we are very helpful to those of us who work on these projects. I am working for Appen in the Aztec Ads project and it has only been a month and they are already telling me that I do not reach the RPH average, I have asked them to guide me regarding this metric indicating an approximate RPH and they do not tell me, they are limited to To say that when I get experience I will improve my RPH. I have been told that they will pay me all the time, but I am afraid that they will give me a leave of absence from the project. Could this be grounds for dismissal?

    Thanks for your help,

    1. Hello,

      RPH is Appen metric the aim of which is to pay less to its raters and to take the difference as profit (they invoice Google actual time worked by rater), additionally I have information from Google project administrator regarding this metric, in fact there is no official time metric for most of tasks, but in general the average time spent on each tasks should be about 2 minutes.
      I confirm that those raters who do not meet internal Appen RPH will be removed from the project. That is the reason why I recommend to apply for Ads Rating project at Lionbridge and avoid Appen.

  14. Hi Tim,
    I have a question. When you say, time metric for most of tasks are about 2mins, do you see Ad Creative + Landing Page evaluation as Two independant two tasks or one task consist of two pages? Clarification is greatly appreciated, Thank you!

    1. Hello,
      For this type of tasks (Search Ads Evalution), customer has an average estimated time which is 2 minutes to evaluate both Ad Creative and Landing Page.

Leave a Reply