Common Issues of New Raters (Appen Aztec and Lionbridge Ads Rating Projects)

Support_636x350-600x330Workflow 

After you have passed either the Lionbridge Ads or the Aztec ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a chance that you will still see some English tasks come through once in a while.  

Time keeping and quality 

Time keeping is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Timesheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.  

One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.  

What are the quality review tasks? 

Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. 

Reasons for losing your service as an Ads Quality Rater 

It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.  

Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.  

Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.  

IP address conflict/Network sharing

In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.

Multi-accounts

Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.

No tasks are available, all task options are not available anymore except the QR code

This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.

Lionbridge (Ads rating project) vs Appen (Aztec)

Productivity 

In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.

Appen has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH matric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen has again invented their own metrics to pay less to its raters.

Working hours and flexibility

At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.

At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.

Payments

Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.

Appen pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.

Payoneer As a Payment Method for Appen, Raterlabs, Leapforce and Lionbridge

Payments rates

In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.

More info about payment rates is available here:

Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Leapforce, Lionbridge and Appen When Combining Several Projects?

General Support

The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of monthly practice sets and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.

Optional Agreements

It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.

The Final Word 

If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too. 

Advertisements

5 thoughts on “Common Issues of New Raters (Appen Aztec and Lionbridge Ads Rating Projects)

  1. Hello Tim

    After two months into working with LB, my account has been blocked and I got an email explains that my quality has been poor(around sixty). So I went on to check my feedback in the portal. There are 6 tasks that were reviewed and given feedback to.

    However, about half of the feedback are wrong. I think the auditors failed to check accuracy of the news, clicking further into the links. So I sent an email explaining the possible mistakes of them and indicate which tasks that I can agree with. After one week that my account has been blocked, I got another email instructs me to complete additional learning path but nothing related to my questions.

    Is this audit final and not disputable? Should I continuing asking for explanation or just accept this?
    What if this quality assessment stands, how much more errors can I make?

    NB. I have been always met the required hours, but I missed about 1-2 hours from maximum the week before the blockage.

    Please help. Thanks you.

  2. I have been working on ads rating project for a year, never had a task shortage before but since last and I’m facing a task shortage Tuesday. However I still got the 3 options marked on my EOWQ. Is anyone experiencing the same issue with shortage of tasks? I am from USA.

    1. Hello,

      As far as I know all markets are experiencing the same issue (shortage of tasks in Ads rating/Aztec projects).
      I have an assumption that it has to do with updating raterhub tool. Vendors do not provide any info regarding this issue, the only response I got from them is to check raterhub for tasks on a regular basis. I hope the volume of tasks will increase in the near future.

  3. Hello,

    I got such email from Appen:

    Our records show a significant discrepancy between the hours you’ve submitted and estimated hours based on RPH as shown below.

    Week

    Estimated Hours Based on RPH

    Total Rater Hours Submitted

    04/02 – 04/08

    19

    29

    When you have a low RPH it negatively impacts the RPH average of the entire roster and project. It is the project requirement raters meet the average rate per hour on a consistent basis. The average rate per hour varies according to each task and should be made available to you in the judging interface or guidelines. In the meantime, these hours will not be approved until clarification has been received. Please amend your hours if needed.

    *************************************************************************************************************************

    Why do they send this email and how do they calculate RHP? I had lost of time consuming tasks (like video evaluation) and I am sure that I entered correct and reasonable time…moreover there is no any indication for the recommended time for many tasks…Do you think Appen cuts my hours to pay me less on purpose?

Leave a Reply