Get all the information you need to find and apply for these legitimate work at home jobs, and get all the resources you need for becoming a successful rater
Author: Tim Allen
Nowadays, many people are looking for jobs that they can do at home. There is a lot of pro's to doing a work-at-home job. You get to spend more time with your family, you get to work your own hours, and you get freedom from the 9-5 grind. But finding jobs that you can work from home isn't easy. I had a job that I didn't like much at all. I ended up quitting and started my search for something that I could do from home. After a lot of research and checking out a bunch of things online, I ran across Leapforce. They were offering a Search Engine Evaluator position. To me it sounded interesting, and after doing some research I found that its a 100% legit opportunity that pays well. The great thing about it is that you can work any hours you want. Part time or full time. Since then, I've been working at Leapforce for several years now. I've seen a lot of advertisements for work-at-home jobs, and I've heard from a lot of people who would like to have this opportunity. However, this is not an easy job to obtain. Even if you have some experience in internet things like SEO, it can be very difficult to pass the test. I decided after some time that I would like to share my experience and help other people. Wishing you great success! P. S. On this blog I do not disclose any specific info regarding exam tasks, only general tips and recommendations related to the process in general rather than to specific exam tasks. All information on my blog is based on my personal experience, you will not find exam answers on my blog. If you have your own personal experience in working at legitimate work-at-home online project (s), you may share your experience on my blog or send it to me via firstname.lastname@example.org
Appen Global usually has a lot of new and different projects, and a new project has recently come out called Megataxon which is an ad review job. This is an interesting project because most of the Falcon projects usually are rating social network stuff like actual videos from pages or the actual content on Facebook or Instagram. This project focuses on the types of ads that you would see on Facebook. The objective on the project is to decide if the core purpose of the ad is presenting a product, service, or advertising content such as getting you to go to a Facebook page and subscribing to that pages feed. Once you understand what the ad is about, you categorize the ad by using a tool that is specially designed by and for Facebook.
Along with the tool, you get the project's guidelines that go over all the categories, and it does a very good job of explaining the categories and how you should categorize them.
As previously mentioned, the object of the job is to categorize ads as you see them, and you are asked to spend no more than 60 seconds on each ad. This is likely due to the sheer volume of ads that are currently cycling on Facebook. However, there is no minimum view time to apply the tags. With that said, if you are on this project it might take some time to get used to deciding what ad should be categorized where, but after time it is likely that you will pick up speed. It should be noted, however, that if you rush through the ads and categorize them wrong, you could be taken off the project. Once again, they want you to spend no more than 1 minute on each ad with no minimum view time, but you also shouldn’t spend more than a few minutes on each ad. Doing this will surely get the attention of a project manager asking you if there is a problem.
For each ad there are basically 3 steps to putting each one in the proper category. For the first step, you have to choose the core purpose of the ad and decide if it presents a product, a service, or content.
There are times when you can automatically skip the ad if you run into errors, the ad won’t load, the ad is not in the target language for your region, for example the ad is not in English, or there are other issues where you immediately see that the ad is illegal, offensive, or needs to be immediately escalated such as in a case of child exploitation, child nudity, hate speech, acts of terrorism, etc. If you see this bad stuff you have the ability to escalate the Job ID to a manager who handles escalations. If you have to skip the ad for any of these reasons, you just indicate the correct option at the bottom of the Facebook tool.
After you select the purpose of the ad, the next step is to choose a top-level category for the ad. Some of the top level categories have some sub-categories and you will see these appear below the top level categories if there are sub-categories available. This is an extra categorization for some top-level categories. If you select a top level and see a sub-level category appear, you simply choose the best one that fits.
Defining the purpose of the ad simply asks you to choose whether the ad is a Product, a Service, or Content which is defined as the sharing of information that doesn’t provide a specific physical product, service that the reader can buy. Content includes all digital content, articles about an experience with a product, photos of products with no specific call to action, or if the ad uses the ‘Learn More’ button which more than likely leads to content.
Choosing the purpose of the ad should become second nature and you should simply ask yourself ‘What is the core purpose of this ad?’ Ads might reference many different products, services, or content, but your goal is to consider and categorize what is most important for the reader.
The tool itself isn’t difficult and is fairly straightforward. Under the ad, you simply choose the appropriate categories by clicking the radio buttons. Each one of the radio buttons opens with different categories or sub-categories. After you have selected the categories, then on the bottom under that is the submit button. You want to be sure that you have chosen the most correct categories for each ad. Also at the bottom next to the “Submit’ button are the other buttons that we previously mentioned. Those are that the ad has some error, the ad won’t load in the tool, or the ad is not in the English language. You will also see the ‘Other Issue’ button here where you would report an ad that contains something that it shouldn’t that may be offensive or illegal. In this case, the ad would go to one of the managers of the project where its escalated and they take action to remove the ad altogether. You should understand that you have to be careful using this and that the ad should really be offensive and not just something you disagree with or don’t like.
One of the things that is not mentioned in the guidelines is that you might see the ad appear more than once during the week or the month. You should try to remember what you have categorized this ad as and not change the results on the next day if you see the ad again. It’s important that Appen collects precise data. An example of this could be categorizing an ad as something different one day, as opposed to the day before. This will surely get the attention of a product manager. Try to fully understand the categories and email your project manager if you have a question. This will ensure you are able to stay on the project if there is something you don’t understand. Admittedly, there are a lot of products and services on Facebook and categorizing them all correctly might be confusing at first.
The guidelines are really concise and fully detailed and go over the majority of products and services in order to help you categorize them. The guidelines are also very helpful when it comes to deciding what is content or not, and discusses what should fall into those categories.
In conclusion, the Megataxon Falcon project looks to be a very easy job. It might be interesting for some people who want to make a difference in what ads are seen on Facebook, however, after 4 hours of doing this job, it might become tedious on a day to day basis. But for those who don’t mind doing this kind of job, it’s definitely not something incredibly vague or difficult such as some search engine rating jobs.
If you are working on this project, we would like to hear from you and your experience with it. And if you have any questions, you can email me or leave a comment and I will reply as quickly as possible.
As Payoneer is now the only way to get paid by Appen, and if you do not have a Payoneer account yet I invite you to apply for Payoneer account via referral link:
After you have passed either the Lionbridge Ads or the Arrow ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a high chance that you will still see some English tasks come through. In fact, 70% of all tasks will be in English language.
RECORDING AND INVOICING WORKING TIME Appen Arrow and Lionbridge Ads Assessor project
Time management is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Times sheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.
One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.
Appen Arrow blind test and Lionbridge Ads Assessor quality review tasks
Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. In total during a month working cycle there are will be about 150 blind test tasks of different types of tasks, raters are expected to obtain at least 70% of correct answers in the pool of blind test task, raters with lower quality score will be removed from the program.
Reasons for losing your service as anAds Quality Rater
It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.
Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.
Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.
IP address conflict/Internet Network sharing
In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.
Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.
No tasks are available, all task options are not available anymore except the QR code
This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.
Lionbridge (Ads rating project) vs Appen (Arrow)
In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.
Appen has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen(Leapforce) has again invented their own metrics to pay less to its raters.
Working hours and flexibility
At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.
At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.
Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.
Appen(Leapfroce) pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.
In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.
The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of a monthly practice test and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.
It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.
The Final Word
If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too.
Appen Arrow invoice correction required due to low RHP
what is mystical RHP and is this practice legal?
In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.
Appen has productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters.
In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RPH metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer (Google) understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen invented their own metrics to pay less to its raters and make more money by invoicing actual time spent on doing task by all raters to Google. Good business, in fact. In this respect Lionbridge plays a fair game and most of raters get their time sheets approved without any issue.
Our records show a significant discrepancy between the hours you’ve written in your invoice and the amount of work completed. You over-billed more than xx hours for the week of x/x/201X – X/X/201X.
Please check our Chat feature, review our project information page and Intl FAQ. Do not write your missed hours in your invoice time card as the entry will be considered billable.
In the meantime, these hours will not be approved until clarification has been received.
Please let us know if there’s ever an issue outside your control that is impacting your overall productivity such as tool errors.
The productivity rate/ rate per hour (RPH) is calculated by looking at the average of team who worked on a particular task. When you have a low RPH it negatively impacts the RPH average of the entire team and project. It is our expectation that raters meet the average rate per hour on a consistent basis. Listed below are some tips to try if you have not done so already, to hopefully push your rate per hour to the average:
• Time Management: Refrain from multi-tasking while working on EWOQ. Watching videos or listening to music not associated to a task may impact your focus and performance. • Timing Devices:
In order to get a better feel for how fast or slow you are working, you might consider using a desktop timer, Toggl, or stopwatch. Always exit EWOQ when taking a break, no matter how short. Do not count the time spent on breaks towards your billable hours.
A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.
What raters can do in this situation?
Before making any invoice correction, ask Appen for detailed clarification about RHP: what is the RPH ratio for all raters; what is your own RHP ratio, what is recommended RPH ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks, ask why customer does not indicate recommended time for completion for many types of tasks. Be ready that they will not provide any info, because there is no official RHP approved by customer (Google).
Point out that fact there is neither any policy regarding RHP can be found on Appen connect project platform nor customer (Google) does not provide any recommended time for most of tasks.
Notify Google about the issue by sending email to the ads rating project admin (Google Ads Eval: email@example.com) to make them aware that this is some kind of issue that really needs to be investigated. The message should be the following: actual time spent on doing live tasks was not approved due to RHP, what is the RHP ratio for all raters; what is your own RHP ratio, what is recommended RHP ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks, ask why Google does not indicate recommended time for completion for many types of task. FINALLY ask them – is it legal that Appen cuts hours based on mystical RHP which is not trasparant?
Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.
If you have questions or comments then leave them below.
A lot of interesting things have happened over the last five years when you take a look at the three big companies that offer work at home opportunities with jobs in social media evaluation. Those big three companies are of course Appen Global, Leapforce, and Lionbridge.
To get a good idea of the geographical presence of these three companies, you need to look no further than Google trends as far as interest over time along with the search different search terms.
The confusing part of how and when Raterlabs came into play leaving many people wondering if they were the same company as Leapforce. The website looked the same except the logo was changed, and this time when someone went to apply for the search engine rater job, you were redirected to the Raterlabs website. This also had people wondering if the two companies were the same or not.At one point, one of the clients for RaterLabs decided to only work with companies that hire raters as employees and not independent contractors. It’s not known who this is exactly, but considering what happened with the legal situation with Leapforce and Google, it was quite obviously the search engine giant Google. In order for LeapForce to keep their client they created the company RaterLabs and hired people as actual employees, but this is only for the USA.
This brings up two things. You might remember that Leapforce was recently acquired by Appen Global and that there was a situation in where there was confusion on whether independent contractors were considered full-time employees of Google if they were simply independent contractors. One of the biggest mysteries for raters at Leapforce is who they actually work for. After all, paychecks come from Leapforce, but their work all seems to take place via Google’s Raterhub service. Despite all this stuff going on, the press release that Appen was going to acquire Leapforce came out on November 29th, 2017.
From the Google trends charts, you can see a steady rise in interest for Appen Global while the other two companies Leapforce and Lionbridge took a sharp decline. Leapforce as a company still exists, but all hiring is done by Appen Global now since the acquisition. The main reason for this is because it’s easier to simply acquire a company and use their already hired staff. In this way Appen is able to completely control the company and while at the same time, grow much bigger. And while it sounds like company growth would be a good thing for everyone involved, it’s not. As Appen becomes bigger, it can hire more people and pay them less money, which is what is currently happening. The pay rate at Appen has decreased for raters, and because they have so many people internally from their ‘crowdsourcing’ pool, it’s harder to apply and get a job for this position. By the way, Lionbridge also has changed the way they do business in USA (Google rating project) and also made it is raters as part time employees. With this in mind, we wonder why, Appen continues hiring raters as independent contractors for the same project (Inca also known as Yukon) for less pay at $9.00 per hour as opposed to $12.50 per hour at Raterlabs simply because they haven’t had any legal cases so far and they tend to hire people in states with minimum payments rates at the $9.00 per hour rating. This comes even as most states in the USA have a minimum pay rate of $10 per hour and some states even signed a law to increase the min. wage to $15 an hour. It’s almost a guarantee that if you live in these states, you will not likely be hired when they can pay cheaper wages in another state.
Update:Project Inca at Appen, Part-Time team members
On July 1, 2018, independent contractors will move from Appen to RaterLabs. RaterLabs is a separate legal entity owned and operated by Appen. The contractors will become part-time employees amd most likely their hourly rate will be increased. This decision most likely is forced by the customer (Google) and US legislation.
However, it is worthwhile to mention that Leapforce had its own problems in the past when it came to pay as they claimed you would be making a certain amount of money per hour, but that wasn’t accurate. You were actually paid for the time it should have taken you to do the work rather than how long it actually took you to do the work. In other words, they calculate how long they think each task will take you, and pay you for each task no matter how long it took. Work wasn’t consistent enough to make the hourly pay rate either. And the biggest thing, of course, is that training that takes a long time and you’re not paid for it.
This is something that Appen hasn’t changed either. They’ve basically adopted the system which Leapforce uses, but instead, they pay far less than Leaforce did or even as Lionbridge pay. Speaking of Lionbridge, we’ve been talking about Appen and Leapforce. But what about Lionbridge?
If you look at the 5-year chart on Google trends and compare the three companies together, you can see that Lionbridge has just barely managed to stay above the interest level of Leapforce which of course is old news now because of Appen. At the same time, you can see that interest skyrocketed for Appen in January right after it acquired Leapforce. It was the big story at the time, and people that didn’t know about Appen finally found out about them and the work at home jobs they offered.
However, it might be worth mentioning that Lionbridge is probably still the best place to work for when it comes to being a search engine rater. They are known to have a higher pay rate that Appen offers the same job. There are however a couple things that seem to be problems still and that is the 20-24 hourly cap they have for each week, and the fact that there appear to be full-time jobs but part-time employees have no idea how to get into these jobs. With that said, these have been issues for quite some time. Otherwise, Lionbridge is more stable and more rater oriented than Appen which offers a ton of different jobs and doesn’t focus on search engine raters. That is just what you would logically experience from such a big company whose main goal is to increase their client list while utilizing their crowdsourcing at low pay rates.
If you were looking at these companies from a market share perspective, anyone knows that Appen is taking a huge share of the market when it comes to data gathering, translation services, and human-annotated datasets for machine learning and artificial intelligence. But Appen seems to be getting too big for its own good and the bottom line is that the people who work for this company are the ones who miss out on the benefits. Appen seems to also be getting a lot of negative reviews lately and even some people are calling them a ‘scam’ company. Leapforce, since it was acquired, will likely end up being broken down and dismantled and ‘enveloped’ if you will, as more time passes. By the way process has already started as we got confirmation from lots of markets regarding decrease in payment rates in Yukon project at Leapforce. As you may remember we predicted this when Appen acquired Leapforce.
The future for Lionbridge lies in its ability to separate itself by doing things that Appen doesn’t do. If this company is going to survive, they should make a few changes, such as hiring full time employees with benefits, pay for the long training that one needs to go through as a search engine evaluator, and make the pay competitive enough so that people will want to work there instead of the big competition, who doesn’t really have much to offer in the terms of well-payed employment.
We just might get lucky and see Lionbridge make these changes in the future.
Have you had experience with Appen, Leapforce, or Lionbridge? Let me know in the comments and as always, if you have a question, let me know and I’ll answer as soon as possible.
Appen Arrow and Lionbridge Ads rating projects overview
The Arrow Ads project and Lionbridge Ads rating are similar search engine evaluation project that is using the same Rater Hub platform that the other two large companies Appen and Lionbridge are using to rate search engine results depending on a query or intent of the user. Basically, you are doing the same thing as other search engine evaluators, but the difference is that you are judging the quality of ads and advertising-related data as they are calling it. Feedback that you provide is supposed to help the client monitor and improve their advertising as well as functionality.
So what you are doing with this certain project is taking a look at different ads on web pages, and using the Rater Hub tool, you answer some questions on your findings. It sounds easy but answering these questions, and the answers that you provide are coming from guidelines that need to be closely followed. It’s not simply about providing your own feedback, but providing the feedback based on the guidelines themselves and specific metrics.
Appen Arrow and Ads Evaluation program purpose
As a human evaluator, you are not approving or disapproving the ads you evaluate, and you are not deciding directly which ads get shown or don’t get shown. In fact, even though you are providing scores and feedback about individual advertisements and advertisers, your evaluations don’t generally have any direct effect on those advertisements or advertisers. Sometimes the ads you rate are from a long time ago. Sometimes they are in a different form than they originally appeared. Sometimes they were never shown by Google at all, in any form. Basically what you are doing is collecting data to help train the algorithms that will eventually assess the ads being shown on their own. Without human interaction this would not be possible. So this job is very important to the engineers who are collecting the data from you.
This is the main reason why it’s very difficult to pass the assessment part of this project. Your judgments are used to determine whether ads are getting better or worse over time, to see whether particular kinds of ads are working well or not, to see what kinds of ads cause problems, and to see what kind of impact a particular change to our systems will have. When Google engineers design new systems or propose changes to existing systems, the human evaluation data you provide is used to judge whether those proposals succeed or fail. This data is fed to computers that train the automatic algorithms.
This is also a big reason that a lot of raters end up being terminated or losing their contracts. Accuracy and quality of your ratings are very important. You should always check the Ads Guidelines if you aren’t sure about something that you are looking at. Even some raters that have been doing this job for a while still refer to the guidelines.
Appen Arrow and Lionbridge Ads rating program requirements
Getting on this project can be difficult. There is a long assessment required. Its mentioned early on that you will perform all work and including the assessment tasks on a non-shared and non-public computer. That’s pretty straight forward. You’ve got to have your own computer with a high speed internet connection. And according to them you can’t take your laptop to the coffee shop and work from there. So it’s a very good possibility that your home IP address is recorded in the RaterHub, and you need to use this same connection to perform all work. Anything else is strictly prohibited.
Another thing to mention is that you’re going to need to use either Firefox or Chrome web browser but that should hinder anyone since those seem to be the main web browsers being used by the masses. This is because the Rater Hub tool will not work on other browsers.
If you do pass the assessment, then you may be offered a contract position. Usually the contract hours are part-time 20 hours a week, but Aztec states 19 hours, Monday through Sunday. Once you complete the assessment, the results are usually sent within 48 business hours after your deadline to complete the assessment has passed, not 48 hours after you submit the assessment.
As with all projects we can’t stress enough the important of one thing. Thoroughly read the guide they send you and have it open and ready while you are taking the tests. Make sure the minute you get the link to review the guide, that going over it is the first thing you do. Give yourself enough time to digest everything in it at least 1-2 times before taking the tests. The guide itself has to be accessed directly through EWOQ which is part of the RaterHub portal. You’ve got to be logged in using the Gmail address you used when you signed up at Appen/Lionbridge. This Gmail account should be a dedicated email account.
Since you are not able to download the guide, you should bookmark the page with the guidelines so that you can access it quickly and easily. We said that it cannot be mentioned enough and it’s very important that you read and understand the guidelines completely before starting or you will fail the test.
Appen Arrow and Lionbridge Ads program payment rates
Wages on each contractual hour depend on the country and can vary between $4 to $15 per hour. For more information on wages read here:
Agents will invoice vendors for rendered services once per calendar month. Fees to independent agents are paid (via direct deposit) within 30 days of receipt of a valid invoice. The payment is made on monthly basis via direct deposit into your bank account. Appen projects are paid now by Payoneer. If you do not have a Payoneer account yet you can get yours here:
The first set of tasks are multiple choice questions (in total 45 questions) that come straight from the Search Ad instructions in the Ads Guide itself. After you have completed the comprehension questions, you will be shown a set of assessment tasks (in total 190 tasks of different types: search ads evaluation, user interest, key words, etc) that asks you to rate just the first section of a Search Ads task. Once this is done you’ll get a set that asks you to rate just the second section. Finally, you’ll get a set of complete Search Ads tasks, User to Keyword, Close Match Evaluation to rate.
The typical exam tasks is a query and an ad creative generated by a search engine in respect to the user query. You as a rater will need to apply rating rule to determine how much promising or unpromising is the ad in respect to user intent. Once you evaluate the ad then you will be asked to evaluate a landing page associated with the ad and to determine how satisfactory is the page and website content in terms of meeting the user intent needs.
Additionally I would like to give general tips on how to proceed with Aztec/Ads Rating Project qualification:
As usual before proceeding with rating you need to make sure you clearly understand user intent. If not, use Google search to research the query and establish the dominant intent, and the secondary intent if one exists. Sometimes it’s hard to understand what the user intent is without doing the search yourself;
Search Ads Evaluation Tasks
Select appropriate query flag if required. For instance, if the query is in foreign language, or it is clearly a porn related query, make sure you select the appropriate flag. Remember that when you are doing ratings, the language should be in your contracted regional language. So if you are working in English, and you get a page in another language, then flag it appropriately.
Examine the ad creative and answer the question of how promising the ad is. You should refer to a specific section of instructions, and not evaluate the ad creative based on your own perception or personal likes or dislikes. Pay close attention to wording of the ad and the URL. In many cases, wording of the ad may indicate the ad deserves a porn flag, or has reference to another search engine.
Check the query and ad creative for the navigational flag. If the query is looking for a specific page or website and the ad suggests the website, then the ad and the query should be given a navigational flag.
There are a few scenarios in which you should pay special attention to. The first one is when the ad offers to repeat the search in another search engine. In this case, the appropriate rating for ads and LP (Landing Page) are somewhat unpromising, and don’t satisfy the query because the action won’t bring the user closer to his goal. Google wants to provide the user with the quickest possible and satisfying solution for the user intent and query. Sending a user to another search engine defeats that purpose entirely.
Select the ads creative rating, or how promising the ad is in relation to user intent. Once you’ve completed the first part of the task you will then be directed to the Landing Page section.
Visit the landing page and examine the page based on the guideline rules of how satisfactory the page is based on the query or intent. Once again, you should not use your own judgement but instead learn how to apply the guidelines rules. In many cases research on the website is required, for instance a quick product search will show you more information about the page, and help you understand if it meets user intent based on the query.
Make sure to pay special attention to the user location. If you see that the user is looking for something in a specific location, such as a restaurant in a specific location, and you get a restaurant in a completely different location, you should apply a negative rating. In this case it’s a good idea to refer to the guidelines and review typical user location scenarios.
User to Keywords/User Interest Tasks/Close Match Evaluation
Before proceeding to these types of tasks make sure you read and understand instructions, because the logic is different comparing to approach required for search ads evaluation tasks. You will be asked to rater how much is interesting for a user the suggested page/url based on examination of a user current web interest.
Once you have completed all assessment tasks, including the survey, you will temporarily run out of tasks. This is because your answers will be checked before you will be allowed to move forward. Either you will pass this part or not. You will not see the Acquire Tasks button at this time. Remember that you need to wait 48 hours after the assessment deadline and not 48 hours after you submit. The assessment will be complete after you submit all tasks available.
Keep in mind that unlike Yukon project/Personalized Assessor and Maps Analyst (Shasta) projects’ qualifications, the Ads Assessor/Arrow project qualification allows only one attempt to pass the exam. In most cases you will not be allowed to attempt the exam for the second time.
Arrow and Ads Evaluator Expert Support Available
If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.
Appen Arrow vs. Lionbridge Ads rating project
Many people ask me which project is better and which one should I join. In my opinion I think that Lionbridge is definitely better in terms of offering a better payment rate compared to Appen. It’s also important to note that the exam qualification and training are paid by Lionbridge where at Appen it’s not. So take this into consideration when you apply for both of these companies. Three days of unpaid training is a lot for people to deal with and really when you think about it, no one wants to do an exam for three days, and then fail it having put all the time and effort into it to get a job.
As always, if you have any questions or thoughts leave a comment or shoot me an email.
Wishing you great success in your work at home career!
Appen Yukon & Lionbridge Internet assessor Third Party Chrome Extension Ban
If you are a search engine rater for either Appen or Lionbridge, you probably got an email from both of these companies talking about how the use of extensions to track time and your tasks would end up being banned, and they explained their decision in a couple of emails with this one being their last warning to remove them.
“As a follow-up to our communication sent out on March 12th, we wanted to remind you that as of Monday, April 9th, all 3rd party browser extensions must be uninstalled when working on Yukon Search tasks. While 3rd party extensions were developed to aid in rater work, some actually pose security risks and compromise the integrity and the quality of work.”
While it’s not known exactly how and when these security issues were found, or what types of exact security problems were identified, it does put raters back to a problem of keeping track of tasks and time while doing their jobs. It was a lot easier to use some of the extensions that you could add to Chrome such as Rateraid, Greater Acceptor, and LBTimer which were all great extensions for keeping track of your time and tasks.
So now what do you do when it comes to keeping track of your tasks as a Search Engine Rater? Well you could resort to using a pen and paper like they used to do back in the good old days, or you could find some alternatives to these tools that can’t be tracked. And before we get into that, we really have to mention that although we are here to help everyone in any way possible to make their job easier, we are not responsible if you start using some tool that ends up on the banned list and you get caught for it. And for anyone wondering if they can detect these tools being used, they can, and if you get caught using them after the fact, you can lose your job.
Of course we don’t want that to happen to you so we will take a look at some things out there that can help pick up where these tools that were banned left off.
What are alternative and approved way of tracking your time in effective way? Apparently there are some tools out there that can keep the integrity of the browser intact without opening any security holes. And while it’s not fully understood how using these tools compromises the quality of work, it is very possible that the makers of these tools are being used to scrape data as you are using them as time keepers. Anything is possible and the data you are collecting is an invaluable asset. We are not sure if that is the case with these tools, but you might be aware that RaterAide was released in July 2012 and has since become the de facto time tracking tool for raters around the world. It is produced by an independent developer and fellow rater which is probably the reason that both companies decided to ban use of the tool.
But this was not the only tool that was banned. Greater Acceptor is also banned and cannot be used as well as the well-designed LBTimer. All of these tools are designed by third-parties and was likely the main reason behind the decision to ban these from being used in connection with tracking your time and your tasks. If someone really wanted to steal something, it wouldn’t be the time you have spent, or even the tasks that you are working on. However, these tools like any other type of software could be used to track everything else and even spy on your activities and compiling a log of results with your data input. We’re not saying that this has been done, but if you are the companies that offer search evaluation jobs, then you probably should tighten things up a little bit when it comes to data collection. And aside from this, no one knows if these tools were used to exploit or farm data. It’s not likely that either Lionbridge or Leapforce would speak of the specifics about such a topic. And as far as Leapforce is concerned, many of you know that this company was acquired by Appen Global who probably also decided it’s time to tighten things up a little bit when it comes to security.
So now you are a rater and you’re stuck with no tools to use to track tasks and time. Well there are some extensions that you might be able to use and be OK with. Some of these things are tools that were not designed with search engine rating in mind, and work offline. Something that keeps track of your time while being offline is probably key in that the program works as a standalone application and doesn’t need the internet to function.
One of those tools is called Task Timer which can be found here:
This tool allows you to create a list of tasks along with time goals represented in hours. It does keep track of each task that you perform and it has a lot of options so that you can customize it. You can input any amount of tasks and include goals, which could be the time you are allowed on each task which is handy for Yukon or other search engine tasks, and you’re able to start, pause, reset, and edit tasks on the fly. This makes it a perfect replacement for the stuff that these companies have banned. Task Timeris available on Chrome Web Store.
Manual recording of time spent and tasks done in excel file
There are some people who love Excel, and some people who loathe it. However, if you need something right now that is completely standalone whether you have internet or not, and you need to keep track of things, Microsoft Excel can fit the bill. It is also worth mentioning that you can customize Excel beyond your typical applications and even create a template for adding time and tasks.
Auto Page Refresh
As you all probably know, refreshing the page when you are logged into the backend of Lionbridge or Appen to check if new tasks are available is quite a pain. There is a tool that is available that can help with this called Auto Refresh Plus. It is a simple add-on for Chrome that acts as a page refresher which sends notifications to your email about tasks availability/ and new content on the page, such as when tasks show up. Auto Refresh Plus is available on Chrome Web Store.
These are just a few tools that can help you out while Lionbridge and Appen have banned the use of these tools. It could be possible that both companies are coming up with their own solutions for helping track time and the amount of tasks that you do, but for now, this is what we have as Search Engine Raters. Of course the moment that we find something new that will help your job go by a bit easier, we will let you know. As always, if you need help with something pertaining to Search Engine Rating, leave a comment or drop us an email and we reply as soon as possible.
More and more people are looking for freelance jobs these days. Not so long ago a variety of online jobs included just a few options, such as copywriting, customer support, search engine evaluation, and the social media jobs offered by the big companies such as Appen and Lionbridge. Lately, a lot more types of businesses are showing up online needing help from people that own computers or smartphone devices. Now any independent freelancer can choose from a handful of companies that offer jobs like application testing, software testing, and other types of jobs where installing applications and running them for stability which are options to make some extra money. Sometimes an independent contractor can find full-time jobs and become skilled specialists. Some companies come and go and some stand out, being especially interesting and worth everyone’s consideration. With that said, there is a new company that just started out recently called uTest.com.
uTest offers new opportunities to get paid while making a contribution to modern software markets and improving different applications and giving feedback on bugs. The service is dedicated to software and application testing. Professional software testers, developers, and web enthusiasts are able to register on this website and test various apps and writing bug reports that get sent back to the developers. Right after this, they can get offers on their first paid projects. Each one of these offers has some short questionnaire on your equipment and checks to see if you qualify.
The interface of the platform is quite simple, and the payment policy is clear with the ability to get paid by either PayPal or Payoneer. The service also has a friendly community of testers who share their experience with each other. In addition, uTest has its own Academy which helps beginners from all over the world in learning software testing.
To become a freelance software tester, you must be at least 18 years old and have a device such as a PC, tablet, smartphone, and even gaming consoles. uTest works with freelancers from all over the world. No previous testing experience is required. Moreover, on uTest, you can find thousands of free online courses, guides, and webinars to learn more about functional testing and sharpen your skills.
Registration of a new account is fast and simple. You have to fill a form, providing basic information about yourself, such as the full name, location, email address and native languages. After this, you will need to specify the types of devices you own, the models, operating systems of your PC. You can choose multiple devices to expand the range of available software that you are able to test. The service claims that testers are allowed to work on any device that provides a stable internet connection, including smart TVs, gaming consoles, and even fitness trackers. However, currently, mobile devices comprise 70% of all the projects on uTest. That will likely change in the future as the company becomes more well-known and picks up other clients that need software testers.
uTest registration is absolutely free, without any hidden fees.
First of all, you must complete your profile. To do it, select “Expand my Profile” or “Go to Platform” in the navigation menu. The more accurate information you provide; the more projects that you will get matched up with. Thus, we suggest updating your profile anytime you get a new device.
Once you’ve registered an account, you have to prove your qualification skills for certain projects by completing the uTest Sandbox Cycle. You shouldn’t worry about this as it’s all relatively easy stuff. This is a mixed testing cycle that includes elements of exploratory testing as well as a test case. Usually, you get your invitation to the Sandbox within two weeks after the account is registered. But in our experience the moment we registered we started getting invitations to join certain projects. You may also contact community managers to request an invitation. We received several invitations for different jobs that had different requirements such as age or device specific projects.
Generally, the number of test cycle invites depends on your statistics. The more bugs you report; the more jobs you get. This is somewhat how Appen Global works. Thus, we suggest participating in more test cycles and collecting more experience in your profile. After the Sandbox, you can check out the uTest Academy section to find more cycles that will give you basic experience. Another way to make your profile more attractive is the uTest 101 Quiz that is focused on your expectations and rules of the service. The completion of the uTest Quiz gives you higher priority in the distribution of test cycles.
As mentioned previously uTest supports Payoneer and PayPal systems. Testers get paid twice a month, on the 15th and the last day of every month. The transfer to a PayPal or Payoneer account may take up to 24 hours, which is a usual time for these payment services. However, PayPal payments once sent are usually immediate.
Keep in mind that if you use a Payoneer account, you won’t be able to link it to your uTest account unless you’ve earned at least $10. In case your Payoneer account is set up to transfer payments directly to your bank card, you’ll need to have at least $50 in uTest to transfer money to the Payoneer account.
Payoneer now has a sign up program that’s a good incentive to join, lets benefit together!
As Payoneer is now the only way to get paid by Appen, and if you do not have a Payoneer account yet I invite you to apply for Payoneer account via my referral link:
U.S. citizens who earn more than $500 a year on the website may also need to complete a W-9 form for taxes. You are always responsible for doing your own taxes as uTest will never take out taxes. In this case, you can find all necessary information in Accounts & Settings > Payments.
All freelancers get paid in USD only, though the website allows them to see their earnings in any local currency.
uTest has its own rating system, rating freelancers based on their activity and the quality of their work. Ratings are calculated every day. Any tester can be classified as Rated, Proven, Bronze, Silver, or Gold. However, you won’t see any badge on your profile unless you have enough activity points. This rating system considers various parameters of your performance. For example, you can be unrated in some types of testing while being rated Gold in functional testing.
Higher ratings allow you to get higher payout rates. For example, Gold testers earn 10% more on every approved report. Silver and Bronze testers receive additional 5% and 2,5%, respectively. These extras are calculated only for a certain testing type. For example, if you’re a Gold Usability tester, you won’t get extra 10% for Security tests.
The rating system takes into account the percentage of approved and rejected reports. All bugs are classified by their importance. Thus, you don’t have to go for a higher approval percentage. Instead, try to find more valuable bugs. Some testers have 98% of reports approved, but their bugs mostly fall into the “Somewhat valuable” category. Their ratings are lower than those of testers who have more reports approved and end up having a higher percentage of exceptionally valuable bugs.
The important thing that often affects ratings negatively is the submission of test reports. If a tester accepts a test cycle without providing at least one report, this will cause a negative impact on his or her rating. It’s better to decline a test cycle from the very beginning than to accept it and get a negative rating.
If a report was rejected but a tester doesn’t agree with the customer on such a decision, he or she can use the Dispute feature built into the website to provide the customer with more information. However, we suggest avoiding unnecessary use of this feature because a rejected dispute will have a more negative effect on your rating than an initially rejected report.
uTest is a convenient and reliable platform for professional software testers as well as for those who want to try something new. The friendly environment and clear payment policy guarantee the best experience for freelance testers of any skill level. In addition, you can register here for free and join the uTest Academy to learn more about testing and to try this service. If you are a uTest member, let us know what your experience has been in the comments, and as always, leave any questions you might have in the comments and we’ll answer them.
Appen offers more than just rating projects for Google. Currently there is a project called Nile where, as a rater, you can review ads based on your likes, your interests, or your web viewing habits.
You can change these settings in Facebook to adjust what ads you get ‘served’ when doing the ad ratings. The ads are not based on companies that you have already manually liked, but ads that are similar. When starting the project, you get 20 ads to rate. These can be anything from video ads, to just plain text ads or ads with a photo.
Before you are accepted to the Nile project, you are asked for your Facebook profile URL. You must have at least 50 friends on this Facebook profile. Appen simply checks this and nothing else about your profile. You will hear back from Appen within a week if you are accepted into the program. In order to stay in the Nile program, you must work no less than 5 hours per week.
When rating you are asked ‘How much do you feel that this ad is ‘relevant, useful, entertaining, if the ad is offensive, or misleading. You can rate each one of these on a scale from 1-3 using radio buttons.
You are then asked ‘How much do you want to see this in your news feed’ and are shown a scale from 1-5 from ‘I definitely do not want to see this in my feed’ to ‘I definitely want to see this in my feed. Once you have done this, there is an explanation that asks you ‘In your own words, why do you feel this way about the ad.
It is not known if the comments are actually looked at in the ads or not. You don’t need to create a huge explanation. A simple reason why you do or do not like the ad is good enough. When you are finished rating the 20 different ads, you are then asked to wait for one hour to complete what is called a ‘Bakeoff’ task. The Bakeoff task determines which ads you liked and which ads you didn’t. It asks you to choose out of 6 ads in a side-by-side comparison, which one you would like to see in your feed over the other. This task should be completed exactly one hour after you’ve rated the ads. Currently you are allowed to only work one hour per day on Nile. The hour that you wait for the Bakeoff tasks is not paid. So you charge 1 hour for the Nile ratings and the Bakeoff task which takes you less than a minute to do since all you are doing is clicking which ad over the other you want to see.
You must work at least 1 hour per day to Nile is an interesting project and will probably go full time, but currently there is just one hour allowed per day. That’s an extra 30 plus hours per month and totally worth the time. It also pays the same hourly wage as the Google rating program.
Appen Nile Project Qualification
Appen has introduced qualification exam for Nile project. The Nile project exam consists of 30 questions designed in a way to assess your understanding of basic concepts of how to evaluate ads and search results. Again before taking this exam you will be given an access to specific guidelines which you need to study carefully before proceeding with the exam itself. Do not underestimate the complexity of the Nile qualification exam; some of the questions are tricky and confusing in its wording, so without preparation and thoughtful approach you have a very low chance to pass this exam. One attempt of taking this exam is allowed. Once the exam is passed (you will get result immediately after completing the test), you will be offered to sign a contract.
If you are working as a Search Engine Rater or Social Media evaluator for any company that doesn’t hire you directly, then you are known as an independent contractor.Regardless of whether you are an independent contractor, a freelancer, or consultant, you will be considered as ‘self-employed’. What this also means is that you are responsible for doing your own taxes. There can be some advantages to this and also some drawbacks.
Independent Contractor Tax Payer status
If you’ve been working for companies like Appen or Lionbridge, you know that you are basically your own boss. It’s very rare that you have to work with other people in these jobs or have much interaction at all. This can be a good thing or a bad thing. Keep in mind that the people you are working for are not your supervisors. You’re a contract worker which simply means that you provide your services in exchange for a set wage or price. You are largely in control of when, how, and where the work is done. And you can do the work when you want to as long as it gets done.
Another advantage of being an independent contractor is that you will make more money than if you were simply an employee. The reason for this is because companies are willing to pay more for an independent contractor because there aren’t any expensive long term commitments and the company doesn’t have to pay extra money for employment benefits.
The other great thing about being an independent contractor and working on your own is that you get to deduct more business expenses than a typical employee can claim. Think about everything you need to do your job, your computer, your desk, your chair, even the internet connection you are paying for can all be deducted as an expense at the end of the year. This basically means free office equipment.
One of the first things that you will want to do is come up with a plan to save enough money to pay your taxes at the end of the year. Working as a freelancer means that no one is doing your taxes. Your pay is given to you all in one lump sum with no taxes taken out. It is important to understand the basic tax laws. If you are not sure what to do, it is advised to talk to a tax specialist in order to help you save enough tax out of your paycheck.
cons of working as an independent contractor
One of the big things is that contractors aren’t eligible for any type of health or unemployment benefits because you are basically self-employed. So if you need health insurance, you would have to find something on your own that you could pay per month. Now you see why being an employee has its benefits when it comes to taking care of taxes and health insurance. You can still do all this stuff on your own, and it’s not impossible to get your own health insurance, but all that stuff takes time to keep track of.
It should be noted that if you are a US Yukon project rater then you’re a part time employee and your taxes are done for you. There still are other projects that are done on contract basis.
Knowing your taxes and how much you need to deduct is the most important part. As long as you keep enough money to cover taxes then you won’t have to pay anything. The biggest problem here is not withholding the right amount or not enough taxes. A single person in the USA will pay more taxes than a married person will. So it’s important to know exactly what you should be deducting from your pay.
For the USA you must take out self-employment taxes which are Social Security and Medicare contributions.
things that can help you with tax issue
There are some tax calculators online that can be used to accurately deduct the correct amount of money in taxes, including the self-employment taxes. That money should be kept in a separate bank account always so that if anything happens with the main bank account, that money is safe. So create a savings account and put that money into the account and don’t touch it until the end of the year when it’s time to do taxes. It is important to remember that it’s better to save too much than not enough.
When it comes time to do your taxes, it might be good to hire someone that can put together everything for you including your business deductions such as your office equipment or computer that you use to do your work on. There are some limits of course on what you can deduct. Find out exactly what you should have paid and make out a check and send it to the tax government in which you live. In the USA it’s the Internal Revenue Service.
As an independent contractor you won’t pay less or more taxes than if you were an employee. The only difference is that you have to do all the paperwork yourself. However, you are able to take more physical deductions than an employee would because you provide yourself with all the tools you need to do your job on your own.
In closing, it shouldn’t scare you to do your own taxes. It’s not any more difficult than doing your taxes as if you were an employee, and you might actually get the tools you need to do your job for free.
This article goes over the guidelines of the project at Appen called Typeahead e2e. This project is somewhat the same as being a search engine rater and rating queries and results, but you are rating queries for Facebook and their website search engine. They may be trying to get this search engine as tuned as Google, except it’s based on their platform. Facebook is considered a family website, and they are trying to make sure that none of the bad stuff gets through. The main purpose of the job you perform is to detect low quality suggestions, and the ratings tasks are mostly based on your own point of view, which is unusual for a project of this type.
Appen Typeahead e2e – requirements and payment rates
E2E stands for ‘end to end’ but that seems irrelevant at this point. In order to work on this project, you must have a Facebook account. The project has a work volume from 1 to 4 hours per day. This is the usual amount of time on Appen projects. The pay range for this project depends on where you live, but it can range from $6 an hour to $14 an hour. This project is exclusive of Appen and runs in many markets.
Typeahead e2e Qualification
Qualification process is straight-forward, before proceeding with actual exam you will be given a chance to do a practice test which consists of 20 tasks and covers all possible rating scenarios. Once you complete the practice set you will receive feedback and recommendations on your ratings. The next step will be actual exam which also consists of 20 tasks. If you pass this exam successfully you will be moved to production.
Typeahead e2e working process
Typeahead are queries that are given with suggestions when you type something in the search box on Facebook.
The search Typeahead provides suggestions, and it’s your job to see if the suggestions are relevant, or if suggestions are bad. There are a lot of different things that can make them bad. You might recall that Google at one point had a big problem many years back when Google ads were coming up with some violent and hateful content that caused them to lose some advertising with big names. It wasn’t until this happened that Google wanted a real human to look at what ads were being displayed, and what websites were providing as far as search content. The same thing is happening at Facebook and they want to make sure that people are not getting bad results with violent, offensive, hateful, or pornographic content. That is where the human rater comes in, and it’s your job to determine if the results are good or bad, or if they should be flagged.
As mentioned previously, the workflow of this project is to determine the quality of suggestions in the search bar drop down. There are some things that the guidelines go over.
You are presented with a list of categories when doing the search. You make a decision based on what the results are. You would follow these labels and determine what is acceptable and what is offensive/porny (cute new word that Appen or Facebook apparently made up), junky, or acceptable.
It is important to remember that you are rating the quality of the suggestion, and not the quality of the query. Quality doesn’t have anything to do with the query. The query should be looked at to determine if it’s in a foreign language other than the language you are contracted to work in.
Furthermore, a suggestion should be rated 1 if it is offensive, violent or porny, such as “hot girls”, “sexy videos”, etc. The suggestion, at first glance, needs to be obviously offensive or seeking offensive material.
You need to remember that you are not rating the results generated by the Facebook search engine but the suggestion itself. Many common names may generate pornographic results (ex.“mary ann”) but should not be rated as offensive. If it’s junky, then it is because the suggestion is unrecognizable and, most importantly, you think the suggestion is junky/gibberish. Junky entails the suggestion is difficult to understand or is entirely nonsense. If you rate the suggestion as junky, you would then choose a category to support your decision.
A suggestion gets a 3 – Acceptable if it is not junky. This is considered a good suggestion which doesn’t contain foreign language or anything offensive. It is spelled correctly and free of grammatical errors. If a suggestion is rated acceptable, then it must receive an additional rating to indicate whether it is personally recognizable to you. This is a weird task, because you wonder how it can be acceptable if it’s not recognizable. They clear that up by saying ‘The suggestion is UNRECOGNIZABLE based on your own point of view but still makes sense’. They say that you can use Google/Bing to determine if the suggestion is acceptable or junky. But once again, the rating is based on your own personal view.
The job itself doesn’t look very hard, and the work flow should go pretty fast since you only have to choose from 4 different categories, and then choose a label from one of those 4 categories. There are a lot of examples in the guidelines that show you what to do in certain cases in case you are unsure of what to select.
It is interesting to know what this will end up doing to the Facebook pages that rely on certain words that are brought up. Some searches may end up disappearing from the suggestions at all.
As always, if you have any questions about this project, or you are working on this project, leave us a comment and tell us how it’s going.