If you would like to obtain a job as a tester, you are probably wondering what sorts of skills does it take to become one. I would definitely recommend starting with some basic tutorial on “how to test”, you can find those on official uTest Youtube channel, and there are couple of good books for beginners: “Software Testing” by Ron Patton; “Agile Testing: A Practical Guide for Testers and Agile Teams” by Lisa Crispin and many others. But to become a good tester these kind of skills you need to have for starters:
– Great English skills (written);
– Analytical Thinking;
– Attention to Detail;
– Thinking Outside the Box;
There are no restrictions on amount of work, but it really depends on the platform and your rating as a tester. There are projects that I get invited daily. The project really depends on the hardware you have. Mostly there it’s a web or app based projects. For instance – some hotel website, or booking service. There is so much to test, starting with a web forms, ending with validation of every input field. They are easy for beginners to start finding bugs right away. I worked from US, and got what the platform offered me for the region, but I do know that project can be different depending on country you are working from.
Types of tasks
Usually is’s a whole functionality testing of some elements of the site. For instance: the customer wants a tester to be precise in Log-in page. So the tester focuses on those bugs which occur only on a login page, not considering the other parts of the website. Sometimes there are localization task. Let say you know Chinese, and customer requires the testers to check the spelling and grammatical errors of specific language.Most of the tasks are ad-hoc (exploratory testing) for functional issues. (App crashes, buttons not working, etc).
How to become a successful rater at uTest and make decent money
But there is no deep inside mentioning of a rating system and how it works. Let me explain why is it so important on uTest. This will be a more “in depth” article on earning on uTest. There is this system where it takes your results (amount of submitted bugs) and amount of rejected bugs and counts the successful percentage rate.
Based on this rate there are classifications of testers: Rated; Proven; Bronze; Silver; Gold
And there are different Testing Types, for each of which there is a rating: Functional; Usability; Load; Security; Localization; Automation
But don’t worry, most of the tests are functional and Usability, and polishing skills on these Testing Types can set you with a good additional income for each month.Getting more in those ratings, as I mentioned earlier it is the key filter for uTest to invite you to the project. To become a Gold rated testers it takes quite some time and dedication. Not to mention the quality of your bug reports should be on very high level.
Once you get a couple of rejections it is really hard to achieve the Gold rating. The highest rating gets you not only the more amount of offers, but you also get more paid, and those rare and exclusive projects available only for best testers out there.
Apart from this rating there is also a very cool and interesting opportunity on becoming a “Favorite tester”. Once you have basic rating of Silver or Gold you might end up becoming the one.
Let me explain simply how it works. There is certain person which manages a test cycle a TTL or Testing Team Lead. This person is responsible for other testers to follow the rules and code of conduct of Utest platform, and the project in particular. Sometimes there are even NDA’s which are needed to be signed. TTL is responsible for all this and more.
Let’s assume you have submitted 10 bugs in 1 test cycle, and all of those issues are either very valuable or exceptionally valuable bugs for the customer, Testing Team Lead can easily forward your account to the customer, so that they can give you a “Favorite Tester” status.
What does it mean? Well, you get to be automatically invited to each testing cycle this customer has and many more. Some customers always slot a test case for a favorite tester, which leads to more earnings.
I’ll go into more detail with the favorite tester part and test cases. Why is its important?
How much really you can make of uTest?
Once you become a favorite tester, usually it’s a big project with a big company, and the pay is really good. Some test cases can be up to 50$ per/case.
Once you start doing those cases, simultaneously it is easy and very practical to look for the bugs, (the inconsistencies between an actual and expected results). It really pops in the eye, once you get going through a test case.In that moment when you have found an issue and reported it to the system, apart from getting those 50$ for the test case itself, you can earn a lot more for reporting bugs additionally.
Talking about rates paid for bugs filled… Well, it absolutely depends on the person, the amount of details he/she is giving, the amount of devices the person holds, and the overall work ethics a freelancer sticks up to.
Bugs may vary from 3$ for “Somewhat valuable” to 30-40$ (per bug) for “Exceptionally valuable” issue, depending on the customer.
For sure you are willing to know the real numbers, and the sacramental question needs to be answered.“How much really you can make of Utest”? I will repeat myself stating that everything depends from the person. But, you can crank out a fair 1000-1500$ per/month of income depending on your dedication.
Everything is done via PayPal and Payoneer. Sometimes the bugs are not approved by customer for as long as 35 days. After that period, they are approved automatically by the system.
Use this link for applying for signing up to Payoneer and we will both benefit by earning 25$, once your account is loaded with payments.
There are many factors on becoming successful on this platform. Speed is one of them.
Sometimes it’s critical to start testing immediately, as other (hungry) testers will beat you to that critical issues, while you’re yawning.
The same rule is applied for the test cases.
Test Cases are usually limited, and not everyone can claim them, so it’s lethal to be fast.
To make the numbers I mentioned earlier, it really takes time to pump your account to a certain rating level. Usually it will take around month or two to get going, but hey, Rome wasn’t built in a day either.Once you become an experienced tester and acquainted with a platform like a “Pro”, you actually might get invited easily on becoming a Testing Team Lead.
Who are Testing Team Leads?
The roles of this person are really dependent of the customer you are working for.
My roles were:
Writing test cases;
Participating in meetings with other TTL’s, Project Managers and Software Delivery Managers;
Create and control a test cycle;
Check and forward bugs for the customer (yes, TTL has the power to approve or decline bugs).
The income story is completely different, as the TTL starts working on per hour rate, and once again it really depends from the customer on the rate. (It is near to a very entry level junior position in software testing working for an onsite company in the States).
The benefit of becoming TTL on one project is that very rapidly you become noticed by others, and easily can get invited to manage some other cycle.In addition, TTL’s are still allowed to do the functional testing as other participants, though some customers forbid this.
In conclusion, now that I completely moved to an onsite job I can say with a share amount of confidence – Utest is a great place to start. It takes that will power, but sure does pay off. Good luck fellow freelancers!
Bing Search Engine Rating Project (Side by Side Judge, Multimedia Judge) vs Google projects (Yukon/Internet Safety/Internet Assessor)
When getting into jobs as a search engine evaluator, you will find that one company will produce ratings for one search engine, such as Google, while another company will offer jobs as a rater in Bing which is Microsoft’s Search Engine. Like Google, Microsoft also needs research and pages rated just like Google.
As far as the job is concerned, it is basically the same job except you are rating results in different search engines that turn up different results. The pay generally varies depending what company you work for with some reporting earnings in the $12 to $15 dollar per hour range.
Google projects (Yukon/Internet Safety/Internet Assessor)
Some of these jobs are changing and they are being called different things such as ‘internet analyst’ which puts a twist on the tasks you are given. Previously when working as a search engine rater, you were given results that were based on generic search results. However, as time went by, Google started asking raters for permission to use their personal data. Basically, if you were surfing for stereo’s or smartphones, and you started doing this job, you’d get back results based on your surfing habits as linked to your Gmail address that you use.
The job that Raterlabs offers is a Google Search Engine and Google Playstore project where the raters are now called ‘Internet Analyst’ where the tasks you get are based solely on your personal Google use. It’s a part-time position that currently looks to be for the USA region only, but this does change from time to time. The requirements and skills to get this job are nothing that most people don’t already have if they are good at searching or surfing the internet. With this job they want raters to possess a computer with high speed internet, and also an Android phone with version 4.1 or higher, or an iPhone version 4 or higher. The job most likely involves doing ratings on both the computer and the smartphone. Sometimes tasks are available on only one device or the other. So it’s good to have them both because you will likely get smartphone tasks. It is also noted that in the description that they want raters who use Google Play often. There is a possibility that tasks involving Google Play will be available. This job’s testing and qualification is no different from the previous exams for search engine related jobs. There are three parts to the exam leading us to believe it’s the same exact material for the regular rater. The job is basically the same after all, the only difference is now its personalized to you. It is not known what the pay is for this job as that information is confidential and you sign a non-disclosure agreement about not discussing your pay. But it’s likely in the $12 dollar per hour range if its Raterlabs.
Appen Global also offers this type of position even though they also own Leapforce and likely are using them as a source of experience and also taking advantage of the workers who were already employed by Leapforce along with their experience. Appen calls their position Web Search Evaluator which likely has the same test. It is unknown if Appen supplies raters for both Google and Bing but it’s possible that they do both. All the requirements and testing are the same for Appen since they are providing raters for Googles Search Engine. But it is rumored that the pay for this position is much lower than you would get if you worked at Lionbridge or Raterlabs. This is because Appen is a much larger company with a much larger talent pool to take people from. This allows them to pay much lower wages. So in this case it’s actually better to get these types of jobs at other companies like Leapforce/Raterlabs, and Lionbridge. It’s interesting to hear that Appen is doing some interviews from Skype now.
Speaking of Lionbridge, we are getting to them next. This company seems to have a good reputation for paying people better than the other companies for the same type of job. They also offer opportunities for the Search Engine Rater but they likely offer rating jobs for both Google and Bing. Microsoft does use Lionbridge to hire human search evaluators, but Microsoft calls you ‘judges’ instead of raters. Lionbridge was known to pay raters $14 an hour for these positions although that wage has probably fluctuated over time.
Miscrosoft’s HRS project (Multimedia Judge, Side by Side Judge)
You’re probably wondering what the difference is in rating for Google or the Bing search engine. Bing has a project called Human Relevance System or HRS and it’s the system that they use to have human raters judge search results with. The HRS project is similar to the Quality Rater program that Google employs. However, it’s interesting to know that Microsoft’s version has been around since 2004 since MSN Search began generating its own results.
Not much has been written about Miscrosoft’s HRS project, but leaked copies of the guidelines are floating around that go over the process of rating or ‘judging’ websites. The difference between Googles rating guidelines and the HRS is that HRS uses what is called a rating matrix. This matrix combines the searcher intent with how well the returned document satisfies the searchers intent.
You can see the five rating options that judges can use from perfect to bad. This is the absolute heart of how you would rate returned search results. This might actually be a little easier to understand for some raters by judging just the intent and how well the intent is satisfied.
But there is one other aspect of the HRS system and the guidelines go over it in a pretty detailed manner. This is called Freshness and basically you will look at the content to see how up to date it is. Some information can be relevant but be a couple years old when some things have changed and the information isn’t updated or fresh enough to contain those changes. The Freshness system is based on three categories; Fresh Not Important, Very Likely Fresh, and Most Likely Fresh. The Fresh Not Important will be for things where the facts will never change no matter how fresh the article or the page is, while the other two will be based on the articles ability to receive fresh content.
When it comes to the difference between Google and Bing as far as their rating systems, they aren’t really all that different from each other. But if you will be working as a search engine rater, and you are looking for stable tasks, you might want to try and get Google search rating jobs since there is more volume when it comes to this search engine over Bing. Still, both jobs are decent, but it really depends on who you are employed by. Appen seems to be one of those companies that will pay people as little money as possible to make more money themselves. This was their whole point of buying Leaforce; to squeeze out the competition. So this company will pay less. Lionbridge or Raterlabs will be the places to try and get these jobs.
When it comes down to Bing, it’s not a very big project in terms of the number of raters who are hired for this as compared to the raters that are hired for Google. That shouldn’t be a big surprise because we all know that Google has a much higher search volume and a lot more users. The judging platform for HRS is a bit outdated, not convenient to use, and it only works on Internet Explorer of course. Also, the test has a part in where you have to judge the search results for relevance of around 100 items and they give you very little time to do this. Many candidates often fail the test and are unable to pass the exam.
iSoftstone is another vendor and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done in the HRS platform and you have to complete a qualification for each task. There is a guideline for each one of the tasks, but you don’t need to worry because everything is explained in videos and emails. Tasks are categorized as hits or pages and for each hit or webpage that is shown you get 0.01 penny to 0.20 cents US. If you do tasks/hits quickly, then you can make easily $10-$15 dollars US per hour, and some people even more. You might be tempted to go through some of the tasks as quickly as possible because the more you complete, the more money you obviously make. However, sometimes during these tasks you are sometimes tested so you have to be very careful. Too many mistakes and they will remove you from the project and you won’t receive any more hits. ISoftstone is a bit easier to work for in that they don’t have a huge amount of work compared to Appen/Raterlabs.
Basically, what this whole thing boils down to is the individual. Some people might like working for Bing because the workload is lower, but you’re going to make less money because of this. If you need more stable work and higher than average work, then working on a project that involves Google will probably be a better alternative.
Do you work for any of these companies? We would like it if you shared your experience here in the comments. And as always, if you have any questions, let me know by email or in the comments and I will answer them as soon as possible.
Many people fail the Yukon exam at their first attempt because they don’t fully understand Google algorithms’ way of thinking, and you’ve only got a week to study the 164 page guide. Many people fail the first time because this is not an easy exam. Appen and Lionbridge are aware of this fact, and even mention in the welcome email that many people fail the test. In one week after an unsuccessful attempt, you will get the second and last chance to re-take the exam. You will have to go through all parts of the exam again, practical tasks in part 2 (Page Quality) and part 3 (Send to Device) will be different from those you had on your first exam, although there may be a few tasks that come from previous exam.
Why do candidates fail the Appen Yukon and Lionbridge qualification exams?
Lack of effort and sufficient preparation
The most common reason that people fail the Yukon project exam, is that some candidates think that they may get lucky and pass the exam without proper studying, or you may think you know enough about the internet or have good search skills. Be aware that all parts of exam are designed in a way to assess your ability to understand and apply rating concepts and rules based on a 164 page guide. Its not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort within the one week they give you to study for the exam. If you don’t at least study the guide, or go over it a couple times, you will most likely fail the theoretical part (part 1) of the exam.
Poor time management
One of the common reasons for unsuccessful attempt to pass the exam is poor time management. People do exam preparation and taking all 3 parts within 1-2 days. This strategy in many cases leads to exam failure, because the exam requires lots of concentration and attention to details…Thus plan your exam carefully so you have enough time for studying the material and taking the exam without rushing. Basically, don’t wait till the last minute.
Inappropriate understanding and/or implementation of rating rules and concepts
Have you failed the Appen or Lionbridge exams after carefully studying for them? Don’t worry, a lot of people have. Even people with experience in Search Engine Optimization, or even experience in taking exams have failed this test. In fact it is the first thing that Appen mentions before you get the exam. That it is very difficult and many fail. If you have failed, it is most likely you misunderstood the rating concepts and/or applied them in a wrong way; e.g. used personal judgment/common sense, or did not pay attention to details where it was required (e.g. user location for local queries). In this case I recommend you get assistance from an expert, such as myself
Yukon exam and Lionbridge Internet Assessor tutoring and assistance
If you are not confident about going through the Appen or Lionbridge qualification exam , you may always get my expert assistance, support and advice. For detailed information please write to firstname.lastname@example.org
Note: I am not giving out Leapforce and Lionbridge exam answers, so please do not ask. The reasons for this is because part 2 (Page Quality) and Part 3 (Send to Device) tasks are unique practical tasks, tasks in those parts are updated on a regular basis. The best way is to understand what you are doing and get training on the subject before and after you pass the exam. If you are not sure you understand rating concepts well and and you are not sure how to rate exam tasks correctly, then that is when its best to get my assistance which will give you confidence and lead to success in passing the exam.
Welcome to Yukon/Internet Assessor/Internet Safety Evaluator/Rater Exam Simulation Tool
As you may know Appen acquired Leapforce in 2017, since then the company has decided to close it is own project management platform known as Appen Global and move all projects to Appen connect. The group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform.
Appen usually has a lot of new and different projects, and a new project has recently come out called Paradise (previously known as Megataxon) which is an Facebook ad review job. This is an interesting project because most of the Falcon Facebook projects usually are rating social network stuff like actual videos from pages or the actual content on Facebook or Instagram. This project focuses on the types of ads that you would see on Facebook. The objective on the project is to decide if the core purpose of the ad is presenting a product, service, or advertising content such as getting you to go to a Facebookpage and subscribing to that pages feed.
Once you understand what the ad is about, you categorize the ad by using a tool that is specially designed by and for Facebook.For each ad there are basically 3 steps to putting each one in the proper category. For the first step, you have to choose the core purpose of the ad and decide if it presents a product, a service, or content.
Along with the tool, you get the project's guidelines that go over all the categories, and it does a very good job of explaining the categories and how you should categorize them.As previously mentioned, the object of the job is to categorize ads as you see them, and you are asked to spend no more than 60 seconds on each ad. This is likely due to the sheer volume of ads that are currently cycling on Facebook. However, there is no minimum view time to apply the tags. With that said, if you are on this project it might take some time to get used to deciding what ad should be categorized where, but after time it is likely that you will pick up speed. It should be noted, however, that if you rush through the ads and categorize them wrong, you could be taken off the project. Once again, they want you to spend no more than 1 minute on each ad with no minimum view time, but you also shouldn’t spend more than a few minutes on each ad. Doing this will surely get the attention of a project manager asking you if there is a problem.
There are times when you can automatically skip the ad if you run into errors, the ad won’t load, the ad is not in the target language for your region, for example the ad is not in English, or there are other issues where you immediately see that the ad is illegal, offensive, or needs to be immediately escalated such as in a case of child exploitation, child nudity, hate speech, acts of terrorism, etc. If you see this bad stuff you have the ability to escalate the Job ID to a manager who handles escalations. If you have to skip the ad for any of these reasons, you just indicate the correct option at the bottom of the Facebook tool.
After you select the purpose of the ad, the next step is to choose a top-level category for the ad. Some of the top level categories have some sub-categories and you will see these appear below the top level categories if there are sub-categories available. This is an extra categorization for some top-level categories. If you select a top level and see a sub-level category appear, you simply choose the best one that fits.
Defining the purpose of the ad simply asks you to choose whether the ad is a Product, a Service, or Content which is defined as the sharing of information that doesn’t provide a specific physical product, service that the reader can buy. Content includes all digital content, articles about an experience with a product, photos of products with no specific call to action, or if the ad uses the ‘Learn More’ button which more than likely leads to content.
Choosing the purpose of the ad should become second nature and you should simply ask yourself ‘What is the core purpose of this ad?’ Ads might reference many different products, services, or content, but your goal is to consider and categorize what is most important for the reader.
The tool itself isn’t difficult and is fairly straightforward. Under the ad, you simply choose the appropriate categories by clicking the radio buttons. Each one of the radio buttons opens with different categories or sub-categories. After you have selected the categories, then on the bottom under that is the submit button. You want to be sure that you have chosen the most correct categories for each ad. Also at the bottom next to the “Submit’ button are the other buttons that we previously mentioned. Those are that the ad has some error, the ad won’t load in the tool, or the ad is not in the English language. You will also see the ‘Other Issue’ button here where you would report an ad that contains something that it shouldn’t that may be offensive or illegal. In this case, the ad would go to one of the managers of the project where its escalated and they take action to remove the ad altogether. You should understand that you have to be careful using this and that the ad should really be offensive and not just something you disagree with or don’t like.
One of the things that is not mentioned in the guidelines is that you might see the ad appear more than once during the week or the month. You should try to remember what you have categorized this ad as and not change the results on the next day if you see the ad again. It’s important that Appen collects precise data. An example of this could be categorizing an ad as something different one day, as opposed to the day before. This will surely get the attention of a product manager. Try to fully understand the categories and email your project manager if you have a question. This will ensure you are able to stay on the project if there is something you don’t understand. Admittedly, there are a lot of products and services on Facebook and categorizing them all correctly might be confusing at first.
The guidelines are really concise and fully detailed and go over the majority of products and services in order to help you categorize them. The guidelines are also very helpful when it comes to deciding what is content or not, and discusses what should fall into those categories.
In conclusion, the Megataxon Falcon project looks to be a very easy job. It might be interesting for some people who want to make a difference in what ads are seen on Facebook, however, after 4 hours of doing this job, it might become tedious on a day to day basis. But for those who don’t mind doing this kind of job, it’s definitely not something incredibly vague or difficult such as some search engine rating jobs.
If you are working on this project, we would like to hear from you and your experience with it. And if you have any questions, you can email me or leave a comment and I will reply as quickly as possible.
As Payoneer is now the only way to get paid by Appen, and if you do not have a Payoneer account yet I invite you to apply for Payoneer account via referral link:
After you have passed either the Lionbridge Ads or the Arrow ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a high chance that you will still see some English tasks come through. In fact, 70% of all tasks will be in English language.
RECORDING AND INVOICING WORKING TIME Appen Arrow and Lionbridge Ads Assessor project
Time management is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Times sheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.
One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.
Appen Arrow blind test and Lionbridge Ads Assessor quality review tasks
Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. In total during a month working cycle there are will be about 150 blind test tasks of different types of tasks, raters are expected to obtain at least 70% of correct answers in the pool of blind test task, raters with lower quality score will be removed from the program.
Reasons for losing your service as anAds Quality Rater
It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.
Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.
Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.
IP address conflict/Internet Network sharing
In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.
Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.
No tasks are available, all task options are not available anymore except the QR code
This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.
Lionbridge (Ads rating project) vs Appen (Arrow)
In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.
Appen has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen(Leapforce) has again invented their own metrics to pay less to its raters.
Working hours and flexibility
At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.
At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.
Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.
Appen(Leapfroce) pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.
In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.
The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of a monthly practice test and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.
It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.
The Final Word
If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too.
Appen Arrow invoice correction required due to low RHP
what is mystical RHP and is this practice legal?
In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.
Appen has productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters.
In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RPH metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer (Google) understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen invented their own metrics to pay less to its raters and make more money by invoicing actual time spent on doing task by all raters to Google. Good business, in fact. In this respect Lionbridge plays a fair game and most of raters get their time sheets approved without any issue.
Our records show a significant discrepancy between the hours you’ve written in your invoice and the amount of work completed. You over-billed more than xx hours for the week of x/x/201X – X/X/201X.
Please check our Chat feature, review our project information page and Intl FAQ. Do not write your missed hours in your invoice time card as the entry will be considered billable.
In the meantime, these hours will not be approved until clarification has been received.
Please let us know if there’s ever an issue outside your control that is impacting your overall productivity such as tool errors.
The productivity rate/ rate per hour (RPH) is calculated by looking at the average of team who worked on a particular task. When you have a low RPH it negatively impacts the RPH average of the entire team and project. It is our expectation that raters meet the average rate per hour on a consistent basis. Listed below are some tips to try if you have not done so already, to hopefully push your rate per hour to the average:
• Time Management: Refrain from multi-tasking while working on EWOQ. Watching videos or listening to music not associated to a task may impact your focus and performance. • Timing Devices:
In order to get a better feel for how fast or slow you are working, you might consider using a desktop timer, Toggl, or stopwatch. Always exit EWOQ when taking a break, no matter how short. Do not count the time spent on breaks towards your billable hours.
A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.
What raters can do in this situation?
Before making any invoice correction, ask Appen for detailed clarification about RHP: what is the RPH ratio for all raters; what is your own RHP ratio, what is recommended RPH ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks, ask why customer does not indicate recommended time for completion for many types of tasks. Be ready that they will not provide any info, because there is no official RHP approved by customer (Google).
Point out that fact there is neither any policy regarding RHP can be found on Appen connect project platform nor customer (Google) does not provide any recommended time for most of tasks.
Notify Google about the issue by sending email to the ads rating project admin (Google Ads Eval: email@example.com) to make them aware that this is some kind of issue that really needs to be investigated. The message should be the following: actual time spent on doing live tasks was not approved due to RHP, what is the RHP ratio for all raters; what is your own RHP ratio, what is recommended RHP ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks, ask why Google does not indicate recommended time for completion for many types of task. FINALLY ask them – is it legal that Appen cuts hours based on mystical RHP which is not trasparant?
Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.
If you have questions or comments then leave them below.
A lot of interesting things have happened over the last five years when you take a look at the three big companies that offer work at home opportunities with jobs in social media evaluation. Those big three companies are of course Appen Global, Leapforce, and Lionbridge.
To get a good idea of the geographical presence of these three companies, you need to look no further than Google trends as far as interest over time along with the search different search terms.
The confusing part of how and when Raterlabs came into play leaving many people wondering if they were the same company as Leapforce. The website looked the same except the logo was changed, and this time when someone went to apply for the search engine rater job, you were redirected to the Raterlabs website. This also had people wondering if the two companies were the same or not.At one point, one of the clients for RaterLabs decided to only work with companies that hire raters as employees and not independent contractors. It’s not known who this is exactly, but considering what happened with the legal situation with Leapforce and Google, it was quite obviously the search engine giant Google. In order for LeapForce to keep their client they created the company RaterLabs and hired people as actual employees, but this is only for the USA.
This brings up two things. You might remember that Leapforce was recently acquired by Appen Global and that there was a situation in where there was confusion on whether independent contractors were considered full-time employees of Google if they were simply independent contractors. One of the biggest mysteries for raters at Leapforce is who they actually work for. After all, paychecks come from Leapforce, but their work all seems to take place via Google’s Raterhub service. Despite all this stuff going on, the press release that Appen was going to acquire Leapforce came out on November 29th, 2017.
From the Google trends charts, you can see a steady rise in interest for Appen Global while the other two companies Leapforce and Lionbridge took a sharp decline. Leapforce as a company still exists, but all hiring is done by Appen Global now since the acquisition. The main reason for this is because it’s easier to simply acquire a company and use their already hired staff. In this way Appen is able to completely control the company and while at the same time, grow much bigger. And while it sounds like company growth would be a good thing for everyone involved, it’s not. As Appen becomes bigger, it can hire more people and pay them less money, which is what is currently happening. The pay rate at Appen has decreased for raters, and because they have so many people internally from their ‘crowdsourcing’ pool, it’s harder to apply and get a job for this position. By the way, Lionbridge also has changed the way they do business in USA (Google rating project) and also made it is raters as part time employees. With this in mind, we wonder why, Appen continues hiring raters as independent contractors for the same project (Inca also known as Yukon) for less pay at $9.00 per hour as opposed to $12.50 per hour at Raterlabs simply because they haven’t had any legal cases so far and they tend to hire people in states with minimum payments rates at the $9.00 per hour rating. This comes even as most states in the USA have a minimum pay rate of $10 per hour and some states even signed a law to increase the min. wage to $15 an hour. It’s almost a guarantee that if you live in these states, you will not likely be hired when they can pay cheaper wages in another state.
Update:Project Inca at Appen, Part-Time team members
On July 1, 2018, independent contractors will move from Appen to RaterLabs. RaterLabs is a separate legal entity owned and operated by Appen. The contractors will become part-time employees amd most likely their hourly rate will be increased. This decision most likely is forced by the customer (Google) and US legislation.
However, it is worthwhile to mention that Leapforce had its own problems in the past when it came to pay as they claimed you would be making a certain amount of money per hour, but that wasn’t accurate. You were actually paid for the time it should have taken you to do the work rather than how long it actually took you to do the work. In other words, they calculate how long they think each task will take you, and pay you for each task no matter how long it took. Work wasn’t consistent enough to make the hourly pay rate either. And the biggest thing, of course, is that training that takes a long time and you’re not paid for it. This is something that Appen hasn’t changed either. They’ve basically adopted the system which Leapforce uses, but instead, they pay far less than Leaforce did or even as Lionbridge pay. Speaking of Lionbridge, we’ve been talking about Appen and Leapforce. But what about Lionbridge?
If you look at the 5-year chart on Google trends and compare the three companies together, you can see that Lionbridge has just barely managed to stay above the interest level of Leapforce which of course is old news now because of Appen. At the same time, you can see that interest skyrocketed for Appen in January right after it acquired Leapforce. It was the big story at the time, and people that didn’t know about Appen finally found out about them and the work at home jobs they offered.
However, it might be worth mentioning that Lionbridge is probably still the best place to work for when it comes to being a search engine rater. They are known to have a higher pay rate that Appen offers the same job. There are however a couple things that seem to be problems still and that is the 20-24 hourly cap they have for each week, and the fact that there appear to be full-time jobs but part-time employees have no idea how to get into these jobs. With that said, these have been issues for quite some time. Otherwise, Lionbridge is more stable and more rater oriented than Appen which offers a ton of different jobs and doesn’t focus on search engine raters. That is just what you would logically experience from such a big company whose main goal is to increase their client list while utilizing their crowdsourcing at low pay rates.
If you were looking at these companies from a market share perspective, anyone knows that Appen is taking a huge share of the market when it comes to data gathering, translation services, and human-annotated datasets for machine learning and artificial intelligence. But Appen seems to be getting too big for its own good and the bottom line is that the people who work for this company are the ones who miss out on the benefits. Appen seems to also be getting a lot of negative reviews lately and even some people are calling them a ‘scam’ company. Leapforce, since it was acquired, will likely end up being broken down and dismantled and ‘enveloped’ if you will, as more time passes. By the way process has already started as we got confirmation from lots of markets regarding decrease in payment rates in Yukon project at Leapforce. As you may remember we predicted this when Appen acquired Leapforce.
The future for Lionbridge lies in its ability to separate itself by doing things that Appen doesn’t do. If this company is going to survive, they should make a few changes, such as hiring full time employees with benefits, pay for the long training that one needs to go through as a search engine evaluator, and make the pay competitive enough so that people will want to work there instead of the big competition, who doesn’t really have much to offer in the terms of well-payed employment.
We just might get lucky and see Lionbridge make these changes in the future. Have you had experience with Appen, Leapforce, or Lionbridge? Let me know in the comments and as always, if you have a question, let me know and I’ll answer as soon as possible.
Appen Yukon & Lionbridge Internet assessor Third Party Chrome Extension Ban
If you are a search engine rater for either Appen or Lionbridge, you probably got an email from both of these companies talking about how the use of extensions to track time and your tasks would end up being banned, and they explained their decision in a couple of emails with this one being their last warning to remove them.
“As a follow-up to our communication sent out on March 12th, we wanted to remind you that as of Monday, April 9th, all 3rd party browser extensions must be uninstalled when working on Yukon Search tasks. While 3rd party extensions were developed to aid in rater work, some actually pose security risks and compromise the integrity and the quality of work.”
While it’s not known exactly how and when these security issues were found, or what types of exact security problems were identified, it does put raters back to a problem of keeping track of tasks and time while doing their jobs. It was a lot easier to use some of the extensions that you could add to Chrome such as Rateraid, Greater Acceptor, and LBTimer which were all great extensions for keeping track of your time and tasks.
So now what do you do when it comes to keeping track of your tasks as a Search Engine Rater? Well you could resort to using a pen and paper like they used to do back in the good old days, or you could find some alternatives to these tools that can’t be tracked. And before we get into that, we really have to mention that although we are here to help everyone in any way possible to make their job easier, we are not responsible if you start using some tool that ends up on the banned list and you get caught for it. And for anyone wondering if they can detect these tools being used, they can, and if you get caught using them after the fact, you can lose your job.
Of course we don’t want that to happen to you so we will take a look at some things out there that can help pick up where these tools that were banned left off.
What are alternative and approved way of tracking your time in effective way? Apparently there are some tools out there that can keep the integrity of the browser intact without opening any security holes. And while it’s not fully understood how using these tools compromises the quality of work, it is very possible that the makers of these tools are being used to scrape data as you are using them as time keepers. Anything is possible and the data you are collecting is an invaluable asset. We are not sure if that is the case with these tools, but you might be aware that RaterAide was released in July 2012 and has since become the de facto time tracking tool for raters around the world. It is produced by an independent developer and fellow rater which is probably the reason that both companies decided to ban use of the tool.
But this was not the only tool that was banned. Greater Acceptor is also banned and cannot be used as well as the well-designed LBTimer. All of these tools are designed by third-parties and was likely the main reason behind the decision to ban these from being used in connection with tracking your time and your tasks. If someone really wanted to steal something, it wouldn’t be the time you have spent, or even the tasks that you are working on. However, these tools like any other type of software could be used to track everything else and even spy on your activities and compiling a log of results with your data input. We’re not saying that this has been done, but if you are the companies that offer search evaluation jobs, then you probably should tighten things up a little bit when it comes to data collection. And aside from this, no one knows if these tools were used to exploit or farm data. It’s not likely that either Lionbridge or Leapforce would speak of the specifics about such a topic. And as far as Leapforce is concerned, many of you know that this company was acquired by Appen Global who probably also decided it’s time to tighten things up a little bit when it comes to security.
So now you are a rater and you’re stuck with no tools to use to track tasks and time. Well there are some extensions that you might be able to use and be OK with. Some of these things are tools that were not designed with search engine rating in mind, and work offline. Something that keeps track of your time while being offline is probably key in that the program works as a standalone application and doesn’t need the internet to function.
One of those tools is called Task Timer which can be found here:
This tool allows you to create a list of tasks along with time goals represented in hours. It does keep track of each task that you perform and it has a lot of options so that you can customize it. You can input any amount of tasks and include goals, which could be the time you are allowed on each task which is handy for Yukon or other search engine tasks, and you’re able to start, pause, reset, and edit tasks on the fly. This makes it a perfect replacement for the stuff that these companies have banned. Task Timeris available on Chrome Web Store.
Manual recording of time spent and tasks done in excel file
There are some people who love Excel, and some people who loathe it. However, if you need something right now that is completely standalone whether you have internet or not, and you need to keep track of things, Microsoft Excel can fit the bill. It is also worth mentioning that you can customize Excel beyond your typical applications and even create a template for adding time and tasks.
Auto Page Refresh
As you all probably know, refreshing the page when you are logged into the backend of Lionbridge or Appen to check if new tasks are available is quite a pain. There is a tool that is available that can help with this called Auto Refresh Plus. It is a simple add-on for Chrome that acts as a page refresher which sends notifications to your email about tasks availability/ and new content on the page, such as when tasks show up. Auto Refresh Plus is available on Chrome Web Store.
These are just a few tools that can help you out while Lionbridge and Appen have banned the use of these tools. It could be possible that both companies are coming up with their own solutions for helping track time and the amount of tasks that you do, but for now, this is what we have as Search Engine Raters. Of course the moment that we find something new that will help your job go by a bit easier, we will let you know. As always, if you need help with something pertaining to Search Engine Rating, leave a comment or drop us an email and we reply as soon as possible.
More and more people are looking for freelance jobs these days. Not so long ago a variety of online jobs included just a few options, such as copywriting, customer support, search engine evaluation, and the social media jobs offered by the big companies such as Appen and Lionbridge. Lately, a lot more types of businesses are showing up online needing help from people that own computers or smartphone devices. Now any independent freelancer can choose from a handful of companies that offer jobs like application testing, software testing, and other types of jobs where installing applications and running them for stability which are options to make some extra money. Sometimes an independent contractor can find full-time jobs and become skilled specialists. Some companies come and go and some stand out, being especially interesting and worth everyone’s consideration. With that said, there is a new company that just started out recently called uTest.com.
uTest offers new opportunities to get paid while making a contribution to modern software markets and improving different applications and giving feedback on bugs. The service is dedicated to software and application testing. Professional software testers, developers, and web enthusiasts are able to register on this website and test various apps and writing bug reports that get sent back to the developers. Right after this, they can get offers on their first paid projects. Each one of these offers has some short questionnaire on your equipment and checks to see if you qualify.
The interface of the platform is quite simple, and the payment policy is clear with the ability to get paid by either PayPal or Payoneer. The service also has a friendly community of testers who share their experience with each other. In addition, uTest has its own Academy which helps beginners from all over the world in learning software testing.
To become a freelance software tester, you must be at least 18 years old and have a device such as a PC, tablet, smartphone, and even gaming consoles. uTest works with freelancers from all over the world. No previous testing experience is required. Moreover, on uTest, you can find thousands of free online courses, guides, and webinars to learn more about functional testing and sharpen your skills.
Registration of a new account is fast and simple. You have to fill a form, providing basic information about yourself, such as the full name, location, email address and native languages. After this, you will need to specify the types of devices you own, the models, operating systems of your PC. You can choose multiple devices to expand the range of available software that you are able to test. The service claims that testers are allowed to work on any device that provides a stable internet connection, including smart TVs, gaming consoles, and even fitness trackers. However, currently, mobile devices comprise 70% of all the projects on uTest. That will likely change in the future as the company becomes more well-known and picks up other clients that need software testers.
uTest registration is absolutely free, without any hidden fees.
First of all, you must complete your profile. To do it, select “Expand my Profile” or “Go to Platform” in the navigation menu. The more accurate information you provide; the more projects that you will get matched up with. Thus, we suggest updating your profile anytime you get a new device.
Once you’ve registered an account, you have to prove your qualification skills for certain projects by completing the uTest Sandbox Cycle. You shouldn’t worry about this as it’s all relatively easy stuff. This is a mixed testing cycle that includes elements of exploratory testing as well as a test case. Usually, you get your invitation to the Sandbox within two weeks after the account is registered. But in our experience the moment we registered we started getting invitations to join certain projects. You may also contact community managers to request an invitation. We received several invitations for different jobs that had different requirements such as age or device specific projects.
Generally, the number of test cycle invites depends on your statistics. The more bugs you report; the more jobs you get. This is somewhat how Appen Global works. Thus, we suggest participating in more test cycles and collecting more experience in your profile. After the Sandbox, you can check out the uTest Academy section to find more cycles that will give you basic experience. Another way to make your profile more attractive is the uTest 101 Quiz that is focused on your expectations and rules of the service. The completion of the uTest Quiz gives you higher priority in the distribution of test cycles.
As mentioned previously uTest supports Payoneer and PayPal systems. Testers get paid twice a month, on the 15th and the last day of every month. The transfer to a PayPal or Payoneer account may take up to 24 hours, which is a usual time for these payment services. However, PayPal payments once sent are usually immediate.
Keep in mind that if you use a Payoneer account, you won’t be able to link it to your uTest account unless you’ve earned at least $10. In case your Payoneer account is set up to transfer payments directly to your bank card, you’ll need to have at least $50 in uTest to transfer money to the Payoneer account.
Payoneer now has a sign up program that’s a good incentive to join, lets benefit together!
As Payoneer is now the only way to get paid by Appen, and if you do not have a Payoneer account yet I invite you to apply for Payoneer account via my referral link:
U.S. citizens who earn more than $500 a year on the website may also need to complete a W-9 form for taxes. You are always responsible for doing your own taxes as uTest will never take out taxes. In this case, you can find all necessary information in Accounts & Settings > Payments.
All freelancers get paid in USD only, though the website allows them to see their earnings in any local currency.
uTest has its own rating system, rating freelancers based on their activity and the quality of their work. Ratings are calculated every day. Any tester can be classified as Rated, Proven, Bronze, Silver, or Gold. However, you won’t see any badge on your profile unless you have enough activity points. This rating system considers various parameters of your performance. For example, you can be unrated in some types of testing while being rated Gold in functional testing.
Higher ratings allow you to get higher payout rates. For example, Gold testers earn 10% more on every approved report. Silver and Bronze testers receive additional 5% and 2,5%, respectively. These extras are calculated only for a certain testing type. For example, if you’re a Gold Usability tester, you won’t get extra 10% for Security tests.
The rating system takes into account the percentage of approved and rejected reports. All bugs are classified by their importance. Thus, you don’t have to go for a higher approval percentage. Instead, try to find more valuable bugs. Some testers have 98% of reports approved, but their bugs mostly fall into the “Somewhat valuable” category. Their ratings are lower than those of testers who have more reports approved and end up having a higher percentage of exceptionally valuable bugs.
The important thing that often affects ratings negatively is the submission of test reports. If a tester accepts a test cycle without providing at least one report, this will cause a negative impact on his or her rating. It’s better to decline a test cycle from the very beginning than to accept it and get a negative rating.
If a report was rejected but a tester doesn’t agree with the customer on such a decision, he or she can use the Dispute feature built into the website to provide the customer with more information. However, we suggest avoiding unnecessary use of this feature because a rejected dispute will have a more negative effect on your rating than an initially rejected report.
uTest is a convenient and reliable platform for professional software testers as well as for those who want to try something new. The friendly environment and clear payment policy guarantee the best experience for freelance testers of any skill level. In addition, you can register here for free and join the uTest Academy to learn more about testing and to try this service. If you are a uTest member, let us know what your experience has been in the comments, and as always, leave any questions you might have in the comments and we’ll answer them.
Appen offers more than just rating projects for Google. Currently there is a project called Nile where, as a rater, you can review ads based on your likes, your interests, or your web viewing habits.
You can change these settings in Facebook to adjust what ads you get ‘served’ when doing the ad ratings. The ads are not based on companies that you have already manually liked, but ads that are similar. When starting the project, you get 20 ads to rate. These can be anything from video ads, to just plain text ads or ads with a photo.
Before you are accepted to the Nile project, you are asked for your Facebook profile URL. You must have at least 50 friends on this Facebook profile. Appen simply checks this and nothing else about your profile. You will hear back from Appen within a week if you are accepted into the program. In order to stay in the Nile program, you must work no less than 5 hours per week.
When rating you are asked ‘How much do you feel that this ad is ‘relevant, useful, entertaining, if the ad is offensive, or misleading. You can rate each one of these on a scale from 1-3 using radio buttons.
You are then asked ‘How much do you want to see this in your news feed’ and are shown a scale from 1-5 from ‘I definitely do not want to see this in my feed’ to ‘I definitely want to see this in my feed. Once you have done this, there is an explanation that asks you ‘In your own words, why do you feel this way about the ad.
It is not known if the comments are actually looked at in the ads or not. You don’t need to create a huge explanation. A simple reason why you do or do not like the ad is good enough. When you are finished rating the 20 different ads, you are then asked to wait for one hour to complete what is called a ‘Bakeoff’ task. The Bakeoff task determines which ads you liked and which ads you didn’t. It asks you to choose out of 6 ads in a side-by-side comparison, which one you would like to see in your feed over the other. This task should be completed exactly one hour after you’ve rated the ads. Currently you are allowed to only work one hour per day on Nile. The hour that you wait for the Bakeoff tasks is not paid. So you charge 1 hour for the Nile ratings and the Bakeoff task which takes you less than a minute to do since all you are doing is clicking which ad over the other you want to see.
You must work at least 1 hour per day to Nile is an interesting project and will probably go full time, but currently there is just one hour allowed per day. That’s an extra 30 plus hours per month and totally worth the time. It also pays the same hourly wage as the Google rating program.
Appen Nile Project Qualification
Appen has introduced qualification exam for Nile project. The Nile project exam consists of 30 questions designed in a way to assess your understanding of basic concepts of how to evaluate ads and search results. Again before taking this exam you will be given an access to specific guidelines which you need to study carefully before proceeding with the exam itself. Do not underestimate the complexity of the Nile qualification exam; some of the questions are tricky and confusing in its wording, so without preparation and thoughtful approach you have a very low chance to pass this exam. One attempt of taking this exam is allowed. Once the exam is passed (you will get result immediately after completing the test), you will be offered to sign a contract.