How do human raters improve online services? There is a lot of information here about the different kinds of jobs that a rater does for companies such as Appen Global, Lionbridge, and Leapforce. And over the past year or so, there are a lot more jobs than just your average rater job. There are even more types of jobs now than before. Everything from recording your voice for voice recognition applications, to social media evaluation. What isn’t generally talked about it the process and what you are actually providing as a service.
What it mostly comes down to is data. In reality, most of these jobs are simply data collection. Facebook, Instagram, and Google all require some type of data collection in order to create programming for their algorithms. Most of you who work for these companies probably don’t realize what it is that you are providing. In fact, your job is a lot more important than you think.
Google was always at the forefront of providing the best user experience for its searchers or users. And many people think that it is just their algorithm that does all the work. But no one really knew until recently when the Google Guidelines were leaked online that humans actually had something to do with it directly. As a rater, you are not actually rating the website, or giving the website you are looking at, a grade. Instead, you are using the guidelines written by Google engineers, to give the algorithm its data.
The data that you return for doing rating is actually fed to the algorithm where it judges and grades websites based on the real world interactions through raters. It is often thought that the job you are doing is actually affecting those pages and ads that you see. In a way, you are affecting them, but not directly. It is only by the data you collect that will affect websites, ads, and services in the long run once the algorithms are programmed using your data.
The same can be said for Facebook and what some of the social media evaluation jobs do. For instance, the Falcon Video Recommendations program had guidelines that were written so that when raters were looking at videos, the algorithm would know what to do concerning video’s and the content of those videos, whether they were helpful, if they should be flagged, and why they didn’t meet any quality expectation. Facebook ads also have an algorithm which is programmed by the engineers from the resulting data collected from raters. Instagram is no different. This company needs real people to rate and check ads so that its algorithm can be programmed. The videos were judged on how relevant and interesting they were, and why the videos were interesting or not. It is the information that was given on what made a good video good, or a bad video bad that was the most important data needed.
This becomes valuable real world data in which future guidelines and algorithms can be programmed. But once again, when the guidelines will change, the data needed will also change.
This is the main reason why you see some projects come and go, and why some are continual projects. Once one project has enough data, the algorithm is programmed and data collection stops until more data is needed or they make some changes in any general guidelines. Anytime changes are made in guidelines, raters or data collectors are once again needed to evaluate the current content being put online by advertisers, website builders, or even people who run Facebook pages and ads.
Some other types of projects that include the collection of data are voice recognition applications where you get a project that asks you to record your voice saying certain commands. Once enough voice data is collected, the product is programmed and tested. Some of these projects can be very interesting and I’ve seen a lot of them being offered by Appen Global as part of the talent pool network.
All these companies have one thing in common. They seek to improve their user experience. They can only improve the experience by having real world testers or raters look at the actual content, and rate the content based on the carefully crafted guidelines. This data definitely improves services and ads for the best user experience possible. Google, Facebook, and Instagram know that it would be impossible to create this experience without its social media and website raters.
There’s something called the Global Search Engine Evolution and many of you are aware of the fact that IT companies like Microsoft, Yahoo and Google use complicated and complex algorithms for their services. In many cases these methods are not effective all the time and sometimes give users wrong search results. For this reason these systems require constant human assessment to rate results for usefulness. Using human input from thousands of raters companies can create better code to search information on the Web or test new services or products before releasing them to public. And this is the main reason why IT companies hire hundreds of human raters from all over the world. Can you imagine what amount of money is circulating only in this very narrow segment?
Google and other IT companies do not hire directly but outsource this business to its vendors like Appen and Lionbridge. This job comes under different names – search engine evaluator, rater, Internet assessor or social media assessor.
As many services of Google are localized for each country of its presence, applying to this position is only possible if person knows the language, culture peculiarity of users of local searching system. Typically position is opened for bilingual candidates, but there are vacancies only for English speakers.
Usually you don’t work the whole day, it is apart-time job, but 2 hours of work per day is guaranteed. The payment is different for each country and varies from 8$ to 17$ per hour. This position requires work to be done at aprivate place or at home.
A lot of companies are actually starting to hire people that allows them to work from home. There are some big companies that hire a lot of people, and keep these people as independent contractors as part of giant talent pools. If you are an independent contractor and you have worked for any of the large companies like Lionbridge or Appen Global, you probably have been hired to do data collection for many different kinds of jobs. These companies have more than one client, and it seems that they are getting more and more clients for different types of jobs as time progresses and the need for human data collection increases. Some of these jobs can last a long time, and some of them can last only a few days depending on what they are, and what type of data is needed. It might be beneficial for our readers to take a look at some of these companies in case you were wondering about their history, where they came from, and which one is probably best to work at if you want to have this type of job. One thing that many people should understand before working for one of these companies is that you are an independent contractor, contracts can range in many different timeframes and pay. Usually the more contracts you successfully complete with a company, the more contracts or work that you will be offered.
Let’s take a closer look at each company, who they are, and what they offer.
Leapforce is probably one of the biggest providers for search engine analysts and the main reason behind this us because its CEO/President is Daren Russel Jackson who used to work for Google and is responsible for the creation of raterhub which was eventually bought by Google and is used at as the main platform by which websites are graded on according to the guidelines and a user met scale. This company was launched in 2008 as a way to help Google gather data that is incredibly important to forming each algorithm that Google releases. Leapforce also has its hand in some of the social media data gathering for companies such as Facebook. But in general, raterhub was not used at the time when the Facebook project ‘Nile’ was introduced to raters as a second project. Facebook used its own tool for this. Currently, you can still get contracts for this company as a Google search engine rater. There are just a few requirements for this position but they are simple; You must be 18 years old, you must be a resident living in the USA, and you must have a social security number or tax ID, and a US bank account that accepts direct deposit. One thing to understand about this company is that they offer a very limited amount of different projects. If you want to be a search engine rater, then it’s not a problem. But don’t expect to have any type of variety in different projects which can be a nice change when it comes to being stuck on a boring and monotonous job. This is considering you are able to pass the very hard 3 part (1-week process) exam based on 160 page Google Guidelines. The pay is pretty fair at $13.75 per hour to start, and there are some reports of being bumped up to $17.00 per hour, but you’ve got to be at the top of the list of raters with nothing less than perfect scores. They pay only once a month and there aren’t any benefits. One thing interesting to note is that the CEO, Daren Jackson just created another company called Raterlabs which could very well be a spinoff of raterhub. It’s too early to tell at this point what the intention of having more than one company based on this type of work is, but the Raterlabs website is using the exact same layout as the Leapforce site and its unfinished. It could be very possible that Raterlabs seeks to venture out in other areas of data collection, perhaps the social media ads evaluation arena like Appen Global does. I think that the expectations for this company’s longevity and its stability should be in question. On the outside, it really doesn’t look like much is happening with Leapforce, and with the CEO creating a clone to the company, something else might be up that we don’t see yet. It probably has to do with the acquisition of raterhub by Google, but only time will tell. If you are looking to be a search engine rater, then this company might be for you, however that work is only currently part time.
Lionbridge is a much bigger company that has been around for quite some time and was founded in 1996. They mostly started out doing translation services and currently they offer translation, marketing and multilingual communications solutions to more than 800 enterprises worldwide. Lionbridge utilizes what is called the ‘crowd-in-the-cloud’ model which enables them to service a wide variety of client needs by adding a high amount of different types of talent to their pool. They claim 6000+ global staff that are controlling 100k professional cloud workers that provide translation, online marketing, global content management and application testing solutions. It can be safe to say that social media and search engine analysis is also part of the services they provide because of the types of jobs they hire for. These jobs are mostly human data collection and of course some of them may require a different type of skilled person from their pool. However, they hire normal everyday people that understand social networking, and if you can follow basic guidelines, and pass a few exams, then you can have a decent part time job analyzing social media advertising. It makes sense to get involved with such a large company as a home based worker, however size can be an issue when it comes to working with such a large talent pool. Communication with so many people and also having to communicate within the company as far as staff can be challenging if there is not enough people to provide support to their independent contractors when needed. However, personal experience has been good. The company is based in Waltham Massachusetts and has the same type of requirements as Appen to become part of their talent pool which is you must be at least 18 years old, have some type of banking for direct deposit, and have a tax ID number. This company is different than Appen in that it is a global provider for client services instead of just being based in the USA and requiring its workers to be based there. Workers can be global in many different countries. Also, this company provides benefits for its workers unlike Appen. It is interesting to note that this company was just acquired by global private equity and alternative assets investment firm, H.I.G. which is definitely a signal that the company is doing well and expanding its services.
ZeroChaos is also a global provider of workforce management solutions, or in other words, provides a talent pool of different skilled individuals able to hand different types of data collection from large companies. ZeroChaos is based in the USA but also has global operations in Europe, and Asia, and was founded in 1999 by John Riley and Mark Lowrey. The company has more than $3 billion in annual revenue, operates in 45 countries, and has more than 100,000 contingent cloud based workers together in its talent pool. This company focuses mostly on providing a reliable workforce for large companies that need to hire a lot of people quickly without having to go through heavy hiring processes. This company focuses more on the client side from their website, and they hire people from other sources such as Flexjobs, or other companies that outsource to find workers online jobs. In fact, when on their website, there isn’t a portal for independent contractors to apply. They seem to have a very vigorous process by how they bring people into their talent force. They screen every person that is employed including background checks. They focus and pride themselves on being able to provide large companies with the workforces that are needed at a moment’s notice. ZeroChaos hires for a wide variety of jobs including social media type jobs for Facebook and Instagram, and also ads quality rater type jobs. But since this company doesn’t seem to focus much on its independent contractors, and it’s unknown how to apply, it might be very difficult in getting a job with this company. ZeroChaos pays $15 per hour for ad-raters for Google, but some independent contractors complain that there is little work and some social media jobs have been put on hold indefinitely making the prospect of working for the company an unstable one.
Update: As of 08.2017Ads Quality Rater position at Zerochaos has been permanently closed due to customer request (Google). This project has been outsourced to Appen and Lionbridge.
Appen Global was founded in Sydney, Australia by linguist Dr. Julie Vonwiller in 1996, who was later joined by her husband who is an engineer to create one of the largest suppliers of speech and search technology services. This company has an interesting history and started doing data collection for different applications such as handwriting recognition for tablet PC’s running Windows XP, and supplied the speech data for groundbreaking speech to speech translation software. In 2007 they became involved with Google and started to provide search relevance data. This was the beginning for much bigger data collection projects, and now the company works with hundreds of different clients. Positions for search engine rater were only the beginning. Now the company offers jobs collecting search or ad relevant information for Facebook and Instagram. Human raters are key for helping engineers of these companies form their complex algorithms. Also there are a lot of speech data collection jobs within this company once you get hired. What they involve is speaking into a special recording application certain phrases that are pre-written, or in some cases they ask you to give your input. Some of these projects only last a couple of hours, but if you start doing a lot of them and completing them, expect to get more invitations. Appen has what is called ‘Crowdsourcing’ where everyone who is hired can be invited to complete certain projects. This crowdsource is estimated by the company to be at 400,000+, and the company boasts experience in translation services in over 180 languages with 20 years of experience. Support for independent contractors can be tricky and sometimes emails can go completely unanswered. It’s not surprising with this size of company to have some internal problems. However, once you are on a project, the support seems good. And, the more projects that you complete, the more invitations to other projects you will receive. The backend is setup so that you can take certain language tests, set your hours of availability, and apply easily for other jobs. They have the same requirements as the other companies as far as being 18, having a tax ID, and a bank account where you can get direct deposit. Some jobs you can get paid by PayPal, but they now work directly with payment processor Payoneer, so it’s easy to get an account setup to get paid. Average pay is $13.75 per hour for most contracts and the company pays on a monthly basis.