Appen Falcon Typeahead Project

Appen Typeahead Project 

This article goes over the guidelines of the project at Appen called Typeahead e2e. This project is somewhat the same as being a search engine rater and rating queries and results, but you are rating queries for Facebook and their website search engine. They may be trying to get this search engine as tuned as Google, except it’s based on their platform. Facebook is considered a family website, and they are trying to make sure that none of the bad stuff gets through. The main purpose of the job you perform is to detect low quality suggestions, and the ratings tasks are mostly based on your own point of view, which is unusual for a project of this type.  

Appen Typeahead e2e – requirements and payment rates 

E2E stands for ‘end to end’ but that seems irrelevant at this point. In order to work on this project, you must have a Facebook account. The project has a work volume from 1 to 4 hours per day. This is the usual amount of time on Appen projects. The pay range for this project depends on where you live, but it can range from $6 an hour to $14 an hour. This project is exclusive of Appen and runs in many markets. 

Typeahead e2e Qualification  

Qualification process is straight-forward, before proceeding with actual exam you will be given a chance to do a practice test which consists of 20 tasks and covers all possible rating scenarios. Once you complete the practice set you will receive feedback and recommendations on your ratings. The next step will be actual exam which also consists of 20 tasks. If you pass this exam successfully you will be moved to production. 

Typeahead e2e working process

Typeahead are queries that are given with suggestions when you type something in the search box on Facebook.  

The search Typeahead provides suggestions, and it’s your job to see if the suggestions are relevant, or if suggestions are bad. There are a lot of different things that can make them bad. You might recall that Google at one point had a big problem many years back when Google ads were coming up with some violent and hateful content that caused them to lose some advertising with big names. It wasn’t until this happened that Google wanted a real human to look at what ads were being displayed, and what websites were providing as far as search content. The same thing is happening at Facebook and they want to make sure that people are not getting bad results with violent, offensive, hateful, or pornographic content. That is where the human rater comes in, and it’s your job to determine if the results are good or bad, or if they should be flagged.  

As mentioned previously, the workflow of this project is to determine the quality of suggestions in the search bar drop down. There are some things that the guidelines go over. 

You are presented with a list of categories when doing the search. You make a decision based on what the results are. You would follow these labels and determine what is acceptable and what is offensive/porny (cute new word that Appen or Facebook apparently made up), junky, or acceptable.  

It is important to remember that you are rating the quality of the suggestion, and not the quality of the query. Quality doesn’t have anything to do with the query. The query should be looked at to determine if it’s in a foreign language other than the language you are contracted to work in.  

Furthermore, a suggestion should be rated 1 if it is offensive, violent or porny, such as “hot girls”, “sexy videos”, etc. The suggestion, at first glance, needs to be obviously offensive or seeking offensive material.  

You need to remember that you are not rating the results generated by the Facebook search engine but the suggestion itself. Many common names may generate pornographic results (ex.“mary ann”) but should not be rated as offensive. If it’s junky, then it is because the suggestion is unrecognizable and, most importantly, you think the suggestion is junky/gibberish. Junky entails the suggestion is difficult to understand or is entirely nonsense. If you rate the suggestion as junky, you would then choose a category to support your decision.  

A suggestion gets a 3 – Acceptable if it is not junky. This is considered a good suggestion which doesn’t contain foreign language or anything offensive. It is spelled correctly and free of grammatical errors. If a suggestion is rated acceptable, then it must receive an additional rating to indicate whether it is personally recognizable to you. This is a weird task, because you wonder how it can be acceptable if it’s not recognizable. They clear that up by saying ‘The suggestion is UNRECOGNIZABLE based on your own point of view but still makes sense’. They say that you can use Google/Bing to determine if the suggestion is acceptable or junky. But once again, the rating is based on your own personal view.  

The job itself doesn’t look very hard, and the work flow should go pretty fast since you only have to choose from 4 different categories, and then choose a label from one of those 4 categories. There are a lot of examples in the guidelines that show you what to do in certain cases in case you are unsure of what to select.  

It is interesting to know what this will end up doing to the Facebook pages that rely on certain words that are brought up. Some searches may end up disappearing from the suggestions at all.  

As always, if you have any questions about this project, or you are working on this project, leave us a comment and tell us how it’s going.

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest

Advertisements

Maps Analyst Projects at Lionbridge and Shasta Project at Appen

Apple-Maps-IconWe’ve got some new information to share about a position called the “Maps Analyst” at Lionbridge (Codenamed: Shasta project at Appen. It should be mentioned that the recruitment process for this project normally lasts a very limited period of time (at Lionbridge), and usually 1-2 months during different months depending on the country. However, the Shasta project could be available at any time. 

 The project involves evaluating online mapping, and routing related tasks in order to improve content and accuracy of map service: Apple maps.

Raters are required to conduct research, and use online tools to investigate the geographic accuracy of tasks. Work is provided to you as a list of tasks through an online work platform, which is somewhat like raterhub. Most of tasks will be in the language related to the position that you applied for. For example, if you applied for the French role, then you will be rating in French etc. You will also be required to provide comments/feedback in English. 

Map Analyst & Shasta Project Payment and working hours

This project is paid on a per task basis. You need to keep track of the amount of tasks, as well as the type of tasks completed on a day-to-day basis. This information is used to generate your invoice on the billing dates. The payment rate per tasks depends on your market, including the complexity of each task, and payment may vary from, $0.22 to $1.00 per task. Since you are working in a freelance independent capacity, you can manage your working time as you like and work whatever times you want. But this is a part-time job, and the number of hours you can accrue is set at a maximum of 20 per week. You must not exceed the maximum of 20 hours per week unless authorized. Lionbridge expects new Map Quality Analysts to be active and to participate each week. 

 Project compatibility

This project is fully compatible with any other projects at other vendors. You are not allowed to work on another project at Lionbridge, although you are allowed to qualify on another projects if your vendor is Appen. 

Maps Analyst/Shasta project qualification 

In my opinion the qualification process for this particular project is quite difficult if you compare it with other projects. First, you need to study and understand 200 pages of guidelines and instructions within a limited period of time, and then complete a 2 stage exam, which consists of theoretical and practical parts, within 10 days. As with any project, you should read the Guidelines as many times as you can before you start working. There is a lot of information in the guidelines so you may want to read them multiple times. You should also use them while working, and refer to them to ensure you are applying the correct rating. There is no chance to pass this exam without getting a profound understanding of the rating rules and concepts. 

Exam Preparation Approach (Maps Analyst and Shasta project exam)

Lionbridge has very good exam support so candidates should use this to their advantage as much as possible. It is highly recommended to use the below approach in order to increase your chances of successfully passing the exam: 

  1. Read the Guidelines thoroughly and understand them
  2. Read/watch the content on the Pre-Exam Support Portal
  3. Complete the Sample Exam Questions

Completing sample exam tasks will help you understand what kind of exam tasks to expect. A typical maps analyst task is a query; address, chain/organization, point of interest, and a list of results generated by the map service based on user location, user view port, and type of query you will need to provide:

  •  Relevance rating (taking into account distance/user intent issue) 
  • Keep in mind that distance ratings are variable ratings which depend on query type (type of business), user location, area, etc. So each case should be evaluated individually in terms of proximity and distance, that is why there are no specific criteria for distance
  •  Name accuracy (you always need to check name accuracy on official website of organization) 
  •  Address accuracy (check all components of address on official website/UPS ZIP code look up website) 
  •  Pin accuracy (apply rules as per guidelines) 
  • The pin location, name and address accuracy do not affect the relevance rating. If the result meets the requirements for a navigational rating then you should go for it. Relevance and the pin location, name and address accuracy are independent ratings

Finally, you will provide comments if your relevance rating is below good. This just a general overview of typical task/algorithm however it gives a good understanding what kind of tasks to expect in this project. 

Maps Analyst Exam Task

 It is very important that you thoroughly research the query, and the result in every task you are rating. In order to establish the correct rating, you need to understand the query, and you also need to verify that the information returned in a result is correct and accurate including the name, address, location, existence, popularity etc. In order to determine if the information is true, use official sources wherever possible. Include links in your comments whenever you find out that the task returned incorrect data. To determine popularity, you will need to check reviews on reputable sites to decide on the most accurate rating to apply. You do not need to include a comment for every rating, but comments are mandatory for any rating below Good for relevance or below Correct/Perfect for data accuracy. 

  • Reference Guidelines – If you are demoting because of a guideline intricacy, it is useful to reference the guideline/rule that influenced your decision.
  • Be Concise – Don’t waste time writing elaborate comments, be specific and efficient with your comments.
  • Comment in English Only- Your comments will sometimes be reviewed or used by those who are not very familiar with your locale or its language.

As mentioned before you will be given 7 days to complete the exam. The theoretical part will consist of 20 questions which are entirely based on the guidelines. I think this part won’t create any problem because answers to most of questions can be found in the guidelines, average estimated time for completing this part of the exam is 1 – 2 hours. 

 Part 2 is the most difficult part of the qualification, some of the tasks are really difficult, and will require a high level of attention, the ability to apply rating concepts, and analytical thinking. In total you will be asked to rate 20 tasks, but in fact there more tasks because some of them are complex tasks which have a list of results that have to be rated. You will be given almost all types of results to rate (chain, address, transit, POI etc.) So be careful to research and examine each case step-by-step. Keep in mind that only one attempt to pass this exam is allowed. If you need any tips or further information, do not hesitate to contact me.

 Maps Analyst Exam results 

Once you complete this exam within 2-3 days you should expect exam results and a welcome email containing information about access to the supplier and project portal. You also will be given a login and password to the rating tool where you will do your production. 

Project workflow

When you start working, the process will be more smooth because you have already gained a conceptual understanding of the work during testing, plus most of tasks will be in your local language. It is recommended that 2 minutes be taken to rate each task in order to complete your research accurately. Please note this is an average. Some tasks may require more time, while others may require less time. Tasks can be made available any day of the week, at any time of the day. There is no specific time when tasks are launched. If you don’t see tasks, then you will need to check more frequently to see if new tasks have been added to your queue. 

Falcon Crowd Help at Appen Global

survey1Appen Global offers a lot of different types of projects that you are able to work on besides the larger projects such as Falcon Video Recommendations, or any of the other larger Falcon projects. Falcon is simply a code word for a certain division at Appen Global which encompasses a group of clients that have different data needs. Based on my experience, it is not likely that you could start on the Falcon Crowd Help project from the beginning if you are hired by Appen Global. You would be more likely to start on a larger project first. Once you have started working at Appen, and you are on a Falcon project, like any of the social media projects that they offer, you will likely get an invitation to come aboard and start on the Falcon Crowd Help project. This is based on how well you are doing on your current project. In my experience, I have noticed that the longer I am working on a project, the more invitations for different projects I get. Appen uses this system and gauges individual performance. If you do good, you will get invited to work on other projects. Everyone who does a good job on their initial contract will most likely get invited to Falcon Crown Help. From there you would receive different invitations to different projects. This is where this jobs gets exciting. But I’ll get to that later.

By now you are likely wondering what this project is all about. Appen Global has many, many different clients that need different types of data. They could have a client that needs voice data. In this case you would be sent an email with an invitation. Sometimes you will be told what is required in an email, at which time you will also see a tile for each project you are invited for when you log into your Appen Global account. Sometimes the tasks are surveys and usually paid fairly well. Tasks are usually one time tasks in which when completed, are paid for and finished. Sometimes a task will need more data and you might be invited to similar tasks. If you participate and complete survey or quiz tasks, you are likely to be invited to other tasks. I have had 5-10 task invitations in a week. You do not have to accept all these tasks, and some you might not qualify for. For example, a recent tasks wanted voice recordings from a younger audience, and asked if I had children between the ages of 12-17. I don’t meet those requirements, so of course I would have to pass on this task.

So many different types of tasks are available, and one day you might be asked to give your input on an application, or give voice recorded data in which you speak given phrases, record them on your computer, then send back to Appen Global. Some tasks are simple quizzes. The exciting thing is that you are not doing the same tasks over and over. There is a lot of variety which really starts to make your job interesting. Payment on average is $10 per hour depending on the type of task. Of course you aren’t restricted by time and can complete tasks as quickly as you want, as long as they are done right. A recent task paid me $22.00 for a couple hours of work and I ended up finishing in an hour and a half. Not bad money and the tasks aren’t mundane or boring.

There are a few things to point out. Don’t expect to qualify for all tasks. As I previously mentioned in my example above, you will not always be able to meet the requirements. Don’t worry about this though, there are so many other tasks that will come your way. The more you participate, the more tasks you are given. If you don’t do any tasks, you might see a sharp decline in invitations. I tested this and for a week did not get any invitations. It is important to accept at least 1-2 invitations for tasks, and complete them by their deadline so that you will be invited to other tasks. Keep in mind that if you start a task and you don’t complete it by the deadline, you won’t get payment for it.

Falcon Crowd Help does require you to take two tests before being accepted into the team. The first one is the Falcon Crowdsourcing Screening Test. This is a simple test that asks basic questions and is easily passed. The second test is a Spelling test. Keep in mind this is for English-Canada, so you need to have good knowledge of the English language. This is an application process and they really want you to pass this and join their team. Once on the team, you would contribute to exciting and interesting projects. There are not always projects available and they are usually done on a first come first serve basis, but I have had no problem getting into any of the tasks that have been sent me. In a month I have made a couple hundred extra dollars just from doing different tasks. Some tasks simply ask questions based on a program or an application. I was invited to a task that asked me how I would talk to a voice recognition music player. The task didn’t take longer than an hour, and I was paid $25 for it. Helping by supplying data for projects that are in development can be very rewarding, although I have noticed that you aren’t told exactly what you are supplying data for. But that isn’t really important and completing most tasks are very simple.

Falcon Crowd Help can be a lot of fun and while you are working on other projects at Appen Global. You can make a few extra dollars here and there by completing simple tasks. There is no requirement to complete tasks and you aren’t forced to complete any of them, nor will you be penalized if you do not accept the tasks. The only thing is that you will see a decline in invitations. You can accept as many tasks as you can handle, but you must complete the tasks by the required deadline which is always given in the invite email. When you get this email, and if you see that you can qualify, its best to accept the invitation as soon as possible because there is a limit to how many people can be on one task. So you could find they already have enough people. This doesn’t happen often, and I have not seen it happen yet, but I usually accept all tasks that I qualify for.

What is Crowdsourcing Screening Test at Appen?

It is a basic test which is sent to some candidates who applied for any crowndsourcing opportunity, the aim of this test is to  evaluate  English proficiency level and basic PC and research skills. If you have any doubt while completing this test I recommend you to copy/paste test question into Google search  and Google will provide you with answers to a question.

As always, if you have any questions, leave me a comment and I will answer as soon as I can.

Read also this:

Payoneer as a payment method for Appen, Raterlabs, Leapforce and Lionbridge

Appen Rattle Project (The Falcon Video Recommendations)

online-social-medi_2734204bAppen now has a new hot opportunity that anyone with a verified Facebook account can work on. It’s a very unique project that involves watching live video or live video feeds and rating these videos based on their content and whether or not they are useful to you or not. And while it sounds like a very easy job, it can be tricky if you don’t understand what Appen Hill is looking for when it comes to data collection.

Application process and project set up

The process to apply for the job is the same process that you would use normally for any other project that Appen offers. Once you apply for the position, if you are selected, then you will start to receive instructions on qualifying. Yes you need to qualify for this particular job, but it’s nothing too difficult.

How To Apply for Search Engine Rater and Social Media Evaluation Jobs at Appen and Lionbridge

The process will take 2-3 days in which you will be able to review the project guidelines, any additional materials concerning the job, including the presentation, and then completing a ‘Qualification quiz’. It’s very important that you use the same email and I recommend creating a new gmail account just for this purpose. The reason is that the emails that you will get are time sensitive, and having a new email will ensure that they don’t end up in spam or junk. Also it’s very important that you use the email address you have on file with your Appen application. So if you have already applied for Appen, use this same email.

After you have applied for the position, you will get an automated email from Appen saying that you will be contacted for the Rattle Video Recommendations Project. Once you have been contacted, a 6 step process begins. Applying for the project and making sure that your email is verified is the first part.

FACEBOOK ID Requirements

The second part of the process will be setting up the Facebook Profile. You must use your Facebook account and it must be at least 1 year old and have no less than 50-55 friends. Your Facebook account has to be 100% truthful using your name that is on your Appen application. You should not have more than 1 Facebook account as this goes against Facebook TOS. Age group, location, name, and gender must be accurate.

Facebook Rating Tool 

Your Facebook ID will be used to set up your access in the Facebook tool you will use to complete daily assignments. Once set up, you’ll be given a batch of videos/pages which you will then evaluate based on the project guidelines. Through the tool, Appen Project Managers will have access to those batches, your ratings and the comments you provide. Appen does not have access to your personal FB information such as your account, messages, posts or network of friends. There will be no changes to your actual News Feed. Everything will be the same except that you will have access to what is called the FRT Tool.

Very soon you will be getting the Rattle Video Recommendations Project Guidelines. You should take some time to read through this carefully to understand exactly what you will be doing as a video rater, and most importantly, that you give the correct ratings and data. While it is an easy job, it can be a little tricky to understand how you should comment, and how those comments should reflect the numbers rating that you have given. Otherwise, you could find yourself moved off the project quickly.

The next step will be getting your Sharepoint Access where you will be able to download all materials, including the guidelines, for the project. Take the time to get familiar with Sharepoint. This email will come from Appen-Connect Support.

Rattle (Falcon Video recommendation) project qualification quiz

The last step is receiving the instructions on completing the Falcon News Feed Qualification Quiz via Appen Connect. Once you complete the online module, you will need to complete the quiz to qualify. Quiz results are instant after each question. IMPORTANT! You only get one chance to pass this quiz. They will not give a second chance to pass it under any circumstances. Qualification should take around 5 hours (Review of all emails and project documents, review of guidelines and PowerPoint presentation, open forum attendance and quiz).

Once you have completed the Quiz and successfully passed it, you will then be contacted by a Project Manager 1-2 days after the quiz. It is not necessary to email them.

Rattle Video  Recommendation project production 

Production work starts as soon as you have successfully completed the entire process. It is VERY important to once again mention that the deadlines for the process are very strict.

The Falcon Video Recommendations Project requires that you work no less than 5 days per week and one of those days must include a weekend. You can split the days as you like as long as one weekend day is worked. Each batch consists of 28-30 videos in which you are given a total time of 2.25 hours to complete. Time is automatically kept track of in the rating tool. You cannot go over the 2.25 hours per 30 video batch. If you stay on track, this comes out to about 4:00 minutes per video. There is a task at the end called the ‘bakeoff task’ which takes about 5 minutes to complete.

Further details concerning how the system actually works will be laid out in the project guidelines.

Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?