Maps Analyst Projects at Lionbridge and Leapforce

Apple-Maps-IconWe’ve got some new information to share about a position called the “Maps Analyst” at Lionbridge (Codenamed: Shasta project at Leapforce). It should be mentioned that the recruitment process for this project normally lasts a very limited period of time (at Lionbridge), and usually 1-2 months during different months depending on the country. However, the Shasta project could be available at any time. 

 The project involves evaluating online mapping, and routing related tasks in order to improve content and accuracy of map service: Apple maps. Raters are required to conduct research, and use online tools to investigate the geographic accuracy of tasks. Work is provided to you as a list of tasks through an online work platform, which is somewhat like raterhub. Most of tasks will be in the language related to the position that you applied for. For example, if you applied for the French role, then you will be rating in French etc. You will also be required to provide comments/feedback in English. 

Payment and working hours

This project is paid on a per task basis. You need to keep track of the amount of tasks, as well as the type of tasks completed on a day-to-day basis. This information is used to generate your invoice on the billing dates. The payment rate per tasks depends on your market, including the complexity of each task, and payment may vary from, $0.22 to $1.00 per task. Since you are working in a freelance independent capacity, you can manage your working time as you like and work whatever times you want. But this is a part-time job, and the number of hours you can accrue is set at a maximum of 20 per week. You must not exceed the maximum of 20 hours per week unless authorized. Lionbridge expects new Map Quality Analysts to be active and to participate each week. 

 Project computability 

This project is fully compatible with any other projects at other vendors. You are not allowed to work on another project at Lionbridge, although you are allowed to qualify on another projects if your vendor is Leapforce. 

Maps Analyst/Shasta project qualification 

In my opinion the qualification process for this particular project is quite difficult if you compare it with other projects. First, you need to study and understand 200 pages of guidelines and instructions within a limited period of time, and then complete a 2 stage exam, which consists of theoretical and practical parts, within 10 days. As with any project, you should read the Guidelines as many times as you can before you start working. There is a lot of information in the guidelines so you may want to read them multiple times. You should also use them while working, and refer to them to ensure you are applying the correct rating. There is no chance to pass this exam without getting a profound understanding of the rating rules and concepts. 

Exam Preparation Approach (Maps Analyst and Shasta project exam)

Lionbridge has very good exam support so candidates should use this to their advantage as much as possible. It is highly recommended to use the below approach in order to increase your chances of successfully passing the exam: 

  1. Read the Guidelines thoroughly and understand them
  2. Read/watch the content on the Pre-Exam Support Portal
  3. Complete the Sample Exam Questions

Completing sample exam tasks will help you understand what kind of exam tasks to expect. A typical maps analyst task is a query; address, chain/organization, point of interest, and a list of results generated by the map service based on user location, user view port, and type of query you will need to provide:

  •  Relevance rating (taking into account distance/user intent issue) 
  • Keep in mind that distance ratings are variable ratings which depend on query type (type of business), user location, area, etc. So each case should be evaluated individually in terms of proximity and distance, that is why there are no specific criteria for distance
  •  Name accuracy (you always need to check name accuracy on official website of organization) 
  •  Address accuracy (check all components of address on official website/UPS ZIP code look up website) 
  •  Pin accuracy (apply rules as per guidelines) 
  • The pin location, name and address accuracy do not affect the relevance rating. If the result meets the requirements for a navigational rating then you should go for it. Relevance and the pin location, name and address accuracy are independent ratings

Finally, you will provide comments if your relevance rating is below good. This just a general overview of typical task/algorithm however it gives a good understanding what kind of tasks to expect in this project. 

Maps Analyst Exam Task

 It is very important that you thoroughly research the query, and the result in every task you are rating. In order to establish the correct rating, you need to understand the query, and you also need to verify that the information returned in a result is correct and accurate including the name, address, location, existence, popularity etc. In order to determine if the information is true, use official sources wherever possible. Include links in your comments whenever you find out that the task returned incorrect data. To determine popularity, you will need to check reviews on reputable sites to decide on the most accurate rating to apply. You do not need to include a comment for every rating, but comments are mandatory for any rating below Good for relevance or below Correct/Perfect for data accuracy. 

  • Reference Guidelines – If you are demoting because of a guideline intricacy, it is useful to reference the guideline/rule that influenced your decision.
  • Be Concise – Don’t waste time writing elaborate comments, be specific and efficient with your comments.
  • Comment in English Only- Your comments will sometimes be reviewed or used by those who are not very familiar with your locale or its language.

As mentioned before you will be given 7 days to complete the exam. The theoretical part will consist of 20 questions which are entirely based on the guidelines. I think this part won’t create any problem because answers to most of questions can be found in the guidelines, average estimated time for completing this part of the exam is 1 – 2 hours. 

 Part 2 is the most difficult part of the qualification, some of the tasks are really difficult, and will require a high level of attention, the ability to apply rating concepts, and analytical thinking. In total you will be asked to rate 20 tasks, but in fact there more tasks because some of them are complex tasks which have a list of results that have to be rated. You will be given almost all types of results to rate (chain, address, transit, POI etc.) So be careful to research and examine each case step-by-step. Keep in mind that only one attempt to pass this exam is allowed. If you need any tips or further information, do not hesitate to contact me.

 Maps Analyst Exam results 

Once you complete this exam within 2-3 days you should expect exam results and a welcome email containing information about access to the supplier and project portal. You also will be given a login and password to the rating tool where you will do your production. 

Why do people fail Maps Analyst Shasta project qualification exams?

Lack of effort and sufficient preparation

The most common reason that people fail the Maps Analyst/Shasta project exam, is that some candidates think that they may get lucky and pass the exam without proper studying, or you may think you know enough about the internet or have good search skills. Be aware that all parts of exam are designed in a way to assess your ability to understand and apply rating concepts and rules based on a 200 page guide. Its not possible to demonstrate the knowledge of these guidelines without investing a significant amount of effort within the one week they give you to study for the exam. If you don’t at least study the guide, or go over it a couple times, you will most likely fail the theoretical part (part 1) of the exam.

Poor time management

One of the common reasons for unsuccessful attempt to pass the exam is poor time management. People do exam preparation and taking all 2 parts within 1-2 days. This strategy in many cases leads to exam failure, because the exam requires lots of concentration and attention to details…Thus plan your exam carefully so you have enough time for studying the material and taking the exam without rushing. Basically, don’t wait till the last minute.

Inappropriate understanding and/or implementation of rating rules and concepts

Have you failed the Leapforce or Lionbridge exams after carefully studying for them? Don’t worry, a lot of people have. Even people with experience in Search Engine Optimization, or even experience in taking exams have failed this test. In fact it is the first thing that Leapforce mentions before you get the exam. That it is very difficult and many fail. If you have failed, it is most likely you misunderstood the rating concepts and/or applied them in a wrong way; e.g. used personal judgment/common sense, or did not pay attention to details where it was required (e.g. user location for local queries).


When you start working, the process will be more smooth because you have already gained a conceptual understanding of the work during testing, plus most of tasks will be in your local language. It is recommended that 2 minutes be taken to rate each task in order to complete your research accurately. Please note this is an average. Some tasks may require more time, while others may require less time. Tasks can be made available any day of the week, at any time of the day. There is no specific time when tasks are launched. If you don’t see tasks, then you will need to check more frequently to see if new tasks have been added to your queue. 



Falcon Crowd Help at Appen Global

survey1Appen Global offers a lot of different types of projects that you are able to work on besides the larger projects such as Falcon Video Recommendations, or any of the other larger Falcon projects. Falcon is simply a code word for a certain division at Appen Global which encompasses a group of clients that have different data needs. Based on my experience, it is not likely that you could start on the Falcon Crowd Help project from the beginning if you are hired by Appen Global. You would be more likely to start on a larger project first. Once you have started working at Appen, and you are on a Falcon project, like any of the social media projects that they offer, you will likely get an invitation to come aboard and start on the Falcon Crowd Help project. This is based on how well you are doing on your current project. In my experience, I have noticed that the longer I am working on a project, the more invitations for different projects I get. Appen uses this system and gauges individual performance. If you do good, you will get invited to work on other projects. Everyone who does a good job on their initial contract will most likely get invited to Falcon Crown Help. From there you would receive different invitations to different projects. This is where this jobs gets exciting. But I’ll get to that later.

By now you are likely wondering what this project is all about. Appen Global has many, many different clients that need different types of data. They could have a client that needs voice data. In this case you would be sent an email with an invitation. Sometimes you will be told what is required in an email, at which time you will also see a tile for each project you are invited for when you log into your Appen Global account. Sometimes the tasks are surveys and usually paid fairly well. Tasks are usually one time tasks in which when completed, are paid for and finished. Sometimes a task will need more data and you might be invited to similar tasks. If you participate and complete survey or quiz tasks, you are likely to be invited to other tasks. I have had 5-10 task invitations in a week. You do not have to accept all these tasks, and some you might not qualify for. For example, a recent tasks wanted voice recordings from a younger audience, and asked if I had children between the ages of 12-17. I don’t meet those requirements, so of course I would have to pass on this task.

So many different types of tasks are available, and one day you might be asked to give your input on an application, or give voice recorded data in which you speak given phrases, record them on your computer, then send back to Appen Global. Some tasks are simple quizzes. The exciting thing is that you are not doing the same tasks over and over. There is a lot of variety which really starts to make your job interesting. Payment on average is $10 per hour depending on the type of task. Of course you aren’t restricted by time and can complete tasks as quickly as you want, as long as they are done right. A recent task paid me $22.00 for a couple hours of work and I ended up finishing in an hour and a half. Not bad money and the tasks aren’t mundane or boring.

There are a few things to point out. Don’t expect to qualify for all tasks. As I previously mentioned in my example above, you will not always be able to meet the requirements. Don’t worry about this though, there are so many other tasks that will come your way. The more you participate, the more tasks you are given. If you don’t do any tasks, you might see a sharp decline in invitations. I tested this and for a week did not get any invitations. It is important to accept at least 1-2 invitations for tasks, and complete them by their deadline so that you will be invited to other tasks. Keep in mind that if you start a task and you don’t complete it by the deadline, you won’t get payment for it.

Falcon Crowd Help does require you to take two tests before being accepted into the team. The first one is the Falcon Crowdsourcing Screening Test. This is a simple test that asks basic questions and is easily passed. The second test is a Spelling test. Keep in mind this is for English-Canada, so you need to have good knowledge of the English language. This is an application process and they really want you to pass this and join their team. Once on the team, you would contribute to exciting and interesting projects. There are not always projects available and they are usually done on a first come first serve basis, but I have had no problem getting into any of the tasks that have been sent me. In a month I have made a couple hundred extra dollars just from doing different tasks. Some tasks simply ask questions based on a program or an application. I was invited to a task that asked me how I would talk to a voice recognition music player. The task didn’t take longer than an hour, and I was paid $25 for it. Helping by supplying data for projects that are in development can be very rewarding, although I have noticed that you aren’t told exactly what you are supplying data for. But that isn’t really important and completing most tasks are very simple.

Falcon Crowd Help can be a lot of fun and while you are working on other projects at Appen Global. You can make a few extra dollars here and there by completing simple tasks. There is no requirement to complete tasks and you aren’t forced to complete any of them, nor will you be penalized if you do not accept the tasks. The only thing is that you will see a decline in invitations. You can accept as many tasks as you can handle, but you must complete the tasks by the required deadline which is always given in the invite email. When you get this email, and if you see that you can qualify, its best to accept the invitation as soon as possible because there is a limit to how many people can be on one task. So you could find they already have enough people. This doesn’t happen often, and I have not seen it happen yet, but I usually accept all tasks that I qualify for.

What is Crowdsourcing Screening Test at Appen?

It is a basic test which is sent to some candidates who applied for any crowndsourcing opportunity, the aim of this test is to  evaluate  English proficiency level and basic PC and research skills. If you have any doubt while completing this test I recommend you to copy/paste test question into Google search  and Google will provide you with answers to a question.

As always, if you have any questions, leave me a comment and I will answer as soon as I can.

Read also this:

Payoneer as a payment method for Appen, Raterlabs, Leapforce and Lionbridge