Uolo project review

There’s a new project out there from Appen Connect with the codename Uolo which is about researching and evaluating the claims that are made in video content from the Facebook platform, all tasks are evaluated on SRT Facebook rating tool. 

Uolo rating process

The task itself is split into two parts – Task A is the Content Description and Task B: is where you will do the Claim Assessment. Both of these tasks have their own rating queue. 

Task A Queue focuses on providing some information about the Context of different pieces of video content. 

Task B Queue focuses on identifying, researching, and evaluating the Central Claim stated in the video content. 

To sum it all up, you’re going to be checking claims made by the authors of the video you will be assessing. Facebook is doing this project because they want to better understand the information that is shared on their platform and check an individual’s credibility. 

It looks like the guidelines they give you runs about 27 pages long which is very short considering other project guidelines. This is probably because the guidelines that they provide aren’t set in stone and are subject to change – Basically, they are not a strict set of guidelines because this isn’t intended to be a purely objective task. The really interesting thing is that the guidelines state that there will be some variability between people (you) in how you approach the task, and how you will respond to the questions… We say interesting because this is by design. 

With that in mind, you should try as much as possible to follow the instructions which we will go over here in a minute and answer each of the questions to the best of your ability. 

There is a set of tasks that you are given that will ask you to evaluate the Central Claim that is presented in a piece of video content. 

Now if we are to define a Central Claim, it is a statement of fact related to the content’s main point or purpose. Statement of fact means a sentence or idea that the content is conveying as a fact, regardless of whether or not it’s factually accurate. The guidelines seem to be written by someone who is really technical and trying to confuse everyone. Basically, what this means is that you are looking for the main statement or claim made in the video that could be either supported or contradicted by evidence. This is the key element you are looking for regardless of whether or not it would be possible to find the evidence by an internet search. 

So for example, the video capture below shows people attempting to burn a flag in a baseball field, and a baseball player runs up and snatches the flag away. You’ll notice at the bottom of the video there are subtitles that say this happened on April 25th, 1976, and there is a status update message that say the event happened at Dodger Stadium and it was player Rick Monday who took the flag away. 

There are a couple of things you need to look at here. You’ve got a couple of things here that need to be evaluated and those are: Was it really Rick Monday who stopped the two protestors from burning the flag on the date shown AND you should also focus on whether there is evidence that the video is the actual footage of the event described in the text. 

There are some general instructions that you need to keep in mind. To evaluate this claim you will need to be a good searcher using the internet search engine of your choice (most of you will likely choose Google and that’s OK) and you need to provide some information about the evidence you found. We’ll go over this process for you to help you understand. 

It is possible that in some cases a central claim may be a statement of fact, but there might not be any public evidence about the statement of fact itself. 

Rating time frame

The time it takes you to evaluate a piece of content will totally depend on each piece of content. Some video content may be really easy to research and evaluate and might only take you 3-4 minutes to complete. Other videos are going to be more difficult to identify or check the claims, and it could have multiple claims requiring that you spend more time to research, AND enter your responses – It is possible that it could take up to 20 minutes, but most items will fall somewhere in between. 

That said, you are allowed up to 15 minutes of research time for each item. If you do encounter a piece of content that is difficult to evaluate, and you feel like you’re not going to be able to deliver a satisfactory conclusion with your evidence after this time, then you should still answer the task’s questions to the best of your ability based on what you could find. The good thing about this is there is an option to specify that you ran out of time during the task. 

disqualifying issues during rating process

In the Task A Queue you will be asked to provide information about the context of the video content. During your assessment, you might run into some disqualifying issues. 

There are THREE reasons why video content can be disqualified from being rated. 

1. You’re not able to review the content because of a bug or a technical issue. It could be the SRT preview is broken or the video or its caption doesn’t render properly. 

2. If you are hired for ANY APPEN project in the English language, then the content provided for you should be in English. If it’s in a different language you don’t understand then you obviously cannot rate it.

3. The content doesn’t contain a central claim based on a statement of fact which can be supported or contradicted based on evidence. It is possible that the content is solely to convey an opinion or values statement and it doesn’t contain any statement or facts. 

There is additional information in the guidelines that helps to explain when content doesn’t contain a central claim based on a statement of fact. 

Task B: is your Claim Assessment where you are asked to identify the central claim, search for evidence of the claim, and then make an evaluation about whether it’s supported by evidence from trustworthy sources. 

At the start of Task B you are given the opportunity to indicate whether or not the content should be disqualified from review. 

However, the items that you will get in the queue have been identified as very likely rateable and so you should RARELY need to mark something as disqualified. THIS IS VERY IMPORTANT! If you mark things as disqualified and it is reviewed and found to be marked incorrectly, you could get an email from a manager with a warning!

Even if something contains an opinion, there is still a good chance that it’s about a statement of fact being presented in the content. The central claim should be one sentence or idea from the content you are evaluating. 

Usually the central claim in a video is the most prominently displayed claim in the content. It is the thing that the viewer would most likely observe, and a lot of time the central claim will be stated in the title. 

There are a lot of examples in the guidelines that go over the differences of video text that may or may not talk about the central claim – Video text may directly state a claim about what is happening in the video, or the video text may state a claim that is unrelated to what is being shown in the video. In this case you should focus on identifying and evaluating the central claim in the text. 

Many examples are given in the guidelines to help you understand the differences. 

Searching for evidence of the claim will be your main task and making sure you find the proof or evidence of the stated claim from a reputable source is important. Of course, this should be from a different source than the one you are investigating. The evidence you find can either support or contradict it…as long as the evidence comes from a trusted source. 

You can use any search engine of your choice when it comes to hunting down proof or evidence of a stated claim. There are a lot of do’s and don’ts when it comes to searching. It is a good idea to refer to the guidelines when doing this. 

Once you have all your evidence, then you can perform an evaluation on the central claim. You are to provide your best judgement about whether the claim is fully supported, partially supported, partially contradicted, or documented as unverified. 

Even if you aren’t 100% sure about which category fits best, you should still make your best judgement which category fits the content best. 

As always, keep the guidelines handy because it has plenty of examples. If you have any questions about the project, please let me know. And if you are working on this project, then we would like to hear from you!

Payment rates

Appen and Lionbridge Projects – Hourly Rates Across Different Markets

Related Appen connect projects

List of Projects And Code Names For Lionbridge & Appen Connect