We are a language technology research group working on improving the way news is presented to readers. One of the systems we are building will summarise issues via quotes about them, so that you can see what different people are saying about a current issue, policy, or event. We have automatically found several hundred documents that are about one of several issues that were reported in news text. Within these documents we have also automatically identified the quotes and who we think said each quote.
What we don't know is how the speaker of each quote felt about the issue they were talking about. That's where you come in. For each quote in the document we need you to mark on a five-point scale what the speaker was trying to get across about their feelings on an issue (from &amp;amp;amp;amp;quot;strongly agrees&amp;amp;amp;amp;quot; to &amp;amp;amp;amp;quot;strongly disagrees&amp;amp;amp;amp;quot;). From these annotations we will develop a system that can automatically tell the difference between an extreme quote on an issue, and a more moderate one.
To complete these annotations you will use a web interface we have developed for this task, and follow explicit guidelines that describe the annotation task. We will be checking over your annotations and giving you feedback about what we need done differently. Accuracy is critical for us, as our system cannot learn from inconsistent annotations.
This task requires first-rate English fluency, and the ability to follow instructions that sometimes rely on subtle linguistic distinctions. We need several people for this task and we particularly encourage bids from freelancers we have worked with successfully on previous projects.
This project will be structured as follows. We will be asking annotators to complete sets of 100 documents, which we expect will take between 4 to 8 hours. Additionally before the first set of documents we will need you to complete a training set of 20 documents, which we will pay you for at the same per hour rate. As this task requires some training we prefer annotators who can complete several sets of documents. Bids on this project should be your price for completing a set of 100 documents.