sentiment analysis - relevancy check of a twitter post training guide

Sentiment Analysis

Relevancy Check of a Twitter Post

Training Guide

1

Crowdsourcing Asia Sdn. Bhd. © All Rights Reserved - 2016

Task Introduction

● We have launched a new type of paid task called Sentiment Analysis.

● Many companies use digital workers to help analyze what people are saying on social media sites and blogs about their

products and services. These individuals that post comments are expressing “sentiment”.

● We have many companies interested in testing the eRezeki workforce for various types of Sentiment Analysis work. We

recommend you prepare yourself carefully for this task. If you do well, there is the potential for lots of work.

● There are various types of Sentiment Analysis work. They include:

○ Analysing whether sentiment is positive, negative or neutral

○ Categorizing types of sentiment (i.e. is a post about a particular product)

○ Filtering sentiment (i.e. is a post relevent)

● The task we have launched today requires workers to filter sentiment. The company requesting this work wants to

identify individuals that might need their products or services. Your job is to review Tweets and to filter them by choosing

between two options: to “accept” the post if it’s relevant or to “reject” the post if it’s not relevant

● If you qualify and deliver accurate results, you will get access to another task type that will require you to research

additional information about the individual that published the post.

2


A Quick Overview of Sentiment Analysis (1)

● The comments and posts on Facebook, Twitter, Instragram (and others) can be related to a customers view or

comments of a particular product. Companies utilize your analysis of the comments and posts to understand customer

feedback, improve their products or for product promotions.

● Although computer automated Sentiment Analysis can process sentiment, there are many instances where humans

do a better job of understanding language neuances; expressions, cultural variations and the use of slang or social

media abbreviations. Human judgement is often essential to distinguish the tone and meaning of a statement to

determine whether social media posts are relevant or are positive or negative. This is where you come in!

● Take this example, “My car broke down. Just great!” It’s easy to see that the person is frustrated that their car has broken down. By applying your contextual understanding of the statement, you can clearly assess that their sentiment is negative. Without human judgement though an automated approach could mistakingly interpret the example as positive because of the use of the word “great”.

● Also known as opinion mining, Sentiment Analysis, is the process of analysing these comments and posts to identify

and categorize opinions or attitudes, in order to determine whether a person’s opinion towards a particular idea or

product is positive, negative, or neutral. For example, a negative sentiment expressed in the following Tweet:

3


A Quick Overview of Sentiment Analysis (2)

4

● In another example, Expedia Canada changed their adverts after getting a lot of negative feedback on the music they

utilized on their ads.

After the company studied the sentiment of their customers, they learned that customers found that the music they

played on their commercials had become incredibly irritating! Knowing that the feedback was negative on social media

networks, they were able to respond.

● In other sentiment analysis tasks, you are asked to determine whether a social media post is relevant or not, for

example, could it be potentially valuable as a sales leads, used for marketing or promotional purposes, as per the

following example:


How to Access the Task

5

Login to Massolutions*

Locate the first qualification task and achieve a qualification score on the English Proficiency survey of at least 80%

Locate as follows: Qualify > Working with content > Content monitoring > Sentiment analysis(UGC/Social Media) > A Survey to Identify Worker English Language Proficiency

If you pass the first test, now get paid to take the specific task qualification and achieve a qualification score of 90%

Locate as follows: Qualify > Working with content > Content monitoring > Sentiment analysis(UGC/Social Media) > Qualification - Relevancy Check of a Twitter Post

If you’ve qualified, you can now access thousands of paid tasks!

Locate as follows: Work > Working with content > Content monitoring > Sentiment analysis(UGC/Social Media) > Relevency Check of a Twitter Post

*NOTE: Currently, some content, data and functionality used by Massolutions resides on the eRezeki Digital Work Portal. To provide workers with an easy to use worker interface, in particular when training, qualifying or seeking support, we have built Massolutions in a manner that allows the worker to view most content items via Massolutions without the need to also interact with the eRezeki Digital Work Portal. However, in order for all content to display within a Massolutions web page, the worker MUST be concurrently logged into the eRezeki Digital Work Portal.


The First Qualification Test!A Survey to Identify Worker

English Language Proficiency

● The purpose of the survey is to gauge your proficiency with the English language and your familarity with social media, an important element of sentiment analysis.

● It is very important that you read the instructions carefully before answering the survey.

● Since you will be listening to an audio recording to answer one of the survey questions, it is recommended that you prepare a set of headphones or speakers.

● You are required to answer twenty (20) questions.

● Remember to click “ACCEPT” before answering the survey.

● The timer starts once you have clicked accept. Remember to read and answer each question carefully.

● Once you have answered all the questions, click “Submit”.

● Once you have completed the survey, you will receive the notification, “Qualification English Language Proficiency” score 80 or above which qualifies you to the next step. if you receive a score of less than 80 you will not be able to proceed further.

● You can also check your qualification scores on your Massolutions dashboard. 6


The Next Step!A Paid Qualification Test:

Relevancy Check of a Twitter Post

● Once you have passed the survey, you can now take the “Qualification: Relevancy Check of a Twitter Post”.

● In each sentiment analysis task, a key focus point wil be required and it is absolutely necessary for you to know about it. You will not be able to complete the task without knowing the context.

● Please read the instructions carefully. You will need to complete 10 qualification tasks. You must obtain a qualification score of 90 to have access to paid tasks.

● Remember to ACCEPT each task before providing your judgement and do not let the task expire (timer becomes zero).

● Once you click “Submit”, remember that the timer still runs until you click “Next Task”.

● After completing the 10 tasks, you will receive a notification saying “ Your score Relevancy Check of a Twitter Post (CS)” is 90, if you qualified; 60, if otherwise.

● Alternatively, you can also check your qualification scores on your Massolutions dashboard.

7


Further Guidelines:Key Terms

● Twitter - a social media platform where people can share their messages to a group of people or friends.

● Tweet - a message consisting of 140 characters that a person posts on Twitter.

● The @ sign is used to call out usernames in Tweets: "Hello @twitter!" People will use your @username to mention you in Tweets, send you a message or link to your profile.

● A username is how you're identified on Twitter, and is always followed immediately by the @ symbol. For instance, Micheal Jackson is @michealjackson or @mj.

● A (#) hashtag is any word or phrase immediately preceded by the # symbol. When you click on a hashtag, you'll see other Tweets containing the same keyword or topic.

● A Tweet that you forward to your followers is known as a Retweet and is always symbolized as RT, followed by the username of the person who sent the tweet first. Often used to pass along news or other valuable discoveries on Twitter, retweets always retain original attribution.

■ e.g. RT @techdotlondon: Starting 11 April @CrowdWeek offers global stage 4 innovators 2 present collaborative solution buff.ly/1SUmzko

8


● Check the username or the author’s account name. Normally, if the posting of a Tweet has been initiated

automatically by a program, it is often spam or an advertisement. Typically the author’s account name would not show an account that includes an individual’s name. Often, the account profile photo shows an avatar, instead of an image of a person.

This is the username /author of the Tweet and it apears that it was automated and not posted by an individual so this tweet would be rejected.

Further Guidelines:Identifying Whether a Tweet is Spam or an Advertisement (1)

9


● Usually, a Tweet that is an advertisement clearly promotes a product or service and often includes a link in the Tweet to a product promotion page.

● Some of the advertisments may also be posted by individuals but you should be able to identify marketing Tweets by the content of the tweet.

For example:

While this Tweet is posted by an individudal, it’s an advertisement because it’s enticing people to open accounts in order to obtain free Netflix trials.

In this tweet, the author is promoting her website and intending to encourage people to access the link.

Further Guidelines: Identifying whether a Tweet is a

Spam or an Advertisement (2)

10


● A Tweet that is referencing the subject matter being analysed but that reflects negative sentiment should be rejected (i.e. mocking a product or service) .

● Looking at this next example, this Tweet about Star Wars should be rejected if the task is to identify positive sentiment Tweets because although the author might be a fan, the tweet mocks a character.

The individual posting this tweet is using irony by implying that in 2042 we will no longer be watching Netflix as its populatity will have declined.

This Tweet should be rejected because it is mocking a character and therefore the task requires negative sentiment to be rejected.

Further Guidelines: Identifying Tweets that portray

ironic or sarcastic language

11


● Tweets that are related to a key point but have some grammar mistakes are still acceptable as long as you can comprehend or understand what the tweet / message means.

● A tweet that expresses vulgar words and profanity are still acceptable, provided that the message portrays something useful for a company.

● Reject a tweet if it’s:

❖ Nonsense and unclear❖ Only consists of expletives or expressions of vulgarity❖ Bears no relation to the topic❖ For this particular task, in a language other than English❖ If the Tweet simply contains emoticons and/or symbols.

Further Guidelines:Spelling or Gramatical Errors,

Vulgarity and Nonsense

12

sentiment analysis - relevancy check of a twitter post training guide

Internet