Detecting and Tracking Political Abuse in Social

MediaJ. Ratkiewicz, M. D. Conover, M. Meiss, B.

Gonc¸alves, A. Flammini, F. Menczer

Presented by: Jumana Nassour-Kassis

Barack Obama’s 2008 Presidential Campaign

• “The first occupant of the White House to have won a presidential election on the Web.”

• “This election was the first in which all candidates—presidential and congressional—attempted to connect directly with American voters via online social networking.”

• “It has even been called the ‘Facebook election.’”

http://www.usnews.com

Social Networks

• Have become an important part of our lives.

• Provide a platform to discuss issues of public interest.

• User-as-information-producer instead of just receiver. (grassroots)

• Can reach a larger and more diverse audience than social media users.

• In politics: play a crucial role in the successes and failures of political campaigns and causes.

• However:• Little guarantee on information correctness (catchiness and repeatability is

what counts!).

Truth Lies and Politics

• Social media:• Shape political

communication.• Sway voters or influence

pending legislations. • Can be faked!

Political Astroturf: masking the sponsors of a message or organization.

Previous Work

• Mustafaraj and Metaxas (2010): • Describe a concerted, deceitful attempt to cause a specific URL to rise to

prominence in Twitter using a network of 9 fake accounts.

• Created 930 tweets in 138 mins all having link to a URL smearing a candidate for 2009 Massachusetts election.

• In a few hours, it got promoted to the top of google search for that candidate creating a so called ‘twitter bomb.’

=> focused effort can initiate viral spread of information.

Previous Work - Continued

• Mendoza, Poblete, and Castillo (2010): False information is more likely to be questioned by users than reliable accounts of an event.

• Galuba et al. (2010): Predict the spread of URLs through the social network, taking into account: user behavior, user-user influence, and resource virulence.

• Sankaranarayanan et al. (2009): automated breaking news detectionsystem based on the linking behavior of Twitter users.

Previous Work - Continued

• Bollen, Mao, and Pepe (2010), Bollen, Mao, and Zeng (2011): There’s a correlation between the global mood of Twitter users and important worldwide events.

• Diakopoulos and Shamma (2010): Relation between media events and responses among social media users.

• Tumasjan et al. (2010): Information shared on Twitter can predict the results of political elections.

Spam vs. Political Astroturfing

• Same tools: • Mass creation of accounts.• Impersonation of users.• Posting deceptive content.

• Different objectives:• Spam: Persuade users to click a link.• Political astroturfing: Establish a false sense of group consensus about a particular

idea.

• Detection: • Spam: Focus on content; properties of user accounts (e.g. the number of URLs in

tweets or the interval between successive tweets).• Political astroturfing: How the message is delivered.

Spam vs. Political Astroturfing

• Users are more likely to believe a message that they perceive coming from several independent sources or from an acquaintance.

• Legitimate users may be unwittingly complicit in the propagation of Astroturf.

=>Spam detection doesn’t work well for political astroturfing detection.

Goal & Hypothesis

Definition:

• Meme: Something that a person believes based on emotion rather than facts.

• Truthty - Falsely propagated information from organic grassroots memes.

• Goal: The early detection of truthy memes in the Twitter system.

• Hypothesis: Initial stages of truthy memes exhibit identifiable signatures.

Contribution

• Klatsch - A system to analyze, in real-time, the behavior of users and the diffusion of information in social media.

• Truthy- A system to automatically monitor the data stream from Twitter.

• Identify and track social network abuse efforts, that try to mimic the organic spread of information through Twitter.

Analytical Framework

• A generic stream of social networking data as a series of events.

• Events: involves a number of actors (users), some memes(units of information), and interactions among them.

• Each event contributes a unit of weight to the edges

Meme Types – Topics

• Reminder: Meme - Something that a person believes based on emotion rather than facts.

• Meme Types: • Hashtags: To label the topical content of tweets using # (e.g. #obama).

• Mentions: Include another user’s screen name in a post using @ (e.g. @BarackObama).

• URLs: Match strings of valid URL characters that begin with ‘http://’.

• Phrases: The entire text of the tweet.

• A tweet can be included in more than one of these categories.

Example

• Tweet:

“Happening now: President Obama is speaking to @OFAsupporters on how everyone can help #StopGunViolence. http://ofa.bo/f9cR”

• Meme types:• Hashtags: #StopGunViolence.

• Mentions: @OFA.

• URLs: http://ofa.bo/f9cR

• Phrases: President Obama is speaking to OFA supporters in how everyone can help StopGunViolence.

Diffusion Networks

• Directed graph.

• Nodes: individual user accounts.

• Edge from A to B: Flow of information from a to B!• B retweeted a message from A (according to metadata).

• A mentions B in a tweet.

• Weight: incremented each time an event connects 2 users.

Mention vs. Retweet

• Retweet: If B retweets A, then there’s an implicit confirmation that information from A appeared in B.

• Mention: If A mentions B, then A explicitly confirms that A’s message appeared in B’s Twitter feed. Therefore, may or may not be noticed by B.

=>Mention edges are less reliable indicators of information flow compared to retweet edges.

Diffusion Network - Example

• k - Degree. --- Retweet

• s – Strength = weighted degree. – Mention

Truthy:

• Detect relevant memes.

• Collect the tweets that match themes of interest.

• Produce basic statistical features relative to patterns of diffusion.

• Corpus: A sample of the twitter corpus(4-8 million tweets a day).

• http://truthy.indiana.edu/

Meme Detection

• Scan the collected tweets in real-time, and find candidate memes with ( ~305 million tweets ):• Content related to U.S. politics (manually collected keywords) 1.2 million

• That is of general interest. ( the number of tweets with that meme in a sliding window of time exceeds a given threshold.)

• Tracked 600,000 tweets (collected in about a month and a half)

Network Analysis

• Statistics based on the topology of the graph and its largest connected component.

• Skew: A measure of the asymmetry of the distribution of a real-valued random variable about its mean.

• Crowdsourced Annotation.

Network Analysis

• the Google-based Profile of Mood States GPOMS: • Sentiment analysis method.

• Assigns a 6 cell vector to describing mood attributes: calm, alert, sure, vital, kind, and happy.

• Relies on a vocabulary taken from an established psychometric evaluation instrument extended with co-occurring terms from the Google n-gram corpus.

Automatic Classification

• A supervised binary classifier to label legitimate and truthy memes.

• Training: manually annotated + bootstrapping a corpus with: (366 annotated memes)• Turthy- meme is spread in misleading ways by a significant portion of the

users, (61)

• Legitimate-several non-automated users conversing about a topic, (305)

• Remove- memes in a non-English language or otherwise unrelated to U.S. politics.

Automatic Classification

• 2 well-known classifiers:• AdaBoost with DecisionStump.

• Support vector machine (SVM).

• Used 31 features extracted from:• The topology of the diffusion network.

• Sentiment analysis.

• Crowdsourced annotation.

AdaBoost with DecisionStump vs. Support Vector Machine (SVM)• Support Vector Machines (SVMs) and Adaptive Boosting (AdaBoost)

are two successful classification methods.

• AdaBoost-• Feature selection with bootstrapping using Decision Stumps, a weak learning

algorithm.

• The output of the learners is combined into a weight sum.

• SVM –• Each dimension is a feature,.

• SVM maps the samples to this space and tries to find a hyperplane that divides the samples into 2 groups.

Classifier Results

• The training data was resampled to balance the classes prior to classification.

• AUC (area under the classifiers’ ROC curve)-appropriate in the presence of class imbalance.

Confusion Matrix of AdaBoost

The 10 Most Discriminative Features

• Determined by χ2 analysis with 10-fold cross validation.

• Network features such as, mean edge weight in largest connected component, mean strength, number of edges etc. appear to be more discriminative than sentiment scores or the few user annotations that we collected.

Examples of legitimate diffusion networks

#Truthy – injected by the NPR Science Friday radio program

@senjohnmccain – 2 different communities in which the meme was propagated: one by retweets from @ladygagain the context of discussion on the repeal of the “Don’t ask, don’t tell” policy on gays in the military, and the other by mentions of @senjohnmccain.

Detected Astroturf

#ampat:• The #ampat hashtag is used by any

conservative users. • Bursts of activity are driven by two

accounts, @Csteven and @CStevenTucker, which are controlled by the same user!

• To give the impression that more people are tweeting about the same topics.

• Same tweets using both accounts.• Generated a total of over 41, 000

tweets in this fashion.

Example 2• @PeaceKaren 25:

• Unknown owner.• Generated a very large number of tweets

(over 10,000 in four months). • Mainly in support of several Republican

candidates.• @HopeMarie 25:

• Similar behavior to @PeaceKaren 25 in retweeting the accounts of the same candidates and boosting the same websites.

• It did not produce any original tweets.• Retweeted all of @PeaceKaren 25’s

tweets.• Created a ‘twitter bomb:’ for a time,

Google searches for “gopleader” returned these tweets in the first page of results.

Example 3:

Example 3:• “How Chris Coons budget works- uses tax $ 2 attend

dinners and fashion shows”

• This is one of a set of truthy memes smearing Chris

Coons, the Democratic candidate for U.S. Senate from

Delaware.

• A network of about ten bot accounts.

• Inject thousands of tweets with links to posts from the

freedomist.com website.

• Duplicate tweets are disguised by adding different

hashtags and appending junk query parameters to the

URLs.

• To generate retweeting cascades, the bots also

coordinate mentioning a few popular users. When

these targets perceive receiving the same news from

several people, they are more likely to think it is true

and spread it to their followers.

• Most bot accounts in this network can be traced back to

a single person who runs the freedomist.com website.

Summary

• The proposed classification accurately detects ‘truthy’ memes based on features extracted from the topology of the diffusion network.

• Features of truthy memes:• High numbers of unique injection points with few or no connected

components.

• Strong star-like topologies characterized by high average degree.

• Large edge weights between 2 elements.

Future work

• Explore further crowdsourcing: (had only 304 clicks of the truthybutton) may prove useful with more data.

• Try other features with the classifier:• The age of the accounts involved in spreading a meme.

• The reputation of users based on other memes they have contributed.

• Other features from bot detection methods.

Thank You!