election 2010: the view from twitter
DESCRIPTION
Paper presented at the InASA 'Double Vision' conference, Sydney, 26 Nov. 2010.TRANSCRIPT
Election 2010: The View from Twitter
Axel Bruns / Jean BurgessARC Centre of Excellence for Creative Industries and Innovation, [email protected] – @[email protected] – @jeanburgesshttp://mappingonlinepublics.net – http://cci.edu.au/
Image by campoalto
Project: New Media and Public Communication
• ARC Discovery (2010-12) – A$410.000
• Axel Bruns (CI), Jean Burgess (SRF) – QUT, Brisbane
• Lars Kirchhoff, Thomas Nicolai (PIs) – Sociomantic Labs, Berlin
• Project blog: http://mappingonlinepublics.net/
Year 1 Year 2 Year 3
Social network sources:
· YouTube· Flickr· Twitter· blogs
Research tools:
· network crawler· content scraper· content analysis· network analysis
Research tool development and baseline data
Baseline information:
· data extraction· content creation
statistics· patterns in terms
and themes· baseline social
networking map· interconnections
between social network spaces
Content creation patterns
Changes over time:
· short-term statistics· regular / seasonal
patterns
Cluster profiling:
· common themes / patterns
· lead users
Focus on specific events
Cultural dynamics:
· rapid spread of new ideas
· communication across clusters
· thematic discourse analysis
· relationship with main- stream media coverage
Methodology – Twapperkeeper
Analysis
Capture
Identification
#Hashtag Archive
Tweet Statistics and
@Replies
Patterns of Activity over
Time
Networks of @Replies
(short/long term)
Tweet Texts
Keyword /Key PhraseMapping
Data Processing – Twitter
• Typical data structure (#ausvotes):
Data Processing – Twitter
• Tools:
• Gawk – Scripting tool für CSV processing (open source)
• Excel – Data aggregation, pivot tables and charts
• Leximancer / WordStat – Keyword extraction, co-occurence matrices
• Gephi – Network analysis and visualisation (open source)
# Extract @replies for network visualisation## this script takes a CSV archive of tweets, and reworks it into network data for visualisation## expected data format:# text,to_user_id,from_user,id,from_user_id,iso_language_code,source,profile_image_url,geo_type, # geo_coordinates_0,geo_coordinates_1,created_at,time## output format:# from,to,tweet,time,timestamp## the script extracts @replies from tweets, and creates duplicates where multiple @replies are# present in the same tweet - e.g. the tweet "@one @two hello" from user @user results in# @user,@one,"@one @two hello" and @user,@two,"@one @two hello"## Released under Creative Commons (BY, NC, SA) by Axel Bruns - [email protected]
BEGIN {print "from,to,tweet,time,timestamp"
}
/@([A-Za-z0-9_]+)/ {
a=0 do {
match(substr($1, a),/@([A-Za-z0-9_]+)?/,atArray)a=a+atArray[1, "start"]+atArray[1, "length"]
if (atArray[1] != 0) print $3 "," atArray[1] "," $1 "," $12 "," $13
} while(atArray[1, "start"] != 0)
}
# filter.awk - Filter list of tweets## this script takes a CSV or other list of tweets, and removes any lines that don't include RT @username# the script preserves the first line, expecting that it contains header information## script expects command-line argument search={searchcriteria} _before_ the input CSV filename# enclose the search term in quotation marks if it contains any special characters## e.g.: gawk -F , -f filter.awk search="(julia|gillard)" tweets.csv >filteredtweets.csv## expected data format:# CSV or simple list of tweets, line-by-line## output format:# same as above, listing only retweets## Released under Creative Commons (BY, NC, SA) by Axel Bruns - [email protected]
BEGIN { getline print $0
}
tolower($0) ~ search {
print $0
}
2010
-Jun-
23 0
0
2010
-Jun-
23 0
1
2010
-Jun-
23 0
2
2010
-Jun-
23 0
3
2010
-Jun-
23 0
4
2010
-Jun-
23 0
5
2010
-Jun-
23 0
6
2010
-Jun-
23 0
7
2010
-Jun-
23 0
8
2010
-Jun-
23 0
9
2010
-Jun-
23 1
0
2010
-Jun-
23 1
1
2010
-Jun-
23 1
2
2010
-Jun-
23 1
3
2010
-Jun-
23 1
4
2010
-Jun-
23 1
5
2010
-Jun-
23 1
6
2010
-Jun-
23 1
7
2010
-Jun-
23 1
8
2010
-Jun-
23 1
9
2010
-Jun-
23 2
0
2010
-Jun-
23 2
1
2010
-Jun-
23 2
2
2010
-Jun-
23 2
3
2010
-Jun-
24 0
0
2010
-Jun-
24 0
2
2010
-Jun-
24 0
3
2010
-Jun-
24 0
4
2010
-Jun-
24 0
5
2010
-Jun-
24 0
6
2010
-Jun-
24 0
7
2010
-Jun-
24 0
8
2010
-Jun-
24 0
9
2010
-Jun-
24 1
0
2010
-Jun-
24 1
1
2010
-Jun-
24 1
2
2010
-Jun-
24 1
3
2010
-Jun-
24 1
4
2010
-Jun-
24 1
5
2010
-Jun-
24 1
6
2010
-Jun-
24 1
8
2010
-Jun-
24 1
9
2010
-Jun-
24 2
0
2010
-Jun-
24 2
1
2010
-Jun-
24 2
2
2010
-Jun-
24 2
3
2010
-Jun-
25 0
0
0
1000
2000
3000
4000
5000
6000
7000
Total
Prelude: Leadership #spill
23 June, 19:00-00:00:
Speculation
24 June, 08:00-15:00:
Party Vote & Aftermath
#spill Discussion Network (Node size: indegree [most @replies received]; node colour: outdegree [most @replies sent])
#ausvotes: Overall Activity (17 July – 24 Aug. 2010)
#ausvotes: Discussion Network(17 July to 25 Aug. 2010 / All @replies / Node size: Indegree / Node colours: betweenness centrality)
#ausvotes: Mentions of the Parties (normalised per day)
#ausvotes: Mentions of the Leaders (normalised per day)
#ausvotes: Mentions of the Leaders (cumulative)
#ausvotes: Key Themes
#ausvotes: Key Themes (normalised per day)
#ausvotes: Distractions (normalised per day)
Labor’s Twibbon Campaign RTs
#ausvotes: Distractions
Notes and Limitations
• Twapperkeeper relies on #hashtags
• Problem if #hashtags are inconsistent/unclear
• Follow-on @replies and retweets may not continue to use #hashtags
• Casual commenters may not use #hashtags in the first place
• May miss early developments – e.g. #hashtag standardisation
• Twitter as a subset of society:
• Broadband policy and Internet filter over-, asylum seekers underrepresented
• #hashtag use is a further sign of self-selection
• Need to look to Twitter firehose for more comprehensive picture
• Need to track baseline activity to understand how exceptional #ausvotes was
• See more at mappingonlinepublics.net – up next: time-based animations...
• Or find us at @snurb_dot_info and @jeanburgess
http://mappingonlinepublics.net/
@snurb_dot_info@jeanburgess
Image by campoalto