design and prototyping of a social media observatory

31
Design and Prototyping of a Social Media Observatory Karissa McKelvey and Filippo Menczer Center for Complex Networks and Systems Research School of Informatics and Computing Indiana University, Bloomington 1

Upload: karissa-rae-mckelvey

Post on 27-Jan-2015

105 views

Category:

Technology


1 download

DESCRIPTION

Web Observatory Workshop at the International World Wide Web Conference, 2013.

TRANSCRIPT

Page 1: Design and Prototyping of a Social Media Observatory

1

Design and Prototyping of a Social Media Observatory

Karissa McKelvey and Filippo Menczer

Center for Complex Networks and Systems ResearchSchool of Informatics and Computing

Indiana University, Bloomington

Page 2: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 2

Can we use social media as laboratories for social

science?

Page 3: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 3

Political Polarization on Twitter Michael Conover, Jacob Ratkiewicz, Bruno Gonçalves, Alessandro Flammini & Filippo Menczer International Conference on Weblogs and Social Media 2011

Page 4: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 4

Data “mine” ing

Page 5: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 5

Data “ours” ing

Page 6: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 6

truthy.indiana.edu

Page 7: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 7

Design and Prototyping

Page 8: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 8

Social Mediastreaming

sensitive

very large

structured

multiple sources

Page 9: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 9

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 10: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 10

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 11: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 11

Reliability

• Spam and misinformation

Page 12: Design and Prototyping of a Social Media Observatory

celebrities

spam

astroturf

politics

12

Page 13: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 13

Reliability

• Spam and misinformation

• Cleansing and tagging by social, algorithmic, or other means

Page 14: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 14

Page 15: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 15

Reliability

• Spam and misinformation

• Cleansing and tagging by social, algorithmic, or other means

• Sampling bias

Page 16: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 16

Data Collection

• Twitter Streaming API, random sample

• August, 2010 – present

• 5TB Compressed

• Real-time access to data from last 9 months related to 3 themes: US Politics, Social Movements, News

Page 17: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 17

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 18: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 18

Reproducibility

• Standard ontology for the storage, reference, and transfer of these datasets between users.

Page 19: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 19

Model• Events– Post on a social media site

• Users– Actors in the post– Sender, receiver, forwarder, etc

• Meme– Discernable unit of information transfer– Eg, hashtag, URL, user, phrase…

Page 20: Design and Prototyping of a Social Media Observatory

20

Meme Diffusion Networks

truthy.indiana.edu

Retweet (forward) Mention (conversation)

Page 21: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 21

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 22: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 22

Page 23: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 23

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 24: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 24

Page 25: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 25

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 26: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 26

Open Access

• A public and free -- or low-cost -- social observatory enables access to large-scale social media data analytics for non-profit endeavors.

Page 27: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 27

API & Website

• Filterable and searchable interface to find memes of interest

• Endpoints for users to access data programmatically

• Visualizations

truthy.indiana.edu/apidoc

Page 28: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 28

Design Considerations

ReliabilityReproducibilityTopic FilteringVisualizationOpen AccessLegal Compliance

Page 29: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 29

Legal Compliance

• Terms of Service– Tweet text– Derived data

Page 30: Design and Prototyping of a Social Media Observatory

truthy.indiana.edu 30

truthy.indiana.edu

Page 31: Design and Prototyping of a Social Media Observatory

Thanks!