social media academy 2016 keynote presentation, john wihbey

Defining a Digital Storytelling Discipline: Learning, Skills, and Knowledge

John Wihbey Northeastern University

@wihbey

Case study: Northeastern

University undergrads working on Boston Police Department data, “as is” - in a general digital skills

course

Murder data from the 1960sCity of Boston - Homicide data obtained through public

records request

Text text text

Murder data from the 1990sCity of Boston - Homicide data obtained through public

records request

Murder data from the 2000sCity of Boston - Homicide data obtained through public records

request

(http://the-accidental-housewife.blogspot.com/)

29 problems. 1 assignment.

List of problems/errors in structure and format of homicide data

1. Inconsistencies in case column, e.g. “01/06” vs “ ’09/06 ”2. No indication of meaning of red text3. No key for case column IDs4. Different text formats/styles for entire rows and cells5. Inconsistent descriptions of intersection addresses, e.g. “Washington @ Cedar St” vs “Willowood & Woodrow Ave” vs “Shawmut Ave. / Dwight”6. No key for weapon codes7. Race and gender are collapsed into single column8. No codes for race/gender (race: “W”, “B”, gender: “M”, “F”, “H”)9. Some R/G codes are “W/H/M”, “B/N/H” making it impossible to systematically split columns into 2 using the only delimiting character (/)10. Some R/G codes have NO delimiter (e.g., 2009 sheet), so cannot split at all11. Data for 2007 and later have two additional columns not in 2006: “defendant” and “DOA” (no indication of what DOA means)12. Some rows have merged cells13. Some merged cells have multiple values14. Missing data/empty cells – what do these mean?

List of problems/errors in structure and format of homicide data (cont.)

15. Location data is incomplete – no zip code information and Boston assumed as city (except in cases where “Dor” is appended at end of address)16. Only first couple of sheets have column header information; column headers have to be assumed for remaining ones to follow those with labeled headers17. Mysterious unexplained extra characters in date columns (e.g., (w) and xxx)18. Inconsistent syntax for times: 12:00am, 7:10pm, 02:16am, 2:56hrs, 15oo hrs (Letter “o” instead of number 0), 1:49 AM, 7:24 PM, 21:25, 21:32 PM, 12:39P.M., 2:22p.m., 1:49:pm, 12;24AM, :14 am19. Inconsistent syntax for dates: “07/24/2006” vs “2006/7/24” vs “6/31/64” vs “08/31/06”20. Inconsistent syntax for age: “1’7”21. Sheets for 2012/13/14 have new columns not in previous sheets22. Motive/Relation columns look identical but are not both present in all sheets, impossible to know which labels are which in those sheets without column headers23. Simple spelling errors: “Tauma”24. Inconsistent coding: Unk, UNK, unk25. Unexplained “DV” column that only appears in 201326. No explanation for meaning of row breaks – are these separating data rows into groups of some sort? Are these data that one existed but were removed?27. Multiple columns with same (non-unique) headers – “R/G”, “Age”, “DOB” for both VICTIM and DEFENDANTS28. Inconsistent district labels and squadron personnel names29. For cells with multiple data/names in cell merged column, have to assume respective values in adjacent cells are provided in same order

Existential experience of: #datafail & #GIGO risk

Good student requests for clarification

Many noble student attempts at cleaning, analysis, exploration,

viz:

Data viz using Plot.ly

Data viz using Carto

Google Maps

Fun with line graphs - an attempt to look at time-of-day patterns

Experiments in viz for exploratory purposes

D'où Venons Nous / Que Sommes Nous / Où Allons Nous - Paul Gauguin, 1897

2000

2005

2016

Wikimedia

1774

(Nielsen 2006 via www.nngroup.com)

G. Chaucer, The Canterbury Tales (courtesy: library.arizona.edu)

1400s

https://projects.propublica.org/docdollars/

Fatal Force https://www.washingtonpost.com/graphics/national/police-shootings/

http://www.nytimes.com/interactive/2012/01/15/business/one-percent-map.html

http://www.npr.org/news/graphics/2011/10/toxic-air/#4.00/39.00/-84.00

https://offshoreleaks.icij.org/#_ga=1.76851094.2020983486.1475355003

http://projects.latimes.com/value-added/

Labnol.org

Pew Research Center

Polarized Crowd: Two large dense groups with little interconnection

Pew Research Center

Tight Crowd: Highly interconnected group with few isolated participants

Pew Research Center

Brand clusters: Products, services, celebrities discussed by disparate persons

Pew Research Center

Community clusters: Popular topics attracting multiple smaller groups

Pew Research Center

Broadcast networks: Media-centric, with audience proliferating information

Pew Research Center

Support network: Customer complaints, with hub-and-spoke dynamics

Six degrees - Wikimedia

Facebook friends network

Nebraska local politicians; network graph - Matt Waite

http://www.poynter.org/2013/how-to-visually-explore-local-politics-with-network-graphs/218543/

Chicago community - homicide network (Andrew Papachristos et al.)

NYTimes.com

Gary King, et al, IQSS, Harvard

Global supply chain, Sourcemap.com

Opte Project

D'où Venons Nous / Que Sommes Nous / Où Allons Nous - Paul Gauguin, 1897

John Wihbey Northeastern University

@wihbeyj.wihbey@northeastern.edu