Web and Database Tools:Investigative Reporting

Steven Rich, IRE and DocumentCloud

@thestevenrich

#SABEW #drillingdeep

SYW

• Show Your Work

• Data is not a dirty word

• People want to see documents

• Readers love interactivity

• Journalists as gatekeepers still exists but...

• We need a wider gate, especially with investigative pieces

Investigating the Energy Industry

• Must read: “The History of the Standard Oil Company” by Ida Tarbell

• Ida only used words, but who knows what she'd do now?

• If you have the tools, use them

• It's a complicated subject

• If you don't use it (tools)

• You risk losing it (readers)

Types of Tools

• Audio, Photo and Video

• Google

• Document Presentation

• Web Scraping

• Data Wrangling

• Database Managers

• Data Visualization

• Mapping

• Advanced → DIY

The best things in life

ARE FREE

Audio, Photo and Video

• Audioo Audacity

• Photoo Gimp

• Videoo iMovieo Open Movie Editor

Google

•Advanced Search

• Insights for Search

•Trends

•Public Data Explorer

•Google Fusion

•Always coming out with more

DocumentCloud

• Upload your documents

• Analyze, annotate and publish

• Turn your documents into data

• Used in newsrooms across the world

• See others' documents

• Request an account today: free

• Request an account any other time: free

Data Wrangling

• Just because someone gives you data, it doesn't mean it's clean

• If it's not, there are solutions

• Mr. People

• Google Refine

• Mr. Data Converter

• CSV to HTML

Database Managers

• SQLiteo SQLiteMan

• MySQL

• Libre Office (Calc)

• PostgreSQL

Web Scraping

• There's a lot of data out there, but it's not always easy to get

• Always easier to ask for data than to scrape it

• Scraping is good for constantly updating data

• Make friends with your developers

• Does require some programming language

• But Google can help

Mapping/Data Viz

• The energy industry affects many

• Mapping helps answer “so what?” for more

• Google Fusion

• It'll do the trick, but if you want personalization

• Cloudmade/Leaflet

• Mapstraction

• Tableau Public

Advanced

• Looking to customize user experience?

• Like to play around with code?

• APIs

• Django CMS

• Ruby on Rails

• Github

Questions?

Download slides: http://bit.ly/SABEWtools

@thestevenrich | http://stevenri.ch

serich@mizzou.edu