1New Mexico Press Women – April 2010 -- Albuquerque

Doing the Numbers:

Getting the other 50% of the story

Tom JohnsonInstitute for Analytic Journalism

Santa Fe, New Mexicowww.analyticjournalism.com

t o m @ j t j o h n s o n . c o m

2

Today?

Theory is your friend Arithmetic is your friend Examples of sophisticated

analytic journalism Fundamental analytic tools Using Excel to give stories

context

3

“Theory?” said the journalist. “I don’ need no stinkin’ theory: I’m a WRITER!”

4

There is nothing more practical than good theory

Nothing more practical

5

Ecosphere as theory

The ecosphere is a theoretical concept, place, process

6

Species in biosphere

MetabolismLatent EnergyIn

EnergyOut

Species in Biosphere:

7

Datasphere

That theoretical place where all data -- in all its forms -- exists

8

Journalists, et al., in Datasphere

Changing Datasphere:

DataIn Analysis Info

Out

MetabolismLatent EnergyIn

EnergyOut

Species in Biosphere:

9

Changing Datasphere

Shift from I-o-P to 1s and 0s RRAW-P process model:

ResearchReportingAnalysisWritingPrint/Publish/Produce

Journalism is no longer breaking news; no longer just telling stories

10

Changing Datasphere

Journalism’s future =Hub for transactions

Journalists’ future = bringing added value + enriching connections

Analysis and explanation of data (while telling the stories)

Requires new skills

11

Major factors in changing datasphere

Vast quantities of digital data in various formats

Relative ease to get that data New tools to find, retrieve,

analyze and present that data, especially quantitative data

12

New demands for journalists

High-level skills: search, retrieval, revisiting of data and data sites

Reasonable skills in quantitative analysis Basic statistics Basic visual statistics

13

No more math-phobes

No, it is NOT “all about story.”

Little – if any -- room today for the mathematically phobic

14

The quant. half of the story

ANY topic can have a quantitative angle

“Calculus for journalists” Change in amount Change in rate

15

The quant. half of the story

1. Estimate2. Gather the data3. Clean the data (See Ver 1.0)

4. Count Sort, note range (high/low,

median, mean, mode)

5. Measure Proportion Change

16

Fundamental Tools

Search, Save and Retrieve

GIS (Geographic Information Systems)

Data base applications Spreadsheet application

Excel OpenOffice Calc, et al. (free)

17

Examples of heavy-weight Analytic Journalism…

18

USAToday - Analytic Journalism Examples

NB: Great site of 2008 Presidential election maps: http://kottke.org/plus/2008-election-maps/

19

Orange County, California Bus Study - Analytic Journalism Examples

Orange County, California

Bus Study

20

Weather-Hurricane Andrew - Analytic Journalism Examples

Weather - Hurricane Andrew

21

Light-weight examples (to produce daily stories)

Resist journo’s tradition “If it’s stats, they must be correct.”

Remember: “All data is dirty”

“I KNOW what I’m doing!”Don’t expect to do all this alone; enlist colleague/editor/data rabbi

“I have a good memory.”Keep a “lab book” of methods

“Yeah, I always make my deadline.”Always sleep on data story

22

23

Fundamental Tool

Spreadsheets can Help understand phenomena, often

in context Uncover new stories and angles on

old stories Help sell stories to your editor Help tell stories to your readers

Easy sorting of data Basic to high-end statistics Visual graphics tools

24

Einstein card

Create a “lab book” Calculating Percentages Calculating Proportion Calculating Rates Calculating Ratios Calculating Inflation

25

Freeport Traffic Horizontal #1

26

Freeport Traffic Horizontal #1aStep 1a: “Clean” your data

Step 1b: “Interrogate” the data/database

Step 2: Source; Where did the data come from?

Step 3: What are the definitions?Have they remained constant?

Step 4: ESTIMATE the hard-data difference as real numbers and percent

Step 5: Calculate the differences.

27

Freeport Traffic Horiz #2

28

Freeport Traffic Horiz #3-Act. Diff

•“Intero-gate” the data

•Potential stories?

•What next?

29

Freeport Traffic Horiz #4a - % of change#1 - Estimate!

#2 - Look to the Einstein Card!

30

Freeport Traffic Horiz #4b - % of change

31

Vertical Analysis- ProportionProportion= # of Parts/# of Whole Parts

32

Vertical Analysis- Proportion

33

Writing the darn thing

Pay attention to the difference between million and billion

Write like time flows – from left to right; from past to present

Three levels of using numbers

34

Three levels when writing stats…

Data differences: 1-100? Use hard numbers

Data differences: 100-1,000? Use hard numbers and percent

Data differences >1,000? Use percentages

35

Writing the darn thing Pay attention to the difference

between million and billion Write like time flows – from left

to right; from past to present Three levels of using numbers

In a story with stats, do not use the word “significant” casuallyDon’t confuse “correlation” with “causality

36

Prepare for tomorrow

Join www.ire.org Subscribe to the IRE and NICAR

listservshttp://www.ire.org/membership/listserv.html

Think about “dashboards” (free tutorial) http://www.excelcharts.com/blog/products/demographic-dashboard/

GIS (more free tutorials)http://herzoggis.notlong.comhttp://www.esri.com/industries/media/business/journalism.html

ESRI data to IRE members at no cost.http://ow.ly/1nPf6

37

Semi-advanced tips Excel Data Analysis For Dummies

http://www.dummies.com/store/product/Excel-Data-Analysis-For-Dummies.productCd-0764516612.html

Ten ways to clean your datahttp://office.microsoft.com/en-us/excel/HA102218401033.aspx

Text editing with spreadsheetshttp://forjournalists.com/cookbook/index.php?title=Text_editing_with_spreadsheets

Analytic Journalism and Political Races: Campaign Pollshttp://ajandpolraces.pbworks.com/Campaign-Polls

Excel Pivot Tables Tutorial: What is a Pivot Table and How to Make onehttp://chandoo.org/wp/2009/08/19/excel-pivot-tables-tutorial/

Distinctions between Excel and a database; Excel cautions and fixes at…http://www.stanford.edu/class/hrp223/2009/collection/collection_223_20090926.ppt

38

This presentation – with hot links – found at:

www.slideshare.net/jtjohnson

39New Mexico Press Women – April 2010 -- Albuquerque

Doing the Numbers:

Getting the other 50% of the story

Tom JohnsonInstitute for Analytic Journalism

Santa Fe, New Mexicowww.analyticjournalism.com

t o m @ j t j o h n s o n . c o m