Tonight’s Quantitative Thinking Presentation

(Theme TBA Later)

Michael HansenSt. Albans School of Public Service

Washington, DCJune 21, 2007

“There are three kinds of lies: Lies, d______ed lies,

and statistics.”—attributed

to Mark Twain

My background . . .

• Math and music (piano, violin, viola)• M.S., Applied Mathematics, UIUC• 1986–1998: U.S. Government

contractor, primarily in the Pentagon• 1998–present: STA math teacher

Icebreaker Question

• In what city was I born? (Hint: It is one of the ten largest cities in the U.S.)

Your Backgrounds . . .

• Tell me something unusual that is not in your capsule bio

Which of these subjects are you most likely to use later in

life?

Geo

met

ry

Alg

ebra

Sta

tistic

s

Cal

culu

s

6%

17%

67%

11%

1. Geometry2. Algebra3. Statistics4. Calculus

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Why are geometry, algebra, and calculus taught in high

school?

Good

ness

only

knows.

Life

is c

ruel

.

Mat

h teac

hers

are

gr...

Thes

e su

bjec

ts p

rovi

..

0%

61%

17%22%

1. Goodness only knows.

2. Life is cruel.3. Math teachers are

greedy and want full employment.

4. These subjects provide a chance to teach abstract reasoning skills.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

mortar : brick :: glue :

sto

ne

wood

wat

er oil

6%0%

6%

89%1. stone2. wood3. water4. oil

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

arachnid : spider :: marsupial :

liza

rd

kan

garoo

fish

bee

tle

0% 0%

17%

83%1. lizard2. kangaroo3. fish4. beetle

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

What have we been doing here?

• Answer: Gathering statistics.• What is a statistic?• A number computed from data.

Which of these are examples of statistics?

• A batting average• A vote tally for a political candidate• A census (headcount) for a community• A person who is injured in a traffic accident• The statement, “Most Americans support our

troops in Iraq”• The statement, “51% of the adult voters

polled stated that the war in Iraq has been a mistake”

• The statement, “51% of adult American voters think the war in Iraq was a mistake”

Terms• Data = facts (plural); one fact is a datum• Statistic = a number computed from data• Parameter = a number that describes a

population• Key idea: We use statistics to estimate

parameters.• Bias = error in parameter estimation that is

systematic (i.e., tending to one side or the other)

• Sampling error = the inevitable result of trying to estimate parameters with a sample that is smaller than the entire population

• m.o.e. = an estimate of sampling error (m.o.e. shrinks as sample size grows, assuming you have a random sample)

• Note: Bias is avoidable, at least in theory. Sampling error is not.

Fun Chart: Who wins thiselection? (5.8 million votes

cast)

0 1,000,000 2,000,000 3,000,000

Candidate B

Candidate A

Fun Chart: Who wins thiselection? (5.8 million votes

cast)

0 1,000,000 2,000,000 3,000,000

Candidate B

Candidate A

Statistics:

Because of errors in the tabulation process, the m.o.e. for each candidate is at least several thousand votes, possibly as high as 60,000 votes.

The initial tally shows Candidate A with a lead of 537 votes over candidate B.

Who do you think won?

Can

didat

e A, b

y 53

7 ...

Can

didat

e A, m

ainta

i...

Can

didat

e B, p

ulling... T

ie

17%

6%

44%

33%

1. Candidate A, by 537 votes (no recount)

2. Candidate A, maintaining a lead even after the recount

3. Candidate B, pulling ahead after the recount

4. Tie

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Tonight’s Theme(Try to guess it from these examples.)

• Space aliens stare at Earth through powerful telescopes. They observe that most of the people buying diet soda are a bit overweight, to put it mildly. Conclusion: Diet soda makes people fat.

• A teacher enforces discipline in the classroom by deducting one point for each minor infraction (burping, tardiness, etc.) and tells a student who has lost several points, “Your actions have lowered your grade from A minus to B plus.”

• Terrorists hijack airplanes and crash them into buildings, killing several thousand Americans. Spokesmen for the terrorists, as well as a number of international commentators, say that U.S. government policies are to blame.

• A Roman army unit performs poorly in battle, with several of the men deserting in fear. After the battle, the commander decimates the troops (i.e., kills every tenth man present). The troops learn that if they desert, they will be sealing a death warrant for their friends still in the unit.

• A village under Nazi occupation revolts. In reprisal, the Nazis murder nearly all the villagers, but they also lay waste to several neighboring villages. The revolts quickly cease.

• A sheriff running for re-election proudly states, “During my four years in office, the violent crime rate has dropped by 28%. Vote for me to continue the progress!”

What is the theme?

Spac

e al

iens

are

not .

..

Corre

latio

n does

not

...

Dis

torti

on of c

ause

a..

How to

lie

with

sta

tistic

s

0%6%

29%

65%1. Space aliens are

not very bright2. Correlation does

not imply causation

3. Distortion of cause and effect

4. How to lie with statistics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Cause and Effect• A most interesting subject: Even babies are

fascinated by it!• All public policy decisions hinge on cause

and effect• Warning: Remember the “Law of Unintended

Consequences” (examples of refrigerator subsidies, Australian rabbit invasion, AFDC)

• Difficulties:– Mathematics has nothing to say on the subject.– We must turn to the relatively new science of

statistics.– There is only one way to prove cause and effect,

and the conditions are rarely met.– But . . . that does not stop politicians, journalists,

and activists from asserting cause-and-effect relationships.

Real-World Questions to Ponder

• Do CO2 and other greenhouse gases cause global warming?

• What fraction of global warming is caused by human activities (e.g., burning of fossil fuels)?

• Will changing our behaviors cause a reduction or reversal of global warming?

• What government policies on current and alternative fuels will cause changes in people’s behavior?

• To what degree do U.S. energy policies cause changes in other countries’ economies or behaviors?

Making it Real—Cause and Effect in the News

• Do mercury-based preservatives in childhood immunizations cause autism? Many parents whose children suddenly became autistic shortly after having an immunization are convinced that this is so.

• Did Vioxx cause heart attacks and strokes? If so, should Merck be required to pay damages to patients who suffered a heart attack or stroke after taking Vioxx?

• Does second-hand cigarette smoke cause lung cancer or other diseases? If so, should smoking be banned in all indoor locations?

• Do American high school classrooms cause boys to lag behind girls academically? If so, should curricula and teaching styles be revamped? What would be the effects of doing that?

• Do silicone implants cause lupus and other autoimmune disorders in women?

• Did Zicam cause people to lose their senses of taste and smell?

• Do cell phones cause automobile accidents?

Some Common Fallacies• Anecdotal data• Emotional appeals or ad hominem attacks• COI, appearance of COI, or accusations of COI• “Post hoc, ergo propter hoc” (roughly

translated: “after this, therefore caused by this”)

• Similar: Correlation confused with causation• Extrapolation (i.e., assuming trends will

continue)• Overanalyzing time series of uncontrolled

systems (e.g., trying to predict the stock market by using “technical analysis”)

Are there any people who know what they’re doing?

• Yes, a few.• Everyone should take a course in

statistics.• A statistics course is one of the few

courses where you are unlikely to study any actual statistics! (You will study the science and practice of statistics.) Actual statistics are seen in nearly all other courses: history, biology, sociology, etc.

“Of the 1300 randomly chosen adult American voters who were polled, 38% were satisfied with President

George W. Bush’s performance in office. The margin of error is plus or minus 3%.”

I kn

ow exa

ctly

wha

t t..

I kn

ow quite

wel

l wha.

..

I do n

ot know w

hat th

...

I do n

ot eve

n know ..

.

53%

6%6%

35%

1. I know exactly what the statement means.

2. I know quite well what the statement means.

3. I do not know what the statement means.

4. I do not even know whether or not I know what the statement means.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

The

true

param

eter

is...

The

true

stat

istic

is ..

.

The

true

param

eter

is...

The

true

stat

istic

is ..

.

12%18%

71%

0%

1. The true parameter is between 35% and 41%.

2. The true statistic is between 35% and 41%.

3. The true parameter is probably between 35% and 41%.

4. The true statistic is probably between 35% and 41%.

“Of the 1300 randomly chosen adult American voters who were polled, 38% were satisfied with President

George W. Bush’s performance in office. The margin of error is plus or minus 3%.”

Exactly what does this statement mean?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

On the previous slide, why did the correct answer involve the word “probably”?

sam

pling e

rror

bia

s

lack

of r

andom

ness

in...

sam

ple s

ize w

as to

o...

82%

6%6%6%

1. sampling error2. bias3. lack of randomness

in the sample4. sample size was too

small for a believable survey of the entire nation

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

If you are making public policy decisions, why should you take nearly all news reporting (as

opposed to news analysis or commentary) with a grain of salt?

sam

pling e

rror

the

most

acc

urate

rep.

..

med

ia b

ias

anec

dotal

dat

a

0%

24%

65%

12%

1. sampling error2. the most

accurate reporting is usually internal to the government, not coming from the news media

3. media bias4. anecdotal data1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

How to Talk Back to a Statistic

Source: Darrell Huff’s classic 1950’s book, How to Lie With Statistics

• Who says so? (Beware of COI.)• How does he or she know? (What methods

were used to compute the statistic? Many numbers are simply unknowable in a practical sense.)

• What’s missing? (Ask what the m.o.e. is, what assumptions were used, what time period was used, and whether there was a control group.)

• Did someone change the subject? (Beware of “semiattached data,” “gee-whiz” graphs, and extrapolation.)

• Does it make sense? Modern-day buzzterm: FACE VALIDITY.

Regarding Face Validity . . .• Use your common sense and read critically. Even

reputable sources contain errors.• Compute ratios or per capita values. (For any

national budget number, simply take the number of billions and multiply by 3 or 4. Example: A $50 billion program is costing each adult in the country about $200.)

• Excerpt from The Washington Post Magazine on April 2, 2006 (posted on their website):

Adult education is thriving nationwide, with more than 92 million adults taking college classes. At the nearly 70 two- and four-year colleges in the Washington area, an estimated 175,000 adults are enrolled, 40 percent of them on a part-time basis.

• Which statistic in this excerpt lacks face validity?

What is a statistic?

a n

umbe

r that

des

crib

..

a fa

ct

a n

umbe

r com

pute

d ...

a p

erso

n who

is th

e vi

...

0% 6%

94%

0%

1. a number that describes a population

2. a fact3. a number computed

from data4. a person who is the

victim of crime or an accident

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

What is a parameter?

a n

umbe

r that

des

crib

..

a fa

ct

a n

umbe

r com

pute

d ...

an a

djust

able

cons

tan..

94%

6%0%0%

1. a number that describes a population

2. a fact3. a number computed

from data4. an adjustable constant,

or a boundary condition for a problem

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

What is the only way to establish cause and effect?

a c

aref

ul obse

rvat

iona.

..

a c

aref

ul obse

rvat

iona.

..

an e

xper

imen

t

a c

ontro

lled e

xper

imen

t

0%

82%

18%

0%

1. a careful observational study

2. a careful observational study with a sufficiently large sample size and freedom from bias

3. an experiment4. a controlled experiment

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

On a scale of 1 to 7, please rate how you feel about this statement: “I am planning to take a statistics course sometime within the next three years.” (Mark 7 if you

have already taken a statistics course in high school, e.g., AP Statistics.)

Stro

ngly D

isag

ree

Dis

agre

e

Som

ewhat

Dis

agre

e

Neu

tral

Som

ewhat

Agre

e

Agre

e

Stro

ngly A

gree

6% 6%

0%

24%

35%

18%

12%

1. Strongly Disagree2. Disagree3. Somewhat

Disagree4. Neutral5. Somewhat Agree6. Agree7. Strongly Agree1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Q and A

Thank you for your time and attention!

Michael Hansen(e-mail: modd “at sign” modd.net)St. Albans School of Public Service

Washington, DCJune 21, 2007