by Ömer s. benli ethics across the curriculum 2006

by Ömer S. Benli

Ethics Across the Curriculum2006

http://www.csulb.edu/colleges/cba/ethics/

© Copyright 2006, Ukleja Center for Ethical Leadership, CSULB.

Mutant Statistic (or, how bad statistics live on and take on lives of their own)

• “Every year since 1950, the number of American children gunned down is doubled.”

• “The number of American children killed each year by guns has doubled since 1950.”


“Every year since 1950, the number of American children gunned down is doubled.”

• appeared in a Ph.D. dissertation proposal,

• which was a direct quotation from and articles published in a refereed journal in 1995.


• Year Number of Children• 1950 1• 1951 2• 1952 4• 1960 1,024• 1965 32,768 FBI reported 9,960 total homicides• 1970 >1,000,000• 1980 >1,000,000,000 > 4 US population• 1983 8.6 Billion 2 population of Earth• 1995 35 Trillion

“Every year since 1950, the number of American children gunned down is doubled.”


The correct source

• “The number of American children killed each year by guns has doubled since 1950.”

— 1994 Children’s Defense Fund

• The US population has increased by 73% over the same period of time.


“Things to Look Out For”

• Where did the data come from? Who ran the survey? Do they have an ulterior motive for having the result go one way?

• How was the data collected? What questions were asked? How did they ask them? Who was asked?


“Things to Look Out For”

• Be wary of comparisons. Two things happening at the same time are not necessarily related, though statistics can be used to show that they are. This trick is used a lot by politicians wanting to show that a new policy is working.

• Be aware of numbers taken out of context. This is called 'cherry-picking', an instance in which the analysis only concentrates on such data that supports a foregone conclusion and ignores everything else.


on the lighter side …

• 47.3% of all statistics are made up on the spot.

- Steven Wright

• Think about how stupid the average person is; now realise half of them are dumber than that.

- George Carlin


Appeal of round numbers and the problem with predetermined levels

• Our decimal system, is the “artifact” of humans having ten fingers to count! If Martians have eight fingers, they must be using “octal” system.

• It is customary that in statistical analysis “round numbers” are used for significance level. What does a 10%, or 5%, or 1% level of significance really mean?


“How not to lie with statistics: avoiding common mistakes in

quantitative political science” King [1986] • “I know of no […] research in which it makes sense to

use a precise critical value. Any [result] that is significant at 0.05 level is as useful in this discipline as it were 0.06 or 0.04. To [reverse the conclusion of the analysis because the result] is .01 or 0.001 above [or below] a significance level makes little sense. Even if the author has a reason for it, at least the readers could be permitted to come to their own conclusions. My recommendation is to present the [p-value], regardless of what it is; the author can argue whatever he or she wants and readers would still be able to draw their own conclusions.”


one- vs. two-tail hypothesis tests or confidence intervals

• When a test statistic is eventually computed, a two-tail test (say at 10% significance level) is reduced to a one-tail test (at 5% significance level.) Are we to interpret this as a test that started as being “significant” and ending up as “very significant”?


The major problem with hypothesis testing is• that it does not tell us what we want to

know, which is: “Given this sample data (D), what is the probability that Ho is true?”

• But it tells us: “Given that Ho is true, what is the probability that we can get this (or even more “extreme’) sample data (D)?”

• these are not the same things simply because P{Ho | D} ≠ P{D | Ho}.


“Power Analysis”

Types of error in hypothesis tests: • Type-I, α, rejecting the null hypothesis

when the null hypothesis is true; and • Type-II, β, not rejecting the null hypothesis

when the null hypothesis is false. • Power of the test is the probability of not

making Type-II error;’ that is, “the ability of your statistical test to detect true differences of a particular size.”


Power of the test depends on

• sample size, • variability in the sample, • the specified α level, and • the “effect size” one wants to be able to

detect (that is, the hypothesized value of the alternative hypothesis).

Usually one specifies the last three of these parameters and then determines the proper sample size.


p-value

• indicates the probability of a particular set of data being generated by the null model

• it has little to say about size of deviation from model

• especially in the tails of the distribution, where large changes in effect size cause only small changes in p-values.


Use and Misuse of Conditional Probabilities

• “Beware of German tourists” (according to Der Spiegel magazine, most foreign skiers involved in accidents in a Swiss skiing resort came from Germany).

• “Boys more at risk on bicycles” (the newspaper Hannoversche Allgemeine Zeitung reported that among children involved in bicycle accidents the majority were boys).

• “Soccer most dangerous sport” (the weekly magazine Stern commenting on a survey of accidents in sports).

• “Private homes as danger spots” (the newspaper Die Welt musing about the fact that a third of all fatal accidents in Germany occur in private homes).

• “German shepherd most dangerous dog around” (the newspaper Ruhr-Nachrichten on a statistic according to which German shepherds account for a record 31% of all reported attacks by dogs).

• “Women more disoriented drivers” (the newspaper Bild commenting on the fact that among cars that were found entering a one-way-street in the wrong direction, most were driven by women).


The problem with these is the same as in the classical example

• “Four times more fatalities occur on highways at 7 p.m. than at 7 a.m.; thus it is more dangerous to drive in the evening than in the morning!”

• Clearly, this implication is not correct, since

P(accident | 7 p.m.) ≠ P(7 p.m. | accident).


“Four times more fatalities occur on highways at 7 p.m. than at 7 a.m.”

• is

P(7 p.m. | accident) > P(7 a.m. | accident);

which most likely is true.

• However, it is not correct to replace the event “7 p.m.” by the event “accident” and assume

P(accident | 7 p.m.) > P(accident | 7 a.m.)


if it had been correct, the statement

P(accident | 7 p.m.) > P(accident | 7 a.m.),

would imply,

P(accident | 7 p.m.) > P(accident).

“thus it [would be] more dangerous to drive in the evening than in the morning!” but that statement is not correct.


As Huff remarks in his classic work: “How to Lie with Statistics”

• “By the same kind of nonsense […] you can show that clear weather driving is more dangerous than foggy weather. More accidents occur in clear weather, because there is more clear weather than foggy weather.”


How to Clear the Confusion: use of “natural frequencies” rather than

conditional probabilities

Consider for instance the question:

“What is the probability that a person who has been tested positive in a medical test, actually is HIV+?”


Under the Radar, HIV WorsensOctober 16, 2004

• With as many as 950,000 Americans infected with the virus that causes AIDS — and another 40,000 new infections each year — AIDS workers say they are confronting a level of ignorance and misconception reminiscent of the epidemic's earliest days.


End 2002

Caribbean1% North Africa

and Middle East1%

North America2%

Western Europe

1% Eastern Europe/Central

Asia3%

East Asia and the Pacific

3%

Latin America4%

South and Southeast Asia

14%

Sub-Saharan Africa71%

Australia and New Zealand

<1%

People With HIV/AIDS, by Region

Source: UNAIDS, AIDS Epidemic Update 2002.

2004 World Population Data Sheet

Population Mid-2004

United States

293,633,000

http://www.prb.org/datafind/prjprbdata/wcprbdata.asp?DW=DR&SL=

http://www.prb.org/datafind/prjprbdata/wcprbdata.asp?DW=DR&SL=


In the US population

(in 1,000s)Drug(8.3%)

NoDrug Total

NoHIV 23,925 269,125 293,050

HIV 475 475 950

Total 24,400 269,600 294,000

http://www.cdc.gov/nchs/fastats/druguse.htm


How good is the test

• the sensitivity of the test:

P(+|HIV) = 0.99, P(-|HIV) = 0.01

• the specificity of the test

P(+|noHIV) = 0.01, P(-|noHIV) = 0.99


Take a person at random in the US

(in 1,000s) Total

NoHIV 293,050

HIV 950

Total 294,000

Prior Probabilities:• P(noHIV)

= 293,050/ 294,000

=0.996768707

• P(HIV)= 950/294,000

= 0.003231293


Calculating Joint Probabilities

• Combining information about “the average person in the US” and the “reliability of HIV test” into the joint probabilities:

P(state and finding) = P(state) P(finding | state)

Information about the “reliability of HIV test”

Information about “the average person in the US”


Calculating Joint Probabilities

• State: “noHIV” or “HIV” Finding: “+” or “-” test result

P(state and finding) = P(state) P(finding | state)

• P(noHIV and +) = P(noHIV) P(+ | noHIV) = (0.996768707)(0.01) = 0.009967687• P(noHIV and -) = P(noHIV) P(- | noHIV) = (0.996768707)(0.99) = 0.98680102• P(HIV and +) = P(HIV) P(+ | HIV) = (0.003231293)(0.99) = 0.00319898• P(HIV and - ) = P(HIV) P( - | HIV) = (0.003231293)(0.01) = 0.0000323129


Probabilities of Each Finding

P(finding) = P(noHIV and finding) + P(HIV and finding)

P(+) = 0.009967687 + 0.00319898 = 0.013166667

P(-) = 0.98680102 + 0.0000323129 = 0.986833333


Calculating the Posterior ProbabilitiesP(state | finding) = P(state and finding) / P(finding)

• P(noHIV | -) = 0.98680102 / 0.986833333 = 0.999967256• P(HIV | -) = 0.0000323129 / 0.986833333 = 0.000032744• P(noHIV | +) = 0.009967687 / 0.013166667 = 0.7570395• P(HIV | +) = 0.00319898 / 0.013166667 = 0.2429605


Posterior vs. Prior

• P(noHIV | +) = 75.7%

• P(HIV | +) = 24.3%

• P(noHIV) = 99.7%

• P(HIV) = 0.3%

Compute the probabilities for other subpopulations.


Data given as conditional probabilities:

• The probability that an average person in the U.S. is HIV+ is 0.32%.

• If a person is HIV+, the probability that a medical test will show a positive result (the sensitivity) is 99%.

• If a person is not HIV+, the probability of a positive result is 1%.

• What is the probability that that person is actually HIV+?


Data given as natural frequencies:

• Our data tells us that 323 out of every 100,000 persons in the U.S. is HIV+.

• Of these 323 who are HIV+, 320 will be tested positive.

• Of the 99,677 who are not HIV+, some 997 will still test positive in the test.

• Take, for example, a sample of 100 persons who have tested positive for HIV+. How many of these are actually HIV+?


What is the probability that a person tested positive is actually HIV+?

Testedpositive

Not testedpositive

Totals

HIV+ 320 3 323

not HIV+ 997 98,680 99,677

Totals 1,317 98,683 100,000

P(HIV+ | Tested positive) = 320 / 1,317 = 0.2429 = 24.3%.


‘Cookie Jar’ Accounting of Statistical Analysis

• “Data containing 70 repetitive task times are given for each of the two workers.

• John has been doing this task for months, whereas Fred has just started.

• Each time listed is the time (in seconds) to perform a routine task on an assembly line.

• The times shown are given in chronological order.

• Which of the two workers would you rather have (assuming time is the only issue)?”


John

0

10

20

30

40

50 55 60 65 70 75 80 85 More

Performance Times

Fre

qu

ency

Fred

0

10

20

30

40

50 55 60 65 70 75 80 85 More

Performance Times

Fre

qu

ency

John and Fred Comparison

50.055.060.065.070.075.080.085.090.0

0 20 40 60 80

Observation Number (Chronological)

Tim

e In

Sec

on

ds

John

Fred

Linear (John)

Linear (Fred)


How bin widths (or the number of bins) affect a histogram • The histogram is for the Old Faithful data set.• The observations are the time (in seconds) between

eruptions for the Old Faithful geyser in Yellowstone National Park.

• Change the bin width by dragging the arrow underneath the bin width scale in the original applet.

• For large bin widths, the bimodal nature of the dataset is hidden, and for small bin widths, the plot reduces to a spike at each data point.

• What bin width do you think provides the best picture of the underlying data?

http://www.amstat.org/publications/jse/v6n3/applets/Histogram.html.

http://www.amstat.org/publications/jse/v6n3/applets/Histogram.html.

http://www.amstat.org/publications/jse/v6n3/applets/OF.dat


Graphical Misrepresentations


Graphing “pumpkins grown in the gardens of Mary, Joe and Ann.”

[How to Lie and Cheat with Statistics]

http://faculty.washington.edu/chudler/stat3.html


Tufte’s books

http://www.edwardtufte.com/tufte/


Drawing by Charles E. Martin; copyright 1961, The New Yorker.

From Tufte, 2001, p. 56: “Misperceptions and miscommunication are certainly not special to statistical graphics …”

by Ömer s. benli ethics across the curriculum 2006

Documents