statistics as a distillation of everyday...
TRANSCRIPT
![Page 1: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/1.jpg)
March 29, 2007 KNAW Lecture 1
Statistics as a Distillation of Everyday Experience
Gerald van BelleDepartment Biostatistics,
Department of Occupational and Environmental Health Sciences
University of Washington,Seattle, WA.
![Page 2: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/2.jpg)
March 29, 2007 KNAW Lecture 2
Where Are We Going?
Statistics as distillation of everyday experience
1. Variation
2. Causation
Experience can benefit from everyday statistics
![Page 3: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/3.jpg)
March 29, 2007 KNAW Lecture 3
A. Variation in everyday experience
1. Describing and classifying variation2. Selection in the face of variation3. Controlling variation4. Inducing variation5. “Missing data”
![Page 4: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/4.jpg)
March 29, 2007 KNAW Lecture 4
1. Describing and classifying variation• We tell stories of ab”normality”
* Air travel horror stories, laptop disasters,…
• We sort into genres: art, biology, literature* Concept of “population”* Characteristics of population and “sample”
• Variation in time, space, social structures,…* Waves on beach (non-stationarity)* Hierarchy, “social class”
• We make inferences based on limited data* And often get the wrong population* Basis for a great deal of humor* Switch in “expectation”
![Page 5: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/5.jpg)
March 29, 2007 KNAW Lecture 5
![Page 6: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/6.jpg)
March 29, 2007 KNAW Lecture 6
2. Selection in the face of variation
• Need to know selection mechanismSpend very little time on thisAssumption of “missing at random”
• Error of thinking that current observation is representative
• Unintended intentional selection• Two examples:
![Page 7: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/7.jpg)
March 29, 2007 KNAW Lecture 7
![Page 8: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/8.jpg)
March 29, 2007 KNAW Lecture 8
![Page 9: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/9.jpg)
March 29, 2007 KNAW Lecture 9
2. Selection in the face of variation--2
• Need to know selection mechanismRandom selection as gold standard
• “Representativeness”* Kruskal and Mosteller papers* Slippery concept* Large sample vs small sample
![Page 10: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/10.jpg)
March 29, 2007 KNAW Lecture 10
3. Controlling variation
• Clearest examples in sports: Divisions, “junior”,…
• Societal examplesMin, max speed limitsOccupational (noise limits, flying hours)Vergunningen, vergunningen,…
• “Blocking” in statistics
![Page 11: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/11.jpg)
March 29, 2007 KNAW Lecture 11
4. Inducing variation
• Antitrust laws * Increase competition, i.e. variability
• Draft system in sports* Teams more equal, P(win) near 1/2
• Societal* Admission to medical school in Holland* Representativeness (slippery concept)* Key to clinical trials
![Page 12: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/12.jpg)
March 29, 2007 KNAW Lecture 12
5. “Missing” data
1. Serious problem, obviously2. Anatomy of missingness
• Normal (e.g. pediatrician chart)• Transcription error• Just not there (Murphy was here)• Deliberately missing (e.g. extended testing on
subset of patients)• Impacts population of inference• Example:
![Page 13: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/13.jpg)
March 29, 2007 KNAW Lecture 13
![Page 14: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/14.jpg)
March 29, 2007 KNAW Lecture 14
Vulnerability Analysis of Spitfires (sample: 15/400)
![Page 15: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/15.jpg)
March 29, 2007 KNAW Lecture 15
Composite of hits
*
* ***
*
**
***
**
*
Abraham WaldAdvice:
![Page 16: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/16.jpg)
March 29, 2007 KNAW Lecture 16
…as we know,there are known knowns;there are things we know we know.We also know there are known unknowns;that is to say,we know there are some thingswe do not know.But there are also unknown unknowns—the ones we don’t know we don’t know.
Donald Rumsfeld
Another anatomy of missingness
(set to music, see NPR website)
![Page 17: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/17.jpg)
March 29, 2007 KNAW Lecture 17
…as we know,there are known knowns;there are things we know we know.We also know there are known unknowns;that is to say,we know there are some thingswe do not know.But there are also unknown unknowns—the ones we don’t know we don’t know.
Donald Rumsfeld
Non-missing
MCAR/MAR
Non-ignorable
Translation into modern statistics
![Page 18: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/18.jpg)
March 29, 2007 KNAW Lecture 18
B: Causation in everyday experience
1. Aristotle’s four causes2. Hardwired to look for causation3. Hardwired to assume association is
causation4. Hardwired to assign blame (secondary
causes)
(The Dutch: “oorzaak” is closer to Aristotle’s αιτιον)
![Page 19: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/19.jpg)
March 29, 2007 KNAW Lecture 19
1. Aristotle’s four causes
• Material cause (table made of wood)
• Formal cause (four legs and flat top make this a table)
• Efficient cause (carpenter makes a table)
• Final cause (surface for eating or writing makes this a table)
(From S.M. Cohen, U Washington)
![Page 20: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/20.jpg)
March 29, 2007 KNAW Lecture 20
Four questions
• What is the question? was• Is it testable? Was• Where will you get the data? did
____________________________• What will the data tell you? do
The great divide
![Page 21: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/21.jpg)
March 29, 2007 KNAW Lecture 21
Not everything that can be counted counts and not everything that countscan be counted.
Albert Einstein
1. What is the Question? Is it testable?
![Page 22: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/22.jpg)
March 29, 2007 KNAW Lecture 22
![Page 23: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/23.jpg)
March 29, 2007 KNAW Lecture 23
2. Frequent Consulting Scenario
Testable hypothesis
What was the question?
but
![Page 24: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/24.jpg)
March 29, 2007 KNAW Lecture 24
Example from Science (February 23, 2007)
Title Redefining the age of ClovisFront page Flints (pictures)Page 1045 Summary paragraphPage 1067 News storyPage 1122—1126 Article (numbers)
Very different “flavor” for each section
![Page 25: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/25.jpg)
March 29, 2007 KNAW Lecture 25
Question Testable version
![Page 26: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/26.jpg)
March 29, 2007 KNAW Lecture 26
3. Hardwired to look for causation
• Story• Instinctive looking for causes• Challenging in courts
crime in search of criminal, stock market (“if only I had bought Microsoft in 1980”), and science (global warming)
• Life forces us to do this ex post factoWhitehead quote
![Page 27: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/27.jpg)
March 29, 2007 KNAW Lecture 27
4. Hardwired to assume association is causation
1. Story2. Criteria for causation help3. Causation in observational studies is a
great challenge4. R.A. Fisher’s design of experiments
introduced randomization
![Page 28: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/28.jpg)
March 29, 2007 KNAW Lecture 28
5. Challenges to Causation in Observational Studies
1. Selection bias “Where did you get the data?”
• Confounding“What do you think the data are telling you?”
4. Interplay of selection bias and confoundingEffect modification needs to be considered
![Page 29: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/29.jpg)
March 29, 2007 KNAW Lecture 29
6. Hardwired to look for secondary causes
1. Assists in soccer, hockey, basketball2. Tendency to blame i.e. move from
efficient cause to material or formal cause; that is, change the “universe of discourse”
3. Surrogate outcomes in science (great work by Ross Prentice)
![Page 30: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/30.jpg)
March 29, 2007 KNAW Lecture 30
6. Observational vs Experimental Studies
Characteristic Observational ExperimentEthical issues Fewer MoreOrientation Retrospective ProspectiveInference Weaker StrongerSelection bias Big problem Less Confounding Present AbsentRealism More LessCausal plausibility Weaker StrongerResearcher control Less MoreAnalysis More complicated Less
![Page 31: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/31.jpg)
March 29, 2007 KNAW Lecture 31
The speech of numbers
If numbers could talk we could discern the liars from the truth tellers
Numbers are a severe reduction of the real world—the process is as important as the product
![Page 32: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/32.jpg)
March 29, 2007 KNAW Lecture 32
Consequence
In view of variation and hard-wired tendencies we have a lethal mixture.
![Page 33: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/33.jpg)
March 29, 2007 KNAW Lecture 33
APHA Journal, May, 2004
![Page 34: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/34.jpg)
March 29, 2007 KNAW Lecture 34
Recapitulation:
1. What is the question?2. Is it testable?3. Where will you get the data?4. What do you think the data are telling
you?
![Page 35: Statistics as a Distillation of Everyday Experiencevanbelle.org/presentations/2007KNAWLectureReduced.pdfMarch 29, 2007 KNAW Lecture 1 Statistics as a Distillation of Everyday Experience](https://reader033.vdocuments.us/reader033/viewer/2022053001/5f052c967e708231d411a48f/html5/thumbnails/35.jpg)
March 29, 2007 KNAW Lecture 35
1. Variation is fact of life2. “Population” as model 3. Representativeness4. Regression to the mean5. What is the question?6. Testable question? 7. Association and causation8. Causation through randomization
Experience and everyday statistics
Variation
Causation