using randomization methods to build conceptual understanding of statistical inference: day 2
DESCRIPTION
Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2. Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse - Joint Mathematics Meetings Baltimore, MD January 2014. Schedule: Day 2 Saturday, 1/18, 1:00 – 3:00 pm 5. More on Randomization Tests - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/1.jpg)
Using Randomization Methods to Build Conceptual Understanding
of Statistical Inference:Day 2
Lock, Lock, Lock Morgan, Lock, and LockMAA Minicourse- Joint Mathematics Meetings
Baltimore, MDJanuary 2014
![Page 2: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/2.jpg)
Schedule: Day 2Saturday, 1/18, 1:00 – 3:00 pm
5. More on Randomization Tests • How do we generate randomization distributions for various
statistical tests?• How do we assess student understanding when using this
approach?
6. Connecting Intervals and Tests
7. Connecting Simulation Methods to Traditional
8. Technology Options• Other software for simulatiions (Minitab, Fathom, R, Excel, ...)• Bascis summary stats/graphs with StatKey• A more advanced randmization with StatKey
9. Wrap-up• How has this worked in the classroom?• Participant comments and questions
10. Evaluations
![Page 3: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/3.jpg)
• In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug)
• The outcome variable is whether or not a patient relapsed
• Is there convincing evidence to conclude that Desipramine is better than Lithium at treating cocaine addiction?
Cocaine Addiction
![Page 4: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/4.jpg)
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R
R R R R R R
R R R R R R
R R R R R R
Desipramine Lithium
1. Randomly assign units to treatment groups
![Page 5: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/5.jpg)
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R R R R N N
N N N N N N
RR
N N N N N N
R = RelapseN = No Relapse
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R R R R RR
R R N N N N
RR
N N N N N N
2. Observe relapse counts in each group
3. Compare the proportions who relapse
LithiumDesipramine
10 relapse, 14 no relapse 18 relapse, 6 no relapse
1. Randomly assign units to treatment groups
10 1824
ˆ ˆ
24.333
new oldp p
Is this convincing evidence?
![Page 6: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/6.jpg)
• Assume the null hypothesis is true
• Simulate new randomizations
• For each, calculate the statistic of interest
• Find the proportion of these simulated statistics that are as extreme as your observed statistic
Randomization Test
![Page 7: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/7.jpg)
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R R R R N N
N N N N N N
RR
N N N N N N
10 relapse, 14 no relapse 18 relapse, 6 no relapse
![Page 8: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/8.jpg)
R R R R R R
R R R R N N
N N N N N N
N N N N N N
R R R R R R
R R R R R R
R R R R R R
N N N N N N
R N R N
R R R R R R
R N R R R N
R N N N R R
N N N R
N R R N N N
N R N R R N
R N R R R R
Simulate another randomization
Desipramine Lithium
16 relapse, 8 no relapse 12 relapse, 12 no relapse
ˆ ˆ16 1224 240.167
N Op p
![Page 9: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/9.jpg)
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R N R R N N
R R N R N R
RR
R N R N R R
Simulate another randomization
Desipramine Lithium
17 relapse, 7 no relapse 11 relapse, 13 no relapse
ˆ ˆ17 1124 240.250
N Op p
![Page 10: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/10.jpg)
• Start with 48 cards (Relapse/No relapse) to match the original sample.
•Shuffle all 48 cards, and rerandomize them into two groups of 24 (new drug and old drug)
• Count “Relapse” in each group and find the difference in proportions, .
• Repeat (and collect results) to form the randomization distribution.
• How extreme is the observed statistic of 0.33?
Physical Simulation
![Page 11: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/11.jpg)
A randomization sample must:
• Use the data that we have (That’s why we didn’t change any of the results on the cards) AND
• Match the null hypothesis (That’s why we assumed the drug didn’t matter and combined the cards)
Cocaine Addiction
![Page 12: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/12.jpg)
StatKey
The probability of getting results as extreme or more extreme than those observed if the null hypothesis is true, is about .02. p-value
Proportion as extreme as observed statistic
observed statistic
Distribution of Statistic Assuming Null is True
![Page 13: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/13.jpg)
How can we do a randomization test for a correlation?
![Page 14: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/14.jpg)
Is the number of penalties given to an NFL team positively correlated with the “malevolence” of the team’s uniforms?
![Page 15: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/15.jpg)
Ex: NFL uniform “malevolence” vs. Penalty yards
r = 0.430n = 28
Is there evidence that the population correlation is positive?
![Page 16: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/16.jpg)
Key idea: Generate samples that are(a) consistent with the null hypothesis (b) based on the sample data.
H0 : = 0
r = 0.43, n = 28
How can we use the sample data, but ensure that the correlation is zero?
![Page 17: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/17.jpg)
Randomization by ScramblingOriginal sample
MalevolentUniformsNFLNFLTeam NFL_Ma... ZPenYds <new>
1234567891011121314151617181920212223
LA Raiders 5.1 1.19
Pittsburgh 5 0.48
Cincinnati 4.97 0.27
New Orl... 4.83 0.1
Chicago 4.68 0.29
Kansas ... 4.58 -0.19
Washing... 4.4 -0.07
St. Louis 4.27 -0.01
NY Jets 4.12 0.01
LA Rams 4.1 -0.09
Cleveland 4.05 0.44
San Diego 4.05 0.27
Green Bay 4 -0.73
Philadel... 3.97 -0.49
Minnesota 3.9 -0.81
Atlanta 3.87 0.3
Indianap... 3.83 -0.19
San Fra... 3.83 0.09
Seattle 3.82 0.02
Denver 3.8 0.24
Tampa B... 3.77 -0.41
New Eng... 3.6 -0.18
Buffalo 3.53 0.63
Scrambled MalevolentUniformsNFLNFLTeam NFL_Ma... ZPenYds <new>
1234567891011121314151617181920212223
LA Raiders 5.1 0.44
Pittsburgh 5 -0.81
Cincinnati 4.97 0.38
New Orl... 4.83 0.1
Chicago 4.68 0.63
Kansas ... 4.58 0.3
Washing... 4.4 -0.41
St. Louis 4.27 -1.6
NY Jets 4.12 -0.07
LA Rams 4.1 -0.18
Cleveland 4.05 0.01
San Diego 4.05 1.19
Green Bay 4 -0.19
Philadel... 3.97 0.27
Minnesota 3.9 -0.01
Atlanta 3.87 0.02
Indianap... 3.83 0.23
San Fra... 3.83 0.04
Seattle 3.82 -0.09
Denver 3.8 -0.49
Tampa B... 3.77 -0.19
New Eng... 3.6 -0.73
Buffalo 3.53 0.09
Scrambled sample
![Page 18: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/18.jpg)
Randomize one of the variables!
Let’s look at StatKey.
![Page 19: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/19.jpg)
Traditional Inference1. Which formula?
2. Calculate numbers and plug into formula
3. Plug into calculator
4. Which theoretical distribution?
5. df?
6. find p-value
0.01 < p-value < 0.02
43.2t
212r
nrt
243.0122843.0
t
![Page 20: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/20.jpg)
How can we do a randomization test for a mean?
![Page 21: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/21.jpg)
Example: Mean Body Temperature
Data: A random sample of n=50 body temperatures.
Is the average body temperature really 98.6oF?
BodyTemp96 97 98 99 100 101
BodyTemp50 Dot Plot
H0:μ=98.6 Ha:μ≠98.6
n = 5098.26s = 0.765
Data from Allen Shoemaker, 1996 JSE data set article
![Page 22: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/22.jpg)
Key idea: Generate samples that are(a) consistent with the null hypothesis (b) based on the sample data.
How to simulate samples of body temperatures to be consistent with H0: μ=98.6?
![Page 23: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/23.jpg)
Randomization SamplesHow to simulate samples of body temperatures to be consistent with H0: μ=98.6?
1. Add 0.34 to each temperature in the sample (to get the mean up to 98.6).
2. Sample (with replacement) from the new data.3. Find the mean for each sample (H0 is true).
4. See how many of the sample means are as extreme as the observed 98.26.
![Page 24: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/24.jpg)
Let’s try it on
StatKey.
![Page 25: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/25.jpg)
Playing with StatKey!
See the orange pages in the folder.
![Page 26: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/26.jpg)
Choosing a Randomization MethodA=Sleep 14 18 11 13 18 17 21 9 16 17 14 15 mean=15.25
B=Caffeine 12 12 14 13 6 18 14 16 10 7 15 10 mean=12.25
Example: Word recall
Option 1: Randomly scramble the A and B labels and assign to the 24 word recalls.
H0: μA=μB vs. Ha: μA≠μB
Option 2: Combine the 24 values, then sample (with replacement) 12 values for Group A and 12 values for Group B.
Reallocate
Resample
![Page 27: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/27.jpg)
How do we assess student understanding of these methods(even on in-class exams without computers)?
See the blue pages in the folder.
![Page 28: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/28.jpg)
Connecting CI’s and Tests
Randomization body temp means when μ=98.6
xbar98.2 98.3 98.4 98.5 98.6 98.7 98.8 98.9 99.0
Measures from Sample of BodyTemp50 Dot Plot
97.9 98.0 98.1 98.2 98.3 98.4 98.5 98.6 98.7bootxbar
Measures from Sample of BodyTemp50 Dot Plot
Bootstrap body temp means from the original sample
What’s the difference?
![Page 29: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/29.jpg)
Fathom Demo: Test & CI
Sample mean is in the “rejection region”
Null mean is outside the confidence interval
![Page 30: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/30.jpg)
What about Traditional Methods?
![Page 31: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/31.jpg)
Transitioning to Traditional Inference
AFTER students have seen lots of bootstrap distributions and randomization distributions…
Students should be able to• Find, interpret, and understand a confidence
interval• Find, interpret, and understand a p-value
![Page 32: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/32.jpg)
slope (thousandths)-60 -40 -20 0 20 40 60
Measures from Scrambled RestaurantTips Dot Plot
r-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6
Measures from Scrambled Collection 1 Dot Plot
Nullxbar98.2 98.3 98.4 98.5 98.6 98.7 98.8 98.9 99.0
Measures from Sample of BodyTemp50 Dot Plot
Diff-4 -3 -2 -1 0 1 2 3 4
Measures from Scrambled CaffeineTaps Dot Plot
xbar26 27 28 29 30 31 32
Measures from Sample of CommuteAtlanta Dot Plot
Slope :Restaurant tips Correlation: Malevolent uniforms
Mean :Body Temperatures Diff means: Finger taps
Mean : Atlanta commutes
phat0.3 0.4 0.5 0.6 0.7 0.8
Measures from Sample of Collection 1 Dot PlotProportion : Owners/dogs
What do you notice?
All bell-shaped distributions!
Bootstrap and Randomization Distributions
![Page 33: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/33.jpg)
The students are primed and ready to learn about the normal distribution!
![Page 34: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/34.jpg)
Transitioning to Traditional Inference
Confidence Interval:
Hypothesis Test:
• Introduce the normal distribution (and later t)
• Introduce “shortcuts” for estimating SE for proportions, means, differences, slope…
![Page 35: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/35.jpg)
z*-z*
95%
Confidence Intervals
![Page 36: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/36.jpg)
Test statistic
95%
Hypothesis Tests
Area is p-value
![Page 37: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/37.jpg)
Confidence Interval:
Hypothesis Test:
Yes! Students see the general pattern and not just individual formulas!
![Page 38: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/38.jpg)
Other Technology OptionsYour binder includes information for doing several simulation examples using • Minitab (macros)• R (defined functions)• Excel (PopTools ad-in)• Fathom (drag/drop/menus)• Matlab (commands)• SAS (commands)
![Page 39: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/39.jpg)
More StatKeyBasic Statistics/Graphics
![Page 40: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/40.jpg)
Sampling Distribution
Capture Rate
![Page 41: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/41.jpg)
Example: Sandwich Ants
Experiment: Place pieces of sandwich on the ground, count how many ants are attracted. Does it depend on filing?
Favourite Experiments: An Addendum to What is the Use of Experiments Conducted by Statistics Students? Margaret Mackisack
http://www.amstat.org/publications/jse/v2n1/mackisack.supp.html
![Page 42: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/42.jpg)
![Page 43: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/43.jpg)
![Page 44: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/44.jpg)
Student Preferences
Which way of doing inference gave you a better conceptual understanding of confidence intervals and hypothesis tests?
Bootstrapping and Randomization
Formulas and Theoretical Distributions
113 51
69% 31%
![Page 45: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/45.jpg)
Student Preferences
Which way did you prefer to learn inference (confidence intervals and hypothesis tests)?
Bootstrapping and Randomization
Formulas and Theoretical Distributions
105 60
64% 36%
Simulation Traditional
AP Stat 31 36
No AP Stat 74 24
![Page 46: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/46.jpg)
Student Behavior• Students were given data on the second midterm and asked to compute a confidence interval for the mean
• How they created the interval:
Bootstrapping t.test in R Formula
94 9 9
84% 8% 8%
![Page 47: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/47.jpg)
A Student Comment
" I took AP Stat in high school and I got a 5. It was mainly all equations, and I had no idea of the theory behind any of what I was doing.
Statkey and bootstrapping really made me understand the concepts I was learning, as opposed to just being able to just spit them out on an exam.”
- one of Kari’s students
![Page 48: Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2](https://reader035.vdocuments.us/reader035/viewer/2022070500/568168f5550346895de0025c/html5/thumbnails/48.jpg)
Thank you for joining us!
More information is available on www.lock5stat.com
Feel free to contact any of us with any comments or questions.