Dr. Kari Lock MorganDepartment of StatisticsPenn State University

USA

Making computing skills part of learning introductory stats

Royal Statistical Society10/13/16

ASA 2016 Recommendations for Intro StatGAISE: Guidelines for Assessment and Instruction in Statistics Education

1. Teach statistical thinking. ¡ Teach statistics as an investigative process of

problem-solving and decision making. ¡ Give students experience with multivariable thinking.

2. Focus on conceptual understanding. 3. Integrate real data with a context and purpose. 4. Foster active learning. 5. Use technology to explore concepts and analyze data. 6. Use assessments to improve and evaluate student learning.

1. Teach statistical thinking. ¡ Teach statistics as an investigative process of

problem-solving and decision making. ¡ Give students experience with multivariable thinking.

2. Focus on conceptual understanding. 3. Integrate real data with a context and purpose. 4. Foster active learning. 5. Use technology to explore concepts and analyze data. 6. Use assessments to improve and evaluate student learning.

Simulation-Based Inference

Question #1

Does drinking tea boost your immune system?

Tea and Immune Response

Antigensintea-BeveragePrimeHumanVγ2Vδ2TCellsinvitroandinvivoforMemoryandNon-memoryAntibacterialCytokineResponses,Kamath et.al.,ProceedingsoftheNationalAcademyofSciences,May13,2003.

• Participants were randomized to drink five or six cups of either tea (black) or coffee every day for two weeks (both drinks have caffeine but only tea has L-theanine)

• After two weeks, blood samples were exposed to an antigen, and immune system response was measured

• Does tea boost immunity (over coffee)?

Tea and Immune System

�̅�# − �̅�% = 34.82 − 17.70 = 17.12

1. Checkconditions

2. Computestatistic:chooseformula,plugandchug

3. Usetheoreticaldistribution(whichone?df?)

4. 0.025<p-value<0.05

Getting the p-value: Option 1

𝑡 =𝑥0 − 𝑥1

𝑠01𝑛0+ 𝑠11𝑛1

= 2.07

𝑛0 = 11𝑛1 = 10

So what’s a p-value???

• Plugging numbers into formulas does little to help reinforce conceptual understanding

• With a different formula for each test/interval, students often get mired in the details and fail to see the big picture

• We need a better way…

Traditional Inference

Actual ExperimentR R R R R

R R R R R

R R R R RR R R R R

Tea Coffee

R R RR R R R R

R R R

R R R R R

R

R

Actual Experiment

R R RR R R R R

R R R

R R R R R

R R R R13 18 205 1152 55 5647 48 R58

3 11 150 0

21 38 5216 21

Tea Coffee

Actual Experiment

R R RR R R R R

R R R

R R R R R

R R R R13 18 205 1152 55 5647 48 R58

3 11 150 0

21 38 5216 21

Tea Coffee

!!xT − xC =17.12

� Twoplausibleexplanations:¡Teaboostsimmunity

¡Randomchance Whatmighthappenjustbyrandomchance???

R R RR R R R R

R R R

R R R R R

R R R R13 18 205 1152 55 5647 48 R58

3 11 150 0

21 38 5216 21

Tea Coffee

R R RR R R R RR R 13 18 205 11

52 55 5647 48 R58

R R R

R R R R R

R R 3 11 150 0

21 38 5216 21

Simulation

Simulation

R R RR R R R R

R R R

R R R R R15 16 21

18 20 4721 13 R55

38 52 5

52 56 5811 48

Tea Coffee

R R RR R R R RR R 13 18 205 11

52 55 5647 48 R58

R

R R R R R

3 11 150 0

21 38 5216 21

Simulation

R R RR R R R R

R R R

R R R R R15 16 21

18 20 4721 13 R55

38 52 5

52 56 5811 48

Tea Coffee

0 3 R R0 11

!!xT − xC = −12.3

RepeatManyTimes!

We need technology!

StatKeywww.lock5stat.com/statkey

¡ Free¡ Easy to use¡ Online (or offline as chrome app)

p-valueProportionasextremeasobservedstatistic

observedstatistic

RandomizationTest

DistributionofstatisticifH0 true

If there were no difference between tea and coffee regarding immune system response, we would see results this extreme about 2.6% of the time

p-value: The chance of obtaining a statistic as extreme as that observed, just by random chance, if the null hypothesis is true

Simulation-Based Inference• Intrinsically connected to concepts

• Same procedure works for many statistics

• More generalizable (new statistics or designs)

• Minimal background knowledge needed

• Fewer conditions; conditions transparent

Question #2

What is the average mercury level of fish (Large Mouth

Bass) in Florida lakes?

Mercury Levels in Fish

!!n=53!!x =0.527!ppm

Lange, T.,Royals,H.andConnor,L.(2004). Mercuryaccumulation inlargemouthbass(Micropterus salmoides)inaFloridaLake.ArchivesofEnvironmentalContaminationandToxicology,27(4), 466-471.

!µ = ???

Getting a Margin of Error

Population Sample

Sample

Sample

SampleSampleSample

...

Calculatestatisticforeachsample

SamplingDistribution

StandardError(SE):standarddeviationofsamplingdistribution

MarginofError(ME)(95%CI:ME=2×SE)

statistic±ME

• Key idea: how much do statistics vary from sample to sample?• Problem?

• We can’t take lots of samples from the population!

Assessing Uncertainty

Getting a Margin of ErrorPopulation

(???)

statistic±ME

Sample

BestGuessatPopulation

Sample

Sample

Sample

SampleSampleSample

...

Distributionofthestatistic

Calculatestatisticforeachsample

StandardError(SE):standarddeviationofthestatistic

MarginofError(ME)(95%CI:ME=2×SE)

Bootstrapping� What is our best guess at the population,

given sample data?

¡The sample itself!

� Draw samples repeatedly from the sample data (of size n = 53)…

¡… with replacement! (bootstrapping)

� Calculate statistic for each bootstrap sample

� SE = standard deviation of these statistics

We Need Technology!� StatKey: lock5stat.com/statkey� Rossman/Chance: rossmanchance.com/applets� InZight: stat.auckland.ac.nz/~wild/iNZight� R: cran.r-project.org� RStudio: rstudio.com� Fathom: fathom.concord.org� Tinkerplots: tinkerplots.com� JMP: jmp.com� Minitab Express: minitab.com� StatCrunch: statcrunch.com

Red = Free

SE=0.047

statistic±2xSE0.527± 2x0.047(0.433,0.621)

MercuryLevelsinFish

Weare95%confidentthataveragemercurylevelinfishinFloridalakesisbetween0.433and0.621ppm.

95%ConfidenceInterval

Same process for every parameter!Estimatethemarginoferrorand/oraconfidenceintervalfor...• proportion(𝑝)• differenceinmeans(µ1 − µ2)• differenceinproportions(𝑝1 − 𝑝2)• standarddeviation(𝜎)• correlation(𝜌)• ... Samplewithreplacement

CalculatestatisticRepeat...

MercuryandpHinLakes

Lange,Royals,andConnor,TransactionsoftheAmericanFisheriesSociety(1993)

• ForFloridalakes,whatisthecorrelationbetweenaveragemercurylevel(ppm)infishtakenfromalakeandacidity(pH)ofthelake?

Givea95%CIforρ

r =-0.575

Simulation-Based Inference• Students leave the course with…

• Better conceptual understanding (Tintle et al, JSE, 2011; Maurer and Lock, TISE, 2016)

• Better retention of concepts (Tintle et al, SERJ, 2012)

• Broader ability to apply what they have learned

• Familiarity with modern computationally-intensive methods

Fall '14 Spring '15 Fall '15

2040

6080

100

Pos

t Tes

t Sco

re

Conceptual UnderstandingScores on a National Assessment

Averages:

p-value:0.00002

43% 60% 63% National:47%

National Assessment ResultsMost improvement on p-value questions!

T-test SBI Nat’l

Abletoreasonthatasmallerp-valueprovidesstrongerevidenceagainstthenullhypothesisthanalargerp-value.

29%

Abletoreasonaboutaconclusionbasedonastatisticallysignificantp-valueinthecontextofaresearchstudythatcomparestwogroups.

43%

45%

48%

68%

80%

Student Behavior• Students were given data on the second midterm (right after learning about t-intervals!) and asked to compute a confidence interval for the mean

• How they created the interval:Bootstrapping t.test inR Formula

84% 8% 8%

Use data from a sample survey to estimate a population mean or proportion; develop a margin of

error through the use of simulation methods for random sampling

Use data from a randomized experiment to compare two

treatments; use simulation to decide if differences between

parameters are significant

Common Core State Standards in Mathematics (High School)

Traditional Methods Still There� We use simulation-based inference to

introduce inference and build understanding

� We then cover traditional normal and t-based methods and SE formulas (goes quickly!)

� CLT easy to motivate after simulation

� Testing and intervals concepts already there

Sir R. A. Fisher

"Actually, the statistician does not carry out this very simple and very tedious process [the randomization test], but his conclusions have no justification beyond the fact that they agree with those which could have been arrived at by this elementary method."

-- Sir R. A. Fisher, 1936

George Cobb“... the consensus curriculum is still an unwitting prisoner of history. What we teach is largely the technical machinery of numerical approximations based on the normal distribution and its many subsidiary cogs. This machinery was once necessary, because the conceptually simpler alternative based on permutations was computationally beyond our reach. Before computers statisticians had no choice. These days we have no excuse. Randomization-based inference makes a direct connection between data production and the logic of inference that deserves to be at the core of every introductory course.”

-- Professor George Cobb, 2007

Want More?� Simulation-based inference blog:

causeweb.org/sbi

� Videos, presentations, and more: lock5stat.com

� Recordings from the 2016 Electronic Conference on Teaching Statistics (eCOTS): causeweb.org/cause/ecots/ecots16

� Email me: klm47@psu.edu

klm47@psu.edu