getting comfortable with your data ii one way to turn your data into knowledge, and another way...
TRANSCRIPT
![Page 1: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/1.jpg)
GETTING COMFORTABLE WITH YOUR DATA II
One way to turn your data into knowledge,
and another way that’s probably better
Winter Storm 2010Stats workshop
Dave Kleinschmidt
![Page 2: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/2.jpg)
ANOVAWhat is it, anyway?
![Page 3: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/3.jpg)
WHAT YOU WANT
You’ve designed + run your experiment
It sorts observations into groups
Is there any difference between groups?
![Page 4: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/4.jpg)
YOUR DATA IS NOISY
This could be a big problem for you
What if the noise is too big,
and drowns out the effect of your groups?
More importantly, how can you tell?
![Page 5: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/5.jpg)
STATISTICS TO THE RESCUE
Statistical models quantify noise
ANOVA is one kind of model
Mixed-effects models (MEMs) are another
![Page 6: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/6.jpg)
ANOVA
ANalysis Of VAriance
Tells whether group means are identical
(tests a null hypothesis)
Compare variance between groups (good)
with variance within groups (bad—noise)
![Page 7: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/7.jpg)
ANOVA
Figure from PDQ Statistics, Norman and Streiner
![Page 8: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/8.jpg)
ANOVA
If differences between groups outweigh noise within groups, then you can safely reject the null hypothesis
(which is that your experiment did nothing)
![Page 9: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/9.jpg)
ANOVA—ONE LAST NOTEANOVAs come in different flavors:
• One-way ANOVA tests one grouping
• Factorial ANOVA tests multiple crossed groupings
• Repeated-measures ANOVA tests a design where each subject is exposed to each condition (a within-subjects design)
![Page 10: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/10.jpg)
SO WHAT’S THE PROBLEM?ANOVA’s considered the gold-standard
Especially for factorial designs
However, ANOVA makes assumptions:
• Data is perfectly balanced
• Each group has identical variance
• No systematic variability between subjects or items
![Page 11: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/11.jpg)
MIXED-EFFECTS MODELS TO THE RESCUE!
MEMs can represent nearly any sort of variability between subjects/items.
Balance these differences with the need to draw general conclusions about the average character of the whole population
![Page 12: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/12.jpg)
MIXED-EFFECTS MODELS TO THE RESCUE!
Do other nice things, too
• Far more robust to missing data
• Can model nearly any data distribution (not just normal, like ANOVA)
![Page 13: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/13.jpg)
WHAT IS A MEM?
Combines fixed and random effects:
• Fixed effects are deterministic and common to all subjects/itmes
• Random effects vary from subject-to-subject/item-to-item
`
![Page 14: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/14.jpg)
WHAT IS A MEM?
Fixed effects describe how the experimental manipulations affect the observations
Think of it as the slope of a line:
dataij = fixed * xij
(xij is the condition that dataij comes from)
`
![Page 15: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/15.jpg)
WHAT IS A MEM?
Of course, we have to add noise.
If the noise of each subject/item combination is independent, than we just get
dataij = fixed * xij + noiseij
Where all of the noiseijs are independent and normally distributed (with mean zero)
(this is the essence of an ANOVA)
`
![Page 16: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/16.jpg)
WHAT IS A MEM?
What if some subjects are just faster/better than others?
Then we just add another noise term by subjects:
yij = fixed * xij + noise0j + noiseij
Note that this changes the intercept for the line for each subject, but leaves the slope the same for each
`
![Page 17: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/17.jpg)
WHAT IS A MEM?
In the same way, we can let the slope of the line vary a little by subject, too.
This is equivalent to saying that we believe the experimental manipulation affects some subjects more than others.
`
![Page 18: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/18.jpg)
SO WHY DOESN’T EVERYONE USE MEMs?Soon, everyone will (probably).
No pencil-and-paper solution, unlike ANOVA
(but software is widely available now)
ANOVA is the established standard
(but more and more are using MEMs)
![Page 19: GETTING COMFORTABLE WITH YOUR DATA II One way to turn your data into knowledge, and another way that’s probably better Winter Storm 2010 Stats workshop](https://reader036.vdocuments.us/reader036/viewer/2022082819/56649f2c5503460f94c471ae/html5/thumbnails/19.jpg)
LET’S TRY SOME