bigger, better, faster, more! sample size calculation · 2020-01-17 · bigger, better, faster,...
TRANSCRIPT
![Page 1: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/1.jpg)
![Page 2: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/2.jpg)
Bigger, better, faster, more! – Sample size calculationSEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED STUDY? TIPS FORIMPROVING DATA ANALYSIS – SESSION 6
TOM SMEKENS
TYPE NAME DEPARTMENT IN WINDOW
![Page 3: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/3.jpg)
Would you believe me if I said…
“Patient centered care is not related to health outcomes, based on:
my sample of 8 physicians”
“No! Your sample is not…”
my sample of 5000 orthopedic surgeons”
“No! Your sample is not…”
TYPE NAME DEPARTMENT IN WINDOW
![Page 4: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/4.jpg)
TYPE NAME DEPARTMENT IN WINDOW
4
![Page 5: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/5.jpg)
“Not representative”
Bias Variance
TYPE NAME DEPARTMENT IN WINDOW
![Page 6: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/6.jpg)
“Not representative”
Bias Variance
TYPE NAME DEPARTMENT IN WINDOW
![Page 7: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/7.jpg)
Estimating sampling variance
After the study: standard errors, p-values, confidence intervals…
Before the study: sample size calculation
TYPE NAME DEPARTMENT IN WINDOW
![Page 8: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/8.jpg)
TYPE NAME DEPARTMENT IN WINDOW
![Page 9: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/9.jpg)
Thought process
Hypothesis AnalysisSample
size
TYPE NAME DEPARTMENT IN WINDOW
![Page 10: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/10.jpg)
Sample size goals
Hypothesis testing:Power
Estimation:Precision
TYPE NAME DEPARTMENT IN WINDOW
![Page 11: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/11.jpg)
Sample size goals
Hypothesis testing:Power
Estimation:Precision
TYPE NAME DEPARTMENT IN WINDOW
“A large enough sample tomake the right conclusionin most cases”
Conclusion determined byStatistical significance
“A large enough sample tosufficiently narrow down an estimate”
Precision expressed usingConfidence intervals
![Page 12: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/12.jpg)
Methods of sample size calculation
Derivation
TYPE NAME DEPARTMENT IN WINDOW
![Page 13: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/13.jpg)
Methods of sample size calculation
Derivation Simulation
TYPE NAME DEPARTMENT IN WINDOW
![Page 14: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/14.jpg)
TYPE NAME DEPARTMENT IN WINDOW
![Page 15: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/15.jpg)
TYPE NAME DEPARTMENT IN WINDOW
![Page 16: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/16.jpg)
TYPE NAME DEPARTMENT IN WINDOW
![Page 17: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/17.jpg)
Contents
1. Power
2. Populations
3. Proportions
TYPE NAME DEPARTMENT IN WINDOW
![Page 18: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/18.jpg)
TYPE NAME DEPARTMENT IN WINDOW
1. The preponderance of power
![Page 19: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/19.jpg)
Ingredients (hypothesis testing)
1. Expected value"We want to reduce mean systolic bloodpressure by 5 mmHg"
2. Scale"68% of the population is in a range of 20 mmHgaround the mean"
3. P-value threshold= 5%
4. Power= 80%
TYPE NAME DEPARTMENT IN WINDOW
![Page 20: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/20.jpg)
Ingredients (hypothesis testing)
1. Expected value"We want to reduce mean systolic bloodpressure by 5 mmHg"
2. Scale"68% of the population is in a range of 20 mmHgaround the mean"
3. P-value threshold= 5%
4. Power= 80%
TYPE NAME DEPARTMENT IN WINDOW
If your expectations are true, theprobability of getting a statisticallysignificant result
![Page 21: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/21.jpg)
Ingredients (hypothesis testing)
1. Expected value"We want to reduce mean systolic bloodpressure by 5 mmHg"
2. Scale"68% of the population is in a range of 20 mmHgaround the mean"
3. P-value threshold= 5%
4. Power= 80%
TYPE NAME DEPARTMENT IN WINDOW
If your expectations are true, theprobability of getting a statisticallysignificant result
![Page 22: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/22.jpg)
TYPE NAME DEPARTMENT IN WINDOW
(Bacchetti et al., 2011)
![Page 23: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/23.jpg)
TYPE NAME DEPARTMENT IN WINDOW
(Bacchetti et al., 2011)
80% Power
![Page 24: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/24.jpg)
TYPE NAME DEPARTMENT IN WINDOW
(Bacchetti et al., 2011)
80% Power
Budget
![Page 25: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/25.jpg)
TYPE NAME DEPARTMENT IN WINDOW
(Bacchetti et al., 2011)
80% Power
Budget
![Page 26: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/26.jpg)
Ingredients (hypothesis testing)
1. Expected value"We want to reduce mean systolic blood
pressure by 15 mmHg"
1. Scale"68% of the population is in a range of 20 mmHgaround the mean"
1. P-value threshold= 5%
2. Power= 80%
TYPE NAME DEPARTMENT IN WINDOW
![Page 27: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/27.jpg)
How much power, really???
For a 15 mmHg difference?
80%
For a 5 mmHg difference?
15%
TYPE NAME DEPARTMENT IN WINDOW
![Page 28: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/28.jpg)
Results from an underpowered study
8 mmHg: Not significant...
19 mmHg: Significant!
TYPE NAME DEPARTMENT IN WINDOW
= Filters out plausible, realistic results in favor of fantastical, noisy ones
![Page 29: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/29.jpg)
Ingredients (precision)
1. Scale"68% of the population is in a range of 20 mmHgaround the mean"
1. Confidence level= 95%
2. Margin of error"We will be able to estimate the effect, give or take 2 mmHg"
TYPE NAME DEPARTMENT IN WINDOW
= half the width of the confidence interval
![Page 30: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/30.jpg)
Mentimeter
How often have you based your sample size calculations around precision?
I never calculate any sample sizes...
TYPE NAME DEPARTMENT IN WINDOW
![Page 31: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/31.jpg)
TYPE NAME DEPARTMENT IN WINDOW
![Page 32: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/32.jpg)
TYPE NAME DEPARTMENT IN WINDOW
2. What about population size?
![Page 33: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/33.jpg)
Inference: generalize from a sample to...
A specific group (usually people) in a given place and time
Finite population
A process (usually biomedical) that is repeatedly observed
Infinite population
TYPE NAME DEPARTMENT IN WINDOW
![Page 34: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/34.jpg)
Finite Population Correction
By default: assume infinite population
Optional: account for the population size
-> Smaller p-values, narrower confidence intervals...
Only noticeable if sample is > 1% of the population
TYPE NAME DEPARTMENT IN WINDOW
![Page 35: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/35.jpg)
TYPE NAME DEPARTMENT IN WINDOW
3. Properly prespecify probabilistic properties ofproportion prerogation
![Page 36: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/36.jpg)
Comparing proportions
Instead of mean systolic blood pressure,compare the prevalence of hypertension
1. Expected valueScale
2. P-value threshold = 5%
3. Power = 80%
TYPE NAME DEPARTMENT IN WINDOW
![Page 37: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/37.jpg)
Scale
"Hypertension prevalence is 30%.We want to reduce it to 25%."
=> Test a difference of 5%?
What about 10% to 5%?
TYPE NAME DEPARTMENT IN WINDOW
![Page 38: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/38.jpg)
Comparing proportions: 25% vs 30%
Comparison Value Regression method
Percentage point difference - 0.05 Linear
Risk ratio 0.83 Log binomial
Odds ratio 0.78 Logistic
TYPE NAME DEPARTMENT IN WINDOW
![Page 39: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/39.jpg)
Comparing proportions: 25% vs 30%
Comparison Value Regression method
Percentage point difference - 0.05 Linear
Risk ratio 0.83 Log binomial
Odds ratio 0.78 Logistic
TYPE NAME DEPARTMENT IN WINDOW
Study result: 8% to 11%
Comparison Value Regression method
Percentage point difference - 0.03 Linear
Risk ratio 0.73 Log binomial
Odds ratio 0.70 Logistic
![Page 40: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/40.jpg)
Non-inferiority trials
TYPE NAME DEPARTMENT IN WINDOW
"Not inferior"
(Ellis et al., 2015)
Non-inferiority margin: 4.5 percentage pointsStudy result: 1.7 percentage pointsBut: risk increased by 30%!
![Page 41: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/41.jpg)
TYPE NAME DEPARTMENT IN WINDOW
![Page 42: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/42.jpg)
Sample size calculation with categorical data
Before the study After the study
TYPE NAME DEPARTMENT IN WINDOW
![Page 43: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/43.jpg)
Conclusion
TYPE NAME DEPARTMENT IN WINDOW
![Page 44: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/44.jpg)
Sample size calculation
Essential in planning a quantitative study
But: fraught with bad habits (even from statisticians!)
Future prospects:
Simulation instead of formulas
Precision instead of power
![Page 45: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/45.jpg)
Questions / Comments?
Next seminar is on December 17
Why eating ice cream doesn’t cause summer – Association and causation
Presenter: Jozefien Buyze
These days, it’s commonly known by researchers that an association doesn't necessarily imply causation. Nevertheless, it’s common to still find this mistake even in high impact papers. In this session, we discuss confounding and interaction (also known as effect-modification) and why finding an association may be very valuable in some settings but not sufficient in others.
32
![Page 46: Bigger, better, faster, more! Sample size calculation · 2020-01-17 · Bigger, better, faster, more! –Sample size calculation SEMINAR SERIES: HOW TO RUIN YOUR CAREFULLY PLANNED](https://reader033.vdocuments.us/reader033/viewer/2022050400/5f7ded6643276d3100725053/html5/thumbnails/46.jpg)
First name Name Tel/Mobile number E-mail address