power and sample size
DESCRIPTION
Power and Sample Size. Shaun Purcell & Danielle Posthuma Twin Workshop March 2002. Aims of Session. Introduce concept of power and errors in inference Practical 1 : Using probability distribution functions to calculate power Power in the classical ACE twin study - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/1.jpg)
Power and Sample Size
Shaun Purcell & Danielle Posthuma
Twin Workshop March 2002
![Page 2: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/2.jpg)
Aims of Session
Introduce concept of power and errors in inference
Practical 1 : Using probability distribution
functions to calculate power
Power in the classical ACE twin study
Practical 2 : using Mx to calculate power
Practical 3 : Monte-Carlo simulation
![Page 3: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/3.jpg)
Power primer
Statistics (e.g. chi-squared, z-score) are continuous
measures of support for a certain hypothesis
Test statistic
Inevitably leads to two types of mistake : false positive (YES instead of NO) (Type I)false negative (NO instead of YES) (Type II)
YES OR NO decision-making : significance testing
YESNO
![Page 4: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/4.jpg)
Hypothesis testing
Null hypothesis : no effect
A ‘significant’ result means that we can reject the
null hypothesis
A ‘nonsignificant’ result means that we cannot
reject the null hypothesis
![Page 5: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/5.jpg)
Statistical significance
The ‘p-value’
The probability of a false positive error if the null
were in fact true
Typically, we are willing to incorrectly reject the null
5% or 1% of the time (Type I error)
![Page 6: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/6.jpg)
Misunderstandings
p - VALUES
that the p value is the probability of the null
hypothesis being true
that very low p values mean large and important
effects
NULL HYPOTHESIS
that nonrejection of the null implies its truth
![Page 7: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/7.jpg)
Limitations
IF A RESULT IS SIGNIFICANT
leads to the conclusion that the null is false
BUT, this may be trivial
IF A RESULT IS NONSIGNIFICANT
leads only to the conclusion that it cannot be
concluded that the null is false
![Page 8: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/8.jpg)
Alternate hypothesis
Neyman & Pearson (1928)
ALTERNATE HYPOTHESIS
specifies a precise, non-null state of affairs with
associated risk of error
![Page 9: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/9.jpg)
P(T)
T
Critical value
Sampling distribution if HA were true
Sampling distribution if H0 were true
![Page 10: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/10.jpg)
Rejection of H0 Nonrejection of H0
H0 true
HA true
POWER =(1- )
Nonsignificant resultType I error at rate
Type II error at rate
Significant result
![Page 11: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/11.jpg)
Power
The probability of rejection of a false null-
hypothesis
depends on - the significance crtierion ()- the sample size (N) - the effect size (NCP)
“The probability of detecting a given effect size in a population from a sample of size N, using significance criterion ”
![Page 12: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/12.jpg)
Impact of alpha
P(T)
T
Critical value
![Page 13: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/13.jpg)
Impact of effect size, N
P(T)
T
Critical value
![Page 14: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/14.jpg)
Applications
POWER SURVEYS / META-ANALYSES- low power undermines the confidence that can be
placed in statistically significant results
INTERPRETING NONSIGIFICANT RESULTS- nonsignficant results only meaningful if power is high
EXPERIMENTAL DESIGN- avoiding false positives vs. dealing with false negatives
MAGNITUDE VS. SIGNIFICANCE- highly significant very important
![Page 15: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/15.jpg)
Practical Exercise 1
Calculation of power for simple case-control study.
DATA : frequency of risk factor in 30 cases and 30
controls
TEST : 2-by-2 contingency table : chi-squared
(1 degree of freedom)
![Page 16: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/16.jpg)
Step 1 : determine expected chi-squared
Hypothetical risk factor frequencies
Case Control
Risk present 20 10
Risk absent 10 20
Chi-squared statistic = 6.666
E
EO 22 )(
![Page 17: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/17.jpg)
P(T)
T
Critical value
Step 2. Determine the critical value for a given type I error rate,
- inverse central chi-squared distribution
![Page 18: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/18.jpg)
P(T)
T
Critical value
Step 3. Determine the power for a given critical valueand non-centrality parameter
- non-central chi-squared distribution
![Page 19: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/19.jpg)
Calculating Power
1. Calculate critical value (Inverse central 2)
Alpha 0 (under the null)
2. Calculate power (Non-central 2)
Crit. value Expected NCP
![Page 20: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/20.jpg)
http://workshop.colorado.edu/~pshaun/
df = 1 , NCP = 0
X
0.05
0.01
0.001
3.84146
6.63489
10.82754
![Page 21: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/21.jpg)
Determining power
df = 1 , NCP = 6.666
X Power
0.05 3.84146
0.01 6.6349
0.001 10.827
0.73
0.50
0.24
![Page 22: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/22.jpg)
Exercise 1
Calculate power (for the 3 levels of alpha) if sample
size were two times larger (assume proportions
remain constant) ?
Hint: the NCP is a linear function of sample size, and will also
be two times larger
![Page 23: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/23.jpg)
Answers
df = 1 , NCP = 13.333
X Power
0.05 3.84146
0.01 6.6349
0.001 10.827
nb. Stata : di 1-nchi(df,NCP,invchi(df,))
0.95
0.86
0.64
![Page 24: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/24.jpg)
Twin 1
A C E
a c e
Twin 1
A C E
a’ 0 e’
Estimating power for twin models
The power to detect, e.g., common environment
Expected covariance matrices arecalculated under the alternate model :
Fit model to data with value of interest fixed to null value, e.g. c = 0
NCP = -2LLSUB
![Page 25: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/25.jpg)
0.51 0.28
0.41 0.20
Model A C E
1 30% 20% 50%
2 0% 20% 80%
(350 MZ pairs, 350 DZ pairs)
Model Power to detect C
Alpha 0.05 0.01
1
2
Using power.mx script
![Page 26: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/26.jpg)
Using power.mx script
Qu. You observe MZ and DZ correlations of 0.8
and 0.5 respectively, in 100 MZ and 100 DZ twin
pairs. What is the power to detect an additive
genetic effect, with a type I error rate of 1 in
1000?
![Page 27: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/27.jpg)
Absolute ACE effects
Power to detect :
A C E A C
0.1 0.1 0.8 0.02 0.02
0.2 0.2 0.6 0.06 0.09
0.3 0.3 0.4 0.29 0.32
0.4 0.4 0.2 0.95 0.79
150 MZ twins, 150 DZ twins, = 0.01
![Page 28: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/28.jpg)
Relative ACE effects
Power to detect :
A C E A C
0.2 0.2 0.6 0.06 0.09
0.2 0.0 0.8 0.57
0.0 0.2 0.8 0.82
150 MZ twins, 150 DZ twins, = 0.01
![Page 29: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/29.jpg)
Sample Size
NMZ NDZ A C
150 150 0.83 0.53
250 250 0.98 0.86
350 350 1.00 0.96
500 500 1.00 0.99
A:C:E = 2:2:1, = 0.001
![Page 30: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/30.jpg)
NCP and power
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 5 10 15 20
NCP
Power
![Page 31: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/31.jpg)
Relative MZ and DZ sample N
NMZ NDZ A C
150 150 0.83 0.53
500 500 1.00 0.99
500 150 0.99 0.56
150 500 0.95 0.99
A:C:E = 2:2:1, = 0.001
![Page 32: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/32.jpg)
Increasing power
Increase sample size
Increase
Multivariate analysis
Adding other family members
![Page 33: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/33.jpg)
Adding other siblings
Power compared to twins only design
(keeping total # individuals constant)
Power to detect
A C D
+ 1 sibling + ++ ++
+ 2 siblings - ++ ++
![Page 34: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/34.jpg)
Monte-Carlo simulation
Instead of calculating expected NCP under
population parameter values, simulate multiple
randomly-sampled datasets
Perform test on each dataset
Due to random sampling variation, the effect will
not always be detectable
The proportion of significant results Power
![Page 35: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/35.jpg)
P(T)
T
Expected NCP
Critical value
P(T)
T
Critical value
![Page 36: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/36.jpg)
![Page 37: Power and Sample Size](https://reader035.vdocuments.us/reader035/viewer/2022081501/568148bf550346895db5db9e/html5/thumbnails/37.jpg)
More importantly...
Meike says …
“people are going skiing Saturday and all are
welcome to join”