bmi 541/699 lecture 12lindstro/12.assumptions.10.13.pdf · 0.8 rel. freq. hist. of sample sample...
TRANSCRIPT
BMI 541/699 Lecture 12
We have covered:
1. Introduction and Experimental Design
2. Exploratory Data Analysis
3. Probability
4. Distribution of sample statistics
5. Testing hypotheses about the sample mean(s)- One sample t-test- Two sample t-test (two sided p-value)- Confidence interval for the di↵erence of two means- Paired t-test- checking assumptions for t-based methods
1 / 37
Assessing assumptions for T-based methods
What are the assumptions for t-tests and confidence intervalsbased on t-distributions?
• The sample(s) are SRSs from large population(s).
• The distribution of X̄ (s) is approximately normal.
1. The distribution of X (s) not too skew.2. The sample size is large enough given the distribution of X (s)
2 / 37
How do we check these assumptions?
To find out if the sample is an SRS you must find out how the datawere gathered by talking to the person who gathered the data.
We can check the assumption that the distribution of X̄ (s) isapproximately normally using plots.
Either
• we can be reasonably sure that X̄ (s) is approximately normallydistributed.
Or
• We can tell it is not normal or we don’t have enoughinformation to tell. In this case we use other (non-t-based)methods.
3 / 37
To check for normal X̄ .
• plot the data (histogram, dot-plot, strip-plot, box-plot))- is the sample distribution relatively symmetric?- is n large enough for the CLT to hold?
What if we can’t decide?
• simulate the distribution of X̄ assuming the populationdistribution is equal to the sample distribution.
• plot the simulated distribution of X̄ and see if it is close tonormal.
4 / 37
Estimating the distribution of X̄We can create an estimate of the distribution of X̄ by re-sampling(also called bootstrapping).
• the sample consists of n = 20 observations on the variable X .
0.76 1.36 1.59 1.73 2.28 2.90 3.143.26 3.47 3.48 3.92 4.24 4.64 4.925.64 5.77 5.99 6.21 6.57 8.0
x̄ = 3.995• the relative frequency histogram of the sample is our bestestimate for the distribution of X .
x
Freq
uenc
y
0 2 4 6 8
01
23
45
Freq. Hist. of sample
x
Rel
ative
Fre
quen
cy
0 2 4 6 8
0.00
0.10
0.20
Rel. Freq. Hist. of sample
5 / 37
The bootstrap distribution of the sample meanTo create the bootstrap sampling distribution of the sample meanfor our sample:
1) draw a sample of size n=20 observations from the sample atrandom with replacement.
One possible bootstrap sample:0.76 1.36 1.73 2.28 2.28 2.28 3.143.14 3.26 3.26 3.26 3.47 3.47 3.483.48 3.48 3.48 3.48 4.24 6.21
Note that some observations come up more than once andsome are missing
2) calculate mean of the bootstrap sample. x̄b = 3.08
Repeat steps 1 and 2 many (10,000) times.
Plot the relative frequency histogram of the 10,000 bootstrapsample means.
6 / 37
Example 1
x
Rel
ative
Fre
quen
cy
0 2 4 6 8
0.00
0.10
0.20
Rel. Freq. Hist. of samplesample size = 20 , mean = 3.99
xb
Rel
ative
Fre
quen
cy
3 4 5 6
0.0
0.4
0.8
Rel. Freq. Hist. of bootstrap sample meanssample size = 20 , mean = 3.99
The line is the density function for a normal distribution with thesame mean and standard deviation as the bootstrap distribution.
• Sample relatively symmetric.
• CLT is likely to work for this sample.
• T based methods can be used to construct a confidenceinterval or hypothesis test.
7 / 37
Example 3
x
Rel
ative
Fre
quen
cy
0 20 40 60 80
0.00
00.
015
0.03
0
Rel. Freq. Hist. of samplesample size = 20 , mean = 22.25
xb
Rel
ative
Fre
quen
cy
10 20 30 40 50
0.00
0.04
0.08
Rel. Freq. Hist. of bootstrap sample meanssample size = 20 , mean = 22.27
• Sample of size 20.
• The frequency histogram is skewed to the right.
• CLT still work pretty well.
8 / 37
Example 4
x
Rel
ative
Fre
quen
cy
0 1 2 3 4 5 6 7
0.0
0.4
0.8
Rel. Freq. Hist. of samplesample size = 20 , mean = 0.82
xb
Rel
ative
Fre
quen
cy
0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.0
0.4
0.8
1.2
Rel. Freq. Hist. of bootstrap sample meanssample size = 20 , mean = 0.82
• Sample is severely skewed right.
• CLT does not work for n = 20.
• Do not use T based methods.
9 / 37
Example 5
x
Rel
ative
Fre
quen
cy
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
Rel. Freq. Hist. of samplesample size = 20 , mean = 0.51
xb
Rel
ative
Fre
quen
cy
0.2 0.3 0.4 0.5 0.6 0.7 0.8
01
23
45
Rel. Freq. Hist. of bootstrap sample meanssample size = 20 , mean = 0.51
• Sample is symmetric and bimodal.
• CLT works for n = 20.
• Use T based methods.
In general the CLT tends to work well when the distribution of X issymmetric even if not unimodal.
10 / 37
What do we do when the CLT is unlikely to work?
original data
x
Frequency
0 5 10 15
040
80
11 / 37
Skewness - what to do?
If the data are skewed and n is not large enough it’s not safe toassume that the sample mean is normally distributed.
We can sometimes transform data to a scale where the data aremore symmetric.
The CLT will work and the mean in the transformed scale is agood summary of the transformed data.
If the data are:
• positive
• are skewed to the right (long right tail)
Try a square-root, cube-root or log transformation of the data andre-plot.
12 / 37
These three transformations all spread out the observations closeto zero and squeeze the larger observations closer together.
Square-root does the least, cube-root does more and log does themost. original data
x
Frequency
0 5 10 15
040
80120
x^(1/2)
x^(1/2)
Frequency
0 1 2 3 4
020
4060
x^(1/3)
x^(1/3)
Frequency
0.5 1.0 1.5 2.0 2.5
020
40
log(x)
log(x)
Frequency
−2 −1 0 1 2 3
020
40
13 / 37
Example: Two sample comparison of means
A histogram for a two sample comparison (Treatment vs. Control).Control
frequency
0 5 10 15 20 25
04
812
Treatment
frequency
0 5 10 15 20 25
04
812
The standard deviations look di↵erent and the control data alsolook skewed to the right.
14 / 37
Two sample exampleHere is the same data after a cube-root transformation.
Control
x(1 3)frequency
0.0 0.5 1.0 1.5 2.0 2.5 3.0
02
46
8
Treatment
x(1 3)
frequency
0.0 0.5 1.0 1.5 2.0 2.5 3.0
02
46
8
• The distribution in both groups is now closer to symmetric.
• Note that the cube root removes much of the skewness in thecontrols without skewing the distribution of the treatment data.
• We can use a two sample t-test.15 / 37
When using a data transformation always:
• transform all data to be analyzed with the sametransformation
• transform back results (means and CI’s) to the original scaleusing the inverse of the transformation.
16 / 37
Transforming backControl
x(1 3)frequency
0.0 0.5 1.0 1.5 2.0 2.5 3.0
02
46
8
Treatment
x(1 3)
frequency
0.0 0.5 1.0 1.5 2.0 2.5 3.0
02
46
8
After calculating the means and confidence intervals in thetransformed scale
transform them back to the original scale using the inversetransformation.
17 / 37
The estimates in the transformed (cube root) scale are:
let w = 3px be the transformed variable.
• Control group
w = 1.078 95% CI for 3pµCntl = (0.851, 1.306)
• Treatment group
w = 2.157 95% CI for 3pµTmt = (1.962, 2.352)
Notice that the CIs are symmetric around the means in thetransformed scale.
18 / 37
Estimates and confidence intervals for the population means canbe transformed back using the inverse transformation.
In this case the inverse transformation is y3.
• Control group
Cube root scale: wCntl = 1.078 CI = (0.851, 1.306)
Original scale: µ̂Cntl = w
3Cntl = 1.0783 = 1.25 CI = (0.62,
2.23)
• Treatment group
Cube root scale: wTmt = 2.157 CI = (1.962, 2.352).
Original scale µ̂Tmt = w
3Tmt = 2.1573 = 10.04 CI = (7.55,
13.01).
Recall that the “hat” notation means “estimate for”.
After transforming back the CIs are no longer symmetric. Thisreflects the skewness of the data distribution.
19 / 37
If we want an estimate for
µTmt � µCntl
We can use the di↵erence of the estimates for population means inthe original scale (that we transformed back from the cube rootscale)
2.1573 � 1.0783 = 10.04� 1.25 = 8.79
However, there is no confidence interval available for thedi↵erence in the means using standard methods
20 / 37
• Report means and confidence intervals transformed back tooriginal scale
• The CIs in the original scale are no longer symmetric(particularly for the controls). This reflects the skewness inthe original data.
• Di↵erences can only be calculated in the original scale.Di↵erences in the transformed scale cannot be transformedback to the original scale using the inverse transformation.
21 / 37
log transformation with zeros
Example: Data (n = 40) that are strongly skewed to the right.
original scale
Freq
uenc
y
0 20 40 60 80
020
4060
This data set contains 3 zeros.
22 / 37
We would like to use a log transformation but the log of zero isnegative infinity.
In R
> x <- 0:5
> x
[1] 0 1 2 3 4 5
> log(x)
[1] -Inf 0.0000000 0.6931472 1.0986123 1.3862944
If you try to run a t-test on data where some of the values are -Infyou will get an error.
23 / 37
If the data include a small number of zeros and you wish to use alog transformation then either
1) Replace the zeros with a number (call it a) that is smallerthan the smallest non-zero observation and use the logtransformation
2) Use the transformation
w = log(x + a)
The inverse transformation is then
x = exp(w)� a
In both cases choose the number a so that the resultingtransformed data look symmetric.
24 / 37
In our data example:
original scale
Freq
uenc
y
0 20 40 60 80
020
4060
The smallest non-zero value is 0.1.
I’ve decided to replace the zeros with 0.05
25 / 37
log scale
Freq
uenc
y
−2 0 2 4
02
46
810
Note that the transformed data have negative values.This is because some of the observations are smaller than 1.
26 / 37
The mean and 95% CI for mean in the log scale is:
0.427 (0.055, 0.798)
Transformed back using exp(w) we have an estimated mean and95% CI in the original scale:
1.53 (1.06, 2.22)
( | )|1.06 1.53 2.220
Note that the confidence interval is not symmetric back in theoriginal scale.
Also, I changed some zeros to a small number but I still useexp(w) to transform back to the original scale.
27 / 37
Transformation Functions summary
Inverse Inverse Use whenFunction Function In R in R data are
loge(x) e
wlog(x) exp(w) right skewed &
positive
loge(x + a)
⇤e
w � a log(x+a) exp(w)� a right skewed &
positive with some
zeros
x
1/3w
3xˆ(1/3) wˆ3 right skewed &
non-negative
px w
2sqrt(x) wˆ2 right skewed &
non-negative
*Or replace the zeros with a number smaller than the smallestpositive observation before taking the log.
28 / 37
If data include both positive and negative numbers it is oftenbecause the data are di↵erences. Di↵erences are not usuallyskewed.
On the other hand, if the data are ratios they will probably be rightskewed and usually need a log transformation.
29 / 37
Transformation Functions
0 20 40 60 80 100
020
4060
80no transform
original data
trans
form
ed d
ata
0 20 40 60 80 100
02
46
810
x^(1/2)
original data
trans
form
ed d
ata
0 20 40 60 80 100
01
23
4
x^(1/3)
original data
trans
form
ed d
ata
0 20 40 60 80 100
−4−2
02
4
log(x)
original data
trans
form
ed d
ata
30 / 37
Assessing assumptions for T-based methods - Summary
• Assumption 1: The sample(s) are SRSs from largepopulation(s).
- Ask the investigator about how the sample was obtained.
• Assumption 2: The distribution of the sample mean(s) areapproximately normal.
- Plot the data.- If right skew, try a transformation before analyzing the data.- Remember to transform back your sample mean(s) andconfidence interval(s).
- For 2 samples transformation must work for both.- If no transformation works use a non-parametric method (stillto come).
31 / 37
More examples
0 1 2 3 4 5 6
original data
0.0 0.5 1.0 1.5 2.0 2.5
square root
0.5 1.0 1.5
cube root
−8 −6 −4 −2 0 2
log
32 / 37
0 1 2 3 4 5 6
original data
0.5 1.0 1.5 2.0 2.5
square root
0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
cube root
−2 −1 0 1
log
33 / 37
0 1 2 3 4 5 6
original data
0.0 0.5 1.0 1.5 2.0 2.5
square root
0.0 0.5 1.0 1.5
cube root
−25 −20 −15 −10 −5 0
log
34 / 37
0 1 2 3 4 5
original data
1.0 1.2 1.4 1.6 1.8 2.0 2.2
square root
1.0 1.2 1.4 1.6
cube root
0.0 0.5 1.0 1.5
log
35 / 37
0 1 2 3 4 5 6
original data
0.0 0.5 1.0 1.5 2.0 2.5
square root
0.5 1.0 1.5
cube root
−6 −4 −2 0 2
log
36 / 37
0 1 2 3 4 5 6
original data
0.0 0.5 1.0 1.5 2.0 2.5
square root
0.0 0.5 1.0 1.5
cube root
−20 −15 −10 −5 0
log
37 / 37