cive2602 - engineering mathematics 2.2 (20 credits) statistics and probability lecture 9 hypothesis...
TRANSCRIPT
CIVE2602 - Engineering Mathematics 2.2 (20 credits)
Statistics and Probability
Lecture 9
• Hypothesis testing –Examples• Undertaking experiments • t-test for two sample means • P-values• F-tests
Dr D Borman
©Claudio Nunez 2010, sourced fromhttp://commons.wikimedia.org/wiki/File:2010_Chile_earthquake_-_Building_destroyed_in_Concepci%C3%B3n.jpg?uselang=en-gbAvailable under creative commons license
1) Testing a single sample mean
• Does one distribution come from a certain population? • How certain are we that it comes from a different population?
population
sample
??
??
2) Two sample tests on the mean
• Are two distributions different?• Do they come from the same population?• How different?• How certain are we that they are different?
• Is there a difference in the strength of our concrete depending on if we use chemical X or chemical Y? (test sample of each type)
• Does our concrete get stronger if we leave it an additional week to set?
(e.g. sample 1 this week, sample 2 week after)
• Has pollution level in a river changed since a new factory was built?
(before /after sample)
• Has there been a reduction in vibrations to a bridge after modifications
Hypotheses Testing – 2 sample tests ??
We might want to set up an experiment to test for…
©Terraplanner 2007, sourced fromhttp://www.flickr.com/photos/halonfury/1977813099/
Available under creative commons license
Standard Deviation (σ)
Testing X against Y(where Y is population)
Testing X against Y (where Y is another
sample)
σx ~σy
σ is known or
large sample
z - testuse σ (or Sx) = population variance
2 Sample t-test or
2 Sample z-test
σx ≠ σy
or σ is unknown,
small samples
t - testuse σ =Sx
2 Sample t-test
Summary of when to use z-test and t-test
population
sample
??
??
No
‘Student’ t test
• The t-test was developed by W. S. Gossett at around 1899
• Employed at the Guinness brewery in Dublin
• He was responsible for developing procedures for ensuring the similarity of batches of Guinness
• Had to publish anonymously so did so under the name ‘Student’
©Sami Keinanen 2005, sourced fromhttp://commons.wikimedia.org/wiki/File:Guinness.jpg
Available under creative commons license
Ronald Fisher (famous Statistician),
what he says rings even more true today than when he uttered it, back in 1938:
"To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of."
Designing a statistical experiment
http://www.youtube.com/watch?v=BX9iMIC6mcg
Hypotheses Testing (2 sample t-test)
Setting up and performing an experiment
1
)( 22
y
y n
YYS1
)( 22
x
x nXX
S
Formula B: - not equal variances
2
)()( 22222
yx
yx nn
YYXXSSS
Formula A: - assumes equal variances
y
y
x
x
yx
n
s
n
s
YXt
22
)()(
103.1161
133.1161
0)83.2652.301(518.2
33.14
09.36
P-value –value from t-tables(We double it for 2TT so that it can be compared to significance level, that’s because value from tables only shows value for one tail)
1
)( 22
y
y n
YYS1
)( 22
x
x nXX
S
Formula B: - not equal variances
2
)()( 22222
yx
yx nn
YYXXSSS
Formula A: - assumes equal variances
y
y
x
x
yx
n
s
n
s
YXt
22
)()(
103.1161
133.1161
0)83.2652.301(518.2
33.14
09.36
Rejection region for H0
Acceptance region for H0
Critical Level (from tables)
(tcrit)
P-value –value from t-tables(We double it for 2TT so that it can be compared to significance level, that’s because value from tables only shows value for one tail)
3 ways to check if H0 should be rejected (all effectively the same thing) Rejection region for H0
Acceptance region for H0
Critical Level (from tables)
tcrit
-If P-value< significance level (<5%)(p-value found from tables, but doubled for 2TT)
-If value from tables < ½ significance level (2TT)
-If t statistic > tcrit
tcrit=t-value for a particular significance level e.g. 5% (depends on if 1TT or 2TT)
http://www.youtube.com/watch?v=ZFXy_UdlQJg&NR=1
A simple example
• The breaking stress of 66 Oak beams to be used in houses around Yorkshire were measured
• We want to– Describe the data– Test for differences between Red Oak and
Yellow Oak
Good to look at your data before analysis
female male
sex
0.00
50.00
100.00
150.00
200.00
250.00
Box plot
Scatter plot
medianIQR
Red yellow
Oak Type
Breaking strength
Red yellow
Oak Type
Breaking strength
What is the distribution of the data?
The frequency distribution of the breaking stress of Oak beams in 66 Yorkshire houses
WORMS
225.0220.0
215.0210.0
205.0200.0
195.0190.0
185.0180.0
175.0170.0
165.0160.0
155.0150.0
145.0140.0
135.0130.0
Histogram
Fre
qu
en
cy10
8
6
4
2
0
Std. Dev = 19.51
Mean = 174.9
N = 66.00
Approximately normal distribution+/- symmetrical (skewness = 0)
Breaking stress
How do I analyse my data?• Think about analysis before you collect
the data
• What is the form of the data?
• How do I describe the data?
• Hypothesis testingHypothesis testing
• What test do I need?What test do I need?
• What do the results mean?What do the results mean?
Hypothesis testing
• Phrase as a hypothesis: e.g. Strength of beams vary with Oak type
• Rephrase as a Null hypothesis, H0
e.g. “beam strength does not vary with Oak type”
• Test whether data are consistent with H0
• Think of an interesting/appropriate question
Types of null hypothesis
• data follow a given distribution
• a and b are not associated
• means do not differ
• x does not influence y
What test do we need?
• Breaking strength of Oak Beam -Single continuous dependent variable
• Oak Type (Red/Yellow) -Single categorical predictor variable
=> t-test (or ANOVA if more than two)
Could draw a graph show 95% Confidence Intervals of sample mean
• Graph should reflect analysis
• e.g. if analysing means present means,
• Figure legend should be understandable without reference to text
Type of Oak Red Yellow
95% CI
Breaking
strength
Hypothesis testing• Choose the null hypothesis
– H0: breaking strengths of Red and Yellow are equal
• Calculate a test statistic– Mean for Red (x1)= 181, Yellow (x2)= 170
– Calculate t = (x1 - x2)/SE = 11/4.6 (use formula)
– t =2.37
y
y
x
x
yx
n
s
ns
YXt
22
)()(
-Null hypothesis allows us to predict the distribution of the test statistic
-Test the observed value against the predicted distribution assuming the null hypothesis is true
-Is this is significant i.e. larger or smaller than we would expect by chance?
0
0.01
0.02
0.03
0.04
0.05
-4 -2 0 2 4
•If H0 is true, predicted distribution of t for 64 df is
tcrit=-2.00 tcrit=2.00
•There is a probability of 2.1% that H0 is true (p-value= 0.021),
•5% of the time we see t>2.00 or t<-2.00(P-value =0.021, found from tables using t-statistic= 2.37)
t=2.37
Remember n=66, df=v=n-2=64
tcrit-value for 5% significance (from tables)
Therefore, reject H0 and take alternative Hypothesis=> There is a difference between red and yellow oak
Data presentation(what do we need to report)
• The frequency distribution of Oak breaking strength did not differ significantly from normal distribution.– Make it clear that your data meets any assumptions of the tests
used
• The mean (±SD) Breaking strength was 181 kN (±20) for Red Oak (n=30), and 170 kN (±18) for Yellow Oak (n=36)– Present appropriate descriptive statistics
• There was a significant difference in Breaking strength between Red Oak and Yellow Oak (t=2.37, df=64, P=0.021)… – Present the test statistic, degrees of freedom and exact P-value
WORMS
225.0220.0
215.0210.0
205.0200.0
195.0190.0
185.0180.0
175.0170.0
165.0160.0
155.0150.0
145.0140.0
135.0130.0
Histogram
Fre
qu
en
cy
10
8
6
4
2
0
Std. Dev = 19.51
Mean = 174.9
N = 66.00
P-value measures strength of evidence against null hypothesis, not magnitude of effect
SEX
2.22.01.81.61.41.21.0.8
HE
IGH
T
2.4
2.2
2.0
1.8
1.6
1.4
1.2
SEX
2.22.01.81.61.41.21.0.8
HE
IGH
T
2.6
2.4
2.2
2.0
1.8
1.6
1.4
1.2
P<0.0001 P=0.021
Point to note
Interpreting Non-Significant results
• Non-significant result does not mean no effect (could be good to take larger samples!)
• Very hard to prove no effect
• Other techniques can be used
F-test – test on the variance
Example- Comparing Two means with small sample
Five cubes can be made from one mix of concrete. They are tested after 5 days and give the following compressive strengths: 23 17 22 33 25 (say, X)
while another 5 cubes from the same mix are tested after 6 days and give
26 19 29 37 24 (say, Y)
Is there a significant difference between the means for the two samples?
1) –Test variances (not done this yet) – assume underlying variances are the same
SolutionWe have small samples with an unknown σ (population standard deviation)
Use the t-test
nx =5 ny=5
X =24, Y =27Find sample means:
Find sample standard deviation:
1
)( 22
x
x n
XXS
1
)( 22
y
y n
YYS
Formula B:
25.392 S2
)()( 22222
yx
yx nn
YYXXSSS
Formula A:
Degrees of freedom, v = n – 2 = 5+5 -2 = 8
(assume underlying population is Normally distributed)
F-test – test on the variance
XSo far we have only considered testing a sample mean against a population mean (μ).
Here we look at tests on the sample variance to see if two samples have significantly different variances (normally done before comparing means).
2
2
y
x
S
S
Distribution of VariancesIf X and Y are normally distributed with identical variances and if samples of size nx and ny respectively are drawn from each, then the ratio
satisfies an F distribution with
V1 = nx-1 and V2 = ny-1 degrees of freedom.
2xS
2yS
22 )()1(
1XX
nS
xx
22 )()1(
1YY
nS
yy
Remember and are the unbiased estimators of the X and Y population variances given by:
F-test – test on the variance
XSo far we have only considered testing a sample mean against a population mean (μ).
Here we look at tests on the sample variance to see if two samples have significantly different variances.
2
2
y
x
S
S
Distribution of VariancesIf X and Y are normally distributed with identical variances and if samples of size nx and ny respectively are drawn from each, then the ratio
satisfies an F distribution with
V1 = nx-1 and V2 = ny-1 degrees of freedom.
2xS
2yS
22 )()1(
1XX
nS
xx
22 )()1(
1YY
nS
yy
Remember and are the unbiased estimators of the X and Y population variances given by:
F-distribution
F-test
Example – we have two small samples, X and Y
18xn
16yn2002 xS
502 yS
Calculate
2
2
y
x
S
SF
Since our calculated value of F=4 exceeds the value at the 5% significance, we conclude that the two sample variances are different
Degrees of freedom: V1 = nx-1=17
and V2 = ny-1=15
450
200
2.37315,17F
Find F at 5% significance
F – Test-summaryThe F-test is used to test for differences among sample variance. Like
the Student's t, one calculates an F and compares this to a table value.
The formula for F is simply
The variance are arranged so that F>1. That is; s1
2>s22.
We use the F-test as the Student's t test, only we are testing for significant differences in the variances.
1) Invoke the null hypothesis that states that the two variances we are comparing are from the same population. (i.e., they are not statistically different)
2) Calculate the F value (the ratio of the two variances)
3) Look up the table value of F for the degrees of freedom used to calculate both variances and for a given confidence level.
4) If the calculated F is greater than the table value, then the null hypothesis is not correct. Else, the two could have come from the same population of measurements.
2
2
y
x
S
S
T-statistic
B
B
A
A
A
nS
nS
XXt B
22
Difference in the means
‘Sum’ of the standard errors T-statistic =
Assumptions about the T-statistic
You have normally distributed data– You might not have. – You can’t use this test if you don’t.
(there are ways to investigate if a population is likely to be normal- but beyond this course)
- Important to state assumptions you are making
Confidence
• Need something to compare it with
• How certain do you need to be that the means are the same?
• 90% - 1 in 10 you’ll be wrong
• 95% - 1 in 20 you’ll be wrong
• 99%– 1 in 100 you’ll be wrong
What about if more than 2 samples?
- ANOVA The ANalysis Of VAriance (or ANOVA) is a powerful and common statistical procedure
The t-test tells us if the variation between two groups is "significant". Why not just do t-tests for all the pairs of samples.
Multiple t-tests are not the answer because as the number of groups grows, the number of needed pair comparisons grows quickly. For 7 groups there are 21 pairs.
ANOVA puts all the data into one number and gives us one P for the null hypothesis.
http://www.youtube.com/watch?v=BX9iMIC6mcg
Hypotheses Testing (2 sample t-test)
Setting up and performing an experiment - maybe need explanation of p-value 9perhaps cover here!)
Hypothesis testing considerations
Do two samples come from the same population?
Z-test (for very large samples or when σ known)
t-test (for small samples when σ not known) Population should be Normally distributed
Testing if strength of concrete changes if kept in an humidity chamber for a week
We want to find out if leaving concrete in a humidity chamber for a week changes the breaking strength of the concrete.
We have 5 samples to put in the chamber and 6 to leave out in normal drying conditions. Following the week of drying the 11 samples are strength tested.
23 17 22 33 25 (say, X) 26 19 29 37 24 32 (say, Y)The breaking strengths (in KN) are as follows
Is there a significant difference between the means for the two samples?
(assume underlying population is Normally distributed)
Example
21 17 21 33 23 (say, X) 27 25 29 37 24 29 (say, Y)
The breaking strengths are as follows
Is there a significant difference between the means for the two samples? (don’t assume same variances)
SolutionWe have small samples with an unknown σ (population standard deviation)
=>Use the t-test
nx =5 ny=6
X =23, Y =271) Find sample means:
1
)( 22
x
x n
XXS
1
)( 22
y
y n
YYS
2) Find sample standard deviation: Use Formula B (i.e. don’t assume same variances)
3) Degrees of freedom, v = n – 2 = 5+6 -2 = 9
xn
XXge ..
15
..)2317()2321( 2
36
16
..)2919()2725( 2
2.33
Which test statistic to use?
Rejection region for H0
Acceptance region for H0
Critical Level (from tables)
Use the t-test
nx =5 ny=6 X =23, Y =27
0.362 xSDegrees of freedom, v = 9
y
y
x
x
yx
n
s
ns
YXt
22
)()(
H0: μx = μy
H1: μx ≠ μy
H0: the population means are the same
H1: population mean for X is not the same as the population mean for Y.
84.056.3
3
62.33
50.36
0.0)2724(
= 0 :H0 Assumes means are equal
2TT we are looking for any difference (at 5% significance 2.5% in each tail)
So P-value =~ 0.2 x2 =40%
As P-value is higher than 5% significance (i.e.40%>5%) do not reject H0 ->conclude that there is no significant difference in strength between the concrete that has been in the chamber (although might want to try with bigger samples!)
Is there a significant difference between the means for the two samples? (don’t assume same variances)
2.332 yS
From t tables (t-stat =0.84):
Could have also compared the probability of 0.2 from tables with 2.5% (i.e 20% > 2.5) and again would have rejected the H0 (this is just the same)
CIVE2602 - Engineering Mathematics 2.2
Lecture 9- Summary
Hypothesis testing very important technique –used very widely
We have looked at testing to see if there is a difference between 2 means – can test other properties as well
When testing the means- perform an F-test on the variance first
For more than 2 samples we use ANOVA
If X and Y are normally distributed with means μx, and μy, and population standard
deviations σx and σy, and independent random samples of size nx and ny are
drawn from X and Y, then the difference between the sample means YX
is normally distributed with mean: )( YXE μx - μy
Differences of Sample Means: known population Variances (or estimated using very large samples)
y
y
x
x
nnyxYXVar
22
)var()var()(
and variance
y
y
x
x
nnYXSD
22
)(
or standard deviation
x
x
nXSD
2
)(
Compare with:
Choosing the null hypothesis
• “x does not affect y”• “distribution is not different from normal”• “means of 2 groups do not differ”• could be “means of 2 groups differ by
3.1”• NOT “means of 2 groups are different”
Allows a prediction to be made about expected value of a statistic
Summary of Tables in Notes
• P101 of notes - Normal tables -finds probabilities from z-values
• P102 of notes - Reverse Normal tables -finds z-statistics from probabilities.
• P103 of notes - t-tables (reverse) -finds t-statistic from probabilities (can use to look up probabilities from t-stat as well).
• P104 – F-tables, used for F-test on variances (not covered this yet)
Find probability for z=2.12. prob=
0.0170
What are the assumptions of an independent samples t test?
• Assumptions. For the equal-variance t test, -the observations should be independent, random
samples from normal distributions with the same population variance. Formula A
For the unequal-variance t test, -the observations should be independent, random
samples from normal distributions. Formula B
Hypothesis test of two sample means is also referred to as the ‘Student’ T test
• The t-test was developed by W. S. Gossett at around 1899
• Employed at the Guinness brewery in Dublin• He was responsible for developing procedures
for ensuring the similarity of batches of Guinness
• Had to publish anonymously so did so under the name ‘Student’
• T-test is sometimes referred to as "Student's” t-test. http://www.youtube.com/watch?v=snrajeiZLK0&feature=related
©Sami Keinanen 2005, sourced fromhttp://commons.wikimedia.org/wiki/File:Guinness.jpg
Available under creative commons license
We want to find if a new factory being built near the river Aire, has changed the level of pollution in the river.
Before the factory is built we take 30 samples and find that the mean pollution level of these samples to be 21.51 and the standard deviation to be 4.3.
After the factory has been built a further 40 samples are taken from the river and the mean level is found to be 19.51 and sample standard distribution is 3.10.
Q Can we say at 5% significance level that the level of pollution in the river has changed.
i.e. Is there a significant difference between the underlying pollution levels of the river on these two occasions
WORK IN GROUPS OF 2-3
Summary of Procedure
• Compare your t-statistic with the one in the table
• Need to decide on the – degrees of freedom and the – percentile you are trying to check within– Whether it is a 1 tail or a 2 tail comparison– Usually use 2 tails
T-Statistic Tables
3 variants of t-test
• Comparing 2 independent datasets– Equal variances (use Formula A)– Unequal variances (use Formula B)
• Matched pair comparison
Comparing 2 independent samples
• You have two different datasets looking at the same thing. Are the means significantly different?
e.g. Is patient recovery rate faster with drug A or drug B
e.g. Is the fuel economy better with Car Type A or Car Type B
e.g. Is Concrete with Add
Matched pair comparison
• You have two datasets looking at the same thing with pairs that are links. Are the means significantly different?
e.g. Do patients temperatures change after you give the drug A
Procedure for independent samples
• Find the mean of each group• Find the number of observations in each group• Find the standard deviation in each group• Put the numbers into the formula• Calculate the t-statistic
• Compare the t-statics with the ones in the table• Is your value higher than the critical one (you can
disprove the null) or lower (you can’t)
B
B
A
A
A
nS
nS
XXt B
22
Analysis of lots of data
• Measure size of leaves on maple trees from 2 different sites
• Are the means the same at all the sites?
• Perform a t-test
Analysis of lots of data
• Measure size of leaves on maple trees from 10 different sites
• Could perform t-tests between all the combinations
• 1-2, 1-3, 1-4, 1-5, 1-6, 1-7, 1-8,
1-9, 1-10
• 2-3, 2-4, 2-5, 2-6,2-7…..
• 3-4, 3-5, 3-6, 3-7,3-8,
Analysis of lots of data
• 45 different combinations
• Testing at 95% confidence, will find significant differences between the samples just because of the number of tests you are doing.
• Not the best way of doing this!!!!
FisherSir Ronald Aylmer Fisher
Geneticist
‘Fisher was a genius who almost single-handedly created the foundations for modern statistical science’ Hald, Anders (1998). A History of Mathematical Statistics. New York: Wiley.
"I occasionally meet geneticists who ask me whether it is true that the great geneticist R.A. Fisher was also an important statistician" (Annals of Statistics, 1976).
Fisher didn’t believe that smoking caused lung cancer.
Fisher’s analysis
• If we have several samples taken from identical populations we can estimate the standard deviation in 2 ways
1. The spread of values within the samples
2. The spread of the mean of the samples• Providing the samples are from identical
populations the two ways should give approximately the same result
http://www.youtube.com/watch?v=abjHpJ36pIE
http://www.youtube.com/watch?v=snrajeiZLK0&feature=related
T-test information
http://www.youtube.com/watch?v=ZFXy_UdlQJg&feature=related
http://www.youtube.com/watch?v=N4aHi_g0vgQ&feature=related
http://www.youtube.com/watch?v=BX9iMIC6mcg
http://www.youtube.com/watch?v=ZFXy_UdlQJg&NR=1P-value-use
Hypothesis test-use