statistical inference. distribution of the sample mean take a random sample of n independent...
TRANSCRIPT
Statistical inference
Distribution of the sample mean
• Take a random sample of n independent observations from a population.
• Calculate the mean of these n sample values. This is know as the sample mean.
• Repeat the procedure until you have taken all possible samples of size n, calculating the sample mean of each.
• Form a distribution of all the sample means.
• The distribution that would be formed is called the sampling distribution of means.
In many cases we need to infer information about a population based on the values obtained from random samples of the population.
Consider the following procedure:
Mean and variance of the sampling distribution of means
2
.
Consider a population in which and .
Suppose we take a total of samples, each of independent observations, on the random variable
Each sample will have a first observation, a second
X E X Var X
m n X
observation, and so on. Denote by , the random
variable "the th observation".iX
i
1 2
1 2
1 1 1
Then, the sample distribution of means is
n
n
X X XX
n
X X Xn n n
1 2
1 2
1 2
1 1 1
1 1 1
1 1 1
1 1 1
(since )
n
n
n
E X E X X Xn n n
E X E X E Xn n n
E X E X E X E aX aE Xn n n
n n n
1 2
1 2
21 22 2 2
2 2 2
2
1 1 1
1 1 1
1 1 1
1 1 1
(since )
n
n
n
Var X Var X X Xn n n
Var X Var X Var Xn n n
Var X Var X Var X Var aX a Var Xn n n
n n n
n
2
The standard deviation of the sampling distribution is , sometimes written .
This is known as the .
n n
standard error of the mean
The distribution of for normal X X
2
2, ,If then X N X Nn
Read Example 9.13 & 9.14, pp.438-440
The distribution of for non-normal X X
Central limit theorem
2
2
,
For samples taken from a non-normal population with mean and variance ,
by the Central Limit Theorem, is approximately normal and
provided
X
X Nn
30 that the sample size, , is large ( , say).n n
This theorem holds when the population of X is discrete or continuous!!
Read Example 9.15, pp.442-443
Do Exercise 9C, pp.443-444
Unbiased estimates of population parameters
Suppose that you do not know the value of a particular population parameter of a distribution. It seems sensible to take a random sample from the distribution and use it in some way to make estimates of unknown parameters.
The estimate is unbiased if the average (or expectation) of a large number of values taken in the same way is the true value of the parameter.
Point estimates,
ˆ
ˆ
If the random sample taken is of size
the best unbiased estimate of , the population mean, is where
( is the mean of the sample)
the best unbiased estimate
n
xx x
n
2 2
2 2 2
ˆ
ˆ1
of , the population variance, is where
( is the variance of the sample) n
s sn
22
2 2 21ˆ ˆ
1 1Alternatively, use or
xx xx
n n n
Read Example 9.17, p.448
Unbiased estimates of population parameters
Interval estimates
Another way of using a sample value to estimate an unknown population parameter is to construct an interval, known as a confidence interval. This is an interval that has a specified probability of including the parameter.
( , )90% 95% 99%
The interval is usually written , with and known as .The probabilities most often used in confidence intervals are , and .
a b a b confidence limits
95%( , )
0.95
For example, to work out a confidence interval for the unknown mean of a particular population, we construct an interval such that
The interval constructed uses
a b
P a b
the value of the mean, , of a random sample of
size taken from the population.x
nThree questions
• Is the distribution of the population normal or not?
• Is the variance of the population known?
• Is the sample size large or small?
Confidence interval for µ(normal population, known population variance, any sample size)
The goal is to calculate the end-values of a 95% confidence interval. We can adapt this approach for other levels of confidence.
2
, ,For random samples of size , if , then .n X N X Nn
0,1Standardizing, , where .X
Z Z Nn
95% 95%For a confidence interval, we find the -values between which of thedistribution lies.
z
1
0.975
(0.975)
1.960
1.96
The values of are .
P Z z
z
z
1.96 1.96 0.95
1.96 1.96
So
i.e.
XP
n
P X Xn n
2 95%
1.96 , 1.96
If is the mean of a random sample of any size taken from a normal population with
known variance , then a confidence interval for is given by
x n
x xn
n
Read Example 9.19, pp.451-452
Confidence interval for µ(normal population, known population variance, any sample size)
This computer simulation shows 100 confidence intervals constructed at the 95% level. On average 5% do not include µ.
In other words, on average, 95% of the intervals constructed will include the true population mean.
Critical z-values in confidence intervals
The z-value in the confidence interval is known as the critical value.
90%
1.645 , 1.645
confidence interval
x xn n
95%
1.96 , 1.96
confidence interval
x xn n
99%
2.576 , 2.576
confidence interval
x xn n
Confidence interval for µ(non-normal population, known population variance, large sample size)
30In this case, since the sample size is large (say, ), the Central Limit Theorem
may be used.
n
2
,i.e. is approximately normal and .X X Nn
2
30
95%
If is the mean of a random sample of size , where is large ( ),
taken from a non-normal population with known variance , then a
confidence interval for is given by
x n n n
1.96 , 1.96 x xn n
• for a given sample size, the greater the level of confidence, the wider the confidence interval;
• for a given confidence level, the smaller the interval width, the larger the sample size required;
• for a given interval width, the greater the level of confidence, the larger the sample size required.
Note
Read Examples 9.20 – 9.22, pp.454-457
Confidence interval for µ(any population, unknown population variance, large sample size)
2 2ˆIn this case, is used as an estimate for . Ideally, the distribution of
should be normal, but an approximate confidence interval may also be given
when the distribution of is not normal.
X
X
2 2 2
2 2
30 95%
ˆ ˆ1.96 , 1.96
ˆ1
1ˆ
1
Provided that is large, ( , say), a confidence interval for is
,
where , ( is the sample variance)
or
n n
x xn n
ns s
n
xx
n
2
.n
Read Examples 9.23 & 9.24, pp.458-459
Do Exercise 9e, pp.460-461
Confidence interval for µ(normal population, unknown population variance, small sample size)
2
30
When calculating confidence intervals, we have already encountered
the situation when ( ) are taken from a population
with .
n
large samples normal
unknown variance
0,1ˆ
For large samples: , where X
Z Z Nn
30ˆ
But if the sample size is small ( ), no longer has a normal distribution.X
nn
For small samples:
ˆ, where has a -distribution.
XT T t
n
Confidence interval for µ(normal population, unknown population variance, small sample size)
The t-distribution
The distribution of is a member of a family of -distributions. All -distributions aresymmetric about zero and have a single parameter which is a positive integer.
T t t
( )
is known as the of the distribution and we write
or .T t T t
number of degrees of freedom
2 10 0,1
0,1
This diagram shows and along with .
Note that as increases, gets closer andcloser to .
t t N
t vN
ˆ( 1)
For samples of size , it can be shown that
follows a -distribution with degreesof freedom.
n
XT
nt n
Probability density function of the t-distribution
12 2
12
, 1
12
f t n
t
1
0
z tz t e dt
1 !, n n n
Confidence interval for µ(normal population, unknown population variance, small sample size)
2
2
30
95%
If and are the mean and variance of a small sample ( ) from a normal population
with unknown mean and unknown variance , then a confidence interval for is given by
x s n
2 2ˆ ˆˆ,
1
1 0.95
, 95% 1
, where
and is the value from a distribution such that ,
i.e. encloses of the distribution.
nx t x t s
nn n
t t n P t T t
t t t n
To find the required value of , known
as the , the -distribution
tables are used.
t
tcritical value
95%
0.975
90%
0.95
99%
0.995
For a confidence interval, look
under column .
For a confidence interval, look
under column .
For a confidence interval, look
under column .
Read Examples 9.26 & 9.27, pp.466-468 Do Exercise 9f, p.468
Confidence intervals for µ
Read Examples 9.30, pp. 474-475
Read Examples 9.32 & 9.33, pp. 476-478
Do Exercise 9h, Q1-Q4, Q7, Q10-Q12, Q15, Q17-Q21, pp.478-481
Hypothesis testing- exampleA machine fills ice-packs with liquid. The volume of liquid follows a normal distribution with mean 524 ml and standard deviation 3 ml. The machine breaks down and is repaired. It is suspected the machine now overfills each pack, so a sample of 50 packs is inspected. The sample mean is found to be 524.9 ml. Is the machine over-dispensing?
Is the sample mean high enough to say that the mean volume of all packs has increased? A hypothesis (or significance) test enables a decision to be made on this question.
2
.
, 3
Let be the volume of liquid now dispensed. Let the unknown mean of be
Assume the standard deviation has remained unchanged i.e , where .
X X
X N
0
524
: 524
The hypothesis is made that is ml, i.e. the mean is . This isknown as the , and is written H
0null hypothesis
unchangedH
1
524524
Since it is suspected the mean has , the , , isthat the mean is ml, written : H
1alternative hypothesisincreasedgreater than
H
The test is now carried out.
Hypothesis testing- example50
To carry out the test, the focus shifts from , the volume of liquid in a pack, to ,the mean volume of a sample of packs. In this test, is known as the and its distribution is need
X XX test statistic
ed. 2
2
,
3524 524,
50
We know, for a sample of size , . The test starts by assuming
the null hypothesis is true, so . Hence .
n X Nn
X N
The result of the test depends on the location in the sampling distribution of the test value 524.9 ml.
If it is close to 524 then it is likely to have come from a distribution with mean 524 ml and there would not be enough evidence to say the mean volume has increased.
If it is far away from 524 (i.e. in the upper tail of the distribution) then it is unlikely to have come from a distribution with mean 524 ml and the mean is likely to be higher that 524 ml.
A decision needs to be made about the cut off point, , knownas the , which indicates the boundary of the regionwhere values of would be considered from 524 mland therefo
c
xcritical value
too far awayre . The region is known as the
or .critical region rejection regionunlikely to occur
Hypothesis testing- example524 524
Here, if lies in the critical region, we , (that the meanis ml), , (that the mean is greater than ml).
If does not lie
x
x
0
1
reject the null hypothesisin favour of the alternative hypothesis
HH
0 in the critical region, there is not enough evidence to reject , so is . (In this case, is known as the .)
Hx c0 accepted acceptance regionH
%For a significance level of , if the sample mean lies in the critical (or rejection)region, the result is said to be .
%significant at the level
0,1 5%Since the distribution of is normal, instead of finding , the critical value, it is possibleto use the standardized distribution and find the -value that gives in the upper tail.
X c xN z
0.05 1.645We require , and the standard normal table yields .P Z z z 0 1.645
524
3 50
We can now state the : reject if , where
.
Note the rejection criterion should be established the sample is taken.
H zx x
zn
rejection criterion
before
0
1
524.9 524524.9 2.12 1.645
3 50For the sample taken , so . Since , is rejected
in favour of .
x z z H
H
Conclusion: There is evidence, at the 5% level, that the mean volume of liquid being dispensed by the machine has increased.
One-tailed and two-tailed tests
0Say that the null hypothesis is .
1
:In a , the alternative hypothesis looks for an increase or adecrease in
H
one - tailed test
1 0For an increase, is and the critical region is in the .H upper tail
1 0For a decrease, is and the critical region is in the .H lower tail
1
1 0
In a , the alternative hypothesis looks for a change withoutspecifying an increase or decrease and is . The critical region is intwo parts:
HH
two - tailed test
Critical z-values and rejection criteria
Example: At the 1% level
One-tailed tests
Two-tailed tests
Stages in the hypothesis test
Testing the mean, µ, of a population(normal population, known variance, any sample size)
2
2
0 ,
When testing the mean of a normal population with known variance
for samples of size , the test statistic is
, where .
In standardized form, the test statis
X
n
X X Nn
0 0,1
tic is
where .
XZ Z N
n
Read Examples 11.1 & 11.2, pp.514-517
Testing the mean, µ, of a population(non-normal population, known variance, large sample size)
2
2
0 ,
When testing the mean of a non-normal population with known variance ,
provided that the sample size is large, the test statistic is
, where is approximately normal,
X
n
X X X Nn
0 0,1
.
In standardized form, the test statistic is
where .
XZ Z N
n
Read Examples 11.3, pp.517-518
Testing the mean, µ, of a population(any population, unknown variance, large sample size)
2
2
0
ˆ,
When testing the mean of a non-normal population with unknown
variance , provided that the sample size is large, the test statistic is
, where and
X
n
X X Nn
2 2 2
2
2 2
0
ˆ1
1ˆ
1
0,1ˆ
, ( is the sample variance)
or .
In standardized form, the test statistic is
where .
ns s
n
xx
n n
XZ Z N
n
Read Examples 11.4, pp.519-520
Do Exercise 11A, Q1-Q6, Q8-Q10, Q13, pp.522-523
Testing the mean, µ, of a population(normal population, unknown variance, small sample size)
2
0 1ˆ
ˆ
When testing the mean of a normal population with unknown
variance , when the sample size is small, the test statistic is
, where and with
X
n
XT T T t n
n
2 2 2
2
2 2
1
1ˆ
1
, ( is the sample variance)
or .
ns s
n
xx
n n
Read Examples 11.6 &11.7, pp.524-526
Do Exercise 11b, pp.527-528
1 2Testing the difference in means, ,
of two normal populations
1 2 1 2
2 21 1 1 2 2 2, ,
Consider two normal populations and with unknown means, and .
So and and we want to test the difference
between the means of these populations.
X X
X N X N
0 1 2
1 1 2 1 2 1 2 )
The hypothesis might be:
:
: (or or
H
H
0 1 2
0 1 2
:
: 0
Often the test involves the null hypothesis that the means are the same i.e.
or .
H
H
1 1 1
2 2 2
.
.
Take a random sample of size from and find its sample mean
Take a random sample of size from and find its sample mean
n X x
n X x
1 2
1 2 1 2 1 2
2 21 2
1 2 1 21 2
The test statistic is and to consider this sampling distribution we use
X X
E X X E X E X
Var X X Var X Var Xn n
1 2
( )
Testing of two normal populations
2 2
2variances and known
2 21 2
2 21 2
1 2 1 2 1 21 2
1 2 1 2
2 21 2
1 2
,
0,1
If the variances and are known, the test statistic is
where .
In standardized form, the test statistic is
where .
X X X X Nn n
X XZ Z N
n n
2 21 2
1 2 1 21 2
95% 1.96The confidence limits for are .x xn n
1 2
( )
Testing of two normal populations
2
common known variance
2 2 21 2
2
21 2 1 2 1 2
1 2
1 1,
If there is a common population variance ( )
and is known, then the test statistic is
where .
In standardized form, the test statistic is
X X X X Nn n
Z
1 2 1 2
1 2
0,11 1
where .X X
Z N
n n
1 2 1 21 2
1 195% 1.96The confidence limits for are .x x
n n
2Pooled two-sample estimate of 2
2
2 22 2 21 1 2 2
1 21 2
ˆ
ˆ2
If the common population variance, , is unknown, then an unbiased
estimate, , is used instead. This is known as a
, where
( and are n s n s
s sn n
pooled two- sample
estimate
the sample variances)
2
2 22 22 2
1 1 2 2 1 22
1 2 1 2
ˆ
1 1
ˆ2 2
An alternative format for is
i i i ix x x x
x x x x n n
n n n n
1 2The distribution of depends on whether the samples taken
are large or small.
X X
1 2
( )
Testing of two normal populations ,
2
common unknown variance large sample
1 2
21 2 1 2 1 2
1 2
1 2
1 1ˆ,
For large samples the distribution of is approximately normal.
The test statistic is
where .
In standardized form, the test statistic is
X X
X X X X Nn n
X XZ
1 2
1 2
0,11 1
ˆ
where .Z N
n n
1 2 1 21 2
1 1ˆ95% 1.96The confidence limits for are .x xn n
1 2
( )
Testing of two normal populations ,
2
common unknown variance small sample
1 2
1 2 1 21 2
1 2
21 1
ˆ
For small samples the standardized form of the distribution of
follows a -distribution. The test statistic is
where .
X X
t
X XT T t n n
n n
1 2 1 21 2
1 2
1 1ˆ95%
0.975 2
The confidence limits for are ,
where is such that for .
x x tn n
t P T t t n n
Read Examples 11.11 - 11.15, pp.536-542
Do Exercise 11d, Sections A & B, pp.543-546
Paired samplesIt is widely thought that people’s reaction times are shorter in the morning and increase as the day goes on. A light is programmed to flash at random intervals and the experimental subject has to press a buzzer as soon as possible and the delay is recorded.
Experiment 1: Two random samples of 40 students are selected from the school register. One of these samples, chosen at random, uses the apparatus during the first period of the day, while the second sample uses the apparatus during the last period of the day. The means of the two samples are compared.
Experiment 2: A random sample of 40 students is selected from the school register. Each student is tested in the first period of the day, and again in the last period. The difference in reaction times between the two periods for each student is calculated. The mean difference is compared with zero.
Experiment 1 requires a standard two-sample comparison of means, assuming a common variance. There is nothing wrong with this procedure but we could be misled. Firstly, suppose that all the bookworms were in the first sample and all the athletes in the second- we might conclude that reaction times decrease over the course of the day! More subtly, the variations between the students may be much greater than any changes in individual students over the time of day; these changes may pass unnoticed.
In Experiment 2 the variability between the students plays no part. All that matters is the variability of the changes within each student’s readings. The problems with Experiment 1 have vanished! Experiment 2 is a paired-sample test.
The paired-sample comparison of means
Let denote the mean of the distribution of differences between the paired values.d
0
1
: 0
: 0 0 0
Let
(or or as appropriate)d
d d d
H
H
1 2
0
, , ,
0
We have a single set of pairs of values and are interested in the ,
which assuming , are a random sample from a population with mean .
An unbiased estimate of the unknown vari
nn d d d
H
differences
22 21ˆ
1
ance of the population is given by
i
d i
dd
n n
We have created a single sample situation, so previous methods now apply!
95%For example, if the differences can be assumed to have a normal distribution,or if is sufficiently large that a normal approximation can be used, then a confidence interval for is provided by:d
n2 2ˆ ˆ
1.96 , 1.96 , where id dd
d d dn n n
1Alternatively, if the differences can be presumed to have a normal distribution,
but is small, then a distribution can be used.n t n
Paired sample- an example0.001
Suppose that Experiment 2 on the reaction times is carried out, with the following results(in units of seconds):
1%We analyse these data at the significance level.2 2ˆ266 9574 6.650 200.1308The summary statistics are , so that and i i dd d d
0
1
: 0
: 0
The hypotheses being compared are:
d
d
H
H
2
6.6502.97
2.237ˆ
The test statistic is , given by:
d
z
dz
n
02.97 2.3261%
Since , is rejected.There is evidence, at the level,that reaction times increase through the day.
H
Distinguishing between thepaired-sample and two-sample cases
• If the two samples are of unequal size then they are not paired.
• Two samples of equal size are paired only if we can be certain that each observation from the second sample is associated with a corresponding observation from the first sample.
Questions
A significance test for the
product-moment correlation coefficient
0
1
: 0
: 0
(no correlation)
(positive correlation)
H
H
6n
0
1
: 0
: 0
(no correlation)
(negative correlation)
H
H
8n
0
1
: 0
: 0
(no correlation)
(some correlation)
H
H
10n
Read Examples 13.1, pp.603-604 Do Exercise 13a, p.604