1

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing

• General Idea

• How unusual is the result?

• Test statistics

• Type I error (alpha level)

• p-value

• Type II error (beta level) – Power=1-beta

2

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

• Total cholesterol (mg/dl) is measured on a simple random sample of 32 women over the age of 60. Is there evidence that mean cholesterol is different in women of this age group as compared with women under age 50?

• Estimate of TC: (see ejs09b540p36.sas)

32n 244Y 34S • Plot histogram of SRS of n=32 from women <50. (see

ejs09b540p37.sas)Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230

0

5

10

Pe

rce

nt

mn_samp

244

Result is very unusual relative to what we’d expect from sampling.

Conclude the mean is differnet.

3

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

32n244Y 34S

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230

0

5

10

Pe

rce

nt

mn_samp

244

• Histogram of distribution of sample means/standardized value-– need to know mean and variance of TC for women < 50.

– use Z if variance is known, t if variance is estimated

0cal

y

yz

0cal

y

yt

s

• for women < 50:

4

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

32n244Y 34S

244

• Plot Histogram of distribution of sample means under Null H

• or … histogram of standardized values of the difference of the sample mean from the mean TC for women < 50– need to know mean and variance of TC for women < 50.

0cal

y

yz

0cal

y

yt

s

• for women < 50:0 194

35

244 194

35 / 328.08

calz

244 194

34 / 328.32

calt

– use Z if variance is known, t if variance is estimated

5

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Pe

rce

nt

mn_samp 244

6

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 2. Histogram of sample z-scores of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/28/2009 by ejs

-4.0 -3.4 -2.8 -2.2 -1.6 -1.0 -0.4 0.2 0.8 1.4 2.0 2.6 3.2 3.8 4.4 5.0 5.6 6.2 6.8 7.4 8.0 8.6

0

5

10 P

erc

en

t

z

z=8.08

7

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

z=8.3

Figure 3. Histogram of sample t-scores of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/28/2009 by ejs

-4.0 -3.4 -2.8 -2.2 -1.6 -1.0 -0.4 0.2 0.8 1.4 2.0 2.6 3.2 3.8 4.4 5.0 5.6 6.2 6.8 7.4 8.0 8.6

0

5

10

Pe

rce

nt

t

z=8.32

8

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

• Is the result ‘unusual’?– Decide a level of ‘unusualness’– usually set at values so that 5% of time, sample mean

would be further away (also called TYPE 1 Error)– If in either direction, then 2.5% on either side, and

test is called ‘2-sided’• Called 2-sided test

– If unusual is important only in one direction (drug lowers cholesterol), then put all 5% on one side

• Called 1-sided test

– Null hypothesis is ‘usual’ or commonly accepted position.

– Alternative hypothesis is what you want to ‘prove’

0.05level

9

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

Null Hypothesis:

0 0:H

10

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

AlternativeHypothesis

0 0:H

1 0:H

11

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

2-sided test

0 0:H

1 0:H

UnusualUnusual

Criticalregion

12

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

1-sided test

0 0:H

1 0:H

Unusual

Criticalregion

13

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

2-sided test

0 0:H

1 0:H

UnusualUnusual

y

2 ( )p value P Y y

14

Introduction to Biostatistics (PUBHLTH 540) Hypothesis Testing-General Idea

Figure 1. Histogram of sample means of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

1-sided test

0 0:H

1 0:H

Unusual

y

( )p value P Y y

15

Introduction to Biostatistics (PUBHLTH 540) Power of a Test-General Idea

Figure 1. One Sided test for a particular alternative and Histogram of sample means assuming null hypothesis is true based on of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

1-sided test

0 0:H

1 : 227H

Unusual

y

( )p value P Y y

16

Introduction to Biostatistics (PUBHLTH 540) Power of a Test-General Idea

Figure 1. One Sided test for a particular alternative and Histogram of sample means assuming null hypothesis is true based on of n=32 for tc from Population of N=93

Source: ejs09b540p37.sas 10/27/2009 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Per

cent

mn_samp

0

1-sided test

0 0:H

1 : 227H

Unusual

y

( )p value P Y y

17

Introduction to Biostatistics (PUBHLTH 540) Power of a Test-General Idea

0

1-sided test

0 0:H

1 : 227H

Unusual

y

( )p value P Y y

Figure 1. Histogram of sample means of n=32 for tca from Population of N=93

Source: ejs09b540p37a.sas 11/16/2010 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Pe

rce

nt

mn_samp

Unusual

0

1( 206 | 227)

0.999

Power P Y

1 227

1-sided test

0 0:H

1 : 227H

18

Introduction to Biostatistics (PUBHLTH 540) Power of a Test-General Idea

0

1-sided test

0 0:H

1 : 227H

Unusual

y

( )p value P Y y

Figure 1. Histogram of sample means of n=32 for tca from Population of N=93

Source: ejs09b540p37a.sas 11/16/2010 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Pe

rce

nt

mn_samp

Unusual

0

1( 206 | 227)

0.999

Power P Y

1 227

1-sided test

0 0:H

1 : 227H

19

Introduction to Biostatistics (PUBHLTH 540) Power of a Test-General Idea

0

1-sided test

0 0:H

1 : 227H

Unusual

y

( )p value P Y y

Figure 1. Histogram of sample means of n=32 for tca from Population of N=93

Source: ejs09b540p37a.sas 11/16/2010 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Pe

rce

nt

mn_samp

Unusual

0

1( 206 | 227)

0.999

Power P Y

1-sided test

0 0:H

1 : 227H

Figure 1. Histogram of sample means of n=32 for tca from Population of N=93

Source: ejs09b540p37a.sas 11/16/2010 by ejs

170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245

0

5

10

Pe

rce

nt

mn_samp

0

1( 206 | 215)

0.962

Power

P Y

1 215

1-sided test

0 0:H

1 : 215H

1 0.962

0.034

Power

20

Introduction to Biostatistics (PUBHLTH 540) Power- example

• Assume in one population, we know TC for males is normally distributed with mean 220, and variance 1524. Our interest is in mean TC for men in a different population. We would like to know whether TC is less in the other population (vs a null hypothesis that it is equal to or greater than 220). Consider a one sided test of the null hypothesis. Suppose we select a sample of n=25 subjects from the new population. Let us test the null hypothesis that the mean is 220, versus an alternative hypothesis that the mean is 205 based on a one sided test with n=25. What is the power of the test?

• Figure out the rejection region under the null hypothesis in terms of the distribution of sample means.

• Make a sketch indicating the critical region (on the scale of TC).

• Use the z-applet with an assumption that the alternative hypothesis is true to figure the power.

21

Introduction to Biostatistics (PUBHLTH 540) Power- example

• Figure out the rejection region under the null hypothesis in terms of the distribution of sample means.

00

1.645 0.05

1.645 1.645

39220 1.645

5

207

P Z

YP P Y

n n

P Y

P Y

22

Introduction to Biostatistics (PUBHLTH 540) Power- example

• Figure out the rejection region under the null hypothesis in terms of the distribution of sample means.

23

Introduction to Biostatistics (PUBHLTH 540) Power- example

• Make a sketch. 0.6012Power