Hypothesis Testing

Ginger Holmes Rowell, Ph. D.

April 24, 2002

Hypothesis Testing (HT)

•Uses

•Types

•General Procedures

•Example

•Review of Error Types* Type I & Type II

Hypothesis Testing - Uses• Medical – Clinical Trials

* Show that Drug A lowers cholesterol more than the placebo (2-Sample for difference in means)

* Show that the percent of people who have side effects from from Drug A is NOT different from the group taking the placebo (2-Sample for difference in proportions)

Hypothesis Testing - Uses• Agriculture

* Show that fertilizer A works better than fertilizer B

• Psychology Experiments* Show that Intervention A keeps kids off of

drugs better than what has been done traditionally

• Education* Show that teaching method A is more effective

than teaching method B

Two Broad Categories of HT

Parametric Methods: frequently used, more common

Nonparametric Methods: used if sample size is small and populations are not normal(We will not talk about this type of HT.)

Param etric M ethods (Make assum ptions about the underlying distributions, like norm ality)

Z-test t-test

Mea ns

Zfor la rge sa m ple

Proportion

C hi-Squa re

Va ria nce

1-Sa m ple

ZLa rge Sa m ple

tSm a ll Sa m ple

D ifferences in Mea ns

Proportion ZLa rge Sa m ple

D ifferences in Proportions

F test

D ifferences in Va ria nces

2-Sa m ples

O n e-W ay A N O V A

M ean s

C h i-S q u areG ood n ess o f F it Tes t

P rop ortion s

L even e 's o r B artle tt 's

V arian ces

M ore than 2Independent Sam ples

HT General Procedure - Steps

* Conduct a literature review or pilot study

* Form your hypothesis* Set your threshold of error (determine

desired significance level)

* Design your experiment* Determine which HT to use* Determine Needed Sample Size

HT Procedure: Steps Contin.

* Collect your data

* Enter data into appropriate tool (calculator, statistical software)

* Examine data graphically

* Find descriptive statistics• Use these methods to check for

typos, errors, …

HT Procedure: Steps Contin.

* Find your “test statistic” and use it determine whether your data supports the null or alternative hypothesis.

HT Example

•Research Hypothesis* Using CI Applets will help students

learn more about CI’s

•Set Error Limit: 5% for Type I Error

•Design Experiment* Give a pre-test, students use CI

applet, give a post-test

Statistical Analysis

• Hypothesis* Using the java applets will improve content

knowledge for math teachers• Ho: improvement = 0

• Ha: improvement > 0

• Improvement = Posttest score – Pretest Score

• Statistical Test* Paired difference t-test (small sample)

• Average improvement was approximately normal

HT Example

•Decide which HT to use ???

•Decide the needed sample size

•Collect Data* You helped with that part

•Examine Data* Graphically, descriptive statistics* Check for typos, errors, ...

HT Example - DATA

1 4 3 2 -1 3 -1 0 1 1 2 0

Points Improved = post test score – pre test score

(Each test is scored out of 5 possible points)

HT Example - DATA

1 4 3 2 -1 3 -1 0 1 1 2 0

Variable N Mean StDev SE Mean

Points Improved 12 1.3 1.6 0.463

-1 0 1 2 3 4

C1

(with Ho and 95% t-confidence interval for the mean)

[ ]

X_

Ho

Points Improved

HT Example

•What does our data say?

• Is that enough evidence to reject the null in favor of the alternative?* What do you think?* How much evidence do you need?* Especially with this small sample size.

HT Example - The WORK

• Test statistic

• Interpretation: The test statistic tells you the number of standard deviations that the sample mean (or proportion or variance) falls from the hypothesized value.

0

sample mean hypothesized valuestandard deviation of the mean

t

xt

sn

Using Your TI-83 for HT

•Press STAT>TESTS>t-test* Input: DATA0: 0

* LIST: LI

* Freq: 1: >

* select Calculate

HT Example

• Test Statistic = t= 2.70• What does this tell us? (Remember the Empirical Rule)

• If the null is true, the chance of getting a data set like ours or one that supports the alternative even more is small (1% chance).

• We got our data set & did not make any errors.

• Do you believe the null hypothesis is true?

HT Example

• Conclusion* Reject the null in favor of the alternative

hypothesis

• What does that mean in the context of our problem?* We can expect an average improvement in

content knowledge for math teacher who use the Regression applet

HT Example

•Next Question: How much improvement

•Answer: Find a 95% confidence interval* Review – use your calculator to do

this and interpret the result.

Conclusion: Regression Applet

• A statistically significant improvement in their average content knowledge can be expected for math teachers using the Regression applet (Rice Probability Webs). (t=2.7, n=12, p=0.01)

• We expect (with 95% confidence) that the average improvement in content knowledge of regression will be between 0.2 and 2.3 points on a 5 point scale.

Review

•Null Hypothesis

•Alternative

Type I and II Errors

•Type I Error

•Type II Error

•Power of Test

Sample Size

•What is the effect of sample size on statistical power?

Comments

• I hope you * Understand how to think about

forming a hypothesis* Understand that actually testing the

hypothesis is more than looking at the two sample averages and saying whether you think they are different.

Questions

????????