Education 793 Class Notes

Decisions, Error and Power

Presentation 8

2

Review: Chain of reasoning

Population with parameters

Random selection

Sample with statistics

Probability

Underlying distributions of the statistic

Inference

Random selection can take several forms (such as simple, systematic, cluster, or stratified random sampling), and is intended to generate a sample that represents the population.

3

Four Steps

1. State the hypothesisH0 vs. HAlternate

2. Identify your criterion for rejecting H0

  Directional or non-directional testOne-tailed or two-tailed

 

Set alpha level (Prob. incorrectly rejecting H0)

3. Compute test statistic General form: Test = statistic – expected parameter

standard error of statistic

4. Decide about H0

4

Million Dollar Question

• To Reject or Not To Reject?

5

Review: Inferential error

• Type I: Alpha– Rejecting the null hypothesis with the null

hypothesis is really true

• Type II: Beta– Failing to reject the null hypothesis when the null

hypothesis is in fact false

6

Possible outcomes

DecisionIn the population,

H0 is true

In the population,

H0 is false

Reject H0

(H0 false)Type I error

Correct decision

Fail to reject H0

(H0 true)Correct decision

Type II error

7

Considerations

• Basic considerations– What is the statistic of interest?– Is a one-tailed versus two-tailed test appropriate?– What is the nature of the sample?

• Dependent versus independent

• How much power is available?– Power is 1 – – Factors influencing power:

• Increasing the number of observations• Reducing the error in the data

8

Statistical Power

• Power is the probability of correctly rejecting a false null hypothesis. As such, power is defined as:

1 - ß where ß is the Type II error probability

• Two kinds of power analyses:– A priori: Used to identify what the sample size

needs to be to identify a specified effect– Post hoc: Used when the null hypothesis has not

been rejected, and when you want to know the probability that you have committed a Type II error

9

What Power is all about

• Power analysis is about Type II errors, “missed effects” – failing to reject H0: when there really is an effect in the population

• “Power” is the antithesis of “risk of Type II error” – Risk of Type II error = 1 - power – Power = 1 - Risk of Type II error

10

Trade off Between and Power

• Conundrum: As we decrease (Type I Error), Type II Error) increases. The two errors have an inverse relationship. In order to deal with the problem, researchers follow an established set of guidelines. =.05 or .01

• Which error is the most dangerous?

11

Power of a Test

• Example: Ho: =72, H1: <72 =.05, n=36

Review: Given the sample data, a decision is made to reject or not reject the null hypothesis.

=72

zcrit= -1.65

=Type I Error

This distribution is assumed to be true under the Null

12

Power of a Test

• If the null hypothesis is false, then the above distribution is NOT the one we sampled from. We must have sampled from one of the possible alternative distributions, of which there are an infinite number. Recall H1: <72

zcrit= -1.65

=Type I Error

=72

13

Power of a Test

• The steps in computing power require the researcher to set a predetermined difference from the two means (1) that is meaningful (Effect Size).

• With three pieces of information, N, we can estimate the power of a test.

• We will let computer programs do this for us.

14

Factors Affecting Power

• Size of the difference between population means, what the book refers to as 1.– The greater the effect size (1), the greater the

power will be. Common sense, as the difference gets larger, it will be easier to detect.

• Significance level– As the power of a statistical test increases so does

This is the trade off between Type I and Type II Errors.

15

Factors Affecting Power

• Variance– All other factors being equal (, N, ), the

smaller the standard deviation in the population, the greater the power.

• Sample size– By increasing N, we decrease the standard

deviation in the sampling distribution. Hence the power is increased.

16

Power as a function of…

the true difference in

significancelevel

                        

samplesize

17

Next Week

• Chapter 12 p. 333-367 and

Available through JSTOR at http://www.jstor.org/journals/00221546.html