Logic OfSignificance Testing

Logic of Significance Testing

• Statistical Hypothesis: an assumption about a population parameter, where the assumption may or may not be true

• Hypothesis Testing: the formal procedures used by statistics to accept or reject statistical hypotheses

Statistical Hypotheses • Uses a simple random sample from population- if sample data is

not consistent with statistical hypothesis, the hypothesis is rejected

Two Types of Statistical Hypotheses:1. Null Hypothesis (Ho): the hypothesis that sample observations

result purely from chance2. Alternative Hypothesis (Ha): hypothesis that sample observations

are influenced by some non-random cause.

• Ex: If you wanted to determine if a die was fair, the null hypothesis might be the chance of rolling a 2 and the alternative hypothesis would be not rolling a 2.– Ho: p= 1/6– Ha: p = 1/6

– Suppose we rolled the die 50 times and the 2 came up 45 times. We would have to reject the null hypothesis , and claim that the die was not a fair die.

Two ways to make a claim:

“reject the null hypothesis”OR

“fail to reject the null hypothesis”

• Why do you think we cannot say “accept the null hypothesis”?

Hypothesis Tests• Formal process to make a claim about Ho and Ha, based on

sample data.1- State the Hypotheses (both null and alternative)

* Must be stated such that Ho & Ha are mutually exclusive

2- formulate an analysis plan- describes how to use sample data and evaluate the null hypothesis

3- analyze sample data- must find test statistic (mean, proportion, t/z- score) described in analysis plan

4- interpret results. Apply decision rule described in analysis plan.(using test statistic you will either reject H0 or fail to reject H0)

* if test statistic is unlikely reject Ho*** if p-value < α then it is significant, and you reject the null hypothesis if p-value > α then it is not significant, and you fail to reject the null hypothesis

Decision Errors• Type I Error: occurs when a true null hypothesis is

rejected– The significance level is the probability (α ) of

committing a Type I error

• Type II Error: occurs when a false null hypothesis is not rejected– Beta ( β) is the probability of committing a Type II

error

– The Power test is the probability of not committing a Type II error

•

Decision Rules• Two ways for statisticians two describe their decision rules for

rejecting the null hypothesis (Ho)– P-value: the strength of evidence in support of a null hypothesis

– Suppose S is the test statistic. Then the probability of observing a TS as extreme as S is known as the P-value, assuming the null hypothesis is true. If P-value is less than the significance level, we reject the null hypothesis.

– Region of Acceptance: this is a range of values. Region is defined such that the significance level is the probability of making a Type I error• If the TS falls within the acceptance region, the null hypothesis is not

rejected

– Region of Rejection: set of values outside the acceptance region. • If the TS falls within the rejection region, the null hypothesis is rejected

**Both of these methods are comparable

One-Tailed Test• A statistical hypothesis test, where region of rejection is

on only one side of the sampling distribution

• Suppose Ho: μ=20 and Ha: μ>20 The rejection region would be all the numbers to the right of 20 in the sampling distribution

Two –Tailed test• Statistical Hypothesis test where region of rejection is on both

sides of the sampling distribution• Suppose Ho: μ=20 and Ha: μ = 20, this means that the mean is

either greater than or less than 20. – The rejection region consists of the set of values located on both sides of

the sampling distribution where rejection numbers would be on either the left side or right side of acceptance region.