Introduction to Introduction to Hypothesis TestingHypothesis Testing

What is a Hypothesis Test?What is a Hypothesis Test?

A hypothesis test is a statistical method A hypothesis test is a statistical method that uses sample data to evaluate a that uses sample data to evaluate a hypothesis about a populationhypothesis about a population

FalsifiabilityFalsifiability

A good hypothesis is one that is falsifiableA good hypothesis is one that is falsifiable You cannot prove something that cannot You cannot prove something that cannot

be disprovedbe disproved Better yet, you cannot support a Better yet, you cannot support a

hypothesis if you cannot disconfirm ithypothesis if you cannot disconfirm it What are some examples of hypotheses What are some examples of hypotheses

that cannot be falsified?that cannot be falsified? What are examples of ones that can?What are examples of ones that can?

What Are The Steps For What Are The Steps For Hypothesis Testing?Hypothesis Testing?

First we state the null hypothesis HFirst we state the null hypothesis H0.0.

What is the null hypothesis?What is the null hypothesis? This states that in the general population This states that in the general population

there is no change, no difference, or no there is no change, no difference, or no relationship.relationship.

Basically it says the opposite of what we are Basically it says the opposite of what we are hoping to show.hoping to show.

Hypothesis Testing ContinuedHypothesis Testing Continued

Then we state the alternative hypothesisThen we state the alternative hypothesis What is the alternative hypothesis?What is the alternative hypothesis? This states that there is a change, a This states that there is a change, a

difference, or a relationship for the general difference, or a relationship for the general populationpopulation

This is where we state what we believe This is where we state what we believe (hypothesize) to be true(hypothesize) to be true

Why Do We Do This?Why Do We Do This?

There is no way to PROVE a hypothesis. There is no way to PROVE a hypothesis. You can only support a hypothesis, or You can only support a hypothesis, or reject it. If you support it 100,000 times, reject it. If you support it 100,000 times, and then on the 100,001and then on the 100,001stst time you reject time you reject it, the hypothesis is not true.it, the hypothesis is not true.

So, we seek to reject the null, and thus, So, we seek to reject the null, and thus, conversely we support the alternative.conversely we support the alternative.

The Next Step (Hypothesis The Next Step (Hypothesis Testing)Testing)

Set the evaluation criteriaSet the evaluation criteria By this, we are looking to assess an By this, we are looking to assess an

acceptable level of error by chanceacceptable level of error by chance What do we think is an acceptable What do we think is an acceptable

probability that the data we are looking at probability that the data we are looking at is “different”is “different”

Alpha LevelsAlpha Levels

Usually we use Usually we use α = .05.α = .05. The alpha level or the level of significance is a The alpha level or the level of significance is a

probability value that is used to define the probability value that is used to define the very unlikely sample outcomes if the null very unlikely sample outcomes if the null hypothesis is true. In this case we would hypothesis is true. In this case we would expect to obtain this “outlier” sample in only expect to obtain this “outlier” sample in only 5% of the samples simply by chance.5% of the samples simply by chance.

This corresponds to p = .05.This corresponds to p = .05. In other words, the probability of obtaining In other words, the probability of obtaining

this difference by chance is 5%.this difference by chance is 5%.

Critical RegionCritical Region

The critical region is composed of extreme The critical region is composed of extreme sample values that are very unlikely to be sample values that are very unlikely to be obtained if the null hypothesis is true. The obtained if the null hypothesis is true. The boundaries for the critical region are boundaries for the critical region are determined by the alpha level. If sample determined by the alpha level. If sample data fall in the critical region, the null data fall in the critical region, the null hypothesis is rejected.hypothesis is rejected.

The Next Step (Hypothesis The Next Step (Hypothesis Testing)Testing)

Collect DataCollect Data Compute sample statisticsCompute sample statistics Compute test statisticsCompute test statistics

What Are the Relevant Sample What Are the Relevant Sample Statistics?Statistics?

The mean (M)The mean (M) The sample size (n)The sample size (n)

What Are the Relevant What Are the Relevant Population Parameters?Population Parameters?

μμ σσ Where do we get these parameters?Where do we get these parameters?

From our hypothesesFrom our hypotheses

Now We Calculate the Test Now We Calculate the Test StatisticStatistic

The formula for a z-statistic isThe formula for a z-statistic is z = (M – μ) / σz = (M – μ) / σmm

First we calculateFirst we calculate σσmm = σ/√n = σ/√n

The we use the values to get zThe we use the values to get z Finally we make a decision based on the z Finally we make a decision based on the z

statistic and the alpha level we have statistic and the alpha level we have chosenchosen

Decision?Decision?

Given the calculations we have performed, Given the calculations we have performed, and the alpha levels chosen, are we going and the alpha levels chosen, are we going to accept or reject the null hypothesis?to accept or reject the null hypothesis? We fail to reject the nullWe fail to reject the null

ErrorError

ß

αα

Type I and Type IIType I and Type II

Type IType I Occurs when a researcher rejects a null hypothesis Occurs when a researcher rejects a null hypothesis

that is actually truethat is actually true Occurs at the rate of the alpha level we setOccurs at the rate of the alpha level we set

Type IIType II Occurs when a researcher fails to reject a null Occurs when a researcher fails to reject a null

hypothesis that is really falsehypothesis that is really false No easy calculation. How do we know if we have No easy calculation. How do we know if we have

made this type of error? It is NOT the converse of made this type of error? It is NOT the converse of Type IType I

We must estimate betaWe must estimate beta

PracticePractice

Page 241Page 241 Mu = 18Mu = 18 Sigma = 4Sigma = 4 n = 16n = 16 M = 15M = 15 Alpha = .05Alpha = .05

What Is Meant by Significance?What Is Meant by Significance?

A result is said to be significant or statistically A result is said to be significant or statistically significant if it is very unlikely to occur when the significant if it is very unlikely to occur when the null hypothesis is true. That is, the result is null hypothesis is true. That is, the result is sufficient to reject the null hypothesis.sufficient to reject the null hypothesis.

What factors influence significance?What factors influence significance? The size of the difference.The size of the difference. The variability of the scores.The variability of the scores. The number of scores in the sample.The number of scores in the sample.

Is there a difference between significance and Is there a difference between significance and meaningfulness?meaningfulness?

Assumptions For Hypothesis Assumptions For Hypothesis Tests With z-ScoresTests With z-Scores

All statistical tests are based on a certain set of All statistical tests are based on a certain set of assumptions that, when violated, may bias the assumptions that, when violated, may bias the statistic, and give us misleading resultsstatistic, and give us misleading results

Assumptions for hypothesis tests with z-scoresAssumptions for hypothesis tests with z-scores Random SamplingRandom Sampling Independent ObservationsIndependent Observations The Value of sigma is unchanged by the treatmentThe Value of sigma is unchanged by the treatment Normal sampling distributionNormal sampling distribution

Random SamplingRandom Sampling

It is assumed that the subjects used to It is assumed that the subjects used to obtain the sample data were selected obtain the sample data were selected randomlyrandomly

Independent ObservationsIndependent Observations

The values in the sample must consist of The values in the sample must consist of independent observations.independent observations. Two events are independent if the occurrence Two events are independent if the occurrence

of the first event has no effect on the of the first event has no effect on the probability of the second event.probability of the second event.

Sigma UnchangedSigma Unchanged

Because sigma is unknown we must make Because sigma is unknown we must make an assumptionan assumption We assume that the standard deviation for the We assume that the standard deviation for the

unknown population (after treatment) is the unknown population (after treatment) is the same as it was for the population before the same as it was for the population before the treatmenttreatment

In other words, the treatment affects the In other words, the treatment affects the mean, not the standard deviationmean, not the standard deviation

Normal Sampling DistributionNormal Sampling Distribution

The distribution of sample means must be The distribution of sample means must be normal since we have been using the unit normal since we have been using the unit normal table to identify probabilitiesnormal table to identify probabilities

Directional Hypothesis TestsDirectional Hypothesis Tests

In a directional hypothesis test, or a one-In a directional hypothesis test, or a one-tailed test, the statistical hypothesis (htailed test, the statistical hypothesis (h00

and Hand H11) specify either an increase or a ) specify either an increase or a

decrease in the population mean score. decrease in the population mean score. That is, they make a statement about the That is, they make a statement about the direction of the effect.direction of the effect.

This halves the critical region since it is This halves the critical region since it is only taking into account the one tail.only taking into account the one tail.

Effect SizeEffect Size

Demonstrating a significant treatment effect Demonstrating a significant treatment effect does not necessarily indicate a substantial does not necessarily indicate a substantial treatment effect.treatment effect.

This is because we are looking at the relative This is because we are looking at the relative magnitude of the difference in the sample and magnitude of the difference in the sample and the population mean with respect to the S.E.the population mean with respect to the S.E.

What if n is very large, or sigma is very small?What if n is very large, or sigma is very small? Then a small difference in the means may in fact be Then a small difference in the means may in fact be

significant.significant.

Cohen’s dCohen’s d

One of the simplest and most direct One of the simplest and most direct methods for measuring effect size is methods for measuring effect size is Cohen’s dCohen’s d

Cohen’s d = (mean difference) / (standard Cohen’s d = (mean difference) / (standard deviation)deviation)

PowerPower

The power of a statistical test is the The power of a statistical test is the probability that the test will correctly reject probability that the test will correctly reject a false null hypothesis. That is, power is a false null hypothesis. That is, power is the probability that the test will identify a the probability that the test will identify a treatment effect if one really existstreatment effect if one really exists

What is the relation between power and What is the relation between power and error?error? 1 - 1 - ßß

What Affects Power?What Affects Power?

Effect sizeEffect size Sample sizeSample size Alpha levelAlpha level Number of tails in the testNumber of tails in the test