Hypothesis Testing

By Rama Krishna Kompella

Outline

• The Null Hypothesis• Type I and Type II Error• Using Statistics to test the Null Hypothesis• The Logic of Data Analysis

Research Questions and Hypotheses

• Research question:– Non-directional:

• No stated expectation about outcome– Example:

• Do men and women differ in terms of brand loyalty?• Hypothesis:

– Statement of expected relationship• Directionality of relationship

– Example:• Women will have greater brand loyalty than men

Grounding Hypotheses in Theory

• Hypotheses have an underlying rationale:– Logical reasoning behind the direction of the

hypotheses (theoretical rationale – explanation)– Why do we expect women to have better brand

loyalty? • Theoretical rationale based on:

– 1. Past research– 2. Existing theory– 3. Logical reasoning

The Null Hypothesis

• Null Hypothesis - the absence of a relationship– E..g., There is no difference between men’s and

women’s with regards to brand loyalty• Compare observed results to Null Hypothesis

– How different are the results from the null hypothesis?• We do not propose a null hypothesis as research

hypothesis - need very large sample size / power– Used as point of contrast for testing

Hypotheses testing

• When we test observed results against null:– We can make two decisions:

• 1. Accept the null– No significant relationship– Observed results similar to the Null Hypothesis

• 2. Reject the null– Significant relationship– Observed results different from the Null Hypothesis

– Whichever decision, we risk making an error

Type I and Type II Error• 1. Type I Error

– Reality: No relationship– Decision: Reject the null

• Believe your research hypothesis have received support when in fact you should have disconfirmed it

• Analogy: Find an innocent man guilty of a crime• 2. Type II Error

– Reality: Relationship– Decision: Accept the null

• Believe your research hypothesis has not received support when in fact you should have rejected the null.

• Analogy: Find a guilty man innocent of a crime

Potential outcomes of testingDecision

Accept Null Reject Null

RE No A RelationshipLI RelationshipTY

1 2

3 4

Potential outcomes of testingDecision

Accept Null Reject Null

RE No A RelationshipLI RelationshipTY

Correct decision 2

3 4

Potential outcomes of testingDecision

Accept Null Reject Null

RE No A RelationshipLI RelationshipTY

1 2

3Correctdecision

Potential outcomes of testingDecision

Accept Null Reject Null

RE No A RelationshipLI RelationshipTY

1 Type I Error

3 4

Potential outcomes of testingDecision

Accept Null Reject Null

RE No A RelationshipLI RelationshipTY

1 2

Type II Error 4

Potential outcomes of testingDecision

Accept Null Reject Null

RE No A RelationshipLI RelationshipTY Type II Error

Correctdecision

Type I Error Correctdecision

Function of Statistical Tests

• Statistical tests determine:– Accept or Reject the Null Hypothesis

• Based on probability of making a Type I error– Observed results compared to the results

expected by the Null Hypotheses– What is the probability of getting observed

results if Null Hypothesis were true?• If results would occur less than 5% of the time by

simple chance then we reject the Null Hypothesis

Start by setting level of risk of making a Type I Error

• How dangerous is it to make a Type I Error:– What risk is acceptable?:

• 5%? • 1%?• .1%?

– Smaller percentages are more conservative in guarding against a Type I Error

• Level of acceptable risk is called “Significance level” :– Usually the cutoff - <.05

Conventional Significance Levels

• .05 level (5% chance of Type I Error)• .01 level (1% chance of Type I Error)• .001 level (.1% chance of Type I Error)• Rejecting the Null at the .05 level means:

– Taking a 5% risk of making a Type I Error

Steps in Hypothesis Testing• 1. State research hypothesis• 2. State null hypothesis• 3.Set significance level (e.g., .05 level)• 4. Observe results• 5. Statistics calculate probability of results if

null hypothesis were true• 6. If probability of observed results is less than

significance level, then reject the null

Guarding against Type I Error

• Significance level regulates Type I Error• Conservative standards reduce Type I Error:

– .01 instead of .05, especially with large sample• Reducing the probability of Type I Error:

– Increases the probability of Type II Error• Sample size regulates Type II Error

– The larger the sample, the lower the probability of Type II Error occurring in conservative testing

Statistical Power

• The power to detect significant relationships– The larger the sample size, the more power– The larger the sample size, the lower the

probability of Type II Error• Power = 1 – probability of Type II Error

Statistical Analysis

• Statistical analysis:– Examines observed data– Calculates the probability that the results could

occur by chance (I.e., if Null was true)• Choice of statistical test depends on:

– Level of measurement of the variables in question:

• Nominal, Ordinal, Interval or Ratio

Logic of data analysis

• Univariate analysis– One variable at a time (descriptive)

• Bivariate analysis– Two variables at a time (testing relationships)

• Multivariate analysis– More than two variables at a time (testing

relationships and controlling for other variables)

Variables

• Dependent variable:– What we are trying to predict– E.g., Brand preference

• Independent variables:– What we are using as predictors– E.g., Gender, Product usage history

Commonality across all statistical analysis procedures

• Set the significance level:– E.g., .05 level

• Means that we are willing to conclude that there is a relationship if:

– Chance of Type I error is less than 5%

• Statistical tests tell us whether:– The observed relationship has less than a 5%

chance of occurring by chance

Summary of Statistical Procedures

Variables Procedure

Nominal IV, Nominal DV Chi-square

Nominal IV, Ratio DV T-test

Multiple Nominal IVs, Ratio DV

ANOVA

Ratio IV, Ratio DV Pearson’s R

Multiple Nominal IVs, Ratio DV with ratio covariates

ANCOVA

Multiple ratio Multiple Regression

Q & As