Chapter 15Data Analysis: Testing for

Significant Differences

Chapter 15Data Analysis: Testing for

Significant Differences

Value of Testing for Data

Differences

Analysis of variance (ANOVA)Analysis of variance (ANOVA)

Hypothesis testingHypothesis testing

t-distribution and associated confidence interval estimationt-distribution and associated

confidence interval estimation

Central tendency and dispersionCentral tendency and dispersion

Common to all marketing research projects

Common to all marketing research projects

Basic Statistics and Descriptive

Analysis

15-2

• Mode – most common value in a set of responses; i.e., the question response most often given. (Nominal data)

• Median – middle value of a rank ordered distribution; half of the responses are above and half below the median value. (Ordinal data)

• Mean – average of the sample. (Interval and Ratio data)

• Range – the distance between the smallest and largest values of the variable.

• Standard deviation – the average distance of the dispersion of the values from the mean.

• Variance – the squared standard deviation.

Statistical Measures

15-3

• Inferential statistics – to make a determination about a population on the basis of a sample.

oSample – a subset of the population.

oSample statistics – measures obtained directly from sample data.

oPopulation parameter – a measured characteristic of the population.• Actual population parameters are unknown since the

cost to perform a census of the population is prohibitive.

Analyzing Relationships of Sample Data

15-4

Type I:Rejection of the null hypothesis when, in fact, it is true.(Convict an innocent man)

Type II:Acceptance of the null hypothesis when, in fact, it is false.(Set a guilty man free)

Tests are either one or two-tailed. This approach depends on the nature of the situation and what the researcher is demonstrating.

One-Tailed Tests (provide some direction)“If you take the medicine, you will get better”

Two-Tailed Tests (there is a difference)“If you take the medicine, you will get either better or worse.”

Chapter 15

Hypothesis Testing – Error Types

Actual State of theNull Hypothesis

Fail to Reject Ho Reject Ho

Ho is true

Ho is false

Correct (1-) no error

Type II error ()

Type I error ()

Correct (1- ) no error

Chapter 15

Significance Testing Error Issues

Type I and Type II Errors

Alpha and Beta (chances of making a Type I or Type II error) are related. If we set one very small (Alpha is .0001%), then we make the other very large. We are most concerned with minimizing Alpha, therefore a common percentage for a typical research study is 5% so we are 95% confident in the results.

Chapter 15

Significance Testing Error Issues

When do we do a means test?

When do we do a proportions test?

When do we do a chi-square (frequency distribution) test?

In making these decisions, we consider the level of data. We cannot calculate a mean on nominal or ordinal data, therefore we must do a proportion of frequency distribution test. If we have interval or ratio level data, we typically conduct a means test. If we have nominal or ordinal data, and we have one group or two groups, we usually do a proportion’s z-test. If we have more than two groups, we usually do a chi-square test.

Chapter 15

Types of Hypothesis Tests

One Mean

Used to test whether a sample mean is significantly different from an expected or pre-determined mean.Z-Test - usually for samples of about 30 and above.

t-Test - usually for samples below 30.

Two MeansZ-TestTests the difference between means for two.

More than Two MeansANOVA (Analysis of Variance)Tests the difference between means for more than two groups.

Chapter 15

Types of Mean Hypothesis Tests

Chapter 15

Types of Proportion Hypothesis TestsOne Sample

Z-TestTest to determine whether the difference between proportions is greater than would be expected because of sampling error.

Two Proportions in Independent Samples

Z-TestTest to determine the proportional differences between two or more groups.

More than Two Groups

Chi-square χ2

Test to determine whether the difference between three or more groups is greater than would be expected.

p-valueThe exact probability of getting a computed test statistic that was largely due to chance. The smaller the p-value, the smaller the probability that the observed result occurred by chance. The smaller the p-value, the more likely the test is significant.

For example a p-value of .045 is equivalent to a statistically significant test at a 95.5% level of

confidence (4.5% alpha level).

Chapter 15

Hypothesis Testing Term

• Independent samples – two or more groups of respondents that are tested as though they may come from different populations (independent samples t-test).

• Related samples – two or more groups of respondents that originated from the sample population (paired samples t-test).

• Paired samples – questions are independent but respondents are the same.

Hypothesis Testing

15-12

Analyzing Relationships of

Sample Data

BivariateHypothesisBivariate

Hypothesis

NullHypothesis

NullHypothesis

. . . more than one group is involved.. . . more than one group is involved.

. . . there is no difference between

the group means.

• µ1 = µ2 or that µ1 - µ2 = 0

. . . there is no difference between

the group means.

• µ1 = µ2 or that µ1 - µ2 = 0

15-13

Bivariate Statistical Tests

Cross-tabulation – is useful for examining relationships and reporting

the findings for two variables. The purpose of cross-tabulation is to

determine if differences exist between subgroups of the total sample.

15-14

Chi-Square (X2)

Analysis

. . . test for significance between the frequency

distributions of two or more nominally scaled variables in a

cross-tabulation table to determine if there is any

association.

15-15

Assesses how closely the observed frequencies fit the pattern of the expected frequencies and is referred to as a ”goodness-of-fit” test.

Assesses how closely the observed frequencies fit the pattern of the expected frequencies and is referred to as a ”goodness-of-fit” test.

Used to analyze nominal data which cannot be analyzed with other types of statistical analysis, such as ANOVA or t-tests.

Used to analyze nominal data which cannot be analyzed with other types of statistical analysis, such as ANOVA or t-tests.

Results will be distorted if more than 20 percent of the cells have an expected count of less than 5.

Results will be distorted if more than 20 percent of the cells have an expected count of less than 5.

Chi-Square (X2) Analysis

15-16

Analyzing Data

Relationships• Requirements for ANOVA odependent variable can be either

interval or ratio scaled.oindependent variable is categorical.

• Null hypothesis for ANOVA – states there is no difference between the groups – the null hypothesis is . . . oµ1 = µ2 = µ3

15-17

ANOVAANOVA

Total variance – separated into

between-group and within-group variance.

Total variance – separated into

between-group and within-group variance.

F-test – used to statistically evaluate the differences between the

group means.

F-test – used to statistically evaluate the differences between the

group means.DeterminingStatistical

Significance

DeterminingStatistical

Significance

15-18

ANOVA – Testing Statistical

Significance

The larger the F ratio . . .

The larger the F ratio . . .

. . . the larger the difference in the variance between groups.

. . . the larger the difference in the variance between groups.

. . . the more likely the null hypothesis will be rejected.. . . the more likely the null hypothesis will be rejected.

Based on the F-distribution . . .

Based on the F-distribution . . .

. . . Examines the ratio of two components of total variance and is calculated as shown below . . .

. . . Examines the ratio of two components of total variance and is calculated as shown below . . .

F ratio = Variance between groups

Variance within groups

F ratio = Variance between groups

Variance within groups

. . . implies significant differences between the groups.

. . . implies significant differences between the groups.

15-19

Analyzing Relationships of

Sample DataANOVA – cannot identify which pairs of means are significantly different from each other.

Must perform follow-up tests to identify the means that are statistically different from each other. Including:

•Sheffé

•Tukey, Duncan and Dunn

15-20

Analyzing Data

Relationships

ANOVA(analysis of variance)

ANOVA(analysis of variance)

. . . determines if three or more means

are statistically different from each

other (single dependent variable)

. . . determines if three or more means

are statistically different from each

other (single dependent variable)

. . . same as ANOVA but multiple

dependent variables can be analyzed

together.

. . . same as ANOVA but multiple

dependent variables can be analyzed

together.

MANOVA(multivariate analysis

of variance)

MANOVA(multivariate analysis

of variance)

15-21

Perceptual Maps

. . . have a vertical and a horizontal axis that are labeled

with descriptive adjectives.

. . . have a vertical and a horizontal axis that are labeled

with descriptive adjectives.

To develop perceptual maps – can use rankings, mean ratings, and

multivariate methods.

To develop perceptual maps – can use rankings, mean ratings, and

multivariate methods.

15-22

15-23

Perceptual Perceptual

MappingMapping

DistributionDistribution

AdvertisingAdvertising

Image developmentImage development

New product developmentNew product development

Applicationsin Marketing

Research

Applicationsin Marketing

Research

15-24