Chapter 9Causal Research Designs

and Test Markets

Chapter 9Causal Research Designs

and Test Markets

Definitions – two broad types: Laboratory experiments – take place in

an artificial setting where extraneous variables can be controlled.

Field experiments – take place in a natural setting but extraneous variables cannot be controlled.

Types of Experiments

9-2

o Realismo Controlo Time frameo Cost o Competitive reactions

o Realismo Controlo Time frameo Cost o Competitive reactions

Field experiments – issues to consider

9-3

Variables manipulated by researcher

Variables manipulated by researcher

Measures of effects/outcomesMeasures of effects/outcomes

Independent VariablesIndependent Variables

Dependent VariablesDependent Variables

Control VariablesControl Variables Conditions that we measure to better understand the true effects

Conditions that we measure to better understand the true effects

Extraneous VariablesExtraneous Variables Uncontrolled, unmeasured variables that may affect the dependent variable

Uncontrolled, unmeasured variables that may affect the dependent variable

9-4

ExternalExternal ValidityValidity

ExternalExternal ValidityValidity

Internal Internal ValidityValidityInternal Internal ValidityValidity ReliabilityReliabilityReliabilityReliability

9-5

External validity – the extent to which a causal relationship is true for the defined target population.

External validity – the extent to which a causal relationship is true for the defined target population.

Internal validity – the extent to which the

research design accurately identifies causal relationships.

Internal validity – the extent to which the

research design accurately identifies causal relationships.

Validity – the extent to which the

conclusions drawn from the experiment

are true.

Validity – the extent to which the

conclusions drawn from the experiment

are true.

9-6

History History

Maturation Maturation

Testing Testing

Instrumentation Instrumentation

Selection Bias Selection Bias

Statistical Regression Statistical Regression

Mortality Mortality

9-7

Treatment vs. Treatment Treatment vs. Treatment

Treatment vs. Testing Treatment vs. Testing

Treatment vs. Selection Treatment vs. Selection

Treatment vs. Setting Treatment vs. Setting

Treatment vs. History Treatment vs. History

9-8

Construct validity . . . the extent to which the variables under

investigation are completely and accurately identified prior to hypothesizing any functional

relationships. (Are we measuring what we think we are measuring?)

9-9

Inadequate measurement of the variables Inadequate measurement of the variables

One measurement, one methodology bias One measurement, one methodology bias

Evaluation apprehension – respondent concern of other knowing their responses

Evaluation apprehension – respondent concern of other knowing their responses

Demand characteristics – respondents trying to guess the desired response

Demand characteristics – respondents trying to guess the desired response

Diffusion of treatment – respondents causing bias by discussing the treatment with others.

Diffusion of treatment – respondents causing bias by discussing the treatment with others.

9-10

Time order of the manipulation exposure. Time order of the manipulation exposure.

Including control groups Including control groups

Exclusion of non-similar test subjects. Exclusion of non-similar test subjects.

Matching extraneous variables. Matching extraneous variables.

Randomization of test subjects to treatment groups. Randomization of test subjects to treatment groups.

9-11

X = exposure of an independent variable to a group of test subjects for which the effects are to be determined.

O = observation or measurement of the dependent variable (outcome) for the test subjects.

[R] = random assignment of test subjects to separate groups.

EG = the experimental group of test subjects.

CG = the control group of test subjects.

→= movement through time shown as left-to-right movement.

Vertical alignment of symbols = activities that occur simultaneously.

Horizontal alignment of symbols = refer to a specific group of subjects.

Symbols Used in Experimental Research Designs

9-12

One-Shot Study(EG): X → O1

One-Group, Pretest-Posttest(EG): O1 → X1 → O2

Static Group Comparison(EG): X → O1

(CG): O2

Pre-Experimental Designs

9-13

Posttest-Only, Control Group (Good)(EG): [R] X → O1 (CG): [R] O2

Pretest-Posttest, Control Group (Better) (EG): [R] O1 → X → O2 (CG): [R] O3 → O4

Solomon Four Group (Best)Design 1(EG): [R] O1 → X → O2(CG): [R] O3 → O4Design 2(EG): [R] X → O5(CG): [R] O6

True Experimental Designs

9-14

Non-Equivalent Control GroupGroup 1 O1 → X → O2

Group 2 O3 O4

Separate Sample, Pretest-PosttestSample 1 O1 → (X)

Sample 2 (X) → O2

Quasi-Experimental Designs

9-15

. . . use of a controlled field experiment to collect information on market performance indicators.

•Pros:

o Is most accurate method of forecasting future sales.

oProvides an opportunity to pretest marketing mix variables (particular PRICE).

•Cons:

oExpensive.

oExposes the new product or service to competitors.

oTakes time to conduct.

  

Test Marketing

9-16

9-17

•Representativeness: Do demographics match the total market?

•Degree of isolation: Phoenix and Tulsa are isolated markets – thus better; Los Angeles is not isolated.

•Ability to control distribution and promotion: Are there preexisting arrangements to distribute the new product in selected channels of distribution? Are local media prepared to test variations in promotional messages?

Criteria for Selecting Test Market Locations

9-18