Making Causal Inferences and Ruling out Rival Explanations

29 February

Questions?

• How do we know that X is causing Y?• Did X have any effect on Y?

– If X had not happened would Y have changed anyway?

Hypothesized relationship:

%Women elected in National Parliaments

Party rules gender quotas

Questions?

• How do we know that party quotas causing changes in %women elected?

• Standard Design– Party adopts quotas % women elected

X O Where X = treatment and O = observation

Establishing Causation:

• Co-variation• Time – (x occurs before y)• Consistent with other evidence• Rule out rival explanations

– Example – spurious relationship

Spurious Relationship

a relationship in which two variables that are not causally linked appear to be so because a third variables in influencing both of them

Spurious Relationship

Fire damage in $# of fire engines responding to call

+

Intensity of fire

+ +

(the third variable problem)

Alternative explanations:

%Women elected in National Parliaments

Electoral System

Political Culture

Women’s Labor Force Participation

Access to educational opportunities

Women’s Political Resources

% of women candidates standing for election

Party rules - quotas

Spurious Relationship

%women electedParty quotas +

Political culture

+ +

When choosing a research design?

• When and how to make observations:• Internal Validity

– Ability to establish causality

• External Validity– Ability to generalize

Types of Designs:

• Experimental designs• Control groups

• Quasi-experimental• Non-experimental designs

• Statistical controls

Experiments come in a wide variety of apparent types but all share three basic characteristics:

• Random assignment

• Manipulation of an independent variable

• Control over other potential sources of systematic variance X O1

R

O2

These basic characteristics effectively solve the two basic problems in nonexperimental (correlational) research:

•The directionality problem

•The third variable problem

Random Assignment

Random assignment means that assignment to experimental conditions is determined by chance.

Participants have a equal probabilities of being assigned to a treatment or control group.

This insures that any pre-existing characteristics that participants bring with them to the study are distributed equally among the experimental groups . . . in the long run.

Treatment group = (equivalent to) Control group

Think about example of party quotas a % women elected:

Randomly assign countries to two groups: treatment and control

Theoretically should end up with two groups that have equivalent distributions on all other “third variables” (i.e. culture, % women in labour force, etc.)

Have one group adopt quotas

Observe % women elected, treatment group expected to have higher average for % women elected.

Problems?

• Random assignment might be difficult in this case.

• Turn to quasi-experiments when randomization not possible

To Review - One Group Post-Test Only Design

X O

The simplest and the weakest possible design:

(a) Lack of a pretest prevents assessment of change

(b) Lack of a control group prevents threats from being ruled out.

Threats to Internal Validity

• Selection Threats • Maturation• History• Testing• Instrumentation• Regression• Note: Experimental designs

control for these

Party adopts quotas % women elected

X O

One Group Post-Test Only Design

X O

Without changing the basic nature of this design, it can be improved considerably by adding additional outcome measures:

O1

X O2

O3

Compared to norms or expectations, only O2 should be unusual.

Post-Test Only Design with Nonequivalent Groups

X O

O

Threats:

Selection

One-Group Pretest Post-Test Design

O X O

This very common applied design is susceptible to all threats to within-groups comparisons:

• History

• Maturation

• Testing

• Regression

• Instrumentation

One-Group Pretest Post-Test Design

O X O

One powerful modification is to add pretests:

O O O O O X O

Maturation threats can now be examined and their influence separated from treatment effects.

O O O O O O O O O X O

Untreated Control Group Design with Pretest and Posttest

O1 X O2

O1 O2

Can compare change within groups and across groups

Expect change in treatment group to be greater

Selection still a threat

Conclusions:

• Experiments best for internal validity• May not be good on external validity• In non-experimental designs, use statistical

controls (hold constant all possible “third” variables.