Choosing Research Designs I

Experimental and Quasi-Experimental

Experimental Designs•Experimental research design: The researcher has control over the experiment in terms of sample selection, treatment, environment, etc.

•Experimental designs are typical in psychology, medicine, education, etc.

Experimental DesignsExperiments often discuss pre and post test observations

• POST-TEST ONLY

X O 1

Where:

0t = Observation in time t of experimental group X = Treatment 0c = Control group

Experimental Designs• PRE-TEST POST-TEST

O1 X O2

Note how the above test includes no control group. What is

a control group and why include it?

• PRE-TEST POST-TEST WITH CONTROL GROUP

O1 X O2

O1 O2

Key question of Internal ValidityKey question of Internal Validity

• When we test a hypothesis and either accept it or reject it, how do we know that we made the right decision?

• What about alternative explanations that we did not account for?

• What should we do to gain confidence?

Internal ValidityInternal Validity• Are there other causes for what I am

observing?

• If so, a study will lack internal validity if it cannot rule out plausible alternative explanations.

Internal Validity of a StudyInternal Validity of a StudyInternal Validity of a StudyInternal Validity of a Study

• What you measured and what you saw?

• Your program and your observations?

ObservationObservationObservationObservation

Research ProjectResearch Project ObservationsObservations

What you What you dodo What you What you seesee

Is the relationship Is the relationship causalcausal between... between...

AlternativeAlternativecausecause AlternativeAlternative

causecause

AlternativeAlternativecausecause

AlternativeAlternativecausecause

In In thisthis study study

Program-outcome RelationshipProgram-outcome Relationship

•DID YOU FIND THAT BORING?

THEN HOW ABOUT A JOLT OF CUTENESS!

Experimental Designs• What do you do if you fear that your

simple pre and post test single observations are atypical?

• CONTROL WITH MORE OBSERVATION IN THE PRETEST

O1 O2 O3 X O4

O1 O2 O3 O4

Experimental Designs• Of course, we could also experiment

to see if removing a factor has an effect instead of adding a treatment factor

• PRE-TEST POST-TEST REMOVING THE TREATMENT

_O1 X O2 X O3 X 04

Experimental DesignsResearch design with more power to control

for time

• Pre-test post-test

O1 O2 O3 X O4 O5 O6

• Pre-test post-test with control group

O1 O2 O3 X O4 O5 O6

O1 O2 O3 O4 O5 O6

Did you bring your lab coat?• Considering true experiments are

close to impossible or unethical in political science, we need to apply science differently.

• Hence, this is why we in political science often use Quasi-experimental or non-experimental research designs

Quasi-Experimental• Quasi-experiments: The researcher

does not have control over the experiment, rather the experiment occurs in a “natural” setting.

• Quasi-experimental design are typical in economics, sociology, public administration, urban planning, political sciences, etc.

Quasi-Experimental• A prominent example of this type of research is arguably by Samuel Huntington, who argues that Latin American immigration, especially Mexican, is altering US culture for the worse.

• He essentially predicts that the United States will be a divided, less productive, society, ruining the past culture that has made the United States great.

http://www.foreignpolicy.com/story/cms.php?story_id=2495

Quasi-ExperimentalHuntington points out the following X variables:

• Proximity of Mexico, Latin America to USA• Former ties to Mexico• Magnitude of recent wave of immigration• Duration of recent wave of immigration• Illegality

He does this by observing what is going on today. It is crucial that one understands his assumptions and evaluates what he claims as evidence for his theory