l1 chapter 11 experimental and quasi- experimental designs dr. bill bauer
TRANSCRIPT
l 2
Key Ideas
• History of experimental resign• Key characteristics of experimental
design• Types of experimental design• Steps in conducting experimental
research• Criteria for evaluating experimental
research
l 3
A Brief History of Experimental Designs
• Schuyler 1903: control groups• McCall 1916: randomly selected
groups• McCall 1925: book on experiments• Fisher 1936: statistical methods
book
l 4
A Brief History of Experimental Designs• Campbell and Stanley 1963: 15 types
of experimental designs evaluated in terms of threats to validity
• Cook and Campbell 1979: four types of validity
• 1980 and beyond: computer enhancements to experimental design2
l 5
Key Characteristics of Experimental Designs
• Participants selected and assigned to groups• control• experimental
• An intervention is applied to one or more groups
• Outcomes are measured at the end of the experiment
l 6
Key Characteristics of Experimental Designs• Procedures are designed that
address potential threats to validity• Internal• External• Construct• Statistical Conclusion
• Statistical comparisons of different groups are conducted
l 7
Selecting Participants and Assigning Them to Treatments• Decide on the experimental unit of
analysis to be treated• individual• group or groups• organization
• Randomly assign individuals to groups control for extraneous characteristics that might influence the outcome
l 8
Selecting Participants and Assigning Them to Treatments• Control for extraneous factors
• random assignment (equating groups)
• pretest/posttest• covariates• matching participants• selecting homogenous samples• using blocking variables
l 9
IndependentVariable
IndependentVariable: Typeof Instruction
DependentVariable:Rates of Smoking
Controlling for CovariatesNo Covariates
DependentVariable
Covariate Introduced
Covariate:Parents Who
SmokeVarianceRemoved
Variance
l 10
Matching Process Based on Gender
ExperimentalGroup
ControlGroup
JohnJimJamesJoshJacksonJaneJohannaJulieJeanJeb
l 11
Applying an Intervention or Treatment
• Identify a treatment variable• identify the conditions or levels of
the variable• Manipulate the treatment
conditions
l 12
The Experimental Manipulation of a Treatment Group
Independent Variables1. Age (can’t manipulate)2. Gender (can’t manipulate)3. Types of Instruction (can
manipulate)a. Lecture (control)b. Lecture + Hazard Instruction (Comparison)c. Lecture + Hazard Instruction + slides of damaged lungs (experiment)
Dependent Variable
Frequency of Smoking
l 13
Threats to Statistical Conclusion Validity
• Low statistical power due to low sample size
• Violation of assumptions of statistical tests
• Use of unreliable measures
l 14
Threats to Internal Validity
• History• Maturation• Regression• Selection• Mortality• Interactions with selection
l 15
Threats to Internal Validity
• Diffusion of treatments• Compensatory equalization• Compensation rivalry• Resentful demoralization• Testing• Instrumentation
l 16
Threats to Construct Validity
• Lack of good operational definitions
• Apprehensiveness by participants• Participants “guessing” what the
researcher hopes to find
l 17
Threats to External Validity
• Interaction of selection and treatment
• Interaction of setting and treatment
• interaction of history and treatment
l 18
Treatment Comparisons in an ExperimentPhase 1: Relationship Picture
Error Correction Treatment+
Spelling Accuracy
Phase 2: Timeline Picture Test 1 Test 2 Test 36 Weeks 6 Weeks 6 Weeks6 Weeks 6 Weeks 6 Weeks
Class A: Regular Spelling Practice (Control)Class B: Reduced word list (Comparison)
Phase 3: Statistical ComparisonsClass A Class B Class C F-value
Test 1
Test 2
Test 3
10.3(3.6)
10.7 (3.3)
11.1(3.3)
10.8(4.3)
10.6(3.8)
10.3(3.6)
9.9(3.9)
13.9(4.2)
13.1(3.8)
0.27
4.90*
3.31*
* p<.05
Error Correction Treatment 6 Weeks 6 Weeks 6 Weeks
Class C: Error Correction (Experimental)
l 19
Types of Experimental Designs: Between Groups
• True Experiments• Pre- and posttest• Posttest Only
• Quasi Experiments• Pre- and posttest• Posttest Only
• Factorial Designs
l 20
Types of Experimental Designs: Within-Group or Individual • Time series experiments
• interrupted• uninterrupted
• Repeated measures experiments• Single-subject experiments
• A/B design• Multiple baseline design• Alternating treatments
l 21
Means and Main Effects of Eight groups in Factorial Design Depression
Mean rateof
smoking
Low
Mean rateof
smoking
Mean rateof
smoking
Mean rateof
smoking
Mean rateof
smoking
Mean rateof
smoking
Health lecture
Type of Instruction
Standard lecture
Main Effectsof Type ofInstruction
Main Effects ofDepression
Medium High
l 22
Graphs Showing Main and Interaction Effects
High
LowLow Medium High
High
LowLow Medium High
High
LowLow Medium High
(a) No interaction Effects (Parallel)
Extent of Smoking
Extent of Smoking
Extent of Smoking
(b) Interaction Effects (Crossed)
(c) Interaction Effects (Not Parallel)
(a) Interaction Effects (Crossed)
Standard lecture
Health lecture
Standard lecture
Health lecture
Standard lecture
Health lecture
l 23
Steps in Conducting Experimental Research
• Decide if an experiment addresses the research problem
• Form hypotheses to test cause-effect relationships
• Select an experimental treatment and introduce it
• Identify study participants
l 24
Steps in Conducting Experimental Research
• Choose a type of experimental design
• Conduct the experiment• Organize and analyze the data• Develop an experimental research
report
l 25
Criteria for Evaluating Experimental Research• Does the experiment have a powerful
intervention?• Does it employ few treatment groups
(e.g. only two)?• Will participant profit from the
intervention?• Is there a systematic way the
researcher derived the number of participants per group?
l 26
Criteria for Evaluating Experimental Research• Were there an adequate number of
participants used in the study?• Were valid, reliable, and sensitive
measures or observations used?• Did the study control for extraneous
factors?• Did the researcher control for
threats to internal validity?
l 27
Applying What you Have Learned: An Experimental StudyReview the article and look for the following:
• The research problem and use of quantitative research
• Use of the literature• The purpose statement and research hypothesis• Types and procedures of data collection• Types and procedures of data analysis and
interpretation• The overall report structure