mcgraw-hill © 2006 the mcgraw-hill companies, inc. all rights reserved. experimental research...

McGraw-Hill © 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Experimental ResearchExperimental Research

Chapter ThirteenChapter Thirteen


Experimental ResearchExperimental ResearchChapter ThirteenChapter Thirteen


Uniqueness of Experimental Uniqueness of Experimental ResearchResearch

Experimental Research is unique in two Experimental Research is unique in two important respects:important respects:

1)1) Only type of research that attempts to Only type of research that attempts to influence a particular variableinfluence a particular variable

2)2) Best type of research for testing hypotheses Best type of research for testing hypotheses about cause-and-effect relationshipsabout cause-and-effect relationships

Experimental Research looks at the Experimental Research looks at the following variables:following variables:

Independent variable (treatment)Independent variable (treatment) Dependent variable (outcome)Dependent variable (outcome)


Characteristics of Experimental Characteristics of Experimental ResearchResearch

The researcher manipulates the independent The researcher manipulates the independent variable.variable.

They decide the nature and the extent of the They decide the nature and the extent of the treatment. treatment.

After the treatment has been administered, After the treatment has been administered, researchers observe or measure the groups researchers observe or measure the groups receiving the treatments to see if they differ.receiving the treatments to see if they differ.

Experimental research enables researchers to go Experimental research enables researchers to go beyond description and prediction, and attempt to beyond description and prediction, and attempt to determine what caused effects.determine what caused effects.


RandomizationRandomization

Random assignmentRandom assignment is similar but not identical to is similar but not identical to random selectionrandom selection..

Random assignment means that every individual Random assignment means that every individual who is participating in the experiment has an who is participating in the experiment has an equal chance of being assigned to any of the equal chance of being assigned to any of the experimental or control groups.experimental or control groups.

Random selection means that every member of a Random selection means that every member of a population has an equal chance of being selected population has an equal chance of being selected to be a member of the sample.to be a member of the sample.

Three things occur with random assignments of Three things occur with random assignments of subjects:subjects:

1)1) It takes place before the experiment beginsIt takes place before the experiment begins2)2) Process of assigning the groups takes placeProcess of assigning the groups takes place3)3) Groups should be equivalent Groups should be equivalent


Control of Extraneous VariablesControl of Extraneous Variables

The researcher has the ability to control The researcher has the ability to control many aspects of an experiment.many aspects of an experiment.

It is the responsibility of the researcher to It is the responsibility of the researcher to control for possible threats to internal control for possible threats to internal validity.validity.

This is done by ensuring that all subject This is done by ensuring that all subject characteristics that might affect the study characteristics that might affect the study are controlled.are controlled.


How to Eliminate Threats due to How to Eliminate Threats due to Subject Characteristics?Subject Characteristics?

RandomizationRandomization Hold certain variables constantHold certain variables constant Build the variable into the designBuild the variable into the design MatchingMatching Use subjects as their own controlUse subjects as their own control Analysis of Covariance (ANCOVA)Analysis of Covariance (ANCOVA)


Weak Experimental DesignsWeak Experimental Designs

The following designs are considered weak since The following designs are considered weak since they do not have built-in controls for threats to they do not have built-in controls for threats to internal validityinternal validity The One-Shot Case StudyThe One-Shot Case Study

A single group is exposed to a treatment and its effects are A single group is exposed to a treatment and its effects are assessedassessed

The One-Group-Pretest-Posttest DesignThe One-Group-Pretest-Posttest Design Single group is measured both before and after a treatment Single group is measured both before and after a treatment

exposureexposure The Static-Group Comparison DesignThe Static-Group Comparison Design

Two intact groups receive two different treatmentsTwo intact groups receive two different treatments


Example of a One-Shot Case Study DesignExample of a One-Shot Case Study Design (Figure 13.1)(Figure 13.1)


Example of a One-Group Pretest-Posttest Example of a One-Group Pretest-Posttest DesignDesign (Figure 13.2)(Figure 13.2)


Example of a Static-Group Comparison Example of a Static-Group Comparison DesignDesign (Figure 13.3)(Figure 13.3)


True Experimental DesignsTrue Experimental Designs The essential ingredient of a true experiment is The essential ingredient of a true experiment is

random assignment of subjects to treatment random assignment of subjects to treatment groupsgroups

Random assignments is a powerful tool for Random assignments is a powerful tool for controlling threats to internal validitycontrolling threats to internal validity The Randomized Posttest-only Control Group DesignThe Randomized Posttest-only Control Group Design

Both groups receiving different treatmentsBoth groups receiving different treatments The Randomized Pretest-Posttest Control Group The Randomized Pretest-Posttest Control Group

DesignDesign Pretest is included in this designPretest is included in this design

The Randomized Solomon Four-Group DesignThe Randomized Solomon Four-Group Design Four groups used, with two pre-tested and two not pre-Four groups used, with two pre-tested and two not pre-

testedtested


Example of a Randomized Posttest-Only Example of a Randomized Posttest-Only Control Group Design Control Group Design (Figure 13.4)(Figure 13.4)


Example of a Randomized Pretest-Example of a Randomized Pretest-Posttest Control Group Design Posttest Control Group Design

(Figure 13.5)(Figure 13.5)


Example of a Randomized Solomon Four-Group Design Example of a Randomized Solomon Four-Group Design (Figure 13.6)(Figure 13.6)


Random Assignment with MatchingRandom Assignment with Matching

To increase the likelihood that groups of To increase the likelihood that groups of subjects will be equivalent, pairs of subjects will be equivalent, pairs of subjects may be matched on certain subjects may be matched on certain variables.variables.

Members of matched groups are then Members of matched groups are then assigned to experimental or control assigned to experimental or control groups.groups.

Matching can be mechanical or statistical.Matching can be mechanical or statistical.


A Randomized Posttest-Only Control Group A Randomized Posttest-Only Control Group Design, Using Matched Subjects Design, Using Matched Subjects (Figure 13.7)(Figure 13.7)


Mechanical and Statistical MatchingMechanical and Statistical Matching

Mechanical matchingMechanical matching is a process of is a process of pairing two persons whose scores on a pairing two persons whose scores on a particular variable are similar.particular variable are similar.

Statistical matchingStatistical matching does not necessitate a does not necessitate a loss of subjects, nor does it limit the loss of subjects, nor does it limit the number of matching variables. number of matching variables. Each subject is given a “predicted” score on Each subject is given a “predicted” score on

the dependent variable, based on the the dependent variable, based on the correlation between the dependent variable correlation between the dependent variable and the variable on which the subjects are and the variable on which the subjects are being matched.being matched.

The difference between the predicted and The difference between the predicted and actual scores for each individual is then used to actual scores for each individual is then used to compare experimental and control groups.compare experimental and control groups.


Quasi-Experimental DesignsQuasi-Experimental Designs Quasi-Experimental Designs do not Quasi-Experimental Designs do not

include the use of random include the use of random assignments but use other techniques assignments but use other techniques to control for threats to internal to control for threats to internal validity:validity: The Matching-Only DesignThe Matching-Only Design

Similar except that no random assignment occursSimilar except that no random assignment occurs Counterbalanced DesignCounterbalanced Design

All groups are exposed to all treatments but in a All groups are exposed to all treatments but in a different orderdifferent order

Time-Series DesignTime-Series Design Involves repeated measures over time, both before Involves repeated measures over time, both before

and after treatmentand after treatment


Results (Means) from a Study Using a Results (Means) from a Study Using a Counterbalanced DesignCounterbalanced Design



Possible Outcome Patterns in a Time-Possible Outcome Patterns in a Time-Series DesignSeries Design (Figure 13.9)(Figure 13.9)


Factorial DesignsFactorial Designs Factorial DesignsFactorial Designs extend the number of extend the number of

relationships that may be examined in an relationships that may be examined in an experimental study.experimental study.

They are modifications of either the posttest-only They are modifications of either the posttest-only control group or pretest-posttest control group control group or pretest-posttest control group designs which permit the investigation of designs which permit the investigation of additional independent variables.additional independent variables.

They also allow a researcher to study the They also allow a researcher to study the interaction of an independent variable with one or interaction of an independent variable with one or more other variables (more other variables (moderator variablemoderator variable).).


Using a Factorial Design to Study Effects of Using a Factorial Design to Study Effects of Method and Class Size on AchievementMethod and Class Size on Achievement



Illustration of Interaction and No Interaction in a Illustration of Interaction and No Interaction in a 2 by 2 Factorial Design 2 by 2 Factorial Design (Figure 13.11)(Figure 13.11)


Example of a 4 by 2 Factorial Design Example of a 4 by 2 Factorial Design (Figure 13.13)(Figure 13.13)


Effectiveness of Experimental Designs in Controlling Threats to Internal Validity Effectiveness of Experimental Designs in Controlling Threats to Internal Validity (Table 13.1)(Table 13.1)

Subject Instru- Data Collec-Charac- Morta- Loca- ment tor Charac- Data Col- Matur- Atti- Regres- Implemen-

Design teristics lity tion Decay teristics lector Bias Testing History ation tudinal sion tation

One-shot casestudy – – – (NA) – – (NA) – – – – –

One group pre-posttest – ? – – – – – – – – – –

Static groupcomparison – – – + – – + ? + – – –

Randomized post-test-only controlgroup ++ + – + – – ++ + ++ – ++ –

Randomized pre-post-test controlgroup ++ + – + – – + + ++ – ++ –

Solomon four-group ++ ++ – + – – ++ + ++ – ++ –

Randomizedposttest onlycontrol group with matchedsubjects ++ + – + – – ++ + ++ – ++ –

Matching-onlypre-posttestcontrol group + + – + – – + + + – + –

Counterbalanced ++ ++ – + – – – ++ ++ ++ ++ –

Time-series ++ – + _ – – – – + – ++ –

Factorial withrandomization ++ ++ – ++ – – + + ++ – ++ –

Factorial without randomization ? ? – ++ – – + + + – ? –

KEY: (++) = strong control, threat unlikely to occur; (+) = some control, threat may possibly occur; (–) = weak control, threat likely to occur; (?) = can’t determine; (NA) = threat does not apply


Controlling Threats to Internal Validity Controlling Threats to Internal Validity (Table 13.1)(Table 13.1)

Subject CharacteristicsSubject Characteristics MortalityMortality LocationLocation Instrument decayInstrument decay Data Collector Data Collector

CharacteristicsCharacteristics Data Collector biasData Collector bias

TestingTesting HistoryHistory MaturationMaturation AttitudinalAttitudinal Regression Regression ImplementationImplementation

The above must be controlled to reduce threats to internal validity

mcgraw-hill © 2006 the mcgraw-hill companies, inc. all rights reserved. experimental research...

Documents