Experimental Research

Chapter 8

Introduction

Explanatory research Research that seeks to explain the cause of a

phenomenon, and typically asks “what causes what?” or “why is it this way?”

Introduction

Causal hypothesis A testable expectation about an independent

variable’s affect on a dependent variable

Introduction

Causal hypotheses and experimental designs1. Empirical association 2. Temporal precedence or time order 3. Non-causal relationship

Introduction

Causal hypotheses and experimental designs Experimental designs

A study design in which the independent variable is controlled, manipulated, or introduced in some way by the researcher

Introduction

The classic experiment: Data collection technique or study design? In experimental design the independent

variable is introduced, manipulated, or controlled

What does this mean? The independent variable does not occur naturally,

but it is the result of an action taken by the researcher

Unique feature of the classic experimental design The researcher controls the placement of sample

members into two or more categories of the independent variable

Introduction

The classic experiment: Data collection technique or study design? If practical and ethical, a study can be

designed so that the dependent variable is measured first and then, the independent variable is introduced or manipulated and, finally, the dependent variable is measured again.

Can see whether the introduction of the independent variable comes before change in the dependent variable.

Experimental Designs

Control group Exposed to all the influences that the

experimental group is exposed to except for the stimulus

Experimental Designs

Stimulus The experimental condition of the independent variable

that is controlled or “introduced” by the researcher in an experiment

The researcher tries to treat the two groups exactly alike, except instead of the stimulus, the control group receives no treatments, an alternative treatment, or a placebo

Experimental Designs

Placebo A simulated treatment of the control group that

is designed to appear authentic

Experimental Designs

Internal validity Agreement between a study’s conclusions

about causal connections and what is actually true

Experimental Designs

Experimental designpretest-posttest control group experiment Classic controlled experiment

An experimental design with two or more randomly selected groups (an experimental and control group) in which the researcher controls or “introduces” the independent variable and measures the dependent variable at least two times (pretest and posttest measurement)

Experimental Designs

Experimental designpretest-posttest control group experiment Pretest

The measurement of the dependent variable that occurs before the introduction of the stimulus of independent variable

Experimental Designs

Experimental designpretest-posttest control group experiment Posttest

The measurement of the dependent variable that occurs after the introduction of the stimulus or the independent variable

Experimental Designs

Experimental designpretest-posttest control group experiment Probability sampling

A sample that gives every member of the population a known (nonzero) chance of inclusion

Experimental Designs

• Experimental designpretest-posttest control group experiment

• Random Assignment A technique for assigning members of the sample to

experimental and control groups by chance to maximize the likelihood that the groups are similar at the beginning of the experiment This can be done by flipping a coin to determine which

subject is assigned to which group Assign each subject a number and using either a random

number table or electronic random number generator to select members of each group

Experimental Designs

• Experimental designpretest-posttest control group experiment

• Matching Assigning members of the sample to groups by

matching members of the sample on one or more characteristics and separating the pairs into two groups with one group randomly selected to become the experimental group

Experimental Designs

Experimental designpretest-posttest control group experiment

1. The study uses at least one experimental and one control group, selected using a strategy to make the groups as similar as possible

2. The dependent variable is measured at least two time for the experimental and control groups. The first measurement is before and the second is after the independent variable is introduced

Experimental Designs

Experimental designpretest-posttest control group experiment

3. The independent variable is introduced, manipulated, or controlled by the researcher between the two measurements of the dependent variable. 4. The differences in the dependent variable between the pretest and posttest are calculated for the experimental group(s) and for the control group. The differences in the dependent variable for the experimental and control groups are compared.

Experimental Designs

Experimental Designs

Internal validity and experiments Maturation

The biological and psychological processes that cause people to change over time

Experimental Designs

Internal validity and experiments Testing effect

The sensitizing effect on subjects of the pretest

Experimental Designs

Internal validity and experimentsHistory

The effects of general historical events on study participants

Experimental Designs

Internal validity and experimentsSelection bias

A bias in the way the experimental and control or comparison groups are selected that is responsible for preexisting differences between the groups.

Experimental Designs

Posttest-only control group experiment An experimental design with no pretest

Either because it is not possible to do a pretest or because of a concern that using a pretest would sensitize the experimental group to the stimulus

Same design elements as other experiments: control of manipulation of the stimulus and two or more groups using random selection or assignment

Experimental Designs

Extended experimental design Solomon four-group design

A controlled experiment with an additional experimental and control group with each receiving a posttest only

Experimental Designs

Experimental Designs

Quasi-experimental design An experimental design that is missing one or

more aspects of a true experiment Most frequently random assignment into

experimental and control groups Used when it is not ethical or practical to do a

controlled experiment

Experimental Designs

Experiments in the field Field experiment

An experiment done in the “real world” of classrooms, offices, factories, homes, playgrounds, and the like

Experimental Designs

Experiments in the field Generalizability

The ability to apply the results of a study to groups or situations beyond those actually studied

Focal Research

A Field Experiment in the Classroom by Chris Caldeira Hypothesized that using film clips to teach

sociological concepts in conjunction with the textbook and lecture is more effective in helping students learn core concepts than the lecture and textbook without the clips.

Experimental Designs

Experiments in the laboratory Laboratory research

Research done in settings that allows the researcher control over the conditions, such as in a university or medical setting

Research examples Hobza and Aaron Rochlen (2009) Bushman and Anderson (2009)

Experimental Designs

Experiments in the laboratory Important to evaluate a study’s external

validity – or the ability to generalize the results from the laboratory to the “real world”

Experimental Designs

Experiments in the laboratory External Validity – Issues to Consider

1. Was the situation very artificial, or did it approximate “real life?”

2. How different were study participants from other populations?

3. To what extent did the participants believe that they were up for inspection, serving as guinea pigs or play acting, or have other feelings that would affect responses to the stimulus?

4. To what extent did the researcher communicate his or her expectations for results to the subjects with verbal or nonverbal cues?

Experimental Designs

Experiments in the laboratory Experimenter expectations

When expected behaviors or outcomes are communicated to subjects by the researcher

One way to handle the issue of experimenter expectations is to use a double-blind experiment.

Experimental Designs

Experiments in the laboratory Double-blind experiment

An experiment in which neither the subjects nor research staff who interact with them knows the memberships of the experimental or control groups.

Experimental Designs

Natural experiments A study using a real-work phenomena that

approximates an experimental design even though the independent variable is not controlled, manipulated, or introduced by the researcher

Experimental Designs

Natural experiments Wars, hurricanes, & political events might

effect some groups and not others. Example

Studying the of deployment on military families The researcher cannot control deployment, but two

groups could be created by those families who were deployed and those families who were not deployed

Comparing Experiments to Other Designs Many research questions cannot be studied

using experimental design Large samples are required Not practical, ethical, and possible to

manipulate the independent variable In these scenarios researchers may consider

panel, trend, cross-sectional, or case study designs

Summary

Study design options Refer to Table 8.2 - Summary of Design

Studies Issues to consider in designing a study

Ethical considerations Practicality

Time Cost Feasibility of controlling the independent variable

Quiz – Question 1

An experimental design has the advantage over non-experimental approaches of determining the order in which events occur and

a. controlling the effects of other variables.

b. minimizing the costs of the research.

c. accommodating ethical concerns.

d. proving conclusively that a relationship is causal.

Quiz – Question 2

In a classic experimental design, it is important to take measures of control and experimental groups

a. before and after treatment.

b. at least once during the research.

c. after the treatment.

d. before the treatment.

e. it is not important to take measures.

Quiz – Question 3

In experimental designs, it is important that the control group and the experimental group

a. are as similar as possible.

b. represent very different groups so that the research is generalizable.

c. do not come from the same background.

d. receive the same treatment.