EXPERIMENTS: PART 1

Overview

Experimental versus observational research

Variables Designs

Between-group Within-subject

Similarities and differences Mixed-model

Background on Experiments

Study where a researcher systematically manipulates one variable in order to examine its effect(s) on one or more other variables

Two components Includes two or more conditions Participants are randomly assigned by the researcher

Random = Equal odds of being in any particular condition Examples

People with GAD randomly assigned to three treatments so the researchers can examine which one best reduces anxiety

Students assigned to a “mortality salience” or control condition so the research can examine the impact on “war support”

Variables

Independent Variable Manipulated by the researcher Typically categorical Also called a “factor” that has “levels”

Factor = Type of anxiety treatment Level = CBT (or Psychodynamic or Control)

Dependent Variable Outcome variable that is presumably influenced

by (depends on the effects of) the independent variable

Behavior frequencies, mood, attitudes, symptoms Typically continuous

Variables

Confounds (extraneous variables, 3rd variables) Happens when unwanted differences (age,

gender, researchers, environments, etc.) across experimental conditions

Plan: Think of potential confounds up front Control for them methodologically Measure them to examine whether they have

an effect Control for them statistically

Experimental Designs

Three main designs Between-group design

Also called a “between-subjects design,” or “randomized controlled trial” (if clinically focused)

Within-subject design Also called a “repeated-measures design”

Mixed-model design Combines both of the above

Between-group Design

IV: 2 or more randomly-assigned groups of people

DV: Usually a continuous variable

Within-subject Design

Any time that a study assess participants on the DV on more than one occasion

Example: Participants go through more than one experimental condition

■

Control Pill Control Pill

Similarities

Uses the same type of analyses p-values obtained from t-tests (if two

conditions) or F-tests/ANOVA (if more than two conditions) Is the result statistically significant, reliable,

trustworthy? Cohen’s d used to compute effect size

Tells the number of standard deviations by which two groups differ (kind of like r but on a

scale from -∞ to ∞)

Effect r r2 d

Small ≥ .1 ≥ .01 ≥ 0.2

Medium ≥ .3 ≥ .09 ≥ 0.5

Large ≥ .5 ≥ .25 ≥ 0.8

Cohen’s d

Calculator http://www.psychmike.com/calculators.php Usually use the first formula, requires M, SD,

and n Can calculate by hand with a simple formula,

but it doesn’t account for differences in sample size across conditions, so less accurate

d = = (Mean difference) / standard deviation

s = average standard deviation across groups

s

MM )( 21

Calculation Example: Does athletic involvement improve physical health?

M1 = 6.47

M2 = 6.75

s = (1.87+1.94) / 2 = 1.91 d = (6.47 – 6.75) / 1.91 = -0.28 / 1.91 = -0.15 = 0.15 weak effect!

Report

54. Physical Health

6.4720 125 1.87331

6.7543 175 1.94232

6.6367 300 1.91578

7. High School Athleteno

yes

Total

Mean N Std. Deviation

+/- sign is arbitrary, so usually just dropped

2014 article in Lancet (impact factor: 45.2)

Take-home from the abstract:

Differences

Between-group design required when it is impossible or impractical to put participants through more than one condition

Within-subject design is more powerful More likely to get significant p-value and bigger

effect sizes. Why? It allows each participant to serve as their own control, canceling out a lot of cross-participant variability

Between-group design requires more people Within-subject design is prone to ordering

effects (order of conditions can effect results), such as progressive effects, or carryover effects Solution: Counterbalancing

Mixed-model Design

Many different types, but requires Random assignment of people to different

groups Repeated measurement of dependent

variable over time Benefits of both designs Example: Pre-post between-group design

Experimental Group:

pretest Treatment posttest

Control Group: pretest   posttest