1/15/18

1

EXPERIMENTAL METHODS

Experimental Designs• Experiments – examining behavior in a _______ lab setting

• Always approached __________: Describing behavior using numbers

• Controlled lab setting separates experiments from non-experiments• Allows experimenter to

know exactly what’s causing behavior

Experiments Uncover Causation• Experiments try to control everything that affects participants• Making sure all participants have the same experience

• Same room

• Same temperature / lighting

• Same materials

• Same experimenter

• Same confederate(s)

1/15/18

2

Experiments Uncover Causation• Experiments try to control everything that affects participants• Making sure all participants have the same experience

• Every aspect of their experience is _________• Room is checked and reset

between each participant

• Confederate(s) behavior is all scripted

Experiments = Control• Example:

• Experiment designed to test effectiveness of new test-taking strategy• Experimental group: Comes to lab and is trained and

tested in groups• ______ group: Takes test at home online

Experiments = Control• Example:

• Experiment showed those trained in the new strategy got higher scores

• What might have caused this effect?• Home distractions, “_________,” social facilitation, etc.

1/15/18

3

Experiments = Control• Example:

• Experiment designed to test effectiveness of learned test-taking strategy• Experimental group: Comes to lab, is trained in

groups, then tested

• Control group: Comes to lab, is trained in an unrelated task in groups, then tested

• Experiment showed those trained in the new strategy got higher scores• Since everything was the same except for IV, the

IV likely ________ the effect

Experiments = Control• Experiments cannot control everything

• What if control and experimental groups just happened to differ in intelligence?• We cannot make everyone coming into the lab

have equal intelligence, schooling, etc.

• Randomization = _________• Whatever cannot be controlled can be randomized between conditions• With just 30 people in each condition, most

participant factors get spread out evenly

Experiments Uncover Causation• Through control of environmental factors and randomization of participant factors…• We can know exactly what affects the DV in the experimental groups

• Rules out possible “confounds”• Any variable (environmental/ participant) that

varies along with the ____ and affects the ____

• E.g., Distractions at home (confound) only being present in control group (varies along with IV) affecting test scores (affects the DV)

1/15/18

4

Experiments Uncover Causation• Good experiments = High “______ validity”

• Ability to know exactly what is affecting the DV

• That’s the hallmark of good science!

• All sciences, at their core, are methods of understanding cause and effect

The Basic Experiment• “_____________ Design”

• Participants randomly assigned to…• Experimental Group –

Includes the affecting IV• Control Group – Doesn’t

include the affecting IV

• After the manipulation, the DV is measured in the same way for both groups

Virus Saline

The Basic Experiment• Example: Bargh, Chen, & Burrows, 1996

• Hypothesis: Subtle primes can affect behavior• Experimental Group – Completed word unscrambling

task, where many of the words were related to the ______ (e.g., wrinkle, grey, wise)

• Control Group – Completed similar word unscramb-ling task that didn’t include any elderly-related primes

• After manipulation, experimenters recorded the time it took participants to exit the lab room

1/15/18

5

The Basic Experiment• Example: Bargh, Chen, & Burrows, 1996

• It was expected that those primed with elderly-related words would walk _______ when exiting the study (emulating the elderly)• That’s exactly what

they found across two studies

The Basic Experiment• Example: Daniel Pink’s DRIVE (2010)

• Hypothesis: Higher pay hinders quality of work, because it leads to job-related _______• Computer programmers in India asked to create new

database program (like Excel)• Experimental Group – participants given 2 months

salary for 2 days work• Control Group – given 2 days salary for 2 days work

• Afterward, their work (the program) was judged according to its efficacy and elegance (DVs)

The Basic Experiment• Example: Daniel Pink’s DRIVE (2010)

• Found those who received 2 days salary created a much better product than those paid 2 months salary

• In both examples…• Everything was kept constant between groups

except for the IV• E.g., primed words, salary

• So, even if the results are a bit hard to believe, they were most likely caused by the IV

1/15/18

6

• Advantages• Can be used to infer _________

• Produces results that are clear and easy to understand

• Disadvantages• Cannot test most participant variables as IVs

• E.g., income, intelligence, personality

• Since you cannot directly __________ these factors, you cannot infer that they affect the DV

Basic Experiments

More Complex Experiments• To provide more evidence of an effect, experi-menters can test ______________ of the IV

• Example: Bargh, Chen, & Burrows, 1996• Hypothesis: Subtle primes can affect behavior

• Experimental Group #1 – primed with words related to rudeness (e.g., bother, disturb, annoy)

• Experimental Group #2 – primed with words related to politeness (e.g., considerate, patient, courteous)

• Control Group – primed with words unrelated to either rudeness or politeness

More Complex Experiments• Example: Bargh, Chen, & Burrows, 1996

• Participants told to contact experimenter standing outside• Experimenter talked with

someone else, ignoring participant for up to 10 minutes, unless interrupted

• Recorded whether partici-pant “rudely” interrupted experimenter

1/15/18

7

More Complex Experiments• Example: Daniel Pink’s DRIVE (2010)

• Repeated previous experiment, but added a third group that got paid 2 weeks salary

Perfo

rman

ce

Low

Med

High

2-days pay 2-months pay2-weeks pay

More Complex Experiments• By adding extra levels to the IV…

• Able to ________ effects of IV multiple times• Adds to internal validity (believability) of the results

• E.g., showing effect of priming through both rudeness- and politeness-related words

• Able to show exactly how the IV affects the DV• E.g., at what point does an increased salary begin

to impede performance

Perfo

rman

ce

Low

Med

High

2-days pay 2-months pay2-weeks pay

• Participants assessed both before and after the manipulation• Often used to assess the effectiveness of

“___________”• Example: Effect of behavioral therapy on smoking

• Number of cigarettes smoked per day is recorded• Experimental group: Participants put on fixed

schedule for smoking less and less• Control group: Participants are allowed to lessen

smoking in any way they wish• After 1 month, # of cigarettes per day is assessed

Pretest Posttest Designs

1/15/18

8

• Example: Effects of behavioral therapy on smoking

Pretest Posttest Designs

# of

Cig

aret

tes

per D

ay

1

> 10

Pretest Posttest

5

Experimental group

Control group

• _______ – participants who drop out mid-study

• Creates problems if participants who drop out are different from those who stay in

Pretest Posttest Designs

# of

Cig

aret

tes

per D

ay

1

> 10

Pretest Posttest

5

Experimental group

Control group

• Example: Effects of behavioral therapy on smoking• Does this necessarily mean the treatment worked?

• What if half the experimental group dropped out of the study before it was completed?

• Difference between mortality and non-mortality pars.

• Many people who were heavy smokers dropped, because they couldn’t handle the treatment

Effect of Mortality

# of

Cig

aret

tes

per D

ay

1

> 10

5

Participants who dropped

Participants who stayed

1/15/18

9

• Example: Effects of behavioral therapy on smoking

Effect of Mortality

# of

Cig

aret

tes

per D

ay

1

> 10

Pretest Posttest

5

Experimental group

Control group

Treatment had no effect, except driving away heavy smokers

• Example: Wierzbicki & Pekarik, 1993• Meta-analysis of 125 studies testing effectiveness

of psychotherapy

• Discovered an average of 47% of participants dropped out of each study before its completion

• Which participants were most likely to drop?• Those with more _______ psychological illness

• Dropped because of hospitalization, jailing, suicide, distrust of therapist (paranoia), etc…

• Those who weren’t _______ from therapy

Effect of Mortality

• Advantages• Can be a good way to clearly display effects of

treatment vs. non-treatment

• Sometimes it’s ________ to have a control group (e.g., depriving people of beneficial treatment)

• Helps effect of IV to stand out with fewer pars.Effect of treatment is often clouded by between-person variability• By assessing the same participants before and after

treatment, you take away this extra “noise” in the data

Pretest Posttest Designs

1/15/18

10

• Example: Testing effect of social rejection on self-esteem• Posttest only design: effect of IV has to rise above

within-group variability in DV

Pretest Posttest Designs

Self-

este

em

Low

High

Social Rejection Group (Experimental)

Medium

No Social Rejection Group (Control)

• Example: Testing effect of social rejection on self-esteem• Pretest posttest design: effect of IV is shown in each

individual

Pretest Posttest Designs

Self-

este

em

Low

High

Before Social Rejection

After Social Rejection

Medium

• Effect _________ much more• Each individual participant acts as a replication of

the effect

Pretest Posttest Designs

Self-

este

em

Low

High

Before Social Rejection

After Social Rejection

Medium

(Average Effect of Rejection)

1/15/18

11

• Disadvantages• __________

• Have to make sure most participants stick with the experiment right to the end

• Should track participants who drop out

• Figure out if they are significantly different from those who stay in

Pretest Posttest Designs

• Try to make experiment as short as possible (days vs. weeks makes a big difference)

• Offer _________ (e.g., cash) for returning

• Make it clear how important it is for participants to keep coming back

Pretest Posttest Designs• How can you lessen mortality rates?

• Disadvantages• ______ Effects – simply taking a pretest can affect

the posttest measure

• Practice effect – repeated performance of the same test leads to better performance over time• E.g., taking the same IQ test over and over again

• __________ effect – pretest sensitizes or desensitizes participants to the manipulation (changes reaction)• How might exposure self-esteem measure affect

reactions to subsequent social rejection?

Pretest Posttest Designs

1/15/18

12

• Disadvantages• ______ Effects – simply taking a pretest can affect

the posttest measure

• _______ effect – pretest clues participants into what the hypothesis is

Pretest Posttest Designs

• Most participants want to help the experimenter

• May fake what they think is the desired reaction

• Increase the ____ between pre- and posttest• Participants tend to forget about the pretest pretty quickly

• Add ________ tasks between pre- and posttest• E.g., have participants play a video game

• Try to use pre-tests that seem unrelated to the manipulation• E.g., use an implicit measure of self-esteem (testing

association between self and positive vs. negative words)

Pretest Posttest Designs• How can you lessen order effects?

Type of Participant Usage• ______________ Design – participants are randomly assigned to experimental or control conditions

• ________________ Design – participants are run through both experimental and control conditions• E.g.,

• Participants play round of cyberball where they are rejected• Self-esteem is measured• Participants play round of cyberball where they aren’t

rejected• Self-esteem is measured again

1/15/18

13

Repeated Measures Design• Advantages

• Requires far _______ participants• Saves lots of time /

resources

• Some samples are very difficult/ expensive to attain• E.g., studying effects

of stress on expert meditators (monks from Tibet)

Repeated Measures Design• Advantages

• Cuts out “______” in data caused by individual variability• Helps effects to better stand out

• Sometimes exposure to one condition after another is more naturalistic• E.g.,

• Testing effects of TV ads on mood

• Examining effects of political slogans on attitudes• Investigating effects of online dating profiles on mate

preferences

Repeated Measures Design• Disadvantages

• Again, there is the problem of ______ effects• Example:

• In first round of cyberball, participants are rejected and then their self-esteem is measured

• In second round of cyberball, participants are not rejected and then their self-esteem is measured

• How might participants interpret this order of events?

1/15/18

14

Repeated Measures Design• Disadvantages

• If study contains many conditions, the experiment can end up being really long

• Danger of “_______” participants

• Could cause “satisfyzing”• Participants stop

caring enough to perform to the best of their ability

Better Repeated Measures Design• Counterbalancing can nullify order effects

• Participants assigned to different order of conditions• Allows order effects to cancel

each other out• E.g., initial rejection lowers

self-esteem, but initial acceptance heightens self-esteem

• Allows experimenter to examine differences caused by ______ effects

Making the Most of Participants• ____________ Design - participants are prescreened

and then assigned to different conditions • Important participant factors are equalized between

conditions• E.g., Study of effects of social rejection on self-esteem

• Use prescreening measure of self-esteem• Assign equal numbers of low and high esteem

participants to each condition• Removes “noise” caused by participant factors

• But, you don’t have to worry about order effects, fatigue, or other pitfalls of repeated measure designs

1/15/18

15

• Example: Effect of social rejection on self-esteem• Without prescreening, experimental and control

groups may just happen to differ in self-esteem• Especially likely if sample size is ______

Matched Pairs Design

Self-

este

em

Low

High

Group 1 Group 2

Medium

• Example: Effect of social rejection on self-esteem• With prescreening, you can randomly assign same

number of low- and high-self esteem participants to each condition

Matched Pairs Design

Self-

este

em

Low

High

Group 1 Group 2

Medium

Overview• Experiments show _________ through control or randomization of everything that affects participants

• Different types of experimental designs:

• Posttest only design

• Pretest posttest design

• Independent groups design

• Repeated measures design

• Matched pairs design