experimental design experiment: a type of research study that tests the idea that one variable...

Experimental Design

• Experiment: A type of research study that tests the idea that one variable causes an effect on another variable.

Anatomy of an ExperimentExample

Memory Cues No Memory Cues

N1 = 10 N2 = 10

M1 = 16.2 M2 = 9.9

S1 = 2.49 S2 = 2.33

Independent variable = Memory Training Group

Dependent variable = Memory for personal history

Anatomy of an ExperimentExample

Experimental Control Group Group

Memory Cues No Memory Cues

N1 = 10 N2 = 10

M1 = 16.2 M2 = 9.9

S1 = 2.49 S2 = 2.33

• The study allows the researcher to determine that on variable causes an effect on another variable.

Internal Validity

Conditions to establish internal validity

1. Time-Order relationship

Cause Effect I.V. D.V.

Conditions to establish internal validity

2. No alternative explanations

• The difference between the means is due only to the independent variable.

• Anything else represents a threat to the internal validity of the study

Threats to internal validity

• Non-equivalent control group

– Confound: A way in which the groups differ from each other, other than the independent variable.

– Controlling for confounds• 1. Random assignment to groups• 2. Matching

Threats to internal validity

• Floor or Ceiling effects

– The independent variable has made the groups different from each other, but the dependent variable is unable to detect it.

– Floor effect: The test is so difficult that everyone gets a very low score.

– Ceiling effect: The test is so easy that everyone gets a high score.

– They make the means closer together than they should be.

Threats to internal validity

• Experimenter effect

– The experimenter gives an indication of what they want or expect the subject to do in a particular condition.

• Participant effect

– The participant changes their behavior to fit what they think the researcher is studying.

Ways to address experimenter and participant effects

– Single-blind study: The participant doesn’t know which condition they’re in.• Example: a placebo-controlled condition.

– Double-blind design: Neither the participants or the researcher knows which condition the subject is in.

• The results of the study are generalizable

1. Generalization to different samples– Get the same results if repeat the same study

with a different sample (from the same population)

– Replication

External Validity

2. Generalization to different populations– Get the same results if repeat the same study with a

sample from a different population

3. Generalization to different settings– Get the same results under different conditions– The effect is observed in more than one setting– Example: The effect is observed in real life, not just in

the laboratory

External Validity

Independent Samples T-Test

• Tests the difference between two sample means

Memory Cues No Memory Cues

N1 = 10 N2 = 10

M1 = 16.2 M2 = 9.9

S1 = 2.49 S2 = 2.33

Prediction of the researcher: The mean of the Memory Cues Group will be significantly higher than the mean of the No Memory Cues Group.

Independent Samples T-TestPrediction of the researcher: The mean of the Memory

Cues Group will be significantly higher than the mean of the No Memory Cues Group.

– Example of a one-tailed test

– One-tailed test: One mean is predicted to be higher or lower than the other one.

– Two-tailed test: One mean is predicted to be different from the other one.

Independent Samples T-TestPrediction of the researcher: The mean of the Memory

Cues Group will be significantly higher than the mean of the No Memory Cues Group.

– Example of a one-tailed test

– Alternative hypothesis: The mean of the Memory Cues Group is significantly higher than the mean of the No Memory Cues Group.

– Null hypothesis: The mean of the Memory Cues Group is not significantly higher than the mean of the No Memory Cues Group.

Independent Samples T-Test– No way to know for sure which hypothesis is true.

– We can know the odds that the null hypothesis is true.

– We can decide how unlikely the null hypothesis would have to be before we can’t believe it anymore.

That’s the Alpha Level of the test.

– “α = .05” means “Reject the null hypothesis if the odds are less than 5% that it’s true”

Independent Samples T-TestAn independent samples t-test tells you if the odds are

less than 5% that the null hypothesis is true.

1. Find the number we’re making our decision about• It’s the difference between the two group means• M1 – M2 = 16.2 – 9.9 = +6.3

• We’re comparing this number to a difference of zero.

2. Convert that number to a standard score

– In SPSS, t = +5.85– The difference between the two sample means is 5.85

standard deviations above a difference of zero.

Independent Samples T-Test

3. Find how far from zero that number needs to be to be significant Critical Value for t

• We predicted that this difference would be in the positive direction, so it’s a one-tailed test.• α = .05• Degrees of freedom = N1 + N2 – 2 10 + 10 – 2 = 18

• Critical value = +1.73• Decision rule: If t ≥ +1.73, reject the null hypothesis.

Independent Samples T-Test

Conclusion: The mean of the Memory Cues Group is significantly higher than the mean of the No Memory Cues Group, t (18) = 5.85, p < .05.