PSBE2-08

Research Methods

Week 4

1Tassos Sarampalis

The week’s reading

Chapter 4 (pp. 87-122) Chapter 5 (pp. 123-159)

2

RELIABILITY AND VALIDITY

3

Reliability

The consistency or repeatability of the measurements

The degree of accuracy to which a study reflects or assesses the specific concept that the

researcher is attempting to measure

4

Validity

All measurements are aggregates of:

The ‘true’ value+

Random (chance) error

5

Stability and Equivalence

• Test-retest reliability• Alternate-form reliability• Internal-consistency reliability

6time

Version1

Version1

Version2

Version2

Reliability

Equivalence

Internal Consistency

How related are individual items within the measure?

• Spearman-Brown• Kuder and Richardson Formula 20 (KR20)• Cronbach’s alpha

7

Replication in Research

• What do we mean by ‘replication’?

• Three factors to consider:– When– How is the replication is conducted?– By whom

8

Replication in Research

• What do we mean by ‘replication’?

• Three factors to consider:– When– How is the replication is conducted?– By whom

Early replications are more valuable than later ones

9

Replication in Research

• What do we mean by ‘replication’?

• Three factors to consider:– When– How is the replication is conducted?– By whom

The difference between precise and varied replication

10

Replication in Research

• What do we mean by ‘replication’?

• Three factors to consider:– When– How is the replication is conducted?– By whom

Independence of replicators

11

Validity

• Content-related– How well a measure represents all dimensions

• Criterion-related– Concurrent vs. predictive validity

• Construct-related– Convergent vs. discriminant validity

12

OBSERVATION

13

Observational Methods

• Categories based on intervention by researcher– Observation without Intervention– Observation with Intervention

• Categories based on methods for recording behavior– comprehensive record– selected behaviors

Observation without Intervention

• Naturalistic Observation– observation in natural (real-world) setting without

attempt to intervene or change situation– use when ethical considerations prevent

experimental manipulation• Goals– describe “normal” behavior, examine relationships

among naturally occurring variables– establish external validity of lab findings

Observation with Intervention

• Most psychological research involves intervention

• Three methods in natural settings– participant observation– structured observation– field experiment

Observation with Intervention

• Participant observation– observer is active participant in the natural setting

he or she observes• undisguised: people know they’re being observed

(e.g., Kitchen Stories, 2003 Norwegian film)• disguised: people don’t know they’re being observed

Observation with Intervention

• Problems with participant observation– Reactivity• when people change their usual behavior because

they’re being observed• disguised participant observation controls reactivity

– Observers lose objectivity or become too involved in situation

– Observers influence behavior of people they’re observing (e.g., Hawthorne Effect)

Observation with Intervention

• Structured observation– set up (structure) specific situation in order to

observe behavior– used when behavior is difficult to observe as it

naturally occurs– researchers use confederates to structure

situations– problems: when observers don’t follow same

procedures across observations

Observation with Intervention

• Example of structured observation– Simons and Levin (1998): “change blindness”

http://www.youtube.com/watch?v=FWSxSQsspiQ

Observation with Intervention

• Field Experiment– manipulate independent variable in natural setting

and observe behavior (dependent variable)• two or more conditions to compare (IV)• often use confederates to create conditions• strive for control in natural setting

On Being Sane in Insane Places

Original Article:http://cooley.libarts.wsu.edu/soc3611/Documents/Being_Sane_in_Insane_Places.pdf

Video:http://www.youtube.com/watch?v=Kq-7uvVOoyk

In Fiction:Simpson’s Episode S03E01 – “Stark Raving Dad“

22

Dr. David Rosenhan

Maximising Serendipity

• With– Common sense– Hard work– The right info

– And a bit of luck

23

Raising the flag on Iwo Jima - 1945

Errors in Observational Research

• Non-interactional artifacts– Interpreter biases– Observer biases

• Interactional artifacts

24

Non-reactive Measures

• Archival research

• Physical Traces

• Unobtrusive Observation– Simple– Contrived

25

Selecting Judges

• Identify the type of judge that is most likely to produce accurate judgments

– Search the literature– Test them

26

Rating Biases

• Errors of Leniency– e.g., Halo Effect

• Severity Errors• Errors of Central Tendency• Logical Errors

27

Thank you

28