lecture 11: modes of measurement. how should we measure things? s-data: self-reports or...
TRANSCRIPT
How Should We Measure Things?
S-data: Self-Reports or Self-Judgments I-data: Informant Reports (People who know the
target) B-data: Behavioral Data (Naturalistic Observation,
Laboratory Measures, Certain Personality Tests like the TAT, Physiological Recordings)
L-data: Life Data (e.g., Court Reports, School Records)
Hint: BLIS Source: Funder (2007)
Self-Reports (S-Data)
Structured: a standardized list of questions and set way of responding is given to all participants
Open-Ended (e.g., How do you act in social situations?)
When using self-reports the psychologist believes the answer (more or less)
Aggression: S-DataBuss & Perry (1992) Aggression Questionnaire
Rate each item from 1 (“Extremely uncharacteristic of me”) to 5 (“Extremely characteristic of me”)– Given enough provocation, I may hit another
person.– If somebody hits me, I hit back.– I have become so mad that I have broken things.– I get into fights a little more than the average
person. Physical aggression scale correlated .33 with the
number of minutes hockey players spent in the penalty box for aggressive penalties (Bushman & Wells, 1998).
Comments on S-Data
Nothing tricky about this approach. Can get self-reports using paper surveys, phone
calls, the internet, or via face to face interviews Each of these specific methods has particular
advantages and disadvantages. (See Textbook) General Disadvantages
– Will people tell you the truth?– Can people even tell you the truth?– Do psychologists use S-Data too much?– Do we overwhelm our research participants?
One Relatively Big Problem …
Sometimes people interpret the meaning of questions differently than researchers intend…
Would You Say You “Had Sex” If …
Source: Sanders & Reinisch (1999) - JAMA Sample: 599 Undergraduates from Midwestern
Universities (59% Female) Average Age: 20.7 (SD = 3.1) 79% described themselves as “Moderate to
Conservative” Question: “Would you say you ‘had sex’ with
someone if the most intimate behavior you engaged in was ....”
Selected Results
99.5%99.2%99.7%P-V Intercourse
40.2%43.9%37.7%Oral Sex (Receiver)
39.9%43.7%37.3%Oral Sex (Actor)
2.0%2.9%1.4%Deep Kissing
OverallMenWomenBehavior
New S-Data: Experience Sampling
Daily diary keeping– What are you doing, thinking, or feeling right
now?– Prompted by a random buzzer, pager, or a PDA
alarm.– Asked to immediately write down your answer
Increasingly popular because there is the sense that these provide more “real time” reports of thoughts, feelings, and behavior
Comments on I-Data - Judgments about another person
Advantages– Based on large amount of information– Raters use common sense
Is Sally an outgoing person?
Disadvantages
Disadvantages– Some dimensions are easier to judge than others.– Letter of Recommendation Problem– Limited perspective – Do people act differently
around different people?– Judgments can be wrong or biased– We tend to remember behaviors that are extreme,
unusual, and/or emotionally provocative– What to do when raters do NOT agree?
Correlations between Self and Informant Reports of Internalizing Symptoms (N = 326)
.19 Anxious Arousal
.41 Anhedonic Depression
.39 General Distress
CorrelationDimension
“Our minds are forever sealed off from each other. We can only know each other from
watching what we do.”
David Funder
Self-Peer Correlations
Source: Watson (1989) – “Hello, my name is…”
Extraversion: .41 Conscientiousness: .16 Openness: .10 Agreeableness: .08 Neuroticism: -.01
Realistic Accuracy Model (RAM)
How do we get accurate informant reports?– Relevance: Person must do something relevant to
dimension being judged– Availability: Observer has to be able to see it– Detection: Observer has to actually see it– Utilization: Observer must remember the action
and interpret that piece of information correctly
Is Sally Courageous?
Ratings of Sally’s Courageousness
R: Is there a burning building?
D: Did I see her run into the building?
A: Can I see her run into the building?
U: Do I use this observation when it comes time to make my rating?
Some Reasons for Difficulties
Our thoughts, feelings, behaviors are always there (“Fish and Water Effect”): Fish don’t know they are wet because they are always surrounded by water.
People show a “false consensus” effect. They believe their own behavior is more common than it really is.
People believe their behavior was a natural response to the situation. “What else could I do?”
B-data: Behavioral Data/Observational Data
Examples: Record physiological responses to violent media (e.g., heart rate) or measure testosterone levels in the blood.
Measure aggression in the lab: Hot Sauce Paradigm (Lieberman et al., 1997)
– How much hot sauce will an individual give to a person who has “insulted” them?
– Individuals are told to “Put as much or as little hot sauce as you want” into a cup for another person to consume
Precisely measure how much hot sauce is placed in the cup
Comments on B-Data
Often collected in a laboratory or other settings where there is control
Advantages– Control is a good thing – we can
draw out certain behaviors such as aggression
– Quantifiable and objective
Comments on B-Data Part 2
Disadvantages– Well, what are we really measuring?
What do scores mean?– Tendency to ignore validity issues --
rely on “face validity”– Laboratories can be artificial
New B-Data: Implicit Measureshttps://implicit.harvard.edu/implicit/demo/
Goal is to assess automatic or unconscious thoughts and feelings. Usually attitudes.
Implicit measures are based on reaction times to pairings.
– Make judgments about the similarity between two pairings. How similar are the terms TV and good?
– Faster reactive time indicates a more tightly knit associative network.
Faster responses to negative pairings negative implicit attitude
Faster responses to positive pairings positive implicit attitude
Projective Techniques as B-Data
Assumption: How an individual interprets ambiguous stimuli says something about their psychological functioning
Different from “true” self-reports because projective techniques are seen as tests of personality – how you answer questions is interpreted.– Examples: Rorschach Ink Blots, Thematic
Apperception Test (TAT)
L-Data
L-data are life records. L-data for Aggression: School records (Disciplinary
Actions) and Court Records (Arrests and Convictions)
Advantages– Intrinsic Importance (Crime, Mortality, Marital
Status) Disadvantages
– Life outcomes are complicated and multidetermined
Summary Points
Nature of the research question often dictates which modes of measurement are most appropriate
Multiple sources of data help us “triangulate” on the true score.
Recurring Theme: Multiple methods and approaches are good!– Gordon Allport (1947): “No doors should
be closed in the study of personality” (p. 133-134)