![Page 1: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/1.jpg)
Ch 5: Measurement Concepts Ch 6: Observing Behavior
![Page 2: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/2.jpg)
Reliability Reliability refers to the consistency or stability of a
measure of behavior [p92]
If you weighed yourself now and then at the end of class and you weighed the same both times, you would say the scale is reliable.
True score: The real score on the variable
Measurement error: occurs when a measure yields inconsistent results; the greater the inconsistency, the greater the measurement error
![Page 3: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/3.jpg)
Reliability of Measures
How can we assess reliability?
• Correlation coefficients tell us how strongly two variables are related.
- Pearson Product-Moment Correlation Coefficient (noted as r in text) [p93]
Coefficients range from 0.00 to - 1.00 and 0.00 to +1.00
Sign of the coefficient indicates direction
Value of the coefficient indicates the strength
![Page 4: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/4.jpg)
Reliability of Measures
- 1.00 + 1.000.00
Variables covary in opposite directions
Variables covary in the same direction
![Page 5: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/5.jpg)
Methods of assessing reliability [pp 94-96]
Test-retest reliability: Assesses reliability of a score by measuring the same individuals at 2 points in time
Internal consistency reliability: Assesses reliability of the assessment tool (e.g. test) at one point in time Questions should yield consistent results Split-half reliability – individual’s total score on one half
of the test is correlated with the total score on the other half of the test
![Page 6: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/6.jpg)
Methods of assessing reliability: Interobserver (Interrater) Reliability
A measure of how often two or more observers agree (are consistent) in their observations. [p96]
Nominal scale: percentage agreement
How we assess interobserver reliability:
Number of times two observers agree
____________________ X 100 Number of opportunities to agree
![Page 7: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/7.jpg)
Construct Validity of Measures [pp 97-100]
Refers to the adequacy of the operational definition of variables
Is the measure that is used actually assessing what it is supposed to assess?
If so, it has face validity.
Example: Are facial expressions an adequate measure of happiness?
![Page 8: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/8.jpg)
Indicators of construct validity
Face Validity: The content of the measure appears to reflect the construct being measured.
Content Validity: The content of the measure is linked to the universe of content that defines the construct• Facial expressions are part of a set of behaviors related
to happiness, such as body posture, thoughts, etc.)
Example: Are facial expressions an adequate measure of happiness?
![Page 9: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/9.jpg)
Indicators of construct validity
Predictive Validity: Scores on the measure predict behavior on a criterion measured at a time in the future• SAT (Scholastic Aptitude Test predicts future success at
a university) Concurrent Validity: Scores on the measure are
related to a criterion measured at the same time• People who score high on a verbal anxiety test
experience increased sweating at the same time
• People who have happy facial expressions concurrently report feeling happy
![Page 10: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/10.jpg)
Indicators of construct validity
Convergent Validity: Scores on the measure are related to other measures of the same construct.• A score for happy facial expression is related to a
score for body posture or mood or optimism Discriminant Validity: Scores on the measure
are not related to other measures that are theoretically different.• A score for happy facial expression is not related to
one for intelligence or cleanliness
![Page 11: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/11.jpg)
Measurement Scales
Four levels for quantifying behavior: nominal, ordinal, interval, and ratio [p102-105]
![Page 12: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/12.jpg)
Nominal scale
Classifies behaviors, events, and characteristics into different categories [p102]
No numerical or quantitative properties
Independent variables are often nominal or a categorical variable
![Page 13: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/13.jpg)
OrdinalScale
Measurement scale in which events and behaviors can be rank ordered (i.e, first, second, third, etc.) [p103]
• Allows categories to be ordered first to last, highest to lowest, biggest to smallest, etc.
• Quantitative but no values attached to the intervals
![Page 14: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/14.jpg)
Interval Scale
Measurement scale that allows researcher to specify how far apart two observations are on a given dimension [p104]
Difference between the numbers is meaningful• Intervals are equal in size
• Quantitative but no meaningful zero reference point
![Page 15: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/15.jpg)
Ratio Scale
Measurement scale that is quantitative, with all numerical properties including an absolute zero reference point [p104]
![Page 16: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/16.jpg)
Let’s practice! Identify the measurement scale for the following data:
Circle your marital status: Married Single Divorced Engaged
![Page 17: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/17.jpg)
Let’s practice! Identify the measurement scale for the following data:
Circle your marital status: Married Single Divorced Engaged
NOMINAL
![Page 18: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/18.jpg)
Let’s practice! Identify the measurement scale for the following data:
Do you go to work? Yes No
![Page 19: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/19.jpg)
Let’s practice! Identify the measurement scale for the following data:
Do you go to work? Yes No
NOMINAL
![Page 20: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/20.jpg)
Let’s practice! Identify the measurement scale for the following data:
If you work, how many hours a week do you work? _______
![Page 21: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/21.jpg)
Let’s practice! Identify the measurement scale for the following data:
If you work, how many hours a week do you work? _______
RATIO
![Page 22: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/22.jpg)
Let’s practice! Identify the measurement scale for the following data:
Rate your enjoyment of college on the scale below.
1 2 3 4 5
Not Very Much Very Much
![Page 23: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/23.jpg)
Let’s practice! Identify the measurement scale for the following data:
Rate your enjoyment of college on the scale below.
1 2 3 4 5
Not Very Much Very Much
INTERVAL
![Page 24: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/24.jpg)
Let’s practice! Identify the measurement scale for the following data:
What is your class standing?
Freshman Sophomore Junior Senior
![Page 25: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/25.jpg)
Let’s practice! Identify the measurement scale for the following data:
What is your class standing?
Freshman Sophomore Junior Senior
ORDINAL
![Page 26: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/26.jpg)
Places to Observe on Campus (Spring 2010) 1. Area in the middle of campus, by the Info Trolley. 2. By the food place on campus which is near the gym/pools. 3. By the statue of the boy overlooking the water feature. 4. Entrance of the SE parking structure (people exiting structure) 5. Entrance of the SE parking structure (people entering structure) 6. Entrance of the SW parking structure (people exiting structure) 7. Entrance of the SW parking structure (people entering structure) 8. By the entrance of the bookstore. 9 In front of C Building, by mirror pools 10. Shuttle stop by U Building 11. Bus stop on Colorado in front of library 12. Passenger drop off/pick up (Colorado B. in front of L Bldg) 13. Starbucks
![Page 27: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/27.jpg)
Reactivity
Occurs when individuals change their usual behavior, when they know they are being observed [p101]
Minimize reactivity by:• Allowing time for individuals to become
used to the presence of an observer or the recording equipment
![Page 28: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/28.jpg)
Time sampling: Researchers choose time intervals for making observations
• Systematic: Schedule observations to occur at a regular time
• Random: Use some random means for identifying times for observations
• Event sampling is used for rare events
Situation sampling: Researchers choose different settings, circumstances, and conditions for their observations
Sampling Behavior [p116]
![Page 29: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/29.jpg)
Classification of Observational Methods
Two categories of observational methods:
• Observation without Intervention
• Observation with Intervention
![Page 30: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/30.jpg)
Observation without Intervention
Naturalistic Observation (AKA: field work or field observations): Observation in natural (“real-world”) settings without an attempt to intervene or change the situation [p110]
![Page 31: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/31.jpg)
Participant observation
Observer is an active participant in the natural setting he or she observes [p112-113]
Norah Vincent
“My Life as a Man”
• Undisguised (unconcealed): people in the setting know they are being observed
• Disguised (concealed): people don’t know they are being observed
![Page 32: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/32.jpg)
Observation withoutIntervention
Physical traces: The remnants, fragments, and products of past behavior
Products: Creations, constructions, or other artifacts of earlier behavior
![Page 33: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/33.jpg)
Natural-use traces
Produced without any intervention by the investigator
![Page 34: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/34.jpg)
Controlled-use traces
Produced with some degree of intervention or manipulation by the investigator
![Page 35: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/35.jpg)
Natural use or controlled use trace?
![Page 36: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/36.jpg)
Natural use or controlled use trace?
![Page 37: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/37.jpg)
Natural use or controlled use trace?
![Page 38: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/38.jpg)
Archival records
Public and private documents that describe the activities of individuals, institutions, governments, and other groups [p204]
![Page 39: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/39.jpg)
Archival Research [118-121]A non-reactive measure (or indirect method) for collecting data—when the individual who did the behavior is no longer present
Archival research involves using previously compiled information to answer research questions
![Page 40: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/40.jpg)
Running records & records of specific events
Running records are continuously kept and updated (e.g., check book)
Records of specific events (e.g., diploma)
![Page 41: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/41.jpg)
Archival data are used to:
test hypotheses as part of a multimethod approach
test the external validity of laboratory findings
test hypotheses about past behavior assess the effect of a natural treatment
![Page 42: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/42.jpg)
Multimethod approach
Researchers use a variety of measures to examine a research question
![Page 43: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/43.jpg)
Natural treatments
Naturally occurring events that impact society and individuals
![Page 44: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/44.jpg)
Selective deposit
Occurs when some information is selected to be included in the archival record, but other information is not
![Page 45: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/45.jpg)
Selective survival
Occurs when information is lost or missing from an archival source
![Page 46: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/46.jpg)
Content analysis
The process of making inferences based on objective coding of archival data [p120]
![Page 47: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/47.jpg)
Quantitative analysis
Classifying events and behaviors into categories to count their frequency of occurrence [p109]• Assign numerical values to responses and measures and
then subject the data to quantitative statistical analyses
Ex: Count the number of times gender-stereotypical jobs were assigned to characters in the story.
![Page 48: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/48.jpg)
Qualitative analysis Subjective judgments about the content in an
archival record [p109]• Describe behavior or findings based
on themes that emerge from the data.
• Data are nonnumerical and expressed
in language and images
Ex: Watch the tape of Osama Bin Laden and tell me if you think he is being deceptive.
![Page 49: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/49.jpg)
Three Steps of Content Analysis
1) Identify a relevant source
2) Sample selections from the source
3) Code units of analysis
![Page 50: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/50.jpg)
Observation with Intervention [p114]
Systematic observation: The careful observation of one or more behaviors in a particular setting. Use when: • interest is in only a few very specific
behaviors
• observations are quantifiable
• researcher has developed prior hypothesis
![Page 51: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/51.jpg)
Field experiment
Researcher manipulates an independent variable in a natural setting and observes behavior (dependent variable) [pp 83, 113]
![Page 52: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/52.jpg)
Coding systems for systematic observation… [p115]
are either (1) developed to fit the needs of the particular study or (2) “borrowed” systems developed by others
should be as simple as possible
must allow researchers to easily categorize behaviors
![Page 53: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/53.jpg)
Coding systems can involve:
Comprehensive records of people’s behavior (e.g., complete records, such as video tapes)—a qualitative record
Selecting specific behaviors to record—a quantitative record [pp115-116]
![Page 54: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/54.jpg)
Equipment & Narrative RecordsUsed when researchers want a complete (comprehensive) reproduction of people’s behavior [pp 115-116]
• Examples: video and audio recordings, field notes
![Page 55: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/55.jpg)
Systematic Observation: Methodological Issues [pp115-116]
1. Equipment: can fail
2. Reactivity: the probability that the presence of the observer will affect behavior
3. Reliability: refers to how stable/consistent the measure is over time or between observers
4. Sampling: refers to how participants and behaviors are chosen to be studied
1. Larger samples of participants and multiple samples of behavior can increase both internal and external validity.
![Page 56: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/56.jpg)
Case Studies [pp117-118]
A case study provides a description of an individual.
• Usually the individual is a person, but may also be a setting.
• A psychobiography is a type of case study in which a researcher applies psychological theory to explain the life of an individual.
![Page 57: Ch 5: Measurement Concepts Ch 6: Observing Behavior](https://reader035.vdocuments.us/reader035/viewer/2022062216/56649da55503460f94a90c1b/html5/thumbnails/57.jpg)
Case Studies…
are done when an individual possesses a particularly rare, unusual, or noteworthy condition.
provide unique data about some psychological phenomenon
and the insights gained from them may lead to other research using other methods