Download - Research Methods: Measurement
![Page 1: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/1.jpg)
Statistics II: Measurement & Data Analysis
Brian J. Piper
![Page 2: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/2.jpg)
Goals
• Operationalization• Psychometrics (Reliability & Validity)• Scales of Measurement• Hypothesis Testing• Advanced Topics– Power– Effect Size
![Page 3: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/3.jpg)
Operationalization
• What areas of the brain are important for music appreciation?
• Important = Active– EEG (electrical activity)– PET (sugar use)– fMRI (oxygen use)
![Page 4: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/4.jpg)
Operationalization 1: Spatial Function
• Do men and women differ in their spatial abilities?
• Spatial Function = mental rotation
![Page 5: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/5.jpg)
Operationalization 2: Motor Function
• Do men and women differ in their fine-motor abilities?
• Motor Function = rotary pursuit
![Page 6: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/6.jpg)
Reliability
• Consistency of measurement• Types– Test-re-test reliability
Time 1
Time 2
![Page 7: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/7.jpg)
Split-Half Reliability
• Consistency of measurement on two-halves of test
• Foundations for repeated measurements
Even
Odd
![Page 8: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/8.jpg)
Extension: Short-Form of Wisconsin (Berg) Card Sorting Test (BCST)
![Page 9: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/9.jpg)
Extension: Short-Form of Wisconsin (Berg) Card Sorting Test (BCST)
r(205) = +0.77
Fox et al. (in review). J Biol Biomed Reports.
![Page 10: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/10.jpg)
Validity
• Does a test measure what it claims to?• face “faith” validity: does it seem valid based on
intuition (non-numerical)
![Page 11: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/11.jpg)
Criterion Validity
• Does performance on new measure match with older “gold standard” measure?
• Continuous Performance Tests Example
Reaction Time (Conner’s)
Reac
tion
Tim
e (P
EBL)
![Page 12: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/12.jpg)
Construct Validity
• Does a test measure the construct it claims to?
• Convergent Validity: Does test A correlate (converge) with test B?
• Discriminant Validity: Does test A measure something different (discriminate) than test C?
![Page 13: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/13.jpg)
Measurement Scales (Self-Test)Level Definition
Nominal
Ordinal
Interval
Ratio
![Page 14: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/14.jpg)
Measurement Scales (Self-Test)Level Definition
Nominal categorical, e.g. sex
Ordinal ranking, e.g. Olympic medal
Interval equal spacing, e.g. IQ, ACT, SAT
Ratio true zero, e.g. Reaction Time
![Page 15: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/15.jpg)
Hypothesis Testing
• Null hypothesis (H0): A = B
• Alternative hypothesis (HA): A ≠ B
![Page 16: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/16.jpg)
Alpha
• The cut-off used to decide between H0 and HA
• Probability that finding is not due to chance (p value)
• .05: conventional• .10: liberal (some medical environments)• .01: conservative, large N
![Page 17: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/17.jpg)
Alpha
P value obtained
Decision
.50 H0
.11 H0
.06 H0
.0500000001 H0
.0499999999 HA
![Page 18: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/18.jpg)
Decision Making
HO is True HO is False
Fail to reject H0 Correct decision
Reject H0 Correct decision
Reality
Decision
![Page 19: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/19.jpg)
Decision Making
HO is True HO is False
Fail to reject H0 Correct decision
Reject H0 Type I error Correct decision
Reality
Decision
Type I Error: rejecting H0 when it is true
![Page 20: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/20.jpg)
Decision Making
HO is True HO is False
Fail to reject H0 Correct decision Type II error
Reject H0 Type I error Correct decision
Reality
Decision
Type I Error: rejecting H0 when it is trueType II Error: fail to reject H0 incorrectly
![Page 21: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/21.jpg)
Publication Bias
• H0 results often don’t get shared• Reasons: – Journal prestige– Research ego– Higher standard
• Solution: registry?– Replication?
![Page 22: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/22.jpg)
Solution 1: Effect Size Distribution
• A quantitative index of the magnitude of group difference’s
• Calculated as (Mean1 – Mean2)/SD
# St
udie
s
![Page 23: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/23.jpg)
Solution 1: Effect Size Distribution
• A quantitative index of the magnitude of group difference’s
• Calculated as (Mean1 – Mean2)/SD
# St
udie
s
# St
udie
s
![Page 24: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/24.jpg)
Solution 2: Power Analysis
• Power: the probability that a real effect will be detected
• Probability of Type II error: Beta• Power = 1 - Beta
N Power
50 0.40
100 0.70
500 0.80
1000 0.85
![Page 25: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/25.jpg)
Other Terminology
• Population: all members of identifiable group• Sample: a subset of the population• Confidence Interval: inferential statistic,
contains range of where population mean sits
![Page 26: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/26.jpg)
Margin of Error
• Is accurate if sample is representative of population.
![Page 27: Research Methods: Measurement](https://reader036.vdocuments.us/reader036/viewer/2022062312/5563ca4dd8b42ad6088b4881/html5/thumbnails/27.jpg)
Summary• Operationalization• Reliability & Validity (face, criterion, construct)• Scales of Measurement (nominal, ordinal,
interval, ratio)• Hypothesis Testing: Type I versus Type II error• Advanced Topics– Power– Effect Size