Download - Using and Reporting Measures of Effect Size
![Page 1: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/1.jpg)
Using and Reporting Measures of Effect Size
Roger E. Kirk
Department of Psychology & NeuroscienceBaylor University
![Page 2: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/2.jpg)
Three Categories of Measures of Effect Magnitude
1. Measures of effect size (typically, standardized mean differences)
2. Measures of strength of association
3. A large category of other kinds of measures
2
![Page 3: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/3.jpg)
1. Estimate the sample size required to achieve an acceptable power
2. Integrate the results of empirical research studies in meta-analyses
3. Supplement the information provided by null hypothesis significance tests
4. Determine whether research results are practically significant
Four Purposes of Measures of Effect Magnitude
3
![Page 4: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/4.jpg)
1. Answers the wrong question
What we want to know is the probability that the null hypothesis is true, given our data:
Null hypothesis significance testing tells us the probability of obtaining our data or more extreme data if the null hypothesis is true:
Four Criticisms of Null Hypothesis Significance Testing
4
![Page 5: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/5.jpg)
2. Is a trivial exercise
According to John Tukey
“the effects of A and B are always different—in some
decimal place—for any A and B. Thus asking
‘Are the effects different?’ is foolish.”
Four Criticisms of Null Hypothesis Significance Testing (continued)
5
![Page 6: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/6.jpg)
According to Bruce Thompson
“Statistical testing becomes a tautological search for
enough participants to achieve statistical significance.
If we fail to reject, it is only because we have been too
lazy to drag in enough participants”
Four Criticisms of Null Hypothesis Significance Testing (continued)
6
![Page 7: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/7.jpg)
3. Requires us to make a dichotomous decision from a continuum of uncertainty
The adoption of .05 as as the dividing point between
significance and non-significance is quite arbitrary.
Four Criticisms of Null Hypothesis Significance Testing (continued)
7
![Page 8: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/8.jpg)
4. Does not address the question of whether results are
important, valuable, or useful: that is, their practical
significance.
Four Criticisms of Null Hypothesis Significance Testing (continued)
8
![Page 9: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/9.jpg)
1. Is an observed effect real or should it be attributed to chance?
2. If the effect is real, how large is it?
3. Is the effect large enough to be useful?
Three Basic Questions that Researchers Want to Answer from Their Research
9
![Page 10: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/10.jpg)
“Because confidence intervals combine information
on location and precision and can often be used to
infer significance levels, they are, in general the best
reporting strategy . . . Multiple degree-of-freedom
indicators are often less useful than effect-size
indicators that decompose multiple degree-of-freedom
tests into one degree-of-freedom effects . . .
Recommendation of the APA Publication Manual
10
![Page 11: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/11.jpg)
(1) Cohen’s
Three ways to estimate
Cohen
Glass
Hedges
Effect size
11
![Page 12: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/12.jpg)
= 0.2 is a small effect
= 0.5 is a medium effect
= 0.8 is a large effect
Guidelines for Interpreting
12
![Page 13: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/13.jpg)
(1)
Sample estimators of omega squared and the intraclass correlation
Strength of Association
13
![Page 14: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/14.jpg)
= .001 is a small association
= .059 is a medium association
= .138 is a large association
Guidelines for Interpreting Omega Squared
14
![Page 15: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/15.jpg)
Effect Size Strength of Association Other Measures
Cohen d, f, g, h, q,
Glass g’Hedges gMahalanobis DMean1 – Mean2 Mdn1 – Mdn2
Mode1 – Mode2
RosenthalTangThompson d* Wilcox
Measures of Effect Magnitude ________________________________________________________________
r, rpb, r2, R, R2, , , f2
Chamber re
Cohen f2Contingency coef CCramér VFisher ZFriedman rm
Goodman
Herzberg R2
Kelly
_______________________________________________________________
Abs. risk reduction ARRCliff pCohen U1, U2, U3
Shift functionDunlap CLR
Grisson PSLogit d’McGraw & Wong CLOdds ratioPreece ratio of successProbit d’Relative risk RRSánchez-Meca dCox15
![Page 16: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/16.jpg)
Effect Size Strength of Association Other Measures
Wilcox & Muska
More Measures of Effect Magnitude ________________________________________________________________
Kendall WLord R2
Olejnikrequivalent
ralerting
rcontrast
reffect size
Tatsuoka Wherry R2
_______________________________________________________________
Rosenthal & Rubin BESDRosenthal & Rubin
EScounter null
Wilcoxon
16
![Page 17: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/17.jpg)
(1)
(2)
Two Ways to Estimate the Denominator of Cohen’s
17
![Page 18: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/18.jpg)
Effect of the Unreliability of the Dependent Variable, Y, On the
Proportion of Explained Variance
18
![Page 19: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/19.jpg)
Aspirin group pA = .01259
Placebo group pP = .02166
pA – pP = .01259 – .02166 = –.009
Double-Blind Study of 22,071 Men Physicians
19
![Page 20: Using and Reporting Measures of Effect Size](https://reader035.vdocuments.us/reader035/viewer/2022062310/56816176550346895dd10283/html5/thumbnails/20.jpg)
THE END
20