Effect Size Tutorial:Cohen’s d and Omega Squared

Jason R. FinleyMon April 1st, 02013

http://www.jasonfinley.com/tools

ω2DEAL WITH IT

Effect Sizes to use

• Comparison of means (t test):– Cohen’s d

• Calculate using Pooled SD (I’ll demonstrate)

• Correlation: – r is its own effect size! (or r2, whatever)

• Regression:– R2, R2

change, R2adjusted

• ANOVA:– Eta squared η2 – Omega squared ω2

StandardizedDifference

Proportion ofVarianceExplained

“Strength ofAssociation” (Hays)

Effect size for comparing two groups: Cohen’s d

• Between-Ss or within-Ss t-test

“Effect sizes for comparisons of means are reported as Cohen’s d calculated

using the pooled standard deviation of the groups being compared (Olejnik &

Algina, 2000, Box 1 Option B).”

• Use pooled SD, and say that’s what you did!

Effective range: -3 to 3

Note this is not the raw variance of the sample, but rather the variance

adjusted to be an unbiased estimator of the population variance. That is. It’s

based on using N-1, instead of N.

Condition A Condition B0.5 0.5

0.25 0.50.75 10.5 0.250 0.5

0.25 0.50 00 0.5

mean 0.28 0.47Variance

(adjusted) 0.07 0.07df 7 7

=AVERAGE(D2:D9)

=VAR(D2:D9)

=COUNT(D2:D9)-1

Then just plug the values into a formula in Excel

Effect Sizes for ANOVA: η2 vs. ω2

• Eta squared η2

– Proportion of variance in DV accounted for by IV(s)– Partial eta squared η2

partial

• For designs with 2+ IVs• Prop. var. accounted for by one particular IV

– Range: 0-1– Problems:• η2 is descriptive of the SAMPLE data• Biased: overestimates population effect size

– Especially when sample size is small

Equivalent to R2 in regression!

Effect Sizes for ANOVA: η2 vs. ω2

• Omega squared ω2

– INFERENTIAL: estimates population effect size• Prop. var. in DV accounted for by IV

– Way less biased than η2 (will be smaller)– Partial omega squared– Issues:

• Not reported by SPSS• Can turn out negative (set to 0 if this happens)• Formula slightly different for different designs• Put a hat on it (ESTIMATED)

small: .01med: .06large: .14

1-way between-subjects ANOVA• Overall effect size (we’ll get to partial in a minute)• All values needed are obtained from ANOVA table

=

SPSS output for1-way between-Ss ANOVA

effecterror

HINT: paste the SPSS output into Excel!... Make a template!

1-way within-subjects ANOVA

SPSS output for1-way between-Ss ANOVA

Test for violation of sphericity is not sig., so we can use the “Sphericity Assumed” rows in the tables to follow.

SPSS output for1-way between-Ss ANOVAeffect

effect x subject

subject

Partial Omega Squared

• When 2+ IVs– Prop. var. in DV accounted for by one particular IV,

partialing out variance accounted for by the other IVs.

or

2-way Between-Ss ANOVA:with IVs “A” and “B”

For IV “A”:

Regular Partial

Ntotal = total # subjects in experiment

SPSS output for 2-way between-Ss ANOVAIV A: Feedback ConditionIV B: Practice Condition

Partial

SPSS output for 2-way between-Ss ANOVAIV A: Feedback ConditionIV B: Practice Condition

Regular

2-way mixed ANOVA(IV “A” between-Ss, IV “B” within-Ss)

Pro tip: the AB interaction counts as a within-Ss effect

Effect A

Effect B

Interaction AB

Error A: “subject/A”

Error B, AB:“Bxsubject/A”

For interaction AB:

REMEMBER

• In the first paragraph of your Results section (just Exp. 1 if multiple exps), clearly state the effect sizes you’ll be reporting.

• “Effect sizes for comparisons of means are reported as Cohen’s d calculated using the pooled standard deviation of the groups being compared (Olejnik & Algina, 2000, Box 1 Option B).”

• “Effect sizes for ANOVAs are reported as partial omega squared calculated using the formulae provided by Maxwell and Delaney (2004).”

On the horizon

• Confidence intervals for effect size estimates