prospective power analysis for multilevel designs using...

Janeen LoehrDepartment of Psychology, University of Saskatchewan

Introduction

Prospective power analysis for multilevel designs using SIMR

Research questions: • Does joint performance influence joint agency? • Does explicit feedback enhance performance’s

effect?Experiment design: • Participants to be recruited in pairs• Continuous predictor: Pace Error (pair level)• Categorical predictor: Feedback (implicit, explicit)

(between pairs) • Outcome variable: Joint Agency (participant level)Effect size of interest:• Interaction between Pace Error and Feedback

Participants nested within pairs4

SIMR2,3

References:1. Lane & Hennes (2018). Power struggles: Estimating sample size for multilevel relationships research. J Soc Pers Relat, 35, 7–31. 2. Green & Macleod (2016). SIMR: An R package for power analysis of generalized linear mixed models by simulation. Methods Ecol Evol, 7, 493–498. 3. Green, MacLeod, & Alday, P. (2017). Package “simr.” Retrieved from https://cran.rproject.org/web/packages/simr/simr.pdf4. Loehr (2018). Shared credit for shared success: Successful joint performance strengthens the sense of joint agency. Conscious Cogn, 66, 79-90.

• SIMR2,3 estimates power using Monte Carlo simulation • makeLmer builds an artificial fitted mixed-

effects model with specified parameters • extend increases the number of observations

in the artificial fitted model• powerCurve estimates power at different

sample sizes by repeatedly:1. Simulating a new dataset using the fitted

model provided2. Refitting the model to the simulated dataset 3. Applying the statistical test of interest to the

simulated fit. • Power = proportion of successful tests @ Step 3.

Parameter Estimates:• Fixed effects:

• Random effects:

• Pace error: 0-65 ms over 40 trials

Code: Simulated power:

• Prospective power analysis is necessary given problems caused by low statistical power

• Multilevel designs and mixed-effects model analyses are becoming more common

• Challenges for prospective power analysis for multilevel designs:• Accommodate the structure of the multilevel

model of interest• Estimate values for each of the model’s many

parameters1

• Solution: Simulate power using parameter estimates from analyses of existing datasets1

Dataset b Int b PErr1 (like implicit) 33.30 0.122 (like explicit) 30.90 0.18Difference -2.4 0.06

Dataset 1 Dataset 2Param (level) Estimate (% var) Estimate (% var)Residual 289 (43%) 345 (37%)Int (Dyad) 38 (6%) -Int (Part.) 345 (51%) 549 (58%)PErr (Part.) 0.04 (0.006%) 0.11 (0.01%)Int (Trial) 5 (0.6%) 49 (5%)

Trials repeated within participants

Research question: • Does perceptual overlap between action

outcomes influence the sense of joint agency?Experiment design: • Participants to be recruited in pairs• Categorical predictor: Accompaniment

Distance (near,far) (within pairs)• Outcome variable: Joint Agency (participant

level)Effect size of interest:• Effect of Accompaniment Distance on Joint

Agency

Parameter Estimates:• Fixed effects:

• Random effects:

• 36 trials per condition

Parameter bIntercept 63Accomp. Dist. -5.9

Dataset 1 Param (level) Estimate

Residual 325Int (Dyad) --Int (Part.) 183Accomp. Dist. (Part.) 18Int (Trial) --

Code: Simulated power:

Detailed methods

…

Poster + R code

Poster + R code