Analyzing Data from Small N Designs using Multilevel Models

Eden NaglerThe Graduate Center, CUNY

David Rindskopf, Ph.DThe Graduate Center, CUNY

2

Overview/Intro

What is our current work?

Where did we start?

How does HLM fit into this framework?

3

2 Initial Datasets:

Stuart, R.B. (1967). Behavioral control of overeating. Behavior Research & Therapy, 5, (357-365).

Dicarlo, C.F. & Reid, D.H. (2004). Increasing pretend toy play of toddlers with disabilities in an inclusive setting. Journal of Applied Behavior Analysis, 37(2), (197-207).

4

Stuart (1967):

5

Stuart (1967): Procedures for Getting data into HLM

6

Stuart (1967): Procedures for Getting data into HLM

7

Stuart (1967): Level-1 dataset

8

Stuart (1967): Level-2 dataset

9

Stuart (1967): HLM (Linear model)

Linear Model:POUNDS = π0 + π1*(MONTHS12) + e

10

Stuart (1967): HLM – Linear Model Estimates

Final estimation of fixed effects:Standard Approx.

Fixed Effect Coefficient Error T-ratio d.f. P-value

----------------------------------------------------------For INTRCPT1,P0 INTRCPT2, B00 156.439560 5.053645 30.956 7 0.000For MONTHS12 slope, P1 INTRCPT2, B10 -3.078984 0.233772 13.171 7 0.000----------------------------------------------------------The outcome variable is POUNDS----------------------------------------------------------

POUNDSij ≈ 156.4 – 3.1*(MONTHS12) + eij

11

Stuart (1967): HLM – Quadratic Model

Quadratic Model:POUNDS = π0+ π1*(MONTHS12)+ π2*(MON12SQ)+e

12

Stuart (1967): HLM – Quadratic Model Estimates

Final estimation of fixed effects:Standard Approx.

Fixed Effect Coefficient Error T-ratio d.f. P-value-----------------------------------------------------------For INTRCPT1, P0 INTRCPT2, B00 158.833791 5.321806 29.846 7 0.000For MONTHS12 slope, P1 INTRCPT2, B10 -1.773039 0.358651 -4.944 7 0.001For MON12SQ slope, P2 INTRCPT2, B20 0.108829 0.021467 5.070 7 0.001-----------------------------------------------------------The outcome variable is POUNDS-----------------------------------------------------------

POUNDSij ≈ 158.8 – 1.8(MONTHS12) + 0.1*(MON12SQ) + eij

13

Stuart (1967): HLM – Linear vs. Quadratic Model

134.8

157.7

180.5

203.3

226.2

POUNDS

-12.60 -9.30 -6.00 -2.70 0.60

MONTHS12

-12.00 -9.00 -6.00 -3.00 0137.8

157.2

176.7

196.2

215.7

MONTHS12

POUNDS

-12.00 -9.00 -6.00 -3.00 0138.7

158.8

179.0

199.1

219.2

MONTHS12

POUNDS

Stuart (1967) – Actual Data

Quadratic Model Prediction

Linear Model Prediction

14

Dicarlo & Reid (2004):

15

Dicarlo & Reid (2004): Level-1 dataset

16

Dicarlo & Reid (2004): Level-2 dataset

17

Dicarlo & Reid (2004): HLM – Simple Model

Simple Model:

FREQRND = π 0 + π1*(PHASE) + e

18

Dicarlo & Reid (2004): HLM – Simple Model Estimates

Level-1 Model Level-2 Modellog[L] = P0 + P1*(PHASE) P0 = B00 + R0

P1 = B10 + R1----------------------------------------------------------Final estimation of fixed effects: (Unit-specific model)

Standard Approx.

Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------- For INTRCPT1,P0 INTRCPT2, B00 -0.769384 0.634548 -1.212 4 0.292 For PHASE slope,P1 INTRCPT2, B10 2.516446 0.278095 9.049 4 0.000 ----------------------------------------------------------

LN(FREQRNDij) = -0.77 + 2.52*(PHASE) + eij

19

Dicarlo & Reid (2004): HLM – Simple Model Estimates

LOG(FREQRNDij) = B00 + B10*(PHASE) + eij

For PHASE=0 (BASELINE):LOG(FREQRNDij) = B00

FREQRNDij= exp(B00)

For PHASE=1 (TREATMENT):LOG(FREQRNDij) = B00 + B10

FREQRNDij= exp(B00+B10) = exp(B00)*exp(B10)

Estimates: B00 = -0.77; B10 = 2.52For PHASE=0 (BASELINE):FREQRNDij= exp(B00) = exp(-0.77) = 0.46

For PHASE=1 (TREATMENT):FREQRNDij= exp(B00+B10) = exp(-0.77+2.52) = exp(1.75)

= 5.75

20

In conclusion…

1. Other issues we’ve encountered and explored

2. Issues we’ve encountered, but not yet explored

3. Issues we’ve not yet encountered nor explored