multilevel modeling

Multilevel Modeling1. Overview

2. Application #1: Growth Modeling

Break

3. Application # 2: Individuals Nested Within Groups

4. Questions?

Overview1. What is multilevel modeling?2. Examples of multilevel data structures3. Brief history4. Current applications5. Why multilevel modeling?6. What types of studies use multilevel

modeling?7. Computer Programs (HLM 6

SAS Mixed8. Resources

Multilevel Question What effects do the following

variables have on 3rd grade reading achievement?

School SizeClassroom Climate

Student Gender

What is Multilevel or Hierarchical Linear Modeling?

Nested Data Structures

Several Types of Nesting

1. Individuals Nested Within Groups

Individuals Undivided

Unit of Analysis = Individuals

Individuals Nested Within Groups

Unit of Analysis = Individuals + Classes

… and Further Nested

Unit of Analysis = Individuals + Classes + Schools

Examples of Multilevel Data Structures Neighborhoods are nested within

communities

Families are nested within neighborhoods

Children are nested within families

Examples of Multilevel Data Structures Schools are nested within districts

Classes are nested within schools

Students are nested within classes

Multilevel Data Structures

Level 4 District (l)

Level 3 School (k)

Level 2 Class (j)

Level 1 Student (i)

2nd Type of Nesting

Repeated Measures Nested Within Individuals

Focus = Change or Growth

Time Points Nested Within Individuals

Repeated Measures Nested Within Individuals CarlosDay Energy LevelMonday = 0 98Tuesday = 1 90Wednes. = 2 85Thursday = 3 72Friday= 4 70

Repeated Measures Nested Within Individuals

DAY

543210

EN

ER

GY

100

90

80

70

60

Repeated Measures Nested Within Individuals

DAY

543210

EN

ER

GY

100

90

80

70

60 Rsq = 0.9641

Changes for 5 Individuals

0 1.00 2.00 3.00 4.000

25.00

50.00

75.00

100.00

Time

Ener

gy L

evel

Changes in Energy Level Over the Week

3rd Type of Nesting (similar to the 2nd) Repeated Measures Nested Within

Individuals

Focus is not on change

Focus in on relationships between variables within an individual

Repeated Measures Nested Within Individuals

CarlosDay Hours of Sleep Energy LevelMonday 9 98Tuesday 8 90Wednesday 8 85Thursday 6 72Friday7 70

Repeated Measures Nested Within Individuals (Not Change)

HOURS

9.59.08.58.07.57.06.56.05.5

EN

ER

GY

100

90

80

70

60

Repeated Measures Nested Within Individuals (Not Change)

HOURS

9.59.08.58.07.57.06.56.05.5

EN

ER

GY

100

90

80

70

60

Repeated Measures Nested Within Individuals

2.00 4.50 7.00 9.50 12.000

25.00

50.00

75.00

100.00

Hours of Sleep

Ener

gy L

evel

Repeated Measures Nested Within Individuals (3 Individuals)

Repeated Measures Within Persons

Level 2 Student (i)

Level 1 Repeated Measures Over Time (t)

Nested Data

Data nested within a group tend to be more alike than data from individuals selected at random.

Nature of group dynamics will tend to exert an effect on individuals.

Nested Data Intraclass correlation (ICC)

provides a measure of the clustering and dependence of the data

0 (very independent) to 1.0 (very dependent)

Details discussed later

Brief Historyof Multilevel Modeling

Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. Sociological Review, 15, 351-357.

Burstein, Leigh (1976). The use of data from groups for inferences about individuals in educational research. Doctoral Dissertation, Stanford University.

Table 1Frequency of HLM application evidenced in Scholarly Journals

Journal 1999 2000 2001 2002 2003 Total by journal

American Educational Research Journal 3 5 4 3 ? ~15

Child Development 3 2 6 5 13 29

Cognition and Instruction 1 0 0 0 0 1

Contemporary Educational Psychology 0 0 0 0 0 0

Developmental Psychology 2 1 2 5 7 17

Educational Evaluation and Policy Analysis 2 1 5 2 2 12

Educational Technology, Research and Development 0 0 0 0 0 0

Journal of Applied Psychology 1 1 5 7 6 20

Journal of Counseling Psychology 0 2 1 0 0 3

Journal of Educational Computing Research 0 0 0 0 0 0

Journal of Educational Psychology 1 2 3 6 1 13

Journal of Educational Research 2 0 3 3 5 13

Journal of Experimental Child Psychology 0 0 0 0 0 0

Journal of Experimental Education 0 0 0 0 1 1

Journal of Personality and Social Psychology 4 4 6 5 13 32

Journal of Reading Behavior/Literacy Research 0 0 0 0 0 0

Journal of Research in Mathematics Education 0 0 0 0 0 0

Reading Research Quarterly 0 0 0 1 0 1

Sociology of Education 1 2 5 2 1 11

Total by Year 20 20 40 39 49 ~168

Multilevel ArticlesFrequency of Studies Employing HLM in Education or Related Journals

0

25

50

1999 2000 2001 2002 2003

Year

Freq

uenc

y

Total for 19 Journals Reviewed

Journal of Personality and Social Psychology

Child Development

Journal of Educational Research

Some Current Applications of Multilevel Modeling

Growth Curve Analysis Value Added Modeling of

Teacher and School Effects Meta-Analysis

Multilevel Modeling Seems New But…. Extension of General Linear Modeling

Simple Linear RegressionMultiple Linear Regression

ANOVAANCOVA

Repeated Measures ANOVA

Multilevel Modeling Our focus will be on observed

variables (not Latent Variables as in Structural Equation Modeling)

Why Multilevel Modelingvs. Traditional Approaches?

Traditional Approaches – 1-Level

1. Individual level analysis (ignore group)

2. Group level analysis (aggregate data and ignore individuals)

Problems withTraditional Approaches

1. Individual level analysis (ignore group)

Violation of independence of data assumption leading to misestimated standard errors (standard errors are smaller than they should be).

Problems withTraditional Approaches

1. Group level analysis (aggregate data and ignore individuals)

Aggregation bias = the meaning of a variable at Level-1 (e.g., individual level SES) may not be the same as the meaning at Level-2 (e.g., school level SES)

Multilevel Approach

2 or more levels can be considered simultaneously

Can analyze within- and between-group variability

What Types of Studies Use Multilevel Modeling?

Quantitative

Experimental *Nonexperimental

(Survey, Observational)

How Many Levels Are Usually Examined?

2 or 3 levels very common

15 students x 10 classes x 10 schools

= 1,500

Types of Outcomes

Continuous Scale (Achievement, Attitudes)

Binary (pass/fail) Categorical with 3 + categories

Software to do Multilevel Modeling

SPSS Users2 SAV Files: Level 1

Level 2

HLM 6 (Menu Driven) (Raudenbush, Bryk, Cheong, &

Congdon, 2004)

HLM 6

Software to do Multilevel Modeling

SAS Users

Proc Mixed

Resources (Sample…see handouts for more complete list)

Books Hierarchical Linear Models: Applications and

Data Analysis Methods, 2nd ed. Raudenbush & Bryk, 2002.

Introducing Multilevel Modeling. Kreft & DeLeeum, 1998.

Journals Educational and Psychological Measurement Journal of Educational and Behavioral Sciences Multilevel Modeling Newsletter

Resources (cont)(Sample…see handouts for more complete list)

Software HLM6 SAS (NLMIXED and PROC MIXED) MLwiN

Journal Articles See Handouts for various

methodological and applied articles Data Sets

NAEP Data NELS:88; High School and Beyond

Self-Check 1 A teacher with 1 classroom of 24

students used weekly curriculum-based measurements to monitor reading over a 14 week period. The teacher was interested in individual students’ rates of change and differences in change by male and female students.

Self-Check 1 How would you classify this

situation?

(a) not multilevel(b) 2-level(c) 3-level

Self-Check 2 A researcher randomly selected

50 elementary schools and randomly selected 30 teachers within each school. The researcher was interested in the relationships between 2 predictors (school size and teachers’ years experience at their current school) and teachers’ job satisfaction.

Self-Check 2 How would you classify this

situation?

(a) not multilevel(b) 2-level(c) 3-level

Self-Check 3 60 undergraduates from the research

participant pool volunteered for a study that used written vignettes to manipulate the interactional style (warm, not warm) of a professor interacting with a student.  30 randomly assigned students read the vignette depicting warmth and 30 randomly assigned students read the vignette depicting a lack of warmth.  After reading the vignette students used a questionnaire to rate the likeability of the professor.

Self-Check 3 How would you classify this

situation?

(a) not multilevel(b) 2-level(c) 3-level

(Select ONLY one)

Growth Curve Modeling

Studying the growth in reading achievement over a two year period

Studying changes in student attitudes over the middle school years

Research Questions What is the form of change for

an individual during the study?

Research Questions What is an individual’s initial

status on the outcome of interest?

Run

Research Questions How much does an individual

change during the course of the study?

Rise RisebRun

Research Questions What is the average initial

status of the participants?

Research Questions What is the average change of

the participants?

Research Questions To what extent do participants

vary in their initial status?

Research Questions To what extent do participants

vary in their growth?

Research Questions To what extent does initial

status relate to growth?

Research Questions To what extent is initial status

related to predictors of interest?

Research Questions To what extent is growth related

to predictors of interest?

Design Issues How many waves a data

collection are needed? >2 Depends on complexity of growth

curve

Design Issues Can there be different numbers

of observations for different participants?

Examples Missing data Planned missingness

Design Issues Can the time between

observations vary from participant to participant?

Example: Students observed 1, 3, 5, & 7 months 1, 2, 4, & 8 months 2, 4, 6, & 8 months

Design Issues How many participants are

needed?

More is better Power analyses > 30 rule of thumb

Design Issues How should participants be

sampled?

What you have learned about sampling still applies

Design Issues What is the value of random

assignment?

What you have leaned about random assignment still applies

Design Issues How should the outcome be

measured?

What you have learned about measurement still applies

Example Context description

A researcher was interested in changes in verbal fluency of 4th grade students, and differences in the changes between boys and girls.

ID    Gender         Time______                              t0    t4    t7

1    0  20    30    302     0          40    44    493 0          45    40    604     0         50    55    595     0          42    48    536 1          45    52    617 1          39    55    638 1          46    58    689 1          44    49    59

Example Level-1 model specification

0 1 1*( )fluencyY Time error

Example Level-2 model specification

0 00 01 2

1 10 11

*( )*( )

G G Gender errorG G Gender

Example Combined Model

00 01 10

11 2 1

*( ) *( )*( ) *( )

fluencyY G G Gender G TimeG Gender Time error error

Example SAS program

proc mixed covtest; class gender; model score = time gender time*gender/s; random intercept / sub=student s;

Example SAS output – variance estimates

Covariance Parameter Estimates  Standard ZCov Parm Subject Estimate Error Value Pr Z Intercept Student 62.5125 35.9682 1.74 0.0411Residual 14.1173 4.9912 2.83 0.0023

Example SAS output – fixed effects

Solution for Fixed Effects  StandardEffect Gender Estimate Error DF t Value Pr > |t| Intercept 39.8103 3.7975 7 10.48 <.0001time 1.5077 0.3295 16 4.58 0.0003Gender F 5.7090 5.6962 16 1.00 0.3311Gender M 0 . . . .time*Gender F 1.0692 0.4943 16 2.16 0.0460time*Gender M 0 . . . .

Example Graph – fixed effects

0

25.00

50.00

75.00

100.00

SCO

RE

0 2.50 5.00 7.50 10.00

TIME

GENDER = 0GENDER = 1

Example Conclusions

Fourth grade girl’s verbal fluency is increasing at a faster rate than boy ’s.

Persons Nested in Contexts

Studying attitudes of teachers who are nested in schools

Studying achievement for students who are nested in classrooms that are nested in schools

Research Questions How much variation occurs within

and among groups?

To what extent do teacher attitudes vary within schools?

To what extent does the average teacher attitude vary among schools?

Research Questions What is the relationship among selected

within group factors and an outcome?

To what extent do teacher attitudes vary within schools as function of years experience?

To what extent does student achievement vary within schools as a function of SES?

Research Questions What is the relationship among

selected between group factors and an outcome?

To what extent do teacher attitudes vary across schools as function of principal leadership style?

To what extent does student math achievement vary across schools as a function of the school adopted curriculum?

Research Questions To what extent is the relationship

among selected within group factors and an outcome moderated by a between group factor?

To what extent does the within schools relationship between student achievement and SES depend on the school adopted curriculum?

Design Issues

Consider a design where students are nested in schools

How should schools should be sampled?

How should students be sampled within schools?

Design Issues

Consider a design where students are nested in schools

How many schools should be sampled?

How many students should be sampled per school?

Design Issues

What kind of outcomes can be considered?

Continuous Binary Count Ordinal

Design Issues How will level-1 variables be

conceptualized and measured?

SES

How will level-2 variables be conceptualized and measured?

SES

Terminology Individual growth trajectory – individual growth

curve model A model describing the change process for

an individual Intercept

Predicted value of an individual’s status at some fixed point

The intercept cold represent the status at the beginning of a study

Slope The average amount of change in the

outcome for every 1 unit change in time

I ntercept & Slope I llustration

0

5

10

15

20

25

0 1 2 3 4 5 6 7 8 9 10Time

Scor

e

RiseRisebRun

Run

intercept

Curvature =Acceleration=Quadratic Component

0

5

10

15

20

25

30

35

0 1 2 3 4

Time

Scor

e

HLM Hierarchical Linear Model

The hierarchical or nested structure of the data

For growth curve models, the repeated measures are nested within each individual

Levels in Multilevel Models Level 1 = time-series data

nested within an individual

0 1 *( )Y Time error

Levels in Multilevel Models Level 2 = model that attempts

to explain the variation in the level 1 parameters

0 00 01

1 10 11

*( )*( )

G G Sessions errorG G Sessions error

More terminology Fixed coefficient

A regression coefficient that does not vary across individuals

Random coefficient A regression coefficient that does

vary across individuals

More terminology Balanced design

Equal number of observations per unit Unbalanced design

Unequal number of observation per unit Unconditional model

Simplest level 2 model; no predictors of the level 1 parameters (e.g., intercept and slope)

Conditional model Level 2 model contains predictors of level 1

parameters

Estimation Methods Empirical Bayes (EB) estimate

“optimal composite of an estimate based on the data from that individual and an estimate based on data from other similar individuals” (Bryk, Raudenbush, & Condon, 1994, p.4)

Estimation Methods Expectation-maximization (EM)

algorithm An iterative numerical algorithm

for producing maximum likelihood estimates of variance covariance components for unbalanced data.