hierarchical linear modeling: understanding applications ...€¦ · 16/05/2016 · introduce...

Hierarchical Linear Modeling:

Understanding Applications in the MSP

Projects

NSF # DRL1238120

The work of TEAMS is supported with funding provided by

the National Science Foundation, Award Number DRL

1238120. Any opinions, suggestions, and conclusions or

recommendations expressed in this presentation are those

of the presenter and do not necessarily reflect the views of

the National Science Foundation; NSF has not approved or

endorsed its content.

2

Strengthening the quality of the MSP project evaluation

and building the capacity of the evaluators by

strengthening their skills related to evaluation design,

methodology, analysis, and reporting.

3

Website at http://teams.mspnet.org

Online Help-Desk for submitting requests

Assistance with instruments

Consultation and targeted TA

Webinar series on specific evaluation topics

White papers/focused topic papers

4

Hierarchical Linear Modeling: Understanding

Applications in the MSP Projects

Presenters:

Karen Drill, RMC Research Corporation

Emma Espel, RMC Research Corporation

Moderator:

John Sutton, RMC Research Corporation,

TEAMS Project PI

5

6

Introduce Hierarchical Linear Modeling (HLM) principles and techniques

Discuss appropriate use of HLM within MSP projects

Provide concrete examples of the use of HLM within MSP projects

Goals:

NSF # DRL1238120

7

What is HLM?

When to use HLM

Example HLM Use

Pro Tips: What (not) to do

Webinar Sections

Tools & Resources

8

What is HLM?

When to use HLM

Example HLM Use


Webinar Sections

Tools & Resources

9

What is HLM?

How familiar are you with HLM?

A complex form of

ordinary least squares

regression

Can be used to analyze

variance in outcome

variables when predictor

variables are at different

hierarchical levels

10

What is HLM?

Linear regression attempts

to model the relationship

between two variables by

fitting a linear equation to

observed data.

11

Review of Linear Regression

Math interest

Ma

th a

ch

ieve

me

nt

Y’= 𝐵0+ 𝐵𝑌𝑋𝑋 + 𝜀

Y’ = The predicted value

𝐵0 = Y-intercept—the value of Y’ when X = 0

𝐵𝑌𝑋 = Slope—the regression coefficient for

predicting Y

X = Independent variable or predictor

𝜀 = Error

12


0%

50%

100%

0 1 2 3 4 5

Y’= 0.002 + 0.180 𝑋 + .210

Student interest in math

Pe

rce

nt

co

rre

ct

on

ma

th c

on

ten

t e

xam





predicting Y


𝜀 = Error

13


0%

50%

100%

0 1 2 3 4 5

Y’= 0.002 + 0.180 𝑋 + .210


Pe

rce

nt

co

rre

ct

on

ma

th c

on

ten

t e

xam





predicting Y


𝜀 = Error

14


0%

50%

100%

0 1 2 3 4 5

Y’= 0.002 + 0.180 𝑋 + .210


Pe

rce

nt

co

rre

ct

on

ma

th c

on

ten

t e

xam





predicting Y


𝜀 = Error

15


0%

50%

100%

0 1 2 3 4 5

Y’= 0.002 + 0.180 𝑋 + .210


Pe

rce

nt

co

rre

ct

on

ma

th c

on

ten

t e

xam





predicting Y


𝜀 = Error

16


0%

50%

100%

0 1 2 3 4 5

Y’= 0.002 + 0.180 𝑋 + .210


Pe

rce

nt

co

rre

ct

on

ma

th c

on

ten

t e

xam

Based on linear regression

17

HLM Similarities to Linear Regression

Models the relationship between the

observed to the expected

18


Can be cross-sectional or longitudinal

19


20

Differences from Linear Regression

Level-3 (school)

Level-2 (teacher)

Level-1 (students)

Green = Level 1

Orange = Level 2

21


Level-3 (school)

Level-2 (teacher)

Level-1 (students)

Intraclass Correlation (ICC)

Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5

22


HLM: Multiple Levels Ecological Fallacy: One Level

Green = Level 1

Orange = Level 2

23

Questions?

24

MSP Scenario and HLM Equations

25

MSP Scenario and HLM Equations

You are the evaluator of an MSP that is implementing an innovative

math curriculum for 6th graders.

You are interested in whether implementing this curriculum influences

students’ math achievement scores.

Your sample also includes a matched comparison group of teachers

not implementing the curriculum.

To what extent does teacher implementation of the math curriculum

influence students’ math achievement scores?

Variables

Y = Students’ achievement scores (level-1 outcome)

X = Female (level-1 predictor)

W = Treatment (the math curriculum) (level-2 predictor)

26

To what extent does teacher implementation of the math

curriculum influence students’ math achievement scores?

(level-1)𝑌𝑖𝑗= 𝛽0𝑗 + 𝛽1𝑗 (𝐹𝑒𝑚𝑎𝑙𝑒𝑖𝑗)* + 𝑟𝑖𝑗

𝑌𝑖𝑗= dependent variable measured for 𝑖th level-1 (student) unit

nested within the 𝑗th level-2 (teacher) unit

𝑌𝑖𝑗 = students’ math achievement score

27



*dummy coded

(level-1)𝑌𝑖𝑗= 𝛽0𝑗 + 𝛽1𝑗 (𝐹𝑒𝑚𝑎𝑙𝑒𝑖𝑗) + 𝑟𝑖𝑗

𝛽0𝑗 = intercept for the 𝑗th level-2 (teacher) unit

𝛽0𝑗 = best estimate for predicting

math achievement for males

28




𝛽1𝑗 = regression coefficient associated

with 𝑋𝑖𝑗 for the 𝑗th level-2 (teacher) unit

𝛽1𝑗 = level-1 slope

𝛽1𝑗 = the effect of being female on math achievement

29




(𝐹𝑒𝑚𝑎𝑙𝑒)𝑖𝑗 = value on the level-1 (student) predictor

(𝐹𝑒𝑚𝑎𝑙𝑒𝑖𝑗) = value for female (0 = not female, 1 = female)*

*dummy coded

30


curriculum influence students’ math content achievement scores?


𝑟𝑖𝑗 = random error associated with the 𝑖th level-1 unit (student)

nested within the 𝑗th level-2 (teacher) unit

𝑟𝑖𝑗 = deviation for each student from the fitted model

31


curriculum influence students’ math content knowledge?

(level-1) 𝑌𝑖𝑗= 𝛽0𝑗 + 𝛽1𝑗 (𝐹𝑒𝑚𝑎𝑙𝑒𝑖𝑗) + 𝑟𝑖𝑗

(level -2) 𝛽0𝑗= 𝛾00 + 𝛾01 (𝑇𝑥)1𝑗 + 𝑢0𝑗

𝛽0𝑗 = intercept for the 𝑗th level-2 unit

32




(level -2) 𝛽0𝑗= 𝛾00 + 𝛾01 (𝑇𝑥)1𝑗 + 𝑢0𝑗

Υ00 = level-2 intercept

Υ00 = mean math achievement

for comparison schools

33




(level -2) 𝛽0𝑗= 𝛾00 + 𝛾01 (𝑇𝑥)1𝑗 + 𝑢0𝑗

Υ01 = level-2 slope for treatment

34




(level -2) 𝛽0𝑗= 𝛾00 + 𝛾01 (𝑇𝑥)1𝑗 + 𝑢0𝑗

(𝑇𝑥)1𝑗 = value on the level-2 predictor

(𝑇𝑥)1𝑗= value for treatment (0 = no treatment, 1 = treatment)*

*dummy coded

35




(level -2) 𝛽0𝑗= 𝛾00 + 𝛾01 (𝑇𝑥)1𝑗 + 𝑢0𝑗

𝑢0𝑗 = random effects of the 𝑗th level-2 unit

adjusted for treatment on the intercept

𝑢0𝑗 = unique effect for each school on mean math achievement

36



37

HLM Challenges

x

Insufficient power at level -1 or level-2

38

HLM Challenges: Power

Measures need strong psychometric properties

39

HLM challenges: Meeting model assumptions

Level-1 residuals need to be independent and

normally distributed

40

HLM challenges: Meeting model assumptions

41

Questions?

42

What is HLM?

When to use HLM

Example HLM Use


Webinar Sections

Tools & Resources

Does the data have a nested

structure?

Is there sufficient

power at the highest level?

Does your ICC reach an

acceptable level?

Are assumptions

met?

Is there sufficient

power at the lowest level?

HLM is likely a good choice

Consider HLM with

reservations

Consider regression or another more appropriate

design.

NO

NO

YES

YES

YES

NO

When to use HLM

YES

NO


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

NO

NO

NO

YES

YES

YES

NO

When to use HLM

YES

NO


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

YES

NO

When to use HLM

YES

NO


structure?

Is there sufficient


Is there an adequate ICC to warrant multi-

level modelling?

Are assumptions met?

Is there sufficient power at the lowest level?


Consider HLM with reservations


design.

YES NO

NO

NO

NO

YES

YES

YES

YES

When to use HLM

Are you familiar with Power Analysis? Are you familiar with Optimal Design?


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

YES

NO

When to use HLM

YES

NO


structure?

Is there sufficient


Is there an adequate ICC to warrant multi-

level modelling?


Is there sufficient power at the lowest level?




design.

YES NO

NO

NO

YES

YES

NO

Bonus! WWC Recommends clustering adjustment for single-level analyses with multiple levels for significant findings.

1. Compute test statistic for effect size 2. Adjust test statistic and degrees of freedom for effect size 3. Identify significance value

Handy Resource: http://www.air.org/resource/wwc-phase-i-computation-tools-4-15-10

When to use HLM


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

YES

NO

When to use HLM

NO


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

YES

NO

When to use HLM

NO

YES


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

NO

When to use HLM

NO

YES


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

NO

When to use HLM

NO

YES

YES

To what extent does teacher participation in the MSP contribute to student

science content knowledge?

Is HLM appropriate?

57

When to use HLM

Scenario

You are the evaluator of an MSP designed to train teams of teachers in

science content knowledge for 8th graders.

Why or Why Not?

Major activities include an intensive summer institute, learning teams of

involved teachers, teacher leaders, and research activities.

Teachers randomly assigned to training or not (cluster randomized trial)

N teachers = 17 Tx, 42 Control

N students = 2,025


structure?

Is there sufficient



acceptable level?

Are assumptions

met?

Is there sufficient



Consider HLM with

reservations


design.

YES NO

NO

NO

YES

YES

YES

NO

When to use HLM

YES

NO

60

When to use HLM: Power

Raudenbush, S. W., et al. (2011). Optimal Design Software for Multi-level and

Longitudinal Research (Version 3.01) [Software]. Available from

www.wtgrantfoundation.org.

http://sitemaker.umich.edu/group-based/optimal_design_software

Does the data have a nested structure?

Is there sufficient power at the highest level?

What is the ICC?


Is there sufficient power at the lowest

level?



Consider regression or another more

appropriate design.

YES NO

NO

NO

YES

YES

YES

NO

When to use HLM

Keep in mind as you move forward with analysis planning.

Decision

OLS Regression was used to analyze the data

due to insufficient power to detect an effect of

the program.

63

When to use HLM (or not)

Do you agree? Why or Why not?

64

Questions?

65

What is HLM?

When to use HLM

Example HLM Use


Webinar Sections

Tools & Resources

66

When to use HLM

Scenario: Reading HLM Reports

You are the evaluator of an MSP designed to train teams of teachers in

science content knowledge for 8th graders.

Major activities include an intensive summer institute, learning teams of

involved teachers, teacher leaders, and research activities.

N teachers = 148 Tx, 150 Control*

Teachers randomly assigned to training or not (cluster randomized trial)

N students = 2,358**

To what extent does teacher participation in the MSP contribute to student

science content knowledge (assuming all students have scores for the

standardized state science test)?

You are developing the analysis plan for this project.

* Teacher level variables: MSP teacher, MSP Leader

**Student level covariates: Gender, Title I status, Individualized Education Plan (IEP), Hispanic, English

Language Learner, prior Normal Curve Equivalent score (NCE)

67

Example HLM Use

Model 1 Model 2 Model 3 Model 4

Est. SE Est. SE Est. SE Est. SE

Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235

Normal Curve Equivalent

(NCE) pretest

1.244 0.124 1.168*** 0.112

MSP Teacher 2.522* 0.897

MSP Leader 1.045 1.349

HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3

Intraclass Correlation (ICC) .077

Exhibit X. Summary of Regression Analyses of the Effects of MSP Teacher Participation on Student Science Achievement in Grades 4-8 (N = 2,358)

*p < .05, **p < .01, ***p < .001

The first model is always a null

model.

68

Example HLM Use



Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235


(NCE) pretest

1.244 0.124 1.168*** 0.112



HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3



*p < .05, **p < .01, ***p < .001

On average, participants had a

NCE score of 52.004, with a

standard error of 1.660.

69

Example HLM Use



Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235


(NCE) pretest

1.244 0.124 1.168*** 0.112



HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3



*p < .05, **p < .01, ***p < .001

Deviance indicates model fit,

and lower deviance indicates

better fit.

70

Example HLM Use



Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235


(NCE) pretest

1.244 0.124 1.168*** 0.112



HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3



*p < .05, **p < .01, ***p < .001

7.7% of the variance in science

achievement was due to variation

between teachers.

71

Example HLM Use



Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235


(NCE) pretest

1.244 0.124 1.168*** 0.112



HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3


*p < .05, **p < .01, ***p < .001

Exhibit X. Summary of Regression Analyses of the Effects of MSP Teacher Participation on Student Science Achievement in Grades 4-8 (N = 2,358) Model 2 typically adds

Level 1 predictors

72

Example HLM Use



Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235


(NCE) pretest 1.244 0.124 1.168*** 0.112



HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3


*p < .05, **p < .01, ***p < .001


Model 3

typically

adds

predictors of

interest

73

Example HLM Use



Intercept 52.004 1.660 52.025 1.664 51.200 0.483 52.214 0.560

Gender 1.323 0.731 1.323 0.731 1.283 0.731

Title I -0.956 1.972 -0.956 1.973 -0.904 2.010

IEP -17.525*** 1.250 -17.524*** 1.251 -17.505*** 1.250

Hispanic -10.348*** 1.250 -10.348*** 1.250 -10.287*** 1.250

ELL -6.659*** 1.236 -6.659*** 1.236 -6.686*** 1.235


(NCE) pretest 1.244 0.124 1.168*** 0.112



HLM Deviance 15,939.2 15,362.9 15,341.6 15,330.3


*p < .05, **p < .01, ***p < .001


Deviance decreased from Model 1 to Model 4.

74

Example HLM Use

Model 4

Est. SE

Intercept 52.214 0.560

Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112



HLM Deviance 15,330.3


*p < .05, **p < .01, ***p < .001


Model 4 is the final model.

75

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


On average, students scored 52.214 NCE

units.

78

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


On average, students with an IEP scored

17.505 points lower than those without.

79

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


On average, Hispanic students scored

10.287 points lower than non-Hispanic

students.

80

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


On average, ELL students scored 6.686

points lower than non-ELL students.

81

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


For every NCE unit score on the pre-test,

students gained 1.168 NCE units on the

post-test, on average.

82

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


Students who had an MSP Teacher scored

2.522 points higher than those who did

not.

83

Example HLM Use

Model 4

Est. SE


Gender 1.283 0.731

Title I -0.904 2.010

IEP -17.505*** 1.250

Hispanic -10.287*** 1.250

ELL -6.686*** 1.235


(NCE) pretest 1.168*** 0.112





*p < .05, **p < .01, ***p < .001


There was no difference in scores for

students in classes that were taught by

MSP Leaders compared to those who were

not.

84

Questions?

85

What is HLM?

When to use HLM

Example HLM Use


Webinar Sections

Tools & Resources

86


Make sure all variables are dummy

coded appropriately, with 1/0 for

each category and a reference

group.

Race X1 X2 X3 X4

White 1 0 0 0

Black 0 1 0 0

Hispanic 0 0 1 0

Other 0 0 0 0

87

Think strategically about centering your

variables.

Uncentered: Xij

Group-mean centered: Xij − 𝑿 j

Grand-mean centered: Xij − 𝑿 ··


88

Strategically build your model.

Null Model

Covariates

Level 1 Predictors of Interest

Level 2 Predictors of Interest


89

Make sure to report relevant statistics. According to Abt Associates’

Guide for rigorous MSP evaluations and reporting:


90

What is a challenge you have faced when running HLM or

considering whether or not to use HLM?


91

What is HLM?

When to use HLM

Example HLM Use


Webinar Sections

Tools & Resources

Software Options for HLM Analysis

92

SPSS

Tools and Resources

93

Tools and Resources

Resources

SSI Website to download HLM and find resources:

http://www.ssicentral.com/hlm/resources.html

Schochet, P. Z., Puma, M., & Deke, J. (2014). Understanding variation in treatment effects in

education impact evaluations: An overview of quantitative methods (NCEE 2014–4017).

Washington, DC: U.S. Department of Education, Institute of Education Sciences, National

Center for Education Evaluation and Regional Assistance, Analytic Technical Assistance and

Development. Retrieved from http://ies.ed.gov/ncee/edlabs.

Raudenbush, S. W., et al. (2011). Optimal Design Software for Multi-level and Longitudinal

Research (Version 3.01) [Software]. Available from www.wtgrantfoundation.org or

//sitemaker.umich.edu/group-based/optimal_design_software.

Variance Almanac of Academic Achievement : https://arc.uchicago.edu/reese/variance-

almanac-academic-achievement



http://ies.ed.gov/ncee/edlabs

http://www.wtgrantfoundation.org/





https://arc.uchicago.edu/reese/variance-almanac-academic-achievement









94

Questions?

95

TEAMS Resources & Tools

TEAMS MSP Project Document

Self-Appraisal

Purpose of the Evaluation

Evaluation Design &

Measurement

Analysis

Generalizability,

Representativeness, Utility

http://teams.mspnet.org/

index.cfm/27152

Contact Us!

Karen Drill

[email protected]

Emma Espel

[email protected]

RMC Research Corporation

111 SW Columbia St., Suite 1030

Portland, OR 98201-5883


633 17th St., Suite 2100

Denver, CO 80202-1620

96

John T. Sutton, PI

[email protected]

Dave Weaver, Co-PI

[email protected]


633 17th St., Suite 2100

Denver, CO 80202-1620

Phone: 303-825-3636

Toll Free: 800-922-3636

Fax: 303-825-1626


111 SW Columbia St., Suite 1030

Portland, OR 97201-5883

Phone: 503-223-8248

Toll Free: 800-788-1887

Fax: 503-223-8399

97

Follow Up Survey

Please Complete the Feedback

Survey

URL: http://www.rmccorp.com/TEAMS/

Username: 23569

Password: TEAMS

Thank You!

NSF # DRL 1238120

http://www.rmccorp.com/TEAMS/



hierarchical linear modeling: understanding applications ...€¦ · 16/05/2016 · introduce...

Documents