Assessing Local Item Dependence in Building

Explanation Tasks

Han Bao & Dr. Robert J. MislevyUniversity of Maryland, College Park

Han Bao & Dr. Robert J. MislevyUniversity of Maryland, College Park

AERA April 2005

------An Application of the Multidimensional Random

Coefficients Multinomial Logit Item Bundle Model

We would like to thank Nancy Butler Songer and Amelia Wenk Gotwals of the BioKIDS project for permission to use the data in the example, and their time, patience, and insights in helping us understand the substantive aspects of the

modeling described here. We are grateful to Larry Hamel and Cathy Kennedy for lots of help along the way.

Item Bundle MRCML Model

2

Assumption: Bundle Independence implies that a bundle of items are expected to be dependent and are assumed local independence across bundles.

Item Bundle MRCML Model --- Item bundle nested in Multidimensional Random Coefficient Multinomial Logit (MRCML) model

CK

Kcjcj

cjcjCC

ab

abBAjXP

1

''

''

)exp(

)exp(),,;(

,),(),,,,(1

21

C

cnncnnCnn UPUUUP

BioKIDS Assessment Summary Content Areas Biodiversity Simple Machine

Inquiry Skills Hypothesis/Predictions Building Explanations from

Evidence Interpreting data Re-express data

3

Two Structurally Similar Examples from BioKIDS

4

BioKIDS04

Two Structurally Similar Examples from BioKIDS

5

BioKIDS05

Analysis Design

6

I. Dimensionality

One-Dimension----------

Two-Dimension----------

Five-Dimension---------

Biodiversity+

Combined Inquiry

Analysis Design

7

Parameterization of Not bundled Model of BioKIDS04 (Claim)

Parameterization of Not Bundled Model of BioKIDS04 (Evidence)

II. Dimensionality

Analysis Design

8

Parameterization of Bundled Model of BioKIDS04

Note: item bundle parameter =( , , , , ) , for example represent for step difficulty parameter associated with response category of evidence equals 1 conditioning on claim equals 0, etc.

01 ceb 02 ceb 1cb 1ceib 12 ceb 01 ceb

Results of Chi-Square

9

BioKIDS04

BioKIDS05

Note: Chi-Square critical value is 15.0863

Results of Deviance

10

A multidimensional item bundle analysis suggest that the BioKIDS Fall 2003 Pretest exhibits item local dependence in at least one cluster of items based on a common stimulus situation.

Taking dimensionality and idiosyncratic features of test format into consideration makes the analysis of local item independence more accurate and meaningful.

Modeling such data using the MRCML item bundle model can deal with item dependence and dimensionality simultaneously and reduce distortions in item parameter estimates as well as proficiency estimates.

Thank you ! 11

Conclusions