irt models for polytomous response data · pdf filenominal response data • nominal...
TRANSCRIPT
![Page 1: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/1.jpg)
IRT Models for Polytomous Response Data
University of KansasItem Response Theory
Stats Camp ‘07
![Page 2: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/2.jpg)
Overview• General Theory
• Polytomous Data Types:– Graded Response– Partial Credit– Nominal Response
• Polytomous IRT Models:– Graded Response– Partial Credit– Nominal Response
![Page 3: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/3.jpg)
General Theory of Polytomous Response Models
• Item response models for polytomous data extend the general underlying IRT premise…– Modeling item response behavior as a function of
(possibly multiple) latent traits.
• …to data types that are not only limited to binary responses.– The type of model is dependent on the type of data
that have been collected.
![Page 4: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/4.jpg)
A Polytomous Starter Example
• Imagine you have the following data from a set of “examinees”:– The number of baskets
made by each examinee out of 10 shots from various locations on a basketball court.
![Page 5: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/5.jpg)
Basketball Data
• The idea of the number of successfully made baskets out of a set number of attempts could possibly follow a well known distribution.– The Binomial Distribution:
( ) xnx ppxn
xXP −−⎟⎟⎠
⎞⎜⎜⎝
⎛== 1)(
![Page 6: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/6.jpg)
Binomial, Continued…
Binomial Distribution Parameters• n is the number of trials (10 for each location in our
example).• x is the number of made baskets for each location.• p is the probability of a made basket for each trial
(assumed constant).
( ) xnx ppxn
xXP −−⎟⎟⎠
⎞⎜⎜⎝
⎛== 1)(
![Page 7: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/7.jpg)
Linking IRT and the Data
• If we could adapt IRT for these data, here is what IRT could provide for the basketball example:– A measure of the tendency for an “examinee”
to hit shots from around the court (the examinee’s latent variable).
• Could be thought of as an examinee’s shooting ability.
– A measure of the relative difficulty of each shot location (the model item parameters).
![Page 8: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/8.jpg)
Linking, Continued• The idea behind polytomous IRT models is to
adapt the IRT portion of the model so that the data are modeled appropriately.
• In this case, the binomial model can be adapted by using the binary IRT model response function in place of the binomial model p parameter.
• See: – Safrit, Costa, and Costa (1989)– Spray (1990)
![Page 9: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/9.jpg)
Binomial IRT Model
• A Two-Parameter Logistic Binomial IRT model would then be:
( )
( )jij
jij
bDa
bDa
ij eeP −
−
+= θ
θ
θ1
)(
• Where:
( ) ijijij xnij
xij
ij
ijiijij PP
xn
xXP −−⎟⎟⎠
⎞⎜⎜⎝
⎛== )(1)()|( θθθ
![Page 10: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/10.jpg)
Binomial IRT Response Function
Binomial IRF
0
0.2
0.4
0.6
0.8
1
-4-3
.4-2
.9-2
.4-1
.9-1
.4-0
.8-0
.3 0.2
0.72
1.24
1.76
2.28 2.
83.
323.
84
Theta
P(X=
x|Th
eta) P(X=0|Theta)
P(X=1|Theta)P(X=2|Theta)P(X=3|Theta)P(X=4|Theta)
Open “binomial response demo.xls” for file demo
![Page 11: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/11.jpg)
Graded Response Models
(AKA Ordered Categorical Responses)
![Page 12: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/12.jpg)
Graded Responses
• One of the more straight-forward polytomous data types is that of the graded-response format.
• Graded response data consists of a score that is an ordinal number, typically ranging from 0 to M.– 0, 1, 2,…, M+1
• Higher scores represent better performance on the item.
![Page 13: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/13.jpg)
Example Graded Response ItemFrom the 2006 Illinois Standards Achievement Test (ISAT):
www.isbe.state.il.us/assessment/pdfs/Grade_5_ISAT_2006_Samples.pdf
![Page 14: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/14.jpg)
ISAT Scoring Rubric
![Page 15: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/15.jpg)
Additional Example Item• Cognitive items are not the only ones where graded
response data occurs.
• Likert-type questionnaires are commonly scored using ordered categorical values.– Typically, these ordered categories are treated as continuous
data.• Methods of analysis typically include Factor Analysis.
– More on this during Friday’s discussion.
• Consider the following item from the Satisfaction With Life Scale (e.g. SWLS, Diener, Emmons, Larsen, & Griffin, 1985)…
![Page 16: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/16.jpg)
SWLS Item #1
I am satisfied with my life.1. Strongly disagree.2. Disagree.3. Slightly disagree.4. Neither agree nor disagree.5. Slightly agree.6. Agree.7. Strongly agree.
![Page 17: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/17.jpg)
Graded Response Model
• An extension of the 2PL model for graded response data is Samejima’s Graded Response Model (1969).
• The graded response model specifies the likelihood that an examinee of a given ability will respond provide a response that receives a grade of xij (xij = 0,…,M).
![Page 18: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/18.jpg)
Graded Response Model
• Samejima’s Graded Response Model:
( ) ( )( )( )ijxij
ijxij
ij bDa
bDa
iijijixe
exXPP −
−
+=≥= θ
θ
θθ1
|*
• Where:
( ) ( ) ( )ixixiijij ijijPPxXP θθθ *
1*| +−==
![Page 19: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/19.jpg)
Graded Response Model Parameters
( ) ( )( )( )ijxij
ijxij
ij bDa
bDa
iijijixe
exXPP −
−
+=≥= θ
θ
θθ1
|*
• For each item j: one discrimination parameter aj.• For M-1 difficulty parameters bm.
– b1 < … < bm
• P(Xij ≥ 0) = 1.0• P(Xij ≥ M) = 0.0
![Page 20: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/20.jpg)
Graded Response Function
Graded Response IRF
0
0.2
0.4
0.6
0.8
1
-4-3
.4-2
.9-2
.4-1
.9-1
.4-0
.8-0
.3 0.2
0.72
1.24
1.76
2.28 2.
83.
323.
84
Theta
P(X=
x|Th
eta) P(X=0|Theta)
P(X=1|Theta)P(X=2|Theta)P(X=3|Theta)P(X=4|Theta)
Open “graded response demo.xls” for file demo
![Page 21: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/21.jpg)
Nominal Response Models
![Page 22: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/22.jpg)
Nominal Response Data
• Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric values.– Rather, item responses are in the form of nominal
categories.
• Information gained from the use of such models can be useful for detecting which distracter options are better than others in multiple choice tests.
![Page 23: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/23.jpg)
Nominal Response Model Features
• The nominal response model is a model for the categorical responses possible within an item of a test.
• The model specifies the probability that an examinee with a given value of the latent trait selects response option m.
![Page 24: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/24.jpg)
Example Nominal Response Item
![Page 25: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/25.jpg)
Additional Item Types
• Non-cognitive tests can also contain differing item types that could be modeled using a Nominal Response Model.
• For example, consider an item from a questionnaire about political attitudes…
![Page 26: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/26.jpg)
Example Nominal Response Item
Which political party would you identify yourself with?
1. Democrat2. Republican3. Independent4. Green5. Unaffiliated
![Page 27: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/27.jpg)
Nominal Response Model
• An extension of the 2PL model for nominal response data is Bock’s Nominal Response Model (1972).
• The nominal response model specifies the likelihood that an examinee of a given ability will selection option kj of item j.
![Page 28: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/28.jpg)
Nominal Response Model
• Bock’s Nominal Response Model:
( )( )
( )∑=
−
−
== m
h
ba
ba
iijjhijh
jkijk
e
ekXP
1
|θ
θ
θ
![Page 29: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/29.jpg)
Nominal Response Model Parameters
( )( )
( )∑=
−
−
== m
h
ba
ba
iijjhijh
jkijk
e
ekXP
1
|θ
θ
θ
• For each level k of item j: one discrimination parameter ajk and one difficulty parameter bjk.
• No additional constraints on the parameter values.
![Page 30: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/30.jpg)
Nominal Response FunctionOpen “nominal response demo.xls” for file demo
Nominal Response IRF
0
0.2
0.4
0.6
0.8
1
1.2
-4
-3.3
-2.7 -2
-1.4
-0.8
-0.1
0.52
1.16 1.8
2.44
3.08
3.72
Theta
P(X=
m|T
heta
)
P(X=a|Theta)P(X=b|Theta)P(X=c|Theta)P(X=d|Theta)
![Page 31: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/31.jpg)
Conclusion
• Numerous polytomous IRT models exist.– Each extends the basic philosophy of IRT to a
a data type that is not binary.• Some differ with respect to how each model is
parameterized for the same data.• Some differ with respect to the differing types of
data to be modeled.
• Each polytomous IRT model specifies the behavior of an examinee as a function of a latent trait (often representing ability).
![Page 32: IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric](https://reader034.vdocuments.us/reader034/viewer/2022051202/5a84135b7f8b9a9f1b8b739f/html5/thumbnails/32.jpg)
Next…
• Estimation of Parameter for IRT Models–Estimate person parameters
when item parameters are known
–Joint estimation of person and item parameters