the polytomous unidimensional rasch model: understanding its response structure and process acspri...
DESCRIPTION
Ingredient 1 Invariance of comparisons Rasch’s requirement for invariance of parameters estimates leads to models with sufficient statistics The model for dichotomous responses is a special caseTRANSCRIPT
The Polytomous Unidimensional Rasch Model: Understanding its Response Structure and Process
ACSPRI Social Science Methodology Conference, Sydney, December 2006
Mailing addressDavid AndrichMurdoch University Murdoch 6150Western AustraliaEmail: [email protected]
Acknowledgements
• The research was supported in part by an Australian Research Council Linkage grant with the Australian National Ministerial Council on Employment, Education, Training and Youth Affairs (MCEETYA) Performance Measurement and Reporting Task Force; UNESCO’s International Institute for Educational Planning (IIEP), and the Australian Council for Educational Research (ACER) as Industry Partners.
Ingredient 1Invariance of comparisons
• Rasch’s requirement for invariance of parameters estimates leads to models with sufficient statistics
• The model for dichotomous responses is a special case
Ingredient 2Standard formats
Fail < Pass < Credit < Distinction
Never < Sometimes < Often < Always
Strongly Disagree < Disagree < Agree < Strongly Agree
Ingredient 3The dichotomous Rasch model
• Criterion
• The dichotomous model
),,,()},(|,,);,{( jinjninjnijinnjnjnini yyyyfyYyYr
in
in
eeyY
y
ni
1}Pr{
)(
Ingredient 4The Guttman Structure
Items 1 2 3 Total
Score x }|),,Pr{( 321 xyyy nnn
I+1=4 Guttman response patterns 0 0 0 0 1 1 0 0 1 0.667 1 1 0 2 0.678 1 1 1 3 1
2I– I–1 =4 Non-Guttman response patterns 0 1 0 1 0.248 0 0 1 1 0.085 1 0 1 2 0.235 0 1 1 2 0.087
Ingredient 5Design of an experiment
_____________________________________________________________________ Fail (F) Inadequate setting Insufficient or irrelevant information given for
the story… Pass (P) Discrete setting Discrete setting as an introduction, with some
details which also show some linkage and organisation. …. Credit (C) Integrated setting There is a setting which, rather than simply
being at the beginning, is introduced throughout the story. Distinction (D) Integrated and manipulated setting: In addition to the setting
being introduced throughout the story, … _____________________________________________________________________
Experimentally independent responses
Inadequate setting F
Discrete setting P
Integrated setting C
Integrated and manipulated
setting D
Judge 1 Not P P
Judge 2 Not C C
Judge 3 Not D D
Requirements of data
• 1. The success rate at P is greater than that at C, and that the success rate at C is turn be greater than at D
• 2. Relative success rates are independent of the locations of the essays on the continuum. (The dichotomous Rasch model)
DCP ˆˆˆ
iny 1 iny 2 iny 3
0
1niQ 0
2niQ 0
3niQ
1
1niP 0
2niQ 0
3niQ
1
1niP 1
2niP 0
3niQ
' 1
1niP 1
2niP 1
3niP
0
1niQ 1
2niP 0
3niQ
0
1niQ 0
2niQ 1
3niP
1
1niP 0
2niQ 1
3niP
0
1niQ 1
2niP 1
3niP
1)},...,...,Pr{( 3
1
121
k
m
k
ynki
ynkinminkiinin
inkinki QPyyyy
The Guttman subspace
iny 1 iny 2 iny 3
0
1niQ 0
2niQ 0
3niQ
1
1niP 0
2niQ 0
3niQ
1
1niP 1
2niP 0
3niQ
' 1
1niP 1
2niP 1
3niP
1)},...,...,Pr{(''
3
1
121
k
m
k
ynki
ynkinminkiinin
inkinki QPyyyy
Define
},...,2,1,0{;1
i
m
knikni mxyxX
i
Inferred category and the
total score xX ni
1niy
2niy 3niy
0 Fail 0 0 0 1 Pass 1 0 0 2 Credit 1 1 0 ' 3 Distinction 1 1 1
A sub subspace
Inferred category and the
total score xX ni
1niy
2niy 3niy
0 Fail 0 0 0 1 Pass 1 0 0 1,0
' 2 Credit 1 1 0 ' 3 Distinction 1 1 1
Probability of a response in the Guttman Space
DQPyyyyk
m
k
ynki
ynkinminkiinin
inkinki
1)},...,...,Pr{(''
3
1
121
iiii nimninininimninininimninininimninini PPPPQQPPQQQPQQQQD ............... 321321321321
DQQQPPPPxXinimnixnixnixnininini /.......}|Pr{ 21321
'
1}|Pr{0
'
im
xni xX
The doubly conditioned outcome space xx ,1
'
nixnixnix
nix
nini
ni PQP
PxXxX
xX
}|Pr{}|1Pr{}|Pr{
''
'
}|Pr{}|1Pr{}|Pr{
}|Pr{ ''
''
,1
xXxXxX
xXPnini
nixxninix
nixnixxxni PYxX }|1Pr{}|Pr{ ',1
Reverse Process
Let
Define
Then
Indicating the model is the same
},...,2,1,0{, ini mxxX
nixnini
ni PxXxX
xX
}Pr{}1Pr{}Pr{
nixnix PQ 1
DQQQPPPPxXinimnixnixnixnininini /.......}|Pr{ 21321
'
Inferring an experimentally independent outcome space
• Given the Guttman space , we infer the existence of a complete space of which is a subspace. In this complete space we can infer experimentally independent responses.
' '
The PRM
Deee
ee
ee
e
xX
iimnixnixn
ixn
in
in
in
in
ni
/1
1...1
11
...11
}|Pr{
12
2
1
1
'
i
x
kiknm
x
x
ni e0
0
ni
x
ni
x
kikn
exX
/}|Pr{ 0'
Equivalences of corresponding thresholds in the spaces
Outcome space Response Px Cx Dx
}|1Pr{ nixY
iPn
iPn
ee
1
iCn
iCn
ee
1
iDn
iDn
ee
1
}|Pr{ ',1 xxni xX
iFPn
iFPn
ee
1
iPCn
iPCn
ee
1
iCDn
iCDn
ee
1
iFPiP iPCiC iCDiD
{ }|{}| ',1 xixix
Simulation 1 Judges Item
number Generated locations
Estimates from the dichotomous RM in a full space
Estimates from the PRM in a Guttman space
Novice 1 (P) 1 -1.1 -1.125 -1.047 Novice 2 (C) 2 -0.1 -0.119 -0.133 Novice 3(D) 3 1.2 1.267 1.196 Expert 1(P) 4 -1.7 -1.685 -1.680 Expert 2 (C) 5 0.3 0.227 0.232 Expert 3 (D) 6 1.4 1.435 1.433 RMSQ 0.018 0.017 Fit 2 =6.081, df=4
P <0.193
Simulation 2Judges Item
number Generated locations
Estimates from the dichotomous RM in a full space
Estimates from the PRM in a Guttman space
Novice 1 (P) 1 -1.1 -1.125 -1.132 Novice 3 (C) 2 1.2 1.267 1.452 Novice 2(D) 3 -0.1 -0.119 -0.169 Expert 2(P) 4 0.3 0.227 0.122 Expert 1 (C) 5 -1.7 -1.685 -1.651 Expert 3 (D) 6 1.4 1.435 1.379 Extreme scores eliminated
1214 1214
RMSQ 0.018 0.054 Fit 2 =0.932, df=4
P <0.920
Item S121 Marking Key 2: all parts recognisable in shape, size, position, and orientation. 1: most parts recognisable in shape, size, position, and orientation. 0: few parts recognizable in shape, size, position, and orientation.
Item S004 Marking Key 2:rectangle well drawn 1:correct daisies chosen, but rectangle poorly drawn 0:not a four sided figure