latent regression models. where does the probability come from? why isn’t the model deterministic....
DESCRIPTION
Two different models Random sampling Stochastic subjectTRANSCRIPT
![Page 1: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/1.jpg)
Latent regression models
![Page 2: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/2.jpg)
Where does the probability come from?
• Why isn’t the model deterministic.• Each item tests something unique
– We are interested in the average of what the items assess
• Stochastic subject argument• Random sampling of subjects
![Page 3: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/3.jpg)
Two different models
Pr ; |ni ni i nX x
Pr ; ,ni ni i nX x
Random sampling
Stochastic subject
![Page 4: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/4.jpg)
A Random Effects (Sampling) Model -- 1
2~ ,N
2222
1; , exp22
g
Part 1: A Model for the population
equivalently
![Page 5: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/5.jpg)
A Random Effects Model -- 2
Part 2: A Model for the item response mechanism
Pr ; | ; |
exp1 exp
ni ni i ni i
ni i
i
X x f x
x
Example: SLM (but can be any)
![Page 6: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/6.jpg)
A Random Effects Model -- 3
Part 3: Putting them together
2 2; , , ; | ; ,ni i ni if x f x g d
The unknowns are: 2, iand is not an unknown, it is variable of integration
What is analysed is item response, what is estimated is itemparameters and population parameters
![Page 7: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/7.jpg)
Why do this?
• Solves some theoretical estimation problems– For non-Rasch models
• Provides better estimates of population characteristics
![Page 8: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/8.jpg)
Problems with point estimates
ˆvar var
var var
var
n n n
n n
n
e
e
![Page 9: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/9.jpg)
Problem with discreteness
• For a 6-item test, there are only 7 possible ability estimates to assign to people, those getting a score of 0,1,2,3,4,5,6. (raw score is sufficient statistic for ability)
• Suppose we want to know where the 25th percentile point is. That is, 25% of the population are below this point. We need extrapolation.
![Page 10: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/10.jpg)
The Resulting JML Ability Distribution
Score 0
Score 1
Score 2
Score 3Score 4
Score 5
Score 6
Proficiency on Logit Scale
![Page 11: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/11.jpg)
Distribution for a six item test
Score 0
Score 1
Score 2
Score 3Score 4
Score 5
Score 6
Proficiency on Logit Scale
![Page 12: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/12.jpg)
Traditional approach is a two-step analyses
First estimate abilities
Then compute population estimates such as mean, variance, percentiles using
leads to biased results due to measurement error.
In the case of the population variance, we can correct the bias (disattenuate) by multiplying by the reliability.
But in other cases, it is less obvious how to correct for the bias caused by measurement error.
n̂
n̂
![Page 13: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/13.jpg)
Distribution of Estimates is Discrete
• One ability for each raw score• Ability estimates have a discrete distribution• We imagine (and the model’s premise) is a
continuous distribution• The distribution of ability estimates is
distorted by measurement error
![Page 14: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/14.jpg)
Solutions
• Direct estimation of population parameters (directly via item responses, and not through the estimated abilities)
• Complicated analyses that take into account the error– Not always possible
![Page 15: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/15.jpg)
MML: How it works — 1
• Item Response Model for item i:
• Population Model (discrete)
i
iixf
exp1
exp/1
g()-1.5 0.1-0.5 0.2
0 0.40.5 0.21.5 0.1
![Page 16: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/16.jpg)
MML: How it works — 2
1 1/ 1.5 1.5
1/ 0.5 0.5
1/ 1.5 1.5
1/
/
i i
i
i
i
f x f x g
f x g
f x g
f x g
f x g d continuous case
![Page 17: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/17.jpg)
The Implications — 1
• If , then contains parameters – Note that no ability parameters are involved, only
population parameters.• Use maximum likelihood estimation method
to estimate the item difficulty parameters and population parameters.
• Thus, we directly estimate population parameters through the item responses
2,~ Ng xf
,,,,, 21 I
![Page 18: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/18.jpg)
Bayes Theorem
PrPr |
Pr
Pr Pr | Pr
A BA B
B
A B A B B
PrPr |
Pr
Pr Pr | Pr
A BB A
A
A B B A A
Pr | Pr Pr | Pr
Pr | PrPr |
Pr
A B B B A A
B A AA B
B
![Page 19: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/19.jpg)
The Idea of Posterior Distribution
• If a student’s item response pattern is x then the posterior distribution is given by
Pr | PrPr |
PrB A A
A BB
| ||
|
f g f gh
f f g d
x x
xx x
Pr | PrPr |
Pr
x
xx
![Page 20: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/20.jpg)
The Idea of Posterior Distribution
• Instead of obtaining a point estimate for ability, there is now a (posterior) probability distribution
• incorporates measurement error for
the uncertainty in the estimate.
n̂
|h x
|h x
![Page 21: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/21.jpg)
The Resulting JML Ability Distribution
Score 0
Score 1
Score 2
Score 3Score 4
Score 5
Score 6
Proficiency on Logit Scale
![Page 22: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/22.jpg)
Resulting MML Posterior Distributions
Score 0
Score 1
Score 2
Score 3 Score 4
Score 5
Score 6
Proficiency on Logit Scale
![Page 23: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/23.jpg)
MML EAP Estimates – an aside
Score 0
Score 1
Score 2
Score 3 Score 4
Score 5
Score 6
Proficiency on Logit Scale
![Page 24: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/24.jpg)
MML EAP Estimates – an aside
• Biased at the individual level• Discrete scale, bias & measurement error
leads to bias in population parameter estimates
• Requires assumptions about the distribution of proficiency in the population
![Page 25: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/25.jpg)
Distribution for a six item test
Score 0
Score 1
Score 2
Score 3Score 4
Score 5
Score 6
Proficiency on Logit Scale
![Page 26: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/26.jpg)
Estimating proportions below a point based up posterior distributions
![Page 27: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/27.jpg)
More General Form of the Model
expPr ; , , ,
expn
n nn n n
n
z
x b AξX x b A ξ
z b Aξ
2
2
~ ,
~ ,
N x y z
N
Y β
Item response model
Population model
![Page 28: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/28.jpg)
The underlying population distribution is a mixture of two normal distributions, with different means ( and ).
Population Not Normal
• E.g., sample consists of grades 5 and 8.
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69
1 2
![Page 29: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/29.jpg)
• where x=0 if a student is in group 1 and x=1 if a student is in group 2. In this case, we estimate , and . Note that is the difference between the means of the two distributions. That is, group 1 has mean (as x=0), and group 2 has mean (as x=1).
Latent Regression - 1
2,~ xNg
2
![Page 30: Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested](https://reader036.vdocuments.us/reader036/viewer/2022070605/5a4d1ace7f8b9ab0599708eb/html5/thumbnails/30.jpg)
Latent Regression - 2
• We call “x” a “regressor”, or a “conditioning variable”, or a “background variable”. We can generalise to include many conditioning variables.
2,~ zyxNg