Download - Csse 2014 hmm presentation_ta_ed
![Page 1: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/1.jpg)
Machine Learning Algorithms: Applications to Educational Data
Saad Chahine, PhD May 26, 2014
![Page 2: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/2.jpg)
Machine Learning “Field of study that gives computers the ability to learn without being explicitly programmed.” (Arthur Samuel, 1959)
“A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” ( Tom Mitchel (1997), Machine Learning, McGraw Hill | Web Page http://www.cs.cmu.edu/~tom/ )
![Page 3: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/3.jpg)
Two Main Types of Algorithms
Supervised learning:• What we are commonly used to in educational research• We know the data and outputs • We have an idea of the kids of analysis we plan to run (e.g., Linear
Regression)
Unsupervised learning:• Used less often in educational research • We try to find a hidden structure to data that may not be labeled • We have more of an intuition of what we are trying to find (e.g. K-
Means Cluster)
![Page 4: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/4.jpg)
My Interest in Machine Learning
Q: Can we begin to build software programs that learn who we are and can then provide individual learning supports through the use of assessment and feedback?
![Page 5: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/5.jpg)
Markov Models “The future is independent of the past, given the present.” (translation Andrey Markov, 1856-1922)
Limitation – Only takes into account current state and the most recent prior state
![Page 6: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/6.jpg)
Hidden Markov Models - A method(s) of finding a hidden
(latent) structure with a sequential data set
Ghahramani, Z.(2001) An introduction to hidden Markov models and Bayesian networks. International Journal of Pattern Recognition and Artificial Intelligence, 15(1): 9-42.
![Page 7: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/7.jpg)
Piaget Developmental Data
• Visser & SpeekenBrink (2010). depmixS4: An R package for hidden Markov Models. International Journal of Statistical Software, 36(7). http://dare.uva.nl/document/361939
• Data from: Jansen, B.R.J., & van der Maas, H.L.J. (2002). The development of children’s rule use on the balance scale task. Journal of Experimental Child Psychology, 81(4), 383–416.
• Siegler, R.S. (1981). Developmental sequences within and between concepts. Number 46 in Monographs of the Society for Research in Child Development. SRCD.
![Page 8: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/8.jpg)
depmixS4 – Balance Data > data(balance)- 779 participants - Ages from 5-19 years - 4 distance items
![Page 9: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/9.jpg)
CAPP - OSCE• Clinical Assessment for Practice Program
(CAPP)• A program of the College of Physicians
and Surgeons of Nova Scotia (CPSNS)• Objective Structured Clinical Exam (OSCE)• Multiple stations with sequences &
competencies
![Page 10: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/10.jpg)
CAPP OSCE Dataset • 434 observations • 31 participants • 14 stations • 9 measures of competency (Coded
P/F) • 13 different case IDs
![Page 11: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/11.jpg)
My Learning • I conducted the balance data
analysis first• Then I began to examine the OSCE
data• The next slides compare the two as
preliminary analysis
![Page 12: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/12.jpg)
Balance • “Used Age as a
covariate on class membership”
• 3 State Model best• Converged in 77
iterations • loglink = -917.50• AIC = 1867• BIC = 1942
OSCE• Used CASE ID as a
covariate on class membership
• 2 state model best• Converged in 55
iterations • loglink = -1757.81• AIC = 3555• BIC = 3637
![Page 13: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/13.jpg)
Balance • Probabilities at
zero values of the covariates
• 0.001, 0.988, 0.009
OSCE• Probabilities at
zero values of the covariates
• 0.606, 0.394
![Page 14: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/14.jpg)
Balance
OSCE
![Page 15: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/15.jpg)
Balance
OSCE
![Page 16: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/16.jpg)
Balance
OSCE
![Page 17: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/17.jpg)
What's Next • OSCE data did not fit as well as the
Balance data – More Years may help • Learning HMM further and potential
application to performance assessments
• Experiment with different covariates in the datasets
![Page 18: Csse 2014 hmm presentation_ta_ed](https://reader036.vdocuments.us/reader036/viewer/2022082916/54c34d034a795956188b45c6/html5/thumbnails/18.jpg)
Acknowledgements • Acknowledge the use code by Visser
& SpeekenBrink (2010) in depmixS4 package
• Thank you to CAPP & Bruce Holmes
for the use of OSCE data