advances in learning analytics and educational data mining
TRANSCRIPT
Advances in Learning Analytics (LA) &
Educational Data Mining (EDM)
Mehrnoosh Vahdat, Luca Oneto, Alessandro Ghio,
Davide Anguita, Mathias Funk, and Matthias Rauterberg
April 22-24, 2014
European Symposium on Artificial Neural Networks,Computational Intelligence and Machine Learning
http://www.icephd.org www.SmartLab.ws
Outline
• Introduction
• What is LA/ EDM?
• Data and Methods of LA/ EDM
• Applications of LA/EDM
Outline
• Introduction
• Addressed Questions
• Importance : Shift in data resources
• What is LA/ EDM?
• Data and Methods of LA/ EDM
• Applications of LA/EDM
Introduction• Automatic analysis enhances the learning experience [1]
• Availability of educational data empowers LA/ EDM• to inform and support learners, teachers, and institutions
• to understand and predict personal learning needs/ performance [2]
• LA and EDM help • discover the hidden patterns from raw data [3]
• respond to educational questions and problems [4]
• The combination of LA and EDM• to new insights on learners’ behavior
• improve the TEL methods in a data-driven way
• personalization and adaptation
EDM LA
Image credit: http://www.collegestats.org/
Importance: Shift in data resources
• Development of public data repositories (PSLC DataShop)
• Data from MOOCs
• Advantages of open data resources: • Used as benchmarks to develop new algorithms [5]
• Availability of data resources is a motivation for research in the field [6]
Outline
• Introduction
• What is LA/EDM?
• LA/EDM Goals
• Similarities and Differences of LA/EDM
• EDM/ LA Process
• Stakeholders
• Challenges
• Data and Methods of LA/ EDM
• Applications of LA/EDM
What is LA/ EDM?• LA is:
• a multi-disciplinary field involving machine learning, artificial intelligence, information retrieval, statistics, and visualization [1]
• the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs [7]
• EDM is concerned with:• developing, researching, and applying computerized methods to detect patterns in
large collections of educational data that would otherwise be hard or impossible to analyze due to the enormous volume of data within which they exist [8]
LA/ EDM Goals
• To predict future learning behavior
• To Discover/ improve domain models
• To Study the effects of pedagogical support by learning software
• To advance scientific knowledge about learning through models
• To assess students’ learning performance
• To improve the learning process and guide students
• To provide feedback and adapt learning recommendations to behavior
• To evaluate learning materials and courseware
• To detect abnormal learning behaviors and problems [9, 10]
Similarities of LA and EDM
• Emergence of data-intensive approaches
• Improving education • By improving assessment, how problems are
understood, and how interventions planned/selected
• Improving the analysis of data• To support research and practice in education [11]
Differences of LA and EDM
• Origins• LA: Semantic web, outcome prediction, systemic intervention
• EDM: Educational software, student modeling, predicting outcomes
• Human judgment vs. automation
• Adaptation and personalization• LA: Empowers instructors, learners
• EDM: Automates adaption, provides automated feedback
• Techniques and methods• EDM: Finds new patterns, develops new algorithms. A bottom-up approach
• LA: Applies existing methods, tests learning. A top-down approach [9, 11]
• They are complementary• LA: Holistic framework, understands systems in full complexity
• EDM: Reductionist viewpoint, seeks new patterns, modifies algorithms [12]
LA/ EDM Process
In LA/ EDM process, data is collected and analyzed, and after post processing, feedback and interventions are made in order to optimize learning (based on [1, 13])
Stakeholders
• Educators: • To design, decide, plan, building the systems
• To know the needs of students, mistakes
• For real-time insight into the performance,
help at-risk students
• Learners/ Users: • To get recommendations about activities/resources, learning tasks/ path
• To receive information about their progress/performance compared to peers
• To be motivated and encouraged
• Administrators: • To improve system, adapt to needs
• To organize the resources [14, 15, 16]
Image credit: Slides from Hendrik Drachsler
Challenges• Cost
• Of applications, techniques, storing data, developing algorithms
• Data• Interoperability: all data levels together
• Standards: IEEE standard, Experience API
• Reliability: user’s role in activity data
• Context and time: make sense of unorganized information
• Ethical obligations• Privacy and anonymity [9, 17, 18]
• Connection with cognition, metacognition and pedagogy• Is under-represented
• Need to be strong with learning sciences [19]
Image credit: https://safeguardingstudentlearning.net
Outline
• Introduction
• What is LA/EDM?
• Data and Methods of LA/ EDM
• Data Environments
• Data Categories
• Data Features
• LA/ EDM Methods
• Applications of LA/EDM
Data Environments
• Student information systems (SIS)
• Social media
• Web-based courses
• Traditional learning management systems (LMS)
• Adaptive intelligent educational systems
• Personal learning environments (PLE)
• Open data sets [1]
Image credit: http://chronicle.com
Data Categories
• Students-performed actions with a given outcome• Source view during a time span, finishing an activity with a given result
• Student profile• Age, interest, gender, etc. [17]
Image credit: http://gettingsmart.com
Data Features
• Heterogeneous
• Hierarchical• Various levels of data nested inside one another
• Time • Length of practice sessions, time to learn
• Sequence • Order of concepts, practice, tutoring
• Context• For explaining the results, to know where a model works or may not work [9]
LA/ EDM Methods
The most popular methods:
1. Classification
2. Clustering
3. Regression (logistic/multiple)
4. Discovery with models [12]
Image credit: http://www.oracle.com
EDM Methods
• Prediction • Classification
• Regression
• Density estimation
• Clustering
• Relationship mining • Association rule mining
• Correlation mining
• Sequential pattern mining
• Causal data mining
• Distillation of data for human judgment
• Discovery with models [5]
LA Methods
• Social network analysis• Relationship interaction to identify disconnected students
• Social or attention metadata• Using the metadata provided by educators, learners, and provide data about learning
resources
• Learning Registry: sharing learning resource data [9]
Outline
• Introduction
• What is LA/EDM?
• Data and Methods of LA/ EDM
• Applications of LA/EDM
• European Projects
• Success Cases
• Trends of Future Research in LA/EDM
• Special Session Contributions
Applications of EDM/LA
Most popular approaches:
• Student modeling: • to represent the user, adapt the teaching experiences to meet learning
requirements of individual [20]
• Prediction of performance
• Increase of students’ and teachers’ reflection
• Awareness improvement of provided feedback
• Assessment services [12]
European projects
• European Commission recently granted: • Research projects
• Implementations
• Community building
• Examples: LACE, PELARS, LEA’s BOX, Next-Tell, WeSpot, and WatchMe.
http://www.laceproject.euhttp://www.pelars-project.euhttp://www.leas-box.euhttp://next-tell.euhttp://wespot.nethttp://www.project-watchme.eu
Success cases
• It is challenging to find many successful cases• from research into practice
• Apply LA/ EDM to improve educational outcome by measuring learning behavior/ performance of the students
• LACE FP7 European project: • created a knowledge base of evidence: Evidence hub
• captures effective evidence of the field, assign particular evidence types and sectors
Image credit: http://evidence.laceproject.eu/
Example
Course Signals
• Detects early warning signs by predictive modeling
• Provides interventions to students at risk (email)• Place students in risk groups (traffic light)
• Help instructors provide individual feedback
• Gives customized feedback to students
• Retention rate and grades are improved [21, 22]
Image credit: http://www.purdue.edu
Trends of future research in LA/ EDM
• Developing user-friendly mining tools
• Developing decision supports and recommendation engines
• Standardization of methods and data
• Developing context-adapted models
• Integration with the e-learning system
• Specific data mining techniques
• Advancing the anonymization of data [1, 8, 9, 11]
Special Session Contributions • Adaptive structure metrics for automated feedback provision in java
programming. B. Paassen, B. Mokbel, and B. Hammer
• Human algorithmic stability and human rademacher complexity. M. Vahdat, L. Oneto, A. Ghio, D. Anguita, M. Funk, and M. Rauterberg
• High-school dropout prediction using machine learning: A danish large-scale study.
N.B. Sara, R. Halland, C. Igel, and S. Alstrup
• The prediction of learning performance using features of note taking activities.
M. Nakayama, K. Mutsuura, and H. Yamamoto
• Enhancing learning at work. how to combine theoretical and data-driven approaches, and multiple levels of data?
V. Kalakoski, H. Ratilainen, and L. Drupsteen
• Weighted clustering of sparse educational data. M. Saarela and T. Kärkkäinen
References• [1] M. A. Chatti, A. L. Dyckhoff, U. Schroeder, and H. Thüs. A reference model for learning analytics. International Journal
of Technology Enhanced Learning, 4(5):318–331, 2012.
• [2] W. Greller and H. Drachsler. Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 13(3):42–57, 2012.
• [3] G. Siemens. Learning analytics: envisioning a research discipline and a domain of practice. In International Conference on Learning Analytics and Knowledge, 2012.
• [4] N. Bousbia and I. Belamri. Which contribution does edm provide to computer–based learning environments? In Educational Data Mining, 2014.
• [5] R. Baker and K. Yacef. The state of educational data mining in 2009: A review and future visions. JEDM-Journal of Educational Data Mining, 1(1):3–17, 2009.
• [6] K. Verbert, N. Manouselis, H. Drachsler, and E. Duval. Dataset-driven research to support learning and knowledge analytics. Educational Technology & Society, 15(3):133–148, 2012.
• [7] http://www.solaresearch.org/about
• [8] C. Romero, S. Ventura, M. Pechenizkiy, and Ryan S. Baker. Handbook of educational data mining. Data Mining and Knowledge Discovery Series. Boca Raton, FL: Chapman and Hall/CRC Press, 2010.
• [9] M. Bienkowski, M. Feng, and B. Means. Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. In US Department of Education, Office of Educational Technology, 2012.
• [10] W. He. Examining students’ online interaction in a live video streaming environment using data mining and text mining. Computers in Human Behavior, 29(1):90–102, 2013.
• [11] G. Siemens and R. Baker. Learning analytics and educational data mining: towards communication and collaboration. In international conference on learning analytics and knowledge, 2012.
References• [12] Z. Papamitsiou and A. Economides. Learning analytics and educational data mining in practice: A systematic
literature review of empirical evidence. Journal of Educational Technology & Society, 17(4):49–64, 2014.
• [13] D. Clow. The learning analytics cycle: Closing the loop effectively. In International Conference on Learning Analytics and Knowledge, 2012.
• [14] C. Romero and S. Ventura. Educational data mining: A survey from 1995 to 2005. Expert systems with applications, 33(1):135–146, 2007.
• [15] G. Siemens and P. Long. Penetrating the fog: Analytics in learning and education. Educause Review, 46(5):30–32, 2011.
• [16] Drachsler, Hendrik, and Wolfgang Greller. "The pulse of learning analytics understandings and expectations from the stakeholders." Proceedings of the 2nd international conference on learning analytics and knowledge. ACM, 2012.
• [17] A. Del Blanco, A. Serrano, M. Freire, I. Mart´ınez-Ortiz, and B. Fernández-Manjón. E-learning standards and learning analytics. can data collection be improved by using standard data models? In Global Engineering Education Conference, 2013.
• [18] K. Gyllstrom. Enriching personal information management with document interaction histories. PhD thesis, The University of North Carolina at Chapel Hill, 2009.
• [19] R. Ferguson. Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5):304–317, 2012.
• [20] A. Pena-Ayala. Educational data mining: A survey and a data mining-based analysis of recent works. Expert systems with applications, 41(4):1432–1462, 2014.
• [21] K. E. Arnold and M. D. Pistilli. Course signals at purdue: Using learning analytics to increase student success. In International Conference on Learning Analytics and Knowledge, 2012.
• [22] http://www.purdue.edu/uns/x/2009b/090827ArnoldSignals.html
Advances in Learning Analytics (LA) & Educational Data Mining (EDM)Was presented by Mehrnoosh VahdatAt European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning
Bruges - April 22-24, 2014
http://www.icephd.org www.SmartLab.ws
http://goo.gl/ouywVU
@MehrnooshV
These slides are provided under the Creative Commons Attribution Licence: http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.