using assessment data
TRANSCRIPT
![Page 1: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/1.jpg)
Assessment data
@glynnmark
![Page 2: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/2.jpg)
Overview
• Introduction
• Learning Analytics
• LMS analytics example and issue
• Assessment analytics
![Page 3: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/3.jpg)
Audience participation
![Page 4: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/4.jpg)
Who am I?
Teacher
Blogger
Tweeter
Learner
Student
Father
husband
Chemist
Dr
Sports fan
BrotherFirst aider
Public relations
executive
Learning technologist
![Page 5: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/5.jpg)
National Institute for Digital Learning
National
Institute for
Digital
Learning
Open
Education
Unit
Digital
Learning
Research
Network
Teaching
Enhancement
Unit
![Page 6: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/6.jpg)
Teaching Enhancement Unit
TEACHINGENHANCEMENT
UNIT
Onlineand Blended Learning
Support
Awards and Grants
Credit Earning Modules
Professional Development
Workshops
![Page 7: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/7.jpg)
Learning Analytics
Learning analytics is the collection and
analysis of data generated during the learning
process in order to improve the quality of
both learning and teaching.
![Page 8: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/8.jpg)
So much student data we could use …
Academic Performance
• CAO and Leaving cert, University exams, course preferences, performance relative to peers in school
Demographics
• Age, home/term address, commuting distance, socio-economic status, family composition, school attended, census information, home property value, sibling activities, census information
Online Behaviour
• Mood and emotional analysis of Facebook, Twitter, Instagram activities, friends and their actual social network, access to VLE (Moodle)
Physical Behaviour
• Library access, sports centre, clubs and societies, eduroam access yielding co-location with others and peer groupings, lecture/lab attendance,
![Page 9: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/9.jpg)
Levels of Analytics
Descriptive• What has happened?
Diagnostic• Why this happened?
Predictive• What will happen?
Prescriptive• What to do?
![Page 10: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/10.jpg)
Learning analytics is a moral practise which should align with core organisational principles
The purpose and boundaries regarding the use of learning analytics should be well defined and visible
Students should be engaged as active agents in the implementation of learning analytics
The organisation should aim to be transparent regarding data collection and provide students with the opportunity to update
their own data and consent agreements at regular intervals
Modelling and interventions based on analysis of data should be free from bias and aligned with appropriate theoretical and
pedagogical frameworks wherever possible
Students are not wholly defined by their visible data or our interpretation of that data
Adoption of learning analytics within the organisation requires broad acceptance of the values and benefits (organisational
culture) and the development of appropriate skills
The organisation has a responsibility to all stakeholders to use and extract meaning from student data for the benefit of students
where feasible
This study was carried out following core principles
Outlined in PLS
Intervention is interactive
Students are able to change their data by engaging with Moodle more
All data is calculated using the same algorithm therefore cannot be biased
Data used is from Moodle usage alone and therefore does not in any way
define an individual
All researchers have the appropriate skills required to handle the data at
hand
Results of the study will be used to better understand how ro increase
studen engagement
Guiding Principles
![Page 11: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/11.jpg)
Total Moodle Activity
![Page 12: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/12.jpg)
Modules which work well …
• Have periodicity (repeatability) in Moodleaccess
• Confidence of predictor increases over time
• Don't have high pass rates (< 0.95)
• Have large number of students, early-stage
![Page 13: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/13.jpg)
One example module – ideal !
![Page 14: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/14.jpg)
Notes on model confidence
• Y axis is confidence in AUC ROC (not probability)
• X axis is time in weeks
• 0.5 or below is a poor result
• Most Modules start at 0.5 when we don't have much information
• 0.6 is acceptable, 0.7 is really good (for this task)
• The model should increase in confidence over time
• Even if confidence overall increases, due to randomness the confidence may go up and down
• It should trend upwards to be a valid model and viable module choice
![Page 15: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/15.jpg)
LG116: Introduction to Politics
Students / year = ~110
Pass rate = 0.78
![Page 16: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/16.jpg)
LG116 – Predictor confidence (ROC AUC)
![Page 17: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/17.jpg)
Discussion
• Good points
• Bad Points
http://padlet.com/markglynn/cxbh3efm9e0z
![Page 18: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/18.jpg)
Assessment data
![Page 19: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/19.jpg)
What data?
• Module title
• Assessment Title
• Opening and closing date
• Description
• Category i.e. weighting
• Student grade i.e. category total
• Student overall grade i.e. Course total
![Page 20: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/20.jpg)
Privacy versus Visibility
-3 +30
![Page 21: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/21.jpg)
Relative grade
![Page 22: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/22.jpg)
Relative Grade (close up)
![Page 23: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/23.jpg)
Relative assessment grade – Lecturers view
-3 +30John
-3 +30
Paul
-3 +30
Ringo
-3 +30
George
57%
86%
65%
43% ˇ
![Page 24: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/24.jpg)
Coordinators view
-3 +30Chemistry
-3 +30
Physics
-3 +30
Biology
-3 +30
Maths
57%
86%
65%
43% ˇ
Student name: John Lennon
![Page 25: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/25.jpg)
CRM project
![Page 26: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/26.jpg)
Discussion
• Good points
• What next?
http://padlet.com/markglynn/cxbh3efm9e0z
![Page 27: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/27.jpg)
What else?
• Module title
• Assessment Title
• Opening and closing date
• Description
• Category i.e. weighting
• Student grade i.e. category total
• Student overall grade i.e. Course total
![Page 28: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/28.jpg)
Teachers Perspective
![Page 29: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/29.jpg)
Calendar
![Page 30: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/30.jpg)
Rubric Analysis
![Page 31: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/31.jpg)
Rubric Analysis
![Page 32: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/32.jpg)
Track individual student engagement with video
![Page 33: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/33.jpg)
video engagement analytics: measure attention during playback
![Page 34: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/34.jpg)
Discussion
• Good points
• What next?
http://padlet.com/markglynn/cxbh3efm9e0z
![Page 35: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/35.jpg)
Conclusions and next steps
• Huge but simple opportunity
• Minimal investment
• Please share
![Page 36: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/36.jpg)
Acknowledgements
![Page 37: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/37.jpg)
Moodle ModificationsLearning Technology Services
www.lts.iesomerandomthoughts.ie
@ghenrick
![Page 38: Using Assessment data](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a0d3b81a28ab3b108b483e/html5/thumbnails/38.jpg)
Team behind the Moodle Analytics project
John BrennanOwen Corrigan
Aly EganAlan F. Smeaton
Sinéad Smyth