learning analytics: seeking new insights from educational data
DESCRIPTION
CPUT Fundani TWT - 22 May 2014 Analytics is a buzzword that encompasses the analysis and visualisation of big data. Current interest results from the growing access to data and the many software tools now available to analyse this data in Higher Education, through platforms such as Learning Management Systems. This seminar provides an overview of current applications and uses of learning analytics and how it can help institutions of learning better support their learners. The illustrative examples look at institutional and social media data that together provide rich insights into institutional, teaching and learning issues. A few simple ways to perform such analytics in a context of Higher Education will be introduced.TRANSCRIPT
![Page 1: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/1.jpg)
Learning Analyticsseeking new insights from educational data
Andrew Deacon
Centre For Innovation in Learning and TeachingUniversity of Cape Town
Teaching and Learning with Technology workshop, CPUT, 2014
![Page 2: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/2.jpg)
Outline
• What is changing with ‘big data’
• Three eras of social science research
• Three ways educational data is analyzed
• Changing roles of analytics with more data
![Page 4: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/4.jpg)
Learning Analytics
… is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.
https://tekri.athabascau.ca/analytics
![Page 5: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/5.jpg)
ASKING NEW QUESTIONSThree eras of social research
![Page 6: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/6.jpg)
Three eras of social research
1. Age of Quételet collect data on simple & important questions
2. Classical period get the most information from a little data
3. Present day big data deluge of data and questions
![Page 7: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/7.jpg)
[1] UCT Student Experience Survey
• Understand students’ overall experience
• Data to effect change, improve decisions and policies, affirm good practices & quality assure
• Good practice
![Page 8: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/8.jpg)
![Page 9: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/9.jpg)
[2] Are streams being disadvantaged?
Within Degree Type: • Differences in
mean final mark are significant
• Across years, differences in means are similar
• Differences in 2013 are not unusual
Change in mode of delivery
![Page 10: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/10.jpg)
[3] UCT and social media
Prominent links to:– Facebook– Flickr– LinkedIn– Twitter
![Page 11: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/11.jpg)
Twitter: UCT chatter
• Looked at 6 months of data April – Sept 2011
• Selected tweets with a UCT hashtag or text#UCT, #Ikeys, University of Cape Town, …
• Attributes tweet amplification, app used, location
• Dataset Just over 5,000 tweets
![Page 12: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/12.jpg)
Twitter: apps & locations
27%
36%
20%
17%
1 2 3 4
Smartphone geo-location
Cell phones
Blackberry
![Page 13: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/13.jpg)
Twitter: tweeter relationships
Frequent tweeters:1. Drama student
(162)
2. UCT Radio (132)
3. Science student(84)
![Page 14: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/14.jpg)
Twitter: viral #UCT
Varsity Cup final
Helicopter crash
6 months of tweets
![Page 15: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/15.jpg)
Flickr: helicopter crash at UCT
Ian Barbour - http://www.flickr.com/people/barbourians/
![Page 16: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/16.jpg)
Twitter: helicopter crash at UCT
2 hours after the
event
• Peak of 140 tweets in 5 minutes
• Media organisations tweets get re-tweeted
• Crash or hard-landing?
![Page 17: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/17.jpg)
Facebook: all friend relationships
Paul Butler http://www.facebook.com/notes/facebook-engineering/visualizing-friendships/469716398919
![Page 18: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/18.jpg)
1st year course combinationsat UCT Health
Sciences
Engineering
Humanities
Science
Commerce
![Page 19: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/19.jpg)
DIFFERENT ANALYTICAL TOOLSThree approaches to educational data
![Page 20: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/20.jpg)
Three approaches to educational data
1. Psychometrics placing measures on a scale (e.g., in assessment)
2. Educational Data Mining focus on learning over time (e.g., in schools)
3. Learning Analytics typically wider contexts (e.g., in universities)
![Page 21: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/21.jpg)
[3] Developing learning analytics
![Page 22: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/22.jpg)
Students’ use of Vula in a course
Site visits
Chat room activitySectioning
of students
Polling of students
Content accessed
Submission of assignments
Submission of assignments
![Page 23: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/23.jpg)
Purdue University's Course Signals
• Early warning signs provides intervention to students who may not be performing well
• Marks from course• Time on tasks• Past performance
Source: http://www.itap.purdue.edu/learning/tools/signals
![Page 24: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/24.jpg)
Advisors – U Michigan
• Advisors are key element• Data from LMS – Measures to compare students
(LMS performance and LMS usage)– Classifications
(<55% red and >85% green)– Visualizations of student performance
• Engagement with advisors – Dashboard
![Page 25: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/25.jpg)
Measures to compare students
• LMS Gradebook and Assignments – Student score as percentage of total– Class mean score as percentage of total
• LMS Presence as proxy for ‘effort’– Weekly total– Cumulative total
![Page 26: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/26.jpg)
Classifications of cohort
Comparisons are intra-classPerformance Change Presence Rank Action
>= 85% Encourage
75% to 85% < 15% Explore
>= 15% < 25% Explore
>= 15% >= 25% Encourage
65% to 75% < 15% < 25% Engage
< 15% >= 25% Explore
>= 15% Explore
55% to 65% >= 10% Explore
< 10% Engage
< 55% Engage
![Page 27: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/27.jpg)
Advisor support
• Shorten time to intervene– Weekly update– Contact ‘red’ students– Useful to prepare for consultation
• Contextualizing student performance– Longitude trends (course and degree) – Identify students who don’t need support
![Page 28: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/28.jpg)
Example of tools: RapidMiner
![Page 29: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/29.jpg)
Sociogram of a discussion forum
Dawson (2010)
![Page 30: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/30.jpg)
Words used by Lecturers vs Students
Used more by Students
Used more by Lecturers/tutors
‘Weiten’ –textbook author
Marks;thanks;
test;Tut;guys
Week;pages
![Page 31: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/31.jpg)
MOOC Completion Rates
http://www.katyjordan.com/MOOCproject.html
![Page 32: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/32.jpg)
![Page 33: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/33.jpg)
CHANGING ROLES OF ANALYTICSThe future with more data
![Page 34: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/34.jpg)
Concerns about Big Data thinking
• Does Big Data…– change the definition of knowledge– increase objectivity and accuracy– analysis improves with more data– make the context less critical– availability means using the data is ethical– reduce digital divides
See (Boyd & Crawford 2012)
![Page 35: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/35.jpg)
Effective visualisations
The success of a visualization is based on deep knowledge and care about the substance, and the quality, relevance and integrity of the content.
Tufte (1981)
![Page 36: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/36.jpg)
Correlation and causation
• Correlation does not imply causation
– Covariation is a necessary but not a sufficient condition for causality
– Correlation is not causation (but could be a hint)
![Page 37: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/37.jpg)
Future scenarios
• Analytics in educational research:– More data means asking new questions– Interpreting data in a student’s context– Open up discussions and possibilities– New ethical considerations
• Visualisations and analytics tools:– Good open source software is available – Encourage people to engage with learning analytics
![Page 38: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/38.jpg)
Software references
• Gephi – network analysis, data collection• NodeXL – network analysis, data collection• TAGS – Twitter data collection (Google Drive)• Word cloud – R package (wordcloud)• RapidMiner – Data mining, predictive analytics• Excel – spreadsheet, charts• R – statistical analysis, graphs
![Page 39: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/39.jpg)
Literature references• Boyd, D., Crawford, K. (2012) Critical Questions for Big Data, Information,
Communication & Society, 15:5, 662-679• Dawson, S. (2010) ‘Seeing’ the learning community: An exploration of the
development of a resource for monitoring online student networking. British Journal of Educational Technology, 41(5), 736-752.
• Deacon, A., Paskeviciusat, M. (2011) Visualising activity in learning networks using open data and educational analytics. Southern African Association for Institutional Research Forum, Cape Town.
• Berland, M., Baker, R.S., Blikstein, P. (in press) Educational data mining and learning analytics: Applications to constructionist research. To appear in Technology, Knowledge, and Learning.
• Hansen, D., Shneiderman, B., Smith, M.A. (2011) Analyzing Social Media Networks with NodeXL: Insights from a Connected World, Morgan Kaufmann Publishers, San Francisco, CA.
• Tufte, E. (1981) The visual display of quantitative information. Cheshire, Conn.: Graphics Press.
![Page 40: Learning Analytics: Seeking new insights from educational data](https://reader035.vdocuments.us/reader035/viewer/2022070313/5549d2cbb4c9051c778b4db3/html5/thumbnails/40.jpg)
South African references