learning analytics: more than data-driven decisions
DESCRIPTION
An overview of learning analytics as well as recent examples from higher education and current projects underway at the University of Michigan.From The Horizon Report, 2011:"Learning analytics promises to harness the power of advances in data mining, interpretation, and modeling to improve understandings of teaching and learning, and to tailor education to individual students more effectively. Still in its early stages, learning analytics responds to calls for accountability on campuses across the country, and leverages the vast amount of data produced by students in day-to-day academic activities. While learning analytics has already been used in admissions and fund-raising efforts on several campuses, “academic analytics” is just beginning to take shape."TRANSCRIPT
Learning Analytics:More Than Data-Driven Decisions
Steven LonnResearch Fellow
USE Lab, Digital Media Commonswww.umich.edu/~uselab
1
USE LabUniversity of Michigan
http://umich.edu/~uselab
Acknowledgements
• USE Lab:– Stephanie D. Teasley– Andrew Krumm– R. Joseph Waddington
• John Campbell• John Fritz• Tim McKay• David Wiley
2
USE LabUniversity of Michigan
http://umich.edu/~uselab
What is Analytics?
3
+ +
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Lives
4
USE LabUniversity of Michigan
http://umich.edu/~uselab5
Analytics in Our Lives
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Work
6
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Work
6
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Work
6
What does one DO with all this d
ata?
USE LabUniversity of Michigan
http://umich.edu/~uselab
Data Collected at . .
7
What kind of data is already available those
“in the know?”
USE LabUniversity of Michigan
http://umich.edu/~uselab
• High school GPA• SAT & ACT• Parental education• First generation college student?• Socio-economic status• Admission “rank”• AP tests & scores
8
Admissions
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
• Gender• Ethnicity• Age• Michigan residency• Country of origin & citizenship• Athlete?
9
Demographics
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
• Cumulative GPA • Specific course grades• Major / minor• Number of Michigan credits• Number of transfer credits• Credits / grades in subsets (e.g., math courses)
10
Academic Record
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
• CTools (courses, projects, etc.)• Library (Mirlyn, website, electronic journals)• Wolverine Access• Other UM tools (LectureTools, SiteMaker,
UM.Lessons, MFile, Webmail, etc.)
11
Other Places Data is Gathered...
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
Current Use of Data...
12
USE LabUniversity of Michigan
http://umich.edu/~uselab
What if...• Identify:
– Who needs the most help– Most successful sequence of courses– Most / least successful portions of a course
• Notify:– Instructors about their students– Students about their performance compared to peers– Academic advisors about students “at risk”– Staff about their resources (e.g., library use)
13
USE LabUniversity of Michigan
http://umich.edu/~uselab
Milestones
• Stage 1: Extraction & reporting of transaction-level data
• Stage 2: Analysis and monitoring of operational performance
• Stage 3: What-if decision support (e.g., scenario building)
• Stage 4: Predictive modeling & simulation
• Stage 5: Automatic triggers of business processes (e.g., alerts)
14
-- Goldstein & Katz, 2005
USE LabUniversity of Michigan
http://umich.edu/~uselab
!"#$%&"#'()#*+,""#'-#.//#&(0&1&02,+#"$20)($"3
USE LabUniversity of Michigan
http://umich.edu/~uselab
Signals
• Purdue University
• System developed in 2007
• Use of analytics for:
– improving retention
– identifying students “at risk” of academic failure
16
USE LabUniversity of Michigan
http://umich.edu/~uselab
Signals
• NBC Nightly News Clip: http://www.msnbc.msn.com/id/21134540/vp/32634348
• Aired August 31, 2009
17
USE LabUniversity of Michigan
http://umich.edu/~uselab
Signals• 6-10% improvement in retention• 58% of students using report seeking help b/c of
Signals use
• Controlled by the instructor• Course-by-course• Does not show students direct comparison with
their peers
19
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
• Student-controlled
• Designed to promote student agency & self-regulation
• Low impact for the instructor
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• ITS UM-Data Warehouse– One place where all data can be aggregated and reported
out.– Currently includes:
• Student Dataset• eResearch
• Financial• Human Resources• Payroll
• Physical Resources
21
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• M-STEM Academy & USE Lab– 50 Engineering students per cohort– Use CTools data to better inform
mentor team• When do they need mentoring /
direction to resources?
– How do mentors & students make use of this data?
– How does behavior change?
22
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• M-STEM Academy & USE Lab– 50 Engineering students per cohort– Use CTools data to better inform
mentor team• When do they need mentoring /
direction to resources?
– How do mentors & students make use of this data?
– How does behavior change?
22
!"!!#$
%!"!!#$
&!"!!#$
'!"!!#$
(!"!!#$
)!"!!#$
*!"!!#$
+!"!!#$
,!"!!#$
-!"!!#$
%!!"!!#$
./0$-$ ./0$%*$ 123$&$ 123$-$ 123$%*$
!"#$"%
&'(")!*+%&,)
-'&")
../0)123)/*45+%"6)788)
4567/85$9:;35$
<=2>>$?@/32A/$
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
23
Social Network Analysis
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• Tim McKay– Arthur F. Thurnau
Professor of Physics
• Taught into Physics courses for years
• Director: LS&A Honors Program
• Used LS&A ART tool to track student progress.
24
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• Studied nearly 50,000 students over 12 years
• Can predict final grades within 0.5 grade dispersion
• Next project: use an e-coach programmed with analytics data to motivate ALL students
26
USE LabUniversity of Michigan
http://umich.edu/~uselab
Issues to Ponder• Who is the audience?
– Students, Instructors, Advisors, Deans, Staff, Others?
• Who has the control?
– Issues of burden?
• Which views?
• Privacy concerns?
– Is their an institutional obligation?
• Is Learning Analytics just a fad?
• Others?
26
USE LabUniversity of Michigan
http://umich.edu/~uselab
Further Reading• Campbell, J., Deblois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era.
EDUCAUSE Review, 42(4), 40−57.
• Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education, 14(2), 89-97. doi:10.1016/j.iheduc.2010.07.007
• Goldstein, P., & Katz, R. (2005). Academic analytics: The uses of management information and technology in higher education — Key findings (key findings) (pp. 1–12). Educause Center for Applied Research. http://www. educause.edu/ECAR/AcademicAnalyticsTheUsesofMana/156526
• Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588−599. doi:10.1016/j.compedu.2009.09.008.
• Morris, L. V., Finnegan, C., & Wu, S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education, 8(3), 221−231. doi:10.1016/j.iheduc.2005.06.009.
27!"#$#%&'((%)%*+'((,-./012#3-