open learning analytics panel presentation - lak 15
TRANSCRIPT
Open Learning Analytics (OLA)THE 5TH INTERNATIONAL LEARNING ANALYTICS & KNOWLEDGE CONFERENCE
Josh BaronASSISTANT VICE PRESIDENT
INFORMATION TECHNOLOGY FOR DIGITAL EDUCATION
MARIST COLLEGE
Alan BergCOMMUNITY OFFICER
APEREO LEARNING ANALYTICS INITIATIVE
Sandeep JayaprakashLEARNING ANALYTICS SPECIALIST
MARIST COLLEGE
LESSONS LEARNED – VALUE OF AN OPEN PLATFORM
Open Academic Analytics Initiative
OAAI: Overview and ImpactEDUCAUSE Next Generation Learning
Challenges (NGLC)
Funded by Bill and Melinda Gates Foundations
$250,000 over a 15 month period
Goal: Leverage Big Data concepts to create an
open-source academic early alert system and
research “scaling factors”
Student Aptitude Data (SATs, current GPA, etc.)
Student Demographic Data (Age, gender, etc.)
Sakai Event Log Data
Sakai Gradebook Data
Predictive ModelScoring
Identifies students “at risk” to not complete
course
SIS
Dat
aLM
S D
ata
OAAI Early Alert System Overview
Intervention Deployed“Awareness” or Online
Academic Support Environment (OASE)
“Creating an Open Academic Early Alert System”
Model DevelopedUsing Historical Data
Step #1: Developed model using historical data
Academic Alert Report (AAR)
Research DesignDeployed OAAI system to 2200 students across four institutions• Two Community Colleges
• Two Historically Black Colleges and Universities
Design > One instructor teaching 3 sections• One section was control, other 2 were treatment groups
Each instructor received an AAR three times during the semester:• Intervals were 25%, 50% and 75% into the semester
Institutional Profiles
Spring ’12 Portability Findings
Fall ’12 Portability FindingsConclusion
1. Predictive models are more
“portable” then anticipated.
2. It is possible to create generic
models that are then “tuned” for
use at specific types of
institutions.
3. It is possible to create a library of
open predictive models that
could be shared globally.
Intervention Research Findings Final Course Grades
Analysis showed a statistically significant
positive impact on final course grades
• No difference between treatment groups
Saw larger impact in spring then fall
Similar trend amount low income students
50
60
70
80
90
100
Awareness OASE Control
Fin
al G
rad
e (%
)Mean Final Grade for "at Risk" Students
Instructor Feedback
"Not only did this project directly assist my students by guiding students to resources to help them succeed, but as an instructor, it changed my pedagogy; I became more vigilant about reaching out to individual students and providing them with outlets to master necessary skills.
P.S. I have to say that this semester, I received the highest volume of unsolicited positive feedback from students, who reported that they felt I provided them exceptional individual attention!
JAYAPRAKASH, S . M., MOODY, E . W., LAURÍA, E . J . , REGAN, J . R . ,
& BARON, J . D. (2014). EARLY ALERT OF ACADEMICALLY AT -RISK
STUDENTS: AN OPEN SOURCE ANALYTICS INITIATIVE . JOURNAL
OF LEARNING ANALYTICS, 1(1) , 6 -47.
More Research Findings…
Lesson Learned #1Openness will play a critical
role in the future of Learning Analytics
Intersections between openness and Learning AnalyticsOpen Source Learning Analytics Software• Weka, Kettle, Pentaho, R, Python etc.
Open Standards and APIs for Learning Analytics• Experience API, IMS Caliper/Sensor API
Open Models - Predictive models, knowledge maps, PMML etc.
Open Content/Access – Journals, whitepapers, policies documents
Openness or Transparency with regards to Ethics/Privacy
NOT anti-commercial – Commercial ecosystems help sustain OSS
Lesson Learned #2Software Silos Limit Learning Analytics
Software Silos vs. PlatformsMany learning analytics solutions today are“tool” or “software-centric”• Analytics tools are built into existing software such as the
Learning Management System (LMS)
Can make it harder to capture data and integrate across systems (limits Big Data)
A platform solution would allow institutions to collect data from across many systems• A “modularized platform” approach allows institutions to use all or just some components
• Integration points allow data to “flow” in for processing and results to flow out
Apereo Learning Analytics Initiative (LAI)Goal: Operationalize outcomes from Learning Analytics research as means to develop, maintain and sustain an open platform for Learning Analytics
Current Proof-of-Concept Projects◦ University of Amsterdam – Larrisa (open-source Learning Record Store)
◦ Marist College – Learning Analytics Processor (LAP)
◦ Uniformed Services University – OpenDashboard
◦ Sinclair Community College – Student Success Plan
◦ Unicon – OpenLRS and commercial support services
Contact: Alan Berg, Community Officer Email: [email protected], Wiki Page: https://confluence.sakaiproject.org/x/rIB_BQGitHub: https://github.com/Apereo-Learning-Analytics-Initiative
Strategic Vision: Open Learning Analytics Platform
Collection – Standards-
based data capture from any potential source using Experience API and/or IMS Caliper/Senor API
Storage – Single repository
for all learning-related data using Learning Record Store (LRS) standard.
Analysis – Flexible Learning
Analytics Processor (LAP) that can handle data mining, data processing (ETL), predictive model scoring and reporting.
Communication –
Dashboard technology for displaying LAP output.
Action – LAP output can be fed
into other systems to trigger alerts, etc.
Library of Open Models
Learning Record Store & Data Collection• OpenLRS is a secure, standards-based,
standalone Learning Record Store built to fill the need for a high i/o storage mechanism for an open learning analytics environment
• Technical Stack• Spring-Boot
• Pluggable Datastores (redis, elasticsearch, mongodb)
• xAPI integrations to get activity streams
• Roadmap• Integration & Support for IMS Caliper
Learning Analytics Processor Architecture
OAAI Early Alert & Risk Assessment API
Sakai Admin tool
activities.csv
grades.csv
Learning Analytics Processor (LAP)
Student ID, Course ID, Risk Rating
Demographics from SIS
Go!
grabs files
OAAI XML
Kettle pipeline applies model
outputs results
..
.
.
------------------ EXTRACT -------- TRANSFORM ------- LOAD ---------
RESTful API
Student Success Plan
• Case Management Software
• Academic Advising Tools (MAP)
• Early Alert / Faculty Access (EAL)
• Student Interface (Tasks, MAP, Self
Help)
• Student Information System (SIS)
Integration
• Reporting Tools / Data Collection
Student Success Plan
Open Dashboard
• Web application that provides a framework
for displaying analytics visualizations and
data views called “cards”.
• Cards represent a single discrete
visualization or data view but share an API
and data model
• LTI compliant
• Widget(Card) library for Learning Analytics
LAK15 Hackathon - Open Dashboards
Early Alert Insights ChartCourse Engagement Pathways – Resource & Content Access
Demo
Demo Overview• Three core components of a collection of
open source applications and services that
represent the “Analytics Diamond”
• Can be used individually or collectively
• Work with a shared infrastructure and data
model
Technologies:• AngularJS
• Spring-Boot
• Pluggable Datastores
(redis, elasticsearch, mongodb)
Sakai
OpenLRS
Learning
Analytics
Processor
Open
Dashboard
xAPI
LTI
API
API
AWS
Local
Engaging with the Community
Engaging with Apereo Learning Analytics Initiative (LAI)
We believe in Do-ocracy. If you see an opportunity or area of
enrichment then you should take leadership and the community will support you. Bear this in mind as you ask me questions
ExamplesAlan - community officer, organizes hackathons & workshopsSandeep - warding incubation process, analyticsPatrick - communications officer, student requirementsKate - marketing/communications, EvangelistJosh - many roles (not even going to start)Gary - builds the living daylights out of LAI. Sanity check, etc.
Engaging with Apereo Learning Analytics Initiative (LAI)Where to start? LEVEL 1 NINJA (in no particular order):
• Review the homepage
https://confluence.Sakaiproject.Org/display/LAI/learning+analytics+initiative
• Read the notes from the regular meetings
• Join the mailing list: [email protected]
(subscribe by sending a message to [email protected])
• Join the calls (every other Wednesday) :
https://confluence.Sakaiproject.Org/display/LAI/community+hangouts
• Review github: https://github.Com/apereo-learning-analytics-initiative
• Meet us at a BOF or online.
• Take on a role on a subject you care about
Engaging with Apereo Learning Analytics Initiative (LAI)Where to start? LEVEL 2 NINJA (in no particular order):
• Buy us/ME beer
• Host a hackathon or workshop
• Present at a conference
• New project / consortium building / grant proposal
• Enrich a current product
• Add parts to Apereo LAI
• Consider co-developing
• Act as a communication channel between organizations
• Surf, JISC , Apereo
• SoLAR, LACE
• Unicon,uva,marist,hull,oxford, <<your name here>>
Discussion and Q&AJ O S H : J O S H . B A R O N @ M A R I S T. E D U
A L A N : A . M . B E R G @ U VA . N L
S A N D E E P : S A N D E E P. J AYA P R A K A S H 1 @ M A R I S T. E D U