scaling up learning analytics

Scaling Up Learning Analytics

Dr Doug ClowInstitute of Educational Technology, The Open University, [email protected]@open.ac.uk

#laceproject

22

CC-BY – You are free to:copy, share, adapt, or re-mix;

photograph, film, or broadcast;

blog, live-blog, or post video of

this presentation provided that:You attribute the work to its author and respect the rights and licences associated with its components.

#laceproject

The LACE project

3

K12 Workplace

HEI Community-building through events & communication channels/social media(cross-disciplinary HEI, K12, Workplace)

Technology transfer & best practice

Organized 22 events, and contributed to 33(tutorials, workshops, conferences, etc.

Bett, 20 January 2016

European support action aimed at integrating communities working on LA from schools, workplace and universities

4

LACE Reception & Annual Meeting

5.30–6.30pm Sundown bar

Sign up for an invitation!

5English oak Quercus roburCC BY-SA Jean-Pol GRANDMONT https://commons.wikimedia.org/wiki/File:Quercus_robur_JPG_(d1).jpg

1. What is learning analytics?2. What is your goal?3. ROMA framework4. Apply the framework5. Next steps

Scaling up learning analytics

Learning analytics …

… the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.

8

solaresearch.org

Cloud Chamber at the German Electron Synchrotron DESYPhoto public domain: http://commons.wikimedia.org/wiki/File:DESYNebelkammer.jpg

- Erik Duval http://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/

“collecting traces that learners leave behind and using those traces to improve learning”

Clow, LAK12, 2012

12

The Open University (OU)

Jennie Lee Building, The Open UniversityPhoto CC BY-NC Bea de los Arcos https://flic.kr/p/pR2xWF

To use and apply information strategically to retain students and enable them to progress and achieve their study goals.

13

University of Technology, Sydney (UTS)

Dr Chau Chak Wing Building, UTS, by Frank GehryPhoto CC BY-SA Brickworks Building Products https://flic.kr/p/rt7Rxx

A university where staff and students understand data and, regardless of its volume and diversity, can use and reuse it, store and curate it, apply and develop the analytical tools to interpret it.

What’s your goal for scaling up learning analytics?What’s your policy objective?

Scaling up learning analytics: Discuss with your neighbour

The Rapid Outcomes Modelling Approach (ROMA)

Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in context: overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.

Adapted from: Young, J., & Mendizabal, E. (2009). Helping researchers become policy entrepreneurs: How to develop engagement strategies for evidence‐based policy‐making. ODI Briefing Papers. London, UK: ODI.

Define (and redefine) your

policy objectives

Political context (UTS)

17

Lead: Deputy VC & VP T&L

Pilot projectsAdvanced Analytics InstituteConnected Intelligence Centre

UTS Building 11Photo CC BY extramaster https://flic.kr/p/xXJLbx

Political context & Key stakeholders (OU)

18Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.

http://www.slideshare.net/sbs/designing systemic learning analytics at the open university‐ ‐ ‐ ‐ ‐ ‐ ‐

Apply the framework to your situation


Define (and redefine) your

objectives

resources

23

• Current state of play in UK HE & FE• Code of practice for learning analytics• Network meeting, London, soon

http://analytics.jiscinvolve.org/

Effective Learning Analytics

• LAK conferences• LASI workshops• Flare local meetings• Storm PhD training• Journal of LA• … and more!

www.solaresearch.org

www.laceproject.eu

• Blog bit.ly/lace-blog• Newsletter bit.ly/lace-newsletter• Learning Analytics Review

bit.ly/lace-review-papers• Become an Associate Partner!

26

LACE Reception & Annual Meeting

5.30–6.30pm Sundown bar

Sign up for an invitation!

What’s your next step?


28

Organisations are built to resist change

• Rock and the river?• Big lever

Vernal Falls, YosemitePhoto CC BY Mary PK Burns https://flic.kr/p/6zUzqh

Thanks to:People:• LACE at the OU: Rebecca Ferguson, Andrew Brasher,

Bart Rientes, Simon Cross, Linda Norwood Michelle Bailey, Rebecca Wilson, Evaghn De Souza, Natalie Eggleston, Oliver Millard, Gary Elliot-Citigottis.

• LACE project partners: CETIS (Bolton), OUNL, Skolverket, HIOA, Kennisnet, ITS, ATiT.

• The learning analytics community, including SoLAR, IEDMS, those I’ve met at LAK and LASI

• Bett and venue staff

Funders:• LACE: European Commission 619424-FP7-ICT-2013-11

30

“Scaling Up Learning Analytics” by Doug Clow, Institute of Educational Technology, The Open University, was presented at Bett, London, on 20 January 2016.

@[email protected]

This work was undertaken as part of the LACE Project, supported by the European Commission Seventh Framework Programme, grant 619424.

These slides are provided under the Creative Commons Attribution Licence: http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.

www.laceproject.eu@laceproject

cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/

Policy objectivesOU Strategic Analytics Investment Programme

32

VisionTo use and apply information strategically to retain students and enable them to progress and achieve their study goals.

This vision requires• Discursive changes

to the communication of data and analytics

• Procedural changes in how learners are supported

• Behavioural changes associated with sustainable change in learner support.

Define (and redefine)

your policy objectives

Political contextMapping people and processes



Key stakeholdersOU Strategic Analytics Investment Programme

34

Define(and redefine)


A community of stakeholders working in different areas:• Intervention and Evaluation• Data Usability• Ethics Framework• Predictive Modelling• Learning Experience Data• Professional Data• Student Tools

Key stakeholders are• University administrators• Students• Educators

Desired behaviour changesOU Strategic Analytics Investment Programme

35



VisionTo use and apply information strategically to retain students and enable them to progress and achieve their study goals.

Desired behaviour changes• Staff will use and apply information

strategically• Students will extend their learning

journeys• Students will complete their

learning journeys• Students will set learning goals• Students will work effectively

towards study goals

Engagement strategyOU Strategic Analytics Investment Programme

36



• Data in action is provided to stakeholders through a live portal, enabling them to understand learner behaviour and make adjustments and interventions that will have an immediate positive impact.

• Data on action is a more reflective process that takes place after an adjustment or intervention.

• Data for action takes advantage of predictive modelling and innovation in order to isolate particular variables and make changes based on a variety of analysis tools.

Internal capacity to effect changeOU Strategic Analytics Investment Programme

37



Includes• Recruitment• Capacity building• Developing an ethical framework

for the use of learning analytics.

MonitoringOU Strategic Analytics Investment Programme



Policy objectivesUniversity of Technology, Sydney, Australia

39

VisionA university where staff and students understand data and, regardless of its volume and diversity, can use and reuse it, store and curate it, apply and develop the analytical tools to interpret it.

Teaching and learningEnsure that all stakeholders have the capacity to understand and interpret contemporary data‐rich environments.ResearchEnable researchers to think and act differently when designing their research methodologies and practices.AdministrationIdentify opportunities to obtain, generate, visualize, and communicate data and analyses that can improve core business outcomes.UniversityMine existing institutional data to identify areas that can provide direct evidence or assistance to staff and students.

Political contextUniversity of Technology, Sydney, Australia

40Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in context:

overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.

• Project initiated and led by Deputy Vice‐Chancellor and Vice‐President (Teaching and Learning)

• Pilot projects were completed successfully to secure ongoing funding.

• Critical to the success of the initial pilot projects was the existence of an Advanced Analytics Institute with internationally regarded researchers in big data, data sciences and analytics sciences.

• This enabled the establishment of a Connected Intelligence Centre.

Key stakeholdersUniversity of Technology, Sydney, Australia

41



• 190 staff attended a one‐day ‘Data Intensive University Forum’, thus beginning a university‐wide conversation.

• Working party included Deputy Vice‐Chancellors, a senior member of library staff, and representatives of all faculties and administrative areas with relevant expertise

• Stakeholder buy‐in and ongoing participation in the project have been critical to its success.

Desired behaviour changesUniversity of Technology, Sydney, Australia

42



• Provide information that can be used to decrease student attrition

• Provide a more detailed understanding of factors affecting low pass rates in subjects with very high failure rates over time

• Provide students with more information about their own study and engagement patterns

• Enable a more fine‐grained understanding of the influences of a range of possible interventions on pass rates and completions

• Provide valuable input to learning futures projects

Engagement strategyUniversity of Technology, Sydney, Australia

43



• Give attention to institutional culture, ensuring engagement and buy‐in from key stakeholders through good communication and governance

• Invest in pilot projects of significant concern to the university and reporting of outcomes

• Invest in infrastructure: tools, applications, services

• Invest in expertise: recruitment of critical staff

• Provide leadership and engage institutional leaders

Internal capacity to effect changeUniversity of Technology, Sydney, Australia

44



• Students and staff must be sufficiently numerate and equipped to make use of the analyses that analytics projects produce.

• A subject has been developed and trialled with staff

• The course develops students’ ability to engage with complex, extended arguments underpinned by numerical data as a key to participation as informed citizens in issues of significance to our culture and society

• The course available as an elective and will become compulsory

MonitoringUniversity of Technology, Sydney, Australia

45

• UTS has been engaged in a variety of learning analytics projects to assess scale and impact

• For example, the Outreach Programme rings as many commencing undergraduate students as possible. Early results consistently show a significant decrease in attrition in the group of students contacted.

