Download - EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links
![Page 1: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/1.jpg)
Learning design and learning analytics: building the linksEMMA Summer School, Ischia, July 2015
![Page 2: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/2.jpg)
Rebecca Ferguson
• The Open University (200,000 learners)• Informal learning:• YouTube (28.9 million views)
FutureLearn (1.8 million learners)OpenLearn (37.9 million visits)
• Making use of big data for 45 years
• Learning analytics research / events• LACE project – learning analytics
community exchange
2
Lead on MOOC evaluation at The Open University, UK
http://www.laceproject.eu/
![Page 3: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/3.jpg)
Workshop overview
09.45 IntroductionLinking learning analytics, learning design and MOOCs10.15 Questions and answers10.25 Move to workshop area10.30 Group work11.15 Workshop end
3
You can view and download the original version of these slides athttp://www.slideshare.net/R3beccaF
![Page 4: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/4.jpg)
Learning analyticsWhat are they, and how are they used?
4
![Page 5: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/5.jpg)
What are learning analytics?
High-level figuresBrief overviews for internal and external reports
Academic analyticsFigures on retention and success, for the institution to assess performance
Learning analyticsUse of big data to provide actionable intelligence for learners and educators
5
![Page 6: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/6.jpg)
Educators use analytics to
• Monitor the learning process• Explore student data• Identify problems• Discover patterns• Find early indicators for success• Find early indicators for poor marks or drop-out• Assess usefulness of learning materials• Increase awareness, reflect and self reflect• Increase understanding of learning environments• Intervene, supervise, advise and assist• Improve teaching, resources and the environment
6
Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
![Page 7: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/7.jpg)
Learners use analytics to
• Monitor their own activities and interactions• Monitor the learning process• Compare their activity with that of others• Increase awareness, reflect and self reflect• Improve discussion participation• Improve learning behaviour• Improve performance• Become better learners• Learn!
7
Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
![Page 8: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/8.jpg)
Analytics example: UK schools
8
• Aligned with clear aims• Huge and sustained effort• Agreed proxies for learning• Clear and standardised visualisation• Driving behaviour at every level
BUT• Stressed, unhappy learners• Analytics with little value for learners or educators• Omission of key areas, such as collaboration
![Page 9: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/9.jpg)
Analytics example: Course SignalsDeveloped at Purdue University
9
Arnold, K E, & Pistilli, M (2012). Course Signals at Purdue: Using Learning Analytics To Increase Student Success. Paper presented at LAK12, Vancouver, Canada.
![Page 10: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/10.jpg)
Analytics example: SNAPPNetwork analysis
10
![Page 11: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/11.jpg)
Analytics example: iSpotHeading
11
![Page 12: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/12.jpg)
Making the links between learning
analytics and learning design
12
![Page 13: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/13.jpg)
Learning design in MOOCs
●Puts the learning journey at the heart of the design process●Provides a set of tools and information to support a learner-
activity based approach ●Helps to show the costs and performance outcomes of
design decisions●Enables the sharing of best practice●Helps MOOC designers to choose and integrate a coherent
range of media, technologies and pedagogies
●Enables a consistent and structured approach to review and analytics
13
Mor, Y, Ferguson, R, & Wasson, B. (2015). Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology, 46(2), 221-229.
![Page 14: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/14.jpg)
MOOC learning design tools
• MOOC design template
• MOOC planner
• MOOC map
• Journey planner
14
![Page 15: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/15.jpg)
Design template analytics
15
Learning outcome How this is assessed
1. Be able to define an ecosystem.
2. Have joined the iSpot community and obtained identifications for animals, plants or fungi.
1. Multiple choice. Week 1, question 5
2. Self report.
Analytics
1. How many attempted that question? How many got it right 1st / 2nd / 3rd time? How many followed the link back to resources?
2. Access to iSpot data. Use of MOOC hashtag. Persistence over time. Ethical implications of tracking off-site.
Short description of course and learning outcomes
![Page 16: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/16.jpg)
MOOC planner
• Delivered• Reflection• Collaboration• Conversation• Networking• Browsing• Assessment
16
Blocking out types of learning activity
Conole, Gráinne. (2010). Learning design – making practice explicit. Paper presented at ConnectEd, Sydney, Australia. http://cloudworks.ac.uk/cloud/view/4001
![Page 17: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/17.jpg)
MOOC planner analytics
Delivered Content (reading, watching, listening and observing)Analytics: amount of content viewed, dwell time
Reflection (thinking, considering and reflecting)Analytics: returns to the same material, reflection exercises completed, quality of reflection
Collaboration (constructing, collaborating, defining and engaging)Analytics: collaboration exercises completed, quality of collaboration
Conversation (debating, arguing, questioning, discussing…)Analytics: number and length of contributions, quality of discussion
Browsing (exploring, searching, finding and discovering)Analytics: Number of click-throughs to external links, number of visits, number of resources
Assessment (answering, presenting, demonstrating, critiquing…)Analytics: Assessments completed, scores, dwell time on hints, persistence in answering questions
17
![Page 18: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/18.jpg)
MOOC map analytics
●How long did you expect learners to spend on these key elements?
●How long did learners actually spend on the key elements
●How many missed out these elements?
●How many jumped ahead to these elements?
●Which types of element are consistently (un)popular?
●How many left the MOOC at these points?
18
The MOOC map identifies key elements of the course
Ass
imila
tive
Info
rmat
ion
Han
dlin
g
Pro
duct
ive
Exp
erie
ntia
l
Ada
ptiv
e
Com
mun
icat
ive
Ass
essm
ent
Org
anis
atio
n
0
100
200
300
400
500
600
Min
utes
![Page 19: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/19.jpg)
MOOC journey planner analytics
19
Relationships between tools, resources, activities & narrative
A framework for data collection
![Page 20: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/20.jpg)
Analytics to solve problemsAnalytics could filter discussions or group learners
20
You have been actively engaged in the discussions, which is excellent, thank you, but with more than 23,000 participants it
means that our responses and comments risk getting lost.
This will be primary school material for some of you and exactly the opposite for others. It is just not possible to tailor the
material to each of you […]Introduction to Forensic Science:University of Strathclyde
![Page 21: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/21.jpg)
Start with the pedagogy
• How do people learn?
• How can we use data to facilitate that process in our MOOC?
• Which elements are learners struggling with?
• Which sections engage them the most?
• What prompts them to ask questions?
• Are they finding assessment challenging?
• What misconceptions have learners shown?
• Are there any accessibility issues?
• How can analytics be used to obtain desired learning outcomes?
21
![Page 22: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/22.jpg)
Learning analytics and design
Learning design – helping to identify useful analytics●What do learners need to know in order to –
network, collaborate, browse or reflect?●What do educators need to know to support them?
Learning design – helping to identify gaps in the data●What data do we need to collect? Learning design – helping to identify gaps in our toolkit●Which design elements can we look at easily?●Which ones still pose problems?
Learning design – helping to frame & focus analytics questions●What did they learn?… in relation to learning outcomes●Were they social?... when they were collaborating●Did they share links?... when encouraged to browse●Did they return to steps?... when encouraged to reflect
22
Making the links
![Page 23: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/23.jpg)
Q & ANext: if you would like to exlore the relationship between learning design and learning analytics
further, join us downstairs in the workshop area
23
![Page 24: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/24.jpg)
Learning outcomes
24
Make a note of the learning outcomes
Promotional video for Lancaster University’s FutureLearn MOOC: Soils: Introducing the World Beneath Our FeetYou can access this without registering at futurelearn.com
![Page 25: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/25.jpg)
Learning outcomes
By the end of the course, learners should know:●What soils are made of●How soils are formed●The variety of soils around the world●Why soil is so important●What life exists within soils●Threats to soil●Why we need to protect soil
25
FutureLearn MOOC from Lancaster University
![Page 26: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/26.jpg)
Exploring, searching, finding
26
Screenshot from Lancaster University’s FutureLearn MOOC: Soils: Introducing the World Beneath Our FeetYou can currently register for this course free of charge at futurelearn.com
![Page 27: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/27.jpg)
Exploring, searching, finding
27
Screenshot from Lancaster University’s FutureLearn MOOC: Soils: Introducing the World Beneath Our FeetYou can currently register for this course free of charge at futurelearn.com
![Page 28: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/28.jpg)
What are soils made of?Delivered Content (reading, watching, listening and observing)Ask learners to watch a video lecture on soils
Reflection (thinking, considering and reflecting)Ask ‘What are the key constituents of soils and where do they come from’, and show learners a video
Collaboration (constructing, collaborating, defining and engaging)Ask learners to put together what they already know, and come up with a list of the key constituents of soil
Conversation (debating, arguing, questioning, discussing…)Ask learners to discuss an audio recording about soil
Browsing (exploring, searching, finding and discovering)Point learners to some key resources and ask them to develop an initial list of the constituents of soil
Assessment (answering, presenting, demonstrating, critiquing…)Ask learner to post a list of five constituents of soil, and then comment on another learner’s list 28
![Page 29: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/29.jpg)
Types of learning behaviour
Delivered Content (reading, watching, listening and observing)
Reflection (thinking, considering and reflecting)
Collaboration (constructing, collaborating, defining and engaging)
Conversation (debating, arguing, questioning, discussing…)
Browsing (exploring, searching, finding and discovering)
Assessment (answering, presenting, demonstrating, critiquing…)
29
![Page 30: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/30.jpg)
Group activity15 minutes
https://www.futurelearn.com/
Take each learning outcome of the soil science MOOC and briefly outline a different type of
learning activity for each one (delivered, reflection, conversation,
collaboration, browsing or assessment).
30
![Page 31: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/31.jpg)
Example analytics questionsWatch a video lecture on soilsIs the video engaging learners? How long did they watch? How many skipped it?Reflect on key constituents of soilsAre learners engaging in behaviours that indicate they are reflecting? Did they return to the video? Did they pause the video?Discuss an audio recording about soilDoes this work as a discusssion activity? How many engaged? How detailed were the comments? Were they key words in the discussion the same as key words in the recording?Develop an initial list of the constituents of soilWhat mistakes do I need to correct? What elements did they identify? Which elements dd they miss out? Post a list of five constituents of soil, and then commentWhat type of general feedback can I provide as a model for this activity in future? What were common mistakes? What did they generally get right?
31
![Page 32: EMMA Summer School - Rebecca Ferguson - Learning design and learning analytics: building the links](https://reader035.vdocuments.us/reader035/viewer/2022081517/58eeda981a28ab62448b46a3/html5/thumbnails/32.jpg)
Group activity15 minutes
https://www.futurelearn.com/
For each of your learning activities, identify questions you would like answered in order
to improve learning and teaching on the MOOC
What types of data would help you to answer these questions?
32