designing and evaluating incidental learning
TRANSCRIPT
www.maseltov.e
u
Designing and evaluating incidental learning
Andrew Brasher, Ann Jones, Agnes Kukulska-Hulme, Mark Gaved, Eileen Scanlon, Lucy Norris
IET, The Open University
www.maseltov.e
uMaseltov project
“Mobile Assistance for Social Inclusion and Empowerment of Immigrants with Persuasive Learning Technologies and Social Network Services” www.maseltov.eu
Joanneum Research, AustriaCURE - Centre for Usability Research & Engineering, AustriaUniversity of Applied Sciences, FH Joanneum, AustriaAthens Information Technology, Greecebusuu.com -, Spain
Universitat Oberta de Catalunya, SpainOpen University, UKCoventry University, UK Czech Technical University, Czech RepublicTelecom Italia SpA, ItalyFluidtime Data Service GmbH, AustriaFundacian Desarrollo Sostenido, SpainVerein Danaida, AustriaMigrants Resource Centre, UKPearson Publishing, UK
www.maseltov.e
uContents
• Overview of the Incidental Learning Framework• Evaluation & use
– Workshop at plenary meeting– expert researchers– Iliinsky’s work
• Challenges–Use in evaluation: analysis of MK trial data
• Conclusions
www.maseltov.e
uResearch questions
Research question driving the development of the Incidental Learning Framework
www.maseltov.e
uGoals of the Incidental Learning Framework
• to facilitate the creation of technology rich learning opportunities which emanate from incidental learning i.e. learning that is spontaneous and unplanned;
• to encourage links and triggers to structured and reflective learning to back up and deepen learning that happens incidentally;
• to enhance analysis and evaluation of incidental learning;
• to support discussions around software system design;
• to facilitate the communication of learning design ideas both visually and textually.
www.maseltov.e
uReview of frameworks
• Design frameworks– to support the planning and instantiation of new
learning events and situations• Analytical frameworks
– for analysing, understanding and evaluating learning events
• Learner models – to describe the state of the learner so that a
system can react appropriately
www.maseltov.e
uReview of frameworks
• Design frameworks• Analytical frameworks• Learner models
Design framework
Analytical framework
www.maseltov.e
uExamples of frameworks
Analytical• Theory of Learning for
the Mobile Age (Sharples et al., 2007)
• A Pedagogical Framework for Mobile Learning (Park, 2011)
Design• Ecology of Resources
design framework (Luckin, 2010)
• Four dimensional framework (de Freitas et al., 2010)
• Language learning defined by time and place(Kukulska-Hulme, 2012)
Learner model• Advances in learner &
skill modeling in intelligent learning environments (Desmarais and Baker, 2011)
www.maseltov.e
uKolb’s learning cycle
www.maseltov.e
uILF version 1
Outcomes
Social
Time
Tasks
Tools
Place
Learner’s journey
Tim
e
Incident: understanding notice board
Language tool
Start
TextLens
Task: finding station
Task: asking directions
Reflect/plan
Structured learning
Language Tool/Social
Task: navigating
Navigation tool
Language Tool
Language Tool/Social
www.maseltov.e
uExample, stage 1
www.maseltov.e
uExample, stage 2
www.maseltov.e
uEvaluation
• Workshop: adding detail to scenarios • Expert focus group
www.maseltov.e
uEvaluation
The framework • “generates helpful discussions about the learner
journeys which immigrants might make”• Exposes “misunderstandings and different
interpretations of the same problem space through different domain experts views”
• The current representation was sufficient to engage participants and encourage discussion.
(Gaved et al., 2013)
www.maseltov.e
uEvaluation
• Ordering of dimensions was questioned• Outer dimension of “learner’s journey” was
debated: - too general? - how to show e.g. social inclusion in to a new community
(Gaved et al., 2013)
www.maseltov.e
uExtension ideas (1)
Visualisation of specific learner journeys, showing opportunities for planned and incidental language learning
Gaved et al. (2012)
www.maseltov.e
uExtension ideas (2)
Visualisation of the range of affordances that may be associated with particular places that a MASELTOV service user is likely to encounter during their daily activities
www.maseltov.e
uExtension ideas (3)
Visualisation of the range of dimensions associated with motivation and concerns that might be felt by the service user while engaging with mobile situated language learning
www.maseltov.e
uChoosing visual properties
Iliinsky, N. (2013)
www.maseltov.e
uPlace
• Not just a location - also contextual information. • ‘A Place is a geographical and historical context for
Names and Locations’ (Gillies, 2011).• a ‘Place’ has a name and a location, and each may
be applicable for a particular time period. “Andrew’s workplace” currently has - Name: “Jennie Lee
Building” - Location: map
reference SP 88626 37058
www.maseltov.e
uLearning outcomes
Categorical Related
www.maseltov.e
uLinear view
Place
Task
Tools
Outcomes
Social
Time Evening
Language learning
Practise work related vocabulary
Understand work related vocabulary
Sleep/Break19.00 23.00 07.00
Language learning
Practise past tense
Communicate successfully in work related situation
Breakfast08.00
www.maseltov.e
uData gathering
• Pre trial questionnaire:paper
• Post trial interviews: face-to-face
E.g. Lesson access duration: 2’39s
Synchronous & located with MApp use
• Event data from the MApp
• Usage of other apps
• Geo contextual data
Asynchronous & not located
with MApp use Weekly survey: online
www.maseltov.e
uParticipant’s use of Mapp services
I have really liked the English course, it is very practical for the times when you have to use English for different occasions..it has helped me a lot The second is the translation tool because now I take a photo of everything I don't understand and at that moment I know what it says ... It has helped me a lot.
I have learned a lot more English language
I have learned to develop myself in different aspects of speaking English with the help of the course
www.maseltov.e
uUse in analysis
• Example language lesson data key value timestamplesson transport 22/01/2015 13:22publication mas_trav 22/01/2015 13:22score 90.90908813 22/01/2015 13:22lesson plan_journey 24/01/2015 18:41publication mas_trav 24/01/2015 18:41score 0 24/01/2015 18:41lesson plan_journey 24/01/2015 18:41publication mas_trav 24/01/2015 18:41score 0 24/01/2015 18:41lesson transport_chaos 24/01/2015 20:07publication mas_trav 24/01/2015 20:07score 100 24/01/2015 20:07lesson body_and_health 24/01/2015 20:59publication mas_health 24/01/2015 20:59score 85.7142868 24/01/2015 20:59
www.maseltov.e
uConclusions
• Development of ILF as a design tool –Work to do is guided by literature and
experimental findings• Development of ILF as an analytical tool
–Data available complex and multi-faceted;– Initial experiments started; –Transfer of design view not straightforward?–Learner models?;–Suggestions welcome!
Levels of abstraction?
Level of detail?
www.maseltov.e
uTo do
• Include stuff from conclusions slide from 2012 Calrg presentation
• …feedback from eval D7.12.• Other stuff….
• Issues – learning outcomes can be shown at gps locations, but need to show learning journey
• To point to new developments – show new version
www.maseltov.e
uExtensions
• D7.1.2 i.e.- SCAMP- Learning journey
www.maseltov.e
uReferences
• de Freitas, S., Rebolledo-Mendez, G., Liarokapis, F., Magoulas, G. & Poulovassilis, A. 2010. Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. British Journal of Educational Technology, 41, 69-85.
• Desmarais, M. C. & Baker, R. S. J. D. 2011. A review of recent advances in learner and skill modeling in intelligent learning environments. User Modeling and User-Adapted Interaction, 22, 9-38.
• Kukulska-Hulme, A. 2012. Language learning defined by time and place: A framework for next generation designs. In: D´ıAZ-VERA, J. E. (ed.) Left to My Own Devices: Learner Autonomy and Mobile Assisted Language Learning.
• Iliinsky, N. (2013). Choosing visual properties for successful visualizations. Retrieved 24/4/2014, from http://bit.ly/successfulvis
• Luckin, R. 2010. Re-Designing Learning Contexts: Technology-Rich, Learner-Centred Ecologies (Foundations and Futures of Education). 208.
• Park, Y. 2011. A Pedagogical Framework for Mobile Learning: Categorizing Educational Applications of Mobile Technologies into Four Types. International Review of Research in Open and Distance Learning, 12, 78-102
• Sharples, M., Taylor, J. & Vavoula, G. 2007. A Theory of Learning for the Mobile Age. In: Andrew, R. & Haythornwaite, C. (eds.) The Sage Handbook of Elearning Research. London: Sage.