eurocall2014 speakapps presentation - speakapps and learning analytics
DESCRIPTION
Authentic oral language production: A conceptual framework for implementing learning analytics within the SpeakApps projectTRANSCRIPT
Authentic oral language production and interaction in CALL: An evolving conceptual framework for the use of learning analytics
within the SpeakApps project
www.speakapps.eu
Project Overview SpeakApps 2 speakapps.eu
Lifelong Learning Programme Nov 2013 – Oct 2014
KA2 LANGUAGES, Accompanying Measures
Development of tools and pedagogical tasks for oral production and spoken interaction
Partners Associated Partners • Institut Obert de Catalunya• University of Southern Denmark• University of Nice• University of Jÿvaskÿla• Ruhr-Universitat Bochum• Polskie Towarzystwo Kulturalne "Mikolaj Kopernik“• Fundació Pere Closa
SpeakApps...
Partners and Associate Partners with various competencies, skills, backgrounds and contexts: Computer Assisted Language Learning Traditional universities and open/distance universities Learning technologists, linguists, pedagogists, information
technologists and programmers – Educators! Variety of linguist backgrounds and contexts including lesser used
languages, to national/international languages
Challlenging at times but extremely engaged Binding was the conceptual and theoretical framework of
the project
An Evolving Conceptual Framework*
Maxwell (2013) *
Fundamental concepts, assumptions, theories, experience and expectations
Theories, Concepts, Assumptions and Experience Materials that promote meaningful language interaction
opportunities (Brandl, 2002) Authentic materials provide learners with the opportunity to fulfil a
social purpose in the language community for which it was intended (Grellet, 1981; Lee, 1995; Little, Devitt, & Singleton, 1989)
* Maxwell, J.A. (2013) Qualitative Research Design an Interactive Approach, 3rd Ed., Thousand Oaks: Sage
Conceptual Framework…
Theories, Concepts, Assumptions and Experience Task-based approaches closely aligned with action-oriented approaches to language derived
from sociocultural and cultural historical activity theoretical to language teaching and learning (see for example Blin, 2010; Blin & Appel, 2011; Blin & Thorne, 2011)
Teacher and learner agency being at the centre of curriculum and task design (van Lier, 2004; Engeström, 2006; Lipponen & Kumpulainen, 2011)
Emphasis in many online language classes is usually placed on three of the core language
skills writing, reading and listening (Appel, Santanach, Jager, 2012)
Developing oral language skills is accepted as being problematic in part because of time
constraints independent of the learning environment
Ephemeral nature of speaking makes it difficult for both students and teachers to provide
feedback and to remember what was said
Provide a way of offering students more and enhanced practice outside the classroom
Data-driven decision making
Introduction to Analytics
Pervasiveness of technology has facilitated the collection of data and the creation of a variety of data sets, big data
Application has spread across domains and prompted business and societal applications
Google analytics – Adwords etc. (http://www.web2llp.eu/) Smart cities – Policing using predicative models to prevent crime
(Santa Cruz police department - http://edition.cnn.com/2012/07/09/tech/innovation/police-tech/)
Ultimate aim to inform decision making from resource allocation to improved services etc.
How? By using a variety of data mining techniques for discovery of patterns and/ or validation of hypothesis/claims
Educational Analytics
Data available in education from a variety of sources LMS Institutional systems Google for education User generated content, social networks
Ferguson (2012)* provides a useful overview of the educational analytics field and suggests the following divergence in focus between:
Educational data mining focuses on the technical challenge: How can we extract value from these big sets of learning-related data?
Learning analytics focuses on the educational challenge: How can we optimise opportunities for online learning?
Academic analytics focuses on the political/economic challenge: How can we substantially improve learning opportunities and educational results at national or international levels?
)
Educational Analytics…
Long and Siemens (2011:32) – aptly describes the challenge:
But using analytics requires that we think carefully about what we need to know and what data is most likely to tell us what we need to know.
(http://net.educause.edu/ir/library/pdf/ERM1151.pdf)
* See: Ferguson, Rebecca (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6) pp. 304–317.
Analytics Process*
Establish the objective or claim of the LA exercise
Three Stage iterative process 1. Data collection and Pre-processing
Data preparation and cleaning, removal of redundant data etc. time stamps Application of established and evolving data mining techniques to complete
this2. Analytics and action
Explore/analyse the data to discover patterns Data visualisation and representation
3. Post-processing Adding new data from additional data sources Refining the data set Identify new indicators/metrics Modify variables of analysis Choose a new analytics method
* See: Chatti, M.A., Dyckhoff, A.L., Schroeder, U. and Thüs, H. (2012) ‘A reference model for learning analytics’, Int. J. Technology Enhanced Learning, Vol. 4, Nos. 5/6, pp.318–33
What? What kind of data does the system gather,manage, and use for the analysis?
Who? Who is targeted by the analysis?
Why? Why does the system analyse the collected data?
How? How does the system perform the analysis of the collected data?
* See: Chatti, M.A., Dyckhoff, A.L., Schroeder, U. and Thüs, H. (2012) ‘A reference model for learning analytics’, Int. J. Technology Enhanced Learning, Vol. 4, Nos. 5/6, pp.318–33
10
Learning Analytics Reference Model*
What? Data and Environment: Which systems Structured and/or unstructured data
Who? StakeholderTeachersStudents Instructional designers Institutional stakeholders
11
Learning Analytics Reference Model*
Why? ObjectiveMonitoring and analysisPrediction and intervention Tutoring and Mentoring Assessment and feedback Adaptation Personalization and recommendationReflection
Challenge to identify the appropriate indicators/metrics
12
Learning Analytics Reference Model*
How? MethodStatistics: most LMS produce statistics based on
behavioural data Data mining techniques and others (long list)
Classification (categories known in advance) many different techniques from Data mining
Clustering (categories created from the data similar data clustered together based on similar attributes not known in advance)
Association rules mining leads to the discovery of interesting associations and correlations within data
Social Network Analysis…
13
Learning Analytics Reference Model*
Claim is that student and teacher oral & video recordings should be time limited to maintain the attention of the listener
Currently we recommend a maximum of one minute for learner recordings and two minutes for teacher recordings
At present this claim is based on experience Evidence to support decision-making which will impact:
Resource allocation to refine the tool – time limitation Learner agency Instructional and task design
14
SpeakApps Pilot
15
SpeakApps Pilot & LA Reference Model
What? What kind of data does the system gather, manage, and use for the analysis
LMS data i.e. technical information i.e. device, browser, versions etc.behavioural data i.e. time stamps, click tracking, user generated content such as surveys, peer-feedback
Who? Who is targeted by the analysis?
Students, Teachers, Instructional Designers and Developers
Why? Why does the system analyse the collected data?
Students – adapt Teachers – tutoringInstructional designers – adapt taskDevelopers – interface adaptation
How? How does the system perform the analysis of the collected data?
Statistics based on behavioural data and the analysis of user generated data – possible qualitative follow-up
Data Types and Sources Aggregate and integrate data produced by students from multiple
sources Challenge to source, combine and manipulate data from a wide variety of
sources and in many formats
Over reliance on behavioural data from LMS, varied data sources Structured data i.e. data from LMS etc., other institutional systems, connected
devices Unstructured data i.e. other sources user generated content/data i.e.
Facebook - social network modelling, online dictionaries, translation tools thesaurus etc.
16
Concluding Remarks
Student Agency in LA Students as active agents – voluntarily collaborate in
providing and accessing data
Designing interventions (if appropriate in the context) and the agency of the student: Student at the centre of interpretation Data representation to facilitate interpretation Requires specific skills of interpretation
17
Concluding Remarks…
Ethical and Educational Concerns
Use of data based on transparent opt-in permission of students following established research principles Students understand that data is collected about them and actively
buy-in Privacy and stewardship of data
Emphasis on learning as a moral practice resulting in
understanding rather than measuring (Reeves, 2011)
18
Concluding Remarks…
Challenging to realise the specific objectives of stakeholders Teachers v Instructional designers in online education
Designing and focusing indicators
Quantitative and qualitative subjectivity and tension Answering the why?
Sentiment analysis and forums
19
Concluding Remarks…
We welcome your input and thank you
20
SpeakApps – www.speakapps.eu