how do learning analytics “act” in education?
DESCRIPTION
ESRC Code Acts in Education seminar series http://codeactsineducation.wordpress.com University of Edinburgh, 9th May 2014TRANSCRIPT
How do Learning Analytics “act” in Education? Simon Buckingham Shum Knowledge Media Institute, Open University UK simon.buckinghamshum.net twitter @sbskmi
ESRC Code Acts in Educa/on seminar series h.p://codeactsineduca7on.wordpress.com University of Edinburgh, 9th May 2014
h.ps://twi.er.com/Wiswijzer2/status/414055472451575808
Siri is smart…
“Find code acts in education”
Siri is smart…
“Find code acts in education”
h.ps://twi.er.com
/Wisw
ijzer2/status/414055472451575808 “Note: check the
huge difference between knowing and measuring…”
A word of introduction… What does this tell you?
Learning Technology
KMi, OU
AI & Argumenta7on
Learning Disposi7ons
Human-‐Centred Informa7cs
Learning Analy7cs
Seman7c Scien7fic Publishing
Dialogue / Issue / Argument Mapping
A word of introduction… What does this tell you?
Critical scepticism to economic modelling
Critical scepticism to economic modelling
The conversation you don’t want to have with your learning analytics wonk
LAK13 Panel: Educational Data Scientists: A Scarce Breed http://people.kmi.open.ac.uk/sbs/2013/03/lak13-edu-data-scientists-scarce-breed
John Behrens (Pearson)
Data ethics and societal readiness ny7.ms/1k5rY72
Data ethics and societal readiness ny7.ms/1k5rY72
In what senses do learning analy1cs “act” in educa1on?
make the invisible visible
make solids permeable
make the ephemeral persistent
Educa7onal (indeed all)
Researchers
In what senses do learning analy7cs “act” in educa7on?
make the invisible visible
make solids permeable
make the ephemeral persistent
Learning Analy7cs
In what senses do learning analy7cs “act” in educa7on?
Educa/onal researchers have always done this — what’s new is that these tools are now in the hands of those who actually produce the ac/vity
outcome
In what senses do learning analy7cs “act” in educa7on?
By changing the system dynamics
researchers / educators / instruc7onal designers administrators / leaders / policymakers
intent
outcome
In what senses do learning analy7cs “act” in educa7on?
By changing the system dynamics Faster feedback loops could enable more rapid adapta7on: of agents’ behaviour, and of learning resources and designs
researchers / educators / instruc7onal designers administrators / leaders / policymakers
intent
Delega7on of authority to define goals, analy7cs, and implica7ons
In what senses do learning analy7cs “act” in educa7on?
Distribution of power between educators, learners, leaders, community…?
?
?
In what senses do learning analy7cs “act” in educa7on?
What data, concepts and relationships do the analytics designers seek to model?
Ontologically
Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, pp. 277, 278, 281
“Classification systems provide both a warrant and a tool for forgetting [...] what to forget and how to forget it [...] The argument comes down to asking not only what gets coded in but what gets coded out of a given scheme.”
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM. Eprint: http://oro.open.ac.uk/32823
“A marker of the health of the learning analytics field will be the quality of debate around what the technology renders visible and leaves invisible.”
In what senses do learning analy7cs “act” in educa7on?
What thresholds, samples, relationships, patterns, etc. do the algorithms encode and seek? On what basis is a recommendation engine proposing interventions?
Algorithmically
governingalgorithms.org Learning Analy7cs
In an increasingly algorithmic world […] What, then, do we talk about when we talk about “governing algorithms”?
governingalgorithms.org
A technology or an epistemology?
Barocas, S., Hood, S. and Ziewitz, M. (2013). Governing Algorithms: A Provoca1on Piece. Social Science Research Network Paper 2245322. DOI: h.p://dx.doi.org/10.2139/ssrn.2245322
Secrecy, obscurity, inscrutability
Agency, automa7on, accountabili7es
A typology of algorithms by genre?
The inscrutability of algorithms
Norma7vity, bias, values
In what senses do learning analy7cs “act” in educa7on?
What meaning-making does the representation and interaction design encourage?
Semio7cally
In what senses do learning analy7cs “act” in educa7on?
epistemology
pedagogy assessment
Knight, S., Buckingham Shum, S. and Littleton, K. (In Press, 2014). Epistemology, Assessment, Pedagogy: Where Learning Meets Analytics in the Middle Space. Journal of Learning Analytics. Open Access Eprint: http://oro.open.ac.uk/39226
the ‘middle
space’ of learning analytics
in the context of computation...
What epistemological assumptions are shaping the assessment regime, and hence the pedagogy? What questions are analytics used to help answer?
In what senses do learning analy7cs “act” in educa7on?
Analytics reports at the organisational and national levels come with consequences at different scales — sometimes punitive, often impacting millions of people.
Poli7cally
Figure from Doug Clow: h.p://www.slideshare.net/dougclow/the-‐learning-‐analy7cs-‐cycle-‐closing-‐the-‐loop-‐effec7vely (slide 5)
In what senses do learning analy7cs “act” in educa7on?
All of the above are encapsulated in any learning analy7cs deployment
28
What kinds of learners? What kinds of learning?
What data could be generated digitally from
the use context? How is it ‘cleaned’?
Does your theory predict pa.erns signifying
learning?
What human +/or sooware interven7ons /recommenda7ons?
How to render the analy7cs, for whom, and will they
understand them?
What analy7cal tools could be used to find such
pa.erns?
In what senses do learning analy7cs “act” in educa7on?
Learning Analy7cs approaches act in educa7on…
Ontologically Algorithmically Semio7cally
Systemically
Poli7cally
Authority?
>>> Learning Analy7cs cri7cal template… <Learning Analytics Approach>
Ontology …
Algorithms …
Semiotics …
Epistemology …
Pedagogy …
Assessment regime …
System Dynamics …
Authority …
Politics …
examples
English primary school data RAISEonline sta7s7cs
for a school, on ‘core’ subjects (maths, literacy), a.endance and demographics
Visualising “attainment” and “progress”
Which analytics could reflect the progress that ‘Joe’ has made on so many other fronts other than his SATS?
34
student modelling by an adap1ve tutor
37
https://grockit.com/research
Adaptive platforms generate fine-grained analytics on curriculum mastery
analy1cs for learning disposi1ons
Qualities of effective learners Factor analysis of the literature plus expert interviews: identified seven dimensions of effective “learning power”, since validated empirically with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)
Qualities of effective learners
40
ELLI: Effective Lifelong Learning Inventory
Quantifying learning dispositions agency; identity; motivation; responsibility
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823 http://learningemergence.net/2012/04/30/learning-powered-learning-analytics
A wholis7c visual, intended to build intrinsic mo7va7on, invi7ng stretch, providing a new language, provoking conversa7on that 7es to the learner’s iden7ty
Quantifying learning dispositions agency; identity; motivation; responsibility
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
2020? personal data cloud generates my dispositional profile for reflection >>> help me take responsibility for my own learning
Shaofu Huang: Prototyping Learning Power Modelling in SocialLearn http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium Simon Knight: http://people.kmi.open.ac.uk/knight/2014/02/knowledge-in-search
Social network patterns, teamwork effectiveness
and initiation of relationships
Questioning, arguing and search behaviours
reveal intrinsic curiosity and epistemic commitments
Tagging/sharing/blogging/social patterns
reveal how you see connections between
ideas
Perseverance, grit, tenacity; overcoming
panic/stress when challenged — through
behavioural and somatic traces
Open source approach to analytics: towards true transparency and emergence for complex systems?
http://www.solaresearch.org/mission/ola
conclusion analy1cs act in educa1on — on mul1ple dimensions an analy1cs approach perpetuates an educa1onal worldview — so let’s ensure this is
inten1onal – not accidental...