in focus presentation: analytics: as if learning mattered
DESCRIPTION
Analytics: as if learning mattered Presentation from 'In Focus: Learner analytics and big data', a CDE technology symposium held at Senate House on 10 December 2013. Conducted by Adam Cooper (Co-Director, Cetis) Audio of the session and more details can be found at www.cde.london.ac.uk.TRANSCRIPT
![Page 1: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/1.jpg)
Analytics(as if learning mattered)
Adam Cooper,In Focus: Learner analytics and big data symposium,University of London, December 10th 2013
![Page 2: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/2.jpg)
Structure of the Talk
• My Perspective
• Threats
• Avoiding the threats
Image: cb
a Jaskirat Singh Baw
a
![Page 3: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/3.jpg)
PERSPECTIVE
![Page 4: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/4.jpg)
Perspective of this TalkWhere am I coming from?
UK
Higher Ed
InstitutionCetis is based in the Institute for Educational Cybernetics at the University of Bolton.
“Our mission is to develop a better understanding of how information and communications technologies affect the organisation of education from individual learning to the global system.”
![Page 5: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/5.jpg)
Defining “Analytics”
Analytics is the process of developing
actionable insights
through
problem definition
and the application of statistical models and
analysis against existing and/or simulated future
data.
![Page 6: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/6.jpg)
Learning vs Business Venn
Insights to support
educational aims and
objectives
Insights to support
operational and
strategic activity.
![Page 7: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/7.jpg)
Learning vs Business Venn
Retention
Student Satisfaction
Employability
Achievement
Applications
Easiest Case forInvestment
![Page 8: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/8.jpg)
Structure to Manage ComplexityThe SMT – “what makes a viable organisation?”
Subject/Discipline Member - “what makes a good chemist?” (knowledge, approach to problems, creativity, attitudes, values...)
Course Team – “what curriculum?” (topics, suitable LOs and assessment methods to match our typical student career etc)
Class Teacher – “where are my students collectively and individually?” (intellectually, emotionally, meta-cognitively...)
Student/Learner – time, place and method of studySelf-organising and self-regulating
![Page 9: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/9.jpg)
The Intersection Doppelgänger
Retention
Student Satisfaction
Employability
Achievement
Applications
Values
Epistemological
Cause & Effect Models
Moral Imperatives
![Page 10: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/10.jpg)
The “Accommodation”“The introduction of these techniques [Learning Analytics] cannot be
understood in isolation from the methods of educational management as
they have grown up over the past two centuries. These methods are
conditioned by the fact that educational managers are limited in their
capability to monitor and act upon the range of states which are
taken up by teachers and learners in their learning activities. ... Over
the years, an accommodation has developed between regulatory authorities,
management and teaching professionals: educational managers indicate the
goals which teachers and learners should work towards, provide a
framework for them to act within, and ensure that the results of their activity
meet some minimum standards. The rest is left up to the professional skills
of teachers and the ethical integrity of both teachers and learners.”Prof. Dai Griffiths, “Implications of Analytics for Teaching Practice in Higher Education”
![Page 11: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/11.jpg)
TWO THREATS(+1 AND A BIT)
![Page 12: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/12.jpg)
Threat #1: Interventionism
Image: via Wikimedia Commons p
![Page 13: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/13.jpg)
Threat #2: BI Tradition
• Centralised projects• IT + senior leader• Management reports• KPIs
Image: Paul H
odge
Image: via W
ikimedia Com
mons p
Hhultgren
![Page 14: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/14.jpg)
So, you want to optimise student success?
• Are the following well-specified?• “optimise student success”• “maximise the probability of success”
• What does success look like?• Really?• Says who?• Is a “good education” indicated by student success?
![Page 15: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/15.jpg)
Does everyone understand?Attainment: an academic standard, typically
shown by test and examination results
Progress: how far a learner has moved between assessment events
Achievement: takes into account the standards of attainment reached by learners and the progress they have made to reach those standards
Enrichment: the extent to which a learner gains professional attitudes, meta-cognition, technical or artistic creativity, delight, intellectual flexibility, ...
![Page 16: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/16.jpg)
+1: Technological Solutionism‘Recasting all complex social situations either as neatly defined problems with
definite, computable solutions or as transparent and self-evident processes that
can be easily optimized—if only the right algorithms are in place!—this quest is
likely to have unexpected consequences that could eventually cause more
damage than the problems they seek to address.
I call the ideology that legitimizes and sanctions such aspirations “solutionism.” I borrow this
unabashedly pejorative term from the world of architecture and urban planning, where it
has come to refer to an unhealthy preoccupation with sexy, monumental, and narrow-
minded solutions—the kind of stuff that wows audiences at TED Conferences—to problems
that are extremely complex, fluid, and contentious … solutionism presumes rather
than investigates the problems that it is trying to solve, reaching “for the answer
before the questions have been fully asked.” How problems are composed matters
every bit as much as how problems are resolved.’
Evgeny Morozov, “To Save Everything Click Here: The Folly of Technological Solutionism”, 2013
![Page 17: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/17.jpg)
3M and Six-Sigma“McNerney axed 8,000 workers (about 11% of the workforce), intensified the
performance-review process, and tightened the purse strings at a company that had
become a profligate spender. He also imported GE's vaunted Six Sigma program—a
series of management techniques designed to decrease production defects and
increase efficiency. Thousands of staffers became trained as Six Sigma ‘black belts’.
...
Now his successors face a challenging question: whether the relentless emphasis on
efficiency had made 3M a less creative company. ... Those results are not
coincidental. Efficiency programs such as Six Sigma are designed to identify
problems in work processes—and then use rigorous measurement to reduce
variation and eliminate defects.
...
Indeed, the very factors that make Six Sigma effective in one context can
make it ineffective in another. Traditionally, it uses rigorous statistical
analysis to produce unambiguous data that help produce better quality,
lower costs, and more efficiency. That all sounds great when you know
what outcomes you'd like to control. But what about when there are few
facts to go on—or you don't even know the nature of the problem you're
trying to define?”Source : Bloomberg Business Week, June 2007.
![Page 18: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/18.jpg)
The Bit
Data:
it may be big
but its not clever
![Page 19: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/19.jpg)
The Bit: Net Scale Fascination (delusion?)
Data:
it may be big
but its not clever
![Page 20: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/20.jpg)
• Hendrik’s photo
Source unknown – thanks to Hendrik Drachsler for passing it on.
![Page 21: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/21.jpg)
Threat Rating=UBI*(I+D)UTS
Where:UBI = Uncritical BI tradition factor
I = Interventionist tendenciesD = Big data fascinationUTS = Uncritical technological solutionism
This is not serious...
![Page 22: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/22.jpg)
Why am I Worried?
From
: “An
alyti
cs in
Hig
her E
duca
tion:
Ben
efits
, Bar
riers
, Pr
ogre
ss, a
nd R
ecom
men
datio
ns”
(ECA
R Re
port
)
![Page 23: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/23.jpg)
NEUTRALISING THREATS
![Page 24: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/24.jpg)
Data Is Useless Without the Skills to Analyze It
‘...employees need to become:
Ready and willing to experiment: Managers and business
analysts must be able to apply the principles of scientific
experimentation to their business. They must know how to construct
intelligent hypotheses. They also need to understand the principles
of experimental testing and design, including population selection
and sampling, in order to evaluate the validity of data analyses.
...
Adept at mathematical reasoning: How many of your managers
today are really “numerate” — competent in the interpretation and
use of numeric data?’Jeanne Harris, HBR Blog, Sept 2012
![Page 25: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/25.jpg)
Is This a Real Problem?
From
: “An
alyti
cs in
Hig
her E
duca
tion:
Ben
efits
, Bar
riers
, Pr
ogre
ss, a
nd R
ecom
men
datio
ns”
(ECA
R Re
port
)
![Page 26: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/26.jpg)
Make Some Progress By:
• Directly investing in specialists• E.g. OU, statisticians and data wranglers• ECAR Survey Report 2012: “invest in people over tools...
hiring and/or training... analysis”
• Operating a “shared service”• Higher Education Statistics Agency (HESA)• Universities and Colleges Admissions Service (UCAS)• Jisc
![Page 27: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/27.jpg)
Data Is Useless Without the Skills to Analyze It
‘...employees need to become:
Ready and willing to experiment: Managers and business
analysts must be able to apply the principles of scientific
experimentation to their business. They must know how to construct
intelligent hypotheses. They also need to understand the principles
of experimental testing and design, including population selection
and sampling, in order to evaluate the validity of data analyses.
...
Adept at mathematical reasoning: How many of your managers
today are really “numerate” — competent in the interpretation and
use of numeric data?’Jeanne Harris, HBR Blog, Sept 2012
![Page 28: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/28.jpg)
We Get Further with “Soft Capability”
• Appreciation of the limitations of analytics technologies
• Expertise to comprehend, critique, and converse(as opposed to originate or implement)
![Page 29: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/29.jpg)
Visualisations are the Opiumof the People
(c)2008, Ryan Block/Engadget
![Page 30: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/30.jpg)
Sparklines: Tufte vs ...
![Page 31: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/31.jpg)
Significance and Sample Size“Flagged for action because 74% of students felt
they were well supported by their supervisor compared to 80% in similar institutions (difference >5%)”
“We need to talk to Mr. Benn; the drop-out rate doubled in his class this year.”
Correlation is not Causation“The predictive model indicates that students with
attributes {X, Y, Z} are only 50% likely to complete.”
![Page 32: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/32.jpg)
What Does This Give Us?(aside from people less likely
to cut themselves on sharp tools)
![Page 33: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/33.jpg)
![Page 34: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/34.jpg)
Absorptive Capacity
“...the ability of a firm to recognize the value of new,
external information, assimilate it, and apply it to
commercial ends is critical to its innovative capabilities. We
label this capability a firm’s absorptive capacity and suggest that
it is largely a function of the firm’s level of prior related
knowledge. [...] We formulate a model of firm investment in
research and development (R&D), in which R&D contributes to
a firm’s absorptive capacity, and test predictions relating a
firm’s investment in R&D to the knowledge underlying technical
change within an industry.” Cohen and Levinthal, Admin. Science Quarterly 35(1), 1990
![Page 35: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/35.jpg)
How to do LA as if learning mattered
1.Build soft capability.
2.Engage in participatory (collaborative) design.
3.Develop & prototype technology (& models)
organically.
4.Reflect on the effect LA innovation has on practice.
5.Design and develop with the educator IN the
system.
6.Optimise the environment, not only the outcome.
7.Be realistic about scale/quality of data and its
meaning.
Personal Learning Analytics?• Changed dyamic• Less “big brother”
![Page 36: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/36.jpg)
Image: cbn Jesse Millan
ParticipatoryDesign
LearningAnalytics
![Page 37: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/37.jpg)
Thank-you for your attention
These slides are at: http://is.gd/AaiLM
![Page 38: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/38.jpg)
Reading and ReferencesJacqueline Bichsel, “Analytics in Higher Education: Benefits, Barriers, Progress, and
Recommendations”Available from http://www.educause.edu/ecar
Cohen and Levinthal (1990), “Absorptive Capacity: A New Perspective on Learning and Innovation” Admin. Science Quarterly 35(1)http://faculty.fuqua.duke.edu/%7Echarlesw/s591/Bocconi-Duke/Papers/C10/CohenLevinthalASQ.pdf
Adam Cooper, “What is Analytics? Definition and Essential Characteristics” http://publications.cetis.ac.uk/2012/521
Jeanne Harris, “Data is Useless Without the Skills to Analyze it”http://blogs.hbr.org/2012/09/data-is-useless-without-the-skills/
Bloomberg Business Week, “At 3M, A Struggle Between Efficiency And Creativity”http://www.businessweek.com/stories/2007-06-10/at-3m-a-struggle-between-efficiency-and-creativity
Dai Griffiths, “CETIS Analytics Series: The impact of analytics in Higher Education on academic practice”http://publications.cetis.ac.uk/2012/532
Evgeny Morozov, “To Save Everything Click Here: The Folly of Technological Solutionism”ISBN-10: 9781610391382
Phil Winne, “Improving Education” (lecture)http://www.youtube.com/watch?v=dBxMZoMTIU4
![Page 39: In Focus presentation: Analytics: as if learning mattered](https://reader035.vdocuments.us/reader035/viewer/2022062701/5537fbe1550346592f8b466e/html5/thumbnails/39.jpg)
Licence
“Analytics (as if learning mattered)” by Adam Cooper, Cetis,
is licensed under the Creative Commons Attribution 3.0 Unported Licence
http://creativecommons.org/licenses/by/3.0/
Case studies and white papers:
Cetis Analytics Series,
http://publications.cetis.ac.uk/c/analytics
Me:
http://blogs.cetis.ac.uk/adam
Cetis:
http://www.cetis.ac.uk
Learning Analytics Community Exchangehttp://www.laceproject.info