Download - Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013
![Page 1: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/1.jpg)
Analytics(as if learning mattered)
Adam Cooper,In Focus: Learner analytics and big data symposium,University of London, December 10th 2013
![Page 2: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/2.jpg)
Structure of the Talk
• My Perspective
• Threats
• Avoiding the threats
Image: cb
a Jaskirat Singh Baw
a
![Page 3: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/3.jpg)
PERSPECTIVE
![Page 4: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/4.jpg)
Perspective of this TalkWhere am I coming from?
UK
Higher Ed
InstitutionCetis is based in the Institute for Educational Cybernetics at the University of Bolton.
“Our mission is to develop a better understanding of how information and communications technologies affect the organisation of education from individual learning to the global system.”
![Page 5: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/5.jpg)
Defining “Analytics”
Analytics is the process of developing
actionable insights
through
problem definition
and the application of statistical models and
analysis against existing and/or simulated future
data.
![Page 6: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/6.jpg)
Learning vs Business Venn
Insights to support
educational aims and
objectives
Insights to support
operational and
strategic activity.
![Page 7: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/7.jpg)
Learning vs Business Venn
Retention
Student Satisfaction
Employability
Achievement
Applications
Easiest Case forInvestment
![Page 8: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/8.jpg)
Structure to Manage ComplexityThe SMT – “what makes a viable organisation?”
Subject/Discipline Member - “what makes a good chemist?” (knowledge, approach to problems, creativity, attitudes, values...)
Course Team – “what curriculum?” (topics, suitable LOs and assessment methods to match our typical student career etc)
Class Teacher – “where are my students collectively and individually?” (intellectually, emotionally, meta-cognitively...)
Student/Learner – time, place and method of studySelf-organising and self-regulating
![Page 9: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/9.jpg)
The Intersection Doppelgänger
Retention
Student Satisfaction
Employability
Achievement
Applications
Values
Epistemological
Cause & Effect Models
Moral Imperatives
![Page 10: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/10.jpg)
The “Accommodation”“The introduction of these techniques [Learning Analytics] cannot be
understood in isolation from the methods of educational management as
they have grown up over the past two centuries. These methods are
conditioned by the fact that educational managers are limited in their
capability to monitor and act upon the range of states which are
taken up by teachers and learners in their learning activities. ... Over
the years, an accommodation has developed between regulatory authorities,
management and teaching professionals: educational managers indicate the
goals which teachers and learners should work towards, provide a
framework for them to act within, and ensure that the results of their activity
meet some minimum standards. The rest is left up to the professional skills
of teachers and the ethical integrity of both teachers and learners.”Prof. Dai Griffiths, “Implications of Analytics for Teaching Practice in Higher Education”
![Page 11: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/11.jpg)
TWO THREATS(+1 AND A BIT)
![Page 12: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/12.jpg)
Threat #1: Interventionism
Image: via Wikimedia Commons p
![Page 13: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/13.jpg)
Threat #2: BI Tradition
• Centralised projects• IT + senior leader• Management reports• KPIs
Image: Paul H
odge
Image: via W
ikimedia Com
mons p
Hhultgren
![Page 14: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/14.jpg)
So, you want to optimise student success?
• Are the following well-specified?• “optimise student success”• “maximise the probability of success”
• What does success look like?• Really?• Says who?• Is a “good education” indicated by student success?
![Page 15: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/15.jpg)
Does everyone understand?Attainment: an academic standard, typically
shown by test and examination results
Progress: how far a learner has moved between assessment events
Achievement: takes into account the standards of attainment reached by learners and the progress they have made to reach those standards
Enrichment: the extent to which a learner gains professional attitudes, meta-cognition, technical or artistic creativity, delight, intellectual flexibility, ...
![Page 16: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/16.jpg)
+1: Technological Solutionism‘Recasting all complex social situations either as neatly defined problems with
definite, computable solutions or as transparent and self-evident processes that
can be easily optimized—if only the right algorithms are in place!—this quest is
likely to have unexpected consequences that could eventually cause more
damage than the problems they seek to address.
I call the ideology that legitimizes and sanctions such aspirations “solutionism.” I borrow this
unabashedly pejorative term from the world of architecture and urban planning, where it
has come to refer to an unhealthy preoccupation with sexy, monumental, and narrow-
minded solutions—the kind of stuff that wows audiences at TED Conferences—to problems
that are extremely complex, fluid, and contentious … solutionism presumes rather
than investigates the problems that it is trying to solve, reaching “for the answer
before the questions have been fully asked.” How problems are composed matters
every bit as much as how problems are resolved.’
Evgeny Morozov, “To Save Everything Click Here: The Folly of Technological Solutionism”, 2013
![Page 17: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/17.jpg)
3M and Six-Sigma“McNerney axed 8,000 workers (about 11% of the workforce), intensified the
performance-review process, and tightened the purse strings at a company that had
become a profligate spender. He also imported GE's vaunted Six Sigma program—a
series of management techniques designed to decrease production defects and
increase efficiency. Thousands of staffers became trained as Six Sigma ‘black belts’.
...
Now his successors face a challenging question: whether the relentless emphasis on
efficiency had made 3M a less creative company. ... Those results are not
coincidental. Efficiency programs such as Six Sigma are designed to identify
problems in work processes—and then use rigorous measurement to reduce
variation and eliminate defects.
...
Indeed, the very factors that make Six Sigma effective in one context can
make it ineffective in another. Traditionally, it uses rigorous statistical
analysis to produce unambiguous data that help produce better quality,
lower costs, and more efficiency. That all sounds great when you know
what outcomes you'd like to control. But what about when there are few
facts to go on—or you don't even know the nature of the problem you're
trying to define?”Source : Bloomberg Business Week, June 2007.
![Page 18: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/18.jpg)
The Bit
Data:
it may be big
but its not clever
![Page 19: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/19.jpg)
The Bit: Net Scale Fascination (delusion?)
Data:
it may be big
but its not clever
![Page 20: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/20.jpg)
• Hendrik’s photo
Source unknown – thanks to Hendrik Drachsler for passing it on.
![Page 21: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/21.jpg)
Threat Rating=UBI*(I+D)UTS
Where:UBI = Uncritical BI tradition factor
I = Interventionist tendenciesD = Big data fascinationUTS = Uncritical technological solutionism
This is not serious...
![Page 22: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/22.jpg)
Why am I Worried?
From
: “An
alyti
cs in
Hig
her E
duca
tion:
Ben
efits
, Bar
riers
, Pr
ogre
ss, a
nd R
ecom
men
datio
ns”
(ECA
R Re
port
)
![Page 23: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/23.jpg)
NEUTRALISING THREATS
![Page 24: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/24.jpg)
Data Is Useless Without the Skills to Analyze It
‘...employees need to become:
Ready and willing to experiment: Managers and business
analysts must be able to apply the principles of scientific
experimentation to their business. They must know how to construct
intelligent hypotheses. They also need to understand the principles
of experimental testing and design, including population selection
and sampling, in order to evaluate the validity of data analyses.
...
Adept at mathematical reasoning: How many of your managers
today are really “numerate” — competent in the interpretation and
use of numeric data?’Jeanne Harris, HBR Blog, Sept 2012
![Page 25: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/25.jpg)
Is This a Real Problem?
From
: “An
alyti
cs in
Hig
her E
duca
tion:
Ben
efits
, Bar
riers
, Pr
ogre
ss, a
nd R
ecom
men
datio
ns”
(ECA
R Re
port
)
![Page 26: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/26.jpg)
Make Some Progress By:
• Directly investing in specialists• E.g. OU, statisticians and data wranglers• ECAR Survey Report 2012: “invest in people over tools...
hiring and/or training... analysis”
• Operating a “shared service”• Higher Education Statistics Agency (HESA)• Universities and Colleges Admissions Service (UCAS)• Jisc
![Page 27: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/27.jpg)
Data Is Useless Without the Skills to Analyze It
‘...employees need to become:
Ready and willing to experiment: Managers and business
analysts must be able to apply the principles of scientific
experimentation to their business. They must know how to construct
intelligent hypotheses. They also need to understand the principles
of experimental testing and design, including population selection
and sampling, in order to evaluate the validity of data analyses.
...
Adept at mathematical reasoning: How many of your managers
today are really “numerate” — competent in the interpretation and
use of numeric data?’Jeanne Harris, HBR Blog, Sept 2012
![Page 28: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/28.jpg)
We Get Further with “Soft Capability”
• Appreciation of the limitations of analytics technologies
• Expertise to comprehend, critique, and converse(as opposed to originate or implement)
![Page 29: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/29.jpg)
Visualisations are the Opiumof the People
(c)2008, Ryan Block/Engadget
![Page 30: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/30.jpg)
Sparklines: Tufte vs ...
![Page 31: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/31.jpg)
Significance and Sample Size“Flagged for action because 74% of students felt
they were well supported by their supervisor compared to 80% in similar institutions (difference >5%)”
“We need to talk to Mr. Benn; the drop-out rate doubled in his class this year.”
Correlation is not Causation“The predictive model indicates that students with
attributes {X, Y, Z} are only 50% likely to complete.”
![Page 32: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/32.jpg)
What Does This Give Us?(aside from people less likely
to cut themselves on sharp tools)
![Page 33: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/33.jpg)
![Page 34: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/34.jpg)
Absorptive Capacity
“...the ability of a firm to recognize the value of new,
external information, assimilate it, and apply it to
commercial ends is critical to its innovative capabilities. We
label this capability a firm’s absorptive capacity and suggest that
it is largely a function of the firm’s level of prior related
knowledge. [...] We formulate a model of firm investment in
research and development (R&D), in which R&D contributes to
a firm’s absorptive capacity, and test predictions relating a
firm’s investment in R&D to the knowledge underlying technical
change within an industry.” Cohen and Levinthal, Admin. Science Quarterly 35(1), 1990
![Page 35: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/35.jpg)
How to do LA as if learning mattered
1.Build soft capability.
2.Engage in participatory (collaborative) design.
3.Develop & prototype technology (& models)
organically.
4.Reflect on the effect LA innovation has on practice.
5.Design and develop with the educator IN the
system.
6.Optimise the environment, not only the outcome.
7.Be realistic about scale/quality of data and its
meaning.
Personal Learning Analytics?• Changed dyamic• Less “big brother”
![Page 36: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/36.jpg)
Image: cbn Jesse Millan
ParticipatoryDesign
LearningAnalytics
![Page 37: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/37.jpg)
Thank-you for your attention
These slides are at: http://is.gd/AaiLM
![Page 38: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/38.jpg)
Reading and ReferencesJacqueline Bichsel, “Analytics in Higher Education: Benefits, Barriers, Progress, and
Recommendations”Available from http://www.educause.edu/ecar
Cohen and Levinthal (1990), “Absorptive Capacity: A New Perspective on Learning and Innovation” Admin. Science Quarterly 35(1)http://faculty.fuqua.duke.edu/%7Echarlesw/s591/Bocconi-Duke/Papers/C10/CohenLevinthalASQ.pdf
Adam Cooper, “What is Analytics? Definition and Essential Characteristics” http://publications.cetis.ac.uk/2012/521
Jeanne Harris, “Data is Useless Without the Skills to Analyze it”http://blogs.hbr.org/2012/09/data-is-useless-without-the-skills/
Bloomberg Business Week, “At 3M, A Struggle Between Efficiency And Creativity”http://www.businessweek.com/stories/2007-06-10/at-3m-a-struggle-between-efficiency-and-creativity
Dai Griffiths, “CETIS Analytics Series: The impact of analytics in Higher Education on academic practice”http://publications.cetis.ac.uk/2012/532
Evgeny Morozov, “To Save Everything Click Here: The Folly of Technological Solutionism”ISBN-10: 9781610391382
Phil Winne, “Improving Education” (lecture)http://www.youtube.com/watch?v=dBxMZoMTIU4
![Page 39: Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013](https://reader033.vdocuments.us/reader033/viewer/2022051514/549406ffac79593b2e8b4a1a/html5/thumbnails/39.jpg)
Licence
“Analytics (as if learning mattered)” by Adam Cooper, Cetis,
is licensed under the Creative Commons Attribution 3.0 Unported Licence
http://creativecommons.org/licenses/by/3.0/
Case studies and white papers:
Cetis Analytics Series, http://publications.cetis.ac.uk/c/analytics
Me:
http://blogs.cetis.ac.uk/adam
Cetis:
http://www.cetis.ac.uk