learning from usain bolt: integrating ipsative assessments into our learning
DESCRIPTION
Opus Learning's Ken Currie spoke at the The e-Assessment Question 2014 Conference and Exhibition in London about the integration of ipsative assessments into our Opus Learning courses. To find out more please email Ken under [email protected]TRANSCRIPT
Learning from Usain Bolt: integrating ipsative assessments
into our learning
12th eAssessment Question
Ken CurrieOpus Learning (& CAPDM)
27th March 2014
1
This Talk
• Three parts:– Who are we, and why is eAssessment an issue for
Opus– What are we trying to do with eAssessment– Our approach to Delivery, Support and
eAssessment• Fourth theme?
– Implementation
2
Lots out there
• Online Learning and Assessments is a very active area– Lots of experts– Lots of non-experts– Huge amount of literature
• When I was asked to stand in for my colleague I thought– “what is there to add to an area pinned down by
such expertise?”
3
Useful Quote
• “Large scale online learning could provide continuity for learners throughout their lives between different education systems.” – Association of Learning Technology Draft Green
Paper 2013.• Then you realise that the game is changing
– Advancement is very fluid– As the owner of a problem we can make a
contribution
4
History of Opus Learning
• A spinout from CAPDM.com– Developing and delivering Masters programmes
for over 20 years• Opus delivers highly interactive courses
– Developing tools to manage and monitor learning– Marketing through partner colleges worldwide
• An SQA Approved Centre
5
Opus Learning
• Opus is a wholly online SQA Approved college– Offering HNC/Ds and other courses– With its own students and tutors
• Opus is also a B2B provider of– Ready packaged courses and programmes– A full own-brand ‘College in a Box’– Consultancy for aspiring SQA centres
• Opus /CAPDM is a custom developer & publisher of Online Distance Learning
6
Opus Business Model
• Focussed, highly scalable and repeatable• No significant technical limitations on its
geographic market• Minimal marginal costs of servicing additional
students– economies of scale for Opus and its partners
• Defence against price competition?– Quality of delivery systems, plus content
&structure of learning materials7
Partnering?
• Opus (and its sister company CAPDM) are:– Looking for technology partners, including Moodle
development– Looking for learning partners, including developing
more content and units• And assessment options!
8
Issues 1: the Design Challenge
• To build, deliver and manage– 99 HN courses for Opus– 18 masters level courses for Edinburgh Napier– … + others!
• To manage > 100 academics/teachers/authors• To manage very large domains of content• To deliver to many platforms
– Moodle, Blackboard, ePub, tablet/iPad, PDF, …
9
SQA: the Start Point
• SQA provide ‘arrangement’ documents for all– Frameworks (e.g. HND Business) and – Units (e.g. Business Accounting)
• Unit specifications of:– Outcomes– Knowledge and/or Skills– Evidence Requirements– + Assessment Guidelines
10
HND Business: Unit by Unit
• HND Business: each student has to– Sit 6 formal examinations– Deliver 6 documented portfolio exercises– Complete 22 other assessment reports
• total word count of 34,500
• An HE Diploma?– 12 modules each with final examinations or
assignment coursework
11
Assessments & the SQA
• Why are we trying to ‘change’ things?• Units are stand alone
– Can be delivered in a very flexible manner– But very clear pointers as to how assessment
should be carried out• No longer appropriate to base an assessment
of competence on a final three hour written examination?
12
Issues 2: Assessment
• Assessment can take many forms– Formative, e.g. through topic quizzes– Summative, e.g. exams and assignments– But not Ipsative, e.g. through comparison of self
with earlier marks?• Ipsative?
– I had to look this up:• “the practice of assessing present performance against
the prior performance” (Wikipedia)
• “Be the best that you can be” (www.ipsative.co.uk)13
Any link to Usain Bolt?
• No!• But we like his approach: Bolt, Sep 2013.
– "In The 200m, If I Can Master The Bend There Is Room For Improvement"
• Bolt plots his progress against every conceivable parameter on a daily basis– Continued, measured improvement– Should we not do the same?
14
Our Response: a Digital Work Book
• Longitudinal assessment– Augments examination and assignment
• Part of our ‘Social Learner’ aim– No faith in social learning (students creating the
learning content)– Great belief in breeding social learners– Many levels from ‘none’ to
• Interact with content for own ends (e.g. DWB)• Comment locally with others (e.g. Scotsman)• Interact and discuss with all peers (e.g. Forums)
15
Ipsative via the DWB
• Students build up a Reflective Log, over time– A longitudinal measure (Digital Work Book)– Continually improve it– Shared with the tutor (peers potentially)
16
A Component of Learning Analytics
• MOOCs are gathering global analytics– Measure everything but the individual
• Opus is looking to move towards developing the ‘Digital Self’– How you model yourself as an individual, and
access your data– How you are measured at all times– Our initial response – the DWB – begins to
recognise the potential of ‘always-on’ learning
17
Interact with Content Objects
Learner interacting with content, leaving traces
Student interacts with content
Makes notes, responds to quiz, send messages, etc
Capture notesStore in MLE
Print off record of learning The Digital Workbook
18
Social Networks
• Opus is looking to build 000s of small groups– MOOCs are looking for a group of 000s
• Our social networks are small scale but highly linked (see Anderson 2003)
• Our goal: 5,000 students, but– 100 centres with 50 students– Model the individual, or a small group
19
How do students interact when learning
Student
Environment
Content Tutor
Student/Student
Content/ContentTutor/Tutor
Environment/Environment
Student/Tutor
Tutor/Environment
Student/Environment
Student/Content
Content/Environment
Tutor/Content
Based on diagram inAnderson 2003 20
Big Data?
• The analytics of the individual are of more interest to us– Not specifically looking at group behaviour– Individuals, in depth and across courses
• Digital Self profiles may be limited– E.g. me to Facebook and Linked-In (no pedometer
around my waist!)– How do we enhance profiles?
21
Role of the Learning Materials
• Authors are encouraged to develop clear ‘concept maps’ for students to follow– Learning paths must go through key ‘concept
gateways’– Evidence of competent/critical thinking gathered
• The core is learning content that– Is interactive– Measures its own consumption– Builds ‘Digital Self’ data
• How to make it more intelligent? 22
Student Learning Aims
• Our aims for our learners– To see how they change over time– To teach these learners how to learn
• Their interface– Our content, systems and support– Must be smart
23
Analytics: Some Conclusions
• Analytics and tracking, via the DWB, etc., appears to– Provide the potential of personalised learning
within mechanical presentation of study– Permit formative assessment in the content,
driving tracking and learning progress– Provide feedback on Tutor effectiveness and the
effectiveness of the pedagogy underpinning the content
24
The Digital Workbook is
• A portfolio embedded in the content
• A dashboard for learning • Rich seam of data and
behaviours for mining• Ongoing formative
assessment, and potential summative assessment
• A permanent record (e.g. a PDF) for the student 25
Tracking through the learning content
• Opus content is massively connected to the students Work Book
• Every reflection, exercise, quiz, short response etc. is logged within the learning content and context
• Tutors can view, track and compare Work Books
26
Digital Work Book: its Role
• A key assessment tool – An integral part of the design and the content– Can be coupled with ‘concept gateways’
• Evidence gathering– For demonstration of competent/critical learning
• In the future it will become• our main assessment toolset• our main behaviour manager• But not the only one …
27
Learning Objective Profiling
• Build Learning Objectives into every course– Ensure that all content relates to at
least one LO– Weight all formative
assessmentquestions againstat least one LO
28
( c ) c o p y r ig h t C A P D M L td . 2 0 0 7
Lea rn in gOb j ec tiv es
C o u rse m a p
A
D
C
B
KLO1 KLO2
KLO3
KLO7
KLO10
KLO4 KLO5 KLO6
KLO8 KLO9
KLO11
Aggrega tion s
A’
D’
C’
B’
Lea rn in gOu tcom es
A ssessm en tA im s
Mapp ing?
Learning Objective Profiles
• Couple Learning Objectives to Personalised Feedback
• Developbalancedcoursesand assessment
29
Content-based Study Planner
30
• Encourages students to plan & plot their progress– Allows tutors a
‘helicopter view’ of student progress and an early warning of problems
– Use DWB to seethe detail
In Conclusion
• Do not just set exams and give badges– Track and Assess– Intervene and support– Audit and learn lessons
• Build a community for the whole qualification• Record learning and make available for future
generations• Continually improve learning content
31
DWB Examples – Source Page
33
34
35
Assessor/Tutor Activity Model in Distance Learning
1. Monitor VLE activity, identify any candidate with lack of progress and contact course leader for action.
2. Monitor Digital Workbook (if used) and identify any outliers in progress, contact course leader with details.
3. Respond to direct tickets from candidate, making use of FAQs for responses, if new response is required, copy response to course leader.
4. Responding to assessments; remediation advice
36
Opus StrategySmall Private Open Courses
• Organise cohorts in groups of 10s, not groups of 10k’s or 100k’s
• Localise tutors and assessors• Build communities including assessors• Track activity and behaviour at keyboard level and at
cognitive level• Record activity – perhaps award a badge for partially
completed activity• Guide the learners towards completion, intervene as
necessary37
Creating the Social Learner
• Our DWB is part of a social learner strategy– Rich interaction spaces– Personal interaction with the content (DWB)– Interaction on specific issues (traversing the web)– Social learner (Forums, Facebook, etc.)
• Manage forum and virtual space activity
38