![Page 1: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/1.jpg)
![Page 2: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/2.jpg)
Electronic management of assessment
Maximising the benefits technology can offer
![Page 3: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/3.jpg)
» Background and context
» Headline findings from EMA landscape review
» Consulting on priorities
» From challenges to solutions
» Solution scoping activity
Overview
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 3
![Page 4: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/4.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 4
Background and context
bit.ly/jisc-assess and http://bit.ly/jiscdsaf
» Background and context
› Recently completed – a 3 year programme exploring technology-enhanced assessment and feedback
» Four main themes:
› Feedback and feed forward
› Enhancing student employability
› Strategies for change
› Electronic management of assessment (EMA)
![Page 5: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/5.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 5
» EMA - benefits and drivers are clear
» Challenges surfacing relating to process, practice and technology
» Working in collaboration with the Heads of e-Learning Forum (HeLF)and UCISA to maximise the benefits technology can offer in the management of assessment
Electronic management of assessment project
![Page 6: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/6.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 6
» Aims:
› To gather examples of practices, policies & process as well as approaches to implementation
› To identify the challenges around EMA, in terms of pedagogy/culture, process or technology and to identify existing solutions where they can be found
› To provide interim guidance to institutions on the state of play and effective practice with approaches to implementation
› To make recommendations to institutions, systems providers and Jisc on priorities to address
The EMA study
![Page 7: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/7.jpg)
» HeLF EMA surveys
» Jisc assessment and feedback programme: 30+ institutions
» Online questionnaire: 70 institutions
» Think Tank: 32 institutions
» Other Jisc learning and teaching related projects
» Literature review
» Interviews with learning providers
Information sources
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 7
![Page 8: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/8.jpg)
» Nuanced responses but broad trends visible
» Localised initiatives but beginning to scale up
» System integration is a key problem area
» EMA exposes variability in business processes
Headline findings
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 8
![Page 9: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/9.jpg)
Loose system integration
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 9
![Page 10: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/10.jpg)
Variation in business processes
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 10
![Page 11: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/11.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 11
What are the pain points in EMA?
Pain points
![Page 12: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/12.jpg)
» Anonymity: both ends of spectrum motivated by student pressure
» Resistance: many stakeholders take narrow view; little joined-up thinking
» Scaling up: may expose technology issues & ‘harden’ staff attitudes
» Feedback: storage & access issues; good practice can be hard to scale
» Practice: may be simple but process workarounds cause frustration and resentment
» Technology: limited range of core products how hard can it really be?
» Formative approaches: less well developed than summative
What causes the pain?
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 12
![Page 13: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/13.jpg)
A lifecycle model
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 13
![Page 14: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/14.jpg)
Priorities 1
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 14
Top 10 EMA challenges
![Page 15: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/15.jpg)
Priorities 2
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 15
Top 20 EMA challenges
![Page 16: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/16.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 16
» Review of the EMA landscape across UK HE and FE completed
» Identified and prioritised the challenges around EMA
» Workshops in Dec 2014 and Jan 2015 explored possible solutionsto these prioritised challenges
Where are we now?
ema.jiscinvolve.org/
![Page 17: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/17.jpg)
Where next?
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 17
Phase 2.1Oct – Aug 2015
Second phase of consultation, refining and prioritising challenges (speaking as a ‘single voice’)
Moving from challenges to solutions (Oct – Jan)
Solution development (Feb – July)
Moving from challenge to solutions –
4 projects going forward
Ongoing capture of resources –‘EMA toolkit’
Landscape review –
challenges prioritised
![Page 18: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/18.jpg)
» EMA Requirements Map
» Feedback hub
» Reliable submissions
» EMA systems integration web resource
» Assessment & feedback toolkit
Four projects
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 18
![Page 19: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/19.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 19
» Project to identify, validate and specify key EMA requirements and workflows, through development of generic models that represent the diversity of practice
» Providing clarity and transparency on workflows (particularly from submissionto return of grades). Helps systems suppliers better support good pedagogic practice and helps institutions review their own practice
EMA requirements map
![Page 20: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/20.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 20
» Develop a visual way of presenting workflows; map current systems to workflows; and engage with suppliers and developers to fill gaps
EMA requirements map
![Page 21: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/21.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 21
» System independent tool for aggregating, organising and presenting feedback at a programme level, for both staff and students. Also enables interaction around feedback between staff and students
Feedback hub
![Page 22: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/22.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 22
» Students benefit from aggregatedview of their feedback to support self-reflection on progress; lecturers see a more holistic view of students’ progress and better understand an individual’s progression to identify where intervention or support is needed
Feedback hub
![Page 23: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/23.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 23
» Resource to address lack of interoperability between marking systems and student records systems and subsequent need for ‘workarounds’ by staff
» Helps institutions find solutions to EMA systems integrations issues via improved workflows and by sharing existing integrations (use cases and code)
Systems integration
![Page 24: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/24.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 24
» Gaps can be surfaced and prioritised for potential development
» A community of practice might support this resource
Systems integration
![Page 25: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/25.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 25
» Web-based toolkit of resources: practices, processes, policies, technical integrations etc based around the assessment and feedback lifecycle
» Examples of solutions , enabled by technology, based on pedagogy and underpinned by research, that can be repurposed for local contexts
EMA toolkit
![Page 26: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/26.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 26
» Addresses the question of ‘what does good assessment design look like?’ and enhances assessment literacies of staff and students
» Navigation routes through the toolkit e.g. by identified workflow and stakeholder groups (academic staff, administrative staff, student)
EMA toolkit
![Page 27: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/27.jpg)
» Choose one project area to work on
» The aim is to help to shape the detail of these projects and your facilitator will provide guidance. Discuss:
› What would you like to see from this project and its outputs?
› What form should the outputs take?
› How might you use the outputs within your context?
› What do you have that you can contribute?
› How can we best reflect the needs of HE and FE within this project?
Activity
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 27
![Page 28: Electronic management of assessment - Jisc Digital Media 2015](https://reader031.vdocuments.us/reader031/viewer/2022032714/55ab51f61a28ab28208b4677/html5/thumbnails/28.jpg)
10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 28
» Landscape report available from: bit.ly/jisc-ema
» Join the conversation on the blog: ema.jiscinvolve.org/and on twitter #jiscassess
» Join the mailing list:
www.jiscmail.ac.uk/tech-enhanced-assessment
» Guides, case studies, videos and other resources on assessment and feedback: bit.ly/jisc-assess
Find out more