technology- enhanced examinations at the university of bradford john dermo, centre for educational...
TRANSCRIPT
Technology-enhanced examinations at the University of Bradford
John Dermo, Centre for Educational Development,
University of Bradford
University of Edinburgh 23rd April 2012
2
Technology enhanced exams at UoB
• A strategic approach to technology enhanced assessment
• Drivers: including the student perspective• Developing a facility for e-assessment• Innovative e-assessment at UoB
3
e-Learning Pathfinder Programme“Embedding Support Processes for e-Assessment”
Pathfinder Project 2007-8
http://www.bradford.ac.uk/educational-development/educational-research/pathfinder-caa/
4
JISC Institutional Exemplar Project“Integrating thin client systems and smart card technology for flexible, accessible and secure E-assessment.”
ITS4SEA Project 2007-9
http://www.jisc.ac.uk/whatwedo/programmes/institutionalinnovation/its4sea.aspx
Institutional Innovation Programme
5
Drivers: reasons for focus on online assessment
• University e-Strategy (2004-9)• LTA strategy and Academic Framework
(2009-14)• National Student Survey• Increased use of e-learning• Student expectations
6
• In 20 years HE has seen radical and unprecedented change (Education Act, 1992; Dearing Report,
1997; Roberts Report, 2003; Leitch Report, 2006; Browne, 2010; CSR, 2010)
• Learners have different expectations and assumptions about their HE experiences
• The student body has become dramatically more heterogeneous
• Impact of student fees
7
NATIONAL UNION OF STUDENTS’ PRINCIPLES OF EFFECTIVE ASSESSMENT (2009)
1. Should be for learning, not simply of learning
2. Should be reliable, valid, fair and consistent
3. Should consist of effective and constructive feedback
4. Should be innovative and have the capacity to inspire and motivate
5. Should measure understanding and application, rather than technique and memory
6. Should be conducted throughout the course, rather than being positioned as a final event
7. Should develop key skills such as peer and reflective assessment
8. Should be central to staff development and teaching strategies, and frequently reviewed
9. Should be of a manageable amount for both tutors and students
10. Should encourage dialogue between students and their tutors and students and their peers
8
“Please, Sir, may I have some more exams?”
Becka Colley, Dean of Students, University of Bradford. Keynote speech at eAssessment Scotland, 2011http://www.slideshare.net/rcurrant/please-sir-may-i-have-some-more-exams
With apologies to Charles Dickens and Ronald Searle
9
There was a strong preference to retain two exam sessions. This was felt to be more manageable in terms of revision. Students were not in favour of one exam session at the end of the academic year. A staged approach with summative assessment was viewed as the most appropriate. Also, January exams give an opportunity for students to ‘trial’ their exam revision and writing skills.
[UBU feedback on review of the academic year, June 2011]
Student Union Feedback
10
• Dedicated support• Move towards integrated
processes and systems• Workflow model and clear roles
and communication channels• Approved policies• Training materials and sessions,
briefings• Develop new flexible space for
e-assessment
Developments at UoB
11
100 seat e-assessment cluster
12
13
Smartcards: link each exam to a different card
VMware and Sun Ray servers
14
Benefits of e-assessment suite for online assessments
• Can re-image software available very quickly between exams
• Easy to control software available, flexible using Appsense • Can link assessment to card to allow multiple simultaneous
exams in single location• Easier to invigilate – more secure• Room easier to find• More pleasant environment• Easier to manage extra time students
15
First used January 2009, since when the use for summative e-assessment has gradually increased:
Usage: 2500 different students per year, from more than 30 modules covering range of subject areas across disciplines
“I’ve already told the DVC that we will need another cluster like this within 2 years”
Associate Dean (Teaching and Learning) Life Sciences
“This is just what I always wanted but we couldn’t do on the PCs!”
Lecturer in School of Management
16
• Growing interest in e-assessment
• Further demand on facilities, especially for summative assessment
• More efficient use of space (smartcards, flexible scheduling)
• “Overflow” room to increase capacity to 130
• Single rooms available when necessary
17
Some Student Feedback
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
120
The room is appropriately set out for a summative assessment.
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
The questions were easy to read and answer on the computer screen.
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
I preferred doing the assessment on the computer rather than on paper.
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
Using computerized tests would be appropriate for other summative tests or exams.
18
Varied use of e-assessment suite
• exam periods: block booked by exams office for summative e-assessment
• term time – managed by room bookings for• one off e-assessments• regular teaching sessions
• otherwise: open 24/7 as self access cluster for students
19
24/7 Cluster Use
• Room in regular use• “vibrant environment”• Pleasant learning space• Encourages group work• Demand during exam period• But… limitations of thin-clients?
20
Challenges
How to manage the resource 365 days a year?Who owns the facility?How to get the most out of the investment?How to manage larger groups?Booking the room?Server capacity to allow 100+ logins? Expanding the use of the room to include essay exams (cf interest in e-submission and e-marking)
21
Integrating Institutional Processes
22
But don’t forget the human processes which go with the technical systems
23
Innovation requires commitment and acceptance of responsibility across the institution
instructors course teams invigilators students department admin exams office disabilities office senior managementIT services support teamsroom bookings line managersestates
24
Lessons Learned at Bradford?
Challenges are not just technology-basedCommunication is keyYour systems are only as good as the data in themAutomated systems require humans to play their part
25
We all know what an exam looks like, don’t we??
From this… …to this.
26
Examples of authenticity
Department of Biomedical Sciences: • Combination of web-based genetic
research tool (BLAST) with MCQs to deliver authentic, real-life tasks which require application of knowledge and understanding in a real-life professional context
• Can be assessed automatically = great for large numbers of students
27
School of Medical Sciences: • Use of formative “e-assessment for
learning” (in F42 and via VLE) to deliver automatic generic feedback to students.
• Work has been carried out on delivering question level feedback and topic based feedback – students would appear to best appreciate a combination of these.
Examples of authenticity
28
School of Optometry: • Use F42 for “Active problem-based learning” • Formative group based case studies• Information is automatically and gradually
released during process• Students make joint decisions, then receive
email feedback. • Group focus initially then individual
summative process
Examples of authenticity
29
• “Hybrid” assessments
• Some questions automatically marked, others done by a human
• More efficient use of human resources to allow time to be spent on students and feedback processes.
Other Developments
30
• School of Health Studies, School of Management and School of Lifelong Education and Development:
• Using e-peer assessment• Students grading performance and/or give
feedback collaborative group tasks. • This assessment can take place either all
together in the e-assessment suite or remotely via the VLE.
Other Developments
31
“Activity-based Student-centred” (AbSc) teaching and learning: an Engineering perspective at Bradford
• 8 e-Assessments in 12 weeks in F42 (including 6 bi-weekly assessments, one e-assessment on lab work and one mid-term e-assessment)
• Supported by online quizzes – self assessment with feedback done in own time in VLE
• Final summative e-assessment in F42
Other Developments
32
F42
