overview of helfelectronic management of assessment surveyresults 2013
DESCRIPTION
Overview of HeLFElectronic Management of Assessment SurveyResults 2013TRANSCRIPT
![Page 1: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/1.jpg)
Overview of HeLFElectronic Management of
Assessment SurveyResults 2013
Dr Barbara Newland, Brighton
Lindsey Martin, Edge Hill
Alice Bird, Liverpool John Moores
![Page 2: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/2.jpg)
Electronic Management of Assessment (EMA) includes a wide range of activities so the following definitions were used:
Definitions
eSubmission electronic submission of an assignment
eMarking electronic marking (including offline marking eg in Word)
eFeedback electronic feedback (ie text, audio, video but not hard copy)
eReturn electronic return of marks
![Page 3: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/3.jpg)
To identify current practice with regard to Electronic Management of Assessment (EMA) in UK HE
To gain a snapshot of the strategic overview identifying key issues relating to strategic change, policies and practices
To reflect on longitudinal developments from findings of 2011 and 2012 surveys.
Aim
![Page 4: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/4.jpg)
A network of senior staff in institutions engaged in promoting, supporting and developing technology enhanced learning
Over 138 nominated Heads from UK Higher Education institutions
A regular programme of well attended events
Represents the interests of its members to various national bodies and agencies including the Higher Education Academy and JISCwww.helf.ac.uk
Heads of eLearning Forum (HeLF)
![Page 5: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/5.jpg)
The survey was available to HeLF members who were asked to respond with regard to their knowledge of their own institution.
The survey was available in March/April 2013 and took about 10 minutes to complete
The questions were a mixture of closed multiple-choice and multiple selection as well as open response type
Participants were assured that all data collected in the survey would be held anonymously and securely
No personal data was asked for or retained unless the participant indicated a willingness to participate in the follow-up activity
The results are being analysed using quantitative and qualitative methods
52 responses from HeLF members – 38% response rate
Methodology
![Page 6: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/6.jpg)
More positive attitudes towards EMA and its normalising within their institutions
Challenges in relation to buy-in, take up and roll-out processes, functionality, service disruption and standardisation and whether the latter is desirable and achievable.
Main findings
![Page 7: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/7.jpg)
Institution-wide policy
eSubmission eMarking eFeedback0
5
10
15
20
25
30
35
40
45
50
YesNo
![Page 8: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/8.jpg)
Closer to having an institution-wider policy than a year ago?
eSubmission eMarking eFeedback0
5
10
15
20
25
30
YesNoDon't know
![Page 9: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/9.jpg)
Is the move to eFeedback leading to more standardisation in terms of the feedback provided?
University-wide
Department-wide
Neither
![Page 10: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/10.jpg)
Which software does your institution recommend for eFeedback in text format?
Turnitin (stand-alone)Turnitin (integrated into VLE)VLEHome grownOther
![Page 11: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/11.jpg)
Current practice
05
10152025303540
University-wideSome department-wideIndividual academics
![Page 12: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/12.jpg)
Is there evidence of increased use of mobile devices?
eSubmission eMarking eFeedback0
2
4
6
8
10
12
14
16
Academic staffStudents
![Page 13: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/13.jpg)
Have you experienced critical issues with the eSubmission, eMarking or eFeedback software over the past year?
YesNoDon't know
![Page 14: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/14.jpg)
Technical issues◦ service interruptions and outages◦ limited functionality and bugs with integration tools
Institutional circumstances◦ lack of university standardisation◦ lack of institutional policy◦ lack of consistency across Schools which meant that
the existing technology could not meet all the requirements of Schools
Skills - staff skills gap
Critical issues
![Page 15: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/15.jpg)
eSubmission is a high stakes activity
Concerns about the service being hosted by a third party and also with quality standards
Other challenges:◦ unsatisfactory problem solving◦ unannounced upgrades and maintenance downtimes
during assignment submission periods◦ service not being able to cope with large numbers of
students◦ lost assignments although these were eventually
recovered
Technical concerns
![Page 16: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/16.jpg)
Where Turnitin unavailability coincided with submission deadlines, one institutional representative reported that they had to issue advice to academics to extend deadlines.
For another institution individual staff made their own contingencies, for example, paper submission and paper feedback.
“Caused a lot of stress!”
Solutions?
![Page 17: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/17.jpg)
Does your institution have a fully integrated EMA approach?
YesNoUnder consideration
![Page 18: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/18.jpg)
Does your EMA system cope with the following marking processes?
Anon
ymou
s m
arking
Double
mar
king
Exte
rnal e
xam
iner
s0
5
10
15
20
25
30
YesNoDon't know
![Page 19: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/19.jpg)
How would you rate attitudes to eSubmission?
Positive Negative Neutral Don't know0
5
10
15
20
25
30
35
40
45
50
Academic staffAdministrative staffStudents
![Page 20: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/20.jpg)
How would you rate attitudes to eFeedback?
Positive Negative Neutral Don't know0
5
10
15
20
25
30
35
40
45
Academic staffAdministrative staffStudents
![Page 21: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/21.jpg)
How would you rate academic staff attitudes to eMarking?
PositiveNegativeNeutralDon't know
![Page 22: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/22.jpg)
‘There has been a change in attitude towards eFeedback, with a number of members of staff recognising that they already do this in some form.’
It is regarded as an urgent sector-wide agenda and a common student expectation, and for many institutions it is increasingly becoming embedded in departmental practice
Changing attitudes
![Page 23: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/23.jpg)
Students
Staff
Senior management
National agendas
‘Student use. Getting positive stories from staff and student users.’
‘Positive student feedback through module evaluations and staff, student consultative committees.
Drivers
![Page 24: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/24.jpg)
Consultation with all stakeholders
Leadership by senior management - buy-in and support
Effective communication strategy - visible campaigns,
online awareness-raising and support resources
Departmental champions
Digital literacy - the provision of training
Support from central/departmental learning technologists
Technical – robust, easy to use systems integrated with
other central systems.
Critical success factors
![Page 25: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/25.jpg)
While mandatory policy has been a key driver in some institutions, a number of respondents suggest that inclusive policy and flexible processes work well
‘It depends very much on the culture of your organisation. Here a partnership model between central services, and School / faculty admin and academics tends to work. Top down imposition of systems is less successful. In other institutional cultures this may be different.’
Institutional differences
![Page 26: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/26.jpg)
‘e-Submission is quite straight forward. However, we could have managed the process of e-marking and feedback better. Academic staff need a lot of time to come round to the idea if they are changing years of established practice.’
‘Never underestimate the effort involved with winning hearts and minds of colleagues.’
Challenges and concerns
![Page 27: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/27.jpg)
Assessment is mission critical – it determines whether students achieve, progress and gain awards (and at which level)
All stakeholders in the assessment process are wary of making errors and need to be convinced of the merits in changing long-established practices
Change in practice supported by technology is only worthwhile if it effectively enhances stakeholders’ experiences and/or improves the overall process workflow
Change management - EMA
![Page 28: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/28.jpg)
‘There is a general acceptance of e-submission policy, with fewer complaints, and anecdotal evidence points to greater academic participation in emarking and efeedback’.
Usage has moved from individual early adopters to more widespread, formalised use. There is an increased appetite for standardisation across faculties. However, a very small proportion of respondents report no, little or slow moving change in attitudes.
Summary
![Page 29: Overview of HeLFElectronic Management of Assessment SurveyResults 2013](https://reader035.vdocuments.us/reader035/viewer/2022081516/5549805ab4c905b96a8b475b/html5/thumbnails/29.jpg)
Dr Barbara Newland [email protected]
Lindsey [email protected]
Alice [email protected]
Acknowledgement: Dr Rachel Masika, University of Brighton
Contact details