School of DentistryFACULTYOFMEDICINEANDHEALTH

EVALUATION OF AN ELECTRONIC MARKING SYSTEM FOR AN UNDERGRADUATE DENTAL OSCE ASSESSMENT

Yorke V.C., Hallam J. and Wardman M.J.

OSCEisahighstakesclinicalassessmentandthesystemsusedtomark,recordandprocesscandidateperformancedatamust,aswellasbeingrobust,beefficienttousebybothexaminersandadministrativestaff.

OSCEsattheSchoolofDentistry,UniversityofLeeds

• Typically,100studentstakeOSCEsateachsitting,with14-16stations

• Opticallyreadpapersheetshavetraditionallybeenusedformarking• Significantadministrativetimehasbeenneededtocollate,check

andprocessapproximately1400papers• Markshavenotbeenavailabletoanalyse immediatelymeaningthat

releaseofmarksanddeliveryoffeedbackisnotastimelyasrequiredbyeducationalqualitystandards.

AnewelectronicsystemofmarkingforOSCEhasrecentlybeenintroducedinLeedswiththeaimofimprovingefficiency.Examinersandadministrativestaffmayhaveconcernsthatelectronicsystemsarenotasreliableastraditionalpaperformatsandaboutthetransitiontoatechnologybasedsystem

Introduction

Conclusion

Aims of the Study

Materials and Methods

Results

.

ToevaluatetheintroductionanduseofanelectronicmarkingsystemforanundergraduatedentalOSCE

• Todeterminetheadaptabilityof,andeaseofusefor,examiners• Togainaninsightintotheexperiencesofexaminersand

administratorsintheuseofthenewmarkingsystem

Trainingintheuseofanelectronicmarkingsystem(usingvideoandverbalpresentation)wasprovidedtoallstaffwhoexaminedayear4undergraduatedentalprogramme OSCEwhenthenewsystemwasintroducedin2017.Aquestionnairewasdistributedtoallexaminers(39)andadministrativestaff(6)followingcompletionoftheOSCE.UniversityofLeedsEthicsResearchapprovalwasgained130117/JH/221.

https://www.youtube.com/watch?v=l5Rjy3dY0A4Marking system training video

• Bothexaminersandadministrativestaffadaptedeasilytotheimplementationofanewelectronicmarkingsystemandwouldchoosetousethesystemagain.

• Previousandcurrentuseoftechnologyshouldbeconsideredwhendecidingontrainingrequirements.

• ThisstudymaybeofinteresttootherdentalschoolsconsideringtheimplementationoftechnologybasedOSCEmarkingsystems.

”I thought the system was great! The

training video was very good, and simple to follow.”-

Examiner

“Despite my initial hesitation, it was a

very positive experience- I think we should continue to use

this system as it makes all our lives

easier!!”- Administrator

80%ofexaminersand75%ofadministrativestafffoundtheelectronicmarkingsystemeasytouse.Currentandprevioususeoftechnologywasfoundtobeinfluentialintheneedforfuturetraining:

Overall,90%ofexaminersand85%ofadministrativestaffhadapositiveexperienceusingthesystemand100%ofallstaffsurveyedwouldchoosetousethesystemagain.