effective course evaluation

16
Effective Course Evaluation The Future for Quality and Standards in Higher Education A summary report from interviews among senior academics and student representatives

Upload: eric-bohms

Post on 11-Apr-2017

110 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Effective Course Evaluation

Effective Course EvaluationThe Future for Quality and Standards in Higher Education

A summary report from interviews among senior academics and student representatives

Page 2: Effective Course Evaluation
Page 3: Effective Course Evaluation

Acknowledgements 4

Introduction by Eric Bohms,

Managing Director, Electric Paper Ltd 5

Executive Summary 6

Improving Response Rates 7

Improving Student Feedback 8

Improving Turnaround Time 9

Improving Survey Administration 10

Improving The Student Experience 11

Conclusion 12

Evaluation Potential 13-14

CONTENTS

Page 4: Effective Course Evaluation

4 Effective Course Evaluation

The editors of this report, Phil Smith and Owen Morris at Communications Management,would like to thank the following people who were interviewed in July and August 2011:

• Professor Susannah Quinsee, Director of Learning Development, City UniversityLondon.

• Professor Ian Marshall, Deputy Vice-Chancellor (Academic), Coventry University.

• Professor David Coates, Dean of School of Life Sciences’ Learning and TeachingDivision, University of Dundee.

• Professor Glenn Burgess, Pro-Vice-Chancellor of Learning and Teaching, University of Hull.

• Professor Andy Pitsillides, Chair of Teaching Quality Committee, Royal VeterinaryCollege, University of London.

• Professor Huw Morris, Pro-Vice-Chancellor (Academic), University of Salford.

• Professor Alan Speight, Pro-Vice-Chancellor (Student Experience and Academic QualityEnhancement), University of Swansea, and member of Quality Assurance Agency Boardof Directors.

• Alex Bols, Head of Education and Quality, National Union of Students.

• Alex Smith, former Academic Affairs Officer, University of Leicester Students’ Union, and member of Quality Assurance Agency Board of Directors.

• Alex Nutt, current Academic Affairs Officer, University of Leicester Students’ Union.

ACKNOWLEDGEMENTS

Page 5: Effective Course Evaluation

Effective Course Evaluation 5

The 2011-12 academic year is a big one for the Higher Education sector. This is the yearwhen, more than at any previous point at time, UK universities will have to provide clearevidence of the ‘value’ of studying at their institution.

Universities are digesting the implications of the Higher Education White Paper, published inJune 2011, which has asked institutions to be more accountable to students on teaching qualityand to provide greater transparency in areas such as the student experience. Specifically thepaper says that “universities will be expected to publish online summary reports of studentsurveys of lecture courses, aiding choice and stimulating competition between the bestacademics”. Therefore, gaining effective student feedback on courses and lecturers to enabletransparent reporting is a must.

At Electric Paper we work with over 600 universities in the UK and worldwide to help them toevaluate their courses through effective student feedback via our automated paper and onlinesurvey management system EvaSys. We hope that this report will draw attention to the latestissues and trends around effective course evaluation which, in turn, will help us to develop oursupport for the Higher Education sector. But why is it so important that we get this right?

Firstly, and most importantly, with the increase in tuition fees from 2012, universities can nolonger afford to leave teaching and learning quality in the hands of academics alone. TheUniversity’s executive needs to have visibility around the quality of the course and lecturer, andestablishing the best way to tackle the ‘process’ of getting student feedback is crucial.

Secondly, the National Student Survey (NSS) is currently the primary method of gatheringfeedback on the quality of students’ courses in order to contribute to public accountability andhelp inform the choices of future applicants. Gaining comprehensive, instant, feedback oncourses and lecturers can assist universities in improving overall quality, and NSS scores.

Thirdly, as many universities have taken the decision to charge the full £9,000, it will becomeeven more important for students to distinguish between institutions on issues of quality.Quickly capturing feedback across all courses, providing students and their parents withevidence-based feedback about the quality of teaching and learning, will be a big advantage.

Fourth, individual universities need to stand out from the crowd. With the introduction of ‘KeyInformation Sets’ universities need to do everything they can to provide indicators of quality.Capturing, and responding positively to, student feedback on course evaluation can only helpboost recruitment targets and reputation.

Finally, implementing an effective process for capturing student feedback around courses andlecturers is a money-saver for universities. Institutions spend a huge amount of time andmoney on survey administration, and approaches are inconsistent (e.g. with management ofsurveys sitting centrally in one university, and departmentally in another). An effective processfor course evaluation can offer significant cost and efficiency savings, addressing the hiddencosts in survey administration and freeing up staff expertise for other areas.

Eric Bohms, Managing Director, Electric Paper Ltd

INTRODUCTION

Page 6: Effective Course Evaluation

6 Effective Course Evaluation

• Universities need to work harder atfeeding back to students the actions theywill be taking as a result of input providedfor course and lecturer evaluation surveys.

• End-of-module evaluation is a particularstumbling block in the provision offeedback to students - and feedback canbe slow - but moving to mid-moduleevaluation can help to improve theprocess.

• Ideally students want the opportunity toexpress their views on courseimprovements at a time that theirfeedback benefits them directly.

• Universities need to embrace newtechnologies to improve turnaround time- but effective feedback can be gained viaa combination of paper and onlinesurveys.

• Universities should establish a moreconsistent (centralised) approach to surveyadministration - including a standard setof survey questions - to enable effectivebenchmarking at course and institutionallevel. However, individual departmentsshould have the flexibility to includebespoke questions for particular courses.

• In-class student involvement in surveyadministration can increase commitmentas they are stakeholders in the process.

• Effective course evaluation is necessaryfor universities to provide a clear evidencebase to demonstrate their ‘value’ tostudents.

The purpose of this report is to examinethe latest issues and trends in courseevaluation. In-depth interviews have beenconducted with 10 senior academics andstudent representatives to draw out ageneral sense of how universities arecurrently approaching the ‘process’ ofstudent surveys, and what improvementsneed to be made to gain more effectivestudent feedback on courses and lecturers.

This qualitative research is particularlyrelevant in the context of the 2011 HigherEducation White Paper, which has askeduniversities to be more accountable tostudents on teaching quality and to providegreater transparency in areas such as thestudent experience. Rising tuition fees in2012 mean that gaining (and respondingeffectively to) student feedback on coursesand lecturers is increasingly important foruniversities as they have to provide clearevidence of the ‘value’ of studying at theirinstitution.

This report finds that:

• Many universities seeking feedback oncourses and lecturers via surveys struggleto achieve a meaningful response fromstudents, which is partly due to studentsbeing inundated with requests tocomplete surveys.

• Student representatives have indicatedthat students are not effectively engagedin the feedback process and, for some,providing feedback can even beintimidating.

EXECUTIVE SUMMARY

Page 7: Effective Course Evaluation

Effective Course Evaluation 7

Quality Assurance Agency Board ofDirectors, agreed that there is a need foruniversities to ensure that students are“properly engaged” and even “incentivised”to take part in the process. However,“providing feedback can be intimidating forstudents,” he said. Alex Nutt, whosucceeded Smith at Leicester’s Students’Union added that “despite the processbeing anonymous some students feelintimidated by the idea of giving negativefeedback in case it reflects on them”.

Coventry University has managed toincrease its response rates by using paperand moving to mid-module surveys.“Historically universities conduct end-of-course, or end-of-module surveys, but bythe time the feedback has been analysedand results published the students havegone away,” said Professor Ian Marshall,the University’s Deputy Vice-Chancellor(Academic). “We moved to online surveys,but the response was dreadful, so last yearwe introduced mid-module surveys andwent back to paper. The response wassuper, and we are now able to turn aroundfeedback in two weeks maximum.” Headded that if a module is perceived to beunderperforming “we will focus on those”.

At Coventry University mid-modulesurveys are handed out in class by 120senior student representatives who areinterviewed and selected by the University’sStudents’ Union. The University of Hull isalso working closely with its Students’Union to develop its approach toimproving response rates. “What we havefound is that students are tolerant ofproblems, but they want us to listen, feedback quickly, and respond,” said ProfessorBurgess.

Many universities seeking feedback oncourses and lecturers via surveys are stillstruggling to achieve a meaningfulresponse from students - that is, aresponse rate of over 50% that will ensurestatistical validity in order to evaluateteaching quality and make improvements.

Professor Glenn Burgess, Pro-Vice-Chancellor of Learning and Teaching at theUniversity of Hull, said the problem wasthat students were simply inundated withrequests to complete surveys. “A key issuefacing universities in relation to gainingeffective course feedback from students isaround achieving significant response ratesto any questionnaire-based approach.Students are increasingly overloaded withsurveys - from the National Student Surveydownwards - so you need a good reason toencourage participation. It’s also importantto appreciate the qualitative/quantitativedivide in surveys. Often the most usefulinformation is gained through qualitativefeedback, but this is also the most difficultto achieve as it requires more time fromrespondents.”

This is a view shared by ProfessorSusannah Quinsee, Director of LearningDevelopment at City University London,who said that “a lot of course evaluation” isgeared towards the National StudentSurvey. “Ideally you want to engage indialogue with students, because this ismore useful than knowing that half theclass love a module and half the class hateit. You want to know more about what themiddle group think. You also need moreopen questions rather than tick box, butstudents are not so keen on thesequestions. So there is an issue over studentengagement, and an associated issuearound the extent to which students engagein the language used in evaluation surveys.”

Alex Smith, former Academic AffairsOfficer at the University of Leicester’sStudents’ Union, and a member of the

THE NEED TO...IMPROVE RESPONSE RATES

“We introduced mid-module surveysand went back to paper and theresponse was super”

Page 8: Effective Course Evaluation

8 Effective Course Evaluation

not going to feel the benefits of anyimprovements made,” said City’s ProfessorSusannah Quinsee. “Ideally you wouldwant to explore in-module evaluation,which we are doing, but that takes a lot oftime both in terms of implementation andanalysis.”

Student representatives agree that end-of-module evaluation is no longersufficient. “It’s a flawed system,” saidSmith. “In my experience universities oftenask for student feedback at the end of acourse, but by then it’s too late, and thestudents do not see any results from thefeedback they’ve given. Universities oftenprofess a need to get the feedback systemright but I don’t think this has necessarilyhappened yet as, by only asking forfeedback at the end of a course, they aregathering entirely new opinions from adifferent cohort of students and thereforecannot compare like-for-like.”

Professor Alan Speight, Pro-Vice-Chancellor (Student Experience andAcademic Quality Enhancement) at theUniversity of Swansea, and a member ofthe Quality Assurance Agency Board ofDirectors, said that establishing aneffective system for student feedback wasvital for universities’ reputation. “There is aneed to dig down to obtain feedback as itdoes not take much to make students feeldisgruntled. Universities are only as strongas their weakest link and poor delivery by asubset of staff that goes unchecked couldmake a lot of difference to the overallteaching reputation of an institution.”

“Students are more interested inoutcomes - so it’s important foruniversities to be very clear on what theyare able to do, and equally be honest onwhat they are not able to,” addedCoventry’s Professor Ian Marshall.

Through this research a clear gap hasemerged around the extent to which UKuniversities currently feed back tostudents the action they will be taking asa result of input provided in course andlecturer evaluation surveys.

“Students need to provide feedback, thenthe universities need to take that on boardand do what they can with it, and thenprovide clear feedback to the students onthe actions and outcomes,” said ex-University of Leicester Students’ UnionAcademic Affairs Officer, Alex Smith. “Thefeedback loop must underpin everything.”

Alex Bols, Head of Education and Qualityat the National Union of Students (NUS),said that “in the majority of cases”students who participate in courseevaluation surveys, in his experience, arethen not told what happens as a result ofthe process. “It’s important for universitiesto close the loop and tell students whathas happened - or hasn’t happened - as aresult of the feedback provided and why.This should not be an autopsy at the endof a course, but a process embeddedthrough the learning experience so that itis of benefit to the student giving thefeedback and their experience.”

End-of-module evaluation appears to bea particular stumbling block in theprovision of feedback to students. Forexample, City University London is threeyears into a JISC-funded four-year projecton curriculum design, and staff havelooked at the issue of effective courseevaluation. “What we’re finding is thatevaluation for a lot of students is notmeaningful because they are being askedto give feedback on a course or modulethey are just completing, and are therefore

THE NEED TO...IMPROVE STUDENT FEEDBACK

“It's important for universities to closethe loop and tell students what has orhasn't happened”

Page 9: Effective Course Evaluation

Effective Course Evaluation 9

universities need this for both instantfeedback and for internal benchmarkingleading to organisational improvement.

Student groups agree. “By examining the process and proving that they areinterested, and capable of acting upon, thefeedback they receive in a timely manneruniversities could build partnerships withtheir students to add value all-round,” saidthe NUS’ Alex Bols. “Students should beable to express how their course could beimproved while it impacts on them.”

Exploiting innovative new technologiescould support the requirement to improveturnaround time, according to ProfessorHuw Morris, Pro-Vice-Chancellor(Academic) at the University of Salford,who in his previous role as Dean ofManchester Metropolitan UniversityBusiness School led the trial of coursesurveys via mobile phones. “Going forwardI anticipate the Higher Education sectorwill need to utilise online devices tocapture student feedback, but at the sametime ensure this is not done in an intrusivemanner. Some element of compulsion forstudents in providing feedback will also behelpful in ensuring that the results arerepresentative of underlying views.”

In addition to module evaluation, there isa clear need for feedback on theunderlying quality of the students’educational experience, Professor Morrissaid. “The National Student Survey and, toa lesser extent, the International StudentBarometer, are helpful and mostuniversities respond to these. But while weknow about student happiness andsatisfaction, we don’t know that muchabout how their educational experiencesprepare them in the longer term. Herethere is a need to draw on evidence fromother tools which assess the contributionof the students’ educational experience totheir career prospects and personal andsocial development in the future.”

Relating to the issue of student feedback isthe challenge that universities face inturning around the findings of courseevaluation surveys.

Professor Andy Pitsillides, Chair of theTeaching Quality Committee at the RoyalVeterinary College, University of London,admitted that the process can be “veryslow” in Higher Education. “As academicswe know that students want quickfeedback as it helps their learning, yet weare restricted by the annual cycle ofsurveying. The exercise is also paper-driven, so it is more cumbersome than wemight like, and we are looking at a processof electronic feedback to augment thisapproach.”

It’s a major issue facing the sector,according to Swansea’s Professor AlanSpeight. “Turnaround time is vital, but a bigproblem at the moment. In some areassurveys may be paper-based and manuallyprocessed, which means that someone hasto physically input the results and is open tohuman error. Also the feedback may comeback when it is too late for the staff to doanything about it as they are, by that time,already committed to a teaching pattern forthe next academic session, which in turnmakes the students wonder why theirthoughts have not been listened to.”

A survey of 100 university administratorsby Electric Paper, conducted in 2011, found74% of respondents agree that both paperand online surveys is a requirement inHigher Education. It’s not a question ofeither/or, the respondents said, becauseboth methodologies can provide highresponse rates and efficiency savings forthe right surveys in the right context.However, turnaround time is key as

THE NEED TO...IMPROVE TURNAROUND TIME

“As academics we know that studentswant quick feedback as it helps theirlearning”

Page 10: Effective Course Evaluation

10 Effective Course Evaluation

Other universities are making movestowards centralisation, such as the Universityof Hull. “We have a central code of practicefor survey administration which provides thefoundation for all modular and programmesurveys,” said Hull’s Professor GlennBurgess. “But within the University there iscurrently no consistency on the level ofapproach and questions being asked locally,and we have plans to change that. Thisbrings a pressure on resource for centralprocesses and for us to introduce a particularway of working. It also links to wider policyissues - for example, how is feedback linkedto staff appraisal, what happens in terms ofresponding to poor feedback, and howpublic do you make this information.”

The University of Swansea’s Professor AlanSpeight said that there were historicchallenges to overcome for many institutionsin order for the feedback process to becomecentralised. “The difficulty has been thatsurveys carried out by different departmentshave evolved organically and been developedin silos that are tailored just to the needs ofone faculty. This results in a lack of coreinformation across the institution, mixedresponses and no consistent use of data.There should be a unified approach thatincludes core questions and specifies the waythe feedback is processed - which allowsbenchmarking and consistency. A commonset of core questions should be ownedinstitutionally, with subject areas able toselect from a bank of optional additionalquestions.”

As with the Coventry University model,Professor Speight suggested that studentscould be brought on board to help co-ordinate the process.

To improve response rates, student feedbackand turnaround time, universities need tohave a more consistent approach to surveyadministration. Through this research it hasbecome clear that the management ofsurveys sits centrally in one university, anddepartmentally in another. In addition, a 2011Electric Paper survey found that academicsare managing their own surveyadministration at 1 in 3 institutions - and thatmost universities are facing ‘absolute chaos’around survey management.

What is clear from interviews with senioracademics is that centralisation of courseevaluation feedback analysis is required.Following the arrival of Vice-Chancellor,Professor Paul Curran, City UniversityLondon has introduced a centralisedmodular evaluation system. “We now have astandard set of questions for surveys,managed centrally, which individual schoolscan add to if they wish, and the results ofthese are now part of staff appraisals,” saidCity’s Professor Susannah Quinsee. Thescheme being rolled out by CoventryUniversity is managed by a central unitbased in student services, while theUniversity of Salford has establishedstandard questionnaires for module, leveland programme-centred evaluation whichare co-ordinated by a central planning andperformance department.

Professor David Coates, Dean of the Schoolof Life Sciences’ Learning and TeachingDivision at the University of Dundee, said“the process of receiving and thendisseminating feedback needs to be clean toensure that those doing the teaching willreceive the knowledge they need”. He added:“Firstly, there needs to be consistency in thefeedback the institution receives as a whole,but equally courses should not always becompared like-for-like. A hub-and-spokemodel works best where individualdepartments are evaluated but then thisfeeds into a holistic view of the institution.”

THE NEED TO...IMPROVESURVEYADMINISTRATION

“A common set of questions should beowned institutionally, with subject areasable to select from optional additionalquestions”

Page 11: Effective Course Evaluation

Effective Course Evaluation 11

and lecturer feedback method so therecan be constant improvements that willdetermine a student’s opinion of theirinstitution based on direct experience.”

A wider issue beyond module evaluation is the need for UK universities to movetowards surveying “all student groups”,according to Salford’s Professor Huw Morris. The National Student Survey andInternational Student Barometer do not,he said, cover UK and other EU secondand third year students at Levels 4 and 5,taught postgraduate UK and other EUstudents at level 7, or UK and other EUpostgraduate research students at levels 7and above.

The UK may also wish to take note ofdevelopments across the Atlantic, Professor Morris said. “In the USA there are other survey methods which are morecomprehensive and which focus moreclearly on contribution. For example, Pewstudies and the National Survey ofStudent Engagement provide morecomprehensive and possibly more candidand easily accessible measures of studentexperiences in particular institutions andparticular colleges, schools anddepartments. With the changes in tuitionfees, I think we will see more informationof this kind being expected by studentsand their parents because they will wantto measure return on investment.”

However, City’s Professor SusannahQuinsee, said she had “difficulty” with theoverall notion of students as consumers.“That implies a customer-supplierrelationship where education is aboutpartnership and dialogue. Universities doneed to get students more involved inprogramme design, and evaluation andfeedback is all part of that, but studentsalso need to work with universities to tell us what data they find useful, whatthey expect, and above all what they find meaningful.”

The volatile Higher Education landscape -the increase in student tuition fees andthe focus on student-led decision-making- means that universities are underpressure to be more accountable andtransparent on issues of quality.

Alex Nutt, Academic Affairs Officer at theUniversity of Leicester Students’ Union,said that prospective students and theirparents “will want to know as much aspossible” in order to make their universitydecision, citing examples from the qualityof lectures to resources in the library. “Ialso think they will want to know thatinstitutions take the concerns of studentsseriously and that education is seen as acollaborative partnership between theuniversity and the students, not just abusiness transaction.”

“There is a going to be a race toimprove in Higher Education and theresults will be there for all to see, whereaspreviously things that were not done aswell may have been brushed under thecarpet,” explained Nutt’s predecessor,Alex Smith. “It’s brilliant news forstudents who to all intents and purposesare consumers and have the right to knowthis information.”

Universities are only too aware of theneed to provide a clear evidence base todemonstrate their ‘value’. “Of paramountimportance in the future will be thestudent experience as this will determinehow to promote the institution to theoutside world and potential students,”said Dundee’s Professor David Coates.“With the National Student Survey, andthe introduction of ‘Key Information Sets’,it is vital that there is an effective course

THE NEED TO...IMPROVE THE STUDENT EXPERIENCE

“It’s brilliant news for students who toall intents and purposes are consumersand have the right to know thisinformation”

Page 12: Effective Course Evaluation

However, universities also recognise thatcurrent evaluation processes needdevelopment - there is a clear need tosupport academics in improving the qualityof a course by providing comprehensive,instant, student feedback - and researchindicates that a combination of both paperand online surveys is a requirement inHigher Education survey administration.

What is clear is that all universities need toembrace this agenda. Prospective students(and their parents) applying for degreeprogrammes from 2012-13 will be lookingfor detailed and transparent information tohelp them distinguish between HigherEducation institutions on issues of qualityand standards in relation to courses.

Individual universities need to stand outfrom the crowd - and providing clearbaseline reporting about the quality ofteaching and learning is going to be criticalgoing forward.

As one interviewee for this report summedup: “There is a lot of activity in the sectoraround understanding the student betterand communicating with them - and thiscould be a crucial differentiation foruniversities in the next five years.”

Interviews with senior academics andstudent representatives for this report haveuncovered a strong feeling that universitiesneed to improve their approach to studentfeedback on courses and lecturers viamodule surveys.

The core perspective coming out ofuniversities is that they have difficulty inachieving a “meaningful” response fromstudents to surveys, and turnaround timefrom response to feedback is not as quick asit could or should be.

From the student perspective there is anissue around the extent to which studentsfeel engaged in the feedback process, notleast because universities do not do enoughto feed back the outcomes of their surveyinput, and the extent to which they feel thebenefit of any actions taken as a result oftheir feedback.

Some universities are moving from end-of-module evaluation to mid-moduleevaluation, which may help to address theseissues. Universities also recognise the needfor a more consistent, and centralisedapproach to survey administration whichwill potentially enable more effectiveinstitutional benchmarking.

12 Effective Course Evaluation

CONCLUSION

Page 13: Effective Course Evaluation

higher response rates.” Between 2009 and2010 more than 45,000 questionnaires werecompleted, surveying over 400 graduateteaching assistants, 600 courses, and 700permanent teachers. Once the completedsurveys were received Electric Paper scannedup to 3,000 a day, providing an instant,individual, daily report for eachcourse/teacher combination. Whereaspreviously some staff had to wait weeks,turnaround times improved dramaticallywith results returned quickly and in an easy-to-read format. “The swift turnaround ofclear and accurate results was a keyrequirement for the new system and EvaSysmore than met expectations,” Mr Page said.“EvaSys was selected after an extensivetendering exercise because it offered themost flexible yet dedicated system for ourneeds - it’s a clear winner in our eyes.”

Queen Mary, University of LondonAs part of its five-year strategic plan QueenMary, University of London (QMUL)required a system which would allow centralcollection and analysis of studentevaluations for its 16,000 students to ensurerigorous monitoring and assessment oftaught programmes. In 2010 QMULcontracted Electric Paper’s EvaSys product,which provides flexible and automatedcourse and module evaluations includingpaper and online surveys and detailedreporting. “We wanted to ensure that allstudents are invited to give anonymousfeedback on every module and that suchfeedback is collated and used systematicallyto assure and enhance the quality ofQMUL’s taught provision,” said EmmaWynne, Assistant Academic Registrar atQMUL. In the initial pilot roll-out, nine outof 17 schools participated, surveying over380 modules. Results were completedwithin three weeks with an average responserate of 59%, with some modules as high as73%. As the EvaSys system fulfilled the

In this report senior academics and studentrepresentatives have expressed an interestin improving course evaluation practices inuniversities.

Electric Paper Ltd, a software solutionprovider to the education sector based in theUK, Germany and Switzerland, has developedEvaSys Survey Management Suite, a hybridpaper and online survey automation systemwhich enables Higher Education institutionsto improve and efficiently manage theirmodule evaluations and other studentfeedback surveys.

By managing the process centrally throughthe web-based system, institutions are ableto reduce administration time and costswhilst distributing detailed analysis tostakeholders in a timely manner. Using bothpaper and online surveys, EvaSys utilises in-class evaluations to maximise response ratesand therefore validating results.

In July 2011 Electric Paper announced itslatest product release, EvaSys Education V5.0.

At the time this report was published inAugust 2011 Electric Paper was working withover 600 universities worldwide, andexamples of its UK university partnershipsare below.

London School of Economics andPolitical ScienceThe London School of Economics andPolitical Science (LSE) required a systemwhich would allow it to quickly and efficientlysurvey its 9,000 students and gain feedbackon its courses and teaching. LSE turned toElectric Paper’s EvaSys survey managementsystem, which allows the flexibility to evaluatemodules using both online and paper-basedsurveys automatically and without the needfor manual data entry. “There were two maingoals when introducing EvaSys at theSchool,” explained Mike Page, Head of ARDSystems and Business Processes at LSE. “Wewanted to improve the timeliness andaccuracy of survey results, and to encourage

Effective Course Evaluation 13

POTENTIAL FOR PROCESS IMPROVEMENT

Page 14: Effective Course Evaluation

overhaul of the process was needed. As aresult, Electric Paper’s EvaSys was purchasedby St Andrews in November 2010 and usedto produce that semester’s moduleevaluations. In addition, a core set ofquestions was developed to includecompulsory questions relating to modules,individual lecturers and individual tutors,supported by a maximum of eight optionalquestions from a pool of 16. Oncecompleted, the questionnaires were scannedand instant analysis reports produced.Reports were automatically emailed tomodule coordinators and to lecturers toreceive individual feedback tied to theinternal review of teaching and learning.This alerted senior management to anyareas of concern, and to identify goodpractice that deserved commendation anddissemination. “The introduction of EvaSyshas simplified the module feedback processimmensely,” said Carol Morris, Director ofthe Centre for Academic, Professional andOrganisational Development at St Andrews.“Results are now available in days ratherthan months allowing more time to monitorquality and implement enhancements toteaching and learning.”

objectives of the scheme, senior staffresponsible for taught programmes andQMUL’s Vice-Principal recommended theadoption of EvaSys throughout theUniversity commencing in October 2011.“The feedback from the schools was that thereports generated by EvaSys are rapid,informative and clearly presented,” MsWynne said. “From a programmemanagement and quality enhancementperspective, the presentation of results ingraphical format - including School andCollege averages - was particularly useful inidentifying areas of concern and examplesof excellent practice.”

University of St AndrewsThe University of St Andrews in Scotland hasestablished a reputation as one of Europe’sleading and most distinctive centres forteaching and research. Previously, while runcentrally, academics requested their ownindividualised questionnaire resulting inover 1,000 unique versions with ensuingdifficulties in longitudinal comparison of thequality of delivery and content of themodule. The University established that areview of the feedback system and an

14 Effective Course Evaluation

POTENTIAL FOR PROCESS IMPROVEMENT

Page 15: Effective Course Evaluation

Electric Paper works with over 600 universities toprovide effective student feedback systems

www.electricpaper.co.uk

Need help in implementing

centralised moduleevaluation?

Page 16: Effective Course Evaluation

Electric Paper LtdInternational House, 1-6 Yarmouth Place, Mayfair, London, W1J 7BUTelephone: 0203 145 3258Email: [email protected]