A consistent approach to the evaluation of teaching:a tool for ‘Raising the Bar’
Professor Suzanne CholertonPVC (Learning and Teaching)
What’s included
• Overview of Newcastle University
• Module evaluation in context
• Process and outcomes
• Challenges for staff and students
• Engaging staff and students
• What have we learned?
• What’s next?
3
Newcastle University
• Vision
• We aim to be a world-class, research-intensive university, to deliver teaching and facilitate learning of the highest quality, and to play a leading role in the economic, social and cultural development of the North East of England.
• Structure
• 3 Faculties; 24 Schools; 11 Research Institutes
• Students
• UG - 16,000 (+1000 at overseas campuses)
• PG – 6000 (4000 Taught)
• Staff
• 5300 (+120 at overseas campuses)
Module Evaluation in Context
• Project 2012
– Institutional response to the significant changes in HE
– What do I get in return for my £9k fee?
• Drivers
– Quality assurance of provision
– Reward and recognition
– Students as partners
• Approach
– Consistency
– The use of a single system
– A common set of questions (including staff-specific)
– Migration away from paper to online
– Academic Unit ownership
4
Module evaluation - the process
• 2011/12:
– Development of core questions including staff specific
– Began to re-consider policy Purchase of Evasys
• 2012/13:
– All modules evaluated using core questions
– Piloted in eight schools in Semester 1 and 14 in Semester 2
– Introduced wider range of question sets
– Evaluation of core questions and system by staff though Evasys
• 2013/14:
– EvaSys use a requirement for all modules
– Included staff specific questions
– All campuses involved
– All evaluation online
– Additional questions sets and module specific question facility available
5
6
Fieldtrip Core
Summary
Staff specific
Module specific
3. Fieldtrip modules
Design, Studio or
Performance Core
Summary
Staff specific
Module specific
4. Design, Studio or
performance modules
Module Evaluation Question Structures
Supervision focussed
Clinical CBPS Core
Summary
Staff specific
Module specific
5. Clinical CBPS modules
Independent study Core
Summary
Module specific
2. Independent
Study modules
Supervision focussed
Staff specific
Supervision focussed
Core
Summary
Staff specific
Seminar core
Module specific
Practical sessions core
School staff specific
1. Most modules
School or Subject specific
Group/ Team work core
Compulsory set of centrally owned questions
Optional set of centrally owned questions
School can choose whether to write and use these questions
Module evaluation - the process continued
• 2014/15:
– 1068 modules evaluated in Semester 1
– 695 staff contributors
– 105,387 individual evaluations
– 52,774 responses = average response rate 50% (48% in 2013/14)
Success is dependent upon winning hearts and minds
Challenges for staff
• Change management– System– Access and data use– Standardisation
• Staff concerns about use of online rather than paper evaluations. Common worries include:– Response rates– Electronic interface encourages students
to write inappropriate comments– Which students respond
9
Engaging staff throughout the process
• Involvement of academic and administrative staff at all stages (including sceptics)
• Broad consultation• Treating concerns sensitively • Explain the rationale and
demonstrate the benefits• Work with academic units and
individuals to address issues• Provision of supporting materials
for academic and administrative staff for their use and for use with students
Module evaluation resources
11
Challenges for students
• Barriers to student understanding and engagement with evaluation process- Concerns about anonymity- Lack of knowledge about how feedback is used- ‘Survey fatigue’
• Skills gap: lack of guidance/support for students about how to give constructive feedback
12
Engaging students: central support
• Invitation emails
• ‘You Said, We Did’ webpage
• Video interviews
• Staff Student Committees
• ‘Thinking About My Evaluation’ campaign week, in collaboration with NUSU
13
Online resources
14
Thinking About My Evaluation Campaign
15
Engaging students: good practice in Schools
• Induction lectures
• Shout outs in teaching sessions
• Staff photos on VLE
• Student newsletters
• Discussion forums and text messaging
16
What have we learned?
• That investing in the system gets you only part of the way
• More about culture change than system implementation
• Getting policy right is crucial• Impossible to get core questions agreed by all• Efforts need to be sustained• Collecting staff specific feedback presents
challenges and opportunities• Students need to see their feedback is being
considered
What’s next?
• Continue to support schools with implementation and use of outputs
• Benchmarking and interpretation of scores• Continued work with EvaSys on information
flow and reporting• Increased use of data in promotions and
annual PDR• The use of Evasys for consistency in stage
evaluation • MyImpact (T)
18
THANK YOU
19