giving all students a voice – introducing online evaluation of teaching and learning within qut
DESCRIPTION
Giving All Students a Voice – Introducing Online Evaluation of Teaching and Learning within QUT. Presented at Association for Tertiary Education Management Queensland Branch Conference “Taking it to the Streets” By Margot Duncan Jude Smith Sam Nielsen 1 June 2007. Aims of Presentation. - PowerPoint PPT PresentationTRANSCRIPT
Queensland University of Technology
CRICOS No. 000213J
Giving All Students a Voice – Introducing Online Evaluation of
Teaching and Learning within QUT
Presented at
Association for Tertiary Education Management Queensland Branch Conference
“Taking it to the Streets”
By
Margot DuncanJude Smith
Sam Nielsen
1 June 2007
CRICOS No. 000213Ja university for the worldrealR
Aims of Presentation
• To overview QUT’s new approach to student evaluation
• To highlight challenges related to implementing the new approach to surveying
• To discuss lessons about implementation of a major reform learned from the process
CRICOS No. 000213Ja university for the worldrealR
The Need for ChangeQUT Senior Executive responded to recommendations from a review of Quality Teaching that:
• Teaching evaluation should be universal for all teachers every semester in every unit;
• Teaching and unit data should be provided routinely and form part of corporate reporting internally;
• Individual teaching results should be reported (minimally) to the teacher, supervisor and relevant managers;
• Data should form part of performance management and development discussions; and
• Advice and support should be provided to enable teaching data to be used more consistently, systematically and appropriately in both performance management and in responding to students.
CRICOS No. 000213Ja university for the worldrealR
The New Evaluation Scheme at QUT
QUT has been improving student evaluations processes and policies during 2007:
• a new suite of internal surveys encompassing teaching, units, courses and experience has been introduced:– Learning Experience Survey– First Year Experience Survey– Exit Year Experience Survey– Mid Year Experience Survey
• The surveys are fully administered and reported online• The surveys allow QUT students unprecedented opportunity to
provide feedback about individual teachers and units
CRICOS No. 000213Ja university for the worldrealR
Taking it to the Streets…
• The new systematic, online approach to student evaluation has raised concerns and involved challenges for students, professional staff, academics, and QUT’s senior executive.
To meet these challenges the project team had to “Take it to the streets”
CRICOS No. 000213Ja university for the worldrealR
Perspective
This presentation focuses upon the implementation from the
perspective that the outcomes are not yet fully known.
CRICOS No. 000213Ja university for the worldrealR
Challenges and Issues
STOP
CRICOS No. 000213Ja university for the worldrealR
40,000Students
In the streets:
• students don’t think surveys are ever used to improve teaching
• students feel they are being over-surveyed
• students don’t have time to complete lengthy surveys
• students doubt the confidentiality of their survey comments
Challenges: Students
QUT has almost 40,000 students.
CRICOS No. 000213Ja university for the worldrealR
In the streets:
• staff doubt the validity and accuracy of survey data
• staff are concerned about promotional implications of survey data
• staff have limited support in innovating their teaching practice based on survey data
• staff have issues with previous online surveying efforts
3,000Academic Staff
QUT has over 3,000 Academic Staff.
Challenges: Academic Staff
CRICOS No. 000213Ja university for the worldrealR
Professional staff: 2 divisions 3 campuses9 faculties
In the streets:• professional staff have been frustrated with previous complex systems & the demands of managing mandatory evaluations
Professional Staff from 9 faculties and 2 Divisions are involved.
• each faculty and division is structured differently & has different evaluation requirements
• evaluation communication channels between faculties and divisions have not been established
Challenges: Professional Staff
CRICOS No. 000213Ja university for the worldrealR
In the streets:
• Senior Executive feels pressure from AUQA and LTPF.
• Policy changes have union implications.
Challenges : QUT Senior Executive
CRICOS No. 000213Ja university for the worldrealR
3TechnicalSystems
In the streets:
• key data systems were missing from university practice
• existing technical systems were disconnected
Technical staff working with 3 university-wide systems are involved.
Challenges : Technical Staff
CRICOS No. 000213Ja university for the worldrealR
Project Challenges
In the streets:
• Demanding deadlines
• Limited funding
• Politics
• Uncertainty about roles
• Aware of challenges for others and need for solution satisfying all
Challenges: Project Management
Many limitations and conflicting issues
CRICOS No. 000213Ja university for the worldrealR
Moving ahead…Approach to and lessons learned
from implementing change
CRICOS No. 000213Ja university for the worldrealR
…to get all stakeholders working together effectively to ensure continuous improvement of teaching through listening to the student voice.
1. Keep the project goal in mind.
CRICOS No. 000213Ja university for the worldrealR
2. Consider All Perspectives
Knowledge from other implementations Use of available researchConstructive feedback
CRICOS No. 000213Ja university for the worldrealR
… do not seek to achieve more than is possible… do not over-promise… be willing to enhance the product in the future… effectively use project committees and project sponsorship
3. Manage Project Scope
CRICOS No. 000213Ja university for the worldrealR
4. Bring stakeholders on the journey
CRICOS No. 000213Ja university for the worldrealR
“Always be prepared to inform and be informed by stakeholders”
Human Resources Staff
IT Help desk staff
Undergraduate Students
Corporate Systems Managers
Learning & Teaching Consultants
T&L Support Staff
International Students
Senior StaffAcademic Staff
Faculty Admin Staff
Sessional Staff
Corporate Performance Staff
Post Graduate Students
Course Coordinators
Off-shore Coordinators
The Project Team
5. Enable Communication
CRICOS No. 000213Ja university for the worldrealR
6. Take every opportunity and use every method to communicate
OnlineResources
Forums
ReferenceGroups
Emails
Focus Groups
CRICOS No. 000213Ja university for the worldrealR
We used:• Steering groups• Reference groups• Working Parties• Committees
OnlineResources
Forums
ReferenceGroups
Emails
Focus Groups
We used:• Student Focus groups for survey questions & Interface design• Staff Focus groups for survey questions• Staff & students in survey trials
We used:• DVC & Senior Staff email communications to Academic staff• Staff information Packs• Staff Information Web Site• Staff & Student Information Web Site• Staff Teaching Resource Web site
We used:• HR workshops for Academic PPR• Teaching Portfolio Workshops• Faculty-based Workshops• Campus based Staff Forums• IT Helpdesk training• T&L Support Staff training
We used:• Student advertising campaign through postcards, posters, monitor displays, unit website messages.
CRICOS No. 000213Ja university for the worldrealR
7. Confront stakeholders’ concerns.
CRICOS No. 000213Ja university for the worldrealR
Student Confidentiality
Response Rates
PPR Processes
Seamless experience for students & staff
Closing the Loop Sustainability
of systems & processes
T&L Support
CRICOS No. 000213Ja university for the worldrealR
8. Find the shortcuts
… Cross organisational boundaries … Include and utilise expertise
CRICOS No. 000213Ja university for the worldrealR
9. Trust your Intuition.
•Many choices did not have ‘the’ obvious answer •Many discussions had an unspoken political aspect •Intuition as an indicator that there is a need for further ‘unpacking’ of a problem
CRICOS No. 000213Ja university for the worldrealR
10. Think outside the box.
CRICOS No. 000213Ja university for the worldrealR
11. Keep going.