www.monash.edu.au platform influences on survey delivery and response rate presenter phillip adams...
TRANSCRIPT
www.monash.edu.au
Platform Influences on Survey Delivery and Response Rate
Presenter
Phillip AdamsEvaluations Manager
Australasian Higher Education Evaluation Forum 2008Friday 2 October, 2008
www.monash.edu.au
Presentation Outline
• CHEQ (Centre for Higher Education Quality)• The Way We Were• The Response• The Outcomes
www.monash.edu.au
CHEQ
• Established in September 2000
• Lead and support quality assurance and improvement
• various areas of teaching, research, research training and support services
www.monash.edu.au
The Evolution
Two drivers for the change –
• Quality was an integral part of the higher education sector
• Need for institutional measures of academic activity
www.monash.edu.au
Unit EvaluationsThe way we were
• 1998 – 2002
• item bank
• academics create own survey
• 2002 – 2004
• faculty takes responsibility
• faculty-wide questionnaire
• conduct at least every five years
www.monash.edu.au
The issues
• 1998 – 2002
• singular, aggregated report per unit/subject
• no university benchmarking capacity
www.monash.edu.au
The issues
• 2002 – 2004
• patchy take up
• saw mix of academic & faculty reports
• still no university benchmarking capacity
• few opportunities for monitoring improvement
www.monash.edu.au
Major change in 2005
• 10 common items across university
• Up to 10 faculty items
• To be taken of each unit each year it is offered
• Results posted on a common website
• Results to be systematically considered by faculty
www.monash.edu.au
Re-engineering Technology And Processes
• both paper-based and online surveys for all units
• large volume of survey responses in short turnaround time
• system capable of storing data in any survey, online or paper, in single location
• improved access to results for staff and key stakeholders
• reduced cost of survey development and processing
• allowed automated reporting on WWW
2005, CHEQ introduced Survey Management System (SMS). Significant benefits from change in technology and processes:
www.monash.edu.au
PDF survey
• Benefits
• able to fill in without downloading
• presentation appealing
• Drawbacks
• practically clumsy
• poor accessibility for visually impaired
www.monash.edu.au
PDF survey limitations
• Three main types of student queries/complaints
• students unfamiliar with a fillable PDF
• browser version incompatibilities
• web browser setup
• Plus
• lack of accessibility for visually impaired
• disenfranchised students
www.monash.edu.au
Our response
• prepared “how-to” information sheets
• initially assisted VI over phone and with customised survey
• shifted from PDF to HTML form
www.monash.edu.au
Outcomes
• student familiarity
• accessible for visually impaired
• assisted over phone
• special form design considerations
• No browser compatibility issues
www.monash.edu.au
Outcomes
• Zero complaints about accessAccess Problems
157
109
0 00
20406080
100120140160180
2005 2006 2007 2008
Years
Nu
mb
er o
f co
mp
lain
ts
www.monash.edu.au
Outcomes
• Increase in web based response rate Annual Online Response Rate
26.330.8
33.8
42.8
0
10
20
30
40
50
2005 2006 2007 2008 (s1)
Evaluation period
Res
po
nse
rat
e (%
)
www.monash.edu.au
Conclusion
• Quality cycle relies on the effectiveness of the evaluation system
• Better meet the needs of stakeholders
• Survey more user friendly
•Positive influence on response rate