Programme Wide Evaluation – Biological Sciences

Karen Howie

E-Learning Project Officer

• Project Summary– Running since Summer 2004– Aim: to roll out WebCT to all of the School’s

1st-3rd year courses– Staff

• 1 full-time since Oct 2004• Project Manager (Dr Paul McLaughlin)• 1 part-time paid by BTO (Dr David Barrass)

• What are we evaluating?– Staff and student opinion :

• Template development• Improvement in e-learning tools and usage of the tools• The service we provide

– Project progression

– Extra: general background information about students– Would like to measure:

• Does E-Learning provision improve student marks?

• How we gauge student opinion– Course questionnaires– E-Learning questionnaire to all years 1-3

students – Analysing course site usage– Anecdotal evidence – staff/student meetings,

talking to students, helping students– Focus groups for specific items of interest –

ie. Clickers project

• Course Questionnaires– Annual– Standard questions– Large number of questionnaires now moved

onto WebCT– Change in average response rate

• Paper 04/05: 56%• Online 05/06: 51%• Saves staff time & paper!

Comparison between Semester 1 courses in 2004/2005 and 2005/2006 - % of students which felt the course area was a

valuable resource

0

10

20

30

40

50

60

70

80

90

100

Course

Per

cen

tag

e

04-05

05-06

Is the WebCT course area a valuable resource?

0

20

40

60

80

100

120

U01224 BI0001 U00579 U01212 U00761 IC0002 BI0022 BI0012 PY0001 U00717

Course

Per

cen

tag

e

04-05

05-06

Comparison between Semester 1 courses in 2004/2005 and 2005/2006 - % of students which felt the course area

was easy to use

How easy is the WebCT course area to use?

• E-Learning Questionnaire for Students– Ask a wider variety of questions– Response rate: 16% (of ~ 1700 students)– Provided a ‘response’ document

• Summary of statistics• Answered repeated queries

– Ran last year (04-05) and in progress this year

– Only online last year. Paper and online this year

04/05 E-Learning Student Questionnaire

Where do you access the course web-sites from?

65 70 75 80 85

Computer labs in university

Home

Halls

Lo

cati

on

Number of students

• Analysing Course Site Usage– Uptake by staff has been better this year– Uptake by students generally better also

• Measured using crude approximation of tools available for logging available in WebCT

– Change in course staff can make noticeable difference

• Anecdotal Evidence– Talking to students during tutorials/sessions– Staff-student meetings– Discussion postings & emails

• Focus Groups– Have run 5 in Biological Sciences for the ‘Clicker’

project (ref Simon’s presentation)– Tend to get lots of interesting things discussed – not

just clickers, room sizes, furniture layout, use of WebCT, etc

• How we gauge staff opinion– They tell us!– E-Learning Questionnaire for Staff

• Response rate of 31% (all known staff involved with course teaching)

• Mostly positive and some ambiguities spotted

– Improvement in up-take, particularly in Honours courses

• Evaluating Project Progression– Steering group with external consultant– External consultant – Stephen Clarke, Head

of E-Learning, University of Birmingham– Valuable to have!

• Future Plans– Run optional student training and use it to get

some feedback from students– Keep running other forms of evaluation– Honours evaluation and standardising

courses over Honours