17 module evaluation

8/8/2019 17 Module Evaluation

1/3

A Web Based Module Evaluation System

Interim Innovation Project report to the Learning & Teaching Committee

Nadine Foster (Dept of Physiotherapy Studies), Nikki Rochford (Dept of Physiotherapy Studies), Barry

Smalley (School of Chemistry & Physics), Stephen Bostock (Staff Development & Training).

Aim of Project

To develop a web-based system to facilitate the rapid analysis of module evaluations. The module

evaluation system will be based on one currently in use by the Physics Department.

Background and Purpose

The Department of Physiotherapy Studies has demonstrated a keen determination to improve the

already high standard of quality assurance mechanisms used within the department. Whilst the

department feels its present mechanism of module evaluation is effective, its reliance on paper

questionnaires and the resultant time required in the analysis and production of useful data, renders thisprocess rather intensive in terms of staff time and of questionable cost-effectiveness. The relevance of

these issues in a quality assurance programme is recognised and the aim of this project was to improve

the efficiency and cost-effectiveness of the module evaluation process.

We further proposed to develop this into a portable system using the Physiotherapy Undergraduate

course as a pilot for a more general system. This evaluation system will also facilitate the use of

information technology by students and will respond to the needs for impartial collection of student

views without unnecessary use of staff time.

The Department of Physiotherapy Studies further recognised that the paper module evaluation forms

were not fully student-centred. It was felt that they did not consistently investigate students attitudes

towards their own responsibility for the learning process. It was therefore intended that the modular

evaluation form developed for use on the web would investigate the extent to which the student

actively engages in the learning process.

We also recognised the need to be able to compare the students learning experiences across modules.

It was expected, therefore, that the web-based module evaluation system would facilitate comparison in

the student experience across modules. The development of this web-based module evaluation system,

within the Physiotherapy Department would act as a pilot to evaluate the transferability of the system

to other departments within the University. This will fit well with the aims of the Quality Assurance

initiatives to enable comparison across departments and Faculties.

It was further expected that this project would contribute to the embedding of IT within the BSc (Hons)

Physiotherapy programme, as well as the promotion of the transferable skills of life-long and active

learning, via student reflection.

Objectives

o To develop a standard module evaluation form for the BSc Physiotherapy Studies (Hons)

programme, which would facilitate comparability across modules

o To further develop & enhance the supporting software to analyse and present the evaluation

data producing meaningful web-based summary data.

o To develop a user-friendly tool which does not require a high degree of computer literacy to

use

o To pilot the developed evaluation questionnaire forms within the Physiotherapy programme

o To investigate how feasible it might be to use this web-based system of module evaluation

across the wider University.

08/09/10


2/3

Method

In order to meet the above objectives, the following procedure was undertaken:

1. Current module evaluation tools used within academic environments were reviewed for their

appropriateness and pertinence, additionally focus groups and staff discussion were utilised to

identify the main areas considered to be important in a module evaluation.2. From these a draft questionnaire was then developed.

3. The draft questionnaire was which was reviewed by all staff and representatives of the student

body for relevance and appropriateness and further developed on the feedback.

4. The developed module evaluation form was then adapted to a web based system, which was

then piloted within all first semester physiotherapy modules.

Attached to this report are examples of the form and the outputs.

Results of Pilot

1. Comparability between web based and paper evaluations.

To ascertain if the computer results were comparable to the results gained from a paper evaluation thestudent cohorts were randomly assigned to two groups, the first undertook the computer evaluation and

the second completed the same evaluation form but on paper.

The following table gives a comparison of the results obtained from these two sources.

Modal score Modal score

for web version for paper version

Module 1 4 4

Module 2 4 4

Module 3 4 4

Module 4 4 4

Module 5 4 3

Average number of students giving a low score (1,2) for a question,

out of about 38 students (and range of values for any one question)

for web version for paper version

Module 1 2.7 (range 0-22) 6.7 (range 0-31)





It is clear that the method of delivery did not systematically affect the results obtained from the module

evaluation.

2. Staff Evaluation

A questionnaire was completed by all module leaders to determine their views of the new system:

Staff preference

All 5 staff preferred the new evaluation form

2 staff preferred computer system to the paper based one

3 staff stated no preference between the two systems.

Benefits

All staff stated that the main benefit of the new system was the amount of time saved.

The ease of access to results was also considered to be a benefit.

08/09/10


3/3

3. Student Evaluation

Students also were asked to complete a questionnaire concerning their attitudes towards the new

evaluation system. The following graphs illustrate the results of this audit.

Overall, students were happy with the new system. A larger number of students who had completed

paper evaluations stated a preference for paper, but they had had no experience of the web basedsystem. Those who used the web form, on balance, preferred it. They were positive about the design of

the form. Various improvements were suggested, many of which had already been considered. A large

majority of students felt the medium made no difference to their answers, and those that thought it did

were not consistent in the effect they thought it would have. A majority thought it would be necessary

to book a PC lab to complete the web forms were the computer system adopted.

Further developments

o The standard web form is now being used for all physiotherapy modules, Years 1, 2 and 3.

o Aprogramme evaluation questionnaire had additionally been developed and was piloted at the

end of May 2002; the results are pending.

o The web report now provides summary data for inclusion in the course report. The usefulness

of this will be determined after summer.

o The transferability of the system to other departments is now to be determined.

Conclusion

It is expected that, with minor amendments, this method of module evaluation will be embedded within

the plans for the new BSc (Hons) Physiotherapy course due to commence October 2002. This is

expected to result in a more cost-effective and efficient method of evaluation. Added benefits for

Physiotherapy Studies include the increasing use of IT by students and the improvement of information

quality gathered as part of module evaluation. The system design will explore long-term issues such as

ensuring portability. It is intended that the transferability of this tool to other departments will have

been explored and recommendations made for campus-wide availability.

08/09/10

Q 1 P r e f e r p a p e r o r

0%

10 %

20 %

30 %

40 %

50 %

60 %

70 %

p ape r e it he r w eb

web paper Q2.Was form design helpful?

0%

20%

40%

60%

yes maybe no

web paper

Did the medium make a differen

0%

20%

40%

60%

80%

100%

yes maybe no

web paper Q2.3 Need a PC lab session? (web

0%

10%

20%

30%

40%

50%

60%

70%

yes m aybe no
http://simeon/Exit%20Questionnaire%20Physiotherapy%20Studies%202001-2002.htmhttp://simeon/Exit%20Questionnaire%20Physiotherapy%20Studies%202001-2002.htm

17 module evaluation

Documents