Pergamon

Srudm m Educarronal Evaluation. Vol. 23, No. 2. pp. 141-157. 1997 0 1997 Elsewer Science Ltd

Pruned in Great Britain. All rights reserved 0191-491x/97 $17.00 + 0.00

SO191-491X(97)00009-6

EVALUATING THE EFFECTIVENESS OF EDUCATIONAL INNOVATIONS: USING THE STUDY PROCESS

QUESTIONNAIRE TO SHOW THAT MEANINGFUL LEARNING OCCURS

David Kember, Margaret Charlesworth, Howard Davies, Jan McKay, and Vanessa Stott

Hong Kong Polytechnic University, Hung Horn, Kowloon, Hong Kong

Educational Innovations

This article advocates the use of repeated applications of inventories of approaches to learning as an appropriate technique for the evaluation of educational innovations. The method is particularly appropriate for innovations aiming to introduce more meaningful forms of learning as this is a dimension measured by these instruments. The article presents case studies showing how the measure can be used for innovations introduced in naturalistic settings.

There is a voluminous literature on educational innovations. Different teaching methods, models of teaching, the latest educational media, new curricula, alternative assessment methods and many other forms of innovation have been proposed as leading to better teaching and improvements in student learning outcomes. For such claims to be believed they need some form of evaluation. The traditional approach to seeking evidence is to use the science-based experiment and control method. This approach, though, has been subject to criticism and suffers from limitations where the aim is to introduce an innovation into a genuine educational setting, rather than to conduct a laboratory-type trial. The first difficulty lies in ensuring that the experiment and control are genuinely comparable. Educational media comparison studies, for example, have been criticized on this basis by a number of writers (e.g., Clark, 1983, 1985; Levie & Dickie, 1973; Schramm, 1977). Clark (1983) suggests that unintentional content differences between treatments often confound results. Compelling evidence for this assertion comes from the observation

141