Ten recommendations for evaluating training

Evaluating Training

Ten recommendations
drawn from the psychological literature

Dr Tony GillieThe Gillie Partnership Limited

An overview of evaluating training, together with an expanded version of this presentation and a more detailed white paper reviewing the psychological literature on evaluation of training is available from

http://www.thegilliepartnership.co.uk/evaluating-training-overview.html

Why evaluate training?

Organisations evaluate training (or attempt to) for a number of reasons.

These ten recommendations are based on the assumption that
organisations send people on training courses to change their
behaviour back in the workplace.

This idea of maximising the transfer of learning from a training programme back to the workplace is central to this presentation.

Recommendation 1

Always ask delegates simple questions about the venue, the materials, and the trainer(s)

Research has shown that how people feel at the end of a training course has essentially no relationship with what they do back in the workplace.

However, since it's important that simple 'domestic' issues don't get in the way of learning, it's worth evaluating them on an
ongoing basis.

Research reported by Alliger (1997), in Personnel Psychology, 1997, pp341 - 358

Recommendation 2

Always ask delegates a simple question with a numeric response about how useful the course has been to them in their current job

Research has shown that how useful people say a training course has been is directly related to whether or not they will transfer their learning back to the workplace.

Use a simple scale of 1 to 10, asking how useful the course has been, with 1 meaning no use, and 10 meaning invaluable.

Recommendation 3

Don't waste resources on trying to measure how much people have learned

The research evidence on this is quite clear. The relationship between what someone learns on a course and whether they
apply that knowledge back at work is extremely low.It is lower for instance than the relationship between how useful someone says a course has been and whether they will apply their knowledge back at work (see recommendation 2).

Recommendation 4

Don't waste resources on trying to measure
behaviour change at the end of a course

This may seem counter-intuitive, but since the strength of the
relationship between measures of behaviour change and the transfer of that behaviour back to the workplace is the same as the strength of the relationship between how useful people say they found the course and the transfer of behaviour back to the workplace, it makes sense to measure the easier of the two: see
recommendation 2.

This is so counter-intuitive it's worth restating.

In determining whether or not someone will transfer behaviour from a trainng course back to the workplace, simply asking them how useful they found the course is as predictive as any measure of behaviour change taken on the training course itself.

Since behaviour change is notoriously difficult to measure, why bother when a much simpler measure will do the job just as well?

Recommendation 5

Build role plays of real situations back in the workplace into courses in order to build beliefs of self-efficacy

Research has shown that conservatively, there is a 28% gain in work performance that can be directly attributed to whether or not an individual believes they can do a particular task in a particular situation. This feeling of 'self-efficacy' appears to be a major predictor of whether or not people will transfer learning back to the workplace.

Research by Stajkovic & Luthans (1998) in Psychological Bulletin, volume 124, pp 240-261

Recommendations 6, 7

Try to make sure line managers work with trainees to discuss ways of using new skills

Try to make sure trainees have an opportunity to use new skills within a short time of returning to the workplace

It is important that trainees have the chance to put their new
learning into practice shortly after returning to work. In this context, that means within weeks, not months. Line managers should be encouraged to actively support their staff during this transition.

This may be linked to the notion of self-efficacy discussed in recommendation 5.

A trainee may leave a course believing that they can do something, but without the opportunity to try it back in the workplace, their feelings of self-efficacy may well decline, reducing the probabilty that they will even attempt the behaviour.

Recommendation 8

Make sure the culture of the organisation is one where there is explicit and implicit support for people applying new skills

The traditional view of training, that whether an individual changes their behaviour in the workplace as a result of training is largely dependent on the individual themselves, turns out not to be the case.Whether or not learning is successfully transferred back to the workplace is mostly determined by the organisational culture the trainee returns to.

Key features of cultures where learning is transferred successfully include:

the extent to which supervisors and co-workers encourage and set goals for trainees to use new skills and behaviours;

whether there were features of the trainee's job that served as a reminder to use newly acquired skills or knowledge, and

whether the organisation offers implicit or explicit rewards to trainees who used the skills acquired in training (e.g. more responsibility, pay rises, etc.)

Recommendations 9, 10

Consider using 360 degree feedback to assess general changes in behaviour

Consider using actual job performance as part of the evaluation process

The ultimate test of whether training an individual has been successful is whether or not they are able to demonstrate their new skills in the workplace.Asking co-workers through a 360 feedback exercise, or simply asking the trainee to perform the task (e.g. present to the Board) will allow you to use their actual performance as evaluation.

More information on evaluation of training, based on a review of the psychological literature is available on our website at

http://www.thegilliepartnership.co.uk/evaluating-training-overview.html

Main sources used in these recommendations:

Alliger, G. M., Tannenbaum, S. I., Bennett, Jr., W., Traver, H., & Shotland, A. A metaanalysis of the relations among training criteria. Personnel Psychology, 1997, 50, 341358.

Kirkpatrick, D. L. Techniques for evaluating training programmes. Journal of the AmericanSociety of Training Directors, 1959, 13, 39 and 2126; 14, 1318 and 2832.

Rouiller, J. Z., & Goldstein, I. L. The relationship between organisational transfer climate and positive transfer of training. Human Resource Development Quarterly, 1993, 4, 4, 377390.

Stajkovic, A. D., & Luthans, F. Selfefficacy and workrelated performance: a metaanalysis. Psychological Bulletin, 1998, 124, 2, 240261.

Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. Applying trained skills on the job: the importance of the work environment. Journal of Applied Psychology, 1995, 80, 2, 239252.