An Examination of Learning Processes During Critical

Incident Training: Implications for the

Development of Adaptable Trainees

Andrew Neal, Stuart T. Godley, Terry Kirkpatrick and Graham Dewsnap, Wendy Joung, and Beryl Hesketh

2006

Overview • Purpose: Examine the process that trainees learn

decision making skills during a critical incident training program.

• Discuss that rule and exemplar learning may be responsible for the acquisition of decision making skills.

• Results suggest trainees use both types of learning, and they can also be affected by different aspects of training structure and design.

Introduction • Based off previous work by Ford

and Kraiger (1995)

• Aim: Address gaps in research at a theoretical, methodological, and practical level.

Category Learning• Historical viewpoint vs. Dual Process Model

• Issue: Majority of research supporting dual process models were conducted in university lab settings.

o First goal – determine whether trainees use the same learning strategies as participants in a controlled lab setting.

Training Design and Structure

• If trainees do use both rules and examples, its important for trainers to understand how the design and structure of a training influences the use of these strategies.

o Context and Practice

Practice • Traditional research suggests:

o Practice makes perfect!

• Dual Process Model o Practice allows fine tuning of decision making skills. o Common issue: Individuals will continue to use examples even if they

have access to a well learned rule.

• Suggests context may be more effective.

Training for Adaptability

• Need for new techniques addressed by Feltovich et al. (1997).

• Exceptions are more likely to be recognized if individuals adopt a belief structure that emphasizes the complexity of task/domain.

o Can be achieved through explicit instruction and learning from examples.

Overview of Experiments Experiment

1Experiment

2

• Assess trainees reliance on rule and exemplar learning.

• Examine how aspects of training design and structure affect the use of rule and exemplar learning.

Experiment

3Determine whether explicit instruction and task practice can reduce the incidence of inappropriate generalization.

Experiment 1 • Goal – Determine the extent to which trainees use

rules and examples when learning to make tactical decisions.

• Lecture, training examples (feedback), then followed by test examples. o Used inclusion and exclusion items

Hypotheses • Hypothesis 1: Performance on novel items will be

significantly better than chance.

• Hypothesis 2: Performance on the inclusion items will be better than performance on novel items.

• Hypothesis 3: Performance on exclusion items will be worse than performance on novel items.

Method • Setting – introductory training program

• Participants were 35 new recruits to New South Wales Fire Brigades. o 33- Male, 2 – Femaleo Average age of 30.9 o Average experience of 3.1 years

• Materials included a lecture (15 minutes), eight training examples, and 12 test examples which used a picture of a fire and along with an auditory script.

Analysis and Results• Repeated measures ANOVA was completed along with

Planned contrasts on confidence, accuracy, and reaction time.

• All hypotheses were supported.

• Results are consistent with the prediction that trainees would use both when learning strategies to make tactical decisions.

Experiment 2 • Aim: Examine the effects of practice and context on the use

of rules and examples.

• Hypothesis 1: Performance on novel items will improve from phase 1 to phase 2.

• Hypothesis 2: The differences in performance on the inclusion items and exclusion items will remain stable over practice.

• Hypothesis 3: The differences on the inclusion and exclusion items will be larger when the contextual features of the training and test items are matched.

Method • Design – split item pool in half and counterbalance the

order of the presentation of the two pools.

• Participants were 36 new recruits to the NSW Fire Brigades

• Materials used were the same as Experiment 1.

• Procedure – Lecture, then phase 1 was given test items, then phase 2 was given the test items.

Analysis• Repeated measures ANOVA

• Planned contrasts were run, which assessed the effects of practice on the novel items and the effects of practice and context on the inclusion and exclusion items.

• Classification accuracy, reaction time, and confidence were the dependent variables.

Results • Results supported all 3 hypotheses.

• Practice by itself doesn’t appear to be enough to prevent trainees from generalizing inappropriately from prior examples.

• Retrieval and use of prior examples were dependent on contextual overlap in training the test items.

What to take away… • Trainers can enhance trainees ability to use

general rules by increasing the number of practice examples.

• Trainers can enhance trainees ability to use examples by ensuring context from training matched transfer environment.

Experiment 3 • Aim: Examine the effectiveness of training

strategies designed to prevent trainees from generalizing a simple rule to cases in which it doesn’t apply.

• Used explicit instruction regarding exceptions to the rule. o Near and Far Exceptions

Hypotheses • Hypothesis 1: Providing trainees with explicit

instruction regarding exceptions to a simple rule will enhance performance on near and far exceptions.

• Hypothesis 2: Providing trainees with examples that illustrate exceptions to a simple rule will enhance their performance on near and far exceptions.

Methods • 2x2x(3) • Between group factors:

o Instruction – Routine and exceptions instruction o Practice – Routine and exceptions practice

• Within group factor o Item type

• 16 items (8 – routine, 4-near exceptions, and 4 – far exceptions)

• Half of the participants (exceptions instruction group) received additional information regarding exceptions in the lecture.

Methods• Participants included 91 recruits to the NSW Fire

Brigades. o Had 16 weeks of training o All Male

• Procedure was the same as Experiment 2, except there was only one training and test phase.

Analyses• Series of repeated measures ANOVA.

• Planned contrasts were run, assessing the effects of instruction and practice on the routine items, near and far exceptions.

o Classification accuracy, reaction time, and confidence were dependent variables.

Results • Providing general instruction regarding

exceptions to the rule was more harmful than helpful. Trainees became fixated on exceptions.

• Possible Solution: Presenting exceptions in the set of practice examples might be more effective than explicitly instructing trainees about these exceptions.

General Discussion • Trainees use both rules and examples.

• Results from lab setting do generalize to training contexts.

• Results supported that the learning processes would respond differentially to design variables, such as practice and context. o Can help trainers make design choices accordingly

• Such learning processes could be used in training needs analysis, and training design and evaluation.

Strengths Limitations• Unsure of lasting

effects • Limited information in

lecture, practice, and time to study material.

• Tasks used • Simulated scenarios • Effects could reflect

how lectures and examples were constructed.

• Study Design which allowed us to be able to draw inferences about the results.

Discussion Questions• Do you think such findings can be generalized to

other areas in which critical incident training is used?

• Do you agree that we should integrate more learning strategies such as the dual process model into the training research? Would it increase trainee effectiveness?

• Study suggests results from lab setting can be generalized to a training context, do you feel this is just reflective of this study?