the investigation of human error in atm simulation … investigation of human error in atm...

EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL

EUROPEAN AIR TRAFFIC MANAGEMENT PROGRAMME

This document is issued as EATMP Method and Tool. The contents are notmandatory. They provide information and explanation or may indicate best practice.

The Investigation of Human Errorin ATM Simulation

The Toolkit

Edition Number : 1.0Edition Date : 02.07.2002Status : Released IssueIntended for : EATMP Stakeholders

Page ii Released Issue Edition Number: 1.0

DOCUMENT CHARACTERISTICS

TITLE

The Investigation of Human Error in ATM SimulationThe Toolkit

EATMP Infocentre Reference: 020514-02

Document Identifier Edition Number: 1.0

HRS/HSP-002-REP-06 Edition Date: 02.07.2002

Abstract

This document represents a technical addition to the first report in Phase 2 of the ‘HumanError in ATM’ Project (HERA 2). It concerns the further development and investigation ofhuman error, its prediction, detection and management within the Air Traffic Management(ATM) system. The purpose of the work is to assess how human errors are generatedwithin a simulated ATM environment and, more importantly, how they are managed. Thisreport describes in detail the tools and methods developed to record the error, and theirrecovery, made in the ATM environment.

KeywordsHuman Error Human Performance Error Management Risk Assessment Simulation

Contact Persons Tel UnitAnne Isaac +32 2 729 3957 EUROCONTROL Headquarters, DIS/HUM

Michiel Woldring +32 2 729 3566 EUROCONTROL Headquarters, DIS/HUMAuthors

A. Isaac, C. Duchenne, L. Moulin and G. Hindson

STATUS, AUDIENCE AND ACCESSIBILITYStatus Intended for Accessible via

Working Draft � General Public � Intranet �

Draft � EATMP Stakeholders � Extranet �

Proposed Issue � Restricted Audience � Internet (www.eurocontrol.int) �

Released Issue � Printed & electronic copies of the document can be obtained fromthe EATMP Infocentre (see page iii)

ELECTRONIC SOURCEPath: H:\Carine's files\Deliverables\Former 'Own_use' subdirectory\RELEASED\HFs\HERA

Host System Software Size

Windows_NT Microsoft Word 8.0b 213 Kb

The Investigation of Human Error in ATM Simulation – The Toolkit

Edition Number: 1.0 Released Issue Page iii

EATMP InfocentreEUROCONTROL Headquarters96 Rue de la FuséeB-1130 BRUSSELS

Tel: +32 (0)2 729 51 51Fax: +32 (0)2 729 99 84E-mail: [email protected]

Open on 08:00 - 15:00 UTC from Monday to Thursday, incl.

DOCUMENT APPROVAL

The following table identifies all management authorities who have successively approvedthe present issue of this document.

AUTHORITY NAME AND SIGNATURE DATE

Please make sure that the EATMP Infocentre Reference is present on page ii.

HERA Project Leader

A. ISAAC

26.06.2002

ChairmanHRT Human Factors Sub-Group

(HFSG)M. WOLDRING

26.06.2002

ManagerEATMP Human Resources Programme

(HRS-PM)M. BARBARINO

26.06.2002

ChairmanEATMP Human Resources Team

(HRT)A. SKONIEZKI

27.06.2002

Senior DirectorPrincipal EATMP Directorate

(SDE)W. PHILIPP

01.07.2002


Page iv Released Issue Edition Number: 1.0

DOCUMENT CHANGE RECORD

The following table records the complete history of the successive editions of the presentdocument.

EDITIONNUMBER EDITION DATE INFOCENTRE

REFERENCEREASON FOR

CHANGE PAGES AFFECTED

0.1 01.08.2001 Working Draft All

0.2 18.02.2002 Proposed Issue All(Document configuration)

1.0 02.07.2002 020514-02 Released IssueAll

(Last document configurationadjustments)


Edition Number: 1.0 Released Issue Page v

CONTENTS

DOCUMENT CHARACTERISTICS............................................................................ ii

DOCUMENT APPROVAL ......................................................................................... iii

DOCUMENT CHANGE RECORD ............................................................................. iv

EXECUTIVE SUMMARY ............................................................................................ 1

1. INTRODUCTION.............................................................................................. 3

2. GATHERING HUMAN ERROR DATA IN SIMULATION................................. 5

2.1 Introduction .............................................................................................................. 5

2.2 Phase 1: The Observation of the Controllers ......................................................... 6

Preparation of the Observations .................................................................................... 6

2.2.1 Presentation of the goals of the observations to the ATCOs ........................................... 62.2.2 Preliminary list of observable operational ATC events..................................................... 62.2.3 Checklist for the materials and equipment for the observations....................................... 8

Direct Observations ........................................................................................................ 9

2.2.4 Instructions for the observers ........................................................................................... 92.2.5 Observation notebook..................................................................................................... 102.2.6 Radar screen recording form.......................................................................................... 12

2.3 Phase 2: The Debriefing between the Observers................................................. 14

2.3.1 Pre-interview notebook ................................................................................................... 142.3.2 Principles and questions for the interview preparation................................................... 16

2.4 Phase 3: The Interview with the Observed Controllers ....................................... 26

2.4.1 Instructions for the interview ........................................................................................... 262.4.2 Interview notebook.......................................................................................................... 262.4.3 Situation control and risk assessment forms.................................................................. 28

REFERENCES ......................................................................................................... 31

ABBREVIATIONS AND ACRONYMS...................................................................... 31

CONTRIBUTORS..................................................................................................... 33


Page vi Released Issue Edition Number: 1.0

Page intentionally left blank


Edition Number: 1.0 Released Issue Page 1

EXECUTIVE SUMMARY

This document represents a technical addition to the report ‘The Investigation of HumanError in ATM Simulation’ (EATMP, 2002a), produced under Work Package 1 of Phase 2 ofthe ‘Human Error in ATM’ Project (HERA 2), and dealing with human error observation in areal-time simulated environment and analyses with the HERA Technique (see EATMP,2002b).

It describes the methodology for capturing data in an ATC simulation situation and thesubsequent error recording and analyses.

There are three main phases to this observation methodology: firstly, the observation itself,secondly, the debriefing of the expert observers and, finally, the interview with the observedATC personnel.

Typically, for each of the phases there are several sets of materials which need toaccompany the activities to allow the best possible data capture. These include a descriptionof the documents and information regarding the equipment used, as well as the tools andmethods to help support the data gathering itself.


Page 2 Released Issue Edition Number: 1.0




1. INTRODUCTION

The general objective of the ‘Human Error in Air Traffic Management (HERA)’Project is to investigate several specific areas associated with the prediction,detection and management of human error in ATM, and to develop methodsfor the implementation of the results of these concepts at various levels of theair traffic safety management within Europe.

The objective of Phase 2 of the HERA Project (HERA 2) is to explore moreintensively the potential operational applications of this error analysistechnique, in relation to four specific safety-related areas. These arerepresented in four Work Package (WP) deliverables as follows:

• the development of an approach using the HERA Technique to investigatehow human error can be detected and managed within a real-timesimulated ATC environment;

• the investigation of the potential of the HERA classification as aprospective tool within ATM (error prediction);

• the development of an approach using the HERA classification techniquefor safety management within ATM;

• the development of a training course on the HERA analysis technique forincident investigators and safety managers within several ECAC States.

This document represents a technical addition to the WP 1 deliverable(EATMP, 2002a) dealing with human error observation in a real-timesimulated environment and analysis with the HERA Technique (see EATMP,2002b). It describes the methodology for capturing data in a simulation ATCenvironment and the subsequent error recording and analysis.



2. GATHERING HUMAN ERROR DATA IN SIMULATION

2.1 Introduction

There are three main phases of data gathering within the methodologyexplored in the HERA Project in a simulation environment. These are:

• Phase 1: The Observation of the Controllers,

• Phase 2: The Debriefing between the Observers,

• Phase 3: The Interview with the Observed Controllers.

Typically, for each of the phases there are several sets of materials whichneed to accompany the activities to allow the best possible data capture.These include preparation documents and information regarding equipment tobe used.

The observation materials developed are listed hereafter and correspond tothe three phases of the observation method:

• Phase 1: The observation of the Controllers

Preparation of the Observations

� Presentation of the goals of the observations to the ATCOs� Preliminary list of observable operational ATC events� Checklist for the materials and equipment for the observations

Direct Observations

� Instructions for the observers� Observation notebook� Radar screen recording form

• Phase 2: The Debriefing between the Observers� Pre-interview notebook� Principles and questions for the interview preparation

• Phase 3: The Interview with the Observed Controllers� Instructions for the interview� Interview notebook� Situation control and risk assessment forms



2.2 Phase 1: The Observation of the Controllers

Preparation of the Observations

2.2.1 Presentation of the goals of the observations to the ATCOs

It is important in any simulation environment that those involved all understandthe purpose of the simulation in progress. Normally, a simulation is used tofind out how controllers use new or adapted equipment and procedures in asafe and efficient way. However, when human performance parameters arebeing observed, it is essential that the controllers understand why and what isbeing observed and noted. It is obvious that the controllers in the simulationwill be concerned with regard to what is being recorded, and perhaps moreimportantly, what the data will be used for. In this regard, it is suggested thatall simulations which include the recording and data gathering of error anderroneous behaviours should be preceded with a short presentation explainingexactly what will happen. All controllers should be given the opportunity towithdraw from any part of the data gathering and any recorded data (bothvideo and tape) should either be offered to the controller concerned ordestroyed.

2.2.2 Preliminary list of observable operational ATC events

Prior to any observation a list of observable operational ATC events can begenerated to help the observers with their task. The list is not error-centred buttakes into account the performance of the observed controller. Eachsimulation will be different but the following list includes the generic tasksundertaken by the Radar Controller (RC). Adaptations of this list could bemade for various other functional positions.

Radar Controller’s position

Flight plan data� No reaction to unusual data on the strips (flight level, aircraft type)� Inadequate or incorrect annotations on the strips, taking the flight plan

peculiarities into consideration� No reaction to an abnormal planner annotation on the strip� Incoherent strip position on the boardAircraft under control� No immediate response to a call from a pilot� Communication to/from the pilot:

- no reaction to a pilot error (call and/or acknowledgement)- error in the information delivered

� No annotation on the strip� Incorrect annotation on the strip



Other controller attitudes / activities / behaviours� On the pilot’s initiative (particular request):

- no immediate response- inadequate or irrelevant controller response

� On the controller’s initiative:- HMI: incorrect / no annotation- pilot-controller communication:

- no reaction to a pilot error (call and/or acknowledgement)- incorrect information delivered

- clearances:- inadequate instruction regarding the surrounding parameters

(traffic, airspace organisation, ops room organisation, militaryactivity)

- inappropriate reaction (observer’s point of view)- checking / scanning: no reaction to a pilot not following delivered

instructions- Radar-Planner coordination: irrelevant reaction (misunderstanding)

Handling conflicting traffic� No annotation on the strips� Provisional annotation (Radar or Planner)� Radar-Planner negative reaction� Short-term Conflict Alert (STCA)

Transfer to next sector� Pilot-controller communication:

- no reaction to a pilot error (call and/or acknowledgement)- incorrect information delivered

� Strip incorrect position� Inaccurate/Incorrect exit configuration (heading, flight level)� R/T coordination too early / too late (very subjective observable attitude)� Adjacent sector refusal� No coordination with the adjacent sectorStrategy and overall management� Incorrect measurements on the HMI� Obvious changes in strategy� Forgotten instructions (resume navigation after spacing, cleared to

requested flight level)� No / late / unnecessary sector splitting (observer’s point of view)



Observable behavioural attitudes� Uncertainty expressed verbally or by gesture (i.e. face expression)� Radar screen or strip panel pointing� Other behavioural attitudes: interjections, irritations, etc.

2.2.3 Checklist for the materials and equipment for the observations

It is recommended that, for each controller position, two observers should bepresent. Ideally, there should be one with ATC knowledge (ATCO) and onewith human performance/human factors knowledge.

The quantities listed below are for a three-day observation campaign on twoparallel positions.

Before leaving, material/equipment to take:

� 2 video cameras + tripods

� Videotapes (for 2 cameras: 6 tapes of 2 hours each, appropriate formats)� Audiotapes (for interviews: 6 tapes of 90 minutes each)� 2 audiotape recorders� 2 microphones (audio-recordings of PC-RC communications + interviews)� 2 sound-mixing tables (to mix sound signals: RC-pilot radio

communications + PC-RC off-line communications on the camera audioinput)

� Batteries� Extension cables and adaptors (electrical)

Documents to prepare (Photocopy):

� Overhead transparencies or PowerPoint presentation of the goals of theobservations for the ATCOs

� Observation instructions (2 for the ATCOs of simultaneous sessions)� Observation notebooks (12- one each for the ATCO and psychologist / HF

personnel for each session)� Pre-interview notebooks (6 for the psychologist / HF personnel for each

session)� Interview notebooks (6 for the psychologist / HF personnel for each

session)� Interview instructions (2 for the simultaneous interviews)� Interview questioning aids (2 for the simultaneous interviews)� Risk assessment forms (18 for the ATCO observers + 6 for the observed

ATCOs)



On the premises of the simulation sessions:

� 2 television sets� 2 debriefing rooms� 1 overhead projector / computer + projector (to present the approach to

the ATCOs)� Wiring cables + locked room to leave equipment at night

Direct Observations

2.2.4 Instructions for the observers

The following information should be recorded for each controller on eachposition.

For each observation, it is important to take note of:

• the observation time: time code indicated on the position;

• the aircraft concerned;

• the approximate area where the A/C is positioned on the radarscreen:

• whether the deviation was detected/recovered, who detected/recovered it and how (verbal remark, action, etc.);

• contextual elements.

Note every action, event or state of the airspace that seems notable(outstanding):

• because it represents a deviation regarding what could be expectedfrom the situation;

• because it is clearly an error or a violation;

• because it seems strange to you;

• because you don’t understand;

• because you wouldn’t have done it this way;

• because it is particularly well done;

• etc.

A B

C D



Note every subjective observation on the tactics/strategy implemented bythe ATCO, the way he/she behaves, with regards to safety.

At the end of the observation session, return to each observation and takenote of the contextual elements of the situation.

2.2.5 Observation notebook

The following is the template to be used during observation sessions.

OBSERVATION NOTEBOOK

DATE:

OBSERVER:

OBSERVATION:

POSITION OBSERVED:

SECTOR AND SIMULATION INFORMATION:



INFORMATIONTime Code: A/C:

Display area:OUTSTANDING EVENT

DETECTION / RECOVERYWho: How:

CONTEXT



2.2.6 Radar screen recording form

The following can be used to record the radar information. All strips which areof interest should also be collected for future reference.

RADAR SCREEN CONTENT

DATE:

OBSERVERS:

OBSERVATION:

POSITION OBSERVED:



Time:



2.3 Phase 2: The Debriefing between the Observers

2.3.1 Pre-interview notebook

Following the radar recording session and with the notes which have beentaken, the two observers (ATCO and psychologist / HF personnel) need todebrief each other to establish exactly what has happened and what questionsthey should ask the observed controller. Obviously the two experts will have adifferent interpretation of the events and it is essential that they both agree onwhat has happened in the observation session. The pre-interview notebookcan be used for this preparation.

PRE-INTERVIEW NOTEBOOK

DATE:

OBSERVERS:

OBSERVATION:

POSITION OBSERVED:



No Time Disp.area

A/C Outstanding Event Obs. Post-coding Question(s)



2.3.2 Principles and questions for the interview preparation

Reminder of the interview principles

Remain at the level of the actual facts and of the description concerningthe outstanding events.

• Avoid generalities, re-focus on the event/episode being discussed(specific occurrence of the task).

• Specify the context of the commented event/episode.

• Question the progress of action across time, the anticipations and thehistory:

Ex: When did you begin to think about this?When did you expect results?When did you realise it was good/bad?

• Use a descriptive questioning technique: “What?”, “When?” and above all“How?”:

Ex: How did you do this?What did you have in mind?What made you see that …?

• Avoid “Why?” which implies judgement and evaluation of the activity.

Support and monitor the interview

• Be attentive to the rhythm of speech: a slow rhythm is generallyassociated with introspection (also indicated by the eyes suddenly lookingupwards).

• Do not hesitate to re-phrase ideas which were not clearly expressed.Nevertheless, be careful to avoid suggesting an answer.

• Re-question the denials and go beyond the “I don’t know”, “I can’tremember”, “yes/no” and “hums …”.

Ex: I don’t know. - And if you don’t know, what do you know?ORHow do you know that you don’t know?I don’t remember. - What do you recall?

• Do not forget to use the Flight Progress Strips of the aircraft of concern.

• Do not hesitate to refer to the agreement reached at the beginning of theinterview in case the controller becomes unhappy.



STCA or apparent conflict

• Ask the controller to describe what happened exactly.

- Look for factual information about the conflict (planes, distance, etc.).

- Ask the controller to assess the seriousness of the conflict.

- Reconstruct the sequence of events that have resulted in theconflict/STCA (actions that have led to …).

• Ask the controller to explain how they handled/managed the conflict:

- The moment at which the problem was detected (timing):

Had you perceived the conflict before the STCA alert? Whenexactly?

- The moment at which the conflict was dealt with:

When have you started to deal with the conflict?

- How was the conflict dealt with?:- action, decision made, strategy,- time when decisions were implemented,- expectations regarding the results of the actions,- expected delays.

- Explore the operational context of conflict resolution:- workload, number or aircraft, number of tasks performed,- operational constraints.

Late or early transfers

• Ask the controller to describe the situation:

What happens with this A/C, at this moment?

• IF there is no response or the controller disagrees:

- late transfer: point out the distance between the A/C and the end of thesector;

- early transfer: have the controller comment the distance between theA/C and the end of the sector.



• Ask the controller to explain whether the time of the transfer wasintentional or not:

What has motivated the transfer at this time?Have you transferred the A/C voluntarily at this moment?What has led you to transfer the A/C at this moment?

• IF the controller recalls another story, directly or indirectly linked to thetime of the transfer:

1. Have the controller describe/explain completely the time of thetransfer.

2. Create a new observable attitude to take into account the new story,and have the controller describe this new story.

3. Question the controller’s priorities, and the most important activityundertaken.

Exit of sector

Potential problems: wrong level, wrong heading, wrong sector, etc.Aim to rebuild the story the consequences of which have been observed.

• Ask the controller to describe the problem:What can you tell me about the level/heading/exit sector of thisA/C?

• Rebuild the story:How has the A/C reached this level/heading/sector?

- Question whether it was intentional or not.

- Question the controller’s expectations and expected delays when thelast instruction was delivered to the A/C.

• Question the seriousness of the deviation.

- Ask the controller to explain what the norm is, what was required inthis specific situation.

- Ask the controller to assess the seriousness of the deviation withregards to the norm and to what was required.Have the controller assess the ‘tolerance’ of the deviation.

- Ask the controller to assess the frequency and routine aspect of thedeviation.



Strip annotations

In case of no annotation:

• Ask the controller to explain whether it was intentional or not.

• IF it is an actual omission: have the controller rebuild the operationalcontext (interruption, noise, secondary task, etc.).

In case of erroneous annotation:

What can you tell me about this annotation?

• Characterise the error type:

- Action slip:- Identify what the intention was.- Have the controller rebuild the operational context (interruption,

noise, secondary task, etc.)- Incorrect intention:

- Rebuild the story explaining the wrong intention:What has led you to write this annotation?

• Ask the controller to assess the seriousness of the deviation and itsconsequences (‘tolerance’, margins).

• Question the time of possible detection.Ask the controller to explain how they recovered the error and what theirstrategies were.

In case of incorrect annotation on the strip (incorrect and/or ambiguous):

What can you tell me about this annotation?

• Rebuild the story:What has led you to note …?

• Ask the controller to explain whether it was intentional or not.

• Ask the controller to assess the seriousness of the deviation.



Communications with the pilot

In case of no immediate response:

• Ask the controller to explain the time when they perceived the pilot call.

• Ask the controller to explain what has caused a late response:

- Establish the link to the operational situation (context).- Question the allocation of priorities.

In case of erroneous communication:

What can you tell me about this communication with the pilot?


- Slip of the tongue:- Identify what the intention was.- Have the controller rebuild the operational context (interruption,

noise, secondary task, etc.).- Incorrect intention:

- Rebuild the story explaining the wrong intention:What has led you to …?

• Ask the controller to assess the seriousness of the deviation and itsconsequences (‘tolerance’, margins).

• Question the time of possible detection.

• Ask the controller to explain how they recovered the error and what theirstrategies were.

In case of incorrect communication (incomplete, implicit and/orambiguous):

What can you tell me about this communication with the pilot?

• Rebuild the story:

What has led you to deliver the instruction this way …?

• Ask the controller to explain whether the possible implicit communicationwas intentional or not.

• Ask the controller to assess the seriousness of the deviation.



Behaviours and attitudes

In case of final action or verbalisation:

• Ask the controller to describe the action / content of the verbalisations:

What have you said/done at this time?

• Ask the controller to explain the objectives of the actions / verbalisations:

What was the goal / the objective?

• Try to identify the contents of the thoughts and logic of the controller:

Can you tell what you had exactly in mind at this time?

• Try to establish the origin of the action / verbalisations and of the choiceof the action / (way to decide which action or verbalisation):

What has encouraged you to think this way?

• Question the induced expectations: aim, projection, prediction, expectedeffects, impact on activity.

In case of an attitude (irritation, interjection, etc.):

• Ask the controller to describe the attitude:What have you said/done exactly at this time?

• Identify the origin and the reason for the attitude.

• Identify the contents of the thoughts and logic of the controller:

Can you tell what you had in mind at this time?

• Explain the objective of the attitude:

What was the goal / the objective?

• Question the induced expectations: aim, projection, prediction, expectedeffects, impact on activity.



No reaction to an error committed by someone else

Warning: Do not write down the lack of reaction. This is a very sensitive area.Do not give the controller the impression that it is their fault as theycould be discouraged and not answer.

Involve the observer’s lack of understanding:

I’m not sure of what I’ve seen, I don’t understand, but the pilotdidn’t seem to do what you asked.

The controller did/said something that I don’t understand here.

Did the controller forget to do something …?

IF the controller has seen the erroneous action:

• How did he take into account / cope with the error?

• When did he start to cope with the error?

• Did he have other things in mind (operational context, workload)?

IF the controller has not seen the erroneous action:

• What were the contents of his thoughts at the time of the error?

• What were his preoccupations at the time (expectations, workload, focuson a problem, etc.)?



Change of strategy

• Ask the controller to describe the situation:What can you tell me about the instruction you just delivered to thisA/C?

IF the controller recognises the change of strategy (clear):

• Question the decision-making process on the change of strategy:How have you decided to …?What has led you to …?

• Question the expectations and the delays expected on the results.

IF the controller does not recognise or does not invoke any modificationin the control strategy:

• Rebuild the story:

- Question the elements that have led to the action and that have led tothe time of the action.

- Question the expectations and the delays expected on the results.

• Ask the controller about what they were thinking at the time of the action.



Omission

Two types of omissions can be distinguished:

1. Acknowledged but not performed pilot’s requests (e.g. The pilot asks for aheading change; the controller says “OK” but does not do it or does notcall the pilot back).

2. Not asking about heading or flight level changes when an A/C is deliveredwith incorrect flight parameters or after a modification of the flightparameters by the controller for conflict management purposes or anyother reason.

IF the omission has actually been managed by the controller or byanother person:

• Question the time when it was managed in a neutral way (do not point outthe omission), and try to find out from the controller whether an actioncould have possibly been undertaken before that.

• Ask the controller to describe the situation:

What can you tell me about this instruction?Tell me / describe what you are doing here?

• Rebuild the story of the omission and its management:

- Question the elements that have motivated the action and the time ofthe action.

- Question the opportunities for earlier action.

- Question the expectations and the delays expected on the results.

• Question the time of the detection:When did you realise that you had forgotten …?How did you realise that you had forgotten …?

• Assess the omission and its potential consequences:

- Assess the seriousness of the omission with regards to theprescriptions.

- Question the ‘tolerance’ of the omission.



IF no omission management is observed:

• If some consequences are observed:

- Rebuild the story of the omission.- Assess the seriousness of the omission.

• In the absence of any consequence, refer back to the beginning of theomission sequence: Have the controller describe the situation and theobject of the omission.

Miscellaneous errors

Other errors - any action for which there is a difference between what thecontroller has said and what the controller has actually done. (Note: It isimpossible to talk about actual errors without two sources of information: theaction and the intention.)

• Have the controller describe the situation.


- Action slip, slip of the tongue:

- Identify what the intention was.- Rebuild the operational context (interruption, noise, secondary

task, etc.)

- Incorrect intention:- Rebuild the story:

What has led you to …?

• Assess the seriousness of the deviation and its consequences (‘tolerance’of the deviation, margins).

• Question the time of possible detection.

• Ask the controller to explain how he recovered the error and what thestrategies were.



2.4 Phase 3: The Interview with the Observed Controllers

2.4.1 Instructions for the interview

INTERVIEW INSTRUCTIONS(to be read by/to the interviewee)

Now we are going to debrief together the simulation exercise we haveobserved.

In the course of this exercise we have made observations on your activity. Wehave taken notes of the things that have appeared important to us and wewould like your comments and explanations during this interview.

We have reviewed the film of the radar screen and voice tapes, and haveorganised our information and notes in order to prepare this interview. Wehave noted down all the observations that we would like to discuss with you.

In order to help you verbalise and comment on your activity, we are going toreview the film of the radar screen and ask you some questions. You may findsome of these questions surprising, meaningless or fundamentally obvious.It may also happen that the first answers that come to your mind seem trivialto you. Nevertheless, try to answer all questions. They may help you think andreflect on what happened.

Of course, if you have personal comments to make on sequences we have notplanned to debrief, feel free to ask us to stop the film in order to do so.

Do not hesitate to take time to think about your answers and to let us knowwhen you cannot or do not want to answer for personal reasons.

The interview will last approximately 1½ hours and we are going to take notesthroughout the meeting.

Do you agree to actively participate in this interview?

Then, let’s go …

2.4.2 Interview notebook

Having prepared agreed questions the psychologist should lead thequestioning with the controllers, whilst the ATCO monitors the technicalaspects of the discussion.



INTERVIEW NOTEBOOK

DATE:

OBSERVERS:

INTERVIEWER:

OBSERVATION:

POSITION OBSERVED:

No. QUESTION(S) ANSWERS



2.4.3 Situation control and risk assessment forms

(for the ATCO observer and for the observed ATCO)

Situation control and risk assessment formATCO observer sample

To be filled by the ATCO observer

- In the course of the interview, for each observation commented (oneform per observation).

- At the end of the interview, for the global simulation session (oneform).

For each observation commented on by the observed controller in the courseof the interview make a subjective evaluation of the risk taken, of the degree ofsituation control and of the extent to which safety has been impaired or notimpaired, using the three scales proposed hereafter.

At the end of the interview make a global assessment of the simulationsession using the same three scales (note «G» as Global in the «Time code»column hereafter).

Observation number Time code Aircraft concerned

- How would you assess the risk taken by the ATCO?

- How would you assess the degree of control on the situation by the ATCO?

- Do you think safety has been impaired?

No risk taken Low risk taken Average risktaken

High risk taken(playing with themargins of the

system)

Very low Low Good Very good

Not particularly Little Much Very much



Situation control and risk assessment formObserved ATCO sample

To be filled by the observed ATCO at the end of the interview

This questionnaire will allow you to assess your feeling of situation controlduring the simulation session. For some of the questions you should use theattached scales.

• Have you felt uneasy or threatened at any specific time during thesimulation session?

If yes:

- Please record the moment and the aircraft concerned:

Please, answer the following questions in relation to dealing with thesituation:

- How would you assess the risks you have taken?

- How would you assess your degree of situation control?

- Do you think safety has been impaired?

- How have you recovered the control of the situation?

• Globally, how do you assess your degree of situation control during thewhole simulation session?

No risk taken Low risk taken Average risktaken

High risk taken(playing with themargins of the

system)


Not particularly Little Much Very much




REFERENCES

EATMP Human Resources Team (2002a). The Investigation of Human Errorin ATM Simulation. HRS/HSP-002-REP-05. Ed. 1.0. Released Issue.Brussels: EUROCONTROL.

EATMP Human Resources Team (2002b). The Human Error in ATM (HERA)Technique. HRS/HSP-002-REP-03. Ed. 0.3. Proposed Issue. Brussels:EUROCONTROL.

ABBREVIATIONS AND ACRONYMS

Abbreviations and acronyms used in this technical document are also used inthe main report (EATMP, 2002a). Please refer to page 39 of the main reportfor full designations.



CONTRIBUTORS

NAME ORGANISATION STATE

Capt. M. O’LEARY British Airways UK

B. CONSIDINE EUROCONTROL, IANS

A. GUERRA NAV Portugal Portugal

P. MANA EUROCONTROL, Headquarters

B. WILPERT Technical University Berlin Germany

Document Configuration

C. HELLINCKX EUROCONTROL, Headquarters