integrated tower working position evaluation report …€¦ · the evaluation was performed using...

ITWP

Integrated Tower Working Position

Date: July 27, 2007

Version: 1.0

Status: Public

ITWP – WP4.2 (ITWP Evaluation Results) - Page i Draft


EVALUATION REPORT SESSION 1 (APRIL 2007)

Reference: ITWP Evaluation Report Status: Public Edition No: V1 Date: 27/07/07

Manager(s) Stephane Dubuisson

and Roger Lane

Visa :

Support

Endorsement:

Director(s) Peter Eriksen

Visa :

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public

ITWP – WP4.2 (ITWP Evaluation Results) - Page 1

Table of Contents

1 INTRODUCTION ........................................................................................................................................ 6

1.1 DOCUMENT SCOPE.................................................................................................................................. 6 1.2 STRUCTURE OF THE DOCUMENT.............................................................................................................. 6 1.3 ACRONYMS............................................................................................................................................. 6 1.4 REFERENCE DOCUMENTS........................................................................................................................ 7

2 EXPERIMENT CONTEXT AND OBJECTIVES..................................................................................... 8

2.1 EVALUATION OBJECTIVES ...................................................................................................................... 8 2.2 EXPERIMENTAL CONSTRAINTS................................................................................................................ 8 2.3 EXPERIMENTAL ENVIRONMENT .............................................................................................................. 9

2.3.1 ATCO positions ................................................................................................................................. 9 2.3.2 Pseudo-pilots positions.................................................................................................................... 10 2.3.3 Experimental variables.................................................................................................................... 11 2.3.4 Experimental Plan........................................................................................................................... 12 2.3.5 Participants ..................................................................................................................................... 13

2.3.5.1 Training exercises organisation .............................................................................................................. 13 2.3.5.2 Evaluation exercises organization .......................................................................................................... 14

3 DATA COLLECTION AND ANALYSIS ................................................................................................ 15

3.1.1 Observations.................................................................................................................................... 15 3.1.2 Post exercise questionnaire............................................................................................................. 15 3.1.3 Human factors team debriefings...................................................................................................... 16 3.1.4 Final questionnaire ......................................................................................................................... 16

4 SYNTHESIS OF RESULTS...................................................................................................................... 17

4.1 TRAINING ............................................................................................................................................. 17 4.2 ITWP ENVIRONMENT ........................................................................................................................... 17

4.2.1 ATC Controller Tasks...................................................................................................................... 17 4.2.1.1 GND control position ............................................................................................................................. 17 4.2.1.2 RWY control position ............................................................................................................................ 19

4.3 HUMAN FACTORS ANALYSIS................................................................................................................. 20 4.3.1 Integration of information onto a single screen .............................................................................. 20 4.3.2 Realism of ITWP environment......................................................................................................... 21 4.3.3 System reliability ............................................................................................................................. 21 4.3.4 Presentation of information............................................................................................................. 22 4.3.5 Input devices.................................................................................................................................... 22

4.3.5.1 Mouse..................................................................................................................................................... 22 4.3.5.2 Pen.......................................................................................................................................................... 22 4.3.5.3 Screen size / angle .................................................................................................................................. 22

4.3.6 Use of colours.................................................................................................................................. 23 4.3.6.1 General (consistency) ............................................................................................................................. 23 4.3.6.2 Discrimination........................................................................................................................................ 23 4.3.6.3 Readability ............................................................................................................................................. 23

4.3.7 Ergonomic issues............................................................................................................................. 24 4.3.8 Summary of errors / observations during scenarios........................................................................ 24 4.3.9 Situation awareness analysis........................................................................................................... 24

4.4 HMI OBJECTS AND PROCEDURES.......................................................................................................... 27 4.4.1 Main toolbox window ...................................................................................................................... 27

4.4.1.1 Access to windows ................................................................................................................................. 27 4.4.1.2 Colour, size and position of buttons ....................................................................................................... 28 4.4.1.3 Size, colour of clock............................................................................................................................... 28

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4.4.2 Sub-views window ........................................................................................................................... 28 4.4.3 Map window .................................................................................................................................... 28

4.4.3.1 Menu ...................................................................................................................................................... 28 4.4.3.2 Zoom / Centring / Rings scale ................................................................................................................ 29 4.4.3.3 Label configuration tool ......................................................................................................................... 29 4.4.3.4 EFS bays configuration tool ................................................................................................................... 29

4.4.4 Proposed Menus .............................................................................................................................. 29 4.4.4.1 Clarity of options.................................................................................................................................... 29 4.4.4.2 Scrolling mechanism of the stand menu................................................................................................. 29 4.4.4.3 Ease of use..............................................................................................................................................29

4.4.5 Management of radar labels ........................................................................................................... 29 4.4.5.1 Ease of identification of a/c radar label/tracking .................................................................................... 30 4.4.5.2 Ease of manipulation of radar labels ...................................................................................................... 30 4.4.5.3 Radar labels fonts and sizes.................................................................................................................... 30

4.4.6 RWY alerts....................................................................................................................................... 30 4.4.7 Electronic Flight Strips ................................................................................................................... 30

4.4.7.1 Presentation of information .................................................................................................................... 31 4.4.7.2 Organisation of information in bays ....................................................................................................... 31 4.4.7.3 Organisation of information in bays ....................................................................................................... 31

4.4.8 NAVAIDS window ........................................................................................................................... 31 4.4.9 MET window ................................................................................................................................... 32

4.4.9.1 Minimum information window .............................................................................................................. 32 4.4.9.2 Extended information window ............................................................................................................... 32

4.4.10 Management of stop bars ............................................................................................................ 32

5 DETAILED RESULTS.............................................................................................................................. 33

5.1 SITUATION AWARENESS........................................................................................................................ 33 5.2 ITWP ENVIRONMENT ........................................................................................................................... 40

5.2.1 Controller tasks ............................................................................................................................... 40 5.2.2 Evaluation of realism ...................................................................................................................... 40

5.3 ITWP OBJECTS AND PROCEDURES........................................................................................................ 43 5.3.1 Utility and usability functions ......................................................................................................... 43 5.3.2 General............................................................................................................................................ 46 5.3.3 Input devices.................................................................................................................................... 48 5.3.4 Use of colours.................................................................................................................................. 48

5.4 HMI OBJECTS....................................................................................................................................... 49 5.4.1 Toolbox windows............................................................................................................................. 49 5.4.2 Map windows................................................................................................................................... 49

5.4.2.1 Menus..................................................................................................................................................... 49 5.4.2.2 Zoom / Centring / Rings / Scale ............................................................................................................. 50 5.4.2.3 Labels configuration tool........................................................................................................................ 51 5.4.2.4 EFS bays configuration tool ................................................................................................................... 51 5.4.2.5 EFS configuration tool ........................................................................................................................... 52 5.4.2.6 Proposed menus...................................................................................................................................... 53 5.4.2.7 Management of radar labels ................................................................................................................... 53 5.4.2.8 Extended radar label / EFS window ....................................................................................................... 54 5.4.2.9 RWY alert label...................................................................................................................................... 54 5.4.2.10 Alert window.......................................................................................................................................... 55 5.4.2.11 EFS information ..................................................................................................................................... 56 5.4.2.12 EFS manipulation................................................................................................................................... 57 5.4.2.13 Weather window .................................................................................................................................... 57 5.4.2.14 Navaids window..................................................................................................................................... 59 5.4.2.15 Stop bars management............................................................................................................................ 59

6 CONCLUSION........................................................................................................................................... 60

6.1 SALIENT POINTS OF INITIAL EVALUATION ............................................................................................. 60 6.2 THE WAY FORWARD..............................................................................................................................61

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


7 ANNEXES................................................................................................................................................... 62

7.1 OBSERVATIONS GRID ........................................................................................................................... 63 7.2 POST EXERCISE QUESTIONNAIRE........................................................................................................... 65 7.3 FINAL QUESTIONNAIRE......................................................................................................................... 66

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


EXECUTIVE SUMMARY The aim of ITWP project is to study the integration of the various airport airside system components into an integrated tower working position. Started in April 2006, functional specifications and requirements to be supported by ITWP have been developed based on an initial study (Aug. 06); a possible HMI solution for the EUROCONTROL Experimental Centre has been designed (Nov.06) and a prototype has been developed by end March 2007 to demonstrate the EEC ITWP.

The ITWP prototype has been subject to initial evaluation in early April 2007. Five ATC controllers - Italian, Portuguese, Swedish, German and French - participated in the initial evaluation phase of ITWP, using Stockholm Arlanda airport layout for the simulation environment.

Two control positions (GND, RWY) were subject to evaluation; as the CLD control position was not fully developed. A common environment was designed for both GND/RWY control positions, including a a composite radar screen, mouse/keyboard, and communication peripherals. CLD control position used a tactile screen and a pointer. Two pseudo-pilot positions were allocated to the GND and RWY roles. Variables handled during experiments addressed the rotation of ATC controllers on the three working positions (i.e. RWY, GND, CLD) and traffic samples (variation of factors such as traffic profile and traffic load). After one day training (including 5 training exercises), 6 exercises (50 minutes each) were played (3 in low traffic conditions, 3 in medium high traffic conditions).

The evaluation was performed using direct observations by the EEC ITWP Team during evaluation exercises, post exercise questionnaires oriented towards situational awareness assessment, human factors debriefings, and final questionnaire mainly dedicated to evaluate the utility and usability of HMI objects and associated procedures. 1. Training ATCOs training was considered to be sufficient, though provided for a limited time. Training session duration and proposed exercises were generally accepted to become familiar with ITWP functions. However it was agreed that bringing five controllers into an unknown airport environment with a new HMI environment would require a longer period of training. 2. Evaluation: The expected reduction of adverse effects from the number of displays available for one control position identified in the initial study report was stated. The integration of information onto one screen was accepted and the HMI was considered as intuitive. The facility to open different views, the integration of flight plan and A-SMGCS information, controller-controller coordination improvements (via implementation of Electronic Flight Strips), reduction of moving around from control position were successful. The realism of ITWP prototype could be improved but there was positive feedback about the system response time, behaviour of aircraft and vehicles, coordination with pseudo pilots. Some issues were raised regarding the absence of external view, the number of control positions -only one GND and one RWY control position, and the standardisation of taxi routes. Controllers found it useful to only have the information required in view, but considered that information at this stage of the prototype was sometimes difficult to find, to read or even to interpret. A couple of issues were raised regarding check boxes, clarity of menus and ease of use.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


All controllers agreed that the use of colours was consistent and conventional and did not experience visual fatigue; however, the possibility to test other colour settings, adjust brightness and contrast, solve background/foreground discriminating issues, improve readability would be required in a future system. A mixed feedback regarding the use of input devices was stated. Some controllers disagreed that the input devices were appropriate for performing data entries, especially with regard to the use of mouse (too many inputs) and screen size/angle. The controllers generally felt that using the ITWP system increased their situational awareness, opposed to using their existing systems. Especially situation awareness was considered to be good for the following areas: being ahead of the traffic at all times, feeling comfortable with organising and planning ATCOs’ tasks, not being surprised by event ATCOs did not expect, having a good understanding of the traffic and having an acceptable workload level. Progress must be made to ease finding information on the integrated screen, providing required information for performing their tasks, remove the feeling that there were risks in forgetting something important and the tendency to focus on a single problem or area under responsibility. Positive feedback about ITWP functions and HMI objects was recorded. It must be stressed that neither ITWP functionality nor any HMI object was considered by ATCOs as useless or unusable and no functionality was considered as having no operational utility: • RWY alert label, label configuration tool, alert window, sub-views window and Navaids window

were found useful and usable. • Extended radar / Electronic Flight Strips (EFS) window, stop bars management, EFS configuration

tool and map windows were considered useful but it was suggested to improve HMI. • The utility and usability of zoom/centring/rings/scale functionality was acceptable but should be

consolidated. • EFS bays configuration tool, management of radar labels, proposed menus, electronic flight strips

were considered useful but progress shall be made regarding HMI. Feedback gathered from initial ITWP evaluation was positive and encouraging. A few issues were raised and room for improvement was identified. Proposals for improvements from ATCOs were captured and shall be taken into account before the second evaluation phase in Nov 2007. In addition further development is required to improve ITWP functionality such as implementing label management, detecting non conformance to ATC instructions, detecting inconsistent clearances on EFS, improving stop bars operations, providing a departure sequence to ATCOs (integration of DMAN functionality).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


1 INTRODUCTION

1.1 Document scope The present document “Evaluation Report” depicts and analyzes the results of initial ITWP (experiments Session 1). A second experimental session is planned in november 2007 (Nov 20-22). Session 1 experiments took place at the EEC premises in April 2007 (03-05) on the basis of the ITWP evaluation plan [REF.1]. The evaluation report describes ITWP experiments, from the context and the objectives of the simulations to the results, i.e. the data collected and the analysis of these data.

1.2 Structure of the document

Introduction This section describes the purpose of this document, its structure, the reference documents and gives an explanation of terms used throughout the document. Experiment Context and Objectives This section provides feedback on experiments context, evaluation objectives, ITWP environment and methodology aspects. Data Collection and Analysis This section describes the data collected during the experiments. Synthesis of Results This section presents the main results that were captured during the initial evaluation session. Detailed Results This section addresses the evaluation results in detail. Conclusions This section presents initial conclusions drawn from the evaluation session. Annexes This section provides details about the material used during the evaluation session (questionnaires and obervations grid).

1.3 Acronyms ASMGCS Advanced Surface Movement Guidance and Control System ATC Air Traffic Control ATCO Air Traffic Controller CLD Clearance Delivery CTOT Calculated Take Off Time

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


DMAN Departure Manager eDEP early Demonstration and Evaluation Platform EEC Eurocontrol Experimental Centre EFS Electronic Flight Strip FAA Federal Aviation Agency FDA Flight Data Assistant FPS Flight Plan System GND Ground HF Human Factors HMI Human Machine Interface IFR Instrument Flight Rules ILS Instrument Landing System ITWP Integrated Tower WorkingPosition LAN Local Area Network LCD Liquid Cristal Display OCVSD Operational Concept Validation Strategy R&B Range and Bearing RWY Runway SID Standard Instrument Departure SSR Secondary Surveillance Radar TWR ToWeR VFR Visual Flight Rules

1.4 Reference documents [REF.1] ITWP Evaluation Plan, WP41_v020.doc, 30/03/2007. [REF.2] ITWP: HMI Solution document, v 0.1, 21.12.2006 [REF.3] ITWP - Functional requirements and associated HMI requirements document, v0.21,

16.10.2006 [REF.4] ITWP - initial study report document, v1.0, 21.08.2006

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


2 EXPERIMENT CONTEXT AND OBJECTIVES

2.1 Evaluation objectives Three main hypotheses have been subject to observations during Evaluation Phase S1; Evaluation Phase S2 will supplement initial obervations after improvements and provision of new functionality. The hypotheses to test were as follows: • H1: ITWP provides the ATCOs with the required minimum environment to manage their day-to-

day tasks; • H2: ITWP provides pertinent information, interaction and dialogs functionality; • H3: ITWP helps the controller in managing her/his traffic in a safe and efficient manner. For verifying these hypotheses, the Evaluation Phase S1 concentrated on four evaluation themes (T1 to T4):

• T1: assessing if ITWP environment allows to perform day-to-day controller tasks (RWY, GND, CLD)

• T2: assessing HMI functionality/objects, i.e. the overall pertinence of information presented to controllers (GND, RWY, CLD), ITWP controller dialogs and interaction, system feedback.

• T3: assessing efficiency aspects;

• T4: assessing safety aspects. Note: More details can be found in [[REF.1]], sections 2.4 and 2.5.

2.2 Experimental constraints A number of limiting factors were identified in [[REF.1]], section 4, as influencing the evaluation: • Absence of external view, the ITWP platform is not equipped with a system reproducing the

external view. As looking outside represents an important activity of TWR controller work (GND and RWY) in fine weather conditions, the lack of external view limits the realism.

• Absence of labels anti-overlap facility;

• Limited number of control positions;

• Absence or limited number of vehicles;

• Limited possibility to adapt set-up configuration;

• The majority of participants1 to evaluation had a limited knowledge of the airport platform and topology (Arlanda airport) which is used during experiment sessions.

1 except Swedish ATCOs

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


2.3 Experimental environment The experimental environment was detailed in the Evaluation Plan. Main characteristics are: • Operational environment: one airport environment (Arlanda airport) was used during the

experimental session;

• Two control positions (i.e. GND and RWY control positions) were subject to evaluation. The CLD control position is not fully developped and will be evaluated in november 2007 (session 2). However limited feedback on CLD position (use of tactile screen) were captured during evaluation session 1.

• The controllers made rotations on all positions (GND, RWY, CLD).

Figure 1: ITWP environment (April 2007)

2.3.1 ATCO positions A common ITWP environment was designed for GND and RWY ATCO positions; each of these positions included:

• A composite (i.e. air and ground) radar screen; in the ITWP prototype, the composite radar screen (A) concentrates all information needed by the controller. At current it is a 21” flat screen.

note: possibility to replace the flat screen by a tactile screen will be studied in the future.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


• Peripherals of interaction with the radar screen include a mouse (B) and a keyboard (C).

• Communication peripheral: Audiolan (D) -AudioLAN is a radio-telephone system using voice-over-IP Internet technology- , including headset and microphone (E).

A

B

C

D

E

ITWP environmentITWP environment

Figure 2: Control Position Environment

2.3.2 Pseudo-pilots positions Two pseudo-pilot positions were in charge of managing the traffic from pilot viewpoint. One was in charge of the traffic in contact with CLD and GND controller and one pseudo-pilot position handled the traffic in contact with the RWY controller. Pseudo-pilot positions are outside of the scope of ITWP evaluation but issues raised by pseudo-pilots in managing the traffic were captured for evolution purpose.

The pseudo-equipment of these positions was as follows:

• A composite (i.e. air and ground) radar screen; in the ITWP prototype, the composite radar screen (A) concentrates all information needed by the controller. At current it is a 21” flat screen.

• Peripherals of interaction with the radar screen include a mouse (B) and a keyboard (C).

• Communication peripheral: Audiolan (D) -AudioLAN is a radio-telephone system using voice-over-IP Internet technology- , including headset and microphone (E).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Figure 3: Pseudo-pilot positions

2.3.3 Experimental variables The variables handled by the experimenter were as follows:

• Rotation of the controllers on three working positions (i.e. runway, ground and clearance delivery positions).

• Traffic samples defined according to several factors such as traffic load (low, medium-high) and traffic profile (standard, departing or arriving traffic peak).

Other possible variables such as an event variable (Vdet) to insert disruptive events will be used during the second ITWP evaluation session (november 2007).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Variables Description Arguments

Vrot Rotation variable that materialises the different controllers’ potential roles (inter run session) on working positions.

There were three control positions:

• 1 RWY control position, • 1 GND control position,

• 1 CLD control position.

Vtra Variable dealing with traffic conditions (RWY configuration, traffic load).

The objective of the operation of this variable was to offer a significant range of operational situations. Operated factors in April 2007 were as follows:

• Traffic load (low, busy) and traffic characteristics (arrivals, departures, number of regulated flights, etc.).

Vdet “Event” variable such as insertion of unattended or disruptive events.

The objective for using this variable was to offer a wide range of operational situations while making them more complex (deteriorated conditions), for instance: • Closure of one or several taxiway(s) due to works

in progress, • Runway incursion phenomenon causing a risk of

collision, etc.

Table 1: independent variables

2.3.4 Experimental Plan Different exercises were elaborated to cover simulation needs and to cope with the different phases of the project, encompassing tests, pilots’ training, ATCOs’ training and exercises. Evaluation Phase S1 was split in:

• Training phase; this phase aimed at briefing and familiarising participants with project objectives and scope and HMI functionality; then an exercise was run in order for ATCOs and pseudo-pilots to be familiar with the HMI. Rotations between ATCOs were organised that enabled them to train on GND, RWY and CLD HMIs.

• Experimental session: three exercises were run, each of them having a 50 minutes planned duration. Exercises were broken up in two phases:

• [T0; T0+10’] interval: transitory phase used to launch the traffic, provide necessary instructions to participants, explain conditions (traffic load, meteorological, etc.) and prepare the transfer to exercise phase.

• [T0+10’; T0+50’] interval: exercise phase.

The following statements applied to all runs: • Each simulation run implied the intervention of 3 controllers (a Runway controller, a Ground

controller and a Clearance Delivery controller) and 2 pseudo-pilots.

• Shifting controller roles and positions between simulation runs were made.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


2.3.5 Participants Evaluation Phase S1(Apr 3-5, 2007)

Five ATC controllers - Italian, Portuguese, Swedish, German and French - participated in the initial evaluation phase of ITWP.

Figure 4: ITWP Evaluation Team

ATC controllers swapped between RWY, GND and CLD controller roles and pseudo-pilot positions (see sections below). An EEC ATC controller played a pseudo-pilot role whenever necessary to enable four debriefings with the human factors team.

2.3.5.1 Training exercises organisation Evaluation Phase S1(Apr 3-5, 2007) Five training exercises (TE1 to TE5) were played to ensure each participant to be trained on each control position (RWY, GND, CLD) and pseuso-pilot position. note: in the table below, ‘S’ stands for Swedish ATC controller, ‘G’ for German ATC controller, ‘F’ for French ATC controller, ‘P’ for Portuguese ATC controller, ‘I’ for Italian ATC controller.

GND RWY CLD PP GND PP RWY

TE1 S G I F P

TE2 F P G I S

TE3 I S P G F

TE4 G F S P I

TE5 P I F S

Table 2: Training organisation ( Evaluation Phase S1)

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


2.3.5.2 Evaluation exercises organization Evaluation Phase S1(Apr 3-5, 2007) Two different traffic samples were prepared, one in low traffic conditions, one in medium/high traffic conditions.

GND Ctrl RWY ctrl CLD PP GND PP RWY Debriefing with HF Team

EXL1 F S I P G -

EXL2 P G S I EEC (*) F

EXL3 I F EEC (*) S P G

EXH4 S P F G EEC (*) I

EXH5 G I P F EEC (*) S

EXH6 F S I P G -

Table 2: Organisation of evaluation exercises ( Evaluation Phase S1)

EEC (*): Four debriefing sessions have been organised with Human Factors Team during evaluation exercises (i.e. EXL2, EXL3, EXH4 and EXH5). This explains the adaptations made to organisation during these exercises where an EEC controller played a pilot role (or CLD role) while a participant to evaluation was interviewed by the Human Factors team.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


3 DATA COLLECTION AND ANALYSIS The ITWP initial evaluation was made through:

• Observations made by the Human Factors team and EEC experts during evaluation exercises (see section 3.1.1);

• Post-exercise questionnaires oriented towards the evaluation of situational awareness (see section 3.1.2) ;

• Final Questionnaire mainly dedicated to assess the utility and usability of HMI (see section 3.1.3)

• Human Factors debriefings. Qualitative data analysis was performed with a direct analysis by summarizing the questionnaires (situation awareness, final questionnaires) answers, through synthesizing notes taken during observations and notes taken during debriefing sessions.

3.1.1 Observations Observing the controllers’ activity during the simulation phase helped in collecting data concerning the way how the HMI was used and in identifying specific points and their context of appearance to be discussed during the debriefing sessions. Human factors observers were implied during exercises to gather information via the observations grid. The data was gathered through direct observation. The observers were provided with designed observation grids to simplify and structure the recording. These were structured on all human factors and activities about human activity, efficiency and safety. The observers were more particularly focused on potential issues, difficulties and specific events regarding:

• Controller tasks (RWY, GND, CLD); • HMI usability, information, dialogs, interaction and system feedback; • ATCOs errors if any;

• Possible safety issues.

note: observations grid is provided in annex (see section 7.1).

3.1.2 Post exercise questionnaire After each run, a questionnaire was proposed to all participants in order to gather relevant information about situation awareness; The situation awareness questionnaire is one of the two human factors tools for measuring situation awareness in Air Traffic Management systems developed by the SHAPE (Solutions for Human-Automation Partnerships in European ATM) programme within EUROCONTROL. The SHAPE Questionnaire (SASHA_Q) is a questionnaire-based measure concerned with measuring the controller situation awareness. It is a post-exercise self-rating technique aimed at the individual controller. It consisted of ten questions that were especially designed by taking into account the views of controllers themselves about situation awareness and its indicators (see annex).

note: post-exercise questionnaire is provided in annex (see section 7.2).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


3.1.3 Human factors team debriefings Individual debriefing sessions were also carried out after each run performed with the controllers. The objective was to collect as many opinions (feedback) as possible about particular situations while making use of the working hypotheses and the notes collected during the observations.

3.1.4 Final questionnaire A final questionnaire was distributed to participants at the beginning of the evaluation (morning April 4). This questionnaire was collected at the end of the session. The final questionnaire mainly addressed the utility and usability of HMI objects and functions, as described in the ITWP HMI solution document [REF.2]. It also aimed to assess whether the ITWP environment allows the controller to perform her/his different tasks with sufficient realism. Answers to final questionnaire provided initial assessment and calibration about ITWP functionality and presentation, and helped in determining what should be improved before next evaluation session (Nov.2007).

note: the final questionnaire is provided in annex (see section 7.3).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4 SYNTHESIS OF RESULTS

4.1 Training Before the experiment, each controller team participated in a briefing session aiming at explaining the project and its objectives. A day training was provided before the trials. The objectives of the training were: • Presentation of the session organisation;

• Presentation of the experimental objectives;

• HMIs familiarization.

ATCOs training, though provided for a limited time, was considered to be sufficient. A short training exercise was prepared which consisted of 10 arrivals and 10 departures. Each controller was rotated through the different control positions resulting in 5 training runs being completed. This was generally accepted as sufficient time to become familiar with the ITWP functions in order to perform the evaluation. However, it was understood that bringing five controllers into an unknown airport environment where they have to use a new HMI would in reality require a much longer period of training.

4.2 ITWP environment

4.2.1 ATC Controller Tasks

A number of differences were observed between tasks carried out on the ITWP platform and using either the controllers current system, or their new systems “under development” which were soon to be implemented.

None of these changes were however perceived to be problematic, and there was a general feeling that the ITWP prototype should be able to be flexible so that local procedures/ tasks could be accommodated. Many of the new systems under development closely mirror the functionality of the ITWP system, changing/adapting it to their individual needs.

4.2.1.1 GND control position A) Departing Flights 1. Receiving and organising strips

The task “receive strip” was achieved electronically.

Not all the checks made on strips were consistent. It was commented by several of the controllers that they didn’t routinely check aircraft destination. In several of the systems, information for the aircraft type was also unavailable (i.e. classification as “heavy”) so this was not generally checked, although comments were made that this is a useful piece of information to have, that they would consider integrating into their own systems.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


The task to place the strip in pushback, taxiway, and departure sequence did not change for any of the controllers, although it was sometimes automated.

2. Start-up

Not all controllers had to instruct pilots to start-up the engines -as it is assumed this will be done automatically when the clearance is given-, or transfer the strip to the active bay, which is generally automated. This task was also identified as commonly done by the CLD position, rather than the GND position, depending on local procedures.

3. Push-back

The main difference observed between their existing systems and the ITWP in approving pushback, was in the use of the “visual check” -which could not be carried out using the ITWP prototype. A further difference was in annotating the strip- not all centres recorded information onto the strips, and only Rome did this manually. The Task Analysis in fact states that the recording of information on strips depended on “local procedures” – making the task analysis flexible to this.

4. Taxi clearance

The first task “answer request” is sometimes not carried out, in which case taxiing can also be initiated by the controller (i.e. ‘initiate taxi clearance’).

All controllers agreed that departure information and slot times would then be checked.

The task to “select appropriate route” caused some confusion for many of the controllers, who use very standardized taxi-routes. These standard routes however can still be considered as selecting an ‘appropriate’ route, even if the task of selecting the route is subconscious/ automatic.

5. Aircraft transfer

All controllers agreed on the tasks in this section as applying to their local procedures.

B) Arriving Flights 1. Receiving aircraft

The differences observed between the ITWP and current systems for this task were generally with regards to the mode of transfer of the strip – none of the airports currently employed a FDA (Flight Data Assistant), and the majority of strips were transferred electronically as opposed to physical transfer of paper strips.

2. Giving taxi clearance

To check that the stand is free, many of the controllers said they would almost always do a visual check out of the window in addition to looking on the ASMGCS display. In some cases, the ASMGCS did not even provide this information – thus this is a clear difference with the ITWP system. A change to ITWP is foreseen to show when a stand is occupied.

The other sub-task which is not routinely carried out was removal of the strip from the panel, which was sometimes done automatically. Rather than writing on the strip when taxi-clearance had been given, it was commonly done by a clicking action on the e-strip.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


3. Control of towed aircraft / rescue services

This task was not included, as the ITWP prototype does not yet have the capability to deal with towed aircraft, follow me vehicles or rescue services.

4.2.1.2 RWY control position A) Arriving Flights 1. Monitor approach radar

Not all of the airports have helicopter traffic, so the task “separate from other traffic” did not apply.

2. Landing clearance

It was observed that when using the ITWP the runway check could not be done visually as there was no external view, but in all other towers this is routinely done.

It was also observed that the required braking action information was not available. This is a task which is only done if required -i.e. in special conditions- and most of the controllers interviewed said that they didn’t routinely do this task.

The task of monitoring touch-down/ landing was also a point for discussion, with the lack of external view in the ITWP making this impossible, and certain airport layouts also making it difficult and unlikely for controllers to do this.

The “go-around” instruction was agreed on as a common task among controllers, but this is currently unavailable on the ITWP.

It was also commented that in order to maintain separation, it would be desirable –as exists in many of the current systems- to be able to obtain separation and distance information from the display. There is currently limited availability in the ITWP prototype to measure distance on final but the map will be improved in future versions.

3. Manage aircraft RWY crossings

This was not evaluated as there are no RWY crossings in the current airport layout (Stockholm Arlanda).

4. Carry out vehicle related tasks

This was not simulated in the evaluation.

B) Departing Flights 1. Put strips in departure sequence order

This is pretty much automated in the ITWP (e-strips), although the controller can manually modify the departure sequence if required.

2. Estimates final departure sequence

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


There were some local variations in this task with regards to the information the controllers were provided with; they could only use the information available to them about constraints to estimate the departure sequence (i.e. speed, size, wake vortex classification, departure times, SID were not always available), and local differences (i.e. airport layouts and configurations, number or runways, type of traffic – i.e. mainly IFR or VFR, etc.).

3. Give line-up clearance when next in sequence

Some of the controllers reported that this task was again difficult on the ITWP system, as there was no facility for checking distances (or they were unable/unaware of how to use this facility).

4. Give take-off clearance

Controllers agreed on the majority of sub-tasks, although information for runway condition was unavailable in the ITWP system, and there was also no external view available to check (as they would in their existing systems). Very few of the controllers said that they would record any information on the FPS (i.e. airborne time).

5. Carry out departure after-check

Some of the information checked in the ‘after-check’ on the ITWP task analysis is not available – or controllers were not aware of how to access it (i.e. Squawk, and Altitude information). They all however reported that they generally would double-check this information after departure if using their own systems.

6. Transfer aircraft to departure controller

The only slight difference in tasks between existing systems and the ITWP is that some of the existing systems use paper strips – which are manually transferred/ removed from bay – and the ITWP uses e-strips which are automatically removed from the bay. The action of updating the strips also differs – it is done manually on paper strips, but on a mouse-click in the ITWP system.

Again, vehicle related tasks, other traffic, and rescue services cannot be dealt with using the ITWP system.

4.3 Human Factors analysis

4.3.1 Integration of information onto a single scre en

There was generally a lot of positive feedback about the ITWP system from controllers, who seemed to like having all the information integrated onto one screen. The facility to open different views was thought to be useful. The integration of ASMGCS information and flight data on one screen was thought to be an improvement, and controllers all tended to agree that the screen is big enough to display all the required information adequately.

Another comment was made that the integration of the information onto a single screen helped to identify conflicts (in the RWY position) as the presentation of information was more efficient (i.e. closer together). Controllers liked the feature to be able to right click on labels for flight plan information (“very useful”). It was agreed that the overall reduction of clutter reduced workload – i.e. physical workload was reduced as fewer head movements were required to look at the different displays (i.e. the ASMGCS). Less coordination and moving around from controller position was

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


required. Controllers also thought that the availability of information on one screen is an advantage for enhanced situational awareness. One controller commented that it was easy to form a mental picture of the traffic.

Some negative points were however drawn out, in particular the fact that some data (such as runway braking action, occupied stand, range and bearing etc.) was missing from the screen. It was also noted that controllers had to “look for information”, and sometimes had difficulty in finding the information they needed. This however is probably down to the fact that the system is new to them -although ease of accessibility to information should be continued to be assessed in future workshops.

4.3.2 Realism of ITWP environment

Although the ITWP system is not designed to perfectly mimic a real-life system, a few issues with its practicality and integrity were identified. Firstly, the standardised taxi-routes in the GND position were not considered to be “realistic”, thus controllers were not able to properly predict ground movements and conflicts as strange routes were often chosen.

A comment was also made that with the absence of the external view, it will be impossible for the controllers to be able to validate the concept, as they will be unable to tell whether font sizes, labels, colours, input methods etc. will be adequate in a real-life working situation.

A further issue was raised with respect to the realistic-ness of the simulation set-up - in testing it isn’t really a problem to have just one GND and one TWR position, but the system should really be set with 2 GND and 2 TWR positions to reflect reality and exemplify how to handle more ATCO routes.

Finally, one controller commented that the controllers task is not fully representative of reality – they are required to give too many instructions to a/c (there is no follow me or give way for example) i.e. Hold and Clearances. This was thought to be a problem with the simulation.

4.3.3 System reliability

Several comments were made and issues rose relating to the reliability of the ITWP system. Firstly, controllers commented that in a strip-less environment there will be more head-up time, which is good for maintaining situation awareness and for use of the external view. Overall, controllers said they felt that they could trust the system. There were however a few concerns on what could happen in the real world in the event of failure like losing / missing strips or radar or transponder failure. The observation was also made that by the nature of a ‘simulation’ there was no problem with ‘trusting’ the system, due to the low criticality. In real life however, the issue of ‘trust’ might be a problem i.e. if you suddenly lose strips, miss a strip, system fails etc. However, in general controllers found the system intuitive, and were all able to operate it without too much obvious difficulty.

One controller said they would like to be able to test the system with their own scenarios, as it only seems to be able to cover 50% of scenarios – it doesn’t integrate VFR traffic, helicopters, military etc. There is also limited information available on vehicles, fire, emergency services, etc. The issue of handling VFR traffic was one of the main differences observed and discussed between the ITWP and the controllers existing systems. Controllers seemed keen for the ITWP to integrate a method of handling this into the prototype, as it would make it far more applicable to transfer expertise and aspects of the system into their existing towers.

The issue of familiarity with the system was brought up by several of the controllers – i.e. they are unable to evaluate the effectiveness of the system fully, as they are too unfamiliar with it. In particular,

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


they were unable to locate and access information fast enough.

4.3.4 Presentation of information

Controllers would like to see a default setting for the layout, as it is hard to organise (note: a “PREF” option which allows the screen to be saved is available but was not used during the evaluation).

Others commented that it was good to only have the information required in view, but information was sometimes difficult to find, perhaps due to the unfamiliarity with the system as highlighted above.

When asked whether information was presented in a timely manner, there was a very mixed response. One controller commented that the number of EFS’ presented on the RWY controllers screen could be reduced at certain times (they were coming too early/staying too long) – for example, it is not necessary to have the “pending” strip in the bay when departure is pushing, just after transfer would be preferable. These issues are system parameters and are easily adjusted.

Not all controllers agreed that data was easy to read. The CTOT font should be more visible, and 2 configurations could be useful for Day and Night (background colours etc.). It was generally agreed however that data was easy to interpret, was clear and unambiguous, and easy enough to find on the screen.

A couple of problems were identified with the check boxes – one controller found the boxes too small in the EFS. Some check boxes also did not appear to work – i.e. some of those for closing windows.

Most respondents agreed that it was easy to prioritize between objects on the display; however overlapping of labels can be a problem.

4.3.5 Input devices

Several controllers disagreed that the input devices were appropriate for performing data entries, especially with regard to the use of mouse, pen and screen size / angle.

4.3.5.1 Mouse

Although some find the mouse to be precise and reliable, confidence with the mouse is perhaps more down to practice than anything else. One controller thought there were too many inputs with the mouse in the system. It was also noted that using the mouse is more difficult, as you have to search for the pointer on the screen.

4.3.5.2 Pen

After practice, controllers were comfortable using the pen, and several preferred it. It is easy to use, and gives more direct access to the information. Fatigue was not perceived to be a problem as long as the screen is at a good angle. There were however problems with the hand sometimes obscuring strip information, which is a concern and the sensitivity/ proximity of the pen to the screen.

The touch screen provided in the CLD position was criticised for the fact that it has bigger strips in order to accommodate the pen device.

4.3.5.3 Screen size / angle

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Controllers generally liked the screen, and the size and angle of the screen was fine. It was commented that it should however be lower, to look out.

4.3.6 Use of colours

4.3.6.1 General (consistency)

All controllers agreed that the use of colours was consistent and conventional. Controllers commented that they generally liked the colour choices, and experienced no visual fatigue, especially with the grey base colour. There were however a few comments and suggestions as follows:

Controllers felt that they would like to see other colour settings and the possibility to adjust brightness for different items, such as background, EFS and labels (done through the HMI).

It was suggested that colour should also be used to highlight when events are going out of range – i.e. times etc – to aid in identifying events.

One controller commented that when pointing to the EFS bay, there was a tendency to match the white line to the white pending aircraft that was landing, rather than the rectangular area of the aircraft in concern. This was a confusing use of colour – he suggests that perhaps the pending SC could be light grey instead.

4.3.6.2 Discrimination

Several problems in discriminating between foreground and background colours were identified, as follows:

• The need was identified for a background colour on the strips – for example a light red or pink background could be used for arrivals, and a light blue on departing aircraft, to aid in discrimination. They would also like to see a different colour to distinguish traffic in the push back phase (e.g. yellow or orange) and in the taxi phase.

• The black font is not visible enough on the dark background in the Weather Screen.

• More contrast would be good between strips. They seem to merge together a bit.

• In the CLD position it was difficult to distinguish the grey labels from the grey background.

• Arrival and departure colours were difficult to differentiate. The difference between pending and concerned traffic was not clear.

• There should be a different background colour for pending/ non active traffic.

• Different colours for VFR and IFR would be useful (if VFR traffic were to be integrated into the system).

• A brighter colour would be more readable/ easy to discriminate for arrivals, as the red/brown currently adopted can look black sometimes.

4.3.6.3 Readability

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


There were a few suggestions to improve the readability of specific pieces of information on-screen, as follows:

• CTOT should be bold on the radar label and EFS. The overall colour of the labels could also be a bit bolder to stand out a bit more.

• It would be desirable to have more flexibility in colour settings / brightness etc. in the prototype because there can be a lot of variation in readability due to lighting conditions etc.

4.3.7 Ergonomic issues A couple of minor issues also arose relating to the ergonomics of the system during the debriefing sessions. One controller suggested that there should be a better work console, with a screen cover, to optimise lighting conditions and readability. It was also suggested that a rounded table would be ergonomically better if using the mouse.

4.3.8 Summary of errors / observations during scena rios There were no errors observed during this experiment which can be classified as safety related – and which could be attributed to the functioning of the ITWP – in its current state of development.

4.3.9 Situation awareness analysis

Following analysis of the post exercise questionnaires, the controllers generally felt that using the ITWP system increased their situational awareness as opposed to using their existing systems.

The following graphs summarize the total situation awareness scores for participants, and the average ratings on a per question basis.

note: for more details, see section 5.1

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Column Graph to show Total Situation Awareness scores on post-exercise questionnaires for RWY

and GDN positions

0

10

20

30

40

50

60

70

80

90

100

Average RWY GND

Position

Sco

re o

n S

ituat

ion

Aw

aren

ess

(/10

0)

The above graph shows that situation awareness was on average pretty high (roughly 27/48). Situation awareness was very slightly better in the GND position than in the RWY position, but this difference was too small to be significant.

Column Graph to show Total Situation Awareness scores on post-exercise questionnaires for Busy

vs. Low traffic density

0

10

20

30

40

50

60

70

80

90

100

Average Low Busy

Position

Sco

re o

n S

ituat

ion

Aw

aren

ess

(/10

0)

Situation awareness was also slightly better in the low traffic condition (as could be expected), but this difference was also too small to be significant.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Graph to show average responses on individual questions relating to Situation Awareness

0.001.002.003.004.005.006.00

1 2 3 4 5 6 7 8

Question Number

Situ

atio

n A

war

enes

s R

atin

g on

Lik

ert

Sca

le (

0=lo

w,

6=hi

gh) Average

RWY Controller

GND Controller

Low Traffic Load

Busy Traffic Load

The above graph highlights which questions that situation awareness ratings were lower than average. Generally, ratings for situation awareness were fairly good (>3 on the likert rating scale), however for questions 3, 5, and 7 situation awareness was noticeably lower. No significant differences in responses between positions (RWY vs. GND) or Traffic Load (low vs. high) were found.

These localized differences in situation awareness may have a number of explanations. For example, on Question 3 subjects rated their ability to “find information” on the screen. The low scores corresponding to this question could be easily explained by the controller’s unfamiliarity with the interface and system.

Question 5 related to the risk of forgetting something important – i.e. transferring an aircraft, or communicating a change. This again could be explained by the controller’s unfamiliarity with the system and airport environment, together with the limited current operational experience of some of the controllers.

Question 7 related to whether the subject was surprised by an unexpected event – suggesting an inability to predict future traffic evolution using the system. This could again be explained by the unfamiliarity with the system and interface.

Questions 9 and 10 on the post exercise questionnaire asked controllers to self-rate their Workload and Situation Awareness in the previous exercise. The average response was 4.92 for both workload and Situation awareness – reflecting that controllers felt that workload and SA were both pretty acceptable.

The following graph reflects this:

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Column Chart to show Self-ratings for Workload and Situation Awareness

3.00

3.50

4.00

4.50

5.00

5.50

6.00

Workload Rating SituationAwareness Rating

Question

Rat

ing

(on

ilker

t sca

le,

whe

re

0=lo

w,

6=hi

gh)

Average

Runway Controller

Ground Controller

Average rating in LowTraffic

Average rating in HighTraffic

The graph also reveals that both Situation awareness and Workload ratings were slightly lower for the Runway controller than for the Ground controller. Workload was more acceptable in the low traffic than in the high traffic condition, although Situation Awareness was rated as better in the high traffic condition.

4.4 HMI objects and procedures

4.4.1 Main toolbox window

4.4.1.1 Access to windows

Most respondents agreed that access to windows was adequate, and window overlap was not a problem if well arranged. It was observed however that windows were not always big enough to optimize task performance – i.e. one controller experienced problems with visualization of final approach, i.e. finding the right side of the window.

Another problem identified was that the Labels and EFS were sometimes hidden because of menus (i.e. on one occasion the Stand Information was hidden because of the menu).

One controller felt that it was sometimes not possible to have all the required information up on the screen at once.

Controllers didn’t always want to have the main toolbox window on-screen, as it sometimes obscured view windows, and suggested that it could be collapsible – available to open when needed.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4.4.1.2 Colour, size and position of buttons

Although most controllers were satisfied with this, a couple of suggestions for improvement were given:

• The tool-bar should rotate if we put in on a right or left side and may also disappear (just like windows main tool bar).

• The font of the abbreviation should be different colour than black, and the size of the buttons should be greater.

4.4.1.3 Size, colour of clock

Size and colour of clock was found adequate. However one of the subjects strongly disagreed with this statement, and would like to see a much bigger clock.

4.4.2 Sub-views window

There was a strong agreement that the number of available views was appropriate and that sub-views were useful, although one controller commented that they would like to have 3 views available rather than 2.

The proposed functionality (in the same way as the main view) was also popular, although one controller commented that the zoom factor of the sub-views is different to the zoom factor of the main view, which should perhaps be regulated.

All controllers agreed that they were able to use sub views as an Air Traffic Monitor (e.g. to check separation on final approach and the climb out area), but commented that in order to do this, all ground labels have to be filtered out. It was suggested that a filter be applied on the aircraft on the ground when zooming out

They generally found sub-views easy to use, although one controller commented that it was easy to make mistaken actions in another windows (main or other sub-view) when trying to arrange labels in sub-views.

4.4.3 Map window

4.4.3.1 Menu

It was agreed that all proposed default options are appropriate, and no options were missing. Most controllers were happy with the titles of options; although one commented that the option “ILS” could have several meanings, and should be more clear.

The only real problem identified with menus was that when the map window is opened in the secondary view, the menu is sometimes too long to be fully accessible and you have to enlarge the view to have full access to the options , obscuring other windows.

It was commented that with regards to symbols, a triangle might be clearer than just three arrows. In general, a better colour contrast could be considered.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4.4.3.2 Zoom / Centring / Rings scale

The general consensus on the zooming mouse and drop down menus was that it quite good and easy to handle. It was felt that strips should always be on top of the zoom views.

4.4.3.3 Label configuration tool

One controller suggested that the label selection should include both airborne and ground and that these should be distinguished accordingly.

4.4.3.4 EFS bays configuration tool

There was mixed opinion on the configuration of the EFS bays tool. Some of the controllers for example were not satisfied with the unpredictability of the system and with the fact that after set-up, the functionality group doesn’t save the selection chosen by the controller. It was also suggested that the order of the options such as the time and type of font should be able to be changed by the controller. With regards to options it was mentioned that the menus and font were too small and this required concentration when looking for selection. Some of the controllers stated that the menu that appears when you click on the call-sign overlaps the menu and that perhaps a rolling menu might be more useful. It was also remarked that it was quite difficult to make the menu disappear and this should be changed.

4.4.4 Proposed Menus

4.4.4.1 Clarity of options

Controllers didn’t all agree that all menus were clear and unambiguous. It was commented that in general, Menus are too small to see clearly. One controller criticized the font in particular as being too small, and requiring significant concentration to look to make a selection.

4.4.4.2 Scrolling mechanism of the stand menu

It was suggested that the Parking stand menu should be with sub-menus instead to avoid too much scrolling. Also that clicking on the label the menu should appear without changing (hiding) the call sign: i.e. the menu should disappear after a certain time period (for example 3 sec).

Another suggestion was that on the electronic strips menu, the strip should ‘pop-up’ from the others, as on an Apple Mac Desktop symbol.

4.4.4.3 Ease of use

Not all controllers found the menus entirely easy to use. One in particular didn’t like the menu that appeared upon clicking on the call-sign; it had a tendency to overlap the label, obscuring the stand info on the label. A rolling menu could be a solution to this.

Some controllers also had difficulty in getting the menu to disappear by right clicking or clicking cancel. It should be easier.

4.4.5 Management of radar labels

The majority of the controllers found that it was quite easy to work with the labels as it gives a sense of working with the aircraft. With regard to the options, some controllers felt that the menus and font

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


was a bit small and that it took a great deal of concentration to look for selection. A suggestion was made that by clicking on the label, the menu should automatically appear without changing or hiding the call sign and that this overlapping could be overcome by a rolling menu. The menu could also be set on a time-out phase (i.e. 3 seconds) or else by right clicking or clicking cancel.

4.4.5.1 Ease of identification of a/c radar label/tracking

Some controllers found it hard to find the fields in the label. It was suggested that air ground movements might be a bit easier to control if the aircraft taxi route was shown instead of having to click in order to show it up. It was also commented that track should always be displayed clear and not greyed out, even when aircraft are transferred.

4.4.5.2 Ease of manipulation of radar labels

There did no seem to be any problems with regards to moving radar labels.

4.4.5.3 Radar labels fonts and sizes

One participant also mentioned that CTOT should be put more in evidence by using colour or font size. It was suggested that a background highlight around the label could be efficient than a frame and that perhaps controllers should be able to select an S, M, L (Small, Medium, Large) label size.

4.4.6 RWY alerts

Overall it was considered that conflicts were clearly displayed. However, sometimes the system did not identify conflicts soon enough. The runway conflict information was relatively easy to use but it was suggested that an indication of the type of conflict could be beneficial for example, RWY closed, crossing vehicle, RWY not vacated etc.

With regards to interpretation of radar label information, a pre-configuration to inhibit on certain runway alerts which are not necessarily could be considered. For example, on RWY, a GND alert is not needed. Aircraft should automatically “disappear” after vacating the runway and not on transferral. This should also apply to departures. Having RWY alerts on the GND position can be distracting and therefore not necessary. Care should be taken not to have too many alert windows. It was also stressed that alerts were sufficient in the RWY position and would be useful in the GND position for example in conflicting Aircraft Taxiing.

4.4.7 Electronic Flight Strips

One of the principal advantages of the EFS is that it reduces coordination in the management of flight data leading to better situational awareness.

The general consensus was that the EFS configuration tool was easy to use. With regards to access to the EFS bays, it was suggested that a collapsible e-strip could be put in place and this may take up less space (it is possible to reduce the size of the EFS by deselecting certain fields). There was a little frustration as to the fact that any manual selection which was done wasn’t saved in the system. A few improvements on the presentation of the information on the EFS were suggested such as how RWY indication could be bigger; frequency information could be clearer and how wind information could be improved.

Alerts/alarms on EFS were found clear and easy to interpret.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4.4.7.1 Presentation of information

Certain suggestions were made by the controllers regarding data which they felt was missing from the EFS:

• Availability of the data for the braking action (could be shown in the remarks box of the large MET window);

• Availability of RVR (Runway Visual Range) -available in the extended MET window-;

• Sequence information on light/heavy aircraft (DMAN information is foreseen)

• Indication on Runway Changes (could be shown in the remarks box of the large MET window)

• No squawk was shown on labels. It had to be requested. (SSR is available in the Extended Flight Data Window)

• Bearing and distance information between AC information (R&B) was missing (foreseen in future versions).

• It was generally felt that the text was clear and unambiguous, although one controller mentioned that the font was better in strips than labels, on which it was a bit too small to read comfortably without straining.

4.4.7.2 Organisation of information in bays

It was considered that fields on the strips were quite clear and easy to handle and that indeed eDEP is a big improvement in organising and receiving strips. There was perhaps some difficulty expressed by some of the controllers about the grouping/ungrouping of the strips, the placing of strips in the right position and certain difficulty in cancelling actions and inputs. The sub-groups also tended to overlap each other and this can obscure important information. It was also suggested that it would be appropriate if the strips could be manoeuvred slightly right or left. Overlapping of strips is also an issue and some controllers expressed their wish to be able to save settings and have a default setting for layout of strips and windows.

Some controllers also felt that the strips took up too much space. An interesting suggestion to improve this in the CLD position was instead of a scroll bar to have labels which changed in size, so that labels for a/c nearest in time are larger, showing the full info and those later on are a reduced size.

4.4.7.3 Organisation of information in bays

Most controllers were happy with the order of information on strips. One however felt that the “time” is not as important as call sign so the call sign should come first.

4.4.8 NAVAIDS window

It was suggested that ILS Category II and III status could be needed.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4.4.9 MET window

4.4.9.1 Minimum information window

Overall it was considered that the information provided was sufficient. One controller suggested that it might be more practical if the wind indication was highlighted. It was also stressed that only one wind information at any one time was provided and because there are 2 runways, wind information for both should be observed.

4.4.9.2 Extended information window

It was again remarked that separate wind windows for each RWY should be provided. It was also mentioned that an appropriate means to better locate RVR and braking action at the RWY could be provided. The font on the weather screen could also be improved as the black font on the dark background is not visible enough.

4.4.10 Management of stop bars

The stop bars activation/deactivation function seems to be adequate, useful and easy to use.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5 DETAILED RESULTS

5.1 Situation awareness The post-exercise questionnaire was proposed at the end of each evaluation exercise to GND and RWY ATC Controllers. The following rating scale was used:

1 2 3 4 5 6

strongly disagree

disagree

slightly disagree slightly agree

agree

strongly agree

The overall results are summarised in the table below; 12 opinions were gathered for each assertion; as there was a mix of ‘positive’ (Q1, Q2, Q4, Q8 to Q10) and ‘negative’ assertions (Q3, Q5 to Q7), the average must be carefully read. For each assertion (Q1 to Q10), the distribution of notes and average is provided (example: for assertion Q1, the distribution of answers was: strongly agree (note 6): 1 opinion, agree (note 5): 3 opinions, slightly agree (note 4): 7 opinions, slighthly disagree (note 3): 1 opinion; the average is 4,33). The orange colour (last row of the table below) indicates areas where improvements should be performed (related to proposed assertions).

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10

pos. Q. pos. Q. neg. Q. pos. Q. neg. Q. neg. Q. neg. Q. pos. Q. pos. Q. pos. Q.

notes

6 1 1 1 2 3 2 3

5 3 5 2 2 1 2 6 8 6

4 7 5 6 3 6 2 4 3 1 2

3 1 1 6 4 3 3 2 1 1

2 2 1 6 2

1

Average 4,33 4,50 3,50 3,67 3,75 2,83 4,00 5,00 4,92 4,92 Table: Situation Awareness Questionnaire – distribu tion of answers and average notes

Q1: I was ahead of the traffic at all times. Q2: I felt comfortable with organising and planning my tasks Q3: I had difficulty in finding information. Q4: All the information I needed for performing my tasks was available Q5: There was a risk of forgetting something important (i.e. transferring an a/c on time or communicating a change to an adjacent sector). Q6: I was surprised by an event I did not expect (like an a/c call). Q7: I started to focus on a single problem or a specific area under responsibility Q8: Using information from the integrated screen gave me a good understanding of the traffic. Q9: The level of workload was acceptable in order for me to carry out my required tasks adequately. Q10. Overall, I felt I had adequate Situation Awareness during this exercise

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Two initial conclusions can be drawn from the situation questionnaire: • Situation Awareness was considered good or nearly good for the following areas:

• Being ahead of the traffic at all times; • Feeling comfortable with organising and planning ATCOs’ tasks; • Not being surprised by event ATCOs did not expect; • Having a good understanding of the traffic; • Having an acceptable workload level to carry out their required tasks adequately; • Feeling to have adequate situation awareness during the exercises.

• Situation Awareness must be improved in the following areas:

• Ease in finding information on the integrated screen; • Provision of needed information for performing ATCOs’ tasks; • Feeling that there was risks in forgetting something important; • Tendency to focus on a single problem or specific area under responsibility.

The overall results are detailed below: Q1: I was ahead of the traffic at all times.

Comment: ATC Controllers considered to be slightly ahead of the traffic in all traffic conditions at both RWY and GND control positions (only one slightly disagreed (GND position, busy traffic)). The observed average was 4.33 - between “slightly agree” and “agree” - and the same mean value was observed at RWY and GND control positions. ATC controllers felt slightly more comfortable with the low traffic (average: 4.50) compared to busy traffic (4.17).

0 0

1

7

3

1

0

1

2

3

4

5

6

7

stronglydisagree

disagree slightlydisagree

slightlyagree

agree stronglyagree

1 2 3 4 5 6

average

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Q2: I felt comfortable with organising and planning my tasks Comment: ATC Controllers felt comfortable with organising and planning their tasks, at both RWY and GND control positions; only one ATC controller slightly disagreed (RWY position, busy traffic). The observed average was 4.5 - between “slightly agree” and “agree” but felt more comfortable at the RWY position (4.67) compared to GND position, in busy and low traffic conditions; only one ATC controller slightly disagreed with the statement (RWY position, busy traffic)). Q3: I had difficulty in finding information. Comment: The ATC controllers opinion is mitigated and equally shared between ‘slightly agree’ and ‘slightly disagree’ (observed average: 3,5). No specific example about information difficult to find was

0 0

1

5 5

1

00.5

11.5

22.5

33.5

44.5

5

stronglydisagree


slightlyagree

agree stronglyagree

0 0

6 6

0 00

1

2

3

4

5

6

stronglydisagree


slightlyagree

agree stronglyagree

1 2 3 4 5 6

average

1 2 3 4 5 6

average

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


provided; however some controllers raised the issue of labels overlapping. Q4: All the information I needed for performing my tasks was available Comment: The ATC controllers opinions is mitigated and a large spectrum of answers can be observed (average: 3,67). However, the two negative ‘disagree’ opinions were oberved during the first played exercise (EXL1) exercise at both control positions but with no explanation. Additional information that should be provided was not formally identified. Q5: There was a risk of forgetting something important (i.e. transferring an a/c on time or communicating a change to an adjacent sector).

0

2

4

3

2

1

0

0,5

1

1,5

2

2,5

3

3,5

4

stronglydisagree


slightlyagree

agree stronglyagree

0

1

3

6

2

00

1

2

3

4

5

6

stronglydisagree


slightlyagree

agree stronglyagree

1 2 3 4 5 6

average

1 2 3 4 5 6

average

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Comment: The ATC controllers opinions pointed out the risk of forgetting something important with a 2/3 majority (8 out of 12 opinions), slightly more at RWY positions (Vs GND position). No specific reason was given by ATC controllers and can be attributed to HMI learning (windows, information, labels overlapping, e-strips).

Q6: I was surprised by an event I did not expect (like an a/c call). Comment: The majority of opinions (9 out of 12) indicate that ATC controllers were not surprised by any event they did not expect. However, some ATC controllers indicated that when something in the HMI goes wrong or unexpected, they start to focus on that problem and miss other actions. Especially, when clicking on a wrong button, they might be confused about the outcome. Two ATC controllers (GND position) were confused by the removal of stand information on the label as soon as the aircraft is taxiing.

Q7: I started to focus on a single problem or a specific area under responsibility

0

6

3

2

1

00

1

2

3

4

5

6

stronglydisagree


slightlyagree

agree stronglyagree

0

2 2

4

2 2

0

0,5

1

1,5

2

2,5

3

3,5

4

stronglydisagree


slightlyagree

agree stronglyagree

1 2 3 4 5 6

average

1 2 3 4 5 6

average

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Comment: The majority of opinions (8 out of 12) indicate a tendency of ATC controllers to focus on a single problem or specific area under responsibility. This is mainly due to some HMI unexpected events or outcomes that diverts ATC controllers’attention. Q8: Using information from the integrated screen gave me a good understanding of the traffic. Comment: ATC controllers unanimously noted that the use of information from integrated screen provided them with a good understanding of the traffic (observed average 5.0 (i.e. ‘agree’)). Q9: The level of workload was acceptable in order for me to carry out my required tasks adequately.

0 0 0

3

6

3

0

1

2

3

4

5

6

stronglydisagree


slightlyagree

agree stronglyagree

0 0

1 1

8

2

0123456789

stronglydisagree


slightlyagree

agree stronglyagree

1 2 3 4 5 6

average

1 2 3 4 5 6

average

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Comment: The majority of opinions (8 out of 12) agreed to say that the workload level was acceptable to carry out required tasks adequately for both GND/RWY positions. Only one ATC controller expressed some difficulties (GND position in high traffic). Q10. Overall, I felt I had adequate Situation Awareness during this exercise Comment: A majority of ATC controllers’ opinions (11 out of 12) supported that they had adequate situation awareness during exercises (only one ‘slightly disagreed’ (low traffic conditions, RWY position)).

0 0

1

2

6

3

0

1

2

3

4

5

6

stronglydisagree


slightlyagree

agree stronglyagree

1 2 3 4 5 6

average

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.2 ITWP environment

5.2.1 Controller tasks The evaluation during this phase was oriented towards the perception of ATCOs versus their ability perform main controller tasks. a) ability to perform RWY controller tasks All ATC controllers (no negative opinion) supported during evaluation the idea that they were able to perform RWY controller tasks with ITWP (average: 4.4, between ‘slightly agree’ and ‘agree’).

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The ITWP environment allows RWY controller to perform her/his day to day tasks

4 1

average (4.4)

b) ability to perform GND controller tasks The same tendency (no negative opinion) supported during evaluation the idea that ATC controllers were able to perform GND controller tasks with ITWP (average: 4.2, between ‘slightly agree’ and ‘agree’). The attributed notes underly that improvements can be made for both GND /RWY positions.

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The ITWP environment allows GND controller to perform her/his day to day tasks

4 1

average (4.2)

5.2.2 Evaluation of realism a) overall assessment of ITWP environment realism The ATC controllers supported that the ITWP environment is realistic (average: 4.2, between ‘slightly agree’ and ‘agree’) which is in line with their opinions about the ability to perform their tasks. One ATC controller ‘slightly disagreed’ with this opinion. One raised the issue that “there should be a better console, with screen lower”.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The ITWP environment is realistic 1 2 2

average (4.2)

b) absence of external view Only two ATC controllers’ opinions were gathered2. One raised the issue that “with the lack of outside view, the concept cannot be validated because we don't know if the size of the fonts or colour of the different labels or the way to make the inputs is adequate”. b) realism of behaviour of aircraft and vehicles The level of realism reached by the prototype on this point was considered as rather satisfying. An issue was raised about “some strange routes which have to be checked”.

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The behaviour of aircraft and vehicles is realistic 1 2 2

average (4)

e) system response times

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

System response times are realistic 1 2 1 1

average (4.4)

System response times were considered realistic, no specific comment was raised.

2 It seems that the question was misunderstood by ATCOs.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


f) number of control positions Vs realism

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The number of control positions is realistic 1 1 3

average (3.4)

The number of control positions (one GND, one RWY, one CLD) was considered as an issue: “The system should be set with two GND and two TWR positions to respect reality and give example how to handle more ATCO roles”. The ability to handle more ATCO roles shall be possible in future ITWP evaluations. Information and coordination between ATCO roles shall be supplemented (for instance which information at which time to distribute to GND2 role in a RWY� GND1� GND2 coordination shall be determined). g) coordination with pseudo pilots Vs realism

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The coordination with pseudo pilots is realistic 2 3

average (4.6)

The coordination between ATC Controllers and pseudo pilots was considered realistic. f) System reliability Vs realism

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

System reliability is realistic 1 1 1 1

average (4 opinions) (4.5)

System reliability was considered good (one ATC controller did not answer).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.3 ITWP objects and procedures

5.3.1 Utility and usability functions Assessment of overall utility and usability of main HMI objects/functionality was made (see final questionnaire in annex): • ‘Utility’ refers to the quality of being of practical use , i.e. useful from an operational point of

view. • ‘Usability ’ refers to the effectiveness and efficiency with which controllers can achieve their tasks

in the ITWP environment. It covers practical aspects, such as comprehensibility, readibility, visibility, perceptibility, and comfort of use.

Two statements were made for each HMI object; ATC controllers were asked to make decision on how much they agreed or disagreed with each statement.

1 2 3 4 5 6

strongly disagree

disagree


agree

strongly agree

On the figure below is presented the diagram detailing the interpretation of answers: • when (1<utility ≤ 3.5) and (1<usability ≤ 3.5), the HMI object/functionality is considered useless

and unusable. • when (1<utility ≤ 3.5) but (usability>3.5), this means that the functionality has no real operational

utility but is usable. • when (utility>3.5) but (1<usability ≤ 3.5), this means that the functionality is considered as useful

but that the HMI must be improved. • when both (utility >3.5) and (usability >3.5), the functionality is considered as useful and HMI is

considered usable. Four sub-cases can be identified: • when (utility ≥ 5) but (3.5 <usability<5), the functionality is useful but HMI should be

improved. • when (usabilility ≥ 5) but (3.5 <utility<5), the HMI is usable but the functionality should be

consolidated. • when (3.5 <usability<5) and (3.5 <usability<5), the functionality and HMI should be

improved. • when (utility ≥ 5) and (usability ≥ 5)3, the functionality is useful and the HMI is usable;

3 i.e. assessment of statements are between ‘I agree’ and ‘I strongly agree’

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Table: Utility and Usability functions diagram

The corresponding average assessment is presented below; initial conclusions can be drawn: • No functionality / HMI object was considered as useless and unusable. • No functionality was considered as having no operational utility. • 4 HMI objects / functionality were considered useful but corresponding HMI must be improved. • 5 HMI objects / functionality were considered useful but corresponding HMI should be improved. • 1 HMI object / functionality needs to be consolidated (HMI and functionality). • 5 HMI objects / functionality were considered useful and usable.

Utility

T1.

useful & usable

no operational utility .. but usable

useless .. and unusable

useful .. but HMI must be improved

1 2 3 4 5 6

6

2

2

3

4

4

6

5

Usability

useful .. but HMI should

be improved

usable .. but operational utility to be consolidated

Utility and HMI to be

consolidated

5

4

3

2

1

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Table: Assessment of utility and usability function s (april 2007)

Individual results arepresented in the table below. Utility Usability HMI Object C1 C2 C3 C4 C5 Av. 4 C1 C2 C3 C4 C5 Av. Sub views wndows 5 6 6 5 6 5.6 5 4 5 6 5 5.0 Map windows 5 6 4 5 6 5.2 5 4 4 6 5 4.8 Zoom/centring/rings/scale 4 - 3 4 5 4.0 4 - 3 3 5 3.75 Labels configuration tool 5 6 6 6 6 5.8 5 4 6 6 4 5.0 EFS bays configuration tool 5 6 6 5 6 5.6 5 3 1 4 1 2.8 EFS configuration tool 5 5 6 5 6 5.4 5 5 5 6 4 5.0 Proposed menus 4 6 6 5 5 5.2 4 2 1 4 5 3.2 Management of radar labels 4 6 6 5 6 5.4 5 4 2 4 1 3.2 Extended radar/EFS window 6 - 4 5 6 5.25 6 - 4 6 2 4.5 RWY alert label 6 6 6 6 6 6 6 4 6 6 5 5.4 Alert window 3 6 6 5 - 5.0 4 4 6 6 - 5.0 Electronic Flight Strips 4 6 4 5 6 5.0 4 2 2 4 4 3.2 Weather window 6 6 4 5 - 5.25 5 5 5 3 - 4.5 Navaids window 6 6 4 5 6 5.4 6 5 4 6 5 5.2 Stop bars 6 6 4 5 5 5.2 6 4 3 6 - 4.75

4 C1 to C5 refer to evaluation members. ‘Av.’ stands for ‘Average’

Utility

1 2 3 4 5 6

2

3

4

6

5

Usability

sub-views windows

map windows

zoom/centring/rings/scale

label conf. tool EFS bays conf. tool

EFS conf. tool proposed menus

management of radar labels

extended radar/ EFS window

RWY alert label

alert window Electronic Flight Strips MET window Navaids window

stop bars management

useful and usa ble useful .. but HMI should be improved useful .. but HMI must be improved utility and HMI to be consolidated usable .. but utility to be consolidated no operational utility .. but usable useless .. and unusable

5

4

3

2

1

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Individual results show that: • ‘Usability’ is always below5 ‘Utility ’. This shows that HMI improvement is required. • Assessment regarding ‘Utility ’ is very homogeneous, between 4 and 5 (97% of cases). • Assessment regarding ‘Usability’is not homegeneous and large differences of opinions can be

noted: usability of EFS bays configuration tool and management of radar labels are evaluated between 1 and 5, electronic flight strips between 2 and 4 etc.

note: evaluation member ‘C1’ systematically assessed ‘utility’ and ‘usability’ between 4 and 6.

5.3.2 General

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

HMI is intuitive 1 1 3

average (4.2)

Information is presented simply and in a well organ ised manner 1 1 2 1

average (3.6)

Information is displayed in a timely manner 1 1 1 1

average (3.5)

Data is easy to read (size, font type, colours) 1 2 2

average (3.2)

Use Data is easy to interpret 1 3 1

average (3.8)

Finding data on screen is easy 2 2 1

average (3.8)

Text use clear and simple wording 1 3 1

average (4)

Abbreviations used are unambiguous and do not confu se users 1 2 1 1

average (4.2)

Check boxes (non exclusive buttons) are easy to sel ect 1 1 2 1

average (3.2)

Cursor visibility, movement, size and precision is adequate 1 3 1

average (5)

It is easy to prioritise between objects on the dis play 1 4

average (3.6)

5 or equal (once).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


The initial assessment (april 2007) shows that HMI improvement is required, as statements are quoted between 3.2 and 4.26.

• Issues such as the the selection of check boxes, the readibility of data (size, font type, colours) must be improved.

• Others such as the prioritisation of objects, the interpretation of data, the difficulty to find data on screen, the organisation/presentation of data, the display of data in a timely manner require attention.

It was clear that each controller wanted to set up the screen slightly differently to the normal layout. This involved moving the centre of the map, rotating the map, laying out the EFS bays, having different view windows, position of the MET and clock window. There was no common layout so the function of being able to set up the screen, and in the future save a preferred layout, was seen as an important requirement.

It was requested that the certain windows have priority over others i.e. on top should be the Alert Window, then EFS, then MET window then others. Having windows automatically moving away from each other would be difficult to incorporate.

In the high traffic samples some controllers were finding it difficult to keep up with all of the tasks and EFS management but this is to be expected due to the unfamiliarity with the HMI and Airport and sometimes lack of current controlling experience.

Several minor changes to HMI features were identified for improvement to make the controlling task easier; these include: • Being able to scroll over the symbol and see the planned route on the ground, • Making the actual take off or landing time appear in bold, having trail dots for flights that are not

on the ground, • Reducing the ground clutter when the zoom is large, • Making the data in the MET window larger, • Showing on the EFS when a stand is already occupied, • Having a different symbol for line up and conditional line up, • Reducing the empty space on the EFS, • Making sure that the flights in the runway bay are ordered correctly, • Improving the control of stop-bars and improving the Extended flight data window.

6 Except cursor visibility, movement, size and precision (quoted ‘5’).

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.3.3 Input devices

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Input devices are appropriate for performing data e ntries 3 2

average (3.80)

Input devices are appropriate for selecting objects & positioning cursors 1 2 1 1

average (4.4)

Mouse dynamic characteristics are appropriate 1 1 3

average (4.2)

Individual comment(s): one evaluation member raised the issue of ‘too many inputs with mouse in the system’.

5.3.4 Use of colours

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

The use of coulours is consistent and conventional 2 1


The use of coulours does not reduce screen readibil ity 1 4

average (5.6)

It is easy to discriminate colours 1 3

average (3.75)

Individual comment(s): The following remarks were made:

• The CTOT should be coloured: in brown when the ETOT is in the CTOT window, yellow/orange if ETOT <CTOT window, red if ETOT > CTOT window.

• A background colour is needed on the strips a) light red or pink for arrivals and b) light blue for departing a/c. The use of a dark brown colour for arrivals is not well-chosen

• There should be different colours to distinguish traffic in the pushback phase (e.g. yellow or orange) and in the taxi phase.

• There is a need to see other colour settings, and possibly to adjust brightness for different items like background, EFS and labels.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.4 HMI Objects

5.4.1 Toolbox windows

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Access to windows (i.e. opening/minimizing windows) is adequate 1 2 2

average (4.2)

Abbreviations used are clear (unambiguous) 4 1

average (5.2)

Colour, size and position of buttons is appropriate 1 1 1 2

average (3.8)

Size, clour of clock is adequate 1 1 1 2

average (3.6)

Individual comment(s): The following individual comments were made: • The toolbar should rotate if we put it on a right or left side and may also disappear (just like

windows main toolbar). • The main toolbox window could be colapsable, to open main toolbox window when needed. A

better (or second) way would be to have some actions in "background menu" to open when necessary.

• Sometimes a menu opens under a window view and cannot be visualised. • Size of clock should be bigger.

5.4.2 Map windows

5.4.2.1 Menus

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Proposed default options are appropriate 5

average (5)

Title of options are unambiguous and understandable 1 1 2 1

average (4.2)

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


No option is missing 4 1

average (4.2)

All options are necessary 1 2 2

average (5.2)

Readibility of symbols on map is correct 1 3 1

average (5)

Selection of symbols (parking stands, stop bars, et c.) is correct 2 2 1

average (4.4)

Colours are adequate and do not interfere with othe r objects 1 2 1 1

average (4.4)

Individual comment(s): A few remarks were made by evaluation participants: • When you open the window in the secondary view, sometimes the window is too long to be fully

accessible and you have to enlarge the view to have a full access to the options.

• All ground labels should be fitered out when you use the sub view as an air traffic monitor.

5.4.2.2 Zoom / Centring / Rings / Scale

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Zooming methods are useful and acceptable 1 1 2


Zooming does not interfere with ability to read sym bols/labels/features 1 1 1 1


Centre, Reset (zoom), Rings and Scale functions are useful 1 3


One large problem was when the view window was set with a large scale zoom and the all the ground labels formed a large clutter in the middle. To overcome this there will be a filter for the flights on the ground when the zoom passes >5nm.

Individual comment(s):

• Two evaluation members raised the issue that the zoom factor of the sub views windows is different from factor of the main view;

• It was suggested to ‘apply filter on the a/c on the ground when zooming out’. In addition, one raised the issue that ‘sometimes by a wrong click on the map it shifts suddenly apart unexpectedly’.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.4.2.3 Labels configuration tool

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Difference between ‘MIN’ and ‘SEL’ meaning is clear and unambiguous 2 2 1

average (4)

Default options are appropriate 1 3 1

average (4.2)

Proposed options are adequate and unambiguous 2 2 1

average (4.8)

No option is missing 3 1 1

average (4.6)

All options are necessary 3 2

average (5.4)

Individual comment(s):

• There should be a difference in airborne and ground label. The system should know if label should be displayed as airborne or on the ground.

• The climbing and descending tendency should be implemented.

• Parking is not in accurate position (in the radar label).

5.4.2.4 EFS bays configuration tool

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Access to EFS bays is easy 1 2 2

average (4.2)

Group / ungroup sections functionality is adequate and easy to manage 2 2 1

average (2.6)

Several controllers found that the set up feature for the bays was difficult to use and not ideal for their liking.

Individual comment(s): • The ‘group’ functionality does not save a selection that is manually performed;

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


• There should not be ‘Continue Approach’ in arrival bays; • Arrows could be named something better; • The tool should open closer to the displayed EFS window; • The Controller should be able to change the order of the options (time at the end) and maybe also

the type of the font (bold ..); • Undo option is not available;

note: The use of CA (Continue Approach) and the respective positioning in the runway bay was also felt to be misleading so it was agreed to remove the runway divider and CA clearance on the strip and just have the Runway bay with no divider. Arrival flights would then have the option to be ‘assumed’, ‘on final’, ‘cleared to land’ or ‘go around’.

5.4.2.5 EFS configuration tool

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Default options are appropriate 1 2 2

average (4.2)

Proposed options are adequate and unambiguous 1 4

average (4.8)

No option is missing 2 3

average (3.6)

All options are necessary 1 1 3

average (5.4)

Individual comment(s): • Arrows could be named something better. • The tool should open closer to the displayed EFS window. • The Controller should be able to change the order of the options (time at the end) and maybe also

the type of the font (bold). • ‘Undo’ option is not available.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.4.2.6 Proposed menus

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Options are clear and unambiguous 2 1 2

average (4)

Scrolling mechanism of the stand menu is adequate 1 1 1 1 1

average (4)

Individual comment(s): • Font is too small. You have to concentrate too much for selection • Clicking on the label the menu should appear without changing or hiding the callsign. Menu

should disappear after a certain timing (3s). • Parking stand menu should be with sub menus instead (too much scrolling). • Menus are too small.

5.4.2.7 Management of radar labels

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Radar label information is easy to interpret 1 2 1 1

average (4.4)

Radar label tracking of a/c and vehicle is easy to identify 1 2 2

average (4)

Moving radar label functionality is fast, easy and accurate 1 2 2

average (5)

The status of an a/c (assumed, etc.) is easy to ide ntify 4 1

average (3.4)

Setting up leader length is easy 1 1 1 1 1

average (3.6)

Colours and fonts used are appropriate 1 2 2

average (3.2)

Modifiable fields can be easily accessed 2 3

average (3.6)

The management of the radar labels also caused problems but the functionality to position them has

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


not yet been implemented. Some controllers tried putting the clearances only through the radar label and found this to be useful. Individual comment(s): • It is difficult to find fields in label; • A background faded colour is prefered instead of a frame when label selected; • The leader line should be limited; • Moving labels is not always done as instructed; • Handling the radar labels are not a fully alternative to bays. • The possibility to have different label leader line orientations should be implemented. • There should be an indication (warning) for an incoming a/c in case the stand is occupied by

another a/c.

5.4.2.8 Extended radar label / EFS window

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Access to extended information (rapid or permanent display) is easy 2 1 1


Colours and fonts used are appropriate 1 1 2


Information is well organised, easy to read and int erpret 1 1 1 1

average (4 opinions) (3)

Individual comment(s): • The information is not intuitive; • There are issues in finding fields; • The used font is too small; CTOT must be put in evidence.

5.4.2.9 RWY alert label

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

RWY alerts (info & alarm) are clear and unambiguous ly displayed 2 3

average (5.6)

Colours used are familiar and easy to interpret 1 2 2

average (5)

Alerts are displayed in a timely manner (not too la te or too early) 2 3

average (4.6)

Individual comment(s): • RWY alerts are not needed on GND position; a pre-configuration might be required to inhibate

some alerts or options. • It is suggested that a/c should disappear on RWY position just after vacating and not on

transferring (same with departure a/c). • As red is used for alarms, the text should be white; • It is suggested that it should be possible to set-up alarm time parameters for testing purpose; • An indication of the type of conflict should be good to use (e.g. RWY closed, crossing vehicle,

RWY not vacated, etc.).

5.4.2.10 Alert window

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

RWY alerts (info & alarms) are clear and unambiguou s 1 2 2

average (5.2)

Colours used are familiar and easy to interpret 1 1 3

average (4.8)

Alerts are displayed in a timely manner (not too la te or too early) 2 1 2

Average (5)

All necessary information is displayed 3 1

Average (4 opinions) (4.5)

Individual comment(s): • There are maybe too many alert windows; • There should be a white text on red alarm;

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


• A ring around track should be considered; • An indication of the type of conflict should be provided (e.g. RWY closed, crossing vehicle, RWY

not vacated, etc.).

5.4.2.11 EFS information

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Identifying an arr/dep/vehicle on the EFS is easy 1 4

average (3.8)

Identifying the status (pending, assumed, etc.) of an a/c on EFS is easy 2 2 1

average (3.8)

Information on EFS is clear and unambiguous 1 3 1

average (4)

Colours and fonts used for EFS are adequate 2 2 1

average (3)

Alerts / Alarms are clear, visible and easy to inte rpret 2 3

average (4.60)

All information presented is useful, there is no mi ssing data 1 1 1 2

average (3.8)

Fields that can be modified are easily accessed 2 1 1 1

average (3.2)

Organisation of EFS in bays is appropriate 2 1 1 1

average (3.2)

Criterion used for ordering EFS is adequate and una mbiguous 1 4

average (3.6)

Text used for controller clearances is clear and un ambiguous 1 3 1

average (3.8)

Method for prioritising clearances (most probable i n bold) is appropriate 1 3 1

average (5)

Individual comment(s): • EFS should be able to move slightly on right or left; • The length of an EFS is too big; unused fields should colapse; • The blue colour is better than brown or brighter brown might be acceptable; • EFS fields are too small; • There are too many unfilled areas; • There should be an indication (warning) for an incoming a/c in case the stand is occupied by

another a/c.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.4.2.12 EFS manipulation

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

When required, method for manual organisation of EF S is appropriate 1 1 1 2

average (2.8)

Inputs (clearances, etc.) on the EFS are quick, eas y, and logical 1 2 2

average (3.4)

Marking/unmarking function is easy to use, colour i s appropriate 1 1 3

average (5)

The EFS is removed automatically at the correct mom ent 1 2 1 1

average (4.4)

Giving a wrong clearance via EFS (mistake) is diffi cult to achieve 1 2 1 1

average (4)

Transferring back input clearance should be impleme nted 5

average (6)

It took the controllers some time to manage the positioning of the EFS in relation to other data on the screen and several ended up with windows overlapping or hiding other essential information Individual comment(s): • It is difficult to place the strip in the right position; • Transferring back input clearance should be implemented; • It should be possible to mark different items in label instead of global mark.

5.4.2.13 Weather window Individual comments for minimum and extended weather windows: • A bigger font should be used for the wind; • The wind should appear on RWY strip; • A separate wind windows should be implemented for each RWY TD (Touch Down); • It would be better to locate RVR and braking action at the RWY; • Braking action is missing;

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Minimum Weather Window

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Information provided is visible and easy to interpr et 1 2 2

average (4.2)

The minimum weather information to perform the majo rity of tasks is provided 1 1 2 1

average (4.6)

Information on modified weather information is easy to identify 1 2


Individual comment(s): • The ICAO requirement to have the extreme directions and speed of the wind should be

implemented; • The wind indication must be highlighted;

Extended Weather Window

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Information provided is clear, useful and easy to i nterpret 2 2


All weather information necessary to perform usual tasks is provided 2 2


No additional weather information is required to pe rform control tasks 1 2


Individual comment(s): • Wind arrow should be more clear.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


5.4.2.14 Navaids window

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Access to Navaids information is simple and rapid 1 2 2

average (5.2)

Information provided is clear, useful and easy to i nterpret 1 3 1

average (5)

Knowing if and which navaids serviceability has bee n modified is easy 2 2


Individual comment(s): • ILS CAT II / CAT III indication could be needed;

5.4.2.15 Stop bars management

Str

ongl

y di

sagr

ee

Dis

agre

e

Slig

hly

disa

gree

Slig

htly

agr

ee

Agr

ee

Str

ongl

y ag

ree

Interpretation of stop bars status information is c lear 1 1 1 1


Management of stop bars activation/de-activation fu nction is adequate 1 1 1


Individual comment(s): • Pointing the cursor on the stop bar is difficult (too narrow) • Just clicking on red stopbar in window should make it off. Stop bar should not be in green colour

when off. • Another suggested the opposite, i.e. that stop bars should just be green when off, and take away

the red stop bar by displaying something like a black or white frame on stop bar instead.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


6 CONCLUSION

6.1 Significant points from the initial evaluation The overall feedback gathered from participants to the initial evaluation of ITWP prototype was good. The session organisation was considered correct and time granted to training (one day) was judged adequate to rotate on each control position (five training runs) and test ITWP functionality. However familiarity issues with the ITWP prototype were raised that prevented a sound evaluation of its effectiveness. Even if differences were stated between current controller tasks and those that can be carried out on the ITWP platform, no such modifications were considered problematic as ITWP functionality often keeps up with new systems under development. The integration of information onto one screen was accepted by participants and the HMI is considered as intuitive. The expected reduction of adverse effects from the number of displays available for one control position identified in the initial study report [REF.4] was stated. The facility to open different views, the integration of flight plan and A-SMGCS information, controller-controller coordination improvements, reduction of moving around from control position were successful. The realism of ITWP could be improved but feedback about system response time, behaviour of aircraft and vehicles, coordination with pseudo pilots were positive. Some issues were raised regarding the absence of external view, the limited number of control positions (only one GND and one RWY control position) and the standardisation of taxi routes. Improved information on traffic (vehicles, emergency services), integration of VFR traffic, improved scenarios are required in the future. Positive feedback about situation awareness was recorded regarding the organisation of tasks, the feeling to be ahead of the traffic, the good understanding of the traffic, the workload level; however some issues were raised regarding the ease of finding information, the need for some additional information, the feeling that there could be a risk of forgetting something important or to focus attention on a single problem or area under responsibility. If some credence can be given to unfamiliarity with ITWP and possible consequences (increased time focus to search information or understanding how a specific functionality is implemented), attention should be given to human factors to improve the current situation. A mixed feedback regarding the use of input devices was stated. The reduction of number of mice and keyboards (versus real enviroment) was greeted but some issues were raised regarding the number of inputs to be done via mouse. The use of a touch pen was appreciated even if the hand sometimes obscures some parts of the screen. Regarding utility and usability aspects of HMI objects and procedures, no one was considered useless and unusable or having no operational utility; • RWY alert labels, labels configuration tool, sub-views windows, navaids window and alert

window are considered as useful and usable. • Map windows, stop bars management, extended radar / EFS window, MET window are useful but

corresponding HMI should be improved; • EFS bays configuration tool, management of radar labels, proposed menus and EFS are useful but

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


corresponding HMI must be improved. Comments and possibility for improvement include the organisation of information (especially EFS), readibility, interpretation, priority between objects on the screen, easiness to check boxes, colours (discrimination, colours of buttons, colours and fonts of radar labels, colours and fonts used on extended EFS window), access to information, management of stop bars and configuration of EFS bays.

6.2 The way forward Both refinement of current version and further ITWP development are required. • Refinement of current version in light of initial evaluation results: issues raised during the initial

evaluation must be taken into account; proposed improvements shall be evaluated and implemented to be tested during the second evaluation phase (November 2007).

• Further development is required to focus on:

• Improving the safety net functionality: informing the safety net logic of the aircraft intention via ATCO’s clearances.

• Checking aircraft behaviour with respect to given clearances: checking conformance to clearances, monitoring route deviation;

• Checking aircraft behaviour when clearance has not been given, for instance when an aircraft (or vehicle) enters or crosses runway without clearance.

• Detecting clearances that are not possible, e.g. if a clearance is given to an aircraft type that can not fit down a particular segment (apron, taxiway, closed runway).

• Implementing basic D-Taxi services (for start-up, push-back and taxi). Proposals will be addressed during the next ITWP workshop planned in September 2007.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


7 ANNEXES

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


7.1 Observations Grid The following observations grid was used during ITWP evaluation.

Obervations Grid Date: Session Number:

ATCO Position: Exercise:

Observer:

Controller:

Observations

Time Abbreviation Notes/ Comments

Abbr: OCC (Occupied), INA (Inappropriate), TEC (Technical), CUR (Curiosity), STR (Strategy)

Page:

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Points for consideration in observation Checking and managing flight data Ability to read data Understanding / interpreting the data Ability to scan / find data on the screen Ability to place and move labels. Markers Ability to arrange views / windows / labels Updating strips Speed and ease of inputs Checking weather Accessibility of weather/visibility/wind info. Understanding of weather information Organizing strips Ease of organizing Suitability of order Clarity of presentation Ability to remove and rearrange strips Radar monitoring Ability to check separations Accuracy of picture of climb-out / ground areas Updating FDPS Easy of timely update Intuitiveness of update Coordinating Observe any team interactions of interest Remembering / recalling information Are controllers able to remember key information ? Do they need extra notes / ask questions ? Awareness of the traffic picture Awareness of the traffic evolution ? Ability to access and integrate information on single

screen ? Do controllers identify conflicts in a timely manner ? Errors When and why did errors occur ? Use of controls Use of mouse / pen device

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


7.2 Post exercise questionnaire The following post-exercise questionnaire was oriented towards assessing situation awareness. A set of 10 questions was proposed to ATC controllers after each exercise. Twelve opinions were gathered during ITWP evaluation session 1 (6 exercises x 2 evaluated positions (RWY and GND control positions)). Using the rating scale below, please rate the degree to which you agree/disagree with the following statements

1 2 3 4 5 6

strongly disagree

disagree


agree

strongly agree

Q1: I was ahead of the traffic at all times. Q2: I felt comfortable with organising and planning my tasks Q3: I had difficulty in finding information. Q4: All the information I needed for performing my tasks was available Q5: There was a risk of forgetting something important (i.e. transferring an a/c on time or communicating a change to an adjacent sector). Q6: I was surprised by an event I did not expect (like an a/c call). Q7: I started to focus on a single problem or a specific area under responsibility Q8: Using information from the integrated screen gave me a good understanding of the traffic. Q9: The level of workload was acceptable in order for me to carry out my required tasks adequately. Q10. Overall, I felt I had adequate Situation Awareness during this exercise

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


7.3 Final questionnaire 1. INTRODUCTION Name:

Are you currently in post in a control tower (ATCO) ? Yes No

If ‘Yes’, for how long (number of years) ?

If ‘No’, please specify how long (years) since you were working in a control tower.

2. INSTRUCTIONS Your opinion is an important factor to help us evaluate the effectiveness and functionalityof this system. We would consequently like to ask you to answer thie following questionnaire, relating to your opinions and experience working with the ITWP platform. Your personal experience with the system is particularly important to us. Please note that all information recorded in this questionnaire will be terated as confidential. a) STATEMENTS: The questionnaire contains a number of statements on aspects of the tasks you performed. Please make your decision on how much you agree or disagree with each statement.

1 2 3 4 5 6

strongly disagree

disagree


agree

strongly agree

1 2 3 4 5 6 REF. Statement 1 REF. Statement 1 b) UTILITY and USABILITY : At the end of each section, you will be asked to rate the Utility and Usability of each feature/function of the interface. These are defined as follows: ‘Utility’ - the quality of being of practical use , i.e. useful from an operational point of view. ‘Usability ’ refers to the effectiveness and efficiency with which controllers can achieve their tasks in the ITWP environment. It covers practical aspects, such as comprehensibility, readibility, visibility, perceptibility, and comfort of use. c) COMMENTS – Please feel free to indicate what is meant in your responses in the comments section after each question. At the end of the questionnaire there is also an opportunity to make any practical suggestions for improving the ITWP prototype.

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


3. ITWP ENVIRONMENT

1 2 3 4 5 6

strongly disagree

disagree


agree

strongly agree

1 2 3 4 5 6 E01. The ITWP environment allows GND controller to perform their day-to-day tasks. E02. The ITWP environment allows RWY controller to perform their day-to-day tasks. E03. ITWP environment is realistic The following questions relate to how realistic the system is. Please rate to what extent you agree/disagree that the system reflects reality, wi th respect to the following aspects: 1 2 3 4 5 6 R01. Availability/absence of the external view R02. Behaviour of aircraft and vehicles R03. System response times R04. Number of control positions versus reality R05. Coordination with pseudo-pilots R06. System reliability Comments:

Strongly Disagree

Strongly agree

Strongly Disagree

Strongly agree

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4. HMI OBJECTS AND PROCEDURES

1 2 3 4 5 6

strongly disagree

disagree


agree

strongly agree

4.1 General 1 2 3 4 5 6 G01. HMI is intuitive G02. Information is presented simply and in a well organised manner G03. Information is displayed in a timely manner (not too late or too early) G04. Data is easy to read (size, font type, colours) G05. Data is easy to interpret G06. Finding data on screen is easy G07. Text use clear and simple wording G08. Abbreviations used are unambiguous and do not confuse users G09. Check boxes (non-exclusive buttons) are easy to select G10. Cursor visibility, movement, size and precision is adequate. G11. It is easy to prioritise between objects on the display. Comments: 4.2 Input devices 1 2 3 4 5 6 I01. Input devices are appropriate for performing data entries, I02. Input devices are appropriate for selecting objects & positioning cursors.

I03. Mouse dynamic characteristics (move, action buttons, dims.) are adequate

Comments:

Strongly Disagree

Strongly agree

Strongly Disagree Strongly agree

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


4.3 Use of colours 1 2 3 4 5 6 C01. The use of colours is consistent and conventional. C02. The use of colours does not reduce screen readibility C02. It is easy to discriminate colours Comments: 4.4 HMI Objects 1- Toolbox Windows

1 2 3 4 5 6 HT01. Access to windows (i.e. opening/minimizing windows) is adequate HT02. Abbreviations used are clear (unambiguous) HT03. Colour, size, & position of buttons is appropriate HT04. Size / colour of clock is adequate Comments:

Strongly Disagree

Strongly agree

Strongly Disagree

Strongly agree

ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


2- Sub views windows

1 2 3 4 5 6 HV01. The proposed number of sub-views (2) is appropriate HV02. The proposed functionality (identical to main view) is adequate HV03. A sub view can be used as an Air Traffic Monitor (e.g to check separation on final approach and the climb out area)

Please fill out the following rating scale with respect to utility and usability perspectives, as defined in Section 2 (page 2), using the rating scale as below:

1 2 3 4 5 6

strongly disagree

disagree


agree

strongly agree

1 2 3 4 5 6 HV04. The available sub-viewsare useful HV05. Sub-views are easy to use Comments:



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


3- Map window

Window 1: Menu Window 2: Map

Menu 1 2 3 4 5 6 HM01. Proposed default options are appropriate HM02. Titles of options are unambiguous and understandable HM03. No option is missing HM04. All options are necessary Comments: Map 1 2 3 4 5 6 HM05. Readability of symbols on maps is correct HM06. Selection of symbols (parking stands, stop bars etc.) is easy HM07. Colours are adequate (foreground, background) and do not interfere with other objects (labels, tracks, other symbols)




ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Utility & Usability of Maps in Sub-views 1 2 3 4 5 6 HM08. Maps in sub-views are Useful HM09. Maps in Sub-views are Easy to Use. 4- Zoom / Centring / Rings / Scale

1 2 3 4 5 6 HZ01. Zooming methods (mouse & drop down menu) are useful and acceptable. HZ02. Zooming doesn’t interfere with ability to read symbols/labels/other maps/features

HZ03. Centre, Reset (zoom), Rings and Scale functions are useful Comments: Utility & Usability of Zoom Centring Tool 1 2 3 4 5 6 HZ04. The Zoom Centring Scale range is Useful HZ05. The Zoom Centring Scale range is Easy-to-use 5- Label Configuration Tool

Strongly Disagree

Strongly agree



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Label Configuration Tool 1 2 3 4 5 6 HL01. Difference between ‘MIN’ and ‘SEL’ meaning is clear and unambiguous HL02. Default options are appropriate HL03. Proposed options are adequate and unambiguous HL04. No option is missing HL05. All options are necessary Comments: Now, please rate the usability and utility of the configuration tool with reference to the scales defined in Section 2 (page 2) Utility & Usability of Label Configuration Tool 1 2 3 4 5 6 HL06. Label configuration tool is useful HL07. Label configuration tool is easy to use 6- EFS bays configuration tool

EFS bays configuration tool 1 2 3 4 5 6 HB01. Access to EFS bays is easy HB02. Group / ungroup sections functionality is adequate and easy to manage Comments: Utility & Usability of EFS Bays Configuration Tool 1 2 3 4 5 6 HB03. EFS bays configuration tool is useful HB04. EFS bays configuration tool is easy to use 7- EFS Configuration Tool

Strongly Disagree

Strongly agree



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


EFS configuration tool 1 2 3 4 5 6 HEC01. Default options are appropriate HEC02. Proposed options are adequate and unambiguous HEC03. No option is missing HEC04. All options are necessary Comments: Utility & Usability of EFS Configuration Tool 1 2 3 4 5 6 HEC05. The EFS configuration tool is useful HEC06. The EFS configuration tool is easy to use 8- Proposed Menus

runway menu callsign menu parking/stand menu




ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Proposed Menus 1 2 3 4 5 6 HME01. Options are clear and unambiguous HME02. Scrolling mechanism of the stand menu is adequate Comments: Utility & Usability of Proposed Menus 1 2 3 4 5 6 HME03. The proposed menus are useful HME04. The proposed menus are easy to use 9- Management of radar labels

Management of Radar Labels 1 2 3 4 5 6



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


HRL01. Radar Label information is easy to interpret HRL02. Radar Label / tracking of aircraft and vehicle is easy to identify HRL03. Moving radar label functionality is fast, easy and accurate HRL04. The status of an aircraft (assumed, etc.) is easy to identify HRL05. Setting up leader length is easy HRL06. Colours and fonts used are appropriate HRL07. Modifiable fields can be easily accessed Comments: Utility & Usability of Zoom Radar Labels 1 2 3 4 5 6 HRL08. The radar labels are useful HRL09. The radar labels are easy to use 10- Extended Radar / EFS Window

Extended Radar/ EFS Window. 1 2 3 4 5 6 HER01. Access to extended information (rapid or permanent display) is easy HER02. Colours and fonts used are appropriate HER03. Information is well organised, easy to read and interpret Comments:




ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Utility & Usability of Extended Radar/EFS 1 2 3 4 5 6 HER04. The extended EFS/labels functionality is useful HER05. The extended EFS/labels functionality is easy to use 11- RWY Alert Label

radar label - Information radar label - Information runway alert/ step 1 runway alert/ step 1 (selected label) (minimum label)

radar label - Alarm radar label - Alarm runway alert/ step 2 runway alert/ step 2 (selected label) (minimum label) RWY Alert Label 1 2 3 4 5 6 HRC01. RWY Alerts (Info. & Alarms) are clear, and unambiguously displayed HRC02. Colours used are familiar and easy to interpret HRC03. Alerts are displayed in a timely manner (not too late or too early) Comments: Utility & Usability of RWY Conflict Alert 1 2 3 4 5 6 HRC04. RWY conflict alert information is useful HRC05. RWY conflict information is easy to use



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


12- Alert Window

alert window with runway alerts/ stage 2 Alarm

alert window with runway conflict alerts/ stage 1 (Information) Alert Window 1 2 3 4 5 6 HAW01. RWY Alerts (Info. & Alarms) are clear and unambiguous HAW02. Colours used are familiar and easy to interpret HAW03. Alerts are displayed in a timely manner (not too late or too early) HAW04. All necessary information is displayed Comments: Utility & Usability of Alert Window 1 2 3 4 5 6 HAW05. The alert window and information is useful HAW06. The alert window is easy to use 13- Electronic Flight Strips Example of arrival strips



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Example of departure strips

a) EFS information 1 2 3 4 5 6 HEI01. Identifying an arr/dep/ vehicle on the EFS is easy HEI02. Identifying the status (i.e. Pending, Assume) of an aircraft on EFS is easy

HEI03. Information on EFS is clear and unambiguous HEI04. Colours and font used for EFS are adequate HEI05. Alerts/alarms are clear, visible and easy to interpret HEI06. All information presented is useful, there is no missing data HEI07. Fields that can be modified are easily accessed HEI08. Organisation of EFS in bays is appropriate (correct clearance options appear at the right time)

HEI09. Criterion used for ordering EFS is adequate and sufficient HEI10. Text used for controller clearances is clear and unambiguous HEI11. Method for prioritising clearances (most probable in bold ) is appropriate Comments: b) EFS manipulation 1 2 3 4 5 6 HEM01. When required, method for manual organisation of EFS is appropriate HEM02. Inputs (Clearances etc) on the EFS are quick, easy and logical



ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


HEM03. Marking/unmarking function is easy to use, colour is appropriate HEM04. The EFS is removed automatically at the correct moment HEM05. Giving a wrong clearance via EFS (mistake) is difficult to achieve HEM06. Transferring back input clearance should be implemented Comments: Utility & Usability of EFS Function 1 2 3 4 5 6 HEM07. ITWP EFS functionality is useful HEM08. ITWP EFS is easy to use 14- Weather window

Minimum weather information Extended weather information window window a) Minimum Information Window 1 2 3 4 5 6 HW01. Information provided is visible and easy to interpret HW02. The minimum weather information necessary to perform the majority of tasks is provided

HW03.Information on modified weather information is easy to identify b) Extended Information Window 1 2 3 4 5 6 HW04. Information provided is clear, useful and easy to interpret HW05. All weather information necessary to perform all usual tasks is provided HW06. No additional weather information is required to perform control tasks




ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Comments: Utility & Usability of Weather Information Window 1 2 3 4 5 6 HW07. Weather information window is useful HW08. Weather information window is easy to use 15- Airport Equipment window

Airport Equipment Window 1 2 3 4 5 6 HN01. Access to navaids information is simple and rapid HN02. Information provided is clear, useful and easy to interpret HN03. Knowing if and which navaids serviceability has been modified is easy Comments: Utility & Usability of Airport Equipment Window 1 2 3 4 5 6 HN04. Navaids window is useful HN05. Navaids window is easy to use 16- Stop Bars Management




ITWP


Date: July 27, 2007

Version: 1.0

Status: Public


Stop Bars Management 1 2 3 4 5 6 HS01. Interpretation of stop bars status information is clear HS02. Management of stop bars activation/de-activation function is adequate Comments: Utility & Usability of Stop Bars Function 1 2 3 4 5 6 HS03. The stop bars management function is useful HS04. The stop bars function is easy to use

5. SUGGESTIONS FOR IMPROVEMENT Please feel free to indicate any further suggestion for improving the ITWP prototype

Thank you for your time and cooperation in filling out this Questionnaire.



integrated tower working position evaluation report …€¦ · the evaluation was performed using...

Documents