Wright State University Wright State University

CORE Scholar CORE Scholar

International Symposium on Aviation Psychology - 2013

International Symposium on Aviation Psychology

2013

An Integrated Neuroergonomic Assessment of In-Flight Pilot An Integrated Neuroergonomic Assessment of In-Flight Pilot

Workload Workload

Steven D. Harbour

James C. Christensen

Justin R. Estepp

Tyron M. Gray

Follow this and additional works at: https://corescholar.libraries.wright.edu/isap_2013

Part of the Other Psychiatry and Psychology Commons

Repository Citation Repository Citation Harbour, S. D., Christensen, J. C., Estepp, J. R., & Gray, T. M. (2013). An Integrated Neuroergonomic Assessment of In-Flight Pilot Workload. 17th International Symposium on Aviation Psychology, 621-626. https://corescholar.libraries.wright.edu/isap_2013/11

This Article is brought to you for free and open access by the International Symposium on Aviation Psychology at CORE Scholar. It has been accepted for inclusion in International Symposium on Aviation Psychology - 2013 by an authorized administrator of CORE Scholar. For more information, please contact library-corescholar@wright.edu.

An Integrated Neuroergonomic Assessment of In-Flight Pilot Workload

Steven D. Harbour1, James C. Christensen1, 2, Justin R. Estepp2, and Tyron M .Gray2 Northcentral University1 and Air Force Life Cycle Management Center1, and the Air Force Research

Laboratory2 Prescott, Arizona1 and Wright Patterson Air Force Base, Dayton, Ohio1, 2

This study accomplished an exploration of a workload protocol and a preliminary assessment of pilot workload and situation awareness in-flight during various phases of airborne operation on a tactical airlift aircraft. Initial comparisons were made between the Head Down Display (HDD) which was previously certified by the FAA as a Primary Flight Display (PFD), and the original Head Up Display (HUD) configuration which has not been endorsed as a PFD. Quantitative results were produced to aid in resource decision making regarding cockpit systems for this aircraft. Low and no cost improvements were identified and recommendations were implemented. This work refined a neuroergonomic protocol for workload assessment.

In flight operations, increasing workload at some point decreases pilot or operator attention, leading the way to mishaps such as controlled flight into terrain (CFIT) (Wickens & McCarley, 2008). Consequently, workload assessment is a critical and ongoing issue despite many years of research (e.g., Tsang & Vidulich, 2006). Measuring aircraft pilot or operator workload in an aerospace context presents unique challenges. For example, subjective self-report measures have traditionally been used to assess workload and SA in the cockpit (AFFSA, 2007) post-flight. This time delay, often more than one hour, can degrade recollection accuracy (Baddeley, 2006; Moroney et al., 2005; Southwick, Morgan, & Hazelett, 2007). While simulations may be paused for collection, actual flight operations preclude this. Therefore, subjective workload data from actual flight may be error prone, causing the workload assessment to be problematic. In addition, the authors have observed that military pilots typically may not be completely forthcoming in self-reporting workload. Therefore, non-intrusive, objective measures have long been sought. The present work sought to test and verify the utility of combined subjective and heart-rate based assessment in tactical airlift flights.

METHODOLOGY

The methods were chosen to be minimally intrusive to both the pilot and the mission. The instruments used have been well-established in previous research; this research was not intended to develop new measures but instead to establish a workload assessment protocol and collect real-world data. Subjective (survey) instruments included the Bedford Workload Scale and China Lake Situation Awareness Scale. These were administered to the pilots post flight. In addition heart rate was measured with the use of three-lead ECG (Hankins & Wilson, 1998; Nickel & Nachreiner, 2003.

The objective measures obtained from ECG included heart rate (HR) and heart rate variability (HRV). Both measures have been extensively studied, and in this study are

taken to be broadly reflective of stress experienced by the pilot. To maximize the ecological validity of this study, all data collection was accomplished with actual flight operations. To avoid disrupting training activities, experimenters could not direct the flight profile for each sortie; relevant segments were extracted post hoc. Consequently this study is a quasi-experimental design.

Participants and Procedures

Participants. Eleven participants were recruited from the Ohio Air National Guard population and ranged in ages from 25 to 46. All of the participants had flying experience and were qualified in the aircraft. Participants flew using either the HDD or both the HDD and HUD (See figures 1, 2, 3, 4, & 5). The experimental protocol was reviewed and approved by the Air Force Research Laboratory’s IRB and the Aeronautical Systems Center’s Flight Safety Review Board. As the study was part of their normal duties, no additional compensation was provided.

Tasks. Aircrew piloted a tactical airlift aircraft flying normal scheduled training missions that consisted of some combination of takeoff, cruise, low level, airdrop, and assault landing phases.

Equipment. The military aircraft had an L-3 Communications HDD and a Rockwell Collins Flight Guidance Systems HUD. The electrocardiographic data was collected using a Vitaport system (Temec Instruments B.V., Kerkrade, Netherlands), which is a small, portable pilot-worn physiological data collection system with onboard digital data storage. The three leads were placed on left and right clavicles and sternum; impedances were verified at or below 40 kOhm.

Procedures. After completing written informed consent, pilots were instrumented for ECG. An experimenter accompanied the flights, and used dedicated marker channels along with written notes to time stamp and categorize flight segments and events of interest. A typical sortie was

621

approximately three hours in duration, with changes in pilot flying and maneuvers being practiced. Post-flight, pilots were asked to complete a set of subjective measures for each segment.

Figure 1. Head Down Displays (HDDs).

Figure 2. Pilot’s (HDDs).

Figure 3. Co-Pilot’s HDDs.

Figure 4. Pilot’s HUD.

Figure 5. Co-Pilot’s HUD.

622

RESULTS

Due to the quasi-experimental nature of this study, there are different numbers of pilots represented in each comparison; this will be noted throughout. Due to the unequal numbers and multiple comparisons, omnibus statistics have not been generated. Multiple sorties or repeated maneuvers were averaged together for each pilot and maneuver type; the plotted means and standard errors reflect a single value from each pilot. All data presented here are drawn from the pilot flying. The HR and HRV data were analyzed by extracting 5-minute segments during each maneuver. ECG data were bandpass filtered from .4 Hz to 30 Hz. R wave peaks were marked using QRStool based on threshold detection followed up with visual inspection and correction. Interbeat intervals were exported for subsequent analyses of average HR and SDNN (in this case, standard deviation of the R-R interval).

The first comparison involved 3 pilots, each of whom completed 2 sorties utilizing both the HDD and original HUD configuration to conduct multiple non-precision instrument approaches, uncoupled from the autopilot (Fig. 1).

Figure 1. Average SDNN for instrument approaches conducted solely heads down (HDD) and with the head-up display available (HUD). Bar height represents the mean, and error bars are one standard error of the mean. Note that higher values are associated with decreased stress and workload.

Based on Figure 1, it appears that the use of the HUD lowers workload. This was also reflected to some degree in the self-report surveys of workload and SA (Figs. 2 and 3). However, note that the self report measures exhibit compression and floor effects – self rated workload is very low and SA very high in both cases.

Figure 2. Average subjective workload ratings, on the Bedford 1 to 10 scale. Higher values correspond to higher workload.

Figure 3. Average subjective situation awareness (SA) ratings, on the China Lake 1 to 5 scale. Higher values correspond to lower SA.

A second comparison was drawn from low-level air drop maneuvers. 5 minute segments were generated by counting back from the release point; the comparison for reference was drawn from data during circling back to begin another air drop. A new set of 3 pilots conducted air drops, with each contributing one average value based on 3 to 4 repetitions of the maneuver (Figs. 4-6).

40

50

60

70

80

90

100

110

HDD HUD

SD o

f R-R

Inte

rval

(ms)

Heart Rate Variability, HDD vs. HUD

123456789

10

HDD HUD

Ave

rage

Sco

re

Subjective Workload (Bedford Scale)

1

2

3

4

5

HDD HUD

Ave

rage

Sco

re

Subjective SA (China Lake Scale)

623

Figure 4. Subjective workload (Bedford scale) during air drops and circling. Higher values correspond to higher workload. Note: circling is at a one.

Figure 5. Subjective situation awareness (China Lake scale) during air drops and circling. Lower values correspond to better SA.

Figure 6. Average SDNN for low-level air drops as compared with circling back to conduct another drop. Variance was

negligible during circling, hence the missing error bar. Higher values correspond to decreased stress/workload.

Assault landings (steep, high-speed approach) resulted in the lowest HRV measures (Fig. 7) of all the maneuvers tested. Eight pilots completed at least five assault landings each. The subjective measures again exhibited compression with a mean workload of 2.7 out of 10, and SA 2 out of 5. Figure 7 was shared with the instructor pilots from the participating units; they confirmed that the relative ordering of maneuver types matches their own opinions.

Given the concerns in the literature regarding HRV measures of workload, future analysis will continue with HR measures as well. One example of HR reactivity to a significant event (air traffic announced by ATC in proximity to the aircraft) is presented in Figure 8.

Figure 7. Average SDNN for all the maneuvers analyzed to date. Higher values correspond to decreased stress/workload.

1

3

5

7

9

Air Drops Circling

Subjective Workload

1

2

3

4

5

Air Drops Circling

Subjective SA

40

50

60

70

80

90

100

110

Air Drop Circling

SD o

f R-R

Inte

rval

(ms)

Heart Rate Variability, Air Drop

40

50

60

70

80

90

100

110

Assault Landing

Instrument Approach

HDD

Air Drop Instrument Approach

HUD

Circling

SD o

f R-R

Inte

rval

(ms)

Heart Rate Variability Rollup

624

Figure 8. Heart rate (HR, in beats per minute) over a 5 minute flight segment taken centered on a significant event, specifically air traffic that required maneuvering to avoid. HR is based on IBIs, smoothed with a 30-sample moving window average (29 sample overlap).

DISCUSSION The primary findings of this study address firstly the

workload experienced in tactical airlift flight operations, and secondly methodological concerns regarding single measure workload assessment with military aircrews.

Similar to Wilson and Fisher (1991), we observed noticeable differences in heart activity between different flight segments. Based on both increases in heart-based measures of workload and subjective ratings, assault landings, air drops, and instrument approaches flown without a HUD were identified as areas of concern with regard to pilot stress and workload. The following recommendations were made.

Recommendation I: Workload during Airdrops was found to be high. Autopilot usage would significantly reduce workload. As a result this recommendation was implemented.

Recommendation II: For assault landings, instrument approaches, and low-level flight a HUD, particularly a certified PFD HUD could reduce workload.

The authors who are rated pilots noted that their own observations of pilot workload in flight were not consistent with the subjective ratings. For example, pilots rated hand-flying a low level air drop as 3 out of 10 on the Bedford scale; the anchor text is “Enough spare capacity for all desirable additional tasks”. We observed channelized attention focused on minimizing lateral deviations, with consequent mistaken radio calls and lack of attention to secondary displays. Our observations were more consistent with 6 or perhaps 7 on this scale. While there may be perceived pressure to underrate workload despite assurances of no negative repercussions, we also feel that pilots may misperceive their workload in the absence of a highly visible error. If this is indeed a systematic feature of subjective ratings taken from this type of population

and task, it further underlines the importance of integrating multiple measurement types in conducting workload studies.

Impact:

The results of this experiment show that a

physiological workload assessment reveals decreased workload when using even the original HUD during instrument approaches. There are ongoing concerns about workload during certain phases of flight; this study is providing input to USAF cockpit system upgrade decisions.

CONCLUSIONS

The results presented here indicate that an integrated neuroergonomic protocol and method for workload assessments of pilots and operators of aerospace vehicles in-flight is appropriate. Adding the objective physiological measurement of heart rate augments the veracity and robustness of the findings. With these considerations in mind, workload assessments using both subjective and objective measures, interdependently, provide a more complete picture of pilot workload and SA than does the use of either alone. More research in this area needs to be accomplished, such as adding Electroencephalogram (EEG) data collection, and testing for the effects of varying spatial locations of information displays (ID). Flying and utilizing a display primarily utilizes vision. Therefore, in order to answer the theoretical questions of why or how workload is changed due to ID design, further research should be accomplished investigating individual differences in visual attention and perception as mediating neurocognitive characteristics, thus contributing to further refinements in theories of workload and attention.

REFERENCES Adams, S.R., Kane, R. & Bates R. (1998). Validation of the China Lake

Situational Awareness scale with 3D SART and S-CAT. China Lake, CA: Naval Air Warfare Center Weapons Division (452330D).

Air Force Flight Standards Agency. (2007). White paper: Primary flight reference endorsement process. Washington, DC: Author.

Baddeley, A. D. (2006). Working memory: An overview. In S. J. Pickering (Ed.), Working memory and education (pp. 3–31). Burlington, MA: Elsevier.

Hankins, T.C. & Wilson, G.F. (1998). A comparison of heart rate, eye activity, EEG, and subjective measures of pilot mental workload during flight. Aviation, Space and Environmental Medicine, 69, 360-367.

Moroney, W.F., Biers, D.W., Eggemeier, F.T., and Mitchell, J.A. (1992). A comparison of two scoring procedures with the NASA task load index in a simulated flight task. In Proceedings of the 1992 National Aerospace and Electronics Conference (pp. 734-740). New York: Institute of Electrical and Electronics Engineers.

60

65

70

75

80

85

90

162.0 164.0 166.0

Hea

rt R

ate

[bpm

]

Flight Time [min]

Heart Rate During a Significant Event

Traffic

625

Nickel, P., & Nachreiner, F. (2003). Sensitivity and diagnosticity of the 0.1-

Hz component of heart rate variability as an indicator of mental workload. Human Factors, 45(4), 575-590.

Roscoe, A.H., & Ellis, G.A. (1990). A subjective rating scale for assessing

pilot workload in flight: A decade of practical use. Bedford, UK: Royal Aerospace Establishment.

Southwick, S., Morgan, C., and Hazelett, G. (2007). Memory for trauma:

Accurate, inaccurate, or just plain false? Retrieved May 26, 2010, from Yale University Educational Videos On Line. Web site: http://www.med.yale.edu/psych/education/videos.html

Tsang, P.S., and Vidulich, M.A. (2006). Mental workload and situation

awareness. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (3rd ed., Chap. 9). New York: Wiley.

Wickens, C. D., & McCarley, J. S. (2008). Applied attention theory. Boca

Raton, FL: Taylor and Francis Group. Wilson, G.F., & Fisher, F. (1991). The use of cardiac and eye blink measures

to determine flight segment in F4 crews. Aviation, Space, and Environmental Medicine, 62, 959-961.

626