soldier information presentation and cognitive load

Defence Research and Development Canada Scientific Report

DRDC-RDDC-2021-R083

June 2021

CAN UNCLASSIFIED

CAN UNCLASSIFIED

Soldier Information Presentation and Cognitive Load

Final Report on Soldier System Effectiveness Project Work Breakdown Element 1.5.1

Justin G. Hollands DRDC – Toronto Research Centre Terms of Release: This document is approved for public release.

CAN UNCLASSIFIED

Template in use: EO Publishing App for SR-RD-EC Eng 2021-02-11.dotm © Her Majesty the Queen in Right of Canada (Department of National Defence), 2021

© Sa Majesté la Reine en droit du Canada (Ministère de la Défense nationale), 2021

CAN UNCLASSIFIED

IMPORTANT INFORMATIVE STATEMENTS

This document was reviewed for Controlled Goods by Defence Research and Development Canada (DRDC) using the Schedule to the Defence Production Act.

Disclaimer: This publication was prepared by Defence Research and Development Canada an agency of the Department of National Defence. The information contained in this publication has been derived and determined through best practice and adherence to the highest standards of responsible conduct of scientific research. This information is intended for the use of the Department of National Defence, the Canadian Armed Forces (“Canada”) and Public Safety partners and, as permitted, may be shared with academia, industry, Canada’s allies, and the public (“Third Parties”). Any use by, or any reliance on or decisions made based on this publication by Third Parties, are done at their own risk and responsibility. Canada does not assume any liability for any damages or losses which may arise from any use of, or reliance on, the publication.

Endorsement statement: This publication has been peer-reviewed and published by the Editorial Office of Defence Research and Development Canada, an agency of the Department of National Defence of Canada. Inquiries can be sent to: [email protected].

CAN UNCLASSIFIED

DRDC-RDDC-2021-R083 i

CAN UNCLASSIFIED

Abstract

The Soldier System Effectiveness (SoSE) project sought to improve soldier effectiveness by decreasing physical and cognitive burden on the soldier and increasing resilience, protection, and mobility in an integrated, human-centric soldier system. Information presentation for soldiers has become an important topic given the advent of portable battle management systems (BMSs), such as the Integrated Soldier System—Suite (ISS-S) being fielded by the Canadian Army (CA). The SoSE Decision Aids Work Breakdown Element (WBE) contained a sub-WBE 1.5.1 focused on Display Methodology and Information Requirements. Studies that investigated information presentation methods for soldiers were conducted under SoSE 1.5.1, and are summarized here. These include research efforts on blue force tracking, soldier symbology, head-mounted display, observations of soldiers using BMSs from Bold Quest 17.2, a future urban team experimentation facility, and a novel augmented reality method for navigation assist called Mirror in the Sky (or MitS). There is a concern within the CA that the introduction of BMSs to dismounted soldiers might increase soldier cognitive load and impair soldier performance. Hence, research on cognitive load was also conducted under SoSE 1.5.1. This research included focus group interviews, a scientific literature review, experiments seeking to validate the detection response task (DRT) as a measure of cognitive load for soldiers, field studies using the DRT, and finally a description of how cognitive load was measured in the evaluation of MitS. The international program relevant to SoSE 1.5.1 is also described. Key outcomes of SoSE 1.5.1 research activities are summarized. Three broad results are noted. First, soldier BMSs generally either improve performance and reduce cognitive load relative to conventional methods, or do not markedly impair performance or workload. Second, AR systems do not appear to improve human performance or reduce workload relative to map-based systems, although the technology is rapidly changing and more studies will be necessary as functionality improves. Third, the DRT offers a validated method for real-time measurement of soldier cognitive load in field trials or mission rehearsal. In summary, SoSE 1.5.1 provided insight into factors affecting how information should be presented to soldiers and developed viable methods for measuring soldier cognitive load.

Significance to Defence and Security

This Scientific Report summarizes research outcomes relating to information display and cognitive load for soldiers, conducted under the Soldier System Effectiveness project. Soldier-borne battle management systems were shown to improve navigation and team performance and reduce cognitive load relative to conventional methods, or not markedly impair performance or workload. Augmented reality systems do not appear to improve human performance or reduce workload relative to map-based systems, although the technology is rapidly changing and more studies will be necessary as functionality improves. The detection response task offers a validated method for measuring the cognitive load of soldiers in the field.

CAN UNCLASSIFIED

ii DRDC-RDDC-2021-R083

CAN UNCLASSIFIED

Résumé

Le projet sur l’Efficacité du système du soldat (ESSdt) visait à accroître l’efficacité des soldats en diminuant leur fardeau physique et cognitif et en augmentant leur résilience, leur protection et leur mobilité grâce à un équipement intégré du soldat axé sur la personne. La présentation de l’information à l’intention des soldats est devenue un sujet important avec l’avènement des systèmes portables de gestion du combat (BMS), tels que la suite d’équipement intégré du soldat (S-ÉIS) mise en service par l’Armée canadienne (AC). L’élément de répartition du travail (ERT) sur les aides à la décision de l’ESSdt comportait un sous-ERT 1.5.1 axé sur la méthodologie d’affichage et les besoins en information. On a ainsi mené, dans le cadre de l’ESSdt, des études portant sur les méthodes de présentation de l’information à l’intention des soldats. Celles-ci sont résumées dans le présent document. Elles comprennent des efforts de recherche sur le suivi des forces amies, la symbologie du soldat, le visiocasque, les données d’observation des soldats utilisant les BMS de l’exercice BoldQuest 17.2, la planification d’un futur laboratoire d’expérimentation par une équipe urbaine (EEU) et une nouvelle méthode de réalité augmentée (RA) d’aide à la navigation appelée « Mirror in the Sky » (ou MitS). L’AC craint toutefois que la mise en place de BMS pour les soldats débarqués n’augmente leur charge cognitive et, par conséquent, ne nuise à leur performance. On a donc mené des recherches sur la charge cognitive dans le cadre de l’ESSdt 1.5.1. Ces recherches comportaient des entrevues avec des groupes de discussion, une revue de la littérature scientifique, des expériences visant à valider la tâche d’intervention de détection (TID) comme mesure de la charge cognitive des soldats, des études sur le terrain en utilisant la TID, ainsi qu’une description de la façon dont la charge cognitive a été mesurée lors de l’évaluation de MitS. Une description du programme international relatif à l’ESSdt 1.5.1 figure également, de même qu’un résumé des principaux résultats des activités de recherche de l’ESSdt 1.5.1. Trois résultats généraux figurent dans le présent document. Premièrement, les BMS des soldats vont généralement améliorer la performance et diminuer la charge cognitive par rapport aux méthodes classiques, ou bien ne nuiront pas de façon marquée à la performance ni à la charge de travail. Deuxièmement, les systèmes de RA ne semblent pas améliorer la performance humaine ni réduire la charge de travail par rapport aux systèmes cartographiques. Toutefois, la technologie évolue rapidement et d’autres études seront nécessaires à mesure qu’on améliorera la fonctionnalité. Troisièmement, la TID offre une méthode validée pour mesurer la charge cognitive des soldats lors d’essais sur le terrain ou d’exercices de mission. En résumé, l’ESSdt 1.5.1 a permis de mieux comprendre les facteurs qui influencent la façon dont l’information devrait être présentée aux soldats et d’élaborer des méthodes viables pour mesurer leur charge cognitive.

Importance pour la défense et la sécurité

Le présent rapport scientifique constitue un résumé des résultats des recherches portant sur l’affichage d’information et la charge cognitive des soldats menées dans le cadre du projet sur l’Efficacité du système du soldat. On a démontré que les systèmes portables de gestion du combat vont améliorer la performance de navigation et la performance d’équipe en plus de réduire la charge cognitive par rapport aux méthodes classiques, ou bien ne nuiront pas de façon marquée à la performance ni à la charge de travail. Par contre, les systèmes de réalité augmentée ne semblent pas améliorer la performance humaine ni réduire la charge de travail par rapport aux systèmes cartographiques. Toutefois, la technologie évolue rapidement et d’autres études seront nécessaires à mesure qu’on améliorera la fonctionnalité. Enfin, la tâche

CAN UNCLASSIFIED

DRDC-RDDC-2021-R083 iii

CAN UNCLASSIFIED

d’intervention de détection offre une méthode validée pour mesurer la charge cognitive des soldats sur le terrain.

CAN UNCLASSIFIED

iv DRDC-RDDC-2021-R083

CAN UNCLASSIFIED

Table of Contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

Significance to Defence and Security . . . . . . . . . . . . . . . . . . . . . . . . . i

Résumé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

Importance pour la défense et la sécurité . . . . . . . . . . . . . . . . . . . . . . . ii

Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Report Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 The Integrated Soldier System—Suite (ISS-S) . . . . . . . . . . . . . . . . . 2

2 Information Presentation for Soldiers . . . . . . . . . . . . . . . . . . . . . . . 3

2.1 Blue Force Tracking (BFT) . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Soldier Symbology . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3 Head-up Display (HUD) . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.4 Bold Quest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.4.1 Canadian Platoon ISS-S BMS . . . . . . . . . . . . . . . . . . . . . 7

2.4.2 NZ Platoon—SitaWare . . . . . . . . . . . . . . . . . . . . . . . 8

2.4.3 US Marine Corps Squad . . . . . . . . . . . . . . . . . . . . . . . 9

2.4.4 Bold Quest Summary . . . . . . . . . . . . . . . . . . . . . . . . 9

2.5 Urban Team Experimentation . . . . . . . . . . . . . . . . . . . . . . . 10

2.6 Navigation Assist: Mirror in the Sky . . . . . . . . . . . . . . . . . . . . 10

2.6.1 Mirror in the Sky (MitS) . . . . . . . . . . . . . . . . . . . . . . 11

2.6.2 MitS Summary . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Cognitive Load . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.1 Focus Group Interviews. . . . . . . . . . . . . . . . . . . . . . . . . 14

3.2 Cognitive Load Literature Review . . . . . . . . . . . . . . . . . . . . . 15

3.3 Detection Response Task (DRT) Experiments . . . . . . . . . . . . . . . . . 16

3.4 Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.5 Soldier Performance and Cognitive Load Using ISS-S . . . . . . . . . . . . . 20

3.6 Assessing Cognitive Load for MitS . . . . . . . . . . . . . . . . . . . . 21

4 International Activities Related to Information Presentation and Cognitive Load . . . . . . 22

4.1 NATO Human Factors and Medicine (HFM) Research Task Group (RTG) 319 . . . . 22

4.2 TTCP Human Factors (HUM) JP1 . . . . . . . . . . . . . . . . . . . . . 22

5 Summary and Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

List of Symbols/Abbreviations/Acronyms/Initialisms . . . . . . . . . . . . . . . . . . 31

CAN UNCLASSIFIED

DRDC-RDDC-2021-R083 v

CAN UNCLASSIFIED

List of Figures

Figure 1: An example of the blue force tracking (BFT) map used in [Ho et al. 2013]. . . . . . 3

Figure 2: Examples of augmented reality simulations for blue force tracking (BFT) used by Ho et al. [2019]. From left to right: ribbon, box, and dot conditions. . . . . . . . . 4

Figure 3: Prototype Revision Desert Locust Goggle System fitted with a Recon head-up display (HUD) [Lamb and Hollands, 2019]. . . . . . . . . . . . . . . . . . . . . . 6

Figure 4: The SitaWare battle management system being used by New Zealand Army troops at Bold Quest 2017. Note cabling arrangement. Photo by author, Fort Stewart, Georgia, US. . 9

Figure 5: An example of the simulated virtual reality version of Mirror in the Sky (MitS) used in Reiner et al. [2020]. . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Figure 6: Results found in the cognitive load laboratory experiment described in Hollands, Spivak, and Kramkowski [2019]. . . . . . . . . . . . . . . . . . . . . . . . 18

CAN UNCLASSIFIED

vi DRDC-RDDC-2021-R083

CAN UNCLASSIFIED

List of Tables

Table 1: Summary of Mirror in the Sky Experiments (adapted from [Reiner, 2020]) See text for details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Table 2: Examples of messages presented to participants for the detection response task (DRT) laboratory studies. Message number refers to the position of the message in the temporal sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

CAN UNCLASSIFIED

DRDC-RDDC-2021-R083 1

CAN UNCLASSIFIED

1 Introduction

Defence Research and Development Canada (DRDC) oversees the Army science and technology (S&T) Portfolio. The Army Portfolio includes The Soldier Program, which is purpose-built to address S&T investments needed to achieve the Army’s objectives for improved soldier effectiveness [Williams, 2016]. The Soldier Program included two projects: Future Small Arms Research (FSAR) and Soldier System Effectiveness (SoSE). Both projects ended in 2019/20.

The SoSE project sought to improve soldier effectiveness by decreasing soldier burden and increasing resilience, protection, and mobility in an integrated, human-centric soldier system. The scope included those factors affecting the Soldier System Architecture [Nakaza, Tack, and St. Croix, 2016] in the physiological, cognitive, and technological domains that impact mission effectiveness. SoSE was originally a 5-year project (2015–2020) with approximately $1.5M per year, although a number of adjustments were made as the project progressed. The SoSE project had five Work Breakdown Elements (WBEs). These included: A Soldier System Architecture (1.1), Performance Metrics (1.2), Injury Mitigation (1.3), Sensors and Power (1.4), and Decision Aids (1.5). Within WBE 1.5 there was a sub-WBE called 1.5.1 Display Methodology and Information Requirements with a budget of approximately $50K per annum.

1.1 Report Overview

The original focus of SoSE 1.5.1 was on display methodology and soldier information requirements. Much of this interest was driven by the advent of battle management systems (BMSs) for dismounted soldiers. In 2016 and thereafter work continued on information presentation but there was greater emphasis on cognitive load, reflecting an increased concern about the potential for a BMS to increase soldier cognitive load.

This report is divided into several sections. This first introductory section (Section 1) describes the SoSE project in general terms, and concludes with a description of the BMS being adopted by the Canadian Army (CA), to set the context. The next section (Section 2) is called Information Presentation for Soldiers and includes multiple subsections. The first of these is a treatment of research examining blue force tracking and efforts to develop a symbology for soldiers. Then an assessment of a head-up display (HUD) funded through a related Defence Industrial Research (DIR) project will be considered. Although some of this work preceded SoSE it is clearly relevant to SoSE 1.5.1. An opportunity to observe the use of BMSs by soldiers in the 2017 Bold Quest coalition exercise is also discussed, including an assessment of the BMSs being used by Canada and New Zealand. Then, an experimental field trial that assessed the utility of a BMS against a baseline condition (map and compass) is described, followed by a set of recommendations for a future urban team environment capability. Section 2 concludes with a potential future augmented reality (AR) display technology called Mirror in the Sky (MitS), along with some experimental assessments using a virtual reality (VR) simulation of MitS.

In more recent years the focus of SoSE 1.5.1 shifted to cognitive load measurement. In Section 3 an introduction to the cognitive load construct is provided, along with a brief literature review. The set of current techniques for measuring cognitive load is also described. Under SoSE 1.5.1, one technique called the detection response task (DRT) has been explored. The laboratory experiments conducted with soldiers to validate the DRT—and preliminary results from a field experiment that used the DRT— are

CAN UNCLASSIFIED

2 DRDC-RDDC-2021-R083

CAN UNCLASSIFIED

considered. The results from an assessment of cognitive load using the DRT with MitS are also discussed. Section 4 of this report briefly reviews the international program that SoSE 1.5.1 has supported, and finally Section 5 summarizes the outcomes.

1.2 The Integrated Soldier System—Suite (ISS-S)

With the advent of mobile information technology and networking, it is now possible to provide considerable computing power to the dismounted soldier. There is tremendous potential to this information technology, but also concern that it might overwhelm the soldier, especially in the operational context where the environment already places significant demands on human information processing.

Canadian soldiers have recently been equipped with the Integrated Soldier System—Suite (ISS-S) BMS, which allows dismounted soldiers to communicate via multiple text, radio, and data channels. ISS-S also displays geospatial information in real time, including the locations of individual friendly soldiers. As such, ISS-S has the potential to improve situation awareness (SA) [Endsley, 1995] for tactical decision makers. Alternatively, there is the potential that ISS-S could increase cognitive load and worsen SA, especially if the human-machine interface is designed poorly. For ISS-S to be effective, it should communicate the right information to soldiers at the right time and in the right way, and be easy to use. The same is true for BMSs more generally: allied nations are adopting BMSs with similar functionality, such as the United States (US) Nett Warrior system [Rosen and Walsh, 2011].

In the past several years, within SoSE and in earlier related projects, a number of research studies have been examining how best to present tactical information to soldiers. Although this work did not examine ISS-S per se, it is still relevant to the design of future iterations of ISS-S and BMSs more generally.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

2 Information Presentation for Soldiers

2.1 Blue Force Tracking (BFT)

BFT systems use the global positioning system (GPS) to track entities. In the past, such systems were used for tracking vehicle platforms (e.g., tanks, light-armoured vehicles). More recently, systems like ISS-S have incorporated BFT technology for individual soldiers, with each friendly soldier represented by a blue dot on a map. The position of blue force soldiers is obtained using a GPS emitter mounted on the ISS-S radio. Before ISS-S was fielded, under former project 14dk, Ho, Hollands, Tombu, Lamb, and Ueno [2013] simulated BFT for dismounted soldiers using VBS2 (Virtual Battle Space Version 2). Ho et al. [2013] used this simulation to investigate the effects of BFT on soldier performance in an experiment. In their experiment, 36 soldiers served as participants. Each participant role-played a section commander who had to locate and support another force under enemy contact. Participants performed the task in three different conditions: with a two-dimensional (2D) map having accurate BFT, a 2D map with inaccurate BFT, or a 2D map with no BFT. The inaccurate BFT was meant to simulate a situation where there was error in the GPS signal. To produce inaccurate BFT, the position of each blue dot on the map was shifted in a random direction and distance (up to 10 m) from the true position. Examples are shown in Figure 1. In the case with no BFT, participants were able to obtain necessary information from a simulated radio system. The results showed that when using BFT, soldiers were faster to engage the enemy. They also used their map more frequently compared to the No BFT condition. Surprisingly, the inaccurate BFT did not negatively affect performance.

Figure 1: An example of the blue force tracking (BFT) map used in [Ho et al. 2013].

In a second experiment [Ho, Tombu, Hollands, Ueno, Lamb, and Pavlovic, 2019], which was conducted under SoSE, different AR display methods (described in detail below) were simulated in VBS2 to portray the position information that BFT systems could provide. The participants were 36 soldiers who had experience and training in leading section-sized teams. The experimental scenarios required participants to role play a firebase commander. In each scenario, two assault teams flanked the enemy from either the

CAN UNCLASSIFIED


CAN UNCLASSIFIED

right or the left while the commander was located at a firebase whose mission was to suppress the enemy. This type of attack requires the firebase commander to have high situation awareness (SA) of each assault team position in order to initiate the attack properly and not accidentally fire upon friendly forces. During each attack, unexpected events could jeopardize the attack plan. The participants’ task was to maintain SA on the assault teams, the enemy, the environment, and the time. If an unexpected event occurred, they had to report it as quickly as possible. Last, they had to decide whether they should continue the attack or withdraw.

As in the first experiment, there was a BFT map and a no-BFT condition. However, the second experiment had three additional BFT conditions which used AR (augmented reality): AR-Ribbon, AR-Box, and AR-Dots. With the AR-Ribbon display, a horizontal strip was placed at the top of the display to show the azimuth (bearing) position of each friendly soldier (Figure 2 shows examples of each display type). With the AR-Box display, a rectangular box was drawn around each soldier. With the AR-Dots display a circular dot was placed above the head of each soldier. In each case, the simulation was meant to represent viable display options for a head-mounted AR-display system, with imagery superimposed into the forward field of view. For all BFT display types, there was a condition in which the GPS information was perfectly accurate, and a condition in which it was inaccurate (simulating GPS error). This produced a total of nine conditions. For the AR-Box and AR-Dots displays, the AR imagery showing soldier location and movement was drawn close to the soldier location, but for the BFT Map and AR-Ribbon displays, the imagery was farther from the soldier. As a result, when the BFT was inaccurate, it should have been more apparent in the Box and Dots conditions than in the Map and Ribbon conditions.

Figure 2: Examples of augmented reality simulations for blue force tracking (BFT) used by Ho et al. [2019]. From left to right: ribbon, box, and dot conditions.

The results of the second experiment showed that having BFT supported more accurate event detection and marginally faster event detection times than the no-BFT condition. Accurate BFT also resulted in faster attack decisions in some conditions. BFT accuracy did not affect the participants’ ability to accurately locate and map blue force positions, but they had more confidence in their ability with accurate BFT systems. Surprisingly, however, there was no reliable benefit for AR BFT over BFT on a map, and there were no differences among the AR display formats examined. Full results are described in Ho et al. [2019].

In summary, the two experiments suggest that BFT supports mission performance even when it is not perfectly accurate, and that AR systems appeared to have limited benefit relative to a map-based system (alternatively, one could argue that AR systems were not reliably worse than a map-based system), at least for the tasks examined.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

2.2 Soldier Symbology

There was a history of work examining soldier symbology which continued in the first few years of the SoSE Project. A Change Proposal (CP) was put forward through North Atlantic Treaty Organization (NATO) Joint Symbology Panel (JSP) to amend allied procedural publication (APP)-6 [NATO, 2011]. The goal of the CP was to expand military symbology to include symbols for representing individual soldiers (henceforth, the Soldier Symbol CP). Such symbology would be appropriate for use with BMSs for dismounts such as ISS-S. Directorate of Land Requirements (DLR) 5 was asked to provide input on the Soldier Symbol CP and sought DRDC’s expertise to provide a critical review. The Soldier Symbol CP was based on an earlier recommendation document [Tombu and Lamb, in press] prepared by DRDC for DLR 5, briefed to the US Symbology Standards Management Committee in 2014.

Several details of the Soldier Symbol CP were identified as being problematic [Tombu, 2016]. Specifically, the frame shapes proposed for soldier symbols were complex and overlapping, and in some cases conflicted with existing symbology. A modified soldier symbol set (resulting from work conducted by DRDC under SoSE and described in [Tombu, 2016]) was proposed to address these concerns. This modification used a unique frame shape for friendly entities, and a single frame shape for non-friendly identities. Echelon modifiers were used to indicate non-friendly identities, and changes were proposed to deal with leadership modifiers (e.g., place a line above the left side of the symbol to indicate deputy status). Further, the proposal argued for a restricted set of symbol modifiers. Modifiers that indicate broad military functions (e.g., infantry, medical) and rank codes may not be necessary, and can lead to confusion between unit and soldier symbols. Further, clutter is introduced by rank codes and may not be necessary. Further detail can be found in Tombu [2016]. A version of the modified symbol set proposed by DRDC was included in APP-6D, which was ratified and promulgated in October 2017.

2.3 Head-up Display (HUD)

Through the Defence Industrial Research (DIR) program, DRDC sponsored Revision Military Incorporated to develop a prototype see-through HUD. Revision provided DRDC with two prototype Desert Locust military goggle systems, each fitted with a Recon MOD Live HUD. The HUD was mounted inside the goggles using a small display just below the line of sight, and was controlled by a wrist-mounted control, as shown in Figure 3. The Recon HUD and associated software were designed for commercial skiing applications, although some functionality would be relevant to soldiers (e.g., entity tracking for BFT).

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Figure 3: Prototype Revision Desert Locust Goggle System fitted with a Recon head-up display (HUD) [Lamb and Hollands, 2019].

Using these prototypes, Lamb and Hollands [2019] conducted a heuristic analysis based on multiple workbench testing sessions and three days of field trials. The prototype was assessed against a standardized set of usability heuristics [Nielsen, 1993], as well as the HUD recommendations from the Soldier Information REQuirements (SIREQ) Technology Demonstration Program (TDP). SIREQ was an earlier DRDC research project that investigated the utility of HUDs for soldiers [Kumagai and Massel, 2005], among many other topics [Bossi, 2006]. The Nielsen heuristics are general “rules of thumb,” or guidelines, for software system design. The Nielsen [1993] heuristics indicated multiple problems with the Recon HUD, the most important of which are listed below:

Difficulty in determining battery charge status.

A requirement for an “event” to occur so as to review other data (e.g., speed and distance measures).

The lack of definition of some imagery (e.g., the meaning of a red circle surrounding the user location).

Two different screens for the same functionality (a navigation menu).

Difficulty in distinguishing among three remote control modes (which allowed different but partially overlapping functions).

Lack of error messages.

The comparison of the Recon HUD against the SIREQ HUD recommendations indicated further deficiencies. Most importantly, maps were shown on the Recon HUD, but it had insufficient resolution (428 x 240) to show maps clearly. Furthermore the display was only shown to the right eye (some users are left-eye dominant) and the display occluded the lower right FOV. The small display did not allow a look-through capability (i.e., the imagery was not transparent). Furthermore, although the HUD had the capability to indicate heading, there was considerable lag and it did not always update accurately. The HUD system did have a “buddy tracking” feature similar to a BFT capability. Initial versions of the HUD prototype did not offer an egocentric (track-up) perspective as recommended by SIREQ (although later versions did). Lamb and Hollands [2019] provide a summary of the 23 recommendations for the HUD from a soldier systems perspective.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

2.4 Bold Quest

Bold Quest (BQ) 17.2 is a recurring, large-scale coalition technology demonstration and assessment activity. In October 2017, BQ 17.2 was held at Fort Stewart, Georgia, USA. Multiple DRDC personnel were involved in the “friendly force tracker” thread, which was one of many activities within the broader exercise. The analyst team also included scientists from DST Group (Defence Science and Technology Group, Australia), DTA (Defence Technology Agency, New Zealand), Dstl (Defence Science and Technology Laboratory, United Kingdom), and the US Army Research Laboratory. The DRDC team observed battle management systems (ISS-S and SitaWare) being used in realistic military exercise in a coalition context. The team noted shortcomings and captured the value as perceived by users. A detailed summary is provided in [Banko, 2018].

For the friendly force tracker thread a coalition CANZUS Coy (Canada–New Zealand–United States Company) was struck. This included a New Zealand (NZ) Platoon, a Canadian (CAN) Platoon, and a US Marine Corps (USMC) Squad. The entire Coy (92 soldiers) was equipped with Harris digital radio systems. The CANZUS Coy conducted five missions: Advance to Contact (day and night), linear advance, parallel advance, and a two-day advance.

The New Zealand (NZ) Platoon used the SitaWare BMS (Systematic A/S, Denmark) and the Canadian Platoon used the ISS-S BMS. Both BMSs had a radio for voice communication, a communications headset, a GPS to aid with navigation, and a mobile personal digital assistant (PDA). The USMC Squad did not use a BMS, but their position location information was visible on the ISS-S and SitaWare systems. A near-peer opposing forces (OPFOR) Platoon was provided by the US Army 3rd Infantry Division. Exercise Control (EXCON) provided a Battle Group Headquarters. All soldiers were fitted with the multiple integrated laser engagement system (MILES) to determine whether a shot was a hit or not. Data on use of the BMSs and subjective preferences were collected from CAN and NZ soldiers using questionnaires and through field observations. For the latter, observers focused on four key questions:

1. When did soldiers use the BMS?

2. When did they not use it?

3. How did they use it? and

4. Were there issues?

2.4.1 Canadian Platoon ISS-S BMS

The Canadian Platoon (N = 29) was made up of 70% Regular and 30% Reserve Force personnel.

The Canadian soldiers received 2.5 days of training on ISS-S prior to the exercise [Banko, 2018]. Subjective measures on Canadian soldiers conducted prior to the missions indicated a generally positive disposition towards ISS-S. Reports from the soldiers after using ISS-S indicated that the small screen became quickly cluttered when control measures were drawn by hand. ISS-S was seen as useful for navigating, particularly at night (although light from the display was seen as problematic in night conditions, giving away location), and also for moving units out of line of sight. ISS-S was seen to improve information flow (e.g., situation and contact reports), but not for extensive orders. However, users also noted that ISS-S allowed orders to be easily distributed, and allowed changes during

CAN UNCLASSIFIED


CAN UNCLASSIFIED

operations. Text messaging was seen as an interesting feature. ISS-S was seen as most useful for Coy headquarters personnel and least useful for Riflemen. Riflemen were least satisfied with ISS-S in general, and preferred verbal communications. Soldiers suggested a number of additional capabilities, including pre-typed report words, the ability to see all text (to avoid scrolling off-screen or to the side), and being able to draw straight lines more easily. If implemented properly, such changes could potentially decrease cognitive load. Despite the negative comments, the ISS-S BMS received quite positive ratings overall (M = 6.62 on a 7-point scale). Further details can be found in Banko [2018].

2.4.2 NZ Platoon—SitaWare

For the NZ Platoon using SitaWare, there were considerably more problems. There were different versions of SitaWare: the Frontline version was loaded on Panasonic Toughpad for the Platoon Commander, Sergeant, and the Signallers, whereas the Edge version used by the riflemen was loaded onto a commercial off-the-shelf (COTS) smartphone. In response to items like “BMS improved my SA,” the CAN subjective ratings were more positive than the NZ results. Furthermore, items designed to assess cognitive load associated with the BMS showed higher scores (greater cognitive load) for NZ than for Canada (“BMS distracted me from doing my job” and “BMS preoccupied me” [Banko, 2018]). In 2017, SitaWare was a prototype system in development, and the COTS devices were likely not suitable for field use (e.g., problems with moisture on touchscreen), which may have been a factor in producing the results obtained.

From personal observation over two missions with the NZ Platoon the following problems were noted:

Observations from Mission 1 (Advance to Contact)

o Rain/moisture: Rain on the SitaWare screen caused the cursor to jump around and sometimes caused the system to reboot (blue screen).

o Awkward cabling: The cabling around SitaWare stuck out in front of the device (see Figure 4), and was not rugged enough for field use.

o Disuse: In cases where SitaWare should have been used it was not (instead, pine cones were used to show a tactical plan, a personal cellphone used to track position). There was very little use of SitaWare after H Hour.

o Noise in audio: There were problems with the in-ear audio (e.g., feedback, humming, buzzing in ears).

o Batteries: The batteries drained too quickly.

o Blue on blue: There was at least one case of a friendly misidentified as hostile.

Observations from Mission 2 (Linear Advance)

o Greater Use: Soldiers used SitaWare much more frequently than during Mission 1.

o Frustration: There was frustration with SitaWare but soldiers still tried to use it.

o Crashing systems: Systems crashed on several occasions.

o Stale information: The current position was not always updated. As a result, the Platoon Commander said that he thought that a road was further away than it actually was (a vehicle driving by alerted him to actual position).

CAN UNCLASSIFIED


CAN UNCLASSIFIED

o Checking the system: A considerable amount of time and communication among team members was spent figuring out if others were getting messages or updates on SitaWare.

o Brittle input format: In one case, the Pl 2IC needed to prepare a 9-liner, but could not send it because the system rejected his input.

o Batteries: The batteries drained too quickly.

Figure 4: The SitaWare battle management system being used by New Zealand Army troops at Bold Quest 2017. Note cabling arrangement. Photo by author, Fort Stewart, Georgia, US.

2.4.3 US Marine Corps Squad

The USMC Squad was unable to act seamlessly with rest of Coy and had to use a Liaison Officer frequently. There were delays in timing for the Squad, due to the inability to relay real-time changes to plans. This may have increased cognitive load and reduced SA (due to increased uncertainty) relative to a mission with partners not using BMS technology, although this was not measured formally [Banko, 2018].

2.4.4 Bold Quest Summary

In general the friendly force tracker thread showed that there were significant problems with interoperability given current BMS technology. Data were not easily shareable despite use of Harris radios by all.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Generally SitaWare produced higher ratings of cognitive load than did ISS-S, and produced lower subjective ratings of team SA [Banko, 2018]. Although there are many possible reasons for the discrepancies, it is likely that the difficulties with SitaWare noted above were factors. Canadian soldiers reported that they wanted more time to physically interact with ISS-S equipment, especially how to assemble the vest (problems arose with the equipment during missions and soldiers did not know how to fix the problems, increasing frustration). This result might imply that 2.5 days training with ISS-S might not be sufficient for soldiers to reach a level of proficiency with the system.

From a SoSE 1.5.1 perspective, participation in Bold Quest was valuable because it provided an opportunity to evaluate different BMS systems for soldiers in a realistic setting. There is value in participating in future BQ exercises in support of Canadian Army soldier systems. There is clear interest in methods for assessing cognitive load in the field: “Research should continue focusing on developing objective field measures of…cognitive load” [Banko, 2018, p. 23]. MILES data were not reported in Banko [2018], so it was not possible to relate subjective ratings to shooting performance, although this could potentially be done in future. It should be possible to construct and compare experimental conditions across missions in order to assess cognitive load more objectively than at BQ 17.2. However, given that Bold Quest is a large, multi-national joint exercise, there is a significant amount of coordination and logistics effort necessary. As noted above, BQ 17.2 was much bigger than the “friendly force tracker” thread—this was only one part and had to be fit within the broader Bold Quest construct.

2.5 Urban Team Experimentation

SoSE 1.5.1 supported an assessment of urban team experimentation requirements in FY16–17. Human Systems Incorporated was tasked to determine the best way forward for a future urban team experimentation (UTE) laboratory capability for DRDC – Toronto Research Centre [Tack, Morton, and Arbuthnot, 2018]. To this end, goals and objectives were reviewed and capability requirements for the UTE laboratory were defined. Candidate systems (software and hardware) were also evaluated against these requirements. Furthermore, a framework was defined to help in the development of suitable infantry scenarios to be employed in the UTE. Tack et al. [2018] noted that the VBS2 simulation environment (or its upgrade VBS3) currently in use at DRDC – Toronto Research Centre, covers off most of the requirements. However, it was also suggested that for very specialized requirements, certain game engine software, such as Unity would be more suitable. A scenario framework was described that could organize participants, confederates, and bots in a scenario formation [Tack et al., 2018]. Strategies were described to help choose the desired level of simulation fidelity for a soldier in a Section, a Section Commander in a Platoon, or a Platoon Commander in a Company formation. One pertinent conclusion was that the higher the level of command, the less there was need for highly realistic imagery in the simulation. Alternatively, highly realistic imagery is most necessary for lower levels (e.g., the rifleman).

2.6 Navigation Assist: Mirror in the Sky

The CA Directorate of Land Concepts and Designs has noted that “Commanders and soldiers lack sufficient timely situational awareness” [Bell, 2012, p. 12], which contributes to an identified, persistent “Hard Problem” [Bell, 2012]. One method for improving soldier SA would be to make tactically relevant information available using AR technology, as discussed earlier in Section 2.1, in the context of BFT. Typical AR uses a head-mounted or head-up display, allowing the user to see through the display surface to the real world behind. AR has potential utility for navigation. For example, in the urban terrain context,

CAN UNCLASSIFIED


CAN UNCLASSIFIED

AR could superimpose imagery for a navigation route on the forward field of view (FFOV). As such, the AR imagery is typically aligned with the user’s FFOV (that is, AR is “track up”). Like a track-up map (and unlike a north-up map) such displays are effective for tasks requiring the comparison of real-world objects to those depicted in the display [Yeh, Merlo, Wickens, and Brandenburg, 2003]. One problem for AR is that the virtual objects can occlude the FFOV. In order to maintain SA, the soldier needs to observe potential threats (and friendly soldiers) in any direction, without obstruction.

2.6.1 Mirror in the Sky (MitS)

There is potential to improve urban terrain navigation through the use of innovative AR technology. One such technology, called Mirror in the Sky (MitS), or alternatively SkyMap, has been proposed by, a Toronto-based software development company called Uncharted Software [Kapler, King, and Segura, 2019; Wright, Kapler, and Senior, 2013]. The MitS display technology depicts a simulated map surface high up in the visual field. The surface is like a mirror, reflecting the topographic information below, and is tilted towards the user. As such it does not occlude objects in the FFOV. Figure 5 shows an example. MitS is always “track-up”: that is, the topographic information on the map is aligned with the FFOV. Unlike a 2D track-up map, however, objects that are further away in MitS appear smaller due to linear perspective, just as in the real world. That is, there is a shared perspective between MitS and the FFOV. As a result it should be easier for the observer to compare or link common locations in the FFOV and MitS than with a north-up map, and perhaps a track-up map as well. However, some earlier studies conducted under SIREQ suggested that maps should not be used in HUDs at all, and thus AR maps were not generally recommended [Tack, Kumagai, and Bos, 2005].

Figure 5: An example of the simulated virtual reality version of Mirror in the Sky (MitS) used in Reiner et al. [2020].

Funds from SoSE were used to support a Department of National Defence (DND) / Natural Science and Engineering Research Council (NSERC) grant to the University of Toronto. DND served as a sponsor of the work, along with Uncharted. A virtual reality (VR) simulation of MitS was developed by Uncharted for testing and experimentation purposes. The VR simulation incorporates the properties of MitS but importantly also simulates the imagery for the FFOV. The MitS VR simulation was used for human

CAN UNCLASSIFIED


CAN UNCLASSIFIED

factors experimentation by the University of Toronto [e.g., Reiner, Hollands, Jamieson, and Boustila, 2020]. The VR simulation was also used to represent conventional map interfaces (track-up or north-up).

Using this simulation, several experiments have been conducted to investigate the effectiveness of MitS, which are summarized in Reiner [2020]. Most of the experiments involve a comparison of MitS to track-up or north-up displays using the VR simulation described above. Table 1 lists the various experiments and associated tasks and measures. Some of the studies used soldiers as participants. A pilot session for Study 2 was run with 21 regular force soldiers at The Technical Cooperation Program (TTCP) Contested Urban Environment (CUE) experiment held in Montreal in 2018 (CUE 2018). This helped in customizing the experimental protocol for soldiers.

Table 1: Summary of Mirror in the Sky Experiments (adapted from [Reiner, 2020]) See text for details.

Study Experimental

Tasks Loco

motion Participants MitS

2D map

Measures Results

(Advantage for)

1 Spatial

visualization, Recall

N 28 civ (half w/ VR exp.)

Offset Dome

Track-up

Accuracy, RT, MWL (for both

tasks)

Spatial Visualization: MitS (w/VR experience), Recall: Map

2

Route Planning,

Route Control, Recall

Y 24 mil Offset Dome

North-up

RDS, MWL, Collisions,

Recall

MWL: None, Other

measures: Map

3 Route Control Y 72 (36 civ,

36 mil) Offset Dome

Track-up,

North-up

CT, MWL, Collisions

Track-up Map

4 Route Control

Practice N

20 civ (half for each)

Offset Dome

Track-up

CT, MWL, Collisions

No difference

5 Learning N 42 civ (male,

right-handed)

Dome Track-

up JRD, Map Drawing

Lower initial error: MitS,

Learning: Map

6 Spatial

Visualization N 37 civ

Flat, 4 Domes (1 Offset)

Track-up

Accuracy, RT,

confidence

Flat, Equidistant, Conformal >

Map, Orthographic >

Offset

7 Wayfinding Y 36 civ Offset Dome

Track-up

Error, CT, access

Track-up Map

Civ = civilian, CT = Completion time, JRD = judgment of relative distance, Mil = military, MitS = Mirror in the Sky, MWL = mental workload, RT = response time, VR = virtual reality

CAN UNCLASSIFIED


CAN UNCLASSIFIED

2.6.2 MitS Summary

In general, the findings have shown mixed results for MitS [Reiner, 2020]. It would appear that MitS’ features (scaled presentation and linear perspective) are a strength for some tasks, and a liability for others. While MitS supported spatial visualization tasks (relating information between aid and environment) better than a 2D map, most wayfinding tasks did not show any MitS advantage, or MitS was worse than a 2D map. Spatial learning and recall were particularly ineffective with MitS. The specific implementation of MitS (whether it is flat or hemispheric) also appears to play a role in terms of the efficacy of MitS (Study 6 in Table 1).

However, MitS is as effective as a track-up map for many of the tasks examined, despite its lack of familiarity. Indeed, participants who had more experience with head-mounted virtual reality systems produced better performance with MitS than map (Study 1 in Table 1) [Reiner et al., 2020]. Furthermore, results from Study 4 (see Table 1) showed that with practice (three 30-minute sessions) MitS produced performance as good as that observed with a map. All the studies to date used MitS as a replacement for a conventional map; an alternative would be to use MitS “on demand,” or to provide it as an adjunct to conventional methods in situations where it might be more effective, a design alternative suggested by soldiers [Vasquez, Reiner, et al., 2020]. Work is continuing on this effort under the Human System Performance (HSP) project.

In many cases, the MitS studies listed in Table 1 also examined its effects on cognitive load; those results will be considered in the next section.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

3 Cognitive Load

Starting in early 2017 SoSE 1.5.1 shifted its focus to soldier cognitive load. As noted above, the introduction of BMSs like ISS-S to the dismounted soldier has led to a concern that soldiers might be cognitively overwhelmed. The operational context includes multiple stressors (e.g., noise) which might serve to degrade information processing [Hockey, 1997]. The BMS has the potential to increase cognitive load and worsen SA, especially if the human-machine interface is designed poorly. Reflecting these concerns, the Canadian Army Land Warfare Centre (CALWC) earlier identified cognitive overload as an Army Hard Problem [Bell, 2012].

Beyond ISS-S in particular, DLR needs guidance when choosing validated test methods and metrics to include in the statements of requirements for soldier systems. If such test methods are not yet developed, then they need to be explored so that levels of physical and psychological burden (including cognitive load) can be measured and compared to a validated requirement limit or criteria [Tack, 2018]. Furthermore, the Infantry School needs scientific support and guidance to determine how information is provided to the soldier. Choices about who gets information, how much and when information is provided, and whether soldiers can manage the “push and pull” of devices appropriately are necessary [Tack, 2018].

3.1 Focus Group Interviews

In response to the concern about soldier cognitive load noted above, preliminary front-end analysis work was conducted. Soldiers with operational experience were interviewed in a focus group and asked to recall situations where they had experienced high cognitive load [Bossi, 2016]. The focus group included three non-commissioned officers (1 Master Warrant Officer, 1 Warrant Officer, and 1 Sergeant). Soldiers were interviewed in a multi-hour session. They were asked to verbally describe situations (“vignettes”) where cognitive load was high (“Describe some specific occurrences when you experienced a state of cognitive overload”). The soldiers were asked to distinguish high cognitive load from the soldier’s experience of being “in the black” [Grossman, 2004], which is typically characterized as a high-stress situation (increased arousal, heart rate, release of epinephrine, sweating). Increased stress and arousal have been shown to decrease working memory capacity [Hockey, 1997; Qin et al., 2009; Young, Brookhuis, Wickens, and Hancock, 2015]. As such, stress affects cognitive load (but is not load per se).

The vignettes focused on the use of radios and failures in communications. One example focused on a Platoon Commander in a Company Advance-to-Contact, moving from dense forest to an open area. The Platoon Commander had to keep track of orders while on the move, and was receiving orders from multiple radio systems. Sometimes this situation is referred to as “helmet fire.” There was time pressure due to an imminent H-hour, and radio communications included a mixture of urgent messages and messages that were not of the highest priority [Bossi, 2016]. In another example, a Platoon was attacking insurgents in an urban village, and had difficulty maintaining SA of forward element locations. Radio communications were not reaching all parties, and Section Commanders had lost track of some subordinates. The Platoon was taking fire. The Platoon second-in-command had lost SA and had no clear idea of where the enemy were or how many there were. A third vignette involved a night attack where the Company Commander was overusing the radio and blocking others out, as a result a relatively inexperienced Platoon Commander did not have time to execute the mission. As discussed in the next

CAN UNCLASSIFIED


CAN UNCLASSIFIED

section, level of experience or expertise affects the ability to combine information sources into larger chunks, which affects working memory capacity and cognitive load [Miller, 1956].

Across the vignettes, the soldiers described four factors as influencing cognitive load:

Having too much information to deal with (information overload).

Having information in the wrong form or at the wrong time (information usability).

Lacking the information needed to make decisions or act (paucity of information).

“Going in the black” due to combat stress (combat stress).

From this a preliminary understanding of some situations where soldiers report high cognitive load was obtained. These situations featured a combination of events that required quick reactions or responses, significant time pressure, communication of messages among members who were not co-located, the need to filter information or seek it out, and the need to use that information for planning purposes. Moreover, the messages generally represented an unfolding situation and were auditory (speech over radio), and therefore the messages interfered with each other when they were close together in time. More specifically, the messages about the location of soldiers or enemy forces were often difficult to interpret and sometimes irrelevant.

3.2 Cognitive Load Literature Review

The scientific literature on cognitive load is extensive and has been studied in the laboratory and the field for at least 50 years [Kahneman, 1973]. There are a number of definitions and conceptions of cognitive load in this literature, going by a variety of names (including cognitive load, cognitive workload, mental resources, mental workload, cognitive burden, effort, task difficulty, and working memory capacity). The most prominent of these constructs is mental workload. Mental workload is thought to be multidimensional rather than unidimensional: for example, the National Aeronautical and Space Administration Task Load index (NASA-TLX) [Hart and Staveland, 1988] has six different subscales, including frustration and physical demand. In SoSE 1.5.1, given the interest in whether or not the introduction of information systems to soldiers would lead to cognitive overload, a definition was adopted that focused more specifically on soldier information processing. Thus, Hollands, Spivak, and Kramkowski [2019] defined cognitive load as the information processing demands imposed on a soldier when performing a task requiring the interpretation and comprehension of data.

There is a relatively consistent set of commonly used mental workload measures [Wickens, Hollands, Banbury, and Parasuraman, 2013], any of which could arguably be used or adapted to measure soldier cognitive load. We can place such measures into three basic categories: subjective measures, physiological measures, and performance-based measures. Each type has pros and cons associated with it. For example, subjective measures like the NASA-TLX [Hart and Staveland, 1988] are relatively easy to administer and non-intrusive (they do not interfere with task execution), but require the user to reflect back upon earlier activity, which means that one is assessing memory for workload rather than workload at the time that it is experienced. Physiological measures like heart-rate variability (HRV) and pupil dilation allow continuous measurement during task performance but require setup with sensors, are potentially intrusive, and can be influenced by general arousal factors [Backs, Lennerman, Wetzel, and Green, 2003; Porter, Troscianko, and Gilchrist, 2007]. Brain imaging measure such as functional magnetic resonance imaging (fMRI), electro-encephalography (EEG), and functional near-infrared

CAN UNCLASSIFIED


CAN UNCLASSIFIED

spectroscopy (fNIRS) are sensitive to workload manipulations, but require fairly extensive setup and produce relatively large data sets which require significant time to analyze [Ayaz et al., 2012]. For technical reasons the brain imaging methods have historically been restricted to stationary applications. However, this is changing and EEG and fNIRS are becoming lighter (approx. 350–500 g) and more portable [Hill, Bohill, Lewis, and Neider, 2013; Matthews et al., 2008]. Finally, performance measures typically involve the introduction of a secondary task. As a primary task becomes more or less difficult, performance on the secondary task is measured. Hence secondary task performance becomes a measure of “spare capacity”: if the primary task demands more processing resources, that leaves fewer resources available for the secondary task and its performance should suffer. While the secondary task is necessarily intrusive upon the performance of a primary task, as opposed to a subjective measure like the NASA-TLX, the secondary task method allows real-time workload measurement [Hendy, Liao, and Milgram, 1997].

Within SoSE 1.5.1, the focus was on performance measures of cognitive load. Given the interest of the CA in the potential for cognitive overload that new information technology like BMSs could impose, we were interested in measuring the spare capacity of the soldier using such systems as compared to traditional methods for accomplishing a mission. To examine this, the secondary task technique called the detection response task (DRT) was chosen. The DRT is simple: a signal is presented and the participant responds by pressing a button. As the difficulty of a primary task is increased, the participant becomes slower to respond to the DRT signal and the probability of missing a signal increases.

The DRT has been extensively used for the measurement of driver distraction [e.g., Strayer et al., 2015]; there is an international standard for its use [ISO, 2016] in that domain. Despite its success elsewhere, the DRT has not been applied to soldier tasks until recently. In Australia, scientists at DST Group [Price, Dummin, Cahill, and Al-Janabi, 2020; Price, Cahill, Dummin, and Al-Janabi, 2019] have investigated the utility of DRT in soldier tasks, both in the laboratory and in the field. In both studies they found that DRT performance was improved through the addition of AR imagery to a night vision system in a navigation task. The implication is that cognitive load was reduced with AR relative to without. However, the results did not correlate with subjective measures of workload (NASA-TLX). To our knowledge the field study [Price, Cahill, Dummin, and Al-Janabi, 2019] represents the first empirical use of DRT with soldiers in the field. The question remains, however, whether soldier performance with DRT can be validated as a cognitive load measure for soldiers through explicit manipulation of task difficulty and measurement of both DRT performance and subjective workload in realistic soldier tasks.

3.3 Detection Response Task (DRT) Experiments

Consider an unfolding scenario where tactical messages are presented to a soldier in sequence, and the soldier must interpret them (in order to prepare a situation report later). If the scenario unfolds rapidly, it is likely that the soldier’s cognitive load would be greater than if the events occur more slowly. This situation might be simulated in an experiment and cognitive load measured using a secondary task. The primary task would be monitoring and interpreting the tactical messages. The secondary task would be the DRT. If the message presentation rate is increased (thereby increasing primary task difficulty), DRT performance should degrade. This is the essential logic of the secondary task method. There should be less “spare capacity” to process the DRT signal with fast message presentation relative to slow. If DRT performance is sensitive to the task difficulty manipulation in the presentation of the tactical messages, then it would appear to offer a valid measure of soldier cognitive load.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Traditionally, soldiers communicate tactical information by voice, either directly, or over greater distances through radio. The ISS-S system allows the same information to be transmitted via text. In the first case (listening to speech), the information is perceived using the auditory modality; in the second (reading text), the visual modality is used. Given this new possibility offered by ISS-S, two questions arise. One is whether varying task difficulty would have the same effect on cognitive load across the different modalities. The second is whether soldiers would process text as well as speech, and whether the cognitive load would be generally greater or less with text. To answer these questions an unfolding tactical scenario was simulated and presented to twenty-four soldiers in a laboratory experiment [Hollands, Spivak, and Kramkowski, 2019]. The scenario was based on Platoon House Pearson [Tack and Nakaza, 2016], a Canadianized version of the Kamdesh attack on a US Army outpost in Eastern Afghanistan in 2008 [Tapper, 2012]. Each participant soldier was Warrant-qualified or judged to be at a commensurate level by a subject-matter expert. The participant acted as a platoon commander receiving a series of messages about the scenario from subordinates. Some message examples are found in Table 2. The participant’s task was to build a tactical picture of what is happening from the messages. The experiments had a 2 x 2 factorial design: The message presentation rate (abbreviated MPR hereafter) was slow or fast, and the messages were presented either visually (text) or auditorially (speech). We measured cognitive load using DRT performance, and using subjective ratings (NASA-TLX). We also assessed SA by asking participants to periodically construct a situation report (SITREP) as if to higher levels of command, following the Situation Awareness Global Assessment Technique (SAGAT) [Endsley, 2017]. The resulting reports were scored by a subject-matter expert for accuracy.

The ISO standard [International Organization for Standardization, 2016] was followed for the DRT. A red light emitting diode (LED) served as the DRT signal, and lit up at random intervals between 3 and 5 s. The LED remained on until a response was made or 1 s had elapsed. In order to ensure that participants took the DRT seriously, we asked them to treat it as a realistic military task appropriate to the situation (e.g., checking the status of a thermal sight). We also asked our participants to treat the DRT as of equal importance to the primary task. These were deviations from the ISO standard which were necessary to adapt the DRT to the soldiering context.

The results showed that the fast MPR produced greater cognitive load (as indicated by both DRT performance and NASA-TLX) relative to slow MPR (see Figure 6). Moreover, there was reduced cognitive load for spoken messages (auditory) relative to text (visual), again as indicated by DRT performance and NASA-TLX. This result provides an initial validation of the DRT as a measure of soldier cognitive load, for a number of reasons. First, the DRT should be sensitive to a manipulation of primary task difficulty (increasing demand on working memory resources), and it was. Second, it should also be sensitive to a manipulation of modal interference. According to multiple resource theory [Wickens, 2008], two concurrent tasks drawing upon the same attentional modality (both tasks demanding visual resources) should be performed less well than tasks drawing upon different modalities (one task auditory and one task visual). That is, when two tasks demand the same processing resources, cognitive load should be higher (limited capacity). A secondary task measure should be sensitive to this type of manipulation, and indeed the results showed this effect (lower cognitive load for auditory than visual presentation, with a visual DRT). Finally, the DRT should correlate with conventional workload measures, and it did. Both effects observed with the DRT—task difficulty and modality—were also seen with the NASA-TLX scores.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Figure 6: Results found in the cognitive load laboratory experiment described in Hollands, Spivak, and Kramkowski [2019].

The remarkable aspect of these results is that the cognitive load results perfectly predicted what happened with SA. When cognitive load was lower, SA was better. This was true when task difficulty was reduced (SA better with slow MPR) and when the tasks used different modalities (SA better with auditory messages). There was no interaction between task difficulty and modality manipulations either: the SA advantage for slow MPR occurred equally for both auditory and visual DRT conditions; the auditory message SA advantage occurred equally for both fast and slow MPRs.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Table 2: Examples of messages presented to participants for the detection response task (DRT) laboratory studies. Message number refers to the position of the message in the temporal sequence.

Message Number Message Content

6 1 1 Alpha. Receiving heavy machine gun fire from Technicals. They have

advanced to 400 metres from the East Wall. Over.

20 1 1 Bravo. AmmoCas: All 40mil expended, 3 boxes C9, 2 M72s, 40 x C7

mags. 1 x Rifleman wounded.

26 Receiving RPG fire!

30 1 1 Bravo. 30+ enemy soldiers are massing at the edge of the buffer zone

from the North Wall. They are staying behind cover. We are engaging.

Over.

An alternative explanation for the modality result (speech beats text) is that soldiers are more familiar with voice over radio than they are with text. That is, they are more experienced with auditory than visual message presentation. Thus, the advantage of auditory over visual could have been a result of multiple resources sharing the load (visual DRT with auditory message presentation, as noted above) or because auditory messages were more familiar and therefore easier to process.

To investigate this question, Spivak, Hollands, and Kramkowski [2019] replicated the experiment just described with an auditory signal (a beep) for the DRT. All other design aspects of this second experiment were identical to the first. As in the first experiment, a fast MPR led to poorer DRT performance and increased NASA-TLX scores relative to slow MPR, indicating greater cognitive load in the fast condition. This result indicated that the auditory DRT was also sensitive to task difficulty manipulations. Further, consistent with the first experiment, SA performance was degraded with fast MPR relative to slow.

However, the results of the second experiment were more complex when it came to the use of visual and auditory presentation of the messages. Auditory messages did increase cognitive load (slower responses to the auditory DRT) relative to visual messages, as predicted by multiple resource theory. However, auditory presentation also produced better SA performance than visual presentation. This counterintuitive result might be explained by the fact that speech has been shown to have obligatory access to working memory [Baddeley et al., 1984]. This obligatory access means that speech messages are automatically processed. The automatic processing of speech messages led to better SA performance, but the auditory DRT beeps end up being ignored more frequently (leading to worse DRT performance). This could explain the apparently contradictory results for SA and cognitive load as measured by auditory DRT.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Nonetheless, in combination the two experiments showed that the DRT is sensitive to task difficulty manipulations and correlates with subjective workload, as measured by the NASA-TLX. Furthermore, the effect of task difficulty on cognitive load was the same regardless of whether message presentation was auditory or visual. These results imply that the advantage for auditory message presentation observed in the first experiment was due to the use of the visual DRT. It is also independently sensitive to attentional interference (or modality) effects. This has implications for the use of the DRT in practice: to reduce such effects, it is better to use the auditory DRT when the task is primarily visual, and vice versa. More generally, it is best to keep the modality of the DRT distinct from the primary task. This will help to avoid placing additional cognitive load on the soldier when cognitive load is being assessed with the DRT. In summary, the results indicate that the DRT is sensitive and diagnostic as a cognitive load metric for soldiers.

3.4 Next Steps

The SoSE Project sought to investigate and validate a cognitive load measurement method for soldiers. Although the work under SoSE showed the validity of the the DRT for measuring soldier cognitive load in a laboratory context, the long-term objective is to develop a “DRT Soldier,” which can be used to assess workload in real-time in an operational setting. Although the NASA-TLX is commonly considered the gold standard when it comes to workload measurement, there are two problems with it. The first, as noted earlier, is that it is taken after the fact rather than measuring cognitive load in real time. The second is that it measures a multidimensional construct—mental workload—rather than cognitive load per se. In the SoSE experiments, we observed that the DRT behaves in a predictable way with respect to experimental manipulations, and with respect to the NASA-TLX. The DST Group studies with Australian soldiers showed that the DRT was sensitive to a manipulation of information portrayal (AR imagery present or absent) [Price, Cahill, et al., 2019], which provides an additional indication that it should work in a field setting with soldiers.

3.5 Soldier Performance and Cognitive Load Using ISS-S

An important question with the introduction of ISS-S to the CA is whether soldier performance using ISS-S is improved relative to baseline. In 2018 the SoSE project examined this question in a field study with soldiers from the Third Battalion of the Royal Canadian Regiment (3RCR) at Canadian Forces Base (CFB) Petawawa [Bossi et al., 2021]. Thirty soldiers performed individual tasks and section missions with ISS-S and in a baseline condition dubbed Task Force Afghanistan (TFA). TFA included the use of a map, compass, and the DAGR (Defence Advanced GPS Receiver). Tasks included route planning, navigation, messaging/reporting, mini-missions (emphasizing position awareness, BFT, and SA), and small unit missions. The time taken to navigate and the quality of the route chosen were measured. The results indicated that ISS-S provided a significant advantage for all the tasks relative to baseline, especially with respect to time, in many cases roughly halving the time required to perform the same task. Soldiers completed the NASA-TLX after the task or mission to report subjective workload, and the results indicated lower workload for the ISS-S compared to the baseline condition. However, real-time, objective measures of workload (or cognitive load more specifically) were not taken.

The next step was to adapt the DRT used in the laboratory studies described in the previous section to measure cognitive load in a field context. In 2019 a series of navigation tasks was run at CFB Valcartier with twenty-four soldiers from the Royal 22e Régiment (R22eR). (Although this field trial was funded by the follow-on Human System Performance (HSP) project, it served to demonstrate the cognitive load

CAN UNCLASSIFIED


CAN UNCLASSIFIED

analysis methodologies and metrics developed under SoSE and therefore is summarized here.) At the trial the DRT and measurement of heart rate variability (HRV) served as continuous measures of cognitive load. We found through initial testing that we had to make some adjustments to the DRT in order to use it in the field. In our laboratory studies we used visual and auditory DRT signals. Those studies indicated that the attentional resources used by the DRT should be different from that used by the primary task. For the navigation task in the field the primary modality would be visual, although auditory resources would sometimes also be required (such as listening to spoken instructions). Hence, the tactile version of the DRT was chosen [International Organization for Standardization, 2016]. We mounted the response button on the vest beside the ISS-S tactical user interface (TUI). A further change deliberately violated the ISO standard. In particular, the 3–5 s interval between signals was found to be too frequent and irritating to our pilot participants. Hence the interval was adjusted to 18–30 s for the 2019 field study. The initial results indicated that the DRT response times were correlated with subjective workload estimates (NASA TLX and the Modified Cooper-Harper scale [Wierwille and Casali, 1983]), with all measures showing reduced cognitive load for the ISS-S relative to the baseline condition. DRT accuracy scores were also greater with ISS-S relative to baseline, although the difference was not statistically significant. A scientific report summarizing this study and the results is in preparation [Tack et al., 2020].

Within SoSE we have shown the potential of the DRT as a validated measure of soldier cognitive load. As such it can potentially be used for systems acquisition, requirements definition, the development of TTPs (tactics, techniques, and procedures), or to be able to assess the cognitive load of different training methods. We continue to develop and validate DRT Soldier as a cognitive load metric within the HSP project.

3.6 Assessing Cognitive Load for MitS

The research examining performance measures with Mirror in the Sky (MitS) was discussed in Section 2.6. The purpose of the research program was to determine whether MitS was a useful display method for soldier navigation in urban terrain. Mental workload or cognitive load was also assessed in the MitS studies, using a variety of measures including the DRT, as well as HRV, and NASA-TLX (hence performance-based, physiological, and subjectives measures were all used). Across multiple studies, with varying tasks, cognitive load either tended to be higher with MitS than with conventional map formats (e.g., Study 3 in Table 1) or there was no difference between them. This could be due to a lack of familiarity with the format, or a result of deeper-seated problems. One such problem is that to show the self-marker in MitS means placing it directly above the user’s head, or alternatively stretching the virtual mirror to allow the self-marker to be more easily seen in the FFOV (self-marker offset). The stretching distorts the space and also means that when the user moves the head or body position MitS does not always move in a compatible way (e.g., rightward movement of the closer parts of the map when the user moves leftward). Solis [2019] directly compared a MitS dome projection having an offset against a flat MitS and found greater cognitive load in the former case. However, there are also problems with placing the self-marker directly above. A study that obtained subjective feedback from users showed that soldiers found “looking upˮ was more effortful [Vasquez, Reiner et al., 2020]. Moreover, soldiers found the track-up presentation used by MitS confusing and preferred North-up maps based on their training [Study 3 in Table 1; Vasquez, Reiner, Jamieson, and Hollands, 2020]. Nonetheless, soldiers also suggested that the cognitive load associated with MitS might be reduced by allowing the user to toggle MitS on and off “on demand.” In summary, the specific details of the MitS implementation appear to be important in determining the cognitive load associated with its use.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

4 International Activities Related to Information Presentation and Cognitive Load

There are several international activities in which Canada participates where cognitive load and information presentation are important elements. These include activities within NATO and the Technical Cooperation Panel (TTCP), as well as a trilateral agreement with the Netherlands and Sweden. These activities are described below.

4.1 NATO Human Factors and Medicine (HFM) Research Task Group (RTG) 319

Within the NATO STO (Science and Technology Organization),the HFM (Human Factors and Medicine) Panel recently formed a Research Task Group (RTG) on the topic of Measuring the Cognitive Load on the Soldier (NATO HFM RTG 319). Canada is the lead nation, and membership includes the US, Great Britain, Australia, as well as several European nations. For impact and exploitation, the RTG seeks to develop standardized metrics to provide guidance about cognitive load measurement for system acquisition, design, test and evaluation, and mission rehearsal across a range of scenarios under a three-year program of work.

The objectives of this RTG are to establish a NATO consensus on a set of strategies to measure soldier cognitive load and determine its effects on soldier performance. The topics covered include the following:

Definition of cognitive load (include overload)

Context: Identifying scenarios with high workload

Review of experimental and modelling work: Defence research and academic

Measurement methods

Selection criteria

Integration of measures as a system

Cognitive load measurement systems and system selection

4.2 TTCP Human Factors (HUM) JP1

The Technical Cooperation Panel (TTCP) is the science and technology cooperation organization for five nations (US, United Kingdom, Canada, Australia, and NZ). The HUM (HUMan Factors) Group within TTCP includes Joint Panel 1 (JP1), called Human System Performance—Land. Within TTCP HUM JP1 there are multiple key activities. There is a key activity focused on Information Portrayal, led by the US with active involvement from Australia and Canada. The research programs of these nations as related to soldier information portrayal have been defined and categorized according to a taxonomy. In particular, the US (Natick Soldier Center) is actively researching questions around design guidance for future soldier AR systems, including night vision system development and cueing for situation awareness and navigation, the feasibility of eye-tracking metrics for evaluating AR information displays, and using gaze-contingent displays in the AR context. Australia (DST Group) is examining night vision

CAN UNCLASSIFIED


CAN UNCLASSIFIED

technologies, the incorporation of AR into those systems, the measurement of cognitive load, and defining information requirements for night vision AR systems. These research interests are in obvious alignment with Canadian interests in cognitive load and situation awareness as defined within SoSE and now HSP. A TTCP Emerging Capability Card on Augmented Reality was developed by the Informational Portrayal key activity [Mahoney and Hollands, 2018].

Given the shared interests under the TTCP JP1 Information Portrayal key activity, collaborative research is being defined in the area of information portrayal with AR systems for day and night application. One of the planned collaborative activities was a field trial in AUS that investigated the effectiveness of AR elements within night vision systems for SA during night operations. The trial was conducted in October 2020.

4.3 CAN (Canada) / The Netherlands (NLD) / Sweden (SWE) Trilateral Project Arrangement (PA)

There is a trilateral Project Arrangement (PA) on “Integration of Physical and Cognitive Burden” between Canada, The Netherlands (NLD), and Sweden (SWE) [Binsch, Hollands, van Beurden, and Kwantes, 2017], which is scheduled to end January 2021 and just extended for another year). From a CAN perspective, this PA was set up specifically for the SoSE project. The objective of the PA is to facilitate and enhance knowledge exchange among the participating nations, in order to develop a methodology to minimize future physical, cognitive, and mental burden. It has four focus areas (FAs):

FA-1: Developing validated measures of cognitive burden

FA-2: Individual determinants of physical burden

FA-3: Field study applications

FA-4: Integration of physical and cognitive burden.

Although some of the trilateral PA falls outside the scope of SoSE 1.5.1, we focus here only on the 1.5.1 relevant activity. In 2018, a framework for considering the effects of physical and cognitive load in combination was jointly developed through the collaborative efforts of NLD and CAN. Three dimensions were defined to characterize different situations where cognitive and physical load interact. The first dimension is whether the situation describes an effect of a physical task on a cognitive measure, or a cognitive task on a physical measure. The second is whether the two tasks are coupled or independent (i.e., are they meaningfully related to each other). The third dimension is whether the physical and cognitive tasks are performed in parallel or in series. Current activities include populating each of the eight situation cells within the framework with operational vignettes to serve as illustrations of the situation, and also populating each cell with sample experiments that have been used to assess the situation. Also within the trilateral activity, in order to support the HSP project, CAN is looking at adapting the method developed by NLD for the simulation of AR imagery in a mixed reality environment using a treadmill to simulate soldier locomotion.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

5 Summary and Outcomes

This report has summarized the broad range of research activities conducted under SoSE 1.5.1. First, the studies that investigated information presentation methods for soldiers were described. This included multiple research efforts on various topics: blue force tracking, soldier symbology, head-mounted display, observational results from Bold Quest 17.2, an urban team experimentation facility plan, and a novel augmented reality method for navigation assist (MitS). Second, research on cognitive load was described, including focus group interviews, a brief literature review, experiments assessing the utility of the detection response task for soldiers, field studies using the detection response task, and an evaluation of cognitive load using MitS. Third, the international program relevant to SoSE 1.5.1 was described. The key outcomes of all the various activities under SoSE 1.5.1 are listed below:

Blue force tracking (BFT) supported more accurate event detection, marginally faster event detection times, and faster attack decisions even when it was not perfectly accurate.

Augmented reality (AR) systems for displaying BFT data appear to have no benefit (but also no cost) relative to a handheld map-based system for the hasty attack situation awareness tasks examined.

Symbology for soldier systems developed and modified by DRDC was included in APP-6D, which was ratified and promulgated in October 2017.

A heuristic evaluation of the Recon HUD developed by Revision Eyewear indicated that maps were not useful at low resolution, and that an egocentric (track-up) perspective was needed.

At Bold Quest 2017, subjective ratings for the ISS-S BMS were generally high, although a training time of 2.5 days was seen as limited.

An assessment of the Integrated Soldier System-Suite (ISS-S) and New Zealand SitaWare system at Bold Quest showed greater cognitive load associated with the SitaWare system, with a number of technical issues (e.g., stale information, brittle format, in-ear audio noise) leading to frustration and disuse.

Results from Bold Quest also showed significant interoperability problems between ISS-S and SitaWare BMSs despite the use of the same radio network.

An assessment of urban team experimentation requirements indicated that Virtual Battle Space 2 (VBS2) was a suitable simulation platform, although Unity might be preferred in some instances.

The urban team experimentation assessment indicated that highly realistic imagery is more important for the rifleman than the Platoon or Company Commander.

In studies comparing the AR system Mirror in the Sky (MitS) to map displays, most results showed that MitS was not as effective as more familiar map displays, although it can produce performance as good as conventional maps.

More experience with head-mounted virtual reality systems led to better performance with MitS. Thus, it appears that the amount of experience users had with head-mounted displays plays a role in the usefulness of AR. Although perhaps not suitable as a replacement for current 2D maps, MitS’ role as one of a set of multiple navigation aids deserves further investigation.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Focus groups with soldiers identified four factors that potentially influence cognitive load: information overload; information usability; paucity of information; and combat stress.

The soldier interviews also indicated some common characteristics of situations where cognitive load was high: the need for quick reaction, significant time pressure, message communication (typically about the location of friendly soldiers or enemy forces) among team members who were not co-located (often involving speech over radio), the need to interpret messages and filter out irrelevant information, the need to seek out further information, and the need to use information for planning.

Three general classes of cognitive load measures are available: subjective, physiological, and performance-based. All of these can be adapted for use with soldiers.

One performance-based measure is a secondary task technique called the detection response task (DRT). Under SoSE the DRT was recently validated for use by soldiers in the laboratory using an ISS-S-based message communication scenario, and successfully used in the field to compare soldier navigation with ISS-S against traditional navigation methods. The DRT has also been used in field studies with the Australian Army.

Results from 2018 and 2019 field studies in Canada indicate that ISS-S supports navigation performance better than traditional methods (map and compass) and does not increase cognitive load. Rather, cognitive load was found to be reduced for navigation tasks, as indicated by DRT and subjective methods.

Cognitive load was also assessed with MitS. Generally cognitive load was greater with MitS than with conventional maps, or there was no difference. Problems appear to be related to the placement of the self-marker and the need to look up for extended periods. Soldiers also suggested that the cognitive load associated with MitS might be reduced if it could be toggled on or off “on demand.”

Several international activities supported SoSE 1.5.1 and continue under HSP, allowing Canada to leverage international efforts. In particular the NATO S&T activity HFM RTG-319 is developing a set of standardized methods for measuring cognitive load.

In summary, SoSE 1.5.1 provided insight into factors affecting how information should be presented to soldiers and has developed viable methods for measuring soldier cognitive load. Furthermore these efforts have been linked to relevant international programs and continue under the HSP project.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

References

Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B., and Onaral, B., Optical brain monitoring for operator training and mental workload assessment, NeuroImage, 59, 36–47, 2012.

Backs, R. W., Lennerman, J. K., Wetzel, J. M. and Green, P., Cardiac measures of driver workload during simulated driving with and without visual occlusion, Human Factors, 45, 525–538, 2003.

Baddeley, A. D., Lewis, V., and Vallar, G., Exploring the articulatory loop, Quarterly Journal of Experimental Psychology A, 36, 233–52, 1984.

Banko, D. K., Assessing Dismounted Infantry Battle Management Systems: The Integrated Soldier System—Suite (ISS-S) and Sitaware Edge, Defence Research and Development Canada, Scientific Report, DRDC-RDDC-2018-R202, 2018.

Bell, R. W., Enduring army hard problems: Presentation to the Army S&T Board [slide deck], Canadian Army Land Warfare Centre, Directorate of Land Concepts and Designs, 2012.

Berkun, M. M., Performance decrement under psychological stress, Human Factors, 6, 21–30, 1964.

Binsch, O., Hollands, J. G., van Beurden, M., and Kwantes, P., Integration of physical and cognitive burden, Project Arrangement No. 2017-01 to CAN/NLD/SWE Cooperative Science and Technology Memorandum of Understanding (2003), Human Cognitive and Social Sciences Theme Group, 2017.

Bossi, L., The human factors aspects of dismounted soldier C4I: Lessons learned from SIREQ TD, Presentation slides, 2006.

Bossi, L., Focus group with Queen’s Own Rifle Soldiers re cognitive burden, unpublished meeting notes, 2016.

Bossi, L., Tack, D. W., Nakaza, E. T., Evans, J., Apostoli, C., and Hollands, J. G., Individual and team performance comparison of the Integrated Soldier System—Suite (ISS-S) with in-service baseline: A field study, Defence Research and Development Canada, Scientific Letter, DRDC-RDDC-2021-L026, 2021.

Chee, M. W. L., and Choo, W. C., Functional imaging of working memory after 24 Hr of total sleep deprivation, Journal of Neuroscience, 24(19), 4560–4567, 2004.

Endsley, M. R., Toward a theory of situation awareness in dynamic systems, Human Factors, 37, 32–64, 1995.

Endsley, M. R., Direct measurement of situation awareness: Validity and use of SAGAT, in M. R. Endsley, Situational Awareness, 129–156, 2017.

Grossman, D. A., On combat: The psychology and physiology of deadly conflict in war and in peace, Warrior Science Publications, Millstadt, IL, 2004.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Hart, S. G., and Staveland, L. E., Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology, North-Holland, Amsterdam, Vol. 52, 139–183), 1988.

Hendy, K. C., Liao, J., and Milgram, P., Combining time and intensity effects in assessing operator information-processing load, Human Factors, 39, 30–47, 1997.

Hill, A., Bohil, C., Lewis, J., and Neider, M. Prefrontal cortex activity during walking while multitasking: an fNIR study, In Proceedings of the Human Factors and Ergonomics Society – 57th Annual Meeting, Santa Monica, CA, 1224–1228, 2013.

Ho, G., Hollands, J. G., Tombu, M., Lamb, M., and Ueno, K., Blue force tracking: Effects of spatial error on soldier performance, Proceedings of the Human Factors and Ergonomics Society – 57th Annual Meeting, 182–186, Santa Monica, CA: Human Factors and Ergonomics Society, Defence R&D Canada – Toronto, Scientific Literature, DRDC Toronto SL 2012-092, 2013.

Ho, G., Tombu, M., Hollands, J. G., Ueno, K., Lamb, M., and Pavlovic, N., Blue force tracking for dismounted soldiers, Defence Research and Development Canada, Scientific Report, DRDC-RDDC-2018-R234, 2019.

Hockey, G. R. J., Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework, Biological Psychology, 45, 73–93, 1997.

Hollands, J. G., Spivak, T., and Kramkowski, E. W., Cognitive load and situation awareness for soldiers: Effects of message presentation rate and sensory modality, Human Factors, 61, 763-773, Defence Research and Development Canada, External Literature, DRDC-RDDC-2019-P141, 2019.

International Organization for Standardization, Road vehicles: Transport information and control systems—Detection response task (DRT) for assessing selective attention in driving (ISO 17488:2016) Geneva, ISO, 2016.

Kahneman, D., Attention and effort, Prentice-Hall, Englewood Cliffs, NJ, 1973.

Kapler, T., King, R., and Segura, D. “SkyMap:” World-Scale Immersive Spatial Display. In Proceedings of the 7th ACM Symposium on Spatial User Interaction (SUI 2019). Association for Computing Machinery, New Orleans, LA, 2019.

Kumagai, J. K., and Massel, L. J., Alternative visual displays in support of wayfinding, Defence R&D – Toronto, Contract Report, DRDC Toronto CR 2005-016, 2005.

Lamb, M., and Hollands, J. G., Usability evaluation of Desert Locust military goggle system with Recon MOD Live heads-up display: Preliminary heuristic analysis results, Defence Research and Development Canada, Reference Document, DRDC-RDDC-2018-D095, 2019.

Lan, T., Erdogmus, D., Adami, A., Mathan, S., and Pavel, M., Channel selection and feature projection for cognitive load estimation using ambulatory EEG, Computational Intelligence and Neuroscience, Article ID 74895, 2007.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Liu, Y., Ayaz, H., and Shewokis, P. A., Multisubject “learning” for mental workload classification using concurrent EEG, fNIRS, and physiological measures, Frontiers in Human Neuroscience, 11, 389, 2017.

Mahoney, C. R., and Hollands, J. G., Human machine interfaces: Augmented reality, TTCP Emerging Capability Card, The Technical Cooperation Panel HUM JP1 (Human System Performance – Land), 2018.

Matthews, R., Turner, P. J., McDonald, N. J., Ermolaev, K., McManus, T., Shelby, R. A., and Steindorf, M., Real time workload classification from an ambulatory wireless EEG system using hybrid EEG electrodes, 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Vancouver, BC, 5871–5875, 2008.

Miller, G. A., The magical number seven, plus or minus two: Some limits on our capacity for processing information, Psychological Review, 63, 81–97, 1956.

North Atlantic Treaty Organization, NATO Joint Military Symbology APP-6(C), (STANAG 2019, Edition 6). North Atlantic Treaty Organization, 2011.

Nakaza, E.T., Tack, D.W., and Ste-Croix, C., Canadian Soldier System Architecture (SSA) Demonstrator Development Phases I&II, Defence Research and Development Canada, Contract Report, C21-0219-04751 (under review).

Porter, G., Troscianko, T., and Gilchrist, I. D., Effort during visual search and counting: Insights from pupillometry, Quarterly Journal of Experimental Psychology, 60, 211–229, 2007.

Price, T., Dummin, R., Cahill, L., and Al-Janabi, S., Cognitive burden associated with information quantity and form: A simulated augmented reality study, DST-Group-TR-3678, Defence Science and Technology Group, Canberra, Australia, 2020.

Price, T., Cahill, L., Dummin, R., and Al-Janabi, S., The effect of a prototype augmented reality night visual & thermal helmet-mounted display on dismounted soldier cognitive burden and mission performance,Draft report, Defence Science and Technology Group Edinburgh Australia, 2019.

Qin, S, Hermans, E. J., van Marle, H. J., Luo, J., and Fernández, G., Acute psychological stress reduces working memory-related activity in the dorsolateral prefrontal cortex, Biological Psychiatry, 66, 25–32, 2009.

Reiner, A. J., Evaluation and summary of complex spatial information in augmented reality displays. Draft PhD Dissertation, University of Toronto, 2020.

Reiner, A. J., Hollands, J. G., Jamieson, G. A., and Boustila, S., A mirror in the sky: Assessment of an augmented reality method for depicting navigational information, Ergonomics, 63, 548–562, 2020, Defence Research and Development Canada, External Literature, DRDC-RDDC-2020-P070.

Rosen, J. L., and Walsh, J. W., The Nett Warrior system: A case study for the acquisition of soldier systems, Naval Postgraduate School Sponsored Report, Monterey, CA, NPS-AM-11-187, 2011.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Spivak, T., Hollands, J. G., and Kramkowski, E. W., Cognitive load and situation awareness for soldiers: Using an auditory detection response task, Proceedings of the Annual Meeting of the Human Factors and Ergonomics Society, pp. 272–276, Washington DC: Human Factors and Ergonomics Society, Defence Research and Development Canada, External Literature, DRDC-RDDC-2020-P014, 2019.

Strayer, D. L., Turrill, J., Cooper, J. M., Coleman, J. R., Medeiros-Ward, N., and Biondi, F., Assessing cognitive distraction in the automobile, Human Factors, 57, 1300–1324, 2015.

Tack, D., User needs analysis for burden mitigation support to army stakeholder organizations, Defence Research and Development Canada, Contract Report, DRDC-RDDC-2018-C045, 2018.

Tack, D. W., Kumagai, J. K., and Bos, J. C., Alternative methods for providing survey and route information during open country terrain navigation, Defence R&D – Toronto, Contract Report DRDC Toronto CR 2005-015, 2005.

Tack, D., Morton, A., and Arbuthnot, S., Human Systems Incorporated and CogSim Technologies, Urban team experimentation requirements, Defence Research and Development Canada, Contract Report DRDC-RDDC-2018-C125, 2018.

Tack, D., and Nakaza, E., Cognitive task analysis of target engagements in a near peer scenario, Defence Research and Development Canada, Contract Report, 2020.

Tapper, J., The outpost, Little-Brown, New York, 2012.

Tombu, M., A Critical Review of the Soldier Symbol Change Proposal, Defence Research and Development Canada, Scientific Letter, DRDC-RDDC-2016-L062, 2016.

Tombu, M., and Lamb, M., NATO standard AEP-2013 soldier symbology, NATO Allied Engineering Paper, North Atlantic Treaty Organization, (in press).

Vasquez, H. M., Hollands, J. G., Jamieson, G. A., and Agnew, M. J., A mirror in the sky: The effects of map format and user expertise on navigation performance and mental workload, Defence Research and Development Canada, External Literature, E21-0414-05093, (in press).

Vasquez, H. M., Reiner, A. J., Jamieson, G. A., and Hollands, J. G. A Mirror-in-the-Sky Navigation Aid: Summary of Qualitative Feedback from Soldier and Civilian Users, Presented at the Annual Meeting of the Human Factors and Ergonomics Society, (in press).

Wierwille, W. W., and Casali, J. G., A validated rating scale for global mental workload measurement applications. In Proceedings of the 27th Annual Meeting of the Human Factors Society, Human Factors Society, Santa Monica, CA, 1983.

Wickens, C. D., Multiple resources and mental workload, Human Factors, 50, 449–455, 2008.

Wickens, C. D., Hollands, J. G., Banbury, S., and Parasuraman, R., Engineering psychology and human performance (4th Ed.). Boston: Pearson, 2013.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

Williams, K., The soldier system effectiveness (SoSE) project—project overview, presentation slides, August 2016.

Wright, W., Kapler, T., and Senior, R., System and method for creating and display map projections related to real-time images, United States Patent 8436872, filed May 7, 2013.

Yeh, M., Merlo, J. L., Wickens, C. D., and Brandenburg, D. L., Head up versus head down: The costs of imprecision, unreliability, and visual clutter on cue effectiveness for display signaling, Human Factors, 45, 390–407, 2003.

Young, M. S., Brookhuis, K. A., Wickens, C. D., and Hancock, P. A., State of science: Mental workload in ergonomics, Ergonomics, 58, 1–17, 2015.

CAN UNCLASSIFIED


CAN UNCLASSIFIED

List of Symbols/Abbreviations/Acronyms/Initialisms

2D two dimensional

3RCR Third Battalion of the Royal Canadian Regiment

AC Armée canadienne

APP allied procedural publication

AR augmented reality

BFT blue force tracking

BMS battle management system

BQ Bold Quest

CA Canadian Army

CALWC Canadian Army Land Warfare Centre

CAN Canada

CANZUS Coy Canada–New-Zealand–United States Company

CFB Canadian Forces Base

COTS commercial off-the-shelf

CP Change Proposal

CUE Contested Urban Environment

DAGR Defence Advanced GPS Receiver

DIR Defence Industrial Research

DLR Directorate of Land Requirements

DND Department of National Defence

DRDC Defence Research and Development Canada

DRT detection response task

DST Defence Science and technology

Dstl Defence Science and Technology Laboratory

DTA Defence Technology Agency

EEG electro-encephalography

EEU expérimentation par une équipe urbaine

ERT élément de répartition du travail

ESSdt Efficacité du système du soldat

EXCOM Exercise Control

CAN UNCLASSIFIED


CAN UNCLASSIFIED

FA focus area

FFOV forward field of view

fMRI functional magnetic resonance imaging

fNIRS functional near-infrared spectroscopy

FSAR Future Small Arms Research

GPS global positioning system

ISS-S Integrated Soldier System—Suite

JP Joint Panel

JSP Joint Symbology Panel

HFM Human Factors and Medicine

HUD head-up display

HRV heart-rate variability

HSP Human System Performance

HUM Human Factors

LED light emitting diode

MILES multiple integrated laser engagement system

MiTS Mirror in the Sky

MPR message presentation

NATO North Atlantic Treaty Organization

NASA-TLX National Aeronautical and Space Administration Task Load index

NLD The Netherlands

NSERC Natural Science and Engineering Research Council

NZ New Zealand

OPFOR opposing force

PDA personal digital assistant

PRR Personal role radio

R22eR Royal 22e Régiment

RA réalité augmentée

RTG Research Task Group

S&T science and technology

SA situation awareness

SAGAT Situation Awareness Global Assessment Technique

CAN UNCLASSIFIED


CAN UNCLASSIFIED

S-ÉIS suite d’équipement intégré du soldat

SIREQ Soldier Information Requirements

SITREP situation report

SOSE Soldier System Effectiveness

SWE Sweden

TDP Technology Demonstration Program

TFA Task Force Afghanistan

TID tâche d’intervention de détection

TTCP The Technical Cooperation Program

TUI tactical user interface

US United States

USMC US Marine Corps

UTE urban team experimentation

VR virtual reality

VBS Virtual Battle Space

WBE Work Breakdown Element

CAN UNCLASSIFIED

CAN UNCLASSIFIED

DOCUMENT CONTROL DATA *Security markings for the title, authors, abstract and keywords must be entered when the document is sensitive

1. ORIGINATOR (Name and address of the organization preparing the document. A DRDC Centre sponsoring a contractor's report, or tasking agency, is entered in Section 8.)

DRDC – Toronto Research Centre Defence Research and Development Canada 1133 Sheppard Avenue West Toronto, Ontario M3K 2C9 Canada

2a. SECURITY MARKING (Overall security marking of the document including special supplemental markings if applicable.)

CAN UNCLASSIFIED

2b. CONTROLLED GOODS

NON-CONTROLLED GOODS DMC A

3. TITLE (The document title and sub-title as indicated on the title page.)

Soldier Information Presentation and Cognitive Load: Final Report on Soldier System Effectiveness Project Work Breakdown Element 1.5.1

4. AUTHORS (Last name, followed by initials – ranks, titles, etc., not to be used)

Hollands, J. G.

5. DATE OF PUBLICATION (Month and year of publication of document.)

June 2021

6a. NO. OF PAGES (Total pages, including Annexes, excluding DCD, covering and verso pages.)

39

6b. NO. OF REFS (Total references cited.)

58

7. DOCUMENT CATEGORY (e.g., Scientific Report, Contract Report, Scientific Letter.)

Scientific Report

8. SPONSORING CENTRE (The name and address of the department project office or laboratory sponsoring the research and development.)

DRDC – Toronto Research Centre Defence Research and Development Canada 1133 Sheppard Avenue West Toronto, Ontario M3K 2C9 Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable research and development project or grant number under which the document was written. Please specify whether project or grant.)

02ab - Soldier System Effectiveness (SoSE)

9b. CONTRACT NO. (If appropriate, the applicable number under which the document was written.)

10a. DRDC PUBLICATION NUMBER (The official document number by which the document is identified by the originating activity. This number must be unique to this document.)

DRDC-RDDC-2021-R083

10b. OTHER DOCUMENT NO(s). (Any other numbers which may be assigned this document either by the originator or by the sponsor.)

11a. FUTURE DISTRIBUTION WITHIN CANADA (Approval for further dissemination of the document. Security classification must also be considered.)

Public release

11b. FUTURE DISTRIBUTION OUTSIDE CANADA (Approval for further dissemination of the document. Security classification must also be considered.)

12. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Use semi-colon as a delimiter.)

Geospatial Information; Human Information Processing; Cognitive Load; Mental Workload; Situation Awareness; Dismounted Soldier; Dismounted Soldier System; Human/Soldier Performance; Networked Soldier; Network-Enabled Soldier; Soldier Burden (Physical & Cognitive); Display Technology; Augmented Reality; Virtual Reality

CAN UNCLASSIFIED

CAN UNCLASSIFIED

13. ABSTRACT (When available in the document, the French version of the abstract must be included here.)

The Soldier System Effectiveness (SoSE) project sought to improve soldier effectiveness by decreasing physical and cognitive burden on the soldier and increasing resilience, protection, and mobility in an integrated, human-centric soldier system. Information presentation for soldiers has become an important topic given the advent of portable battle management systems (BMSs), such as the Integrated Soldier System—Suite (ISS-S) being fielded by the Canadian Army (CA). The SoSE Decision Aids Work Breakdown Element (WBE) contained a sub-WBE 1.5.1 focused on Display Methodology and Information Requirements. Studies that investigated information presentation methods for soldiers were conducted under SoSE 1.5.1, and are summarized here. These include research efforts on blue force tracking, soldier symbology, head-mounted display, observations of soldiers using BMSs from Bold Quest 17.2, a future urban team experimentation facility, and a novel augmented reality method for navigation assist called Mirror in the Sky (or MitS). There is a concern within the CA that the introduction of BMSs to dismounted soldiers might increase soldier cognitive load and impair soldier performance. Hence, research on cognitive load was also conducted under SoSE 1.5.1. This research included focus group interviews, a scientific literature review, experiments seeking to validate the detection response task (DRT) as a measure of cognitive load for soldiers, field studies using the DRT, and finally a description of how cognitive load was measured in the evaluation of MitS. The international program relevant to SoSE 1.5.1 is also described. Key outcomes of SoSE 1.5.1 research activities are summarized. Three broad results are noted. First, soldier BMSs generally either improve performance and reduce cognitive load relative to conventional methods, or do not markedly impair performance or workload. Second, AR systems do not appear to improve human performance or reduce workload relative to map-based systems, although the technology is rapidly changing and more studies will be necessary as functionality improves. Third, the DRT offers a validated method for real-time measurement of soldier cognitive load in field trials or mission rehearsal. In summary, SoSE 1.5.1 provided insight into factors affecting how information should be presented to soldiers and developed viable methods for measuring soldier cognitive load.

Le projet sur l’Efficacité du système du soldat (ESSdt) visait à accroître l’efficacité des soldats en diminuant leur fardeau physique et cognitif et en augmentant leur résilience, leur protection et leur mobilité grâce à un équipement intégré du soldat axé sur la personne. La présentation de l’information à l’intention des soldats est devenue un sujet important avec l’avènement des systèmes portables de gestion du combat (BMS), tels que la suite d’équipement intégré du soldat (S-ÉIS) mise en service par l’Armée canadienne (AC). L’élément de répartition du travail (ERT) sur les aides à la décision de l’ESSdt comportait un sous-ERT 1.5.1 axé sur la méthodologie d’affichage et les besoins en information. On a ainsi mené, dans le cadre de l’ESSdt, des études portant sur les méthodes de présentation de l’information à l’intention des soldats. Celles-ci sont résumées dans le présent document. Elles comprennent des efforts de recherche sur le suivi des forces amies, la symbologie du soldat, le visiocasque, les données d’observation des soldats utilisant les BMS de l’exercice BoldQuest 17.2, la planification d’un futur laboratoire d’expérimentation par une équipe urbaine (EEU) et une nouvelle méthode de réalité augmentée (RA) d’aide à la navigation appelée « Mirror in the Sky » (ou MitS). L’AC craint toutefois que la mise en place de BMS pour les soldats débarqués n’augmente leur charge cognitive et, par conséquent, ne nuise à leur performance. On a donc mené des recherches sur la charge cognitive dans le cadre de l’ESSdt 1.5.1. Ces recherches comportaient des entrevues avec des groupes de discussion, une revue de la littérature scientifique, des expériences visant à valider la tâche d’intervention de détection (TID) comme mesure de la charge cognitive des soldats, des études sur le terrain en utilisant la TID, ainsi qu’une description de la façon dont la charge cognitive a été mesurée lors de l’évaluation de MitS. Une description du programme international relatif à l’ESSdt 1.5.1 figure également, de même qu’un résumé des principaux résultats des activités de recherche de l’ESSdt 1.5.1. Trois résultats généraux figurent dans le présent document. Premièrement, les BMS des soldats vont généralement améliorer la performance et diminuer la charge cognitive par rapport aux méthodes classiques, ou bien ne nuiront pas de façon marquée à la performance ni à la charge de travail. Deuxièmement, les systèmes de RA ne semblent pas améliorer la performance humaine ni réduire la charge de travail par rapport aux systèmes cartographiques. Toutefois, la technologie évolue rapidement et d’autres études seront nécessaires à mesure qu’on

CAN UNCLASSIFIED

CAN UNCLASSIFIED

améliorera la fonctionnalité. Troisièmement, la TID offre une méthode validée pour mesurer la charge cognitive des soldats lors d’essais sur le terrain ou d’exercices de mission. En résumé, l’ESSdt 1.5.1 a permis de mieux comprendre les facteurs qui influencent la façon dont l’information devrait être présentée aux soldats et d’élaborer des méthodes viables pour mesurer leur charge cognitive.

soldier information presentation and cognitive load

Documents