introduction - naperville park district repo… · web viewone attempt to solve the problem is a...

Augmented Cognition Mitigation StrategiesNext Generation Concepts

Summary Report

_________________________________

October 14, 2005

Kelly S. Hale, Sven Fuchs, Kay Stanney

Design Interactive, Inc.897 Kensington Gardens Court

Oviedo, Florida 32765

Joseph Juhnke

Tanagram Partners125 North Halsted, Suite 400

Chicago, IL 60661

Prepared ForCDR Dylan Schmorrow


Table Of Contents

1 Introduction..................................................................................................................3

2 Current Mitigation Strategies......................................................................................4

2.1 Context-Sensitive Help........................................................................................4

2.2 Cueing..................................................................................................................5

2.3 Decluttering (Reduce amount of info).................................................................7

2.4 Delegation............................................................................................................9

2.5 Modality Augmentation.....................................................................................10

2.6 Pacing/Scheduling (Reduce rate of info)...........................................................11

2.7 Sequencing.........................................................................................................13

2.8 Task Sharing......................................................................................................14

2.9 Transposition.....................................................................................................14

2.10 Summary............................................................................................................15

3 Conceptual Framework for Mitigation Strategy Selection........................................17

4 Innovative Mitigation Strategies................................................................................22

4.1 Innovative Strategies to Alleviate Bottlenecks..................................................22

4.2 Innovative Strategies to Enhance Situation Awareness (SA)............................26

5 Mitigation Strategies Summary.................................................................................28

6 Operational Example: Real-Time Mitigation Strategies for Tactical Action Officer29

6.1 Operational Scenario.........................................................................................29

7 Conclusions and Future Directions............................................................................33

8 References..................................................................................................................34

Design Interactive, Inc. 2


1 IntroductionDARPA’s Improving Warfighter Information Intake Under Stress (i.e., Augmented Cognition [AugCog]) program aims to extend the information management capacity of human-system dyads in operational environments through real-time system mitigation strategies driven by physiological sensors monitoring operator cognitive state. A general premise of the current AugCog effort is that human information processing (HIP) capabilities are fundamentally the weak link in the symbiotic relationship between humans and computers. Within human information processing there are several ‘bottlenecks’ or points of limited processing capacity, including sensory memory, working memory (WM), attention, and executive function. It has been suggested that through mitigation strategies, the costs of HIP bottlenecks (e.g., degraded human performance due to overload, underload, stress) can be overcome. As physiological sensors become more robust and capable of characterizing cognitive states beyond bottlenecks (e.g., situation awareness metrics), the focus of real-time mitigation may expand beyond alleviating overload to optimizing operator state (i.e., predictive mitigation to avoid overload) to measurably improve performance of the human-system dyad. The design of real-time mitigation strategies must thus be carefully addressed.

To date, system mitigation has been applied in a brute force manner and thus tends to cause context switching (Baldonado, Woodruff & Kuchinsky, 2000), as well as loss in situational awareness (Boiney, 2005); there has generally been an observed cost associated with getting context back. Thus, there appears to be a need to identify more effective mitigation strategies. This effort focused on summarizing current mitigation strategies used in human-systems integration (HSI) as well as techniques utilized in other media (e.g., Fine Arts, Photography, Film, Theatre) that may be adapted to enhance human-system performance within dynamic, information-rich, stressful environments. The goal was to identify theory-driven mitigation strategies from HSI, and leverage the Arts to achieve more effective and innovative, next-generation concepts for mitigation strategies, i.e., think outside the box that WIMPs (Windows, Icons, Menus, Pointing devices) have us bounded by. Through the current effort, a framework was developed to aid real-time mitigation strategy selection. In addition, the repertoire of mitigation strategies was expanded in terms of both the breadth of strategies available and the manner in which each is designed and implemented within varying operational settings. When the project commenced and the scientific literature review was underway, a critical gap in the literature associated with how mitigation strategies are conceptualized was uncovered. Thus in this effort we undertook filling this gap which involved devising a theoretical foundation from which to conceptualize mitigation strategies; specifically a conceptual framework based on Norman’s (1988) ”Seven Stages of Action” has been developed which relates adaptive mitigation activity to a model of the human action cycle of intent, execution and evaluation. The importance of this conceptual model is that it structures and systematically organizes the space of mitigation strategies. Thus the current effort focused more on building a theoretical foundation for mitigation strategies rather than the proposed empirical studies which will be pursued in the follow-on efforts.



2 Current Mitigation StrategiesThe current effort focused on identifying mitigation strategies that may be applied to (1) reduce overload/bottlenecks as identified via psycho-physiological sensors or (2) enhance situation awareness (SA). Bottlenecks that have been the focus of AugCog research thus far include sensory, attention, working memory and executive function bottlenecks. Future research is planned to develop psycho-physiological metrics for SA, as optimization of SA is critical within complex, information-rich environments such as military Command, Control, Communications, Computers, Intelligence, Surveillance and Recognizance (C4ISR) operations. This section describes mitigation strategies that have been developed and are currently used in various interactive systems. Each strategy described may be selectively implemented in real-time (following theoretical mitigation selection plans as outlined in this report) to either alleviate processing bottlenecks and/or enhance SA, thereby enhancing performance for an individual operator within a complex, information rich environment.

2.1 Context-Sensitive HelpBy changing the physical layout, useful affordances of both a physical and cognitive nature can be brought closer to where users need them and at the time they need them – providing context-sensitive help.

Strengths The simple fact that physical and cognitive affordances are available at the right moment can help users notice possibilities they might otherwise overlook (Kirsh, 2000). Context-sensitive help systems take away the task of locating a desired information snippet inside today’s extensive help systems. This is important, as we are apt to lose the thread of our composition (Kirsh was commenting on a word editing task) the deeper we have to go outside of our current environment of activity (Kirsh, 2000).

Weaknesses Automated help systems, however, can be perceived as helpful and non-intrusive (e.g., Microsoft® Word’s auto-correction feature) or highly annoying (“Clippy,” the Microsoft® Office Assistant). Careful consideration of interruption strategies is therefore needed. However, offering help may be generally inappropriate in high-load environments, as it causes distraction from the core task and may disturb the operator’s pace. If external resources are available and can be accurately identified, it may be better to delegate the task.

Specific Execution MethodsIn today’s computer systems, context-sensitive help is implemented in many different ways, mostly in close distance to where it is needed. The right-click context menu, which can be considered a quasi-standard for Windows® environments, adheres to the Gestalt principle of proximity and Fitts’ Law (describes the relationship between distance and size of a pointing target, Fitts, 1954), making the interface more efficient. Context-aware environments (e.g. Microsoft® Visual Basic®) automatically provide help for the



function at the current cursor position, e.g. code reference for a highlighted keyword, or they provide a list of available options (e.g. attributes in an HTML tag).

Software agents can automatically provide assistance in the background. They monitor and analyze user activity and trigger events if particular user action is detected.

2.2 CueingCueing is a common way to compensate for weaknesses in information rich environments, where people receive data at such a rapid rate that it may not be assimilated (Sheridan and Ferrell, 1974). In such cases, users develop and apply prioritizing strategies. However, these strategies are not always accurate. Limited attentional resources also make users vulnerable to the attentional spotlight effect (Posner, Snyder & Davidson, 1980; Townsend, 1971) which decreases their event awareness (signal detection) capabilities outside of their attentional focus, and may leave a high priority event undetected (“miss”). Cues provided by the system notify users of events by directing/redirection attention.

StrengthsResearch has been conducted on the design of cues, e.g. which modality provides the fastest response or what properties make a cue more distractive. Information can be interpreted with minimal cognitive effort if presented in the appropriate modality (Wickens, 2002), and response time to signals differs among modalities, with auditory and haptic cues resulting in faster response times compared to visual cues (ETSI, 2002). The more intrusive a cue is, the more awareness it will create for the cued event, and the more effective it will be in shifting attention. Arroyo, Selker and Stouffs (2002) found that the least used modalities in computer interfaces (i.e. smell and vibration) have bigger disruptive effects, probably because of their novelty.

Weaknesses Arroyo, et al. (2002) findings suggest that people’s backgrounds and worldview determine the effectiveness of various modalities for interruption. Also, if a cue is given at an inopportune moment, slower task performance (e.g., Czerwinski, Curtell & Horvitz, 2000; McFarlane, 1999), more errors (Kreifeldt and McCarthy, 1981), and worse decisions (Speier, Valacich & Vessey, 1999) may be observed. All of these factors may be critical to safety and mission success in C4ISR environments.

To mitigate the disruptive effects of interruption, researchers are investigating systems that reason about when (Horvitz and Apaciple, 2003; Horvitz, Jacobs & Hovel, 1999; Hudson, Fogarty, Atkenson, et al., 2003) and how to interrupt users. McFarlane (2002) evaluated four strategies for coordinating interruption; immediate (with no respect to the user’s current task), negotiated (people have choices about whether to allow interruptions and how and when to handle them), mediated (an attempt to predict people’s interruptibility), and scheduled (provide a degree of reliable expectation about when interruptions will occur). These approaches depend on some model of sensory attention when deciding how and when to interrupt. According to Norman’s (1988) task-



flow model, each task consists of a planning, evaluation and execution phase. Interrupting these phases may be costly (Zijlstra, Roe, Leonova & Krediet, 1999; Monk, Boehm-Davis & Trafton, 2002). Since every subtask is a task itself, high-level tasks (i.e., containing subtasks) have a recursive structure that imposes more cognitive load as more levels of subtasks are present. Therefore, task completion is desirable in order to release these cognitive resources. If this cannot be accomplished, Trafton, Altmann, Brock, and Mintz (2003) suggest a rehearsal strategy to help resume a task after an interruption. Zacks and colleagues conducted studies that provide insights into the ways in which tasks are decomposed hierarchically in the mind (Zacks, Braver, Sheridan, et.al., 2001; Zacks & Tversky, 2001; Zacks, Tversky and Iyer, 2001). These decompositions are linked to distinct patterns of brain activity. Based on this research, Adamczyk and Bailey (2004) suggest a task model that reveals the best timing for an interruption. They also suggest that, besides task status, an interruption manager should dynamically factor in notification relevance, i.e. apply strategies to prioritize subtasks.

Specific Execution MethodsCueing techniques are in use for all modalities. The appropriate modality for a cueing event may or may not be chosen to interfere with the sensory channels of the current task, based on priority assessment and task status. The level of intrusion should relate to the priority of the cued information, however, as cues may interrupt users in their current task causing loss of SA and disorientation when resuming from processing the cue. For example, in a computer application, an auditory signal does not intrude visual workspace but provides a cue that something happened in the background (see Bailey et al., 2000 for more on balancing information awareness and intrusion).

Visual cues may rely on the visual pop-out effect which is implemented by a difference in color, shape, or intensity. Intrusiveness of visual cues is also highly dependent on position: For high levels of intrusion (i.e. cueing of high-priority events), cues should be placed within foveal vision, low priority events may be cued by changes in the peripheral or ambient environment. Auditory cues vary by their nature. They come as generic sounds of different complexity (with more complex sounds being more disruptive), earcons (i.e., real-world sounds that carry a metaphorical meaning), or speech, and can vary in volume or pitch. In more advanced systems, audio output may also be spatialized in order to distinguish information channels or carry spatial information. Haptic cueing is commonly implemented through vibration devices, such as those implemented in pointing devices (e.g., haptic mouse) and cell phones (e.g., meeting setting that gives vibration cue for incoming call) serving as the most common examples. Haptic cues may vary in intensity and pattern, and can be used to provide spatial data (if more than one vibration source is present).

Sometimes, different modalities are combined to create a system of cues. One common application is alarm systems that increase the alarm level step by step, starting with a visual cue (e.g. a flashing light). If this cue is not responded to, the system increases its intrusiveness by adding auditory signals (bells and whistles). Advanced car alarm systems even address suspected intruders by speech before setting off a public alarm (e.g. entreprix SWAT II, http://entreprix.com/swat2.html).



2.3 Decluttering (Reduce amount of info)Clutter is created when screen real estate or channel capacity is occupied by numerous unrelated entities, or if these entities are not well-arranged. Decluttering refers to reducing the amount of information to be displayed. Systems may follow design rules like the Gestalt Principles (i.e., the sum of the whole is greater than its parts; Koffka, 1935) that, correctly applied, can declutter displays and these can facilitate the user’s interpretation of interface state by increasing interface consistency. Some examples are:

- The Gestalt Law of Proximity: Elements that are closer together will be perceived as a coherent object

- The Gestalt Law of Similarity: Elements that look similar will be perceived as part of the same form.

- The Gestalt Law of Good Continuation: Humans tend to continue contours whenever the elements of the pattern establish an implied direction.

- The Gestalt Law of Closure: Humans tend to enclose a space by completing a contour and ignoring gaps in the figure.

A second approach towards decluttering is level-of-detail manipulation, where layers of content may be hidden to simplify the current display to highlight critical information. Strategies include adaptive menus (where options that are rarely used by the operator are hidden), zoom (where users are able to zoom in on specific context area), fisheye views (where a very wide-angle lens that shows an area of interest quite large and with detail with the remainder of the graph/image successively smaller and in less detail; Sakar & Brown, 1992) and preview techniques (where an incomplete, yet representative subset of an environment’s content is presented).

Strengths Interface consistency is helpful to avoid clutter, as it increases learnability and reduces errors (Nielsen, 1989). Consistency should support human perception and cognitive processes such as visual scanning, learning, and remembering (Mahjan & Shneiderman, 1997). Once an interface is learned, cognitive load – necessary to locate and identify elements – is decreased. Consistent color schemes and positions also facilitate orientation and help to reduce clutter. However, as operators deal with increasing amounts of data, unified appearance caused by consistency efforts, may inhibit the richness of information (Gentner & Nielsen, 1996) and therefore reduce the number of orientation cues.

Level-of-detail manipulations can be implemented to take away non-relevant information or advanced features that may not be utilized on a regular basis. These strategies can be used to simplify displays for novice operators, where advanced features would distract from learning basic system features. Preview techniques reduce efforts of assessment and selection of data. Schweiger (2001) used link comments in hypertext structures (mouse-sensitive, popped up next to the mouse cursor) that contained a summary of one or two sentences about the content of the linked page. Similar previews enhanced knowledge acquisition and supported intentional and incidental learning (Cress and Knabel, 2003) by improving the link selection and reducing serendipity effects. Users with link comments



opened significant fewer pages than those without the added information. Link comments also reduced the frequency of backward navigation.

Weaknesses Zooming incorporates a trade-off between detail level and awareness of the global picture. One attempt to solve the problem is a second-view approach, where the global view is presented in a second display, highlighting the zoom area (e.g. Adobe Photoshop). But this solution is still non-optimal, as it requires additional screen real-estate and forces the user to switch between views and mentally integrate the information of both (Sakar & Brown, 1992). Non-linear magnification implemented with ‘fisheye’ lenses is used to achieve a balance between expansion and compression of the data, depending on the user’s focus point (Gutwin & Skopik, 2003). This non-linearity, however, may cause distortion in proportions of objects it is applied to which may lead to decreased speed and an increased need for accuracy in pointing tasks (Gutwin, 2002). When implementing preview techniques, careful decisions are needed about how to reduce the data, as taking away context also eliminates cues that may be helpful to the user.

Specific Execution MethodsToday’s interfaces use various approaches of visual level-of-detail manipulation. Adaptive menus are used in Microsoft® Office, where options that are rarely needed by the user are hidden. In graphic software, layers of content can be shown and hidden as appropriate to not obscure the elements that are currently worked on. Another classic approach to level-of-detail manipulation is zooming – where “zooming in” provides local detail while “zooming out” allows for a high-level overview. This technique is common in electronic maps, such as MapQuest™ (www.mapquest.com).

A fisheye view (Furnas, 1986) has been used to balance scope and detail of information presentation. Gutwin & Skopik (2003) compared a set of magnification techniques and stated that, although the fisheye lens performed best overall, some tasks may be better off with a ‘flat lens’ (maintains relative sizes and positions in a certain area; e.g. on maps, where distance and directions have a meaning) or a panning view (performed best on pointing tasks in high magnification/high accuracy environments; e.g., point-and-click tasks).

Preview techniques such as those implemented in Mini Player mode in Apple®’s iTunes® aim at reducing detail by limiting the amounts of data to be processed by presenting an incomplete, yet representative subset of an environment’s content. Only upon request a detailed view or extended functionality is displayed. Currently, level-of-detail manipulations are not implemented for auditory and haptic modalities. Although one could imagine systems that increase or decrease context based on situation (tone/earcon vs. spoken word vs. spoken sentence), to date most auditory content is statically implemented. The same is true for haptics, except for advanced gaming or force feedback devices.



2.4 DelegationDelegation involves offloading some aspect of current tasking to another operator. Mixed Initiative refers to tasks that are shared between an operator and an intelligent system, where the system takes complete control over certain aspects of a task. The term initiative implies an intelligent system that can actively adapt its status to environmental factors. This technique has been used extensively to decrease operator overload by dynamically appointing tasks to other, less taxed, users or systems.

Strengths Delegation utilizes the capabilities of operators that are better suited for this particular task or parts thereof and can be used when misinterpretation inhibits task performance. The real strength of the delegation is its dynamic workflow optimization schema. When an operator is overloaded, delegation to system or another operator may reduce bottlenecks by reducing the amount of tasking a single operator must complete.

Weaknesses Delegation should be applied with caution, particularly if the global goal is to increase situation awareness. By delegating tasking to outside resources, the current operator reduces his/her awareness of offloaded tasks. This may result in decreased awareness of current state. In addition, automated delegation is difficult as the delegating agent must not only accurately understand the workload of the primary (the person the task was originally sent to) and potential delegates, it must also know which of its potential delegates is best equipped to address this particular situation with the skill sets and situational awareness required. In addition, delegation eliminates the operator’s need for learning. If a task is taken away from an operator as soon as he/she does not perform optimally, he/she will never have the opportunity to improve individual performance on the given task.

Specific Execution MethodsManual delegation is a common practice in the human work place and would seem an intuitive addition for an information flow system. One example is the auto-pilot system in aviation that is turned on by the operator when no events are expected which would require human involvement. However, these systems are mostly passive: The operator initiates the system to take control of certain aspects of the task but is still in charge, as the system only maintains the current state – sometimes capable of limited adaptive actions supported by sensory technology. The user is still required to monitor and make adjustments as needed. Cruise control is another example for delegating parts of a task to a system.

Mixed Initiative is a specific form of delegation, where intelligent systems are assigned responsibility for a task or task component, are already common in current computing environments, car navigation serving as an example: The system detects the current position of the vehicle and adjusts its output accordingly, offloading the task of way-finding from the driver. Also, some car manufacturers have rain sensors which control the wipers and their speed as needed, thus taking away this responsibility from the driver completely.



2.5 Modality AugmentationModality Augmentation can occur in two forms: modality redundancy and modality substitution/switching. Modality redundancy may provide the same information in multiple modalities (e.g., flashing light and auditory beep as a warning), and/or may provide complimentary information (i.e., additional information; e.g., flashing light with auditory speech message stating the problem status) in a second modality. Modality substitution/switching replaces one sensory modality with another (e.g., change flashing light to auditory beep as warning signal).

Strengths According to Wicken’s (2002) Multiple Resource Theory (MRT), separate sensory and cognitive resources are available to process information from different modalities. Therefore, simultaneous processing of competing tasks can be supported by strategically allocating data streams to various multimodal sensory systems. If resources are depleted for one modality, there may still be cognitive capacity left if the information is presented through other sensory channels.

Weaknesses Caution is advised when it comes to modality substitution/switching, as researchers increasingly report evidence that knowledge is grounded in modality-specific systems (Barsalou, Simmons, Barbey & Wilson, 2003), and there is no direct empirical evidence for amodal symbols. Thus, the presentation modality is proposed to influence encoding which may cause association problems when attempting to display the same information in different modalities (as modality switching does) if previous knowledge or interpretation is required for the task. For example, if operators are presented with a visual indication of status, and later presented with an auditory indication of status for the same system, interpreting this second cue as status information may require more processing load due to switched modality. In addition, Pecher, Zeelenberg and Barsalou (2003) report a study supporting the hypothesis that perceptual simulation underlies conceptual processing and that switching from one modality to another during perceptual processing incurs a processing cost, implying effects on operator performance. Also, Arnell and Larson (2002) showed that, under particular circumstances, multimodal stimuli presentation is subject to the attentional blink effect, where the second of two successive stimuli is likely to be missed when falling into a timeframe of about 500ms after the first.

Specific Execution Methods: RedundancyMost existing modality augmentation approaches target the visual channel (e.g. HMDs, HUDs, Starner, Mann, Rhodes & Levine, 1997; Billinghurst, Kato & Poupyrev, 2001). For example, image and pattern recognition technologies facilitate surveillance tasks by identifying and highlighting areas of interest in a video stream, thus enhancing the visual channel with additional, system-generated content. Such approaches are currently implemented in jet fighter cockpits (target tracking), navigation and vision-aid systems for car drivers, and security camera systems. Furthermore, crossmodal systems have been developed (e.g., Lyons, Gandy & Starner, 2000) that enrich visual real-world



environment with artificial auditory information. Audio tour guides in museums may be seen as predecessors of these crossmodal Augmented Reality applications.

Specific Execution Methods: Switching/SubstitutionBell, Boye, Gustafson and Wiren (2000) proposed a system that had the ability to predict and prevent the occurrence of longer error sequences during interaction. E.g., a low confidence score from a speech recognizer or an error indication from another part of the system could be used as a signal to encourage a user to switch from dictating to writing. Also, system activity is currently used as indicator for switching: Microsoft® Outlook® 2003 provides a visual cue for incoming e-mail only if there is no keyboard or mouse activity, assuming that the user is busy with a visual task if engaged in these actions. Only an auditory cue is triggered if the system perceives the user is involved in a visual task (via keyboard/mouse activity).

2.6 Pacing/Scheduling (Reduce rate of info)Pacing or scheduling strategies must be applied to conform to the speed of incoming information streams. This enhances the concept of time beyond its common descriptive character: A functional role is added in which time is relevant as a work requirement or constraint, as information in a control decision, as the outcome of such a decision, or as a property of a task, interface or agent (Hildebrandt, Dix & Meyer, 2004). Scheduling strategies or timesharing skills are one important determinant of performance in multi-task situations (Wickens, 1992).

One pacing strategy is the use of checkpoints, whereby the user monitors elapsed time at predetermined points and compares it to an existing plan. Based on this comparison, the user may opt to speed up, slow down, or maintain the current pace. Another approach in sequential tasking may be to adjust speed of information output to a user’s individual comfort level or abilities. In a study of visual pacing cues, Mamykina, Mynatt and Terry (2001) found that providing a high-level task overview that allows for planning of resource allocation may be equally important as continuous pacing suggestions. They conclude that pacing systems should be intelligent enough to recover from a loss of pace and still provide the user with valuable information, perhaps suggesting recovery strategies. Sensing technology was also suggested to help detect situations potentially dangerous for maintaining a pace.

Goals for paced systems are to minimize cognitive demands for assessing a current pace, provide information on task progress, display ambient cues that can be quickly understood without incurring significant interruption from the current task or demanding more effort than the tasks they are designed to augment, and place knowledge in the world to flexibly support different strategies for managing the pace of a timed task (Mamykina, et al., 2001). According to Hildebrandt, et al. (2004), designers should consider time design strategies when time-related issues occur in a system, such as temporal validity of information, interruption scheduling, temporal reference systems synchronization, multi-tasking, the regularity, periodicity and interleavability of tasks. Particularly highlighted are computer systems that are used in safety-critical systems with hard real-time requirements, and environments where the sequential structure of tasks



becomes increasingly complex, e.g. multi-tasking and interleaving. Therefore, time-design is a highly relevant issue in C4ISR environments.

Strengths A properly executed pacing mitigation would, based on cognitive load, slow incoming information during peak times and speed up information during slower periods. It should have the end goal of getting the user back to a real time status as quickly as possible. This requires the system to understand the user’s cognitive state, task completion rate and incoming task rate.

Weaknesses Using checkpoints requires an undisturbed a priori plan for minimal cognitive demands of pacing (Mamykina, et al., 2001). Additionally, when people are faced with the need to pace themselves while simultaneously performing a cognitively demanding task, pacing is conceptualized as a secondary task that nonetheless requires cognitive resources (Casini & Macar, 1999). With increasing cognitive demands of the primary task, self-pacing abilities decrease as users’ time estimations become inaccurate. These phenomena arise from a competition between temporal and non-temporal processors for limited attention resources (Predebon, 1999; Casini & Macar, 1999; Zakay & Block, 1997). Therefore, people often place cues in their physical environment to support internal mental calculations (Kirsh, 1995), e.g. alarm clocks on the lectern when giving a presentation.

With many pacing systems, particularly for closely coupled systems (i.e. when user and system are highly integrated), users are paced by the machine (Meyer & Hildebrandt, 2002). For example, if System Response Times (SRT) are very short, users will try to keep up with the computer’s rapid work speed (Shneiderman, 1984). While it has been previously argued that an inverse relationship exists between system delay and user productivity (Teal & Rudnicky, 1992), a more recent memorization study with a browsing environment (Meyer & Hildebrandt, 2002) indicates that performance was best for intermediate SRT. It can be argued that short SRTs do not provide sufficient time for proper encoding of information, while long SRTs increase annoyance and external distraction.

Unforeseeable events may not allow humans to keep a predetermined pace in an event-driven scenario. Without any level of awareness, the system keeps on cueing, therefore creating more cognitive load. In such cases, continued notification of a permanently lost pace was found to increase the anxiety level of the user which negatively correlated with performance (Mamykina, et al., 2001).

Specific Execution MethodsCurrent operating systems are event-driven, i.e. they wait for an event and react to it. However, systems do not have active awareness of task status or interpretation capabilities. Paced by predetermined settings (statically implemented or based on user preferences), they solely rely on the user to pace task execution. Sometimes, cues (reminders) are offered if time constraints have to be met.



2.7 SequencingComplex systems often require execution of many tasks at the same time (multitasking). If all tasks present their data at the same time, sensory stores and working memory of the operator may be quickly overloaded (Wickens, 2002). One way to alleviate overload is to present tasks in sequences. Sequencing refers to creating a linear order in which tasks are presented to the user in multi-tasking environments. To avoid cognitive bottlenecks, information is presented one chunk at a time rather than simultaneously. However, in information-rich environments, it is common that multiple events requiring operator response occur at the same time. In this case, sequencing involves decomposition of tasks into smaller portions, and subsequent presentation of subtasks. After one subtask is completed, a subtask of another event may be presented.

Strengths Sequencing may be implemented to ensure critical information is responded to in a timely manner by re-ordering incoming information based on priority (i.e., high priority/critical information is presented before lower priority information). When sequencing is implemented, depiction of task timeline and current status has been found to facilitate operator judgment and performance in selecting appropriate response strategies (St John & Osga, 1999).

Weaknesses Switching tasks requires cognitive resources: Upon exiting a task we must store its state and upon entering another task context, we must recover its task state, i.e. the global task goal, as well as the current point of interruption, have to be kept in mind. If recovering is not successful, users may have to start over if returning cues are not provided (Donmez, Boyle & Lee, 2003). Hence, sequencing may help prevent attentional resources and sensory channels from overload, but its interruption and task-switching patterns can increase working memory load.

When attempting to implement automatic task switching, the trade-off between task priority and cognitive load imposed by task interruption needs to be considered. It is advised to finish tasks if possible to release the associated cognitive capacities, and to interrupt at opportune moments (i.e. between planning, execution and evaluation stages).

Place-keeping strategies assist the task-switching process by providing cues of where the interruption occurred – the cursor is the classic example for place-keeping: It permanently indicates the current position in a text. To some degree, the commonly used principles of location consistency and mapping may guide users in remembering and learning where interface elements are located. A clear or even predictable order of events (such as step-by-step organization of a task) may, as well, facilitate reorientation. Kirsh (2000) suggests that we may reduce the number of interruptions we encounter, or at least their disruptiveness, if we can accomplish our tasks in fewer “environments,” i.e. with less attention-shifting. This implies that task-switching (and therefore the need for place-keeping) should be kept at a minimum, which is supported by Miyata and Norman’s, (1986) notion about cognitive load in hierarchical tasks. These principles may also assist in creating an easier task model, i.e., a clear definition of a task and its subelements



which allows for a more economical use of cognitive resources and may facilitate opportune moments for cueing/interruptions (Adamczyk & Bailey, 2004).

Specific Execution MethodsOne main implication of today’s WIMPs interface paradigm is that the user can perform every task at any point of time by switching windows and applications, therefore creating one’s own sequence of actions, based on individual priority assessments and preferences.

2.8 Task SharingTask sharing distributes cognitive effort across a team of cooperating users distribute cognitive effort, thereby reducing individual stress (Kirsh, 2000).

Strengths Operators often turn to task sharing when information displays present conflicting or confusing information. In such cases, colleagues are brought into the task to help resolve ambiguities and/or clarify the situation for the current operator who is unclear of current status. Thus, operators are able to resolve conflict by utilizing external resources from team members. Unlike delegation, with task sharing the operator stays in the loop – only parts of a task are delegated to another user or the system.

Weaknesses When attempting to implement an automatic task-sharing tool, the restrictions identified for delegation systems apply as well: The tool would need to be informed about potential candidates to share the task with and their current cognitive state.

Specific Execution MethodsTask sharing can occur through pulling in external resources for assistance, e.g. asking colleagues for help or advice without giving the task completely out of hands (i.e. delegation). Cruise control is an example for task sharing with a system: The driver initiates the system to take control of certain aspects of the driving task but is still in charge, as the system only maintains the current state. The user needs to monitor and make adjustments as needed.

2.9 Transposition The human brain perceives and processes different types of information differently. Verbal, spatial, geometric, and musical are some examples of the different cognitive categories (Berz, 1995; Deutsch, 1970; Sulzen, 2001; Wickens, 1984). Studies have shown that while individuals have different limits for each of these cognitive abilities (Sulzen, 2001), all can perceive at least some multiple types of information simultaneously (Wickens, 1984). This provides designers the opportunity to increase the amount of information that can be processed by an individual by monitoring activity for each capability and routing information via untaxed capabilities. Transposition, the cognitive equivalent to Modality Switching, refers to switching the cognitive processing demands (e.g., from verbal to spatial) of a task while maintaining the same input modality. For example, an in-car navigation system may provide a visual map display outlining a given route while a driver is stationary. As the car starts to move (i.e., driver



is involved in highly spatial task of driving), the navigation system may switch to provide route knowledge via verbal information (i.e., text). Often, transposition and modality augmentation compliment one another, and may occur in concert to achieve desired results. In the car navigation system example, the best alternative while driving may be to present route directions via verbal speech (as opposed to text; this will offload both the visual sensory and spatial cognitive channel that are tapped during driving).

Strengths Transposition allows information to be presented in various formats that may enhance information throughput if the operator is overloaded in one processing area (e.g., spatial) but not the other (e.g., verbal). While the transposed information may not be presented in the optimal modality, the critical information is provided to the operator in a secondary form.

Weaknesses Designers must carefully consider how transposition affects cognitive load. Operators may experience a higher cognitive load when processing information presented in a non-optimal format. In addition, context may be altered during transposition that may impact understanding.

2.10 SummaryTable 1 provides a summary of current mitigation strategies organized alphabetically. For each identified strategy, specific execution methods (i.e., examples) are provided that outline how each is currently implemented to achieve two main goals: (1) alleviate overload and (2) enhance situation awareness. While some mitigation strategy implementations are consistent across these two global goals (e.g., cueing), others have specific applications for one goal only (e.g., context-sensitive help).

Table 1: Summary of Current Mitigation StrategiesMitigation Strategy Execution Methods to

Alleviate OverloadExecution Methods to

Enhance SAContext-sensitive Help - N/A - Provide additional context on

request (and in close proximity to current user focus).

- Intelligent agents that monitor user activity and offer context when appropriate

- Interpret system state and present available options

Cueing - Select intrusion level proportional to priority of the cued information. - Auditory cues vary by sound complexity, volume and pitch, and can be

categorized into sounds, earcons and speech.- Haptic cues are currently implemented through vibration.- Visual cues rely on visual popout, and vary in color, shape, intensity, and

position.- Modalities may be combined, e.g. crossmodal cueing: Auditory cue used to

indicate incoming visual stimuli (e.g., tone to indicate incoming email).Decluttering - Use consistency and Gestalt principles to facilitate interpretation.

- Manipulate level of detail by zooming or with preview techniques.



Table 1 (Con’t)Mitigation Strategy Execution Methods to

Alleviate OverloadExecution Methods to

Enhance SADelegation - Currently no automatic

implementations of delegation.- Manual delegation based on

subjective assessment of load and delegation targets.

o Delegate to other humano Delegate to intelligent

agent (mixed initiative)o Get external task support

(task sharing)

- N/A

Modality Augmentation - Redundant Information: Supporting the original perception with additional, identical information in either the same (e.g. HUD) or another channel (e.g., auditory pulsing tone added to visual blinking light warning)

- Complementary Information: Supporting original perception with additional information in either same (e.g., ??) or another channel (e.g. cueing incoming email with a letter symbol and a tone)

- Modality Switching/Substitution: Changing the sensory channel (i.e., presentation mode) to alleviate sensory bottlenecks.

Pacing/Scheduling - Pacing is not used in current systems; applications react to system or user events, or follow manually created a priori schedules.

- Automatic pacing requires dynamic adjustment to accommodate loss of pace due to unforeseeable events

Sequencing - Not used in current systems; contradictory to today’s interface paradigm. - User assesses system and task state to create an appropriate sequence.

Task Sharing - N/A - Pulling in external resource to assist or perform aspects of the task.

Transposition - Changing the cognitive processing content from verbal to spatial or vice versa to alleviate cognitive bottlenecks (e.g., change visual spatial map [showing route] to visual verbal text [written directions])



3 Conceptual Framework for Mitigation Strategy Selection Mitigation strategies to date have been applied in a brute force manner, and thus tend to cause context switching (Baldonado, et. al.,.2000), as well as loss in situational awareness (Boiney, 2005); there has generally been an observed cost associated with getting context back. A framework for real-time system mitigation is needed to provide guidance as to when and how current mitigation strategies should be implemented into a system in real-time to enhance operator performance.

Recent advances in cognitive science and neuroscience have led to development of various physiological and neurological approaches to measuring cognitive load in real-time (e.g., eye movements, pupil diameter, galvanic skin response, EEG, functional magnetic resonance imaging [fMRI]) that may be used to trigger mitigation strategies. While there are various methods available, “EEG is the only physiological signal that has been shown to reflect subtle shifts in alertness, attention and workload that can be identified and quantified on a second-by-second basis” (Berka, Levendowski, Ramsey, et al., 2005, p.90). Thus, EEG metrics have been developed that can accurately identify when an operator experiences a processing bottleneck (i.e., overload), whether that bottleneck be within sensory, attention, working memory or executive function processing. In the near future, EEG sensors that evaluate operator SA in real-time may be developed, which may also be used to trigger mitigation strategies. Thus, while global mitigation goals may differ, a conceptual framework that outlines a process flow is required to direct the order in which mitigation plans and strategies should be implemented.

Building from this knowledge, the conceptual framework for mitigation strategy selection developed under the current effort identifies how real-time psycho-physiological measures can be used to drive cognitive mitigation through interface adaptation. Once psycho-physiological measures indicate less-than-optimal performance (e.g., processing bottleneck, poor SA), a diagnosis agent can determine the character of the problem and human-system interaction can be modified to mitigate this specific performance gap. The outcome of this action should then be reflected in subsequent measurements of psycho-physiological indicators, thus creating a continuous, iterative process to improve human-system interaction. In order to further characterize this closed-loop system, it can be mapped to models developed for the human action cycle, such as Norman’s (1988) “Seven Stages of Action” that describe how people accomplish goals.



Figure 1: Norman’s “Seven Stages of Action” modified for cognitive mitigation.

Mitigation strategies are triggered in stages 1-3 of the model by perceiving the physiological or behavioral activity of the user through automated measures which are interpreted and evaluated by a mitigation managing application (if performance is on target, no mitigation is triggered and stages 1-3 are recursive). Mitigation covers steps four to six (see Figure 2) in the “Seven Stages” model: At step 4, a mitigation goal is formulated in correspondence to any problematic cognitive state detected in step 3. In many cases this mitigation objective will aim to reverse the problem (e.g. “avoid visual overload” if visual channel capacity is found to be exceeded). Step 5 comprises a collection of identified mitigation plans and strategies (Norman’s “intention to act”), each of which addresses one or more objectives, which then lead into a corresponding set of concrete adaptive actions (i.e. modifications of the interface, task reallocation, triggering of instructional strategy). As the adaptation is applied, updated cognitive state metrics become available. The system can thus evaluate the success of the mitigation and initiate further mitigating actions if problems persist.

In selecting a mitigation strategy, various constraints would be considered. These include system state constraints (e.g., what information is currently being presented across the system and how), environmental constraints (e.g., environmental noise), user state constraints (e.g., current processing load), and global user constraints (e.g., individual abilities/preferences).

1 Given a global goal (e.g., reducing operator processing bottlenecks [sensory, attention, working memory, executive function], enhancing SA), the proposed mitigation strategy framework includes five general plans of action (Figure 2). These plans should be considered in the order presented as potential mitigation strategies. If a solution is found that resolves the identified bottleneck while meeting all

Design Interactive, Inc.

1. Perceive psycho-

physiological measures

2. Interpret cognitive state

3. Evaluation/Problem detection

4. Define mitigation

objective to counteract

problem5. Select a mitigation strategy

6. Select interface

adaptation

7. Apply the interface

adaptation

If no Problem detected

18

Figure 2: Mitigation Selection Framework(Step 5 of Figure 1)


constraints, that mitigation strategy will be selected and the cycle will proceed to step 6 (Figure 1).

2 The mitigation strategy selection framework first examines whether all operator sensory and processing (i.e., working memory) resources are optimized. Using data from psycho-physiological sensors (from step 1 in Figure 1), the system can determine if all resources (i.e., sensory modalities, cognitive processing areas) are being used efficiently. If additional resource capacity is available, the mitigation selection system should augment or transpose some information that contributes to the identified bottleneck (most likely based on priority; i.e., highest priority information should be presented in optimal modality and/or through redundant cues; lower priority information may be presented in non-optimal modality). The next option in the mitigation selection framework is to use cueing and intelligent sequencing strategies to direct attention to relevant information.

3 If, given the current situation, cueing strategies are deemed ineffective (e.g., too much information is being displayed), the mitigation selection framework suggests optimizing communication requirements through declutter techniques and/or adding context-sensitive help.



4 The next mitigation plan optimizes the rate of information by implementing pacing/scheduling strategies (e.g., slow down or temporarily stop incoming non-critical chat messages).

5 Finally, if the above plans are deemed ineffective in achieving the global goal, the mitigation selection framework directs delegation of tasks to either an intelligent agent or additional operator or task sharing. Delegation and task sharing are seen as a last resort in meeting global goal of optimizing individual performance within the human-system dyad (i.e., all mitigation strategies to optimize individual performance with the current system have been considered prior to delegation).



Figure 3A: Specific Mitigation Selection Model (Step 4-5 of Figure 1) for Alleviating Bottlenecks



Figure 3B: Specific Mitigation Selection Model (Step 4-5 of Figure 1) for Increasing Situation Awareness



4 Innovative Mitigation Strategies Studies have shown the need for flexible interaction from one user to the next (Langley, 1999). Similarly, the successful future of mitigation strategies lies within the power of individual uniqueness; this is why information display strategies of tomorrow must be inquisitive, adaptive, configurable, and considerate.

To date, augmentation strategies implemented in real-time for optimized HSI have been limited. Further, display strategies for real-time mitigation that have been empirically validated are limited in number and creativity (Schmorrow, Stanney, Wilson, & Young, 2005; Dorneich, Whitlow, Ververs, et al., 2004). Through review of techniques and best practices from other media, innovative information display strategies for dynamic, complex, information rich systems may be developed that can be used to alleviate overload conditions, or existing mitigation strategies may be enhanced and optimized.

The Arts and communications were identified as likely sources for techniques that may be exploited for this purpose. For example, movies and theatre use a linear timeline, but often include multiple strands of action. These individual storylines are decomposed into scenes and then rearranged (Howley, 2004) to create variety and tension. The rapid scene transitions impose a need for continuous reorientation on the part of the audience in order to reconnect the pieces and build a global picture of the plot (Mancini & Buckingham, 2001). This same phenomenon may occur during task switching in sequenced environments. Having evolved beyond traditional desktop paradigms (e.g., WIMP interfaces), the gaming industry can also effectively change context in space or time without affecting one’s situation awareness (Billinghurst, et al., 2001; Szalavari, 1998).

This section lists innovative mitigation approaches and techniques that may be used to dynamically update information presentation schemas to achieve one of two global goals within an information rich, complex operational environment (e.g., military command and control). Section 4.1 addresses the goal of alleviating operator processing bottlenecks. Section 4.2 addresses the goal of enhancing SA. These approaches and techniques are taken from areas outside human-computer interaction such as theatre, movies, gaming, and mass media. After reviewing various strategies used in other media, each strategy was evaluated for its applicability for military command and control environments. These new execution strategies were then categorized into the mitigation selection framework (Figure 2), thus expanding mitigation execution options available under each mitigation strategy category.

4.1 Innovative Strategies to Alleviate BottlenecksThe framework for mitigation strategy selection suggests that all current mitigation strategies can be implemented to alleviate processing bottlenecks with the exceptions of context-sensitive help and task sharing. Context-sensitive help inherently involves adding information, thereby increasing cognitive demand. Task sharing does not necessarily reduce cognitive bottlenecks as operators are still required to be ‘in the loop’ and participate in tasking along with team members. The remaining mitigation strategies identified across the five mitigation plans (Figure 2) can be effective in meeting the



objective of decreasing operator overload. The challenge to designers is to identify innovative ways to mitigation system display in real-time that efficiently optimizes operator state and measurably improves performance. Table 2 lists innovative approaches and techniques for relevant mitigation strategies that may be applied within a complex C4ISR environment to alleviate bottlenecks and enhance HSI within complex, information rich environments. Innovative strategies for pacing and delegation were not found, and thus these strategies are not included in table 2.

Table 2: Innovative Approaches/Techniques for Real-Time Mitigation of BottlenecksMitigation Plan Mitigation

StrategyInnovative Approaches/Techniques

Optimize Individual Processing Capacity

Modality Augmentation

Aural-Spatial Representation

Based on echo and reverberations, people can ‘hear’ how big a room is. This way of encoding spatial data into auditory information may open new implementation strategies for multimodality.

Ambient Sound Background sounds added in post-production phase of a movie are perceived as incidental but in fact function to enhance the drama. Embedding information into the natural environment so that it is not noticed as such (e.g. one notices the weather and acts appropriately without actively attending to it) may allow delivery of information without distraction.

Transposition High-Key/Low-Key Lighting

Lighting style where the scene is evenly lit, suggests a familiar world containing few surprises or mysteries, whereas strongly contrasted areas of light and shadow create a sense of mystery. This principle may be transformed into dynamic color schemes that contain information about system status. A scheme of harmonic colors might indicate a stable operating environment, whereas strongly contrasted colors may indicate that special attention is needed.



Table 2 (Con’t)Mitigation Plan Mitigation


Direct Attention Cueing Environmental Cues

People place cues in their environment (e.g., sticky notes) to increases affordance of information or to serve as a reminder for tasks that will be pursued at a later time. This concept of letting the user “park” information where it is needed allows the operator to temporarily decrease cognitive load.

Induction of Physical Response

To date, haptic cues are usually transmitted by vibration devices. However, low-frequency sound produces a tactile sensation that may be used for cueing in certain environments. Adding more dimensions to a modality may reduce the bottleneck and increase information throughput

Galvanic-Vestibular Stimulation (GVS)

This technique simulates the sensation of movement by applying electromagnetic signals directly to the inner ear. GVS may open entirely new ways of cueing by exploiting a new channel.

Tilt Shot A tilted camera suggests a reaction to a scene or object, usually involving strangeness, imbalance, tension, or the unexpected. E.g., the camera can have an unusual angle after an accident, or shake during an earthquake. Shaking or tilting visual content may be used as ambient cue without distorting or manipulating the content.

Screen Flash A technique implemented in ego-shooter games to indicate hits, where the screen turns red for an instant, but fades to normal rapidly so no detail is lost. Since this cue appears in the foveal focus, it does not require eye gazing or attention shifting and is more likely to be noticed than a peripheral cue.

Music Style/Tempo

Music is often used to create atmosphere in a film. Similar to lighting, background music may be used as an ambient cueing tool. E.g., systems have been developed that indicate stock activities to brokers by changes in music style or tempo. Furthermore, music has been found to affect emotional state which may be utilized to affect stress levels or other parameters of operators.





Direct Attention Cueing Environmental Sound

Shift in constant tone draws attention. Experts can qualify sounds or changes of sound, e.g. mechanics may hear problems with the engine and even be able to diagnose based on these sounds, or computer experts may listen to the frequency of the cooling fan to detect a system problem.

Sequencing Transitions In cinematography, a Cut signifies a shorter time lapse or merely a scene change. The use of Out-/Inpoints (e.g. ending one shot and starting the next shot with the same element) makes the cut less abrupt. A Fade can be used to suggest a passage of time, a journey, or a new location. A Dissolve is used to suggest a special relationship between the scenes that dissolve into one another (a relationship closer than suggested by a fade). Transition techniques may be adapted for sequencing to alleviate the effects of task switching.

Ambient Lighting

Decreased ambient light focuses attention, e.g. theaters use lighting to focus attention and notify (cue) upcoming events. Sequencing may be implemented by fading out irrelevant content, therefore focusing the user’s attention to the current task.

Optimize Communication Requirements

Decluttering Learned Efficiency

If two people share a common level of knowledge, this reduces the amount of information needed to convey a message. The same applies to human-computer interaction, allowing the use of codes, abbreviations, shortcuts, etc., thus reducing the amount of information to be processed and therefore alleviating bottlenecks.

Pattern Monitoring

Looking at macro events (pattern recognition) may reduce the need to attend to details, thus reducing the amount of information to be processed and therefore alleviating bottlenecks.



4.2 Innovative Strategies to Enhance Situation Awareness (SA)The framework for mitigation strategy selection suggests that all current mitigation strategies can be implemented to enhance SA with the exception of delegation. This strategy involves offloading tasking to another operator/system, thereby limiting access to all available information and potentially limiting SA. The remaining mitigation strategies identified across the five mitigation plans (Figure 2) can be effective in meeting the objective of decreasing operator overload. The challenge to designers is to identify innovative ways to mitigation system display in real-time that efficiently optimizes operator SA and measurably improves performance. Table 3 lists innovative approaches and techniques for each identified mitigation strategy that may be applied within a complex C4ISR environment to enhance SA within complex, information rich environments. Innovative strategies for pacing and task sharing were not found, and thus these strategies are not included in Table 3.

Table 3: Innovative Approaches/Techniques for Real-Time Mitigation to Enhance SAMitigation Plan Mitigation


Optimize Individual Processing Capacity

Modality Augmentation

See Table 2

Transposition See Table 2Direct Attention Cueing Control of

Information Stream

People use fill words to maintain attention during an interruption of the information stream (“well”, “uhm”, …)

Validation Feedback

When communicating, people expect cues for transmission success (“uh huh”).

Sequencing Cutaway A shot briefly interrupting one action to provide a glimpse of another also taking place, and then returning to the first action.Can provide additional knowledge without permanently losing focus on the primary task.

Optimize Communication Requirements

Decluttering Establishing Shot

A long shot giving an overview of a scene so the audience is not confused about what is happening and where. Overviews/different perspectives may enhance Situation Awareness.

Context-Sensitive Help

Error Correction Even if signal is improperly transmitted, people can often “fill the gaps” with assumptions based on context.If the system can identify these signal gaps it could provide resolution options to the user.

Change-in-State Indicator

Animators use “onion-skin” technique to track changes from previous frames. Display that/how an object has been manipulated.

Transmission Failure Reconciliation

If a communication signal is not responded to as expected, the sender will verify proper transmission.





Optimize Communication Rate

Pacing Cross Cutting Rapid sequence of shots between two different locations, used to create tension. The sequence builds to a climax and ends with two things coming together.Applied to pacing, cross cutting can be used to create arousal if fatigue or underload is detected.

Slow/Fast Editing

Utilize cutting speed (i.e. shot length) to generate excitement and anticipation (e.g. chase sequence) with short shots, or appear calming and relaxing (e.g. love scenes) with long shots.Relevance for Mitigation: Could be applied to influence the user’s emotional state through sequencing and pacing.



5 Mitigation Strategies Summary Through review of current mitigation strategies, a conceptual framework outlining how mitigation plans and associated strategies may be implemented in an ordered fashion was developed as a first step in creating a real-time human-system closed loop system where physiological metrics can drive innovative, seamless, adaptable system displays. Building from the framework created in this report which outlines how mitigation strategies should be implemented to achieve one of two global goals (alleviate bottlenecks, enhance SA), current and innovative techniques have been described that may be considered based on system and operator state and constraints (i.e., information being presented at any given time, operator load across various channels). Our review of the Arts and other media resulted in various innovative approaches/techniques that may be applied to C4ISR environments to measurably improve operator performance. These approaches were categorized into current mitigation strategies from Figure 3, and may be implemented as alternative mitigation techniques.

To implement the conceptual mitigation selection framework presented here in real-time, each specific execution method must be assigned given IF-THEN constraints to allow automatic evaluation of effectiveness under specific conditions. These constraints must consider all factors that may influence applicability of the specific strategy which may include system state constraints (e.g., what information is currently being presented across the system and how), environment constraints (e.g., environmental noise), user state constraints (e.g., current processing load), and global user constraints (e.g., individual abilities/preferences). The next section provides an example highlighting how the mitigation selection framework would be implemented in an operationally relevant context.



6 Operational Example: Real-Time Mitigation Strategies for Tactical Action Officer

During high intensity operations, the Tactical Flag Command Center (TFCC) provides intelligence, communications and modifications to the operational plan for the Carrier Strike Group (CSG). The Battle Watch Captain (BWC) is present in the TFCC during high tempo; high alert times and is the center of the shared situational awareness. In times when the BWC is not present in the TFCC, the Force Tactical Action Officer (FTAO) assumes the responsibilities of the BWC; in fact, often the FTAO is the BWC. The BWC/FTAO is one of the primary roles within the TFCC and has direct authority from the (CSG) Commander in the decision making process. (Strausser, Kollmorgen & Juhnke, 2005). Thus, the FTAO’s tasks involve monitoring the ship’s area of responsibility for potential threats, and deciding upon an appropriate course of action should a potential threat be identified.

In a complex, stressful environment such as this, physiological measures applied to an FTAO may detect operator processing bottlenecks. Building from the framework developed in this report, a processing bottleneck would trigger the mitigation selection framework (Figure 2). Figure 4 demonstrates how this framework may be applied in an operational context by specifying specific IF-THEN conditions that lead to implementation of specific mitigation strategies.

6.1 Operational ScenarioThis scenario illustrates a “typical” day in the service of Flag Tactical Action Officer (FTAO), Phil Stevenson. References have been placed throughout the story to illustrate relevance to the IF-THEN mitigation selection model (Figure 4). While this scenario is focused on bottleneck resolution, there are also examples of SA improvement mitigations at work.

“Another day, another tussle.” Those words were becoming Phil Stevenson’s mantra lately. It seemed like this conflict was never going to end. Phil and his crew were ranked among the highest performing teams in the fleet; a feat Phil knew had much to do with his augie rig. Phil’s crew had won the “lottery” and been outfitted with some new, just being tested, gear about six months ago. He smirks when he recalls the joke he and his crew made about the hassle it was going to be. That’s all water under the bridge; he’s a convert now.

His watch begins like any other day. He reviews the last watch summary on his personal viewing system while he sips his morning coffee. A process that used to take several hours is compressed into a matter of 15 minutes using the multimodal presentation that is paced to optimize his current state of retention. Displaying the last watch’s progress through task based mission plan by crosscutting information and key decision points from the last watch gives Phil a high level view of occurrences and a complete understanding of mission objectives he has yet to achieve. He even chuckles when he recognizes one of Randy’s signature moves. Randy’s crew is right behind Phil’s on metrics, but it looks like Phil has to do a little catch-up work for him again.



1a) Modality Augmentation KEYVisual resources are not overloaded IMPROVED SA

Augment high priority visual communications with additional BOTTLENECK RESOLUTION

visual informationIf auditory saturation > Substitute auditory communication to visual If auditory saturation > Substitute auditory status representations to visual If auditory saturation > Substitute auditory cues to visual If haptic saturation > Substitute haptic communications to visualIf haptic saturation > Substitute haptic status representations to visualIf haptic saturation > Substitute haptic cues to visual

Auditory resources are not overloadedAugment high priority visual communications with additional auditory informationIf visual saturation > Substitute visual communication to auditoryIf visual saturation > Substitute visual status representations to auditoryIf visual saturation > Substitute visual cues to auditoryIf visual saturation > Substitute haptic communication to auditoryIf haptic saturation > Substitute haptic status representations to auditoryIf haptic saturation > Substitute haptic cues to auditory

Haptic resources are not overloadedAugment high priority visual communications with additional haptic informationIf visual saturation > Substitute visual status representations to hapticIf visual saturation > Substitute visual cues to hapticIf auditory saturation > Substitute auditory status representations to hapticIf auditory saturation > Substitute auditory cues to haptic

1b) TranspositionVerbal resources are not overloaded

Augment verbal resources with additional verbal detailIf spatial saturation > Substitute spatial communication to verbal If spatial saturation > Substitute spatial status representations to verbal If spatial saturation > Substitute spatial cues to verbal

Spatial resources are not overloadedAugment spatial resources with additional spatial detailIf verbal saturation > Substitute verbal communication to spatial If verbal saturation > Substitute verbal status representations to spatial If verbal saturation > Substitute verbal cues to spatial

2a) CueingCritical communications are not being acknowledged

If no response to critical status > Cue status in same modeIf no response to critical status > Cue status in on different least intrusive unsaturated modeIf no response to critical status > Cue status in on different next least intrusive unsaturated mode. Repeat until acknowledgment

2b) Intelligent SequencingExecutive function is overloaded

Present communications in order of priority (Defer lower priority communications)3a) Declutter

Executive function is overloadedReduce information density to minimal amount required to maintain decision-making (all modes)

4a) PacingExecutive function is overloaded

Reduce the rate all communications are presented5a) Mixed Initiative

Executive function is overloadedAutomate tasks to assist decision-making

5b) DelegateExecutive function is overloaded

Redirect communications to available (not overloaded) resources



Figure 4: Operationally Relevant Mitigation Strategy Selection Process“Let’s heat ‘em up team!” Phil announces over group comm. Projection screens are already displaying summaries of team activities and status [1a]. The task for today is to secure a sector that is heavily trafficked by commercial vessels. There’s a good chance that some of the commercial vessels are terrorist platforms hiding amongst the civilians in their typical cowardly manner.

Dynamically assigned their Areas of Interest (AOI)[1a] Phil’s track team begins scanning, and validating traffic. The 5-man team is strong except for Rudy; he’s still pretty green and isn’t adjusting to the predictive load models as well as the others. [2b] Phil notes that Randy’s AOI has already been reduced because of overload. [3a] He stores the view of Randy’s reduced environment into his memory log for later consideration.

Ironically it’s Rudy who finds the first “pheasant,” a carrier class freighter that has a questionable ownership history. “Immediately the rest of the team responds to Rudy’s find. That’s when things get interesting. Sam, one of Phil’s brightest exclaims, “Oh dear, those are dots I didn’t want to connect!” over group comm. A radio transmission comes in from the Nemesis but Phil doesn’t hear it because his attention is focused rightly on Sam’s concerns. “If she’s concerned, I’m concerned,” he thinks. He notes the radio transmission has been added to his text log for later review [2b]. It must not have been important. The computer cues [2a] Phil that it is transferring 3 of the lower priority radio channels to text logs [1a], converting a radio based status report to a graphic representation on his dashboard [1b], and redistributing the high priority channels in different places “in his head.” [1a] Phil really loves spatialized radio comms. They are so much easier to separate.

Before he can speak, the computer has already requested further details regarding Sam’s concern. [5a] Her analysis of traffic patterns, port summaries, and ship logs has shown that 4 of the maybe 200 ships in the AOI have been traveling the same route over the past 20 days. Traffic logs show their actual progress to be normal except for the same 40-mile diversion in the middle of ocean and then back on course. There were no records of any platforms or vessels in that location, so it would be easy to consider the diversion a course deviation to avoid an obstacle, except they all stopped at the same location for 6 hours and then continued back to their planned course. [3a] Phil notes the room has dimmed quite a bit; group cog-load must be high. [3a] A spot over Rudy flashes and catches Phil’s attention. [2a]“Rudy, what’s your status?” “I need auth for a boarding party sir” Rudy responds. The computer whispers in Phil’s head that there is a 10% probability of concern for Rudy’s track. “Rudy, let’s defer for a few minutes. I think we’re going to need our resources.” Phil points at the big screen. Confirmed “civie” vessels have been reduced to mere dots on the screen while the 4 ships Sam has flagged have their routes, and PIMs plotted. [3a] “We need to run these numbers people” Phil announces. The computer has already offloaded two of the vessels to James for analysis and is summarizing determinations on the big screen. [5a][5b] It becomes evident that these four ships, even though they are on different timetables will be converging on the fleet at the same time. The computer whispers statistics in Phil’s head and it becomes evident he needs to react now. They are an hour away from an attack.



Phil orders the weapons teams on alert and cues the rest of the crews on watch of his findings. Communications jump exponentially. The entire fleet is hot now. Comms. are coming in so fast Phil has to rely heavily on the computer’s automated prioritizing and response tools. [5a] At one point a commander sends Phil an updated mission plan and the computer generously offers to allow Phil to dictate his responses and queries. During a few points of very intense communication Phil notices the world seems to slow down. He knows it’s the computer pacing his comms.[4a] to help him concentrate. He looks forward to the cue that he is back on real-time. [3a] Time-late is not Phil’s favorite place. Sometime during the conflict the computer assumed weapon allocation and assignment and Lloyd a “heckovaguy” stationed in Ohio was brought online to assume portions of Phil’s load. [5b] Lloyd’s performance rating was exemplary. Phil knew he had the best help he could get.

Eventually the conflict resolved, no lives were lost and all four of the enemy vessels were captured. It was no surprise that their cargo included an assortment of “dirty” torpedoes. A mess had definitely been averted today.



7 Conclusions and Future DirectionsThis paper reviewed mitigation strategies that are currently implemented to enhance operator performance within traditional human computer interaction systems. Each of these strategies may be implemented within a complex, information rich environment such as military C4ISR environments to measurably improve a number of objectives, including alleviating human processing bottlenecks and enhancing situation awareness. Although this review uncovered a number of mitigation strategies that are available at present, there was no conceptual model to aid designers in determining which mitigation strategy is best under a given set of conditions.

To address this void, a conceptual mitigation selection framework was developed that outlines five mitigation selection plans (optimize information processing capacity, direct attention, optimize communication requirements, optimize communication rate, increase external resources) that may be considered in order to address various global mitigation goals, including alleviating operator processing bottlenecks and enhancing operator situation awareness. In addition, media, communication, and the Arts were embraced in order to identify innovative ways of mitigation that have yet been undiscovered. Although no innovative strategies were found, numerous techniques that may considerably enhance the way mitigation strategies are implemented were extracted and added to the mitigation selection framework. An example of how this conceptual framework may be applied in an operational setting was provided. In follow-on efforts, empirical studies will be conducted to validate the mitigation selection framework within an operational environment.

The conceptual model and associated mitigation strategies (current and innovative approaches to enhancing system design) outlined in this report provide a starting point for realizing the goal of real-time mitigation within complex C4ISR environments by offering a structured approach for strategy selection. Other considerations that need to be addressed in future work include the development of exit policies for each mitigation strategy (i.e., when is mitigation no longer required, e.g. when to return from down-paced presentation to regular speed), how to coordinate the mitigation cycles for different objectives (i.e. reduce overload and enhance situation awareness), and how to apply mitigation framework to team environments. In addition, the conceptual model outlined here may be expanded to mitigate various human state conditions beyond processing bottlenecks and situation awareness. If such conditions can be captured in real-time via psycho-physiological sensors, there may be an opportunity to enhance operator performance, e.g. maximize focus and increase motivation.



8 References

Adamczyk, P, Bailey B.P. (2004). If not now, when?: the effects of interruption at different moments within task execution, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 271 – 278.

Arnell, K.M., Larson, J.M. (2002). Cross-modality attentional blinks without preparatory task-set switching. Psychonomic Bulletin & Review, 9(3), 497-506.

Arroyo, E., Selker, T. and Stouffs, A. (2002) Interruptions as multimodal outputs: Which are the less disruptive? ICMI'02, 479-483.

Bailey, B.P., Konstan, J.A. and Carlis, J.V. (2000) Adjusting Windows - Balancing Information Awareness with Intrusion. Proceedings of the 6th Conference on Human Factors and the Web.

Baldonado, M.Q.W., Woodruff, A., & Kuchinsky, A. (2000). Guidelines for using multiple views in information visualization. In Proceedings of AVI2000, Palermo, Italy (pp. 110-119). Retrieved, July 2005, from http://www2.parc.com/istl/projects/uir/pubs/items/UIR-2000-05-Baldonado-AVI2000-MultipleViews.pdf

Barsalou, L.W., Simmons W.K., Barbey, A.K., Wilson, C.D. (2003). Grounding conceptual knowledge in modality-specific systems. TRENDS in Cognitive Sciences, 7(2), 84-91.

Bell, L., Boye, J, Gustafson, J and Wirén, M. Modality Convergence in a Multimodal Dialogue System. Proceedings of Götalog 2000, Fourth Workshop on the Semantics and Pragmatics of Dialogue, 29-34.

Berka, C., Levendowski, D.J., Ramsey, C.K., Davis, G., Lumicao, M.N., Stanney, K., Reeves, L. Harkness Regli, S., Tremoulet, P.D., Stibler, K. (2005). Evaluation of an EEG-Workload model in an Aegis simulation: Biomonitoring for physiological and cognitive performance during Military Operations. Proceedings of the International Society for Optical Engineering. 5797, 90 - 99.

Berz, W. L. (1995). Working memory in music: A theoretical model. Music Perception, (12), 353-364. Billinghurst, M, Kato, H., & Poupyrev, I. (2001). The Magic Book: Moving seamlessly between

reality and virtuality. IEEE Computer Graphics and Applications, 21(3), 2-4.Boiney, L. (2005). Team decision making in time-sensitive environments. Presented at the 10th

International Command and Control Research and Technology Symposium: The Future of C2. Retrieved, July 12, 2005, from http://www.dodccrp.org/events/2005/10th/CD/papers/175.pdfRetrieved, July 12, 2005, from http://www.dodccrp.org/events/2005/10th/CD/papers/175.pdf

Casini, L. & Macar, F. (1999) Multiple Approaches to investigate the Existence of an Internal Clock Using Attentional Resources, Behavioral Processes 45, 73-85.

Cress, U. & Knabel, O.B. (2003). Previews in hypertexts: effects on navigation and knowledge acquisition. Journal of Computer Assisted Learning, 19, 517-527.

Czerwinski, M., E., Cutrell & Horvitz, E. (2000). Instant Messaging and Interruption: Influence of Task Type on Performance. Proceedings of OZCHI, 356-361.

Donmez, B., Boyle, L., Lee, J.D. (2003). Taxonomy of Mitigation Strategies for Driver Distraction. Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, 1865-1869.

Dorneich, M., Whitlow, S., Ververs, P.M., Mathan, S., Raj, A., Muth, E., Hoover, A., DuRousseau, D., Parra, L., & Sajda, P. (2004) The role of peripheral feedback in emotional experience with music. Music Perception, 22(1), 79-115). DARPA improving warfighter information intake under stress – Augmented Cognition: concept validation experiment


http://www.dodccrp.org/events/2005/10th/CD/papers/175.pdf

http://www.ingentaconnect.com/content/psocpubs/pbr


(CVE) analysis report for the Honeywell team. Prepared under contract: DAAD16-03-C-0054.

ETSI [European Telecommunications Standards Institute]. (2002). Human factors (HF); Guidelines on the multimodality of icons, symbols, and pictograms. (Report No. ETSI EG 202 048 v 1.1.1 (2002-08). Sophia Antipolis, France: ETSI.

Furnas, G. (1986) Generalized Fisheye Views. ACM SIGCHI Bulletin, 17(4), 16-23.Gentner D. and Nielsen, J. (1996). The Anti-Mac Interface. Communications of the ACM, 30(8),

70-82.Gutwin C. and Skopik, A. (2003) Fisheye Views are Good for Large Steeering Tasks.

Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 201-208. Gutwin, C. (2002). Improving Focus Targeting in Interactive Fisheye Views, Proceedings of CHI

2002, 267-274.Hildebrandt, M.A., Dix, A.J., Meyer, H.A. (2004). Time Design. CHI '04 extended abstracts on

Human Factors in Computing Systems, 1737-1738. Horvitz, E. & Apacible, J. (2003). Learning and Reasoning About Interruption. Proceedings of

the Fifth ACM International Conference on Multimodal Interfaces, 20-27.Horvitz, E., Jacobs, A. & Hovel, D. (1999). Attention-Sensitive Alerting. Proc. Uncertainty in

Artificial Intelligence, 305-313.Howley, K. (2004) Breaking, Making, and Killing Time in Pulp Fiction, Scope – an online

journal of film studies. May 2004. Retrieved July 13, 2005, from http://www.nottingham.ac.uk/film/journal/articles/making-breaking-and-killing.htm.

Hudson, S.E., Fogarty, J., Atkeson, C.G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J.C., & Yang, J. (2003). Predicting Human Interruptibility with Sensors: A Wizard of Oz Feasibility Study. Proceedings of the SIGCHI conference on Human factors in computing systems, 257-264.

Kirsh, D. (1995). The Intelligent Use of Space. Artificial Intelligence, 73, 31-68.Kirsh, D. (2000). A Few Thoughts on Cognitive Overload. Intellectica, 30, 19-51.Koffka, K. (1935). Principles of Gestalt Psychology. Lund Humphries: London.. Retrieved

October 13, 2005, from http://www.marxists.org/reference/subject/philosophy/works/ge/koffka.htm.

Kreifeldt, J.G. & McCarthy, M.E. (1981). Interruption as a Test of the User-Computer Interface. Proceedings of the 17th Annual Conference on Manual Control, Jet Propulsion Laboratory, California Institute of Technology, JPL Publication 81-95, 655-667.

Langley, P. (1999). User modeling in adaptive interfaces. In Proceedings of the Seventh International Conference on User Modeling, (p. 357-370). Banff, Alberta: Springer. Retrieved, July 12, 2005, from http://www.cs.utah.edu/classes/cs5350/handouts/adapt.um99.pdf

Lyons, K., Gandy, M., & Starner, T. (2000) Guided by Voices: An Audio Augmented Reality System. Proceedings of International Conference on Auditory Display (ICAD) 2000. April 2000.

Mahjan, R. & Shneiderman, B. (1997). Visual and Text Consistency Checking Tools for Graphical User Interfaces. IEEE Transactions on Software Engineering, 23(11), 722-735.

Mamykina, L., Mynatt, E.,Terry, M.A. (2001). Time Aura: interfaces for pacing, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 144-151.

Mancini, C. & Buckingham, S.S. (2001) Cognitive coherence relations and hypertext: from cinematic patterns to scholarly discourse. Proceedings of the twelfth ACM conference on Hypertext and Hypermedia, 165-174.

McFarlane, D.C. (1999). Coordinating the Interruption of People in Human-Computer Interaction. INTERACT, 295-303.

McFarlane, D. C. (2002). Comparison of four primary methods for coordinating the interruption of people in human-computer interaction, Human-Computer Interaction, 17(1), 63-139.


http://scholar.google.com/url?sa=U&q=http://wearables.cc.gatech.edu/publications/icad2000-voices/

http://scholar.google.com/url?sa=U&q=http://wearables.cc.gatech.edu/publications/icad2000-voices/

http://www.nottingham.ac.uk/film/journal/articles/making-breaking-and-killing.htm

http://portal.acm.org/citation.cfm?id=22339.22342&coll=GUIDE&dl=GUIDE&CFID=48694168&CFTOKEN=52252474


Meyer, H.A. & Hildebrandt, M. (2002). Towards Time Design: Pacing of hypertext navigation by system response times. Ext. Abstracts CHI2002, ACM Press, 824-825.

Miyata, Y. & Norman, D. (1986). Psychological issues in support of multiple activities. In: Norman, D.A. & Draper, S.W. (Eds.), User Centered Systems Design: New Perspectives on Human-Computer Interaction. Hillsdale: Lawrence Erlbaum Associates, 265-284.

Monk, C. A., Boehm-Davis, D. A. & Trafton, J. G. (2002). The attentional costs of interrupting task performance at various stages. Proceedings of the Human Factors and Ergonomics Society 2002, 1824-1828.

Nielsen, J. (1989). Coordinating user interfaces for consistency. ACM SIGCHI Bulletin, 20(3), 63-65.

Norman, D.A. (1988). The Psychology of Everyday Things. Basic Books, New York.Pecher, D., Zeelenberg, R., Barsalou, L.W. (2003). Verifying different-modality properties for

concepts produces switching costs. Psychological Science, 14(2), 119.Posner, M. I., Snyder, C. R. R. & Davidson, B. J. (1980). Attention and the detection of signals.

Journal of Experimental Psychology: General, (109), 160-174. Predebon J. (1999) Time Judgments as a Function of Clock Duration: Effects of temporal

paradigm and an attention-demanding nontemporal task, Perceptual and Motor Skills, 88, 1251-1254.

Sakar, M. & Brown M.H. (1992). Graphical Fisheye Views of Graphs. Proceedings of ACM on Human Factors in Computing Systems, 83-91.

Schmorrow, D., Stanney, K.M., Wilson, G., & Young, P. (2005). Augmented cognition in human-system interaction. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (3rd edition). New York: John Wiley.

Schweiger, W. (2001). Hypermedien im Internet. Verlag Reinhard Fischer, München.Sheridan T. and Ferrell W. (1974). Man-Machine Systems: Information, Control, and Decision

Models of Human Performance. MIT Press, Cambridge, Massachusetts, USA.Shneiderman, B. (1984). Response time and display rate in human performance with computers.

Computing Surveys, 16, 3, 265-285.Speier, C., Valacich, J.S. & Vessey. I. (1999). The Influence of Task Interruption on Individual

Decision Making: An Information Overload Perspective. Decision Sciences, 30(2), 337-360.St John, M. and Osga, G. (1999). Supervision of Concurrent Tasks Using a Dynamic Task Status

Display. Proceedings of the 43rd Human Factors & Ergonomics Society Annual Meeting, 168-172.

Starner, T., Mann, S., Rhodes, B., & Levine, J. (1997). Augmented Reality Through Wearable Computing. M.I.T Media Laboratory Perceptual Computing Section Technical Report No. 397.

Strausser, J., Kollmorgen, G., and Juhnke, J. (2005). DARPA Small Business Innovation Research (SBIR) Program Phase I Report for the Period July 14, 2004 to January 14, 2005. Contract ID W31P4Q-04-C-R253.

Sulzen, J. (2001). Modality based working memory. School of Education, Stanford University. Retrieved, February 5, 2003 from http://ldt.stanford.edu/~jsulzen/james-sulzen-portfolio/classes/PSY205/modality-project/paper/modality-expt-paper.PDF.

Szalavári, Z., Eckstein, E., Gervautz, M. (1998) Collaborative gaming in augmented reality. Proceedings of the ACM symposium on Virtual reality software and technology, 195-204.

Teal S.L., Rudnicky, A.I. (1992). A performance model of system delay and user strategy selection. Proceedings of the SIGCHI conference on Human factors in computing systems, 295-305.

Townsend, J.T. (1971). A note on the identifiability of parallel and serial processes. Perception and Psychophysics, 10, 161-163.


http://portal.acm.org.ezproxy2.lib.depaul.edu/citation.cfm?id=293740&coll=Portal&dl=GUIDE&CFID=55477670&CFTOKEN=77405351

http://portal.acm.org.ezproxy2.lib.depaul.edu/citation.cfm?id=293740&coll=Portal&dl=GUIDE&CFID=55477670&CFTOKEN=77405351

http://ldt.stanford.edu/~jsulzen/james-sulzen-portfolio/classes/PSY205/modality-project/paper/modality-expt-paper.PDF

http://ldt.stanford.edu/~jsulzen/james-sulzen-portfolio/classes/PSY205/modality-project/paper/modality-expt-paper.PDF


Trafton, J. G., Altmann, E. M., Brock, D. P. and Mintz, F. E. (2003).Preparing to resume an interrupted task: Effects of prospective goal encoding and retrospective rehearsal, International Journal of Human-ComputerStudies, 58, 583-603.

Wickens, C. D. (1984). Processing resources in attention. In R. Parasurraman & R. Davies (eds.), Varieties of Attention (pp. 63-101). New York: Academic Press.

Wickens, C. D. (1992). Engineering Psychology and Human Performance (2nd Edition). New York, NY: Harper Collins.

Wickens, C.D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2), 159-177.

Zacks, J., & Tversky, B. (2001). Event structure in perception and cognition. Psychological Bulletin, 127(1), 3-21.

Zacks, J.M., Braver, T.S., Sheridan, M.A., Donaldson, D.I., Snyder, A.Z., Ollinger, J.M., Buckner, R.L. & Raichle, M.E. (2001) Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience 4, 651 – 655.

Zacks, J., Tversky, B., & Iyer, G. (2001). Perceiving, remembering, and communicating structure in events, Journal of Experimental Psychology: General, 130(1), 29-58.

Zakay, D. & Block, R.A. (1997). Temporal Cognition, Current Directions in Psychological Science, 6,(1), 12-16.

Zijlstra, F. R. H., Roe R. A., Leonova, A. B. & Krediet, I. (1999). Temporal factors in mental work: Effects of interrupted activities, Journal of Occupational and Organizational Psychology, 72, 163-185.


introduction - naperville park district repo… · web viewone attempt to solve the problem is a...

Documents