collective mistrust of alarms james p. bliss, ph.d. susan sidone holly mason old dominion university

Collective Mistrust of AlarmsCollective Mistrust of Alarms

James P. Bliss, Ph.D.

Susan Sidone

Holly Mason

Old Dominion University

Collective Mistrust of Alarms - A Collective Mistrust of Alarms - A Few Thoughts Before We Begin...Few Thoughts Before We Begin...

Novelty of this project– Not Automation per se: Alarms inform, do not control– Alarms convey system state to operator– May help to “push the envelope” of etiquette research

“Mistrust/distrust” may be different with alarms Simplistic paradigm deceptive - multiple trust

components involved Information accessibility, technology improvements

means operators expect more from alarm systems Operator mental models very important

Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms - Collective Mistrust of Alarms - IntroductionIntroduction

Investigations of Individual Alarm Mistrust– Aviation (Bliss, 1997)– Mining (Mallett et al., 1992)– Ship Handling (Kerstholt et al., 1996) – Driving (Nohre et al., 1998).

General Findings: People Reacted Slower, Less Frequently, Less Appropriately to Unreliable Alarms.

No Studies of the Impact of Marginally Reliable Alarm Signals on Teams of Operators



Teamed Alarm Reactions– Aviation – Critical Care Units– Nuclear Power Plants– Air Traffic Control Centers

To Effectively React to Alarms, Team Members Must– Share Information– Troubleshoot Systems– Determine Relative Signal Priority– Allocate and Coordinate Reaction Responsibility



Team Member Interdependence Often Varies with the Task and the Environment (Thompson, 1967).

Dependent Teams React to Alarms More Appropriately, More Slowly (Bliss et al., 2002)

Implications of Teamed Alarm Reactions for Human-Automation Etiquette– Human-Alarm Trust– Human-Human Trust– Human-Human(Alarm) Trust



Goals of the Current Research: – Investigate Reactions of Dependent and Independent Teams

to Alarm Signals of Various Reliability Levels. – Determine How Collateral Alarm Systems Mediate Alarm

Mistrust. Approach:

– Dual-Task Approach (Damos, 1991).– Independent Variables Manipulated Using a 2 X 3 Mixed

Design. – Dyads Reacted to Two Separate Alarm Systems.

Temperature Alarm Reliability = 80% true alarms). Pressure Alarm Reliability Fluctuated (40%, 60% or 80%).


Collective Mistrust of Alarms - Collective Mistrust of Alarms - Experimental DesignExperimental Design

Interdependence Manipulated Between Two Groups– Dependent Team Members Required Interaction to React

Appropriately; Independent Team Members Did Not. Pressure Alarm Reliability Manipulated Within Groups

– Pressure Alarms Were 40%, 60%, and 80% Reliable During Sequential Task Sessions. Temperature Alarms Were 80% Reliable.

Dependent Measures– Ongoing Task: Gauge Monitoring Accuracy, Tracking Error.

– Alarm Task: Reaction Speed, Appropriateness; Response

Frequency. Bliss, Sidone, & Mason, 2002

Collective Mistrust of Alarms - Collective Mistrust of Alarms - MethodMethod

Participants: 40 student dyads from Old Dominion University (18-43 yrs) worked for course credit and the chance for a monetary performance bonus.

Primary Task: Multi-Attribute Task (MAT) battery (Comstock & Arnegard, 1992) presented to each member.– Dual-Axis Compensatory Tracking– Gauge Monitoring– Resource Management– Participants Performed the MAT Back-to-Back



Collective Mistrust of Alarms – MAT BatteryCollective Mistrust of Alarms – MAT Battery

Collective Mistrust of Alarms - Collective Mistrust of Alarms - MethodMethod

Auditory and Visual Alarms: Digitized Fire Bell From a Boeing 757/767 simulator. – Alarms Occurred 90° to the Side of the Primary Task. – Alarm Procedure: Determine Whether Corresponding MAT

Gauges Are Out of Tolerance. If so, Reset Gauges and respond to the alarm. If Not, Cancel the alarm and resume the primary task.

Interdependent team members had to communicate because they shared the out-of-tolerance gauges. Independent team members monitored all gauges.


Collective Mistrust of Alarms - Collective Mistrust of Alarms - ProcedureProcedure

Informed Consent Form Experimental Instructions - Dependent team members told to

communicate. MAT Task Practice

– Individual 120-second sessions (Each Subtask)– Combined 200-second session (MAT and Alarms)

Three experimental sessions– Ten alarms presented during each session. – Pressure alarm reliability randomly counterbalanced– Participants Knew Alarm System Reliability Before They Began

Debriefing, dismissal.


Collective Mistrust of Alarms – Collective Mistrust of Alarms – Results (Response Frequency)Results (Response Frequency)


• Response Frequency to Temp Alarms

• No Interaction (p>.05)

• Linear main effect, F(1,38)=129.600, p<.001.

0.750.56

0.41

0.780.690

0.4

0

0.2

0.4

0.6

0.8

1

40% 60% 80%

Pressure Alarm Reliability (True Alarm Rate)

Alar

m R

espo

nses

(%)

Independent

Dependent

Collective Mistrust of Alarms – Collective Mistrust of Alarms – Results (Reaction Appropriateness)Results (Reaction Appropriateness)


0.91

0.90.930.98

0.69

0.86

0

0.2

0.4

0.6

0.8

1

40% 60% 80%


App

ropr

iate

Res

pons

es (%

)

Dependent

Independent

• Significant Interaction, F(2,76)=10.193, p<.001.

• Main Effect for Interdependence, F(1, 38)= 4.000, p=.05.

• Quadratic Main Effect for Reliability, F(1,38)=19,563, p<.001.

Collective Mistrust of Alarms – Collective Mistrust of Alarms – Results (Reaction Time)Results (Reaction Time)


5.5785.9616.126

6.226.6676.631

0123456789

10

40% 60% 80%


Alar

m R

eact

ion

Tim

e (s

ecs)

Independent

Dependent

• No significant interaction

• No Interdependence main effect

• Linear Reliability Main Effect, F(1,38)=8.181, p=.007.

• NOTE: No Primary Task Differences

Collective Mistrust of Alarms - Collective Mistrust of Alarms - DiscussionDiscussion

Results Similar to Past Efforts, Except for Lack of Primary Task Differences. – Multiple alarm systems may have led participants to

rethink their trust levels, a reflection of workload (Bliss & Dunn, 2000).

Alarm designers should consider the effects of multiple alarm systems on operator behavior. – Recognize that complex reaction responsibilities may

cause cognitive load as team members adjust trust levels.



Dimensions of Trust in This Experiment– Basic Trust of the Experimenter (Human-Human)– Trust of the Primary (MAT) Task (Human-Computer)– Trust of the Alarm Task (Human-Computer)

Manipulated by the Experimenter– Trust of Teammates (Human-Human)

Questionable in this Experiment, Due to documented Unfamiliarity



Past reactions to unreliable alarm systems– Fluctuations in physiological responses (Breznitz, 1983)– Degraded performance (Getty et al., 1995) – Complete Lack of Trust (Bliss, 1993)– Complete Trust– Probability Matching – Participants’ Response Rates

Mirror the Perceived Reliability of the Alarm System.– These Patterns Take Time to Appear (Bliss et al., 1996). – Question: What if Researchers Apply human trust

facilitators to human-alarm relationships? WHAT ARE THOSE VARIABLES?



Documented Ways to Improve Alarm Responsiveness– Maximize alarm reliability (Bliss, 1993)– Advertise high alarm reliability rates (Bliss et al., 1995)– Add Redundant Sources of Alarm Information (Bliss et al., 1996)– Augment alarm stimuli and response options (Bliss, 1997).

Etiquette Related Possibilities – Give alarm systems “human” qualities

(include verbiage, etc.)– Make alarm stimuli emotional

(Sorkin et al.’s “likelihood alarm displays”; altering Edworthy’s parameters)

– Vary teammate trustworthiness


collective mistrust of alarms james p. bliss, ph.d. susan sidone holly mason old dominion university

Documents