This article was downloaded by: [University of Connecticut]On: 09 October 2014, At: 22:07Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Theoretical Issues in ErgonomicsSciencePublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/ttie20

Multisensory warning signals for eventperception and safe drivingCharles Spence a & Cristy Ho aa Crossmodal Research Laboratory, Department of ExperimentalPsychology , University of Oxford , Oxford, UKPublished online: 10 Oct 2008.

To cite this article: Charles Spence & Cristy Ho (2008) Multisensory warning signals for eventperception and safe driving, Theoretical Issues in Ergonomics Science, 9:6, 523-554, DOI:10.1080/14639220701816765

To link to this article: http://dx.doi.org/10.1080/14639220701816765

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Theoretical Issues in Ergonomics ScienceVol. 9, No. 6, November–December 2008, 523–554

Multisensory warning signals for event perception and safe driving

Charles Spence* and Cristy Ho

Crossmodal Research Laboratory, Department of Experimental Psychology,University of Oxford, Oxford, UK

(Received 1 May 2007; final version received 19 November 2007)

This article reviews various different approaches to the design of unimodal andmultisensory warning signals, in particular warning signals for use in alertingdrivers to potentially dangerous situations. The design of optimal warning signalsthat are maximally effective in terms of the limitations that constrain humaninformation processing are discussed in light of the latest findings emerging fromcognitive neuroscience research. A new approach to the design of multisensorywarning signals, involving the presentation of warning signals in different regionsof space around a driver, is then critically examined.

Keywords: multisensory warning signal; spatial attention; driving; auditory icon;tactile displays; cognitive neuroscience; neuroergonomics

1. Introduction

Humans are inherently limited capacity creatures; that is, they are able to process only arestricted amount of sensory information at any given time (see Broadbent 1958, Driver2001 for reviews). The inability to simultaneously process multiple sources of informationplaces a number of important constraints on the design and utilisation of vehicularinformation systems (e.g. Burke et al. 1980, Spence and Driver 1997b, Mather 2004, Chanand Chan 2006). What is more, these attentional limitations on driver performance arecurrently being exacerbated by the increasing availability of complex in-vehicletechnologies (Wang et al. 1996, Ashley 2001, Lansdown et al. 2004, Lee et al. 2004,although see also Rockwell 1988, Cnossen et al. 2004), such as navigation systems(e.g. Fairclough et al. 1993, Burnett and Joyner 1997, Dingus et al. 1997), cellular (mobile)phones (e.g. Spence and Read 2003, Strayer et al. 2003, Patten et al. 2004), email (e.g. Leeet al. 2001) and ever more elaborate sound systems (e.g. Jordan and Johnson 1993).Somewhat surprisingly, this proliferation of in-vehicle interfaces has taken place in the faceof extensive research highlighting the visual informational overload suffered by manydrivers (e.g. Hills 1980, Dewar 1988, Bruce et al. 2000, Dukic et al. 2006, although see alsoSivak 1996).

Given the many competing demands on a driver’s limited cognitive resources, it shouldcome as no surprise that driver inattention (including drowsiness, distraction and‘improper outlook’) has been identified as one of the leading causes of vehicular accidents,

*Corresponding author. Email: charles.spence@psy.ox.ac.uk

ISSN 1463–922X print/ISSN 1464–536X online

� 2008 Taylor & Francis

DOI: 10.1080/14639220701816765

http://www.informaworld.com

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

responsible for anything between 26% (Wang et al. 1996) and 56% (Treat et al. 1977) of allroad traffic accidents (see also Gibson and Crooks 1938, Zaidel et al. 1978, Sussman et al.1985, Hughes and Cole 1986, Ashley 2001). Fortunately, however, the last few years haveseen the development of a variety of new technologies, such as sophisticated advancedcollision avoidance systems, designed to monitor the traffic environment automaticallyand to provide additional information to drivers in situations with a safety implication.In fact, the development of these new technologies, known collectively as intelligenttransport systems, means that more information than ever before can now potentially bedelivered to drivers in a bid to enhance their situation awareness and ultimately improveroad safety.

One of the most common types of car accident, estimated to account for arounda quarter of all collisions, is the front-to-rear-end collision (Horowitz and Dingus 1992,McGehee et al. 2002, see also Evans 1991). Research suggests that driver distraction(or inattention) represents a particularly common cause of this kind of accident (no matterwhether the leading vehicle is stationary or moving; Wang et al. 1996, see also Rumar1990). The potential benefits of improving the situation awareness of drivers to roaddangers such as an impending collision is therefore huge. To put this in perspective, it hasbeen estimated that the introduction of a system that provided even a modest decrease inthe latency of overall driver responses (say, of around 500ms) would reduce rear-endcrashes by as much as 60% (Suetomi and Kido 1997). Given that a number of differentforward (or advanced) collision warning systems now exist, it has become increasinglyimportant to determine the optimal means of helping drivers to avoid such collisions(e.g. Graham 1999, Krishnan et al. 2001). Current strategies differ in terms of their degreeof intervention, varying from proactive collision avoidance systems that can initiateautomatic emergency braking responses to collision warning systems that simply presentwarning signals to drivers, prompting them to adjust their speed voluntarily instead(Hunter et al. 1976, Janssen and Nilsson 1993).

A great deal of empirical effort has gone into studying how best to alert and warninattentive drivers to impending danger. A number of researchers have suggested thatauditory and/or tactile (rather than visual) warning signals should be used in interfacedesign (e.g. Munns 1971, Deatherage 1972, Sorkin 1987, Stokes et al. 1990, Horowitz andDingus 1992, Ballas 1994, Fitch and Kramer 1994, Hirst and Graham 1997, Burnettand Porter 2001, Lee et al. 2004, Fitch et al. 2007). The potential advantages of using suchnon-visual signals include the fact that: (1) people can respond more rapidly to auditoryand tactile signals than to visual signals (e.g. Todd 1912, Jordan 1972, Nelson et al. 1990,Nicolas 1997, Ho et al. 2005b); (2) the presentation of such signals should not overload thedriver’s visual system (e.g. Hawkes and Loeb 1962, Veitengruber 1978, Liu 2001, Ho andSpence 2005a, Ho et al. 2005b, McKeown and Isherwood 2007, although see Spence andDriver 1997b, Spence and Read 2003); (3) auditory and tactile signals are inherently morealerting than visual stimuli (e.g. Geldard 1960, Gilmer 1961, Posner et al. 1976, Campbellet al. 1996, Green 2000); (4) auditory and tactile signals are particularly good at capturingattention (e.g. Geldard 1960, Gilmer 1961, Hawkes and Loeb 1961, Jones 1989, Ballas1994); (5) unlike visual cues, auditory and tactile warning signals are not dependent fortheir effectiveness on the current direction of a driver’s gaze (Hirst and Graham 1997,Graham 1999, Stanton and Edworthy 1999, although see Ho and Spence 2007) and can beperceived even when the driver’s eyes are closed (as during blinking) or when the visualsystem is effectively ‘turned off’ (as during saccades, which typically occur several times asecond; Rockwell 1972, Bristow et al. 2005); (6) haptic displays and warning signals tendto be judged as less annoying than many other kinds of warning signal (e.g. McGehee and

524 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Raby 2003, Lee et al. 2004). This review paper is structured as follows. First, currentapproaches to the design of unimodal warning signals are reviewed; next, current researchon multisensory warning signals is evaluated and, finally, a new approach to the design ofa multisensory warning signal taking into account the latest findings emerging fromcognitive neuroscience research is proposed and then critically examined. The discussionwill primarily draw on the findings from a number of studies conducted recently in theauthors’ laboratory.

2. Current approaches to the design of unimodal warning signals

There are several important considerations when assessing the utility of a particularwarning signal. Specifically, signal effectiveness relates to the efficacy with which the signalcan draw the attention of an interface operator in order to communicate the desiredmessage and the ease with which the operator can subsequently produce the appropriateresponse(s). Researchers have typically measured the parameters of response latency andaccuracy when assessing the utility of a given warning signal. This section delineates thecurrent approaches to unimodal auditory and unimodal tactile warning signal design.

2.1. Temporal considerations

In a study looking at the timing of warning signal presentation (i.e. how far in advance of apossible crash a warning signal should be presented), McGehee et al. (2002) assessed theeffectiveness of presenting auditory signals to warn distracted drivers of an impendingcollision with a stationary vehicle on the road ahead in a driving simulator study.They found that the presentation of an ‘advance’ auditory warning signal facilitated driverresponses (in terms of shorter accelerator release times and fewer, and less severe, crashes)relative to either a ‘late’ warning signal (giving only 1 second, as opposed to 1.5 seconds,advance warning), or else to a no warning signal baseline condition (cf. Johansson andRumar 1971). While results such as these highlight the potential benefits of presentingwarning signals to enhance the situation awareness of drivers, it is important to note thata number of potential limitations have also been identified with the presentation of suchearly warning signals.

For example, one might intuitively think that the earlier a warning signal concerninga potential emergency situation is delivered to a driver the better, as it should presumablyallow the driver more time in which to prepare and execute the appropriate behaviouralresponse. However, research has shown that the more advance notice a warning signalprovides to an interface operator of a potentially dangerous upcoming event, the morelikely it is that the warning signal will be classified as a false alarm (Shinar 1978,Parasuraman et al. 1997, McGehee et al. 2002). As soon as the false alarm rate for awarning signal becomes too high, interface operators may perceive it as a nuisance andstart to ignore it (cf. Chambrin 2001). Thus, the danger is that interface operators maybecome ‘desensitised’ to future warning signals (see Wiener 1977, p. 179, Breznitz 1984,Bliss and Acton 2003). As Sorkin (1988) notes, some interface operators may even go sofar as to try and disable alarm signals that they consider to be too distracting or aversive(see also Wiener 1977, p. 178, King and Corso 1993).

The goal for research in this area is therefore to try to provide efficient warning signalsin potentially dangerous driving situations that will not be perceived as a ‘nuisance’ bydrivers (Parasuraman et al. 1997). Given the fact that front-to-rear-end collision warnings

Theoretical Issues in Ergonomics Science 525

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

occur only very rarely (cf. Horowitz and Dingus 1992), warning signals need to be

designed so as to be maximally effective in terms of eliciting the most rapid, and alsoappropriate, behavioural response (see Figure 1). In that way, the warning signals can be

presented closer in time to the external event that they are intended to inform the interface

operator about and thus reduce the likelihood that the signals will constitute a false alarm.

It should be noted here that the more advance notice that a warning signal provides to a

driver regarding a potential upcoming accident, the more likely that the signal willconstitute a false alarm (see the left part of Figure 1). Nevertheless, warning signals still

need to be presented early enough in time to trigger the execution of the appropriate

behavioural response. Note that if the warning signal is presented too late (as shown on the

right of Figure 1), then the warning signal may be perceived by the driver, but there will be

insufficient time for the driver to execute an appropriate response. In other words, thedesign and timing of warning signals needs to be optimised for the entire system

(i.e. considering the human-plus-alarm system as one; cf. Sorkin and Woods 1985,

Horowitz and Dingus 1992). The optimal time window in which to present a warning

signal is represented schematically in the middle section of Figure 1.

2.2. Traditional auditory warning signal design alternatives

Over the years, a number of different approaches to the design of effective auditory

warning signals have been proposed. These include the use of spatially localised auditorywarning signals (e.g. Humphrey 1952, Begault 1993, 1994, Begault and Wenzel 1990,

Bronkhorst et al. 1996, Campbell et al. 1996, Lee et al. 1999, Bliss and Acton 2003,

Svensson and Tap 2003, Ho and Spence 2005a), the use of multisensory warning signals

(e.g. Mowbray and Gebhard 1961, Selcon et al. 1995, Hirst and Graham 1997, Spence and

Driver 1999, Mariani 2001, Kenny et al. 2004, Brown 2005, Ho et al. 2007; see also Haas1995) and the use of synthetic warning signals (known as auditory earcons; Lucas 1995,

McKeown and Isherwood 2007) that have been artificially engineered so as to deliver a

7001500 0

If a warning signal ispresented too early,

it is likely to be judged as annoying, or else to represent

a false alarm

Time to impact (ms)

Optimal window in which to present a

warning signal

Warning signal presented too late

to be effective

Latest point at which a warning signal can

successfully facilitate a crash avoidance response

Moment of impact

Figure 1. A schematic timeline showing the effectiveness of a front-to-rear-end crash avoidancewarning signal as a function of time prior to the moment of impact. Estimated times calculated onthe basis of previous research in the literature (in particular, Green 2000, see also Hills 1980, Dewar1988, Korteling 1990, Horowitz and Dingus 1992, Campbell et al. 1996, Abe and Richardson 2004,2005, 2006, Keller and Stevens 2004).

526 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

certain degree of perceived urgency (see also Van Egmond 2004, Cabrera et al. 2005).To date, these approaches have met with mixed success (see Lee et al. 1999, cf. Rodway2005). For example, researchers have often found that people find it difficult to localiseauditory warning signals, especially when they are presented in enclosed spaces such asinside a car, hence often negating any benefit associated with the spatial attributes of thewarning sound (see Bliss and Acton 2003). Meanwhile, other researchers have reportedfavourably on the potential use of directional sounds in confined spaces (e.g. Doyle andSnowden 1999, Catchpole et al. 2004, Ho and Spence 2005a, Ho et al. 2007, Sound AlertTechnology undated).

Finally, it often takes time for interface operators to learn the arbitrary associationbetween a particular auditory earcon and the appropriate response, as the perceivedurgency is transmitted by the physical characteristics of the warning signal itself(such as the rate of presentation, the fundamental frequency of the sound and/or itsloudness, etc.; Edworthy et al. 1991, Hellier et al. 1993, Haas and Casali 1995, Haas andEdworthy 1996, Graham 1999). It would therefore appear that unless the auditory earconsare associated with intuitive responses (Lucas 1995, Graham 1999), they should not beused in dangerous situations to which an interface operator may only rarely be exposed,since he/she may fail to recall the appropriate actions straight away (see also Doll andFolds 1986, Ballas 1994, Guillaume et al. 2003).

2.2.1. Auditory icons

Given these limitations on the use of traditional auditory warning signals, a number ofresearchers have attempted to investigate whether auditory icons (i.e. sounds that imitatereal-world events; Gaver 1986) might provide more effective warning signals as theyinherently convey the meaning of the events that they are meant to signify (e.g. Lazarusand Hoge 1986, Blattner et al. 1989, Gaver 1989, Gaver et al. 1991, Belz et al. 1999, Kellerand Stevens 2004, Ho and Spence 2005a, McKeown 2005, Ho et al. 2007). Over the years,the effectiveness of a variety of different ‘urgent’ (or danger) warning sounds (icons) havebeen evaluated for their ability to capture attention and, perhaps more importantly, toelicit the appropriate behavioural responses from an interface operator. For example, inone of the earliest studies in this area, sponsored by the American military, Oyer andHardick (1963) evaluated how rapidly different segments of the population of theUnited States responded to a wide variety of auditory alerting signals, including suchattention-capturing auditory icons as the sounds of elephants stampeding and babiescrying (see also Deatherage 1972, Lazarus and Hoge 1986, Ballas 1993).

More recently, researchers have investigated the effectiveness of screeching car tyresounds and car horn sounds to warn car drivers of an impending collision (e.g. Graham1999, McGehee et al. 2002, Ho and Spence 2005a, Ho et al. 2007). For example,participants in Graham’s study had to decide whether or not to brake depending on theparticular combination of road scene and warning signal that was presented to them.They responded significantly more rapidly to the auditory icons than to the tonal orspeech-based warning signals. Meanwhile, Ho and Spence have examined the relativeeffectiveness of spatially presented car horn sounds vs verbal alerts (the words ‘front’ and‘back’) as potential collision warning signals. Figure 2a schematically illustrates theexperimental set-up utilised in Ho and Spence’s laboratory-based driving studies. The leftpanel provides a bird’s-eye view of the experimental set-up, while the panel on the rightshows the schematised timeline of events in a typical trial. The participant is shownholding a steering wheel and monitoring a central rapid serial visual presentation stream of

Theoretical Issues in Ergonomics Science 527

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

letters at fixation for occasionally presented target digits (that he/she has to respond to by

making a button-press response on the steering wheel).Two driving videos were presented to the participants in Ho and Spence’s (2005a)

studies, one from in front of the driver and the other from behind (and viewed by looking

in the rear view mirror; shown in the upper left in Figure 2a). The large video shows the

view ‘out of the windscreen’ on to the road ahead (a car can just be seen on the road in

the distance), while the view from the rear view mirror shows the road scene behind the

participant’s car (once again, a car can be seen in the middle distance). The timeline on the

right of Figure 2a shows key frames from the video where the car in front suddenly braked

and the participant/driver ought to have made a rapid braking response. An auditory

warning signal (the sound of a car horn played from the loudspeaker placed either directly

in front or behind the driver) was presented in time with the onset of the critical driving

event (the onset and offset of the auditory cue is shown at the top of the timeline).

… …

… …

Auditorycue

(a)

(b)

… …

600 ms

Auditory cue every 8500−21500 ms

Driving video

Start of block 1800 ms 15000 ms Time

Critical target visual driving event presented inthe front windscreen or rearview mirror, requiring

either a braking or accelerating response

… …

N ZRSVPstream

RSVP target every2040–6360 ms

6

40 ms 80 ms

Loudspeaker cones placed spatially to the front and rear-left of the participant

Auditorycue

direction

Location oftarget visualdriving event

:

997

1124

1008

895 868

600

700

800

900

1000

1100

1200

[1]Nonpredictivenonspatial carhorn sounds

[2]Nonpredictive

spatial carhorn sounds

[3]Predictivespatial car

horn sounds

[4]Predictivesymbolic

verbal soundcues

[5] Predictive

spatialsymbolic

verbal soundcues

Auditory cue type

Mea

n R

T (

ms)

Cue front : Target front

Cue back : Target front

Cue front : Target back

Cue back : Target back

Overall mean

Figure 2. (a) A schematic diagram showing the experimental set-up and task design used in Ho andSpence’s (2005a) recent study of auditory warning signals. RSVP¼ rapid serial visual presentation.(b) Summary results of mean reaction times (RTs) as a function of auditory cue direction(front vs back) and location of target visual driving event (front vs back) reported across fiveexperiments ([1]–[5]). Error bars indicate standard errors of the means.

528 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Ho and Spence’s (2005a) studies showed that participants’ performance (measured interms of the latency of their braking and accelerating responses) for a given target position(e.g. target front) was faster when the warning signal (or cue) came from the same direction(i.e. cue front) than when it came from the opposite direction (i.e. cue back). Visualinspection of Figure 2b (for auditory cue types 2–5) shows that the valid spatial cuing ofthe critical driving event (i.e. when the direction from which the cue was presentedhappened to coincide with the location of the target driving event) resulted in a facilitationof driver responses (that is, the participants in Ho and Spence’s study responded morerapidly) both when the driving event occurred on the road ahead (a front target requiring abraking response) and when it occurred to the rear of the driven car (i.e. a back targetrequiring an acceleration response). Furthermore, comparison of the cuing effects showsthat spatially localised car horn warning sounds (auditory cue types 2 and 3 that actuallycame from the ‘front’ or ‘back of the driver) were just as effective in eliciting theappropriate braking or accelerating response from drivers as spatially localised verbalprompts (the words ‘front’ and ‘back’ presented from the front or rear of a driver,respectively (auditory cue type 5).

Auditory icons have the advantage over auditory earcons in that their meaning shouldbe more immediately apparent to an interface operator and so people should require lesstime in order to learn the appropriate behavioural responses to such signals (e.g. Begault1994, Lucas 1995, Hempel and Altinsoy 2005, see also Keller and Stevens 2004). However,despite the fact that research has shown that people do indeed tend to respond morerapidly to auditory warning signals as the perceived level of urgency increases (e.g. cf.Johnson and Shapiro 1989, Burt et al. 1995, Haas and Casali 1995), the use of such urgentauditory icons is not without its problems. For example, while the screeching car tyre andcar horn sounds used in Graham’s (1999) study elicited faster responses by drivers than themore typical tonal alert or verbal warning signals, the presentation of these auditory iconsalso resulted in participants making more inappropriate responses than following the tonalor verbal alerts. It would therefore appear that highly urgent signals may elicit such rapidresponses that interface operators can end up responding to the warning signal before theyhave had sufficient time to evaluate the situation properly in order to know what the mostappropriate response would have been (see Graham 1999, Bliss and Acton 2003).

As noted by Ho and Spence (2005a), the more rapid responses following verbal cues ascompared to the spatial auditory car horn cues were accompanied by an increase in errors.Furthermore, while the results of Oyer and Hardick’s (1963) study suggested that thesound of a car horn was one of the most effective auditory warning signals (behind missilealarms, yelper sirens, British air raid sirens and falcon horn sounds; see also McAdams1993), the empirical evidence to date does not provide unequivocal support for the claimthat auditory icons necessarily make particularly effective urgent warning signals. Severalresearchers have also suggested that the modality of warning signals might change as afunction of increasing urgency, with the typical suggestion being a shift from visual toauditory warning signals as the urgency of the situation increases (e.g. Stokes et al. 1990,Horowitz and Dingus 1992, Mollenhauer et al. 1994).

The use of urgent auditory alarm icons may be further limited by the fact that auditoryicons that are perceived as conveying a high degree of urgency are also likely to beperceived as unpleasant (see also Oyer and Hardick 1963, Fidell and Teffeteller 1981,Arrabito et al. 2004, McKeown and Isherwood 2007). For example, McKeown andIsherwood assessed the perceived unpleasantness of 20 different environmental sounds andfound a strong correlation between the perceived urgency of the sounds and howunpleasant people rated them as being. Therefore, while the approach of trying to develop

Theoretical Issues in Ergonomics Science 529

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

auditory warning signals that elicit an intuitive response seems like a good one, thefundamental problem appears to be that any sound that is perceived by users as urgent willmost likely also be judged as unpleasant and thus is unlikely to be accepted by the end userof an interface (see McKeown and Isherwood 2007, cf. Lazarus and Hoge 1986, Fastl2005, p. 141).

2.2.2. Speech warnings

One interesting class of auditory warning signals is constituted by speech warnings. Verbalwarning signals have the advantage that minimal training is required in order for anoperator who understands the language to comprehend and act upon them efficiently(see Burrows 1962, Edworthy and Hellier 2006, cf. Simpson et al. 1987, van Winsumet al. 1999, Bruce et al. 2000). Recently, Ho and Spence (2006) conducted a studyinvestigating the automaticity of verbal signals in orienting a person’s spatial attention.They found no attentional cuing effect of spatially non-predictive (i.e. the cue indicated thetarget position on 50% of the trials), centrally presented verbal directional cues (the word‘left’ and ‘right’) on the discrimination of peripherally presented unmasked visual targets(presented on either the left or right side of a central fixation point). However, using thesame experimental design, they were able to show that spatially non-predictive centrallypresented written word cues resulted in a small but significant attentional facilitation effect(i.e. a speeding-up of responses to targets presented on the cued as opposed to the uncuedside). Ho and Spence further demonstrated that when the visual targets were masked, thepresentation of verbal cues resulted in an improved sensitivity to visual targets on the cuedside. Thus, it appears that verbal directional cues result in the automatic orienting ofspatial attention, at least under certain conditions. (It is perhaps worth noting that thesefacilitatory effects resulting from the presentation of spatially non-predictive cues werefairly modest in size; hence, their utility in applied settings needs to be assessed. However,it should also be noted that the facilitatory effects elicited by such cues are likely to be farlarger in applied settings where the cue is informative/predictive of what is happening inthe environment around a driver.)

However, as suggested by Edworthy and Hellier (2006) in their comprehensive recentreview of complex auditory warnings, the intelligibility of speech warnings by listeners willbe affected by the environment in which they are presented. Given that while driving, aperson may be involved in other concurrent speech tasks such as conversing with thepassenger or on the mobile phone and/or listening to the radio (see Ramsey and Simmons1993), the utility of in-car speech warnings (particularly directional signals such as thosegiven by navigation systems) may be somewhat limited (see also Strayer and Johnston2001, Ho and Spence in press). Furthermore, the meaning of verbal warning signals maynot become apparent until near the end of the message, a situation that is less than idealfor urgent warning signals demanding an immediate response (cf. Burrows 1962, Simpsonand Williams 1980, Nissan 2006). The limited utility of such warning signals for driverswhose first language is different from the one in which the warning signal is presentedshould, of course, also be noted.

2.3. Tactile warning signals

Although researchers have, for many years, been interested in the potential use of tactiledisplays to present information to interface operators (e.g. Gregg 1960, Geldard andSherrick 1965, Hennessy 1966, Fenton and Montano 1968, Deatherage 1972, Hirsch 1974,

530 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Triggs et al. 1974, Sklar and Sarter 1999), current research in this area has primarily been

driven by recent advances in technology that have provided cheap and effective means of

vibrotactile stimulation (e.g. Sorkin 1987, Rupert 2000, Lindeman and Yanagida 2003,

Benali-Khoudja et al. 2004, Ho et al. 2005b, 2006b, Lee et al. 2006). Over the years,

researchers have investigated the effectiveness of vibrotactile stimuli (and also

proprioceptive cues) presented to the torso (e.g. Howell and Briggs 1959, Lindeman

et al. 2003, Ho et al. 2005b, 2006b, 2007, van Erp 2005, Ho and Spence 2007), head (e.g.

Gilliland and Schlegel 1994), hands (e.g. Jagacinski et al. 1979, Burke et al. 1980,

Schumann et al. 1993, Steele and Gillespie 2001, Vitense et al. 2003), wrists (Sklar and

Sarter 1999), buttocks (McGehee and Raby 2003, Lee et al. 2004) and even to the feet

(Godthelp and Schumann 1993, Janssen and Nilsson 1993, Janssen and Thomas 1997,

Kume et al. 1998, Brown et al. 2005). Although, by now, tactile stimulation has been tested

on a wide array of different body parts, no one has systematically examined the

relationship between the body site stimulated and the nature of the task being performed.

Thus, it is uncertain whether tactile stimulation (no matter which body part is stimulated)

will necessarily result in an improvement in the same task, or whether the benefits of

vibrotactile cuing/warning signals might be body site specific (Campbell et al. 1996,

Rinspeed undated). Note though that research on stimulus-response compatibility effects

suggests that there may be important synergies between the stimulation of particular body

parts and specific task requirements (e.g. see Proctor et al. 2005, see also Section 4).

Essentially, the latest findings from cognitive neuroscience research would appear to

suggest that the location of the most effective vibrotactile cuing might critically depend

upon the particular effector (i.e. body part) that the operator (or driver) will likely have to

use to execute the appropriate response.To date, a number of studies have explicitly assessed the utility of vibrotactile cues in

simulated driving scenarios. For example, Janssen and Thomas (1997) reported that

increasing the counterforce (i.e. combined proprioceptive and tactile cuing) on the

accelerator pedal had beneficial effects in a collision avoidance system (see also Janssen

and Nilsson 1993). Meanwhile, Ho and her colleagues (2005b) have investigated the

effectiveness of presenting vibrotactile cues to the front vs rear of a driver’s torso

(see Figure 3a) as a means of warning the driver of an impending collision that was likely

to occur either in front or behind the driver’s car (see also Tijerina et al. 2000).Ho et al. (2005b) demonstrated the beneficial effect of presenting spatial vibrotactile

cues on braking and accelerating responses (see Figure 3b). In particular, Ho et al. showed

that their participants’ performance (measured in terms of the mean RT) for a given target

position (e.g. target front) was faster when the warning signal (or cue) came from the same

(i.e. cue front) direction rather than from the opposite (i.e. cue back) direction.

Visual inspection of Figure 3b shows that the valid spatial cuing of the critical driving

event (i.e. when the direction from which the cue was presented happened to coincide with

the location of the target driving event) resulted in a facilitation of driver responses (that is,

the participants in Ho et al.’s study were able to respond more rapidly) both when the

driving event happened on the road ahead (a front target requiring a braking response)

and when they occurred to the rear of the driven car (i.e. a back target requiring an

acceleration response). Interestingly, the magnitudes of these spatial cuing effects were the

same regardless of the predictability of the cue (i.e. 80% or 50% valid; vibrotactile cue

types 1 and 2 in Figure 3b, respectively) with regard to the location of the subsequent

critical driving event (note that there were two possible event locations, front vs rear, and

hence the 50% valid cue did not provide any spatial information regarding the likely

Theoretical Issues in Ergonomics Science 531

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

location of the critical driving event), thus supporting the automatic attention-capturingcapacity of this particular form of vibrotactile cuing.

Several researchers have compared the effectiveness of vibrotactile cuing to the moretraditional modes of auditory or visual cuing. For instance, Schumann et al. (1993)contrasted various different kinds of tactile feedback to auditory warnings as a means ofsignalling to drivers whether or not to remain in lane when attempting passingmanoeuvres. They found that drivers were more responsive to tactile/proprioceptivecues (such as steering wheel vibrations or force feedback cues resisting the driver’s controlinput to change lanes) as compared to auditory warnings. Lee et al. (2004) also comparedthe effectiveness of auditory and haptic collision warnings presented either in a graded orsingle-staged manner on drivers’ braking performance. Their results showed that hapticwarnings were judged as being less annoying and that graded haptic alerts presented via avibrating seat seemed to be particularly effective in providing a greater margin of safety(cf. Horowitz and Dingus 1992, for related work on the use of graded sequences ofauditory warnings). One potentially important attribute of tactile warning signals is thattheir effectiveness should not be affected by the level of background noise (Mowbray and

600

700

800

900

1000

1100

1200

600

700

800

900

1000

1100

1200

11041065

(b)

(a)

Vibrotactile cues

Vibrotactile cues

[1] Predictive spatial vibrotactile cues [2] Nonpredictive spatial vibrotactile cues

Mea

n R

T (

ms)

Vibrotactilecue

direction

Location oftarget visualdriving event

:

Cue front : Target front

Cue back : Target front

Cue front : Target back

Cue back : Target back

Overall mean

Figure 3. (a) A schematic figure showing the location where vibrotactile stimuli were presented toparticipants in Ho et al.’s (2005b) study of attentional capture by means of the presentation ofvibrotactile cues in a simulated driving task. (b) Summary results of mean reaction times (RTs) as afunction of vibrotactile cue direction (front vs back) and location of target visual driving event (frontvs back) highlighted in the two experiments ([1] and [2]) reported. Error bars indicate the standarderrors of the means.

532 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Gebhard 1961, Brown et al. 1965, Makhsous et al. 2005, cf. Wilkins and Acton 1982,

Ramsey and Simmons 1993).Finally, Van Erp and Van Veen (2001, 2004) have investigated the relative effectiveness

of presenting directional information in a route guidance system by means of unimodal

tactile, unimodal visual and bimodal visuotactile cues. In their studies, tactile cues were

presented via tactors mounted in the driver’s seat below the driver’s thigh, while visual cues

(consisting of distance and arrow symbols) were presented on a LCD display placed to the

left of the steering wheel. They found that their participants responded significantly more

rapidly to tactile or bimodal cues than to visual cues, with their participants giving the

lowest subjective mental effort rating score to the tactile condition. Although their

participants responded more rapidly to bimodal navigational messages than to unimodal

tactile messages (Van Erp and Van Veen 2004), and vice versa in the other

study (Van Erp and Van Veen 2001), statistical analyses of their data failed to reach

significance (see also Nagel et al. 2005, for exciting recent work on a vibrotactile belt

that provided continuously updated information about the direction in which the person

was heading).

2.4. Optimisation of warning signals for driver attention and response

Taken together, the findings from the studies discussed so far would appear to suggest that

it will not be sufficient for future driver warning signals simply to be optimised for their

ability to capture a driver’s attention (Spence 2001, see also Horowitz and Dingus 1992).

Rather, they must also be optimised for (i.e. compatible with) the most appropriate driver

response in any given situation (see Table 1). Typically, behavioural phenomena have

been sub-divided into their perceptual, decisional and response-related sub-components

(e.g. see Kantowitz et al. 1990, Spence 2001, Proctor et al. 2005). The future design of

multisensory warning signals should therefore be optimised to elicit the most effective

response by combining what is known about the different brain pathways involved in

multisensory information processing (emerging from cognitive neuroscience research) and

Table 1. Guidelines for effective warning signal design.

1 ‘Easily learned’. This means that a warning signal should resemble an ecologically valid and/orwell-learned representation. Much research has been conducted on the use of auditory icons(Graham 1999), although what constitutes an easily learned signal is less clear for the case oftactile warning signals (see Brewster and Brown 2004).

2 ‘Intuitive’. This means that little training should be required in order for a driver to understandthe meaning of a warning signal. For instance, a warning signal might be presented in thedirection of the event that demands a driver’s attention in order to facilitate bothstimulus-response compatibility and automatic (or exogenous) spatial attentional orienting.

Several researchershave suggested that tactile signals may be ‘intuitive’, though once again further research isrequired (see Geldard 1961, p. 84, Rupert 2000, Van Erp 2005, Ho et al. 2006a).

3 ‘Attention-grabbing’. This means that a warning signal should be able to break through ongoingattentional control (see Graham 1999, Santangelo et al. 2007, 2008, Santangelo and Spence2007a, b).

4 ‘Division of workload among modalities’. This implies the possibly of using modalities that arenot already (over-)loaded. It should also be borne in mind that there may be a trade-offassociated with presenting stimuli in different sensory modalities in order to reduce overloadthat is associated with the concomitant costs of having to monitor additional sensorymodalities (Spence and Driver 1997b, Spence et al. 2001).

Theoretical Issues in Ergonomics Science 533

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

the particular combinations of multisensory signals that are most appropriate for specific

classes of actions (e.g. for braking, accelerating, mirror checking, turning, etc.) – that is, by

combining the knowledge regarding each of the sub-processes involved in multisensoryattention, event perception, response selection and response execution (see also Zaidel

et al. 1978, Green 2000, contrast this with focus on event perception stressed in early

research, e.g. see Gibson and Crooks 1938, p. 453).If, for instance, one takes the example of turning the steering wheel, several researchers

have already demonstrated that the presentation of tactile/proprioceptive cues via thesteering wheel can facilitate a driver’s turning of the wheel in a particular direction (e.g.

Proctor et al. 2005, cf. Schumann and Naab 1992, Steele and Gillespie 2001, Wang et al.

2003). According to the multisensory approach outlined here, tactile/proprioceptive cues

might become even more effective if they were to be combined with congruent visual cues

on the steering wheel itself (cf. Brown 2005). The authors believe that the use of suchcongruent multisensory cues may be far more effective than one would expect given the

effectiveness of each kind of unimodal cue when presented in isolation (see Santangelo

et al. 2008, Ho et al. submitted). This is because multisensory integration can give rise to

synergistic, or superadditive, outcomes (i.e. to outcomes that are larger than the simplelinear sum of their component parts, e.g. Stein and Meredith 1993, Calvert et al. 2004,

Santangelo and Spence 2007b, although see also Spence and Driver 1999, Holmes and

Spence 2005).

3. Multisensory warning signals

One current topic of interest in multisensory warning signal design is the relative

effectiveness of unimodal vs bimodal and/or multimodal signals. To date, published

studies have demonstrated mixed findings with regard to the superiority of multisensory

over single channel (or unimodal) information presentation (e.g. Kappauf and Powe 1959,Baker et al. 1962, Selcon et al. 1995, Spence and Driver 1999, see also Luo and Kay 1989,

Campbell et al. 1996). One important distinction to be made here is between those

multisensory signals that convey the same (i.e. redundant) information vs those giving

information about different, perhaps interdependent, aspects of the same event (cf. Fidell1982, Partan and Marler 1999, Laurienti et al. 2004). With regard to the former situation,

a number of studies have demonstrated benefits of redundant sources of information

presentation.For instance, Selcon et al. (1995) compared the speed with which participants could

detect and respond to the presentation of a number of different combinations of visualand/or auditory spatial and/or verbal missile approach warning signals. They observed

significantly faster response latencies when two to four sources simultaneously provided

the same information than when only a single type of cuing was used. Hence, the

multisensory warning signals used in Selcon et al.’s study would appear to have resulted in

an additive effect on human performance. Brown (2005) also reported facilitation ofstopping responses at a road intersection when drivers received haptic plus auditory verbal

warnings than haptic warning alone. It should, however, be noted that it is unclear

whether the cues in these applied studies facilitated performance simply by priming the

correct response and/or whether they also led to a genuine attentional facilitation effect

(cf. Miller 1982, Spence and Driver 1999). If performance was mainly facilitated due to thepriming of the appropriate response, the findings in these previous studies might have

overestimated the potential values of the warning signals.

534 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

On the other hand, cognitive neuroscience research reported by Spence and Driver

(1999) has shown bimodal audio-visual warning signals to be no more effective than

unimodal auditory signals in capturing a participant’s spatial attention in their laboratorybased cuing study. Note though that in their study, response priming explanations of their

results were ruled out by cuing the participant’s attention in a dimension (left vs right) that

was orthogonal to the response dimension along which participants were required to

respond (up vs down; see Spence and Driver 1997a, see also Ward et al. 1998, Santangelo

and Spence 2007b).

3.1. Limitations of previous studies

It is perhaps worth noting one limitation with the majority of previous studies of

crossmodal attentional capture, namely that they have typically assessed the ability of aspatial cue (presented either unimodally or bimodally) to capture attention under

conditions where the participants’ only task involves responding as rapidly and accurately

to the frequently presented target stimuli (i.e. under single task conditions; see Spence et al.

2004 for a review). This situation is in many ways quite different from the typical situationin which multisensory warning signals would need to be maximally effective, i.e. when an

interface operator is engaged in another highly attention-demanding task and where the

warning signal needs to be sufficiently salient, and attention-grabbing, to break through.

However, in this regard, recent research by Van der Lubbe and Postma (2005) has shown

that unimodal auditory and visual cues can break through and capture a person’s attentioneven under conditions where they are trying to maintain the focus of their spatial attention

elsewhere (although see Gilmer 1961, Santangelo et al. 2007, Santangelo and Spence

2007a, b on this point). It will therefore be particularly interesting in future studies to

investigate whether multisensory warning signals are any more effective in ‘breakingthrough’ and capturing interface operators’ attention than unimodal warning signals when

the interface operators are involved in another highly attention-demanding task.In order to assess the effectiveness of the presentation of multisensory warning signals

in driving, an (unpublished) experiment was conducted whereby the predictive auditory

car horn sounds and vibrotactile stimuli used previously in Ho and Spence (2005a) and Hoet al. (2005b) were presented simultaneously to participants, with the apparatus, materials,

design and procedure otherwise being exactly the same as before (see Ho 2006). Maximal

facilitation following the presentation of the multisensory warning signals was expected if

multisensory integration functions in an additive manner as predicted on the basis of

certain of the findings emerging from cognitive neuroscience research (see Stein andMeredith 1993). On the other hand, it could be argued that the two stimuli might interfere

with each other in that spatial attention had to be divided between the two stimuli when

they were presented simultaneously. If so, a small facilitatory effect (or even an inhibitory

effect) might be expected.Consistent with the previous findings, significant spatial cuing effects were reported in

this multisensory experiment, with participants responding more rapidly to visual target

events presented from the valid than the invalid direction as the multisensory cue

(see Figure 4). In other words, the participants in Ho’s study responded more rapidly to

the target visual driving event when the cue came from the same direction (i.e. when both

events came from the front, or else both came from the back) rather than from oppositedirections (i.e. when the multisensory cue was presented from the front but the critical

visual driving event occurred from the rear, or vice versa). However, between-experiments

Theoretical Issues in Ergonomics Science 535

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

analyses failed to reveal any statistically significant advantage of multisensory cuing overand above that seen in response to unimodal (auditory or tactile) cuing. These resultswould therefore appear to suggest that the use of multisensory warning signals is no betterthan the best of the individual unimodal warning signals (although see Santangelo andSpence 2007b). However, the results also failed to reveal any performance decrement withthe use of multisensory cues, thus suggesting that the use of redundant cues may not harmperformance either (Ho 2006).

Laurienti et al. (2006) recently reported a study looking at the superadditivity ofmultisensory stimuli as a function of ageing. They found that although sensory sensitivitydeclined with age, the superadditive effects of multisensory stimuli were of a much largermagnitude in older than younger adults. Taken together, these results hint at the potentialuse of multisensory stimuli to complement sensory perception and enhance behaviouralperformance, particularly in the elderly (see also Korteling 1990). The use of multisensorycues also has the advantage that if conditions render one of the cue modalities useless(such as in situations with loud background noise masking the auditory cue, cf. Ramseyand Simmons 1993), the remaining cue modality should nevertheless still prove effective(see also Ho et al. 2007).

There is now a growing body of empirical research comparing the effectiveness ofunimodal feedback signals to multisensory feedback signals in a variety of differentinterface settings (e.g. Sanders and McCormick 1993, Akamatsu et al. 1995, Vitense et al.2003, Cockburn and Brewster 2005). For example, Akamatsu et al. have shown the use ofcombined tactile, auditory and visual feedback signals in a mouse-pointing task to be nomore effective than the use of a unimodal tactile signal by itself. Their results presumablyreflect the fact that an operator may select the most appropriate sensory channel for

Location of visual event

1000

1100

1200

Front Back

Multisensory cue direction

RT

(m

s)

Front (windscreen) Back (rearview mirror)

Figure 4. Summary results of mean reaction times (RTs) as a function of multisensory cue direction(front vs back) and location of target visual driving event (front vs back) highlighted by Ho (2006).Error bars indicate the standard errors of the means.

536 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

feedback depending on the particular task at hand (see also Burke et al. 1980, Cockburn

and Brewster 2005, Barbagli et al. 2006).

3.2. Assessing the effectiveness of multisensory warning signals

Taken together, it could appear that the potential benefit or interference attributable to the

introduction of multisensory warning signals depends to a large extent on the nature of

the task being performed and on the spatial proximity of the cue and target positions(see Gepshtein et al. 2005). In particular, those studies that have succeeded in

demonstrating multisensory performance enhancement are susceptible to a response bias

interpretation. For instance, in a study by Hirst and Graham (1997) on braking responses

following the presentation of collision avoidance warnings, a lower incidence of collisions

and increased time-to-collision braking scores (calculated by dividing inter-vehicle distanceby relative speed, i.e. a higher score indicates better performance) were reported for the

combined use of an abstract head-up visual display with either an auditory tone of 500Hz

or a speech warning (‘danger ahead’), instead of the combined use of a pictorial visual cue

with either the tone or speech cue, but no difference in overall effectiveness as a function ofwhether the visual cue was combined with a non-speech or speech cue.

A possible alternative account of the mixed multisensory facilitation effects reported in

previous studies is in terms of the time-window-of-integration model put forward by

Colonius and Diederich (2004, see also Miller 1982, Mordkoff and Egeth 1993), given that

the occurrence of multisensory integration is dependent on both the temporal and spatialconfiguration of the stimuli presented (see also Stein and Meredith 1993). Colonius and

Diederich’s two-stage model predicts that for multisensory signals to be maximally

effective, they should be presented close together in time in order for multisensory

integration to occur and that the spatial proximity of the signals will determine the

magnitude and sign of this integration effect (e.g. one signal may inhibit the other if themultisensory signals are presented from distinct spatial locations; see also Stein and

Meredith 1993, Kadunce et al. 1997, Holmes and Spence 2005). Note also that for

multisensory warning signals to be effective, designers should avoid presenting

incongruent information even via different sensory channels (e.g. Selcon et al. 1991).One of the major current debates in driving research relates to the ecological validity of

laboratory-based studies and, in particular, their generalisability to real-world driving

situations. It would certainly be very useful, therefore, in future research to investigate the

extent to which the findings from these laboratory-based experiments scale up to modulate

driving performance under more ecologically valid conditions. In this regard, it isinteresting to note that findings similar to those reported by Ho and colleagues (2005b)

have recently been observed in a driving simulator study where vibrotactile cues were

presented on a driver’s waist in order to warn drivers of potential collisions with the

vehicle in front in a car-following task (Ho et al. 2006a). The drivers in this study were

required to respond as rapidly as possible to the sudden deceleration of the lead vehicle,which had its brake lights disabled (cf. Summala et al. 1998), either with (the ‘valid’ and

‘invalid’ cues in Figure 5) or without vibrotactile cues (none in Figure 5). The vibrotactile

cue could either be presented from the same direction as the critical driving event (i.e. from

the driver’s stomach, labelled the ‘valid’ cue type in Figure 5) or from the opposite

direction (i.e. consisting of the vibration of the driver’s back, labelled the ‘invalid’ cue typein Figure 5). Following the presentation of the spatially informative valid vibrotactile cue

from the front (note that this is likely to be the kind of vibrotactile warning signal that

Theoretical Issues in Ergonomics Science 537

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

would actually be implemented in a car), drivers were shown to be able to respond (i.e. tobrake) more than 400ms faster in response to the sudden braking of the lead vehicle ascompared to trials on which no vibrotactile cue was presented to their stomach(i.e. compare the level of performance seen in the ‘valid’ and ‘none’ cue conditions inFigure 5). An intermediate level of improvement was observed when the drivers were givena vibrotactile cue but it came from the wrong direction (see the ‘invalid’ conditionin Figure 5).

The successful replication of the laboratory research findings when using a high-fidelitydriving simulator not only demonstrates the ecological validity of previous laboratory-based findings, but also the feasibility of implementing vibrotactile warning signals inreal-world applications. What is even more promising here is the fact that the facilitationof driving performance attributable to the presence of the vibrotactile cues is much largerthan might have been expected solely on the basis of previous laboratory-based drivingstudies. It will be interesting in future driving simulator studies to investigate whethersimilar performance benefits can be maintained even when the occurrence of thevibrotactile warning signals is made much less frequent than in the studies that have beenconducted so far (i.e. so that the frequency of occurrence of the warning signals moves alittle closer to what might be expected in a functioning in-car system; cf. Hawkes and Loeb1961, 1962, Green 2000). For reference, it is also worth remembering here that Suetomiand Kido (1997) have estimated that a system that reduced the latency of a driver’sresponses by 500ms would reduce rear-end crashes by up to 60%, hence highlighting thepotential importance for real-world driving of Ho et al.’s (2006a) simulator results.

One may, of course, question even the ecological validity of findings emerging fromdriving simulator studies, which, to some extent, once again represent a controlledexperimental environment with different demands (e.g. regarding the perception of riskand safety) than those found in real-world driving. Critics of simulator studies are typicallyconcerned that such differences in demand characteristics may result in differentbehaviours on the part of drivers (e.g. Alm and Nilsson 1995) and also point to thelack of non-visual motion cues in the simulator environment (e.g. Ohita and Komatsu1991). For instance, Green (2000) highlighted the fact that the reaction times (RTs)reported in driving simulator studies are generally faster than those recorded on the road

1000

1100

1200

1300

1400

1500

NoneInvalidValid

Cue type

Mea

n R

T (

ms)

0

2

4

6

8

10

12

14

16

Dis

tan

ce (

m)

RT Shortest headway Lateral deviation

Figure 5. Summary graph of mean braking reaction time (RT) (ms), shortest headway (m) andlateral deviation from the centre of the road (m) results from Ho et al.’s (2006a) recentdriving simulator study plotted as a function of cue type. Error bars indicate the standard errors ofthe means.

538 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

(see also McGehee et al. 2000), but that both methods have limitations in terms of theirecological validity, as compared to the best method of naturalistic observation. However,it should be noted that a number of researchers have argued that driving simulator studiesare both more ethical and more cost effective than on-road testing (see Reed and Green1999, Haigney and Westerman 2001). Although the applicability of driving simulatorstudies to real-world driving should never be asserted unquestioningly, simulated drivingnevertheless offers an acceptable test environment for the better understanding of driverbehaviour (cf. McLane and Wierwille 1975, Kemeny and Panerai 2003).

3.3. Olfactory signals in driving

Finally, it is worth briefly considering other classes of multisensory warning signals thatmay be used in a driving situation, such as, for example, olfactory signals (e.g. seeDeatherage 1972, Washburn et al. 2003, Grayhem et al. 2005, RAC Foundation 2005).Grayhem and colleagues recently reported that the presentation of cinnamon andpeppermint odours could lead to improved alertness during simulated driving, as indexedby subjective ratings in various assessment inventories. Such results suggest a potentiallynovel and ‘subtle’ means of keeping drowsy drivers alert (see also Bounds 1996, Baron andKalsher 1998, Ho and Spence 2005b, Schuler and Raudenbush 2005, Rinspeed undated).

It is possible that ‘unpleasant’ odours, such as the smell of perspiration (i.e. bodyodour) given off by an anxious person, may be even more effective than pleasant odours inalerting a person (see Prehn et al. 2006 for recent work looking at the effects of thepresentation of chemosensory anxiety signals on the startle reflex; see also Chen et al.2006). However, given that recent research has shown that odours appear to be minimallyeffective once people have fallen asleep (e.g. Badia et al. 1990, Carskadon and Herz 2004),further research on the differential effects of various olfactory stimuli on humanperformance at the border between wakefulness and sleep will be needed before thewidespread use of olfactory alerting signals in cars could be recommended.

4. A cognitive neuroscience perspective on multisensory event perception

Recent research suggests that the human brain treats the stimuli (and events) occurring indifferent regions of space differently, which may have a number of important (but as yetunrealised) implications for the design of effective multisensory warning signals.In particular, the brain tends to treat the stimuli occurring in peripersonal space asbeing far more behaviourally relevant and demanding of attention than these stimulioccurring in extrapersonal space (see Rizzolatti et al. 1997, Previc 1998, 2000), presumablybecause they are potentially more immediately threatening and/or dangerous. Whiletraditional warning signals have typically either been presented in extrapersonal space, orelse at the far reaches of peripersonal space, it is proposed here that warning signals maybe more effective (in terms of enhancing the situation awareness and subsequentresponding of drivers and other interface operators) if they are presented in near-peripersonal space instead (see also Previc 2000).

4.1. The space around a driver

In the context of developing novel classes of multisensory warning signals for drivers, it isinteresting to note that the region (or bubble) of protective space built up around

Theoretical Issues in Ergonomics Science 539

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

(i.e. surrounding) the human body for protective purposes has been argued to extend

furthest in the direction of sight (e.g. Horowitz et al. 1964, Hall 1966, Dosey and Meisels

1969), and that it has been proposed that it expands under those conditions where anindividual feels threatened (e.g. Felipe and Sommer 1966, Dosey and Meisels 1969).

This notion of a margin of safety (i.e. a protective space) is similar to the margin of safety

(or flight zone) observed in many animals (e.g. Hediger 1955). However, it is only by

understanding the underlying nature of multisensory information processing and the

brain’s representation of crossmodal space (as increasingly being revealed by cognitiveneuroscience research; see Spence and Driver 2004) that the neuroergonomists of the

future will be able to start designing multisensory warning signals that are genuinely

optimised for the vagaries and information-processing limitations of the human interface

operator (cf. Loomis and Beall 1998, Sarter et al. 2006).

4.2. Functional regions of space

The findings from a recent series of experiments conducted in the authors’ laboratory,

investigating the efficacy of various spatial auditory and vibrotactile warning signals indirecting a driver’s attention to specific locations within visual driving scenes, have shown

that while people can make front vs back discrimination responses to vibrotactile warning

signals presented on their body faster than to auditory signals originating from farther

away (Ho et al. 2005a; cf. Kitagawa et al. 2005), the benefits of such a speeding-up of

responses can be overridden by the attentional facilitation effects that may result when thecue and target events are actually located in the same functional region of space.

In an attempt to distinguish between the relative contributions of response priming and

attentional facilitation to the effects reported in Ho and Spence (2005a) and Ho et al.’s

(2005b) studies (see Figure 6; Ho et al. 2006b), an orthogonal task design (cf. Spence and

Driver 1997a) was used in a subsequent study. This was because, in experiments using anon-orthogonal design, any facilitatory effect observed could reflect the priming of the

appropriate response by the cues, crossmodal spatial attentional facilitation or some

unknown combination of these two effects (note a similar criticism of Selcon et al.’s (1995),

research on multisensory warning signals earlier). Specifically, the participants in Hoet al.’s (2006b) study were instructed to perform a number plate colour discrimination

task. The dimension on which participants made their discrimination responses (red vs

blue) was thus orthogonal to the spatial dimension of interest (front vs back). This means

that the efficacy of the spatial vibrotactile and auditory cues in orienting a driver’s

attention to the visual driving scenes presented to the front vs rear was unrelated to theparticipants’ task of discriminating whether the colour of the number plate changed to

either red or blue (these two possible colour changes occurred equiprobably). The results

of Ho et al.’s study showed that while the benefits of spatial vibrotactile cues presented to

the torso in orienting a driver’s attention was primarily due to the priming of the

appropriate response, the facilitation effect of the spatial auditory cues (originating fromthe same direction as the visual targets) was also attentional (i.e. perceptual) in nature

(see Figure 6).Additional data from an experiment using the same design as in Ho et al.’s (2006b)

experiment 2, with the exception that the auditory cues were now presented from

loudspeaker cones placed close to the front and back of the participant’s head (i.e. 50 cmfrom the centre of the participant’s head – in peripersonal space), showed a similar pattern

of results to the vibrotactile cuing study (Ho et al. 2006b, experiment 1). Overall, though,

540 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

participants responded more rapidly to auditory stimuli presented closer to their head thanto sounds presented further from their head (by approximately 30ms). This would seem tosuggest that peripersonal warning signals do work at some level in terms of an overallspeeding-up of a driver’s reactions, perhaps because auditory stimuli presented close to thehead (i.e. in peripersonal space) are simply more arousing than stimuli presented furtherfrom the head (as well as arriving at the ears slightly earlier; see Spence and Squire 2003).

Spence and Driver (1999) have also argued that the multisensory warning signals thathave been used to date may not have been optimised for the human perceptual system interms of the relative time of onset of the various sensory components of the warning signal.In particular, research suggests that the human brain may be optimised to respond tomultisensory events occurring at a distance of approximately 10m (e.g. Spence and Squire2003). At this distance, the delay in the transmission of auditory signals (i.e. sound waves)through the air cancels out the visual delays associated with the transduction of light at theretina and so both signals are thought to arrive centrally in the brain at the same time(Poppel 1988). As such, there are reasons to believe that humans may respond moreefficiently to audio-visual warning signals presented from close to the interface operator ifthey are presented asynchronously such that the auditory signal is delayed somewhat withregard to the visual signal (i.e. to simulate the optimal arrival time of an event actuallypresented at a distance of 10m; cf. Spence and Driver 1999, Chan and Chan 2006).A similar argument can be made with regard to the presentation of bimodal visuotactilewarnings, given that variable delays in the arrival time of vibrotactile stimuli(when compared to visual stimuli) are associated with their transmission through thehuman body (see Harrar and Harris 2005), depending upon the distance of the stimulationsite from the person’s brain (von Bekesy 1963, Halliday and Mingay 1964, Bergenheim

Attentional facilitation

Response priming Vibrotactile cues

Auditory carhorn sounds

Figure 6. A schematic diagram of the distinction between response priming and attentionalfacilitation effects of the presentation of auditory or vibrotactile cues to a driver to inform of animpeding collision with the car in front. The idea being that an auditory or vibrotactile cue presentedfrom in front of the driver might facilitate their braking response following the onset of braking inthe car in front because of either (or both) of two possible mechanisms. First, the warningsignal might lead to a crossmodal shift of spatial attention resulting in the driver processing thevisual information from the car in front more rapidly than would otherwise be the case.Alternatively, however, the driver might simply use the warning signal in order to preparethe appropriate behavioural response (in this case, transferring his/her foot onto the brake pedal asrapidly as possible).

Theoretical Issues in Ergonomics Science 541

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

et al. 1996). Future research therefore needs to be conducted in order to examine whetherthe transmission latency associated with vibrating distal body sites, such as the feet, canbe overcome by, for instance, any compatibility benefits in terms of responding(e.g. foot responses).

4.3. Multisensory warning signals from different distances

Taken together, the results of these recent studies suggest that while warning signalspresented from close to a driver’s head/body may be more effective in alerting orgenerating a near-automatic driver response, these warning signals do not necessarily leadto any attentional facilitation of the processing of events occurring in the spatial directionindicated by the signals. When these results are taken together with the latest cognitiveneuroscience research findings, they would appear to suggest that the combined use ofsignals presented close to the head and/or body of a driver (or an interface operator) inorder to alert/arouse them, together with signals presented spatially (either virtually oractually) to enhance subsequent processing of any events occurring in that region of space(e.g. a touch on body together with virtual sound indicating the appropriate source of theevent requiring a driver’s attention; cf. Lee et al. 1999, Previc 2000, Menning et al. 2005)may prove to constitute a particularly effective warning signal. The introduction ofspatially presented multisensory warning signals will elicit facilitated attentionaldeployment in both the near and far regions of space, over and above any attentioneffects typically elicited by traditional multisensory warning signals that act primarily as aredundant information source. It should be noted that while the optimal multisensoryintegration account (cf. Stein and Meredith 1993, Colonius and Diederich 2004) predictsthat stimuli should originate from the same spatial location, the account outlined hereinstead predicts that the best effects may be observed when stimuli occur near to the body(i.e. in a different position from the potentially dangerous road event that a driver urgentlyneeds to respond to).

5. Conclusions

Recent cognitive neuroscience research has shown that stimuli occurring in near-peripersonal space are processed by the human brain differently from stimuli presentedin extrapersonal space. Peripersonal stimuli appear to demand an immediate reaction byan organism because they may represent a potential threat or danger. The challenge forthose attempting to design the most effective warning signals for the vehicles of the futurewill be to find ways in which to make the distal events occurring in far extrapersonal space(i.e. the space far away from the driver, outside of the peripersonal space encompassed bythe car’s interior; see Riddoch 1941, p. 221, Horowitz et al. 1964, p. 655, Paillard 1993,p. 40) somehow more behaviourally relevant to the driver (see Gibson and Crooks 1938).The neuroergonomic solution outlined here involves designing multisensory warningsignals that promote rapid (possibly even automatic) and intuitive behavioural responsesby means of a better understanding of the cognitive neuroscience underlying multisensoryinformation processing in humans.

On the basis of the data reviewed here, it would seem likely that any such futurewarning signals, or at least some component of those signals, will need to be presented innear-peripersonal space (hence the notion of ‘peripersonal warning signals’). However,some component of the signals may well also have to be presented in extrapersonal space

542 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

in order to maximise any attentional facilitation that will help the driver to perceive the

distal road events more rapidly and accurately. Fortunately, the available neuroscienceevidence provides a number of clues regarding how such multisensory warning signalscould be configured so as to be maximally effective (such as sounds are more effective

when presented from close behind the head than in front, and that complex auditory cuesappear to be more effective than the pure tones often used in many contemporary tonal

warning signals; see also Bliss 1997, Kitagawa et al. 2005, Kitagawa and Spence 2006,Zampini et al. 2007).

References

Abe, G. and Richardson, J., 2004. The effect of alarm timing on driver behaviour: an investigation of

differences in driver trust and response to alarms according to alarm timing. Transportation

Research Part F: Traffic Psychology and Behaviour, 7, 307–322.

Abe, G. and Richardson, J., 2005. The influence of alarm timing on braking response and driver

trust in low speed driving. Safety Science, 43, 639–654.Abe, G. and Richardson, J., 2006. Alarm timing, trust and driver expectation for forward collision

warning systems. Applied Ergonomics, 37, 577–586.Akamatsu, M., MacKenzie, I.S., and Hasbroucq, T., 1995. A comparison of tactile, auditory, and

visual feedback in a pointing task using a mouse-type device. Ergonomics, 38, 816–827.Alm, H. and Nilsson, L., 1995. The effects of a mobile telephone task on driver behaviour in a car

following situation. Accident Analysis and Prevention, 27, 707–715.Arrabito, G.R., Mondor, T.A., and Kent, K.J., 2004. Judging the urgency of non-verbal auditory

alarms: a case study. Ergonomics, 47, 821–849.

Ashley, S., 2001. Driving the info highway. Scientific American, 285 (4), 44–50.Badia, P., et al., 1990. Responsiveness to olfactory stimuli presented in sleep. Physiology and

Behavior, 48, 87–90.Baker, R.A., Ware, J.R., and Sipowicz, R.R., 1962. Vigilance: a comparison in auditory, visual, and

combined audio-visual tasks. Canadian Journal of Psychology, 16, 192–198.Ballas, J.A., 1993. Common factors in the identification of an assortment of brief everyday sounds.

Journal of Experimental Psychology: Human Perception and Performance, 19, 250–267.Ballas, J.A., 1994. Delivery of information through sound. In: G. Kramer, ed. Auditory display:

sonification, audification and auditory interfaces. Reading, MA: Addison-Wesley, 79–94.

Barbagli, F., et al., 2006. Haptic discrimination of force direction and the influence of visual

information. ACM Transactions on Applied Perception, 3, 125–135.

Baron, R.A. and Kalsher, M.J., 1998. Effects of a pleasant ambient fragrance on simulated driving

performance: the sweet smell of . . . safety? Environment and Behavior, 30, 535–552.Begault, D.R., 1993. Head-up auditory displays for traffic collision avoidance system advisories: a

preliminary investigation. Human Factors, 35, 707–717.Begault, D.R., 1994. 3-D sound for virtual reality and multimedia. Cambridge, MA: Academic Press.Begault, D.R. and Wenzel, E.M., 1990. Techniques and applications for binaural sound manipulation

in human–machine interfaces. NASA Technical Memorandum No. 102279. Moffett Field, CA:

NASA Ames Research Center.Belz, S.M., Robinson, G.S., and Casali, J.G., 1999. A new class of auditory warning signals for

complex systems: auditory icons. Human Factors, 41, 608–618.Benali-Khoudja, M., et al., 2004. Tactile interfaces: a state-of-the-art survey. Proceedings of the 35th

international symposium on robotics. ISR 2004, 23–26 March, Paris, 1–9.

Bergenheim, M., et al., 1996. Experimental evidence for a sensory synchronisation of sensory

information to conscious experience. In: S.R. Hameroff, A.W. Kaszniak and A.S. Scott, eds.

Toward a science of consciousness: the first Tucson discussions and debates. Cambridge, MA:

MIT Press, 303–310.

Theoretical Issues in Ergonomics Science 543

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Blattner, M.M., Sumikawa, D.A., and Greenberg, R.M., 1989. Earcons and icons: their structure

and common design principles. Human–Computer Interaction, 4, 11–44.Bliss, J.P., 1997. Alarm reaction patterns by pilots as a function of reaction modality. International

Journal of Aviation Psychology, 7, 1–14.Bliss, J.P. and Acton, S.A., 2003. Alarm mistrust in automobiles: how collision alarm reliability

affects driving. Applied Ergonomics, 34, 499–509.Bounds, W., 1996. Sounds and scents to jolt drowsy drivers. Wall Street Journal, 6 May, B1.

Brewster, S. and Brown, L.M., 2004. Tactons: structured tactile messages for non-visual

information display. Proceedings of the 5th Australasian user interface conference

(AUIC2004), 15–23.Breznitz, S., 1984. Cry wolf: the psychology of false alarms. Hillsdale, NJ: Erlbaum.

Bristow, D., et al., 2005. Blinking suppresses the neural response to unchanging retinal stimulation.

Current Biology, 15, 1296–1300.Broadbent, D.E., 1958. Perception and communication. Elmsford, NJ: Pergamon.

Bronkhorst, A.W., Veltman, J.A., and van Breda, L., 1996. Application of a three-dimensional

auditory display in flight task. Human Factors, 38, 23–33.

Brown, R.L., Galloway, W.D., and Gildersleeve, K.R., 1965. Effects of intense noise on processing

of cutaneous information of varying complexity. Perceptual and Motor Skills, 20, 749–754.

Brown, S.B., 2005. Effects of haptic and auditory warnings on driver intersection behaviour and

perception. Unpublished Master’s thesis. Department of Industrial and Systems Engineering,

Virginia Tech.Brown, S.B., et al., 2005. Effect of haptic brake pulse warnings on driver behavior during an

intersection approach. Proceedings of the Human Factors and Ergonomics Society 49th annual

meeting. Santa Monica, CA: HFES, 1892–1896.

Bruce, D., Boehm-Davis, D.A. and Mahach, K., 2000. In-vehicle auditory display of symbolic

information. Proceedings of the IEA 2000/HFES 2000 congress, 3-230–3-233.

Burke, M.W., Gilson, R.D., and Jagacinski, R.J., 1980. Multi-modal information processing for

visual workload relief. Ergonomics, 23, 961–975.

Burnett, G.E. and Joyner, S.M., 1997. An assessment of moving map and symbol-based route

guidance systems. In: Y.I. Noy ed. Ergonomics and safety of intelligent driver interfaces.

Mahwah, NJ: Lawrence Erlbaum, 115–137.Burnett, G.E. and Porter, J.M., 2001. Ubiquitous computing within cars: designing controls for non-

visual use. International Journal of Human–Computer Studies, 55, 521–531.Burrows, A.A., 1962. Choice response in quiet and loaded channels with verbal and non-verbal

auditory stimuli. Human Factors, 4, 187–192.Burt, J.L., et al., 1995. A psychophysiological evaluation of the perceived urgency of auditory

warning signals. Ergonomics, 38, 2327–2340.Cabrera, D., Ferguson, S., and Laing, G., 2005. Development of auditory alerts for air traffic

control consoles. Audio Engineering Society, 119th convention, 1–21.Calvert, G.A., Spence, C. and Stein, B.E., eds., 2004. The handbook of multisensory processes.

Cambridge, MA: MIT Press.Campbell, J.L., et al., 1996. Investigation of alternative displays for side collision avoidance systems.

Report No. DOT HS 808 579. Seattle, WA: Battelle Human Factors Transportation Center.Carskadon, M.A. and Herz, R.S., 2004. Minimal olfactory perception during sleep: why odor alarms

will not work for humans. Sleep, 27, 402–405.

Catchpole, K.R., McKeown, J.D., and Withington, D.J., 2004. Localizable auditory warning pulses.

Ergonomics, 47, 748–771.

Chambrin, M.-C., 2001. Alarms in the intensive care: how can the number of false alarms be

reduced? Critical Care, 5, 184–188.

Chan, A.H.S. and Chan, K.W.L., 2006. Synchronous and asynchronous presentation of auditory

and visual signals: implications for control console design. Applied Ergonomics, 37, 131–140.

Chen, D., Katdare, A., and Lucas, N., 2006. Chemosignals of fear enhance cognitive performance in

humans. Chemical Senses, 31, 415–423.

544 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Cnossen, F., Meijman, T., and Rothengatter, T., 2004. Adaptive strategy changes as a function of

task demands: a study of car drivers. Ergonomics, 47, 218–236.Cockburn, A. and Brewster, S., 2005. Multimodal feedback for the acquisition of small targets.

Ergonomics, 48, 1129–1150.Colonius, H. and Diederich, A., 2004. Multisensory interaction in saccadic reaction time: a time-

window-of-integration model. Journal of Cognitive Neuroscience, 16, 1000–1009.Deatherage, B.H., 1972. Auditory and other sensory forms of information presentation.

In: H.P. Van Cott and R.G. Kinkade, eds. Human engineering guide to equipment design.

New York: John Wiley and Sons, 123–160.Dewar, R.E., 1988. In-vehicle information and driver overload. International Journal of Vehicle

Design, 9, 557–564.Dingus, T.A., et al., 1997. Effects of age, system experience, and navigation technique on driving

with an advanced traveler information system. Human Factors, 39, 177–199.Doll, T.J. and Folds, D.J., 1986. Auditory signals in military aircraft: ergonomic principles vs

practice. Applied Ergonomics, 17, 257–264.

Dosey, M.A. and Meisels, M., 1969. Personal space and self-protection. Journal of Personal and

Social Psychology, 11, 93–97.

Doyle, M.C. and Snowden, R.J., 1999. The effect of auditory warning signals on visual

target identification. In: D. Harris, ed. Engineering psychology and cognitive ergonomics,

Vol. 4: Job design, product design and human-computer interaction. Aldershot, UK: Ashgate,

245–251.Driver, J., 2001. A selective review of selective attention research from the past century. British

Journal of Psychology, 92, 53–78.Driving Standards Agency, 2004. The highway code. London: Department for Transport.

Dukic, T., Hanson, L., and Falkmer, T., 2006. Effect of drivers’ age and push button

locations on visual time off road, steering wheel deviation and safety perception.

Ergonomics, 49, 78–92.Edworthy, J. and Hellier, E., 2006. Complex nonverbal auditory signals and speech warnings.

In: M.S. Wogalter, ed. Handbook of warnings. Mahwah, NJ: Lawrence Erlbaum, 199–220.

Edworthy, J., Loxley, S., and Dennis, I., 1991. Improving auditory warning design:

relationship between warning sound parameters and perceived urgency. Human Factors, 33,

205–231.Evans, L., 1991. Traffic safety and the driver. New York: Van Nostrand Reinhold.Fairclough, S.H., Ashby, M.C., and Parkes, A.M., 1993. In-vehicle displays, visual workload and

usability evaluation. In: A.G. Gale, I.D. Brown, C.M. Haslegrave, H.W. Kruysse and

S.P. Taylor, eds. Vision in vehicles – IV. Amsterdam: Elsevier Science, 245–254.

Fastl, H., 2005. Psycho-acoustics and sound quality. In: J. Blauert, ed. Communication acoustics.

New York: Springer-Verlag, 139–162.Felipe, N.J. and Sommer, R., 1966. Invasions of personal space. Social Problems, 14, 206–214.

Fenton, R.E. and Montano, W.B., 1968. An intervehicular spacing display for improved car-

following performance. IEEE Transactions on Man–Machine Systems, MMS-9, 29–35.

Fidell, S., 1982. Comments on Mulligan and Shaw’s ‘Multimodal signal detection: independent

decisions vs integration’. Perception and Psychophysics, 31, 90.Fidell, S. and Teffeteller, S., 1981. Scaling the annoyance of intrusive sounds. Journal of Sound and

Vibration, 78, 291–298.Fitch, G.M., et al., 2007. Toward developing an approach for alerting drivers to the direction of

a crash threat. Human Factors, 49, 710–720.Fitch, W.T. and Kramer, G., eds., 1994. Sonifying the body electric: superiority of an auditory over

a visual display in a complex, multivariate system. Reading, MA: addison-Wesley.Gaver, W.W., 1986. Auditory icons: using sound in computer interfaces. Human–Computer

Interaction, 2, 167–177.

Gaver, W.W., 1989. The SonicFinder: an interface that uses auditory icons. Human–Computer

Interaction, 4, 67–94.

Theoretical Issues in Ergonomics Science 545

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Gaver, W.W., Smith, R.B. and O’Shea, T., 1991. Effective sounds in complex systems:

the ARKOLA simulation. Proceedings of the SIGCHI conference on human factors in

computing systems: reaching through technology, 85–90.Geldard, F.A., 1960. Some neglected possibilities of communication. Science, 131, 1583–1588.

Geldard, F.A., 1961. Cutaneous channels of communication. In: W.A. Rosenblith, ed. Sensory

communications. Cambridge, MA: MIT Press, 73–87.Geldard, F.A. and Sherrick, C.E., 1965. Multiple cutaneous stimulation: the discrimination of

vibratory patterns. Journal of the Acoustical Society of America, 37, 797–801.Gepshtein, S., et al., 2005. The combination of vision and touch depends on spatial proximity.

Journal of Vision, 5, 1013–1023.Gibson, J.J. and Crooks, L.E., 1938. A theoretical field-analysis of automobile-driving. American

Journal of Psychology, 51, 453–471.Gilliland, K. and Schlegel, R.E., 1994. Tactile stimulation of the human head for information

display. Human Factors, 36, 700–717.Gilmer, B.v.H., 1961. Toward cutaneous electro-pulse communication. Journal of Psychology, 52,

211–222.Godthelp, H. and Schumann, J., 1993. Intelligent accelerator: an element of driver support.

In: A.M. Parkes and S. Franzen, eds. Driving future vehicles. London: Taylor & Francis,

265–274.Graham, R., 1999. Use of auditory icons as emergency warnings: evaluation within a vehicle

collision avoidance application. Ergonomics, 42, 1233–1248.Grayhem, R., et al., 2005. Effects of odor administration on driving performance, safety, alertness,

and fatigue. Poster presented at the 27th annual meeting of the Association for Chemoreception

Sciences, Sarasota, FL.Green, M., 2000. ‘How long does it take to stop?’ Methodological analysis of driver perception-

brake times. Transportation Human Factors, 2, 195–216.Gregg, L.W., 1960. Some coding problems in the design of a cutaneous communication channel. US

Army Medical Research Report, 424, 85–102.

Guillaume, A., et al., 2003. Judging the urgency of nonvocal auditory warning signals: perceptual

and cognitive processes. Journal of Experimental Psychology: Applied, 9, 196–212.

Haas, E.C. and Casali, J.G., 1995. Perceived urgency and response time to multi-tone

and frequency-modulated warning signals in broadband noise. Ergonomics, 38,

2313–2326.

Haas, E.C. and Edworthy, J., 1996. Designing urgency into auditory warnings using pitch, speed and

loudness. Computing and Control Engineering Journal, 7, 193–198.

Haas, M.W., 1995. Virtually-augmented interfaces for tactical aircraft. Biological Psychology, 40,

229–238.Haigney, D. and Westerman, S.J., 2001. Mobile (cellular) phone use and driving: a critical review of

research methodology. Ergonomics, 44, 132–143.Hall, E.T., 1966. The hidden dimension: man’s use of space in public and private. London: Bodley

Head.Halliday, A.M. and Mingay, R., 1964. On the resolution of small time intervals and the effect of

conduction delays on the judgement of simultaneity. Quarterly Journal of Experimental

Psychology, 16, 35–46.Harrar, V. and Harris, L.R., 2005. Simultaneity constancy: detecting events with touch and vision.

Experimental Brain Research, 166, 465–473.Hawkes, G.R. and Loeb, M., 1961. Vigilance for cutaneous and auditory signals. Journal of Auditory

Research, 4, 272–284.Hawkes, G.R. and Loeb, M., 1962. Vigilance for cutaneous and auditory stimuli as a function of

intersignal interval and signal strength. Journal of Psychology, 53, 211–218.Hediger, H., 1955. Studies of the psychology and behaviour of captive animals in zoos and circuses.

New York: Criterion Books.

546 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Hellier, E.J., Edworthy, J., and Dennis, I., 1993. Improving auditory warning design: quantifying

and predicting the effects of different warning parameters on perceived urgency.

Human Factors, 35, 693–706.Hempel, T. and Altinsoy, E., 2005. Multimodal user interfaces: designing media for the auditory and

the tactile channel. In: R.W. Proctor and K.P.L. Vu, eds. Handbook of human factors in web

design. Mahwah, NJ: Lawrence Erlbaum, 134–155.Hennessy, J.R., 1966. Cutaneous sensitivity communications. Human Factors, 8, 463–469.

Hills, B.L., 1980. Vision, visibility, and perception in driving. Perception, 9, 183–216.Hirsch, J., 1974. Rate control in man-machine systems. In: F.A. Geldard, ed. Cutaneous

communication systems as devices. Austin, TX: Psychonomic Society, 65–72.Hirst, S. and Graham, R., 1997. The format and presentation of collision warnings. In: Y.I. Noy, ed.

Ergonomics and safety of intelligent driver interfaces. Mahwah, NJ: Lawrence Erlbaum,

203–219.Ho, C., 2006. Multisensory aspects of the spatial cuing of driver attention. Unpublished doctoral

dissertation. Department of Experimental Psychology, University of Oxford, UK.Ho, C., Reed, N., and Spence, C., 2006a. Assessing the effectiveness of ‘intuitive’ vibrotactile

warning signals in preventing front-to-rear-end collisions in a driving simulator. Accident

Analysis and Prevention, 38, 989–996.Ho, C., Reed, N., and Spence, C., 2007. Multisensory in-car warning signals for collision avoidance.

Human Factors, 49, 1107–1114.Ho, C., Santangelo, V. and Spence, C., submitted. Multisensory warning signals: when spatial

location matters. Journal of Experimental Psychology: Applied.Ho, C. and Spence, C., 2005a. Assessing the effectiveness of various auditory cues in capturing a

driver’s visual attention. Journal of Experimental Psychology: Applied, 11, 157–174.Ho, C. and Spence, C., 2005b. Olfactory facilitation of dual-task performance. Neuroscience Letters,

389, 35–40.Ho, C. and Spence, C., 2006. Verbal interface design: do verbal directional cues automatically orient

visual spatial attention? Computers in Human Behavior, 22, 733–748.

Ho, C. and Spence, C., 2007. Head orientation biases tactile localization. Brain Research, 1144C,

136–141.

Ho, C. and Spence, C., in press. The multisensory driver: implications for ergonomic car interface

design. Aldershot: Ashgate.

Ho, C., Spence, C. and Tan, H.Z., 2005a. Warning signals go multisensory. Proceedings of HCI

International 2005, 9, Paper No. 2284, 1–10.Ho, C., Tan, H.Z., and Spence, C., 2005b. Using spatial vibrotactile cues to direct visual

attention in driving scenes. Transportation Research Part F: Traffic Psychology and Behaviour,

8, 397–412.Ho, C., Tan, H.Z., and Spence, C., 2006b. The differential effect of vibrotactile and auditory cues on

visual spatial attention. Ergonomics, 49, 724–738.Holmes, N.P. and Spence, C., 2005. Multisensory integration: space, time, and superadditivity.

Current Biology, 15, R762–R764.Horowitz, A.D. and Dingus, T.A., 1992. Warning signal design: a key human factors issue in an

in-vehicle front-to-rear-end collision warning system. Proceedings of the Human Factors and

Ergonomics Society 36th annual meeting. Santa Monica, CA: HFES, 1011–1013.Horowitz, M.J., Duff, D.F., and Stratton, L.O., 1964. Body-buffer zone: exploration of personal

space. Archive of General Psychiatry, 11, 651–656.Howell, W.C. and Briggs, G.E., 1959. An initial evaluation of a vibrotactile display in complex control

tasks. Tech. Rep. No. (813)-6. Columbus, OH: the Ohio State University Research Foundation.Hughes, P.K. and Cole, B.L., 1986. What attracts attention when driving? Ergonomics, 29, 377–391.Humphrey, C.E., 1952. Auditory displays: I. Spatial orientation by means of auditory signals – an

exploratory study. Report No. APL/JHU-TG-122. Silver Spring, Maryland: applied Physics

Laboratory, The Johns Hopkins University.

Theoretical Issues in Ergonomics Science 547

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Hunter, W.W., Bundy, H.L., and Daniel, R.B., 1976. An assessment of the effectiveness of the

following-too-closely monitor. Chapel Hill, NC: University of North Carolina: Highway Safety

Research Center.Jagacinski, R.J., Miller, D.P., and Gilson, R.D., 1979. A comparison of kinesthetic-tactual and

visual displays via a critical tracking task. Human Factors, 21, 79–86.Janssen, W. and Nilsson, L., 1993. Behavioural effects of driver support. In: A.M. Parkes and

S. Franzen, eds. Driving future vehicles. London: Taylor & Francis, 147–155.

Janssen, W.H. and Thomas, H., 1997. In-vehicle collision avoidance support under adverse visibility

conditions. In: Y.I. Noy, ed. Ergonomics and safety of intelligent driver interfaces. Mahwah,

NJ: Lawrence Erlbaum, 221–229.

Johansson, G. and Rumar, K., 1971. Drivers’ brake reaction times. Human Factors, 13, 23–27.Johnson, T.L. and Shapiro, K.L., 1989. Attention to auditory and peripheral visual stimuli: effects of

arousal and predictability. Acta Psychologica, 72, 233–245.Jones, D., 1989. The sonic interface. In: M.J. Smith and G. Salvendy, eds. Work with computers:

organizational, management, stress and health aspects. Amsterdam: Elsevier, 382–388.Jordan, P.W. and Johnson, G.I., 1993. Exploring mental workload via TLX: the case of

operating a car stereo while driving. In: A.G. Gale, I.D. Brown, C.M. Haslegrave,

H.W. Kruysse and S.P. Taylor, eds. Vision in vehicles – IV. Amsterdam: Elsevier

Science, 255–262.Jordan, T.C., 1972. Characteristics of visual and proprioceptive response times in the learning of a

motor skill. Quarterly Journal of Experimental Psychology, 24, 536–543.Kadunce, D.C., et al., 1997. Mechanisms of within- and cross-modality suppression in the superior

colliculus. Journal of Neurophysiology, 78, 2834–2847.Kantowitz, B.H., Triggs, T.J., and Barnes, V., 1990. Stimulus-response compatibility and human

factors. In: R.W. Proctor and T.G. Reeve, eds. Stimulus-response compatibility: an integrated

perspective. Amsterdam: North-Holland, 365–388.Kappauf, W.E. and Powe, W.E., 1959. Performance decrement on an audio-visual checking task.

Journal of Experimental Psychology, 57, 49–56.Keller, P. and Stevens, C., 2004. Meaning from environmental sounds: types of signal-referent

relations and their effect on recognizing auditory icons. Journal of Experimental Psychology:

Applied, 10, 3–12.Kemeny, A. and Panerai, F., 2003. Evaluating perception in driving simulation experiments.

Trends in Cognitive Sciences, 7, 31–37.

Kenny, T., et al., 2004. Integrated vehicle instrument simulations – i-ViS initial design philosophy.

Imeche 2004, 93–102.King, R.A. and Corso, G.M., 1993. Auditory displays: if they are so useful, why are they turned off?

Proceedings of the Human Factors and Ergonomics Society 37th annual meeting. Santa Monica,

CA: HFES, 549–553.Kitagawa, N. and Spence, C., 2006. Audiotactile multisensory interactions in human information

processing. Japanese Psychological Research, 48, 158–173.

Kitagawa, N., Zampini, M., and Spence, C., 2005. Audiotactile interactions in near and far space.

Experimental Brain Research, 166, 528–537.Korteling, J.E., 1990. Perception-response speed and driving capabilities of brain-damaged and older

drivers. Human Factors, 32, 95–108.Krishnan, H., et al., 2001. Rear-end collision warning system: design and evaluation via simulation.

Transportation Research Record, 1759, 52–60.Kume, Y., et al., 1998. Information transmission through soles by vibro-tactile stimulation.

Transactions of the Virtual Reality Society of Japan, 3, 83–88.Lansdown, T., Brook-Carter, N., and Kersloot, T., 2004. Distraction from multiple in-vehicle

secondary tasks: vehicle performance and mental workload implications. Ergonomics, 47,

91–104.Laurienti, P.J., et al., 2004. Semantic congruence is a critical factor in multisensory behavioral

performance. Experimental Brain Research, 158, 405–414.

548 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Laurienti, P.J., et al., 2006. Enhanced multisensory integration in older adults. Neurobiology of

Aging, 27, 1155–1163.Lazarus, H. and Hoge, H., 1986. Industrial safety: acoustic signals for danger situations in factories.

Applied Ergonomics, 17, 41–46.

Lee, J.D., Gore, B.F., and Campbell, J.L., 1999. Display alternatives for in-vehicle warning and sign

information: message style, location, and modality. Transportation Human Factors, 1,

347–375.

Lee, J.D., Hoffman, J.D., and Hayes, E., 2004. Collision warning design to mitigate driver

distraction. Chi 2004, 6, 65–72.Lee, J.D., et al., 2001. Speech-based interaction with in-vehicle computers: the effect of speech-based

e-mail on drivers’ attention to the roadway. Human Factors, 43, 631–640.Lee, J.D., et al., 2006. Effects of adaptive cruise control and alert modality on driver performance.

Transportation Research Record, 1980, 49–56.Lindeman, R.W. and Yanagida, Y., 2003. Empirical studies for effective near-field haptics in virtual

environments. Proceedings of the IEEE Virtual Reality, 287–288.Lindeman, R.W., et al., (2003). Effective vibrotactile cueing in a visual search task. Proceedings of

the 9th IFIP TC13 international conference on human–computer interaction (INTERACT

2003), 89–96.Liu, Y.-C., 2001. Comparative study of the effects of auditory, visual and multimodal displays on

drivers’ performance in advanced traveller information systems. Ergonomics, 44, 425–442.

Loomis, J.M. and Beall, A.C., 1998. Visually controlled locomotion: its dependence on optic flow,

three-dimensional space perception, and cognition. Ecological Psychology, 10, 271–285.Lucas, P.A., 1995. An evaluation of the communicative ability of auditory icons and earcons.

Proceeding of the 2nd international conference on auditory display ICAD ’94, 121–128.

Luo, R.C. and Kay, M.G., 1989. Multisensor integration and fusion in intelligent systems. IEEE

Transactions on Systems, Man, and Cybernetics, 19, 901–931.McAdams, S., 1993. Recognition of sound sources and events. In: S. McAdams and E. Bigand, eds.

Thinking in sound: the cognitive psychology of human audition. Oxford: Clarendon Press,

146–198.McGehee, D.V., Mazzae, E.N., and Baldwin, G.H.S., 2000. Driver reaction time in crash avoidance

research: validation of a driving simulator study on a test track. Proceedings of the IEA 2000/

HFES 2000 congress, 3, 320–323.McGehee, D.V. and Raby, M., 2003. The design and evaluation of snowplow lane awareness system.

HF-PPC 2003–1. Iowa City: University of Iowa Public Policy Center.McGehee, D.V., et al., 2002. The effect of warning timing on collision avoidance behavior in a stationary

lead vehicle scenario.Human performance:models, intelligent vehicle initiative, traveller advisory and

information systems. Washington, DC: Transportation Research Board, 1–7.

McKeown, J.D., 2005. Candidates for within-vehicle auditory displays. Proceedings of ICAD 05 –

eleventh meeting of the international conference on auditory display, 182–189.McKeown, J.D. and Isherwood, S., 2007. Mapping the urgency and pleasantness of speech,

auditory icons, and abstract alarms to their referents within the vehicle. Human Factors, 49,

417–428.McLane, R.C. and Wierwille, W.W., 1975. The influence of motion and audio cues on driver

performance in an automobile simulator. Human Factors, 17, 488–501.Makhsous, M., et al., 2005. Reducing whole-body vibration and musculoskeletal injury with a new

car seat design. Ergonomics, 48, 1183–1199.Mariani, M., 2001. COMUNICAR: designing multimodal interaction for advanced in-vehicle

interfaces. Proceedings of the human factors in transportation, communication, health, and the

workplace, Human Factors and Ergonomics Society Europe chapter annual meeting, 113–120.Mather, G., 2004. Perceptual and cognitive limits on driver information processing. Imeche, 2004,

85–92.Menning, H., et al., 2005. Spatial auditory attention is modulated by tactile priming.

Experimental Brain Research, 164, 41–47.

Theoretical Issues in Ergonomics Science 549

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Miller, J., 1982. Divided attention: evidence for coactivation with redundant signals.

Cognitive Psychology, 14, 247–279.Mollenhauer, M., et al., 1994. The effects of sensory modality and information priority on in-vehicle

signing and information systems. Proceedings of the 38th annual meeting of the Human Factors

and Ergonomics Society. Santa Monica, CA: HFES, 1072–1077.Mordkoff, J.T. and Egeth, H.E., 1993. Response time and accuracy revisited: converging support for

the interactive race model. Journal of Experimental Psychology: Human Perception and

Performance, 19, 981–991.

Mowbray, G.H. and Gebhard, J.W., 1961. Man’s senses as informational channels.

In: H.W. Sinaiko, ed. Selected papers on human factors in the design and use of control

systems. New York: Dover, 115–149.Munns, M., 1971. Ways to alarm pilots. Aerospace Medicine, 42, 731–734.Nagel, S.K., et al., 2005. Beyond sensory substitution – learning the sixth sense. Journal of Neural

Engineering, 2, R13–R26.Nelson, R.J., McCandlish, C.A., and Douglas, V.D., 1990. Reaction times for hand movements

made in response to visual versus vibratory cues. Somatosensory and Motor Research, 7,

337–352.Nicolas, S., 1997. On the speed of different senses and nerve transmission by Hirsch 1862.

Psychological Research, 59, 261–268.Nissan, 2006, April 20. IC tag child protection test results released – drivers find service useful in

alerting them to children in the vicinity. Available from: http://www.nissan-global.com/EN/

NEWS/2006/_STORY/060420–02-e.html [Accessed 28 April 2006].Ohita, H. and Komatsu, H., 1991. Speed perception in driving – comparison with TV observation.

In: A.G. Gale, ed. Vision in vehicles III. Amsterdam: Elsevier, 415–426.Oyer, J. and Hardick, J., 1963. Response of population to optimum warning signal. Final Report No.

SHSLR163. Washington, DC: Office of Civil Defence.Paillard, J., 1993. The hand and the tool: the functional architecture of human technical skills.

In: A. Berthelet and J. Chavaillon, eds. The use of tools by human and non-human primates.

New York: Oxford University Press, 34–46.Parasuraman, R., Hancock, P.A., and Olofinboba, O., 1997. Alarm effectiveness in driver-centred

collision-warning systems. Ergonomics, 40, 390–399.Partan, S. and Marler, P., 1999. Communication goes multimodal. Science, 283, 1272–1273.Patten, C.J.D., et al., 2004. Using mobile telephones: cognitive workload and attention resource

allocation. Accident Analysis and Prevention, 36, 341–350.Poppel, E., 1988. Mindworks: time and conscious experience. Boston: Harcourt.Posner, M.I., Nissen, M.J., and Klein, R.M., 1976. Visual dominance: an information-processing

account of its origins and significance. Psychological Review, 83, 157–171.Prehn, A., et al., 2006. Chemosensory anxiety signals augment the startle reflex in humans.

Neuroscience Letters, 394, 127–130.Previc, F.H., 1998. The neuropsychology of 3-D space. Psychological Bulletin, 124, 123–164.

Previc, F.H., 2000. Neuropsychological guidelines for aircraft control stations. IEEE Engineering in

Medicine and Biology Magazine, 19, 81–88.

Proctor, R.W., et al., 2005. Implications of compatibility and cuing effects for multimodal interfaces.

Proceedings of HCI International 2005, 11, Paper No. 2733, 1–10.RAC Foundation, 2005, 3 June. The scent of danger. Available from: http://www.racfoundation.org/

releases/030605rac.htm [Accessed 4 June 2005].Ramsey, K.L. and Simmons, F.B., 1993. High-powered automobile stereos. Otolaryngology head and

neck surgery, 109, 108–110.Reed, M.P. and Green, P.A., 1999. Comparison of driving performance on-road and in a low-cost

simulator using a concurrent telephone dialling task. Ergonomics, 42, 1015–1037.Riddoch, G., 1941. Phantom limbs and body shape. Brain, 64, 197–222.

Rinspeed, undated. ‘Senso’ – the car that senses the driver. Available from: http://www.rinspeed.com/

pages/cars/senso/pre-senso.htm [Accessed 30 May 2005].

550 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Rizzolatti, G., et al., 1997. The space around us. Science, 277, 190–191.Rockwell, T.H., 1972. Eye movement analysis of visual information acquisition in driving – an

overview. Proceedings of the 6th Conference of the Australian Road Research Board, 6, 316–331.

Rockwell, T.H., 1988. Spare visual capacity in driving – revisited: new empirical results for an old

idea. In: A.G. Gale, M.H. Freeman, C.M. Haslegrave, P. Smith and S.P. Taylor, eds. Vision in

vehicles II. Amsterdam: Elsevier Science, 317–324.

Rodway, P., 2005. The modality shift effect and the effectiveness of warning signals in different

modalities. Acta Psychologica, 120, 199–226.Rumar, K., 1990. The basic driver error: late detection. Ergonomics, 33, 1281–1290.

Rupert, A.H., 2000. An instrumentation solution for reducing spatial disorientation mishaps. IEEE

Engineering in Medicine and Biology Magazine, 19, 71–80.Sanders, M.S. and McCormick, E.J., 1993. Human factors in engineering and design. 7th ed.

New York: McGraw-Hill.Santangelo, V., Belardinelli, M.O., and Spence, C., 2007. The suppression of reflexive visual and

auditory orienting when attention is otherwise engaged. Journal of Experimental Psychology:

Human Perception and Performance, 33, 137–148.Santangelo, V., Ho, C. and Spence, C., 2008. Capturing spatial attention with multisensory cues.

Psychonomic Bulletin and Review, 15, 398–403.Santangelo, V. and Spence, C., 2007a. Assessing the automaticity of reflexive tactile attentional

orienting. Perception, 36, 1497–1505.

Santangelo, V. and Spence, C., 2007b. Multisensory cues capture spatial attention regardless of

perceptual load. Journal of Experimental Psychology: Human Perception and Performance, 33,

1311–1321.

Sarter, N., Ferris, T., and Hameed, S., 2006. Performance effects of crossmodal links in attention: do

they scale to complex environments? Paper presented at the 7th annual meeting of the

International Multisensory Research Forum, Dublin, Ireland.

Schuler, A. and Raudenbush, B., 2005. Effects of intermittent peppermint odor administration on

alertness, mood, mobility, and sleep patterns. Poster presented at the 27th annual meeting of

the Association for Chemoreception Sciences, Sarasota, FL.Schumann, J. and Naab, K., 1992. On the effectiveness of an active steering wheel in critical driving

situations – a proving ground experiment. Proceedings of the conference on road safety

in Europe, 194–208.Schumann, J., et al., 1993. Breaking up open-loop steering control actions the steering wheel as an

active control device. In: A.G. Gale, I.D. Brown, C.M. Haslegrave, H.W. Kruysse and

S.P. Taylor, eds. Vision in vehicles – IV. Amsterdam: Elsevier Science, 321–332.Selcon, S.J., Taylor, R.M., and McKenna, F.P., 1995. Integrating multiple information sources:

using redundancy in the design of warnings. Ergonomics, 38, 2362–2370.Selcon, S.J., Taylor, R.M., and Shadrake, R.A., 1991. Giving the pilot two sources of information:

help or hindrance. In: E. Farmer, ed. Human resource management in aviation. Aldershot, UK:

Avebury Technical, 139–148.Shinar, D., 1978. Psychology on the road: the human factor in traffic safety. New York: Wiley.Simpson, C.A. and Williams, D.H., 1980. Response time effects of alerting tone and semantic

context for synthesized voice cockpit warnings. Human Factors, 22, 319–330.Simpson, C.A., et al., 1987. Speech controls and displays. In: G. Salvendy, ed. Handbook of human

factors. New York: Wiley, 1490–1525.

Sivak, M., 1996. The information that drivers use: is it indeed 90% visual? Perception, 25,

1081–1090.Sklar, A.E. and Sarter, N.B., 1999. Good vibrations: tactile feedback in support of attention

allocation and human–automation coordination in event-driven domains. Human Factors, 41,

543–552.Sommer, R., 1959. Studies in personal space. Sociometry, 22, 247–260.

Sorkin, R.D., 1987. Design of auditory and tactile displays. In: G. Salvendy, ed. Handbook of human

factors. New York: Wiley, 549–576.

Theoretical Issues in Ergonomics Science 551

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Sorkin, R.D., 1988. Why are people turning off our alarms? Journal of the Acoustical Society of

America, 84, 1107–1108.Sorkin, R.D. and Woods, D.D., 1985. Systems with human monitors: a signal detection analysis.

Human–Computer Interaction, 1, 49–75.Sound Alert Technology, undated. Directional sound in aircraft. Available from: http://

www.soundalert.com/pdfs/localizer_aviation.pdf [Accessed 2 December 2005].Spence, C., 2001. Crossmodal attentional capture: a controversy resolved?. In: C.L. Folk and

B.S. Gibson, eds. Attraction, distraction and action: multiple perspectives on attentional capture.

Amsterdam: Elsevier Science, 231–262.Spence, C. and Driver, J., 1997a. Audiovisual links in exogenous covert spatial orienting. Perception

and Psychophysics, 59, 1–22.Spence, C. and Driver, J., 1997b. Cross-modal links in attention between audition, vision, and touch:

implications for interface design. International Journal of Cognitive Ergonomics, 1, 351–373.Spence, C. and Driver, J., 1999. A new approach to the design of multimodal warning signals.

In: D. Harris, ed. Engineering psychology and cognitive ergonomics, Vol. 4: Job design, product

design and human–computer interaction. Aldershot, UK: Ashgate, 455–461.Spence, C. and Driver, J., eds., 2004. Crossmodal space and crossmodal attention. Oxford: Oxford

University Press.Spence, C., McDonald, J., and Driver, J., 2004. Exogenous spatial cuing studies of human

crossmodal attention and multisensory integration. In: C. Spence and J. Driver, eds.

Crossmodal space and crossmodal attention. Oxford: Oxford University Press, 277–320.Spence, C., Nicholls, M.E.R., and Driver, J., 2001. The cost of expecting events in the wrong sensory

modality. Perception and Psychophysics, 63, 330–336.Spence, C. and Read, L., 2003. Speech shadowing while driving: on the difficulty of splitting

attention between eye and ear. Psychological Science, 14, 251–256.Spence, C. and Squire, S., 2003. Multisensory integration: Maintaining the perception of synchrony.

Current Biology, 13, R519–R521.Stanton, N. and Edworthy, J., 1999. Human factors in auditory warnings. Aldershot, UK: Ashgate.Steele, M. and Gillespie, B., 2001. Shared control between human and machine: using a haptic

steering wheel to aid in land vehicle guidance. Proceedings of the Human Factors and

Ergonomics Society 45th annual meeting. Santa Monica, CA: HFES, 1671–1675.Stein, B.E. and Meredith, M.A., 1993. The merging of the senses. Cambridge, MA: MIT Press.

Stokes, A., Wickens, C., and Kite, K., 1990. Display technology: human factors concepts.

Warrendale, PA: Society of Automotive Engineers.

Strayer, D.L., Drews, F.A., and Johnston, W.A., 2003. Cell phone-induced failures of visual

attention during simulated driving. Journal of Experimental Psychology: Applied, 9, 23–32.Strayer, D.L. and Johnston, W.A., 2001. Driven to distraction: dual-task studies of simulated

driving and conversing on a cellular telephone. Psychological Science, 12, 462–466.Suetomi, T. and Kido, K., 1997. Driver behavior under a collision warning system – a driving simulator

study. SAE Technical Publication, No. 970279, 1242, 75–81.Summala, H., Lamble, D., and Laakso, M., 1998. Driving experience and perception of

the lead car’s braking when looking at in-car targets. Accident Analysis and Prevention, 30,

401–407.Sussman, E.D., et al., 1985. Driver inattention and highway safety. Transportation Research Record,

1047, 40–48.Svensson, M.S. and Tap, H., 2003. Alarms – localization, orientation and recognition. International

Journal of Human–Computer Interaction, 15, 51–66.Tijerina, L., et al., 2000. Preliminary studies in haptic displays for rear-end collision avoidance system

and adaptive cruise control system applications. Report No. DOT HS 808 (TBD). Washington,

DC: National Highway Transportation Safety Administration.Todd, J.W., 1912. Reaction to multiple stimuli. New York: Science Press.

Treat, J.R., et al., 1977. Tri-level study of the causes of traffic accidents. Vol. 1: Causal factor

tabulations and assessments. Vol. 2: Special analyses. Final Report on US Department of

552 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

Transportation. Contract No. DOT-HS-034–3-535–770. Washington, DC: US Government

Printing Office.Triggs, T.J., Lewison, W.H., and Sanneman, R., 1974. Some experiments with flight-related

electrocutaneous and vibrotactile displays. In: F.A. Geldard, ed. Cutaneous communication

systems as devices. Austin, TX: Psychonomic Society, 57–64.

Van der Lubbe, R.H.J. and Postma, A., 2005. Interruption from auditory and visual onsets even

when attention is in a focused state. Experimental Brain Research, 164, 464–471.Van Egmond, R., 2004. Emotional experience of frequency modulated sounds: implications for the

design of alarm sounds. In: D. de Waard, K.A. Brookhuis and C.M. Weikert, eds. Human

factors in design. Maastricht, The Netherlands: Shaker Publishing, 345–356.

Van Erp, J.B.F., 2005. Presenting directions with a vibrotactile torso display. Ergonomics, 48,

302–313.Van Erp, J.B.F., and van Veen, H.A.H.C., 2001. Vibro-tactile information processing in

automobiles. Proceedings of EuroHaptics 2001, 99–104.Van Erp, J.B.F. and van Veen, H.A.H.C., 2004. Vibrotactile in-vehicle navigation system.

Transportation Research Part F: Traffic Psychology and Behavior, 7, 247–256.Van Winsum, W., Martens, M. and Herland, L., 1999. The effects of speech versus tactile

driver support messages on workload, driver behaviour and user acceptance. TNO-Report No.

TM-99-C043. Soesterberg, The Netherlands: tNO Human Factors Research Institute.Veitengruber, J.E., 1978. Design criteria for aircraft warning, caution, and advisory alerting systems.

Journal of Aircraft, 15, 574–581.Vitense, H.S., Jacko, J.A., and Emery, V.K., 2003. Multimodal feedback: an assessment of

performance and mental workload. Ergonomics, 46, 68–87.

Von Bekesy, G., 1963. Interaction of paired sensory stimuli and conduction in peripheral nerves.

Journal of Applied Physiology, 18, 1276–1284.Wang, D.D., Proctor, R.W., and Pick, D.F., 2003. Stimulus-response compatibility effects for

warning signals and steering responses. Proceedings of the 2nd international driving symposium

on human factors in driver assessment, training and vehicle design, 226–230.

Wang, J.-S., Knipling, R.R., and Goodman, M.J., 1996. The role of driver inattention in crashes:

new statistics from the 1995 crashworthiness data system. 40th annual proceedings of the

Association for the Advancement of Automotive Medicine. 377–392.

Ward, L.M., McDonald, J.J., and Golestani, N., 1998. Cross-modal control of attention shifts.

In: R.D. Wright, ed. Visual attention. New York: Oxford University Press, 232–268.

Washburn, D.A., et al., 2003. Olfactory use in virtual environment training. Modeling and Simulation

Magazine, 2 (3), 19–25.Wiener, E.L., 1977. Controlled flight into terrain accidents: system-induced errors. Human Factors,

19, 171–181.Wilkins, P.A. and Acton, W.I., 1982. Noise and accidents – a review. Annals of Occupational

Hygiene, 25, 249–260.Zaidel, D.M., Paarlberg, W.T., and Shinar, D., 1978. Driver performance and individual differences

in attention and information processing. Report No. DOT-HS-803 793. Bloomington, IN:

Institute for Research in Public Safety, Indiana University.Zampini, M., et al., 2007. Audiotactile multisensory interactions in front and rear space.

Neuropsychologia, 45, 1869–1877.

About the authors

Charles Spence is Professor of Experimental Psychology and the head of the Crossmodal ResearchLaboratory in the Department of Experimental Psychology at the University of Oxford. He received hisPhD in Experimental Psychology at the University of Cambridge in 1995. He has published more than 200journal articles over the last decade. He has been awarded the 10th Experimental Psychology Society Prize,the British Psychology Society: Cognitive Section Award, the Paul Bertelson Award, recognising him as

Theoretical Issues in Ergonomics Science 553

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4

the young European Cognitive Psychologist of the year, and, most recently, the prestigious FriedrichWilhelm Bessel Research Award from the Alexander von Humboldt Foundation in Germany.

Cristy Ho is a postdoctoral research scientist at the Crossmodal Research Laboratory, University ofOxford. She received her DPhil in Experimental Psychology from the University of Oxford in June 2006.Her research has focused on investigating the effectiveness of multisensory warning signals in driving. In2006 Cristy was awarded the American Psychological Association’s ‘New Investigator Award inExperimental Psychology: Applied’. This award is given for the most outstanding empirical paperauthored by a young scholar published within the Journal of Experimental Psychology: Applied.

554 C. Spence and C. Ho

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 2

2:07

09

Oct

ober

201

4