Towards Combining UAV and Sensor Operator Roles inUAV-Enabled Visual Search

Joseph CooperDepartment of Computer SciencesThe University of Texas at Austin

Austin, TX USAjcooper@cs.utexas.edu

Michael A. GoodrichComputer Science Department

Brigham Young UniversityProvo, UT USA

mike@cs.byu.edu

ABSTRACTWilderness search and rescue (WiSAR) is a challenging prob-lem because of the large areas and often rough terrain thatmust be searched. Using mini-UAVs to deliver aerial videoto searchers has potential to support WiSAR efforts, buta number of technology and human factors problems mustbe overcome to make this practical. At the source of manyof these problems is a desire to manage the UAV using asfew people as possible, so that more people can be usedin ground-based search efforts. This paper uses observa-tions from two informal studies and one formal experimentto identify what human operators may be unaware of as afunction of autonomy and information display. Results sug-gest that progress is being made on designing autonomy andinformation displays that may make it possible for a singlehuman to simultaneously manage the UAV and its camerain WiSAR, but that adaptable displays that support sys-tematic navigation are probably needed.

Categories and Subject DescriptorsH.5.2 [Graphical user interfaces]:

General TermsHuman Factors

KeywordsEcological Interfaces, Human-Robot Interaction, UnmannedAerial Vehicle, User Study

1. INTRODUCTIONWilderness search and rescue (WiSAR) is a challenging

problem because of the large areas and often rough ter-rain that must be searched. Because the search area growsrapidly and survivability drops considerably as time elapses,it is imperative that WiSAR searches be performed quickly [28].Using a mini-Unmanned Aerial Vehicle (UAV) to provide

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.HRI’08, March 12–15, 2008, Amsterdam, The Netherlands.Copyright 2008 ACM 978-1-60558-017-3/08/03 ...$5.00.

aerial imagery of a search area has the potential for helpingWiSAR, but a number of human-robot interaction problemsmust first be addressed. Because WiSAR efforts can covervast areas, it is desirable to have as many well-trained peopleparticipating in the ground search as possible. This obser-vation suggests that it is desirable to minimize the numberof people required to manage the UAV so that more peoplecan be used in other aspects of the search.

Unfortunately, experience in many aerial search applica-tions indicates that multiple roles must be performed in or-der to use an UAV in search [14, 24]. The four primary rolesare [13]: the incident commander, who is responsible formanaging the search effort; the pilot or UAV operator, whois responsible for aviation and navigation; the payload orsensor operator, who is responsible for analyzing imageryand localizing potential signs of the missing person; andground searchers. In this paper, we address the roles ofthe UAV operator and the sensor operator.

Importantly, the appropriate type of navigation dependsstrongly on whether the search is a hasty/heuristic search1,an exhaustive search, or a search that evaluates high prior-ity regions first. These three categories of search representthree of the four qualitatively different search types encoun-tered in WiSAR [14, 28]. As we evaluate different displayand autonomy paradigms, we will use the ability to recall aflightpath and the ability to quickly cover a region as mea-sures of navigation quality. Similarly, we will use redundantobservations, false alarms, and missed detection as measuresof the quality of detection and localization. Throughout, weassume that aviation is performed by an autopilot.

Ideally, adding well-designed autonomy and informationdisplays may make it possible for a single user to fill boththe UAV operator and sensor operator roles. This paperevaluates what human operators may not be aware of as afunction of emerging autonomy and information display de-signs. The paper uses observations from two informal studiesand one formal experiment. Results suggest that autonomyand information displays may evolve to the point where itis possible for a single human to simultaneously manage aUAV and analyze video, but that adaptable displays thatsupport systematic navigation are probably needed.

2. RELATED LITERATUREThe goal of this work is to support fielded missions in the

spirit of Murphy’s work [5, 7] by designing autonomy and

1A hasty/heuristic search reactively follows a trail of clues,such as footprints, rather than systematically covering anarea.

351

information displays that support WiSAR efforts. Relatedto this goal, there is a great deal of current research dealingwith the human factors of (semi-autonomous) UAVs [17, 8].Typically, a UAV engaged in a search task requires eithertwo operators or a single operator to fill two roles: a pilot,who “flies” the UAV, and a sensor operator, who interpretsthe imagery and other sensors [31]. Lessons from groundrobots suggest that it is sometimes useful to include a thirdperson who (a) monitors the behavior of the pilot and sen-sor operators, (b) protects these operators, and (c) facilitatesgreater situation awareness [5, 7, 11]. We do not explicitlyinclude this as one of the roles in WiSAR. Related to humanroles, other work has analyzed how many unmanned plat-forms a single human can manage [26, 15, 9]. A commontheme of this work work is that the span of human controlis limited so that, for example, it would be difficult to mon-itor information-rich video streams from multiple UAVs atonce, though it is possible to coordinate multiple UAVs a laair traffic control [22, 23].

In the WiSAR domain, literature related to aerial searchis particularly relevant [20, 4]. Recent work includes notonly an evaluation of the WiSAR problem domain [14, 13],but also the evaluation of heuristic algorithms for searchingan environment characterized by a probabilistic descriptionof the person’s likely location [16]. Additional literature in-cludes operator interface work for both UAVs and traditionalaviation displays [6, 27, 1, 10, 32].

The exact type of interaction between a human and on-board autonomy varies widely across UAV platforms. Atone extreme, the Predator UAV essentially re-creates a tra-ditional cockpit inside a ground-based control station, com-plete with stick-and-rudder controls. At the other extremeare architectures employed by research UAVs that measureatmospheric composition by flying pre-programmed flightpaths to obtain precise scientific data [12]. The interac-tions represented by these two extremes typify the extremepoints of several adjustable autonomy scales [29, 19]. Thepresented work uses altitude, attitude, and direction controlalgorithms, plus the ability to autonomously travel to a se-ries of waypoints. Thus, this work is between the extremesof teleoperation and supervisory control.

In addition to designing autonomy, it is necessary to createuser interfaces that support efficient interaction. We use anapproach for UAVs that is similar to the ecological interfacedesign for ground robots developed by others [2, 25].

3. UAV OVERVIEWThe work presented in this paper is done using simulated

UAVs. However, all of the control algorithms and user inter-face designs have been flown in one or more flight test withexperimental UAVs2 that are small and light, with most hav-ing wingspans of approximately 42”–50”and flying weights ofapproximately two pounds. The airframes are derived fromflying wing designs and are propelled by standard electricmotors powered by lithium batteries. Discussions with UtahCounty Search and Rescue indicated that at least 90 min-utes of flight time was required for a reasonable search, sothe BYU MAGICC lab created a custom airframe capableof staying aloft for up to 120 minutes while supporting anavionics sensor suite, a gimballed camera, and an autopilot.

2This description of the UAVs first appeared verbatimin [13]. It is included in this paper since to establish context.

The standard aircraft sensor suite includes 3-axis rate gy-roscopes, 3-axis accelerometers, static and differential baro-metric pressure sensors, a GPS module, and a video cameraon a gimballed mount. The UAV uses a 900 MHz radiotransceiver for data communication and an analog 2.4 GHztransmitter for video downlink. The autopilot is built ona small micro-processor described in [3]. The UAVs areequipped with autopilot algorithms that stabilize the air-craft’s roll angle, pitch angle, attitude, and/or altitude; thealgorithms also provide the ability to fly to a waypoint. Wewill use the waypoint algorithm as the basis for the way theoperator controls the UAV in the complete experiment.

4. INFORMATION DISPLAY PARADIGMSOne of the key factors that determines what a human

operator can perceive is the way information is displayed.Although user interfaces for UAVs are currently evolving ata very rapid rate, there appear to be three complementaryparadigms for presenting information. Although some infor-mation displays borrow ideas from different paradigms, thiscategorization is useful as we evaluate how autonomy andinformation display designs facilitate or inhibit search.

The first paradigm may be called pilot-centered. Thispara-digm seeks to replicate the kinds of sensors and gaugespresented in a manned aircraft [21]. Extensions of a pilot-centered interface include recent ecological display develop-ment for better pilot support [30]. Because WiSAR person-nel are not typically trained as pilots and because it is desir-able to have training efforts focused on rescue and recoveryrather than piloting, we will not address the pilot-centeredparadigm in this paper.

The second paradigm may be called a traditional display.A traditional display uses multiple windows to present amap, live video, and perhaps annotation and communica-tion. For example, Figure 1 presents the flight map window(on the right) that shows a map with the full flight path andcurrent location of the UAV marked. As the craft flies overterrain, the live video window (center) shows video receivedby the UAV camera. The marking window (left) providesa map of the same location as the flight map window andallows UAV operators to annotate the map with relevantitems of information.

Anecdotal evidence in field tests suggests that it is difficultto integrate map information with video information in atraditional display. In an informal study, we simulated asituation where the sensor operator is isolated from the UAVoperator by requiring human participants to identify targetsfrom video obtained during a pre-programmed flight. Theconclusion from this informal study is that it is difficult foran isolated sensor operator using a traditional informationdisplay to distinguish between redundant targets.

Five unbiased participants observed four, five-minute flightson a 19-inch LCD monitor. The four different flight pathscovered approximately the same distance, but followed dif-ferently shaped paths over a wide range of terrains. Par-ticipants marked targets using a regular optical mouse andwere paid $10 for their participation.

During the four flights, four different video presentationswere shown to participants in random, counter-balanced or-der: downward, downward-persistent, angled, angled-persis-tent. The downward trials simulated a camera pointing di-rectly out of the bottom of the craft; the angled trials simu-lated a camera mounted such that the center of the camera

352

Figure 1: Traditional multiple window display.

pointed 45 degrees from the bottom of the craft. The camerahad a 30 degree field of view. The persistent trials assumed agimballed camera that was aimed so that the camera main-tained a constant angle (straight down or 45 degrees fromstraight down) with respect to the normal from the UAV tothe ground. The non-persistent trials kept the camera fixedwith respect to the craft so that when the UAV turned onedirection, the video footprint extended in the other direc-tion.

The lesson from this informal study is that, although de-tection was easy, correctly localizing the target was veryhard to do given the display. This is indicated by the highnumber of targets that were redundantly marked becauseparticipants could not tell that they had seen and localizedthese targets before. Although there were only 10 targetsvisible in each experiment trial, participants marked, on av-erage, 16.35 targets per flight. Additionally, three of thefive participants commented on the difficulty in discerningredundant targets. From video tape of participant faces, wequalitatively observed that participants split their attentionfairly uniformly across the three different windows, spend-ing one to five seconds on each window. This suggests thata key contributing factor to redundant target identificationwas that the participants’ attention was stretched across thethree different windows.

The third paradigm may be called an integrated display.By contrast to the traditional display, integrated displaysmay include satellite imagery, contour plots, a representa-tion of the UAV, the camera footprint, and annotation in-formation projected onto a simulated terrain. For example,Figure 2 displays an iconic representation of the UAV in asynthetic terrain map, with a tethered video heuristicallyprojected to the surface of the terrain. Note that the “car-rot” label in the figure will be discussed in the next section.

Evidence from ground robots indicates that an integrateddisplay paradigm may make searching for objects easier than

Figure 2: An integrated display.

with traditional displays [25]. In an informal experiment,participants analyzed video from a preprogrammed flightover simulated terrain. The conclusion from this study isthat an isolated sensor operator using an integrated displayis largely unaware of the path flown by the UAV.

Eight participants were instructed to localize targets andto remember the flight path. The ability to remember theflight path hypothetically provides information about whethera human operator could localize targets while efficiently per-forming a systematic search. Since remembering what hasbeen searched is necessary for efficiently covering a searcharea, having some recollection of the flight path would benecessary if a single human were to perform both the UAVoperator and sensor operator roles simultaneously. Through-out each flight, targets (spheres) and distractors (pyramids)were visible in the simulated video. At the end of the flight,the interface perspective smoothly zoomed out to show theentire map and instructed the participants to do their best

353

to draw the path the UAV flew. Each flight covered the samedistance, had no overlap, and consisted of five straight seg-ments with four turns of either forty-five degrees or ninetydegrees. Participants observed video from two nominal dis-plays (a chase perspective and a north-up map perspective)and from three combination displays that switched perspec-tives during the middle of the experiment.

Results indicate that targets were easily detected in thisexperiment, but participants were generally incapable of re-membering the automatically executed flight path while fo-cusing on the detection task. Paths seemed almost com-pletely random and participants admitted that they had noclue what the actual flight path was. We tried allowing sub-jects to use a paper and pencil to help with remembering theflight path. With a paper for taking notes, participants per-formed better at remembering the shape of the flight path,but had very little sense of scale or location or even the rel-ative lengths of the five flight segments. This indicates thatthey did not know where the craft had actually flown, butjust that it had made certain turns. We attempted to give asense of scale and location by showing on the map where thecraft started and stopped but participants still failed to drawthe flight path with any degree of recognizable accuracy.

5. AN INTEGRATED DISPLAY AND ACTIVESEARCH

The two informal studies suggest that requiring an oper-ator to integrate information from multiple sources makesit difficult to unambiguously identify targets, and that re-quiring an isolated sensor operator to recall the path is verydifficult. In response to these observations, we hypothesize(a) that an integrated display supports localization betterthan a traditional display, and (b) that giving operators nav-igation control provides them greater awareness of the path.In this section, we present an experiment that tests these hy-potheses. The experiments include the following elements:full autonomy for aviation, a very simple detection task, andautonomy that supports human-directed navigation.

More precisely, we explore the effect of different infor-mation displays on a reactive search task using the displayshown in Figure 2. We studied how well an operator withminimal training could perform a search task operating theinterface using four common control perspectives: chase,north-up, track-up, and split/full-map display. In the chaseperspective, the operator views an integrated map and UAVicon from a virtual position behind and above the UAV. Inthe north-up perspective, the operator views an integratedmap and UAV icon from directly above the UAV, with theorientation of the map always north-up. In the track-up per-spective, the operator again views an integrated map andUAV icon from directly above the UAV, but the orienta-tion of the map changes so that the heading of the UAV isalways to the top of the screen. The split display is simi-lar to the north-up display, but from a much more distantperspective, which means that the operator must rely on asecondary video display since the integrated video has toolittle detail to detect targets.

5.1 DesignParticipants used each of the different perspectives to find

targets randomly distributed according to each of three dif-ferent distributions: uniform, gaussian, or rectangular path.

The uniform and gaussian distributions are restricted to asub-area on the map that covers approximately one-fourthof the map’s area. The path distribution is shown in Fig-ure 3; in the figure, the light speckles indicate targets anddistractors, and the shading represents the probability den-sity function from which the targets and distractors aresampled. Importantly, the three different distributions cor-respond closely to the three previously mentioned WiSARsearch tactics: hasty, high-priority, and exhaustive search.Having targets distributed uniformly across a sub-region ofthe terrain suggests an exhaustive search of that area. Hav-ing targets scattered according to a Gaussian distributionsuggests a high-probability-prioritized search pattern. Hav-ing targets distributed closely along a constrained path sug-gests a hasty search.

Figure 3: Rectangular path distribution.

Colored spheres and pyramids were placed in the world torepresent targets and distractors, respectively. Both typesof objects followed the same distribution, but there were300 pyramids and 10 spheres. Subjects were instructed tolocate and mark the spheres. The pyramids served as dis-tractors (to keep the participant from simply marking anyobject that stood out from the earth-toned terrain imagery).Pyramids also indicated the probability distribution so thatif there were a large number of pyramids in an area, it wasmore likely that there was a sphere in the same area. Thepyramids fill the role of minor environmental clues such asgame trails or vegetation that may not appear in satelliteimagery, but give some hint about where a more importantclue may or may not be when seen through the live video.

Localizing the spheres was accomplished by using the mouseto left-click on the terrain location where the subject be-lieved the sphere to be. When participants marked a loca-tion, a spherical marker stayed in that location. Performinga left-click on an existing mark allowed the subject to dragthe mark around, while performing a right-click deleted themark. Participants also had the option to press the spacebar to take a snapshot of the video. The snapshot left a stillframe of the video at the location the camera was pointingto when the snapshot was taken. Taking snapshots was notnecessary for the task, but was a tool participants could useto get a better look at the video of a particular location orto help mark where the craft had been.

354

Participants operated the UAV with the mouse using a“stick and carrot metaphor” [34]. The “carrot” was a dis-tinct marker (Figure 2) rendered onto the synthetic terrainthat would follow the mouse cursor as long as the Controlkey was down. When the test subject released the Controlkey, the marker stayed where it was and the craft contin-ued to fly toward the mark using the autopilot capability toautonomously fly to the waypoint. When the UAV arrivedat the mark, it first crossed over the point and then beganto orbit the mark until it was moved. Typically the on-board camera pointed thirty degrees forward from straightdown (with respect to the craft), but when the UAV be-gan orbiting a point, the gimballed camera was aimed sothat the center of the camera focused on that point. Thissame control method was used for all four perspectives. Theoperator control interface connected to Aviones, an opensource, moderate-fidelity, physics-based simulation that runsthe same autopilot code and flight-dynamics model as thephysical craft. The simulator generates imagery as it wouldbe seen by the UAV camera using a synthetic terrain model.The experiment used a 19-inch LCD monitor for the primarydisplay and a five-inch auxiliary LCD monitor that showedthe raw video.

Twenty-one naive human subjects participated in the ex-periment. Participants received $12 for their time. Testparticipants were given a sheet of directions and trained oneach interface. For each perspective, training included hav-ing subjects practice controlling the craft, taking snapshots,and marking (localizing) targets. Subjects participated intwelve experimental trials and four practice trials for a to-tal of sixteen trials. Each of the sixteen trials took place insimilar synthetic environments, each with a large flat centralarea and small hills off to the sides. Each participant con-trolled the craft through all four experimental perspectives,which were presented in randomized and counterbalancedorder. After each experimental trial, participants answeredthree questions about the relative difficulty of the task andthen went on to the next trial. The study ended with a fewmore general questions about the interface.

5.2 ResultsThe first observation is that performance shows a strong

learning effect across all experiment conditions. Figure 4shows that true positive marks generally increase while thesubject uses a particular control mode and fall slightly whenthe participant switches to a new perspective. Similarly,false positive and redundant marks3 fall fairly consistentlyover time, rising slightly with the perspective changes.

The second observation is that the split/full-map perspec-tive is significantly worse (p<0.05) in all measures exceptredundant marks; see Figure 5. Additionally, participantsranked the split display as more difficult than all other per-spectives (p<0.0005). Although the video footprint is visiblein the split/full-map perspective, the low video resolutionforces participants to rely on the raw video monitor to de-

3A redundant mark occurs when a subject places two ormore different marks for a single target. Redundant marksindicate that the subject was not able to determine that theyhad previously seen that target; redundant marks indicate alimitation of situation awareness. Note that the only differ-entiating feature between targets in our experiment is theircolor, but targets in a real search may be uniquely identifi-able. Thus, redundant marks may be less of a problem in areal search than in this experiment.

0

2

4

6

8

10

2 4 6 8 10 12

Mar

ks m

ade

Trial

Learning effect over time

True positiveRedundant

False positive

Figure 4: Learning Effect. The vertical lines indicatea change in the perspective used in the experiment.

tect targets. Many participants commented that they usedthe raw video monitor only for the full-map perspective andthat they disliked it.

In the split/full-map display, participants directed the craftusing the interface screen while trying to simultaneouslymonitor the raw video screen for targets. Upon detecting atarget, they returned their attention to the interface screenand searched for the video footprint in order to localize theobject on the terrain. Accurate localization required men-tal rotations to correlate the video with the terrain. Tocope with this difficulty, several participants used the snap-shot feature for the full-map perspective trials. Participantscould concentrate on the raw video monitor with one handon the mouse and the other on the keyboard. When a par-ticipant detected a target from the raw video, participantstook a snapshot. They then switched their attention brieflyto the primary monitor, localized the snapshot, and placedthe sphere mark. Participants who used this strategy quali-tatively did better with the full-map perspective than thosewho did not, but still worse than with other perspectives.

The third observation is that the three distributions varysignificantly in difficulty. Performance is generally best forthe path distribution and worst for the uniform distribution(see Figure 6). The uniform distribution demonstrates moreredundant marks than the path distribution (p=0.0435) andfewer true positives (p=0.0263).

One reason that the path distribution may be easier is thatthe path distribution suggests an obvious coverage strategy:find and then follow the path. Following the path quicklycovers the full probability distribution. Searching the Gaus-sian distribution from the center outward quickly accumu-lates probability at the beginning and gradually tapers offwith time. Finally, a uniform probability distribution overa rectangular area can be accumulated at a constant butsomewhat slow rate.

The fourth observation is that the track-up, north-up,and chase perspectives show roughly comparable levels ofperformance; see Figures 7 through 9, which group variousperformance metrics first by perspective and then by dis-tribution. Although the chase perspective produced more

355

Figure 5: Performance means according to perspec-tive.

true positives than track-up (p=0.0633), subjects generallyperformed comparably well using the chase, north-up, andtrack-up perspective. This is notable because other studieshave found improved performance and operator preferenceusing a track-up perspective [33, 18]. This may be becausethe other studies used an aviation-based control methodwhere commands are given with respect to the craft (e.g.,turn right or left). A track-up perspective helps the opera-tor avoid confusing his or her own left with the craft’s left.The carrot and stick control metaphor, on the other hand,is navigation-based.

The fifth observation is that there are several two-wayinteractions between perspective and distribution. Specifi-cally, chase perspective and north-up perspective are signif-icantly better than track-up and split/full-map perspectivesfor redundant marks under a uniform distribution. Addi-tionally, the chase perspective is significantly better thanthe other three perspectives for redundant marks under thegaussian distribution and the path distribution. By con-trast, the north-up perspective is significantly better thanthe other three perspectives for false positives for the gaus-sian distribution, though the chase perspective is better thanthe other three perspectives under the same metric for thepath distribution. This observation is important because itsuggests that the best perspective for detecting and localiz-ing a target depends on the type of search being performed.We summarize this conclusion and others in the next section.

6. CONCLUSIONS AND FUTURE WORKAlthough it does not yet merit a strong conclusion, the ob-

servations reported herein allow us to speculate that it maybe possible for a single human to simultaneously navigate anarea while localizing objects. Achieving this speculated ob-jective requires that trustworthy aviation is available, thatdetection is sufficiently easy, and that some form of supportis available for systematic and efficient navigation.

In support of this speculation, the observations from theinformal studies and the formal experiment suggest a num-ber of less speculative conclusions. The first conclusion is

Figure 6: Performance means according to distribu-tion.

0

2

4

6

8

10

PathGaussianUniform

Mar

ks

Distribution

Full MapChase

North upTrack up

Figure 7: True positive marks

that having the UAV, camera footprint, and annotation inte-grated into the map appears to improve the ability to localizetargets. This should not be surprising since it should be eas-ier to detect a redundant sign if that sign is displayed next tovideo information in a display; by contrast, traditional dis-plays require an operator to mentally integrate video, map,and annotations. The conclusion that localization is easierin an integrated display is strengthened by the observationthat, under the split/full-map display conditions, partici-pants who took a snapshot when they detected a target andthen used this snapshot to localize the target performed bet-ter than those who did not; having the snapshot integratedinto the map reduced the difficulty of integrating informa-tion from multiple windows.

The second conclusion is that efficient navigation is morechallenging for some search types than others. This conclu-sion is supported by the observations that the path distribu-tion generally produces better performance than the other

356

0

0.5

1

1.5

2

PathGaussianUniform

Mar

ks

Distribution

Full MapChase

North upTrack up

Figure 8: Redundant marks

0

0.5

1

1.5

2

PathGaussianUniform

Mar

ks

Distribution

Full MapChase

North upTrack up

Figure 9: False positive marks

distributions. Simply put, following a path of clues is eas-ier than systematically searching an area beginning in highpriority areas and ending in low priority areas. As it re-lates to the speculation that it may be possible to combineUAV operator and sensor operator roles, the performancedifference between search types suggests a caution and adirection of research. This caution is reinforced by the ob-servation from the second informal trial that a passive sensoroperator may be generally unaware of the path followed bythe UAV operator. However, if “coverage maps”, such as thepreliminary one reported in [14], are used to help an opera-tor understand the quality of the search path, it may still bepossible for an operator to perform an efficient high-priorityor exhaustive search while simultaneously localizing targets.Indeed, a coverage map could be projected into a path planso that the operator could follow an efficient path withoutbecoming a passive sensor operator.

The third conclusion is that exhaustive and high-prioritysearches are probably done more effectively from a north-

up perspective than from a chase perspective; conversely, ahasty search is probably done more effectively from a chaseperspective. Simply put, the observations suggest that hastysearches require more craft-based path changes, while com-plete searches require more map-based planning. This ob-servation suggests that research is needed into how displayperspectives should be changed depending on the type ofsearch that is being performed. Since an incident comman-der may want to change from an exhaustive search to a hastysearch if new evidence surfaces [28], the need for adaptabledisplays may be key for fielded searches.

In addition to future research into adaptable displays andcoverage maps, we have largely ignored aviation and detec-tion. Although the UAV autopilot supports a wide range ofaviation, work is still needed to perform good height aboveground (HAG) maintenance. It should be noted that in atleast three field trials, the absence of good HAG supportcaused problems in the search. A second area of research isthe continued improvement of image enhancement and mo-saicking techniques for improving detection. Initial work inthe importance of temporally local image mosaics indicatesthat such mosaics make detection much easier [14], but sev-eral detection-relevant problems persist including low reso-lution and poor color.

AcknowledgementsThe authors the reviewers for their helpful comments. Thiswork was partially supported by the National Science Foun-dation (grant 0534736), by the Utah Small Unmanned AirVehicle Center of Excellence, and by a Mentored Environ-ment Grant from Brigham Young University. Any opinions,findings, and conclusions or recommendations expressed inthis material are those of the author(s) and do not necessar-ily reflect the views of these supporting organizations.

7. REFERENCES[1] A. L. Alexander and C. D. Wickens. Synthetic vision

systems: Flightpath tracking, situation awareness, andvisual scanning in an integrated hazard display. InProceedings of the 13th International Symposium onAviation Psychology, 2005.

[2] M. Baker, R. Casey, B. Keyes, and H. A. Yanco.Improved interfaces for human-robot interaction inurban search and rescue. In Proceedings of the IEEEConference on Systems, Man and Cybernetics, TheHauge, The Netherlands, October 2004.

[3] R. Beard, D. Kingston, M. Quigley, D. Snyder,R. Christiansen, W. Johnson, T. McLain, and M. A.Goodrich. Autonomous vehicle technologies for smallfixed-wing UAVs. Journal of Aerospace Computing,Information, and Communication, 2, 2005.

[4] F. Bourgault, T. Furukawa, and H. F. Durrant-Whyte.Coordinated decentralized search for a lost target in aBayesian world. In Proceedings of the 2003 IEEE/RSJInternational Conference on Intelligent Robots andSystems, 2003.

[5] J. L. Burke and R. R. Murphy. Human-robotinteraction in USAR technical search: Two heads arebetter than one. In Proceedings of the 13thInternational Workshop on Robot and HumanInteractive Communication (RO-MAN), 2004.

357

[6] G. L. Calhoun, M. H. Draper, M. F. Abernathy,M.Patzek, and F. Delgado. Synthetic vision system forimproving unmanned aerial vehicle operator situationawareness. In J. G Verly, editor, Proceedings of SPIEVol 5802, in Enhanced and Synthetic Vision, 2005.

[7] J. Casper and R. R. Murphy. Human-robotinteractions during the robot-assisted urban searchand rescue response at the world trade center. IEEETransactions on Systems, Man and Cybernetics, PartB, 33(3):367–385, June 2003.

[8] N. J. Cooke, H. Pringle, H. Pederson, and O. Connor,editors. Human Factors of Remotely OperatedVehicles, volume 7 of Advances in HumanPerformance and Cognitive Engineering Research.Elsevier, 2006.

[9] M. L. Cummings. Designing decision support systemsfor a revolutionary command and control domains.Doctoral dissertation, University of Virginia, 2003.

[10] J. L. Drury, J. Richer, N. Rackliffe, and M. A.Goodrich. Comparing situation awareness for twounmanned aerial vehicle human interface approaches.In Proceedings of the IEEE International Workshop onSafety, Security and Rescue Robotics (SSRR), August2006.

[11] J. L. Drury, J. Scholtz, and H. A. Yanco. Awareness inhuman-robot interactions. In Proceedings of the IEEEInternational Conference on Systems, Man andCybernetics, October 2003.

[12] T. Goetzendorf-Grabowski, A. Frydrychewicz,Z. Goraj, and S. Suchodolski. MALE UAV design ofan increased reliability level. Aircraft Engineering andAerospace Technology, 78(3):226–235, 2006.

[13] M. A. Goodrich, L. Cooper, J. A. Adams,C Humphrey, R. Zeeman, and B. G. Buss. Using amini-uav to support wilderness search and rescue:Practices for human-robot teaming. In Proceedings ofthe IEEE International Workshop on Safety, Security,and Rescue Robotics, 2007. Under review.

[14] M. A. Goodrich, B. S. Morse, D. Gerhardt, J. L.Cooper, J. A. Adams, C. Humphrey, and M.Quigley.Supporting wilderness search and rescue using acamera-equipped mini uav. Journal of Field Robotics,2007. To appear.

[15] P. A. Hancock, M. Mouloua, R. Gilson, J. Szalma, andT. Oron-Gilad. Is the UAV control ratio the rightquestion? Ergonomics in Design, 2006.

[16] S. Hansen, T. McLain, and M. Goodrich. Probabilisticsearching using a small unmanned aerial vehicle. InProceedings of AIAA Infotech@Aerospace, May 2007.

[17] Manning the unmanned. In 2004 Human Factors ofUAVs Workshop, Chandler, Arizona, May 2004.Cognitive Engineering Research Institute.

[18] C. Jansen, S. de Vries, and M. Duistermaat.Optimizing the presentation of UAV images in anattack helicopter cockpit. In Proceedings of the HumanFactors and Ergonomics Society 50th AnnualMeeting—2006, pages 131–135, San Francisco, 2006.

[19] D. B. Kaber and M. R. Endsley. The effects of level ofautomation and adaptive automation on humanperformance, situation awareness and workload in adynamic control task. Theoretical Issues inErgonomics Science, 5(2):113–153, March-April 2004.

[20] B. O. Koopman. Search and Screening: GeneralPrinciples with Historical Applications. PergamonPress, 1980. This book has been reprinted in itsentirety in 1999 by the Military Operations ResearchSociety, Inc.

[21] T. W. McLain and R. W. Beard. Unmanned airvehicle testbed for cooperative control experiments. InProceedings of the American Control Conference,June/July 2004.

[22] C. A. Miller, H. B. Funk, M. Dorneich, and S. D.Whitlow. A playbook interface for mixed initiativecontrol of multiple unmanned vehicle teams. InProceedings of the 21st Digital Avionics SystemsConference, volume 2, pages 7E4–1 – 7E4–13,November 2002.

[23] P. M. Mitchell and M. L. Cummings. Management ofmultiple dynamic human supervisory control tasks. In10th International Command and Control ResearchAnd Technology Symposium, 2005.

[24] R. Murphy, S. Stover, K. Pratt, and C. Griffin.Cooperative damage inspection with unmannedsurface vehicle and micro unmanned aerial vehicle athurrican Wilma. IROS 2006 Video Session, October2006.

[25] C. W. Nielsen, M. A. Goodrich, and B. Ricks.Ecological interfaces for improving mobile robotteleoperation. IEEE Transactions on Robotics,23(5):927–941, October 2007.

[26] D. R. Olsen Jr. and S. B. Wood. Fan-out: Measuringhuman control of multiple robots. In Proceedings ofHuman Factors in Computing systems, 2004.

[27] L. J. Prinzel, J. R. Comstock Jr., L. J. Glaab, L. J.Kramer, and J. J. Arthur. The efficacy of head-downand head-up synthetic vision display concepts forretro- and forward-fit of commercial aircraft. TheInternational Journal of Aviation Psychology,14(1):53–77, 2004.

[28] T. J. Setnicka. Wilderness Search and Rescue.Appalachian Mountain Club, 1980.

[29] T. B. Sheridan and W. L. Verplank. Human andcomputer control of undersea teleoperators. Technicalreport, MIT Man-Machine Systems Laboratory, 1978.

[30] D. L. Still and L. A. Temme. OZ: A human-centeredcomputing cockpit display. In TheInterservice/Industry Training, Simulation andEducation Conference (I/ITSEC, 2001.

[31] K. S. Tao, G. K. Tharp, W. Zhang, and A. T. Tai. Amulti-agent operator interface for unmanned aerialvehicles. In Proceedings 18th Digital Avionics SystemsConference, 1999.

[32] C. D. Wickens, O. Olmos, A. Chudy, andC. Davenport. Aviation display support for situationawareness. Technical ReportARL-97-10/LOGICON-97-2, Aviation Research Lab,University of Illinois at Urbana-Champaign, 1995.

[33] C. D. Wickens and T. T. Prevett. Exploring thedimensions of egocentricity in aircraft navigationdisplays. Journal of Experimental Psychology: Applied,1(2):110 – 135, 1995.

[34] Wikipedia. Carrot and Stick Description.http://en.wikipedia.org/wiki/Carrot and stick.

358