The 4th International Conference on Field and Service Robotics, July 14–16, 2003

Distributed Search and Rescue with Robot and Sensor Teams

G. Kantor and S. Singh R. Peterson and D. Rus A. Das, V. Kumar,G. Pereira and J. Spletzer

Robotics Instituate Department of Computer Science GRASP LaboratoryCarnegie Mellon University Dartmouth College University of Pennsylvania

Pittsburgh, PA, USA Hanover, NH, USA Philadelphia, PA, USAssingh@cmu.edu rus@cs.dartmouth.edu kumar@cis.upenn.edu

Abstract

We develop a network of distributed mobile sensor sys-tems as a solution to the emergency response problem. Themobile sensors are inside a building and they form a con-nected ad-hoc network. We discuss cooperative localiza-tion algorithms for these nodes. The sensors collect tem-perature data and run a distributed algorithm to assemblea temperature gradient. The mobile nodes are controlled tonavigate using this temperature gradient. We also discusshow such networks can assist human users to find an exit.We have conducted an experiment to at a facility used totrain firefighters to understand the environment and to testcomponent technology. Results from experiments at thisfacility as well as simulations are presented here.

1 Motivation

We consider search and rescue applications in whichheterogeneous groups of agents (humans, robots, static andmobile sensors) enter an unknown building and dispersewhile following gradients in temperature and concentrationof toxins, and looking for immobile humans. The agentsdeploy the static sensors and maintain line of sight visi-bility and communication connectivity whenever possible.Since different agents have different sensors and thereforedifferent pieces of information, communication is neces-sary for tasking the network, sharing information, and forcontrol.

An ad-hoc network is formed by a group of mobilehosts upon a wireless local network interface. It is a tem-porary network formed without the aid of any establishedinfrastructure or centralized administration. A sensor net-work consists of a collection of sensors and distributed oversome area that form an ad-hoc network. Our heterogeneousteams of agents (sensors, robots, and humans) constitute

distributed adaptive sensor networks and are well-suitedfor tasks in extreme environments, especially when the en-vironmental model and the task specifications are uncer-tain and the system has to adapt to it. Applications of thiswork cover search and rescue for first responders, monitor-ing and surveillance, and infrastructure protection.

We combine networking, sensing, and control to controlthe flow of information in search and rescue in unknownenvironments. Specifically, this research examines (1) lo-calization in an environment with no infrastructure such asa burning building (for both sensors and robots) (2) infor-mation flow across a sensor network that can localize onthe fly for delivering the most relevant and current infor-mation to its consumer, maintaining current maps, and au-tomating localization; (3) using feedback from the sensornetwork to control the autonomous robots for placing sen-sors, collecting data from sensors, and locating targets; and(4) delivering the information gathered from the sensor net-work (integrated as a global picture) to human users. Thepaper will detail our technical results in these 4 areas anddescribe an integrated experiment for navigation in burningbuildings.

2 Localization

Localization in dynamic environments such as posed bysearch and rescue operations is difficult because no infras-tructure can be presumed and because simple assumptionssuch as line of sight to known features can not be guaran-teed. We have been investigating the use of low cost radiobeacons that can be placed in the environment by rescuepersonnel or carried by robots. In this paradigm, as therobot moves, it periodically sends out a query, and any tagswithin range respond by sending a reply. The robot canthen estimate the distance to each responding tag by de-termining the time lapsed between sending the query and

Figure 1: (Left) An ad-hoc network of robots and Mote sensors deployed in a burning building at the Allegheny FireAcademy, Aug 23, 2002 (from an experimental exercise involving CMU, Dartmouth, and U. Penn). (Right) The temperaturegradient graph collected using an ad-hoc network of Mote sensors.

receiving the response. The advantage of such a methodis that it does not require line of sight between tags andthe mobile robot, mak- ing it useful in many environmen-tal conditions that fail optical methods. Note that, sinceeach tag transmits a unique ID number, distance readingsare automati- cally associated with the appropriate tags, sothe data association problem, a difficult issue especially inenvironments that can be visually obscured is solved triv-ially.

Since the position of the tags is unknown to start andcan potentially change during operation, it is necessary tolocalize both the receiver and the beacons simultaneously.This problem is often known as Simultaneous Localizationand Mapping (SLAM). Although generally it is assumedthat a receiver is able to measure both range and bearingto "features", we can assume only that the range to tags isknown and that this measurement may be very noisy.

We have adapted the well-known estimation techniquesof Kalman filtering, Markov methods, and Monte Carlo lo-calization to solve the problem of robot localization fromrange-only measurements [KS02] [SKS02]. All three ofthese methods estimate robot position as a distribution ofprobabilities over the space of possible robot positions. Inthe same work we presented an algorithm capable of solv-ing SLAM in cases where approximate a priori estimatesof robot and landmark locations exist. The primary diffi-culty stems from the annular distribution of potential rela-tive locations that results from a range only measurement.Since the distribution is highly non-Gaussian, SLAM solu-tions based on Kalman filtering falter. In theory, Markovmethods (probability grids) and Monte Carlo methods (par-ticle filtering) have the flexibility to handle annular dis-tributions. Unfortunately, the scaling properties of thesemethods severely limit the number of landmarks that canbe mapped. In truth, Markov and Monte Carlo methodshave much more flexibility than we need; they can rep-resent arbitrary distributions while we need only to deal

Figure 2: A radio tag, approximately 12 x 9 cm in size,with which a robot can communicate to obtain range data.Such tags can be scattered into a burning building as fire-fighters move about or even be deployed by robots them-selves. Since these tags are are placed without careful sur-vey, their position must be calculated along with the posi-tion of the mobile agents themselves.

with very well structured annular distributions. What isneeded is a compact way to represent annular distributionstogether with a computationally efficient way of combin-ing annular distributions with each other and with Gaus-sian distributions. In most cases, we expect the results ofthese combinations to be well approximated by mixturesof Gaussians so that standard techniques such as Kalmanfiltering or multiple hypothesis tracking could be appliedto solve the remaining estimation problem.

We have also extended these results to deal with the casewhen the tag locations are unknown using a geometricallyinspired batch processing method. The basic idea is tostore the robot locations and measured ranges the first fewtimes the landmark is encountered and then obtain an es-timate of landmark position by intersecting circles on theplane. Once an estimate of a new landmark is produced,the landmark is added to the Kalman filter where its esti-mate is then improved along with the estimates of the other(previously seen) landmarks. Because it takes advantage

Figure 3: 13 RF tags in known locations are used to local-ize a robot moving in an open area in conjunction with awheel encoder and gyro. While individual range readingshave a standard deviation of as much as 1.3 m, it is possibleto localize the robot to within 0.3 m of the robots location.Tag locations are denoted by "o". The true path is denotedby the green line and the estimated path is denoted by a redline.

of the special structure of the problem, the resulting ap-proach is less computationally cumbersome and avoids thelocal maxima problems associated with standard batch op-timization techniques.

To collect data for this experiment, we used an instru-mented autonomous robot that has highly accurate (2 cm)positioning for groundtruth using RTK GPS re- ceivers aswell as a fiber optic gyro and wheel encoders. Positionis updated at 100 Hz. We equipped this robot with a RFranging system (Pinpoint from RF Technologies) that hasfour antennae pointing in four directions and a computer tocontrol the tag queries and process responses. For each tagresponse, the system produces a time-stamped dis- tanceestimate to the responding tag, along with the unique IDnumber for that tag. The distance esti- mate is simply aninteger estimate of the distance be- tween the robot and thetag.

The localization experiment was conducted on a flat,area about 30 meters by 40 meters in size. We distributed13 RF tags throughout the area, then pro- grammed therobot to drive in a repeating path among the tags. With thissetup, we collected three kinds of data: the ground truthpath of the robot from GPS and inertial sensors, the deadreckoning estimated path of the robot from inertial sensorsonly, and the range measurements to the RF tags. Resultsfrom our experiments are shown in Figs??and?? Greaterdetails can be found in [DKS03].

Figure 4: In the case that the tag locations are unknown tostart, they can be determined approximately using a batchscheme and then these approximate locations are used in akalman filter to continuously update the position of the tagalong with the position of the robot.

3 Information Flow

Sensors detect information about the area they cover.They can store this information locally or forward it to abase station for further analysis and use. Sensors can alsouse communication to integrate their sensed values with therest of the sensor landscape. Users of the network (robotsor people) can use this information as they traverse the net-work.

We have developed distributed protocols for naviga-tion tasks in which a distributed sensor field guides a useracross the filed [LdRR03]. We use the localization tech-niques presented above to compute environmental mapsand sensor maps, such as temperature gradients. Thesemaps are then used for human and robot navigation to atarget, while avoiding danger (hot areas).

Figure 1(Right) shows the layout of a room in whicha fire was started. We have collected a temperature gra-dient map during the fire burning experiment as shown inFigure 1. The Mote sensors1 were deployed by hand atthe locations marked in the figure. The sensors computedmulti-hop communication paths to a base station placed atthe door. Data was sent to the base station over a period of30 minutes.

1Each Mote sensor (http://today.CS.Berkeley.EDU/tos/) consists of anAtmel ATMega128 microcontroller (with 4 MHz 8 bit CPU, 128KB flashprogram space, 4K RAM, 4K EEPROM), a 916 MHz RF transceiver(50Kbits/sec, 100ft range), a UART and a 4Mbit serial flash. A Moteruns for approximately one month on two AA batteries. It includes light,sound, and temperature sensors, but other types of sensors may be added.Each Mote runs the TinyOS operating system.

X

X

X

X

X

X

X

X

X

XX

X

1

2

12

3

4

5 6

7

8

9 10

11

(112, 126)

(112, 84)

(112, 14)

(168, 135)

(203, 215)

(203, 309)

(280, 413)(203, 418)

(280, 518)

(44, −23)

(0, 0)

(203, 512)

X

X

X

X

X

X

X

X

X

XX

X

1

2

12

3

4

5 6

7

8

9 10

11

(112, 126)

(112, 84)

(112, 14)

(168, 135)

(203, 215)

(203, 309)

(280, 413)(203, 418)

(280, 518)

(44, −23)

(0, 0)

(203, 512)

Figure 5: (Left) The floor map for the directional guid-ance experiment. Arrows indicate the correct direction tobe detected by the Flashlight. (Right) The floor map for thedirectional guidance experiment with the Flashlight feed-back directions marked on it.

3.1 Directional Guidance

We used the structure of the data we collected duringthe fire burning exercise to develop a navigation guidancealgorithm designed to guide a user to the door, in a hop-by-hop fashion. We have deployed 12 Mote sensors alongcorridors in our building and guide a human user out ofthe building. Using an interactive device that can transmitdirectional feedback called a Flashlight [PR02] a humanuser was directed across the field.

The Flashlight prototype we designed and built is shownin Figure 6(left). This device can be carried by a humanuser or placed on a mobile robot (or flying robot) to interactwith a sensor field. Thebeamof the Flashlight is sensor-to-sensor, multi-hop routed RF messages which send or returninformation.

The Flashlight consists of an analog compass, alertLED, pager vibrator, a 3 position mode switch, a powerswitch, a range potentiometer, some power conditioningcircuitry, and a microcontroller based CPU/RF transceiver.The processing and RF communication components of theFlashlight and the sensor network are Berkeley Motes,shown in Figure 6(center,right). A switch selects the sensortype (light, sound, temperature, etc.) When the user pointsthe Flashlight in a direction, if sensor reports of the se-lected type are received from any sensors in that direction,a silent vibrating alarm activates. The vibration amplitudecan be used to encode how far (in number of hops) wasthe sensor that triggered. The potentiometer is used to set

Figure 6: The left figure shows the Flashlight prototype.The center figure shows a Mote board. The right figureshows the Mote sensor board.

the detection range (calibrated in number of network hopsfrom sensor to sensor.) The electronic compass suppliesheading data indicating the pointed direction of the device.

The Flashlight uses one Berkeley Mote(http://today.CS.Berkeley.EDU/tos/) as a main processorand sensor board. The Mote handles data processing tasks,A/D conversion of sensor output, RF transmission andreception, and user interface I/O. It consists of an AtmelATMega128 microcontroller (with 4 MHz 8 bit CPU,128KB flash program space, 4K RAM, 4K EEPROM),a 916 MHz RF transceiver (50Kbits/sec, 100ft range),a UART and a 4Mbit serial flash. A Mote runs forapproximately one month on two AA batteries. It includeslight, sound, and temperature sensors, but other typesof sensors may be added. Each Mote runs the TinyOSoperating system. A moving Flashlight interacts with awireless sensor network consisting of Mote sensors.

We have deployed 12 Mote sensors along corridors inour building and used the Flashlight and the communica-tion infrastructure presented here to guide a human userout of the building. Figure 5 shows the map. The Flash-light interacted with sensors to compute the next directionof movement towards the exit. For each interaction, theuser did a rotation scan until the Flashlight was pointed inthe direction computed from the sensor data. The user thenwalked in that direction to the next sensor. Each time werecorded the correct direction and the direction detected bythe Flashlight. The directional error was 8% (or 30 de-grees) on average. However, because the corridors and of-fice doorways are wide, and the sensors sufficiently dense,the exit was identified successfully. The user was never di-rected towards a blocked or wrong configuration. An inter-esting question is how dense should the sensors be, giventhe feedback accuracy.

4 Control of a Network of Robots

Robots augment the surveillance capabilities of a sen-sor network by using mobility. Each robot must use par-tial state information derived from its sensors and fromthe communication network to control in cooperation withother robots the distribution of robots and the motion ofthe team. We treat this as a problem of formation controlwhere the motion of the team is modeled as an element ofa Lie group, while the shape of the formation is a point inshape space. We seek abstractions and control laws thatallow partial state information to be used effectively and ina scalable manner.

Our platforms are car-like robots equipped with omnidi-rectional cameras as their primary sensors. The communi-cation among the robots relies on IEEE 802.11 networking.By using information from its camera system each robot isonly able to estimate its distance and bearing from theirteammates. However, if two robots exchange their bearingto each other, they are also able to estimate their relativeorientations 6 [SDF+01]. We use this idea to combinethe information of a group of two or more robots in orderto improve the knowledge of the group about their relativeposition.

We have developed control protocols for using such ateam of robots in connection with a sensor network to ex-plore a known building. We assume that a network of Motesensors previously deployed in the environment guide therobots towards the source of heat. The robots can modifytheir trajectories and still find the building exit. The robotscan also switch between the potential fields (or tempera-ture gradients) computed and stored in the sensor network(see Figure 7). The first switch occurs automatically whenthe first robot encounters a Mote sensor at a given location.The robots move toward the fire and stop at a safer distance(given by the temperature gradient). They stay there untilthey are asked to evacuate the building, at which point theyuse the original potential field to find the exit.

5 User Feedback

When robots or people interact with the sensor network,it becomes an extension of their capabilities, basically ex-tending their sensory systems and ability to act over amuch large range. We have developed software that allowsan intuitive, immersive display of environments. Using,panoramic imaging sensors that can be carried by smallrobots into the heart of a damaged structure, the displaycan be coupled to head mounted, head tracking sensors thatenable a remote operator to look around in the environmentwithout the delay associated with mechanical pan and tiltmechanisms.

Figure 7: Three robots switching motion plans in real timein order to get information from the hottest spot of thebuilding. In (b) a gradient of temperature is obtained froma network of Mote sensors distributed on the ground.

The data collected from imaging systems such as visi-ble cameras and IR cameras are displayed on a wearablecomputer to give the responder the most accurate and cur-rent information. Distributed protocols collect data fromthe geographically dispersed sensor network and integratethis data into a global map such as a temperature gradientthat can also be displayed on a wearable computer to theuser.

6 Discussion

The three groups met on August 23, 2002, at the Al-legheny County firefighting training facility to conduct pre-liminary experiments involving a search and rescue exer-cise in a burning building (see Figure 1). A Mote sensornetwork was deployed manually in the building to collecttemperature data and deliver the data to an outside point. A

Figure 8: 360 degree panorama of a room in the burning building taken by a catadioptric system. The image can be viewedusing a combination of a head tracker/head mounted display to enable a digital pan and tilt with very low latency.

network of robots navigated the space. A network of cam-eras took panoramic images and IR images that were sub-sequently used to localize the robots. A network of radiotags was also used for localization. Although these mod-ules were not integrated, the data collected during this ex-ercise was used off-site to test the algorithms described inthis paper. The firefighters who assisted us expressed greateagerness for having the kinds of support our vision pro-vides.

References

[KKS03] Derek Kurth, George Kantor, and Sanjiv Singh.Experimental results in range-only localizationwith radio. In Submitted to IEEE/RSJ Intl.Conf. on Intelligent Robots and Systems, 2003.

[KS02] G. Kantor and S. Singh. Preliminary results inrange only localization and mapping. InIEEEIntl. Conf. on Robotics and Automation, pages1819–1825, 2002.

[LdRR03] Q. Li, M. de Rosa, and D. Rus. Distributed al-gorithms for guiding navigation across a sensornet. InProceedings of Mobicom, 2003.

[PR02] R. Peterson and D. Rus. Interacting with a sen-sor network. InProc. of Australian Conf. onRobotics an Automation, 2002.

[SDF+01] J. Spletzer, A. K. Das, R. Fierro, C. J. Taylor,V. Kumar, and J. P. Ostrowski. Cooperative lo-calization and control for multi-robot manipu-lation. In IEEE/RSJ Intl. Conf. on IntelligentRobots and Systems, 2001.

[SKS02] S. Singh, G. Kantor, and D. Strelow. Recent re-sults in extensions to simultaneous localizationand mapping. InProc. of International Sympo-sium of Experimental Robotics, 2002.