Markov Localization for Reliable Robot

Navigation and People Detection

Dieter Foxyz, Wolfram Burgardy, and Sebastian Thrunz

yUniversity of Bonn, Department of Computer Science III, Bonn, GermanyzCarnegie Mellon University, School of Computer Science, Pittsburgh, PA

Abstract Localization is one of the fundamental problems in mobile robotics.Without knowledge about their position mobile robots cannot e�ciently carry outtheir tasks. In this paper we present Markov localization as a technique for esti-mating the position of a mobile robot. The key idea of this technique is to maintaina probability density over the whole state space of the robot within its environ-ment. This way our technique is able to globally localize the robot from scratchand even to recover from localization failures, a property which is essential fortruly autonomous robots. The probabilistic framework makes this approach robustagainst approximate models of the environment as well as noisy sensors. Based ona �ne-grained, metric discretization of the state space, Markov localization is ableto incorporate raw sensor readings and does not require prede�ned landmarks. Italso includes a �ltering technique which allows to reliably estimate the position ofa mobile robot even in densely populated environments. We furthermore describe,how the explicit representation of the density can be exploited in a reactive colli-sion avoidance system to increase the robustness and reliability of the robot evenin situations in which it is uncertain about its position. The method described herehas been implemented and tested in several real-world applications of mobile robotsincluding the deployments of two mobile robots as interactive museum tour-guides.

1 Introduction

The problem of estimating the position of a mobile robot within its envi-ronment belongs to the fundamental problems of mobile robotics [10,2]. Theknowledge about its position enables a mobile robot to carry out its taskse�ciently and reliably. In general, the problem is to estimate the location ofthe robot, i.e. its current state in its three-dimensional (x; y; �) con�gurationspace within its environment given a map and incoming sensory information.Methods of this type are regarded as so-called map-matching techniques,since they match measurements of standard sensors with the given modelof the environment. The advantage of map-matching techniques is that theydo not require any modi�cations of the environment and expensive specialpurpose sensors. However, there are several problems, these methods have todeal with. First, they must be able to deal with uncertain information comingfrom the inherent noise of sensory data. Second, models of the environment

are generally approximative especially if the environment is populated. Fi-nally, the methods have to deal with ambiguous situations which frequentlyarise for example in o�ce environments with long corridors.

The position estimation techniques developed so far can be distinguishedaccording to the type of problem they attack. Tracking or local techniquesaim at compensating odometric errors occurring during robot navigation.They require, however, that the initial location of the robot is known andoccasionally fail if they lost track of the robot's position. On the opposite sideare the so called global techniques which are designed to estimate the positionof the robot even under global uncertainty. Techniques of this type solve theso-called wake-up resp. initialization and kidnapped robot problems. Theycan estimate the position of the robot without any prior knowledge about it,and they can recover from situations in which the robot is exposed seriouspositioning errors coming for example from bumping into an object.

In this paper we present Markov localization as a technique for glob-ally estimating the position of the robot given a model of the environment.It uses a probabilistic framework and estimates a position probability den-sity over the whole state space of the robot. This allows the technique toglobally estimate the robot's position. In the beginning this technique startswith a uniform distribution over the whole three-dimensional state-space ofthe robot. By integrating sensory input this method keeps track of multi-ple hypotheses and incrementally re�nes the density until it ends up with auni-modal distribution. It furthermore can re-localize the robot in the caseof localization failures. Both properties are a basic precondition for truly au-tonomous robots which are designed to operate autonomously over longerperiods of time. Furthermore, Markov localization can deal with uncertaininformation. This is important because sensors such as ultrasound sensorsas well as models of the environment such as occupancy grid maps are gen-erally imperfect. Our method uses a �ne-grained and metric discretizationof the state space. This approach has several advantages. First, it providesaccurate position estimates which allow a mobile robot to e�ciently performtasks like o�ce delivery. Second, the method can integrate raw sensory inputsuch as a single beam of an ultrasound sensor. Most approaches for globalposition estimation, in contrast to that, rely on assumptions about the na-ture of the environment such as the orthogonality or the types of landmarksfound in that environment. Therefore, such techniques are prone to fail if theenvironment does not align well with these assumptions. Typical examplesof such environments can be found in the experimental results section of thispaper. Furthermore, our Markov localization technique includes a methodfor �ltering sensory input, which is designed to increase the robustness ofthe position estimation process especially in densely populated environmentssuch as museums or exhibitions. This way, our technique can even be ap-plied in dynamic environments in which most of the robot's sensor readingsdo not correspond to the expected measurements, since the senor beams are

re ected by people surrounding the robot or by other un-modelled objects.The technique has been implemented and proven robust in several long-termand real-world applications of mobile robots in populated environments.

The paper is organized as follows. In the next section we will describe themathematical framework of Markov localization. Section 3 introduces thegrid-based representation of the position probability density. It furthermorepresents techniques for e�ciently updating these densities in real-time andalso introduces two �ltering schemes to deal with obstacles in the robot'senvironment that are not contained in the map. Section 4 presents success-ful applications of Markov localization in real-world deployments of mobilerobots. Finally, we relate our approach to previous work in Section 5.

2 Markov Localization

The basic idea of Markov localization is to maintain a probability density overthe whole state space of the robot within its environment [26,29,7,19]. Markovlocalization assigns to each possible pose in the (x; y; �)-space of the robotin the given environment the probability that the robot is at that particularposition and has the corresponding orientation. Let Lt denote the randomvariable representing the state in the (x; y; �) space of the robot at time t.Thus, P (Lt = l) denotes the robot's belief that it was at location l at time t.The state Lt is updated whenever new sensory input is received or the robotmoves. Without loss of generality we assume that at every discrete point t intime �rst a measurement st is perceived and then a movement action at isperformed. Then, given a sensory input st Markov localization updates thebelief for each location l in the following way:

P (Lt = ljst) �t � P (st j l) � P (Lt = l) (1)

In this equation the term P (st j l) is denoted as the perception model since itdescribes the probability of measuring st at location l. The constant �t simplyis a normalizer ensuring that the left-hand side sums up to one over all l. Uponexecuting action at Markov localization applies the following formula comingfrom the domain of Markov chains to update the belief:

P (Lt+1 = l) Xl0

P (Lt+1 = l j Lt = l0; at) � P (Lt = l0 j st) (2)

The term P (Lt+1 = l j Lt = l0; at) is also called action model since it describesthe probability that the robot is at location l upon executing action at at aposition l0.

The belief P (L0) at time t = 0 re ects the knowledge about the start-ing position of the robot. If the position of the robot relative to its map isentirely unknown, P (L0) corresponds to a uniform distribution. If the initialposition of the robot is known with absolute certainty, then P (L0) is a Diracdistribution centered at this position.

3 Grid-based Markov Localization

Grid-based Markov localization | in contrast to other variants of Markovlocalization which use a topological discretization | uses a �ne-grained ge-ometric discretization to represent the position of the robot (see [7]). Morespeci�cally, L is represented by a three-dimensional �ne-grained, regularlyspaced grid, where the spatial resolution is usually between 10 and 40 cmand the angular resolution is usually 2 or 5 degrees. This approach has sev-eral desirable advantages. First, it provides very precise estimates for theposition of a mobile robot. Second, the high resolution grids allow the inte-gration of raw (proximity) sensor readings. Thus, the grid-based technique isable to exploit arbitrary geometric features of the environment such as thesize of objects or the width and length of corridors resp. rooms. It further-more does not require the de�nition of abstract features such as openings,doorways or other types of landmarks as it has to be done for the techniquesbased on topological discretizations.

A disadvantage of the �ne-grained discretization, however, lies in the hugestate space which has to be maintained. For a mid-size environment of size30 � 30m2, an angular grid resolution of 2�, and a cell size of 15 � 15cm2

the state space consists of 7; 200; 000 states. The basic Markov localizationalgorithm updates each of these states for each sensory input and each move-ment operation of the robot. To e�ciently update such large state spaces oursystem includes two techniques which are described in the remainder of thissection. The �rst optimization is the selective update strategy which focusesthe computation on the relevant part of the state space. The second methodis our sensor model which has been designed especially for proximity sensorsand allows the computation of the likelihood P (s j l) by two look-up oper-ations. Based on these two techniques, grid-based Markov localization canbe applied in real-time to estimate the position of a mobile robot during itsoperation.

Global localization techniques, which are based on matching sensor read-ings with a �xed model of the environment assume that the map re ectsthe true state of the world and therefore are prone to fail in highly dynamicenvironments in which crowds of people cover the robots sensors for longerperiods of time. To deal with such situations, our system also includes a �lter-ing technique that analyzes sensor readings according to whether or not theycome from dynamic and un-modelled aspects of the environment. Based onthis technique the robot can robustly operate even in populated environmentssuch as a museum.

3.1 Selective Update

An obvious disadvantage of the grid-based discretization comes from thesize of the state space. To integrate a single measurement st into the belief

~P (s) < P (s j l)~P (s) > P (s j l)

�

Figure1. Basic idea of the selective update technique

state all cells have to be updated. In this section we therefore describe atechnique which allows a selective update of the belief state. The key idea ofthis approach is to exclude unlikely positions from being updated. For thispurpose, we introduce a threshold � and approximate P (st j l) for cells withP (Lt�1 = l) � � by the probability ~P (st). The quantity ~P (st) is given by theaverage probability of measuring the feature st given a uniform distributionover all possible locations. This leads us to the following update rule for asensor measurement st:

P (Lt = ljst) �

(�t � P (st j l) � P (Lt = l) if P (Lt = l) > �

�t � ~P (st) � P (Lt = l) otherwise(3)

Since �t is a normalizing constant ensuring that P (Lt = l j st) sums up toone over all l, this is equivalent to

P (Lt = l j st) �

(~�t �

P (stjl)~P (st)

� P (Lt = l) if P (Lt = l) > �

~�t � P (Lt = l) otherwise(4)

Thus, all positions with a probability less or equal � only have to be updatedwith the normalizing constant ~�t which can simply be computed by summingup over all likely cells and adding to that n � ~P (st) �max fP (Lt)jP (Lt) � �gwhich is an approximation of the probability mass contained in the unlikelycells. In this term n is the number of states with P (Lt) � �. Since all thesestates are multiplied with the same value, it su�ces to update only onevariable instead of all of unlikely cells (see [5]).

At this point it should be noted that the approximation of P (s j l) by~P (s) for a measurement s is a conservative approximation, since P (s j l) isusually below the average ~P (s) at unlikely positions l.

Figure 1 illustrates the key idea of the selective update scheme. In thisexample the belief is concentrated on a single peak which sticks out of the sea.The water level represents the maximum probability of all unlikely states.

Whenever the estimated position corresponds to the true location of therobot, the probability P (s j l) exceeds ~P (s) and the belief is con�rmed sothat the level of the sea goes down. However, if the system loses track ofthe robot's position, then the obtained measurements no longer match theexpected measurements and P (s j l) < ~P (s). Thus the robot's certaintydecreases and the level of the sea increases. As soon as the sea level exceeds�, the so far unlikely states are updated again.

In extensive experimental tests we did not observe evidence that the se-lective update scheme impacts the robot's behavior in any noticeable way.In general, the probabilities of the active locations sum up to at least 0.99.During global localization, the certainty of the robot permanently increasesso that the density quickly concentrates on the true position of the robot. Assoon as the position of the robot has been determined with high certainty,a scan comprising 180 laser measurements is typically processed in less than0.1 seconds using a 200MHz Intel Pentium Pro.

The advantage of the selective update scheme is that the computationtime required to update the belief state adapts automatically to the certaintyof the robot. This way, our system is able to e�ciently track the position ofa robot once its position has been determined. Thereby, Markov localizationkeeps the ability to detect localization failures and to re-localize the robot.The only disadvantage lies in the �xed representation of the grid which has theundesirable e�ect that the space requirement in our current implementationstays constant even if only a minor part of the state space is updated. In thiscontext we would like to mention that recently promising techniques havebeen presented to overcome this disadvantage by applying alternative anddynamic representations of the state space [6,5,12].

3.2 The Model of Proximity Sensors

As mentioned above, the likelihood P (s j l) that a sensor reading s is mea-sured at position l has to be computed for all positions l in each update cycle.Therefore, it is crucial for on-line position estimation that this quantity canbe computed very e�ciently. In [25] Moravec proposed a method to computea generally non-Gaussian probability density function P (s j l) over a discreteset of possible distances measured by the sensor for each location l. In a �rstimplementation of our approach [7] we used a similar method, which unfor-tunately turned out to be computationally too expensive for on-line positionestimation. Furthermore, the pre-computation of all these densities is notfeasible since it requires an enormous amount of memory.

To overcome these disadvantages, we developed a perception model whichallows us to compute P (s j l) by two look-up operations. The key idea is tostore for each location l in the (x; y; �) space the distance ol to the nextobstacle in the map. This distance can be extracted by ray-tracing from

100200

300400

100

200

300

400

5000

0.1

0.2

0.3

0.4

expected distance [cm]

measured distance [cm]

probability

(a)

100200

300400

100

200

300

400

5000

0.1

0.2

0.3

0.4

expected distance [cm]

measured distance [cm]

probability

(b)

Figure 2. Measured (a) and approximated (b) densities for an ultrasound sensor.

occupancy grid maps or CAD-models of the environment. In order to storethese distances compactly, we use a discretization d0; : : : ; dn�1 of possibledistances measured by a proximity sensor and store for each location onlythe index of the expected distance. Thus, to compute the probability P (st j l)it remains to determine the probability P (st j ol) of measuring the value stgiven the expected distance ol.

In our approach, the density P (s j ol) is de�ned as a mixture of a Gaussiandensity centered around the expected distance ol and a geometric distribu-tion [17,11]. The geometric distribution is designed to allow the system todeal with a certain amount of un-modelled objects such as people walkingby. In order to determine the parameters of this model we collected severalmillion data pairs consisting of the expected distance ol and the measureddistance di during the typical operation of the robot. Based on these data weadopted the parameters of the sensor model so as to best �t the measureddata. The measured data and resulting densities for ultrasound sensors aredepicted in Figure 2. The similarity between the measured and the approxi-mated distributions shows that our sensor model yields a good approximationof the data.

Based on these densities we now can compute P (s j l) simply by twonested look-up operations. After retrieving the expected distance ol for thecurrently considered location l, we compute P (s j ol) by a further look-upoperation in the corresponding table.

3.3 Filtering Techniques for Dynamic Environments

The perception model described in the previous section assigns a �xed prob-ability to every pair of measured and expected distances. Although it alsoincorporates a certain amount of un-modelled objects, it is only capable tomodel such noise on average. While this approach showed to reliably dealwith occasional sensor blockage, it is not su�cient in situations where a largeamount of all sensor readings are corrupted. In general, localization tech-

RHINO

(a) (b)

Figure 3. The mobile robots Rhino (a) and Minerva (b)acting as interactive museum tour-guides.

niques based on map-matching are prone to fail if the assumption that themap represents the state of the world is severely violated [14]. Consider, forexample, a mobile robot which operates in an environment in which largegroups of people permanently cover the robots sensors and thus lead to un-expected measurements. The mobile robots Rhino and Minerva which weredeployed as interactive museum tour-guides in the Deutsches Museum Bonn,Germany and in the National Museum of American History, Washington DC

respectively [4,31] were permanently faced with such a situation. Figure 3shows cases in which the robots were surrounded by many visitors while giv-ing a tour through the museum. To deal with such situations, we developedtwo �ltering techniques which are designed to detect whether a certain sensorreading is corrupted or not. The basic idea of these �lters is to sort all mea-surements into two buckets, one that contains all readings believed to re ectthe true state of the world and another one containing all readings believedto be caused by un-modelled obstacles surrounding the robot. Compared toa �xed integration in the perception model, these �lters have the advan-tage that they select the maximum amount of information from the obtainedreadings and automatically adopt their behaviour according to whether theenvironment is empty or highly populated.

Figure 4 contains two typical laser scans obtained in such situations. Ob-viously, the readings are to a large extend corrupted by the people in themuseum which are not contained in the static world model. The di�erentshading of the beams indicates the two classes they belong to: the black linescorrespond to static obstacles that are part of the map, whereas the grey-shaded lines are those beams re ected by visitors in the Museum (readingsabove 5m are excluded for the sake of clarity). Since people standing closeusually increase the robot's belief of being close to modelled obstacles, therobot quickly loses track of its position when taking all incoming sensor dataseriously.

In the remainder of this section we introduce two di�erent kinds of �lters.The �rst one is called entropy �lter which can be applied to arbitrary sensors,

(a) (b)

Figure 4. Typical laser scans obtained when Rhino is surrounded by visitors.

since it �lters a reading according to its e�ect on the belief P (L). The second�lter is the distance �lter which selects the readings according to how muchshorter they are than the expected value. It therefore is especially designedfor proximity sensors.

The Entropy Filter The entropy H(L) of the belief over L is de�ned as

H(L) = �Xl

P (L = l) logP (L = l) (5)

and is a measure of uncertainty about the outcome of the random variableL [8]: The higher the entropy, the higher the robot's uncertainty as to whereit is. The entropy �lter measures the relative change of entropy upon incor-porating a sensor reading into the belief P (L). More speci�cally, let s denotethe measurement of a sensor (in our case a single range measurement). Thechange of the entropy of P (L) given s is de�ned as:

�H(L j s) := H(L)�H(L j s) (6)

Here, H(L j s) is the entropy of the belief P (L j s). While a negative changeof entropy indicates that after incorporating s, the robot is less certain aboutits position, a positive change indicates an increase in certainty. The selectionscheme of the entropy �lter is:

Exclude all sensor measurements s with �H(L j s) < 0.

Thus, the entropy �lter makes robot perception highly selective, in that itconsiders only sensor readings con�rming the robot's current belief.

The Distance Filter The advantage of the entropy �lter is that it makes noassumptions about the nature of the sensor data and the kind of disturbances

occurring in dynamic environments. In the case of proximity sensors, however,un-modelled obstacles produce readings that are shorter than the distancethat can be expected given the map. The distance �lter which selects sensorreadings based on their distance relative to the distance to the closest obstaclein the map removes those sensor measurements s which with probabilityhigher than � (which we generally set to 0:99) are shorter than expected,i.e. caused by an un-modelled object.

0

0.2

0.4

0.6

0.8

1

100 200 300 400 500

prob

abili

ty

measured distance [cm]

Pshort(di j l)Pm(di j l)ol

Figure 5. Probability Pm(di j l) of expected measurement and probabilityPshort(di j l) that a distance di is shorter than the expected measurement.

Consider a discrete set of possible distances d1; : : : ; dn measured by aproximity sensor. Let Pm(di j l) = P (di j ol) denote the probability of mea-suring distance di if the robot is at position l and the sensor detects the nextobstacle in the map. The distribution Pm describes the sensor measurementexpected from the map. This distribution is assumed to be Gaussian withmean at the distance ol to the next obstacle. The dashed line in Figure 5represents Pm for a laser-range �nder and a distance ol of 230cm. Given Pmwe now can de�ne the probability Pshort(di j l) that a measured distance di isshorter than the expected one given the robot is at position l. This probabil-ity is obviously equivalent to the probability that the expected measurementis longer than the ol and can be computed as:

Pshort(di j l) =Xj>i

Pm(dj j l): (7)

Pshort(di j l) is the probability that a measurement di is shorter than expectedgiven the robot is at location l. In practice, however, we are interested in theprobability Pshort(di) that di is shorter than expected given the current beliefof the robot. Thus, we have to average over all possible positions of the robot:

Pshort(di) =Xl

Pshort(di j l)P (L = l) (8)

Based on Pshort(di) we now can de�ne the distance �lter as:

Exclude all sensor measurements di with Pshort(di) > �.

4 Applications of Markov Localization in Real-world

Environments

The grid-based Markov localization technique including the �lter extensionhas been implemented and proven robust in various environments. In thissection we illustrate the application of Markov localization in the context ofthe deployments of the mobile robots Rhino [4] and Minerva [31] as interac-tive museum tour-guide robots (see Figure 3). Rhino [3,32] has a ring of 24ultrasound sensors each with an opening angle of 15 degrees. Both, Rhinoand Minerva are equipped with two laser-range �nders covering 360 degreesof their surrounding.

(a)

robot

(b)

Figure 6. Global localization in the Deutsches Museum Bonn. The left image (a)shows the belief state after incorporating one laser scan. After incorporating two

more scans, the robot uniquely determined its position (b).

4.1 Reliable Position Estimation in Populated Environments

During the deployment of the mobile robots Rhino and Minerva in theDeutsches Museum Bonn resp. the National Museum of American History(NMAH) in Washington DC the reliability of the localization system wasa precondition for the success of the entire mission. Although both robotsused the laser-range �nder sensors for (global) localization, they had to dealwith situations in which more than 50% of all sensor readings were corruptedbecause large crowds of people surrounded the robot for longer periods oftime [14].

Rhino used the entropy �lter to identify sensor readings that were cor-rupted by the presence of people. Since this �lter integrates only readingswhich do not increase the uncertainty, it is restricted to situations in whichthe robot knows its approximate location. Unfortunately, when using the en-tropy �lter, the robot is not able to recover from situations in which therobot loses its position entirely. To prevent this problem, Rhino additionally

Duration: 4.8 hoursDistance: 1540 meters (a)

Duration: 1 hourDistance: 2000 meters

(b)

Figure 7. Typical trajectories of the robot Rhino in the Deutsches Museum Bonn(and) Minerva in the National Museum of American History (b).

incorporated a small number of randomly chosen sensor readings into thebelief state. Based on this technique, Rhino's localization module was able to(1) globally localize the robot in the morning when the robot was switchedon and (2) to reliably and accurately keep track of the robot's position. Fig-ure 6 shows the process of global localization in the Deutsches Museum Bonn.RHINO is started with a uniform distribution over its belief state. The prob-ability distribution given after integrating the �rst sensor scan is shown inFigure 6(a). After incorporating two more sensor scans, the robot knows itsposition with high certainty (see Figure 6(b)). Based on the selective up-date mechanism, the robot could e�ciently keep track of the robot's positionafter determining it uniquely. Figure 7(a) shows a typical trajectory of therobot Rhino in the museum in Bonn. In the entire six-day deployment pe-riod Rhino travelled over 18km. Its maximum speed was over 80cm/sec andthe average speed was 37cm/sec. Rhino completed 2394 out of 2400 requestswhich corresponds to a success rate of 99.75%.

Figure 7(b) shows a trajectory of 2 km of the robot Minerva in the Na-tional Museum of American History. Minerva used the distance �lter to iden-tify readings re ected by un-modelled objects. Based on this technique, Min-erva was able to operate reliably over a period of 13 days. During that timeover 50.000 people were in the museum and watched or interacted with therobot. At all Minerva travelled 44km with a maximum speed of 1.63m/sec.

In an extensive experimental comparison [14,11] it has been demonstratedthat both �lters signi�cantly improve the robustness of the localization pro-cess especially in the presence of large amounts of sensor noise.

4.2 Probabilistic Integration of Map Information into a Reactive

Collision Avoidance System

During the deployments of Rhino and Minerva as interactive museum tour-guides, safe and collision-free navigation was of uttermost importance. In theDeutsches Museum Bonn, the collision avoidance was complicated by the fact

o1

o2

Laser rangefinder

Tactiles

Infrared

Sonars

Stereo camera

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 100 200 300 400 500

P(d)

distance d [cm]

Certain position

Uncertain position

Figure 8. The robot Rhinoand its sensors.

Figure 9. Probability densitiesP (Sm = s) and selected measurement s�.

that many of the obstacles and exhibits could not be sensed by the robot'ssensors although Rhino possessed �ve state-of-the-art sensor systems (vision,laser, sonar, infrared, and tactile). Invisible obstacles included glass cagesput up to protect exhibits, metal bars at various heights, and small podiumsor metal plates on which exhibits were placed (c.f., the glass cage labelled\o1" and the control panel label \o2" in Figure 8). Furthermore, the area inwhich the robots were intended to operate was limited due to un-modelledregions or dangerous areas such as staircases. Therefore, both robots neededa technique which forces the robots not to leave their area of operation.

The collision avoidance system of the robots is based on the dynamicwindow algorithm (DWA) [13], a purely sensor-based approach designed toquickly react to obstacles blocking the robot's path. The basic DWA, justlike any other sensor-based collision avoidance approach, does not allow forpreventing collisions with obstacles which are \invisible" to the robot's sen-sors. To avoid collisions with those, the robot had to consult its map. Ofcourse, obstacles in the map are speci�ed in world coordinates, whereas reac-tive collision avoidance techniques such as DWA require the location of thoseobstacles in robo-centric coordinates.

At �rst glance, one might be inclined to use the maximum likelihood po-sition estimate produced by Rhino's localization module, to convert worldcoordinates to robo-centric coordinates, thereby determining the locationof \invisible" obstacles relative to the robot. However, such a methodologywould be too brittle in situations where the localization module assigns highprobability to multiple poses, which we observed quite frequently especiallywhen the robot moves at high speeds. Our approach is more conservative.Based on the map, it generates \virtual" sensor readings that underestimatethe true distance to the next obstacle in the map with probability 0:99. Morespeci�cally, let Sm be the proximity measurement that one would expect if

all invisible obstacles were actually detectable. Then

P (Sm = s) =Xl

P (Sm = s j Lt = l) P (Lt = l) (9)

is the probability distribution that s is the distance to the next obstacle inthe map given the current belief state of the robot. Now the measurement s�

which underestimates the true distance to the next obstacle in the map withprobability �, is obtained as

s� := maxfs j P (Sm > s) � �g (10)

where

P (Sm > s) :=Xs0>s

P (Sm = s0): (11)

Figure 9 depicts two di�erent densities P (Sm = s) based on two di�erentbelief states. Whereas the solid line corresponds to a situation, in which therobot is highly uncertain about its position, the dashed line comes from atypical belief state representing high certainty. This �gure illustrates, thatthe robot conservatively chooses a very small distance s� if it is uncertainabout its position and a distance close to the true distance in the case of highcertainty.

Our extension of the DWA worked well in the museum. Figure 7 shows a1.6 km-long trajectory of the robot Rhino in the Deutsches Museum Bonn.The location of the \invisible" obstacles are indicated by the gray-shadedareas, which the robot safely avoided. By adding appropriate obstacles tothe map, the same mechanism was used to limit the operational range of therobots. With it, Rhino and Minerva successfully avoided to enter any terrainthat was out of their intended operational range.

4.3 Reacting to People Blocking the Path of the Robot

In the museum, people were not always cooperative. A typical behaviour ofvisitors was to intentionally block the robot's path for longer periods of time.One example of such a situation with Rhino in the Deutsches Museum Bonnis shown in Figure 10. Here a group of visitors tries to challenge the robot byforming a u-shaped obstacle. To deal with such situations, the robot neededthe capability to detect that un-modelled obstacles in the environment areblocking its path. Rhino and Minerva applied the distance �lter to detectsuch situations, since people standing close to the robot lead to readings thatare shorter than expected. If the path of the robot was blocked for a certainamount of time, then Rhino blew its horn to ask for clearance [4]. Minervaused a more sophisticated way of interaction [31]. It possessed a face (see

Figure 11) allowing it to express di�erent moods according to the durationof the blockage. This face was mounted on a pan-tilt head and pointed to theperson standing in the direction the robot was supposed to move. Minervachanged the mimic of the face according to the duration of the blocking.The moods ranged from \happy" (Figure 11(a)) to \angry" (Figure 11(b)).Minerva additionally used di�erent phrases of pre-recorded texts to expressthe desire for free space.

(a) (b)

Figure 10. Typical situation in which visitors try to challenge the robot byintentionally blocking its path.

(a) (b)

Figure 11. The di�erent moods of the robot Minerva according to its progressranged from happy (a) to angry (b).

Rhino's and Minerva's ability to react directly to people was among themost entertaining aspects, which contributed enormously to its popularityand success. Many visitors were amazed by the fact that the robots acknowl-edged their presence by blowing its horn, changing the mimic or explicitelyasking to stay behind the robot. One e�ect was that they repeatedly steppedin its way to get the acoustic \reward." Nevertheless, the ability to detectsuch situations and to interactively express their intentions allowed the robotsto reliably reach their goals.

5 Related Work

Most of the techniques for position estimation of mobile robots developed sofar belong to the class of local approaches resp. tracking techniques which aredesigned to compensate odometric error occurring during navigation giventhat the initial position of the robot is known (see [2] for a comprehensiveoverview). Wei� et. al. [33] store angle histograms derived from laser-range�nder scans taken at di�erent locations in the environment. The position andorientation of the robot is calculated by maximizing the correlation betweenthe stored histograms and laser-range scans obtained while the robot movesthrough the environment. The estimated position together with the odom-etry information is then used to predict the position of the robot and toselect the histogram used for the next match. Yamauchi [34] applies a similartechnique, but uses hill-climbing to match local maps built from ultrasonicsensors against a given occupancy grid map. As in [33], the location of therobot is represented by the position yielding the best match.

A very popular mathematical framework for position tracking are Kalman�lters [24], a technique that was introduced by Kalman in 1960 [20]. Kalman�lter based methods represent their belief about the position of the robot by aunimodal Gaussian distribution over the three-dimensional state-space of therobot. The existing applications of Kalman �ltering to position estimation formobile robots are similar in how they model the motion of the robot. Theydi�er mostly in how they update the Gaussian according to new sensory in-put. Leonard and Whyte [22] match beacons extracted from sonar scans withbeacons predicted from a geometric map of the environment. These beaconsconsist of planes, cylinders, and corners. To update the current estimate ofthe robot's position, Cox [9] matches distances measured by infrared range�nders against a line segment description of the environment. Schiele andCrowley [27] compare di�erent strategies to track the robots position basedon occupancy grid maps and ultrasonic sensors. They show that matchinglocal occupancy grid maps against a global grid map results in a similar local-ization performance as if the matching is based on features that are extractedfrom both maps. Sha�er et. al. [28] compare the robustness of two di�erentmatching techniques against di�erent sources of noise. They suggest a com-bination of map-matching and feature-based techniques in order to inheritthe bene�ts of both. Gutmann, Lu, and Milios [16,23] use a scan-matchingtechnique to precisely estimate the position of the robot based on laser-range�nder scans and learned models of the environment. Arras and Vestli [1] usea similar technique to compute the position of the robot with a very highaccuracy. All these variants, however, rest on the assumption that the posi-tion of the robot can be represented by a single Gaussian distribution. Theadvantage of this approach lies in the high accuracy that can be obtained.The assumption of a unimodal Gaussian distribution, however, is not justi-

�ed if the position of a robot has to be estimated from scratch, i.e. withoutknowledge about the starting position of the robot.

To overcome these disadvantages, recently di�erent variants of Markov lo-calization have been developed and employed successfully [26,29,19,7,18,30].The basic idea of Markov localization is to maintain an arbitrary and notnecessarily Gaussian position probability density over the whole three-dimen-sional hx; y; �i state space of the robot in its environment. The di�erentvariants of this technique can be roughly distinguished by the type of dis-cretization used for the representation of the state space. In [26,29,19,18,30]Markov localization is used for landmark-based corridor navigation and thestate space is organized according to the topological structure of the envi-ronment. Based on an orthogonality assumption [26,29,19] consider only fourpossible headings of the robot. Our �ne-grained and grid-based approach ofMarkov localization has the advantage that it provides accurate position esti-mates and that it can be applied in arbitrary unstructured environments. Thedisadvantage of this technique, however, lies in its computational complexityand space requirements. In order to represent the whole state space of therobot within its environment, usually several million states have to be repre-sented and updated. In this paper we therefore present di�erent techniques toovercome these disadvantages. The result is an e�cient and accurate positionestimation technique for mobile robots. Our implementation of Markov local-ization is able to globally estimate the position of the robot from scratch andto e�ciently keep track of the robots position once it has been determined.Simultaneously, it allows the robot to recover from localization failures. Ina recent experimental comparison it has been demonstrated that Kalman�lter based tracking techniques provide highly accurate position estimatesbut are less robust than Markov localization since they lack the ability toglobally localize the robot and to recover from localization errors [15]. Ourtechnique furthermore includes an approach to �lter out readings comingfrom objects that are not contained in the map. This way our approach,in contrast to other Markov localization techniques, is able to reliably esti-mate the position of the robot even in densely populated and highly dynamicenvironments [4,11,14,31].

6 Discussion

In this paper we presented Markov localization as a robust technique forestimating the position of a mobile robot. The key idea of Markov localizationis to maintain a position probability density over a �ne-grained discretizationof the robot's state space within the environment. This density is updatedwhenever new sensory input is received and whenever the robot moves.

We introduced two techniques for e�ciently updating the density. First,we use a selective update scheme which focuses the computation on the rele-

vant parts of the state space. Second, we apply a sensor model that allows toe�ciently compute the necessary quantities by two look-up operations. Basedon these two approaches, the belief state can be updated in real-time. Ourapproach to Markov localization furthermore includes �ltering techniques sig-ni�cantly increasing the robustness of the position estimation process even indensely populated or highly dynamic environments. We presented two kindsof �lters. The entropy �lter is a general �lter, that can be applied to arbitrarysensors including cameras. The distance �lter has been developed especiallyfor proximity sensors and therefore provides better results with sensors likea laser-range �nder.

Our technique has been implemented and evaluated in several experimentsat various sites. During the deployments of the mobile robots Rhino in theDeutsches Museum Bonn, Germany, and Minerva in the National Museumof American History, Washington DC, our system was successfully applied.The accuracy obtained by the grid-based and �ne-grained discretization ofthe state space turned out to be high enough to avoid even obstacles thatcould not be sensed by the robot's sensors. It furthermore provided the basisfor detecting situations, in which people intentionally block the path of therobot.

There are several aspects which are objectives of future research. First,the current implementation of Markov localization uses a �xed discretiza-tion of the whole state space which is always kept in memory. To overcomethis disadvantage, recently di�erent alternative representations of the densityhave been developed and suggested [6,5,15]. Whereas [6] uses a local grid fore�cient position tracking, we introduced an Octree-based representation ofthe robot's state space in [5] which allows to dynamically adopt the requiredmemory as well as the resolution of the discretization. [15] suggest a com-bination of Markov localization with Kalman �ltering which should lead toa system inheriting the advantages of both approaches. A further restrictionof the current system is that the model of the environment is assumed to bestatic. In this context the problem to combine map building and localizationon-line is an interesting topic for future research.

References

1. K.O. Arras and S.J. Vestli. Hybrid, high-precision localization for the maildistributing mobile robot system MOPS. In Proc. of the IEEE InternationalConference on Robotics and Automation, 1998.

2. J. Borenstein, B. Everett, and L. Feng. Navigating Mobile Robots: Systems andTechniques. A. K. Peters, Ltd., Wellesley, MA, 1996.

3. J. Buhmann, W. Burgard, A.B. Cremers, D. Fox, T. Hofmann, F. Schneider,J. Strikos, and S. Thrun. The mobile robot Rhino. AI Magazine, 16(2), 1995.

4. W. Burgard, A.B. Cremers, D. Fox, D. H�ahnel, G. Lakemeyer, D. Schulz,W. Steiner, and S. Thrun. The interactive museum tour-guide robot. In Proc.ofthe Fifteenth National Conference on Arti�cial Intelligence, 1998.

5. W. Burgard, A. Derr, D. Fox, and A.B. Cremers. Integrating global positionestimation and position tracking for mobile robots: The dynamic markov lo-calization approach. In Proc. of the IEEE/RSJ International Conference onIntelligent Robots and Systems, 1998.

6. W. Burgard, D. Fox, and D. Hennig. Fast grid-based position tracking for mo-bile robots. In Proc. of the 21st German Conference on Arti�cial Intelligence,Germany. Springer Verlag, 1997.

7. W. Burgard, D. Fox, D. Hennig, and T. Schmidt. Estimating the absoluteposition of a mobile robot using position probability grids. In Proc. of theThirteenth National Conference on Arti�cial Intelligence, 1996.

8. T.M. Cover and J.A. Thomas. Elements of Information Theory. Wiley, 1991.

9. I.J. Cox. Blanche { an experiment in guidance and navigation of an autonomousrobot vehicle. IEEE Transactions on Robotics and Automation, 7(2):193{204,4 1991.

10. I.J. Cox and G.T. Wilfong, editors. Autonomous Robot Vehicles. SpringerVerlag, 1990.

11. D. Fox. Markov Localization: A Probabilistic Framework for Mobile Robot Lo-calization and Navigation. PhD thesis, Dept of Computer Science, Univ. ofBonn, Germany, 1998.

12. D. Fox, W. Burgard, F. Dellaert, and S. Thrun. Monte carlo localization|e�cient position estimation for mobile robots. In Proc.of the Sixteenth NationalConference on Arti�cial Intelligence, Orlando, Fl, 1999.

13. D. Fox, W. Burgard, and S. Thrun. The dynamic window approach to collisionavoidance. IEEE Robotics and Automation, 4(1), 1997.

14. D. Fox, W. Burgard, S. Thrun, and A.B. Cremers. Position estimation for mo-bile robots in dynamic environments. In Proc.of the Fifteenth National Con-ference on Arti�cial Intelligence, 1998.

15. J.-S. Gutmann, W. Burgard, D. Fox, and K. Konolige. An experimental com-parison of localization methods. In Proc. of the IEEE/RSJ International Con-ference on Intelligent Robots and Systems, 1998.

16. J.-S. Gutmann and C. Schlegel. AMOS: Comparison of scan matching ap-proaches for self-localization in indoor environments. In Proceedings of the1st Euromicro Workshop on Advanced Mobile Robots. IEEE Computer SocietyPress, 1996.

17. D. Hennig. Globale und lokale Positionierung mobiler Roboter mittelsWahrscheinlichkeitsgittern. Master's thesis, Department of Computer Science,University of Bonn, Germany, 1997. In German.

18. J. Hertzberg and F. Kirchner. Landmark-based autonomous navigation in sew-erage pipes. In Proc. of the First Euromicro Workshop on Advanced MobileRobots. IEEE Computer Society Press, 1996.

19. L.P. Kaelbling, A.R. Cassandra, and J.A. Kurien. Acting under uncer-tainty: Discrete bayesian models for mobile-robot navigation. In Proc. of theIEEE/RSJ International Conference on Intelligent Robots and Systems, 1996.

20. R.E. Kalman. A new approach to linear �ltering and prediction problems.Tansaction of the ASME { Journal of basic engineering, pages 35{45, March1960.

21. D. Kortenkamp, R.P. Bonasso, and R. Murphy, editors. MIT/AAAI Press,Cambridge, MA, 1998.

22. J.J. Leonard and H.F. Durrant-Whyte. Mobile robot localization by trackinggeometric beacons. IEEE Transactions on Robotics and Automation, 7(3):376{382, 1991.

23. F. Lu and E. Milios. Robot pose estimation in unknown environments bymatching 2d range scans. In IEEE Computer Vision and Pattern RecognitionConference (CVPR), 1994.

24. P.S. Maybeck. Autonomous robot vehicles. In Cox and Wilfong [10].25. H.P. Moravec. Sensor fusion in certainty grids for mobile robots. AI Magazine,

Summer 1988.26. I. Nourbakhsh, R. Powers, and S. Birch�eld. DERVISH an o�ce-navigating

robot. AI Magazine, 16(2), Summer 1995.27. B. Schiele and J.L. Crowley. A comparison of position estimation techniques us-

ing occupancy grids. In Proc. of the IEEE International Conference on Roboticsand Automation, 1994.

28. G. Sha�er, J. Gonzalez, and A. Stentz. Comparison of two range-based estima-tors for a mobile robot. In SPIE Conf. on Mobile Robots VII, pages 661{667,1992.

29. R. Simmons and S. Koenig. Probabilistic robot navigation in partially observ-able environments. In Proc. of the International Joint Conference on Arti�cialIntelligence, 1995.

30. S. Thrun. Bayesian landmark learning for mobile robot localization. MachineLearning, 33(1), 1998.

31. S. Thrun, M. Bennewitz, W. Burgard, A.B. Cremers, F. Dellaert, D. Fox,D. H�ahnel, C. Rosenberg, N. Roy, J. Schulte, and D. Schulz. MINERVA: Asecond-generation museum tour-guide robot. In Proceedings of the IEEE In-ternational Conference on Robotics and Automation, 1999.

32. S. Thrun, A. B�ucken, W. Burgard, D. Fox, T. Fr�ohlinghaus, D. Hennig, T. Hof-mann, M. Krell, and T. Schimdt. Map learning and high-speed navigation inRHINO. In Kortenkamp et al. [21].

33. G. Wei�, C. Wetzler, and E. von Puttkamer. Keeping track of position andorientation of moving indoor systems by correlation of range-�nder scans. InProc. of the IEEE/RSJ International Conference on Intelligent Robots andSystems, 1994.

34. B. Yamauchi. Mobile robot localization in dynamic environments using deadreckoning and evidence grids. In Proc. of the 1996 IEEE International Confer-ence on Robotics and Automation, 1996.