Improving Robot Navigation ThroughSelf-Supervised Online Learning

Boris Sofman, Ellie Lin, J. Andrew Bagnell, Nicolas Vandapel, and Anthony StentzRobotics Institute

Carnegie Mellon UniversityPittsburgh, PA 15213

Email: {bsofman,elliel,dbagnell,vandapel,axs}@ri.cmu.edu

Abstract— In mobile robotics, there are often features that,while potentially powerful for improving navigation, prove dif-ficult to profit from as they generalize poorly to novel situa-tions. Overhead imagery data, for instance, has the potential togreatly enhance autonomous robot navigation in complex out-door environments. In practice, reliable and effective automatedinterpretation of imagery from diverse terrain, environmentalconditions, and sensor varieties proves challenging. We introducean online, probabilistic model to effectively learn to use thesescope-limited features by leveraging other features that, whileperhaps otherwise more limited, generalize reliably. We apply ourapproach to provide an efficient, self-supervised learning methodthat accurately predicts traversal costs over large areas fromoverhead data. We present results from field testing on-board arobot operating over large distances in off-road environments.Additionally, we show how our algorithm can be used offline toproduce a priori traversal cost maps and detect misalignmentsbetween overhead data and estimated vehicle positions. Thisapproach can significantly improve the versatility of manyunmanned ground vehicles by allowing them to traverse highlyvaried terrains with increased performance.

I. I NTRODUCTION

A common problem that arises in mobile robotics is thatpotentially powerful sensor data and features are often difficultto take advantage of because they are situation or locationspecific. For instance, camera imagery can potentially detectunpaved road in the desert significantly farther than someladar-based systems can. Detecting such roads from a distanceproved to be a crucial component in Stanford Racing’s win-ning Grand Challenge entry [1]. Unfortunately, such data canprove very resistant to automated interpretation. In particular,classifiers that prove to be powerful indicators of road in aparticular area often do not generalize to new conditions.

Similarly, outdoor robot navigation can benefit from thenow widespread availability of high quality overhead imageryand elevation data from satellite and aircraft [2][3]. Presently,nearly the entire world has been surveyed at1 m accuracy, withhigher resolution (better than25 cm accuracy) data available incertain areas. With this overhead data, many of the difficultiesassociated with autonomous robot operation are alleviated,even with the coarsest of terrain resolution. Systems can thendispense with myopic exploration and instead pursue routesthat are likely to be effective.

Unfortunately, leveraging this tremendous resource on anautonomous robot proves difficult. Building systems that

(a) (b)

Fig. 1. Sample results of terrain traversal cost predictions. 0.35 m resolutioncolor overhead imagery used by our online learning algorithm(a) andcorresponding predictions of terrain traversal costs (b).Traversal costs arecolor-scaled for improved visibility. Blue and red correspond to lowest andhighest traversal cost estimates, respectively.

can reliably interpret overhead image data is far from easy.Additionally, such estimates must be well calibrated withother on-board perception estimates of the terrain, or systemperformance may suffer.

In this paper, we address the problem of learning andinferring between two heterogeneous data sources that varyin density, accuracy and scope of influence. The objectiveis to generalize from one data source, viewed as a reliableestimate, to be able to work with another, which may be highperformance (e.g., long range or high accuracy) but difficultto generalize to new environments. We frame the problemas a simple, linear probabilistic model for which inferenceresults in a self-supervised online learning algorithm that fusesthe estimates from the two data sources. We discuss theadvantages of this framework including reversible learning,feature selection, data alignment capabilities, reliableuse ofmultiple estimates, as well as confidence-rated predictions.

Furthermore, we demonstrate the approach in the contextof long range navigation of a large, unmanned ground vehicleduring various field tests in complex natural environments.As it traverses an environment, the vehicle utilizes its on-board perception system and overhead imagery and elevationdata to learn the mapping from overhead data features tocomputed terrain traversal costs in order to predict traversalcosts elsewhere in the environment where only overhead datais available, effectively extending the range of the vehicle’slocal perception system and allowing more effective navigation

of the environment. This approach removes the necessity ofhuman involvement and parameter engineering, which limitsthe versatility of many robotic systems.

II. A PPROACH

A. Formalization

We approach the problem of leveraging the powerful, butdifficult to generalize, features in a Bayesian probabilisticframework using the notion of scoped learning [4]. The scopedlearning model admits the idea of two types of features:“global” and “local”. Global features are generally useful, andtheir predictive power extends well to new domains, whilelocal ones, which, although often very powerful, typicallygeneralize poorly and are more difficult to take advantage ofin a consistent way. These local features havescopethat islimited to one particular domain. We wish to apply our systemto extend the scope of such features to many possible domains.For our canonical problem of learning to leverage the extendedrange of overhead data, these names may prove counter-intuitive, so we refer to them instead asgeneraland locale-specificfeatures. Features generated from dense, vehicle-basedladar perception serve as our general features, while featuresgenerated from overhead based imagery and elevation dataserve as our locale-specific features. The latter are particularlyvaluable to mobile robots because of their extended range andwidespread availability.

1) Model: The scoped learning approach is a very sim-ple probabilistic model (shown graphically in Figure2) thatcaptures this notion of features that have scope. The outerplate L represents in graphical model notation that there areindependent locales in which the model will be applied [5].These correspond to new areas of the world in which our robotwill operate.

Within the plate, we see a sequence of locale-specificfeatures and corresponding general feature-based estimates.At each point in the sequence, we wish to make predictionsabout c (either all or a subset of them.) Here,c is the truevariable we wish to predict, andc is an estimate of that variablecoming from the general features, whilex are our locale-specific features. The parametersβ common to the locale(plate) capture the relationship between locale-specific featuresand the variables of interestc. The length of our sequence isn.

This learning model captures the idea of self-supervisedlearning [6] in a Bayesian framework and extends the idea tointegrate both the general feature-based estimates and theself-supervised locale-specific estimates. Driven by our application,we are particularly interested in the onlineregressioncase1

where the goal is to infer the true continuous valuesci in anonline fashion as general feature-based estimatesci becomeavailable. We choose a simple model forc as a function ofthe k locale-specific featuresx = (x1, . . . , xk) by modeling

1The original scope learning work [4] was developed in the context ofclassification using discrete features, generative descriptions of those features,and in batch.

the distribution forc givenx as a Gaussian with mean a linearfunction of x and with a variance ofσ2

l :

E(c|β, x) = βT x (1)

We assume that the estimates from the general feature-basedpredictors have Gaussian noise and thus are distributed:

c ∼ Normal(c, σ2g)

We take theσg and σl to be hyper-parameters lying outsidethe locale-specific plate.

Fig. 2. Graphical depiction of the scoped learning model. Hyper-parameters,including priors on the locale-specific parametersβ and noise variances, lieoutside the plate indexed byL and are not depicted.

2) Inference: We develop the inference for the model inan online fashion. Given a new data pointci estimating thetrue variableci, our goal is to compute new estimates2 ofthe variablescj , assuming we have already seen dataD ={{x}1...n, {c}1...i−1}. We can compute this by integrating overthe uncertain parametersβ which describe the relationshipbetween the true variable and the local features.

p(cj |ci, xi,D) =

∫dβ p(cj |β, ci, xi)p(β|ci, xi,D)

We can compute the required distribution overβ as:

p(β|ci, xi,D) ∝ p(β|D)

∫dci p(ci|ci)p(ci|β, xi)

In our linear-Gaussian model, this can be understood asrevising the posterior distribution fromp(β|D) in light of aGaussian likelihood that takes into account noise from bothgeneral and locale-specific features.

Our computation of the posterior distributionp(β|ci, xi,D)is as follows. We first initialize our distribution to the prior dis-tributionp(β). Then, for every training examplei, we multiplyour distribution byp(ci|β, xi). Since the prior distribution andp(ci|β, xi) are normal, the posterior distribution is also normal.We use the notationβ to represent the mean of the posteriordistribution andVβ to represent the variance. Thus, computingp(β|ci, xi,D) is performing a self-supervised learning using aBayesian linear regression model with noise varianceσ2

l +σ2g .

We use our current estimate of the posterior distributionwhen we want to predict a future outcomecj . We are interested

2We assume that our prior onβ is a priori independent of the featuresx

so that inference will remain the same even in the case where thefeaturesbecome available in some sequence.

in predictions in two cases: first, when we have no generalfeature-based estimatecj for a particular cj , and second,when such an estimate is available. In the first case, thepredictive distributionp(c) has meancp = xT β and varianceσ2

p = σ2l + xT Vβx [7]. When we also have an estimatecj ,

inference combines these two estimates:

p(cj) = Normal(σ′2p (

cp

σ2p

+cj

σ2g

), σ′2p )

whereσ′2

p =1

1σ2

p

+ 1σ2

g

.

We note that it is possible to compute the posterior distri-bution in batch, but we prefer to maintain an estimate of theposterior distribution as we receive general feature-based costestimates so that we may immediately apply our algorithm tonew data.

B. Advantages of the Bayesian Learning Approach

Using the online Bayesian scope learning model provides anumber of important benefits.

1) Confidence Rated Prediction:The variance estimateprovided by our algorithm for the probability of eachc can beused as a metric of confidence in the prediction. If a situationarises in which we must choose which one of several predictedoutcomes to trust, we could simply use the one with thesmallest variance.

2) Learning of the Hyper-Prior and Feature Selection:Ouralgorithm depends on a number of hyper-parameter terms thatmay be chosen based on data from multiple locales. We discussways to choose the noise variance termsσl andσg in sectionII-A and the prior distribution on parametersβ in sectionIII-A .It is important to note that the prior onβ may also be chosenas hyper-parameters on multiple locales by adapting Tipping’ssparse hyper-parameter re-estimation procedure3 to our setting[8]. In this way, we can both automate feature selection andbias our algorithm to prefer certain features for new locales.

3) Reversible Learning:A problem that often arises inonline learning is the handling of multiple estimates of aparticular quantity. For instance, in our canonical example,our general feature-based estimatesci may improve as weget closer and denser laser readings of the terrain. It is notappropriate to treat these as independent training examples:while they may differ in their variance, they are generallyhighly correlated. Neither is it useful to simply take the firstestimate available: often this is a poor substitute for all thedata. In our model, since we maintain an exact posteriordistribution that lies inside the exponential family, we mayeffectively removethe effects of training on a data point bydividing out the likelihood term we had used to include it in theposterior [7]. In this way, we always have an estimate of theposterior distribution ofβ using the current best estimateci.Minka has developed an alternate use of this “removal trick”for approximate inference [9].

3The procedure finds the ML-II estimate for the precision (inverse variance)of each weightβi.

III. A PPLICATION TO GROUND ROBOTICS

A. Context

A natural application of our algorithm is to the improvementof autonomous navigation capabilities for unmanned groundvehicles. Our algorithm lends itself well to addressing manyof the issues that arise due to the diversity in the environmentsand data that robotic systems must encounter. In many roboticsystems, variables most relevant to an unmanned ground vehi-cle, such as terrain traversal cost, are computed directly by thevehicle’s on-board perception system through the processingof high density ladar data gathered by on-board sensors.Techniques to process such data are often generalizable tomany environments so that the vehicle’s perception systemdoes not require much adjustment when dealing with newterrain, yet the limited range for this type of data source isa major shortcoming. While data sources such as overheadimagery are widely available with high accuracy, developingfixed techniques to process such locale-specific data sourcesis difficult because even though they may be extremely usefulin specific areas, they do not generalize well to new domainsdue to variations in terrain, lighting conditions, weather, andeven time of data gathering.

We demonstrate how our algorithm can be applied to thedomain of mobile robotics in order to derive the most benefitfrom the availability of both general and locale-specific datasources without any human involvement. The performance ofour algorithm is evaluated through actual field testing in acomplex off-road environment on-board a rugged, all-terrainunmanned ground vehicle. Our results show how combiningthe adaptive performance of our algorithm with the inherentmobility of such a vehicle leads to more efficient navigationofcomplex environments. Additionally, we show how our algo-rithm can be used to detect misalignments between overheaddata and the vehicle’s estimated position, a common problemin many such robotic systems.

For the results in section IV, we chose the hyper-parameterfor noise varianceσ2

l with ML-II and chose an isotropicGaussian with high variance for the prior onβ [7].

(a) (b)

Fig. 3. Robot used for all field tests in its typical operatingenvironment(a) and an illustration of how the online learning algorithmruns on-boardthe robot (b). Algorithm learns mapping from local-specific overhead datafeatures to locally computed terrain traversal costs (computed from generalfeatures) to make prediction elsewhere in the environment.

B. Terrain Traversal Cost Prediction

Our robot performs local sensing using ladar sensors, as-signs traversal costs to the environment from features com-

puted by interpreting the position, density, and point cloud dis-tributions of sensed obstacles (these features generalizeacrossmost domains and therefore serve as ourgeneral features asdefined in SectionII-A ). The robot re-plans in real-time byfinding minimum cost paths through the environment usingthe D* algorithm (see Figure3) [10]. We demonstrate how ouralgorithm can learn to predict terrain traversal costs computedby the on-board perception system of our unmanned groundvehicle from overhead data. We also compare the predictiveperformance of our algorithm to that of a hand-trained classi-fier using superior data sources. We chose to predict traversalcost rather than intermediate results such as slope, density, orpresence of vegetation because traversal cost is the metricthatmost closely governs a vehicle’s navigation strategy throughan environment. Our robot’s perception system is proficientateffectively assessing terrain traversal costs, so it is desirableto be able to mimic its predictive abilities. We will thereforeuse estimates from the robot’s perception system to evaluatethe accuracy of traversal cost predictions.

The characteristics of an environment change with varyingconditions. However, even outdated data can be useful sincemost distinct areas in an environment will maintain uniformityin their characteristics despite these variations. By relaxingrestrictions on the recency of overhead data, our algorithmfurther increases its impact on improving robot navigation.Overhead data is relatively inexpensive and available at variousresolutions for the entire world, so as the quality of globalsurveying improves, the applications of such research willgreatly expand.

C. Features

A set of feature maps for the vehicle’s environment wasgenerated from each overhead data source for use as inputs tothe algorithm (these are ourlocale-specificfeatures as definedin SectionII-A ). In our implementation, HSV (hue, saturation,value) features were used to represent color imagery data whilethe pixel intensity of the black and white imagery data wasused as a single feature. Raw RGB (red, green, blue) colordata was inadequate for our approach due to its sensitivity toillumination variations.

A feature containing the maximum response to a set ofGabor filters of various orientations centered at each pixelwas also generated to capture texture in each type of imagery.Additional features for the black and white imagery data weregenerated by computing the means and standard deviationsin intensity within windows of various size around each pixel.Additional elevation-based features (similar to those describedin [3]) were computed when such data was present. Allfeatures were rescaled to the[−1, 1] range and a constantfeatures was also included.

Finally, clustering of all previously computed features wasperformed that allowed the algorithm to identify patterns in thefeature input space that are relevant to the output regression.The Gaussian Mixture Model algorithm was chosen to clusterthe input data because of its ability to generate membershipfeatures by assigning each data point a fractional degree of

membership in each output cluster (see Figure4). Six clusterswere used in our implementation.

Similar techniques may be used to generate features fromany combination of data sources gathered through a variety ofmethods. We point out that our approach is quite robust to thenumber of clusters and the removal of features.

(a) (b)

Fig. 4. Sample clustering results from using the Gaussian Mixture Modelalgorithm on generated features. Overhead color imagery data (a) used togenerate features and resulting clustering into six clusters (b). Membershipfeatures were generated by computing the fractional degree of membershipof each pixel in each cluster.

D. Training and Prediction

Traversal cost is difficult to quantify, so choosing appro-priate values often requires careful engineering. In ordertoproduce desired behavior when used with a path planningalgorithm such as D*, traversal costs for undesirable areassuch as heavy vegetation must be higher than traversal costsfor ideal areas such as roads (our robot works with traversalcosts in the range of16 to 65535). For example, the robot’son-board perception system assigns traversal costs of16 (theminimum) to roads while grass is assigned a traversal cost of48, implying the robot would be willing to take a detour ofthree times the distance in order to stay on a road as opposedto driving over grass. Meanwhile, dense vegetation is oftenassigned traversal costs of over10000 in order to encouragethe robot to traverse elsewhere except under extreme necessity.Because of these traversal distance ratios, errors in traversalcost estimates in low-cost areas are more detrimental thansimilar errors in high-cost areas. An error of100 to an areaof extremely high traversal cost would have negligible effect,while the same error at an area of desirable terrain wouldradically change the behavior of the robot.

In order to work with a linear model, we deal with traversalcosts within our algorithm on a logarithmic scale, convertingfrom the normal traversal cost space for the purposes of train-ing and prediction. The Gaussian error assumption embeddedin our probabilistic model is a much better approximationwhen we measure error on this scale. Unlike in the regulartraversal cost space, small errors in the log space lead to smallerrors in the traversal distance ratios.

Training examples are constructed from a vector of overheadimagery feature values (xi) and the average of all traversal costestimates that have been calculated within the correspondingarea (ci). As with many robotic systems, the performance ofour robot’s on-board perception system quickly degrades asthe

distance from the robot increases (due to the lowered accuracyand density of sensor data), so the quality of a trainingexample is measured by its proximity to the robot. Rather thanstruggling to decide at which point to utilize an example fortraining, the reversible learning capabilities of our algorithmallow us to maintain an optimal level of predictive abilities byensuring that only the highest quality data available impact itsstate. As the robot approaches locations that had previouslybeen used for training, obsolete examples areunlearned infavor of higher quality training examples available for thoseareas. An example of this training process can be seen inFigure5. Estimates greater than12 meters from the robot areignored since such estimates are unreliable and would onlycorrupt the quality of training in cases where they cannot bereplaced with better estimates.

As the algorithm acquires more training data, its predictiveperformance improves, allowing it to revise previously madetraversal cost estimates. The algorithm specifies a degreeof confidence for each prediction based on the similarityof the example to past training data (as indicated by thevariance estimate), so predictions in which the algorithm lacksconfidence can be ignored in favor of an alternative source ofpredictions or a default value (see Figure5b).

(a) (b)

(c) (d)

Fig. 5. Training progress of online learning algorithm using overhead colorimagery data for traversal of environment shown in (a) is shownin (b) -(d). Dimensions of shown areas are150 m × 150 m. Accumulated groundtruth traversal costs computed by robot’s on-board perception system andvehicle path (shown in red) are overlaid on estimated traversal costs generatedby the algorithm. Lower costs appear as darker colors and predictions thatthe algorithm lacks confidence in (due to insufficient representative trainingexamples) are shown in blue.

E. Applications of Trained Algorithm

This algorithm can be used in a variety of ways to aid inunmanned ground vehicle navigation, both in real-time on-board a robot and offline once it has been trained. In the caseof online terrain traversal cost prediction, the algorithmcanbe used to periodically update traversal cost estimates withina region around the robot so that a real-time path planningalgorithm such as D* can revise the vehicle’s global path toaccount for the changes. As shown in the following section,the use of this algorithm to extend the robot’s field of viewresults in significantly shorter and more intelligent paths.

Since the predictive state of the algorithm can be capturedat any time by the vectorβ at that moment, one can maketraversal cost predictions for a large area without ever havingto traverse or acquire training examples from that area before-hand. Instead, as long as identical features are computed forthe two areas, one can simply drive through a representativearea for a short period of time in order to train the algorithmtomake predictions in a much larger area. The following sectionwill also show that a priori traversal cost maps produced bythis technique are more accurate than even those producedfrom hand-trained techniques that utilize superior data sources.

Even slight mis-registration between overhead data and theestimated position of the robot can significantly hinder theperformance of algorithms such as ours that are sensitive tosuch errors. An advantage of using an online Bayesian linearregression model is the ability to detect such misalignments.

When predicting a new traversal costcj , the model creates apredictive distributionp(c) with a meanµp and a varianceσ2

p.Evaluating the predictive distribution at the traversal cost ci ofa training example gives the probability of having seen thattra-versal cost given its corresponding feature vector. We can usethe probability of having seen all of our data,p(c1, . . . , cn), todetect map misalignments between overhead data sources andestimated vehicle positions by searching through a space ofpotential alignments for the one that maximizes the probabilityof the data.

Sincep(c1, . . . cn), can be computed via the chain rule asthe product of the predictive distributions evaluated at everyci used for training, an estimate of the log data probability isthe cumulative sum of− log σp −

(y−µp)2

σ2p

for every predic-tive distribution. After all examples have been received, theaverage log data probability over all training examples canbe used to compare the quality of an alignment against otheralignments. As shown in the following section, correcting suchmisalignment produces traversal cost maps with better definedobstacles that more accurately reflect the true environment.

IV. RESULTS

A. Field Test Results

The algorithm was tested in real-time on-board our un-manned ground vehicle to measure its impact on naviga-tion performance. The test environments contained a largevariety of vegetation, various-sized dirt roads (often lead-ing through narrow passages in dense vegetation), hills, and

ditches. The vehicle traversed a set of courses defined byseries of waypoints first by using only its on-board perceptionsystem for navigation and also with the help of our onlinelearning algorithm with40 cm resolution overhead imageryand elevation data to supplement the on-board perceptionsystem with traversal cost updates computed within a75 mradius once every2 seconds. The algorithm was initialized foreach course with no prior training (see Figures6 and 7 forsample results). As shown in TableI, our algorithm allowedthe vehicle to complete the courses in26.94% less timewhile traversing7.38% less distance. We found that with ouralgorithm, the vehicle chose to drive further distance on morepreferable terrain in order to avoid difficult or dense areas.Most importantly, while we were forced to manually interveneduring the tests with only the perception system in orderto correct the vehicle’s heading when it became trapped inheavy vegetation and could not escape on its own, no manualinterventions were necessary when using our algorithm.

TABLE I

STATISTICS FORCOURSETRAVERSALS WITH AND WITHOUT THE

ONLINE LEARNING ALGORITHM

Without Algorithm With AlgorithmTotal Traversal Time (sec) 1369.86 1000.82

Total Distance Traveled (m) 1815.71 1681.73Average Speed (m/s) 1.33 1.68

Number of Interventions 1 0

Fig. 6. Comparison of paths executed by our unmanned ground vehicle forshown course when using only on-board perception (in solid red) and withour online learning algorithm used in real-time on-board therobot (in dashedblue). Notice how the online learning path quickly learns toavoid the difficultarea near the cul-de-sac and instead chooses to follow the road to the goal.

B. Field Test Data Post-Processing Results

The algorithm was also used to produce a priori traversalcost maps for a multi-kilometer course over a large area ofcomplex terrain with heavy vegetation and elevation obstacles.The algorithm was trained for about7 minutes using overheadimagery data by driving the vehicle by remote control through

(a) (b)

(c) (d)

Fig. 7. Comparison of paths executed for shown situations when usingonly on-board perception (in solid red) and with our algorithm (in dashedblue) are shown in (a) and (c). Predictions of terrain traversal costs for theenvironment by our algorithm at the time the vehicle chose to avoid the largeobstacles in front of it are shown in (b) and (d). Traversal costs are color-scaled for improved visibility. Blue and red correspond to lowest and highesttraversal cost areas, respectively, with roads appearing in black. In (a) theonline learning path chose to travel slightly further on road in order to avoidthe difficult passage to the left and in (b) our algorithm helped the vehicleavoid the cul-de-sac by executing a path that is42.89% shorter in72.62%

less time.

an entirely separate area that was similar to the main course.The trained algorithm was then used off-line to generate atraversal cost map and plan an initial path through the muchlarger course. Closeups of generated traversal cost maps andresulting planned paths can be seen in Figure8. For compar-ison, we also included the resulting path from a traversal costmap generated by a supervised learning algorithm with human-picked examples from the actual course and features generatedfrom both overhead imagery and high-density elevation data(see TableII for a description of data sources) [3].

TABLE II

TYPES OFOVERHEAD DATA USED BY ONLINE LEARNING (OLL) AND

HAND-TRAINED ALGORITHMS USED TO PRODUCEPRIOR COST MAPS

Algorithm Data Used Resolution

OLL (color) Color imagery 0.35 m

OLL (B & W) Terraserver B & W imagery 1.0 m

Human-Supervised Color imagery 0.35 m

Elevation < 0.2 m

We evaluated the performance of our online learning algo-rithm against the human-trained technique by accumulatingall the traversal costs generated by the vehicle’s on-boardperception system during a traversal of the course shown inFigure8 and comparing those costs to the estimates from eachof the generated prior cost maps. The average absolute errorin traversal cost (on a log scale as described earlier) for each

(a)

(b) (c) (d)

Fig. 8. Circular course with the GPS waypoints for which a priori paths were planned is shown in (a). Shown area is2000 m × 750 m. Our algorithm wastrained during a short traversal of a similar but entirely separate course. Paths generated by the human-trained algorithm (solid red), our algorithm using colorimagery data (dashed cyan), and our algorithm using black andwhite imagery data (dotted blue) are shown. Traversal cost maps produced by our algorithmfor the closeup area in (b) using overhead color imagery and black and white imagery are shown in (c) and (d) respectively. See TableII for description ofdata sources. Traversal costs are color-scaled for improvedvisibility where blue and red correspond to lowest and highest traversal cost areas respectively.

method is shown in Figure9 as a function of training time. Ouralgorithm is shown to outperform the human-trained algorithmusing only imagery data after only a short period of training.However, it should be pointed out that maintaining a tightcorrespondence from traversal costs assigned by the human-trained algorithm to those assigned by the perception systemwas difficult to strictly enforce. This highlights another ad-vantage of the online learning approach over a human-trainedapproach: by relieving the need for manual manipulations oftraversal cost values, the entire system is more adaptable tochanges in both the environment and the perception system.

During post-analysis of this test, we discovered that theoverhead imagery data and the estimated position of thevehicle were in fact misaligned by about1.5 meters. Whilethis result shows that our algorithm is robust to such mapmisalignment, this paper also demonstrates how our algorithmcan be used to detect such errors in alignment in order toachieve optimal performance.

C. Offline Map Alignment

We applied our map alignment technique to a manuallymisaligned log of perception data and overhead imagery

Fig. 9. Average absolute error between log scale traversal costs computedby unmanned ground vehicle’s on-board perception system over the courseof a multi-kilometer traversal of terrain and traversal cost estimates computedusing three techniques: human-trained supervised learningalgorithm usinghigh resolution imagery and elevation data (solid red) and our algorithm usingonly color imagery (dashed cyan) and black and white imagery (dotted blue)as a function of training time by driving in a similar environment. See TableIIfor a description of data sources.

features. A brute force search across all potential map align-ments in 0.35m increments in the four cardinal directionsdetected a misalignment of3.85 m west and4.9 m north.Computed probabilities of observed perception data and thecorresponding improvement in traversal cost estimates canbeseen in Figure10. As expected, correcting the misalignmentimproved the definition of obstacles in the traversal cost mapsand resulted in a stronger correspondence with the actualenvironment, correctly showing that the traveled path is clearlyon the road.

(a)

(b)

(c)

(d)

Fig. 10. Example of how misalignment between overhead data sources andestimated vehicle position can be detected using our algorithm. Computedlog probability of the perception system sensor data encountered over a12.6 m × 12.6 m search space of alignment shifts is shown in (a). Samplecost prediction for area shown in (b) before alignment correction and aftercorrecting detected misalignment of3.85 m west and4.9 m north) appear in(c) and (d) respectively (best alignment is assumed to be that which producesthe highest probability of seen perception data). Darker colors in the imagescorrespond to lower traversal costs.

V. CONCLUSION

We have proposed a self-supervised online learning algo-rithm to learn and infer between different types of data sources

that vary in density, reliability, and scope. By applying thescoped learning model, we were able to generalize from onetype of data source to be able to work with another which maybe difficult to generalize to new environments. As a result, wewere able to extend the scope of such features to many possibledomains.

We showed how the algorithm can be used to improve thenavigation capabilities of unmanned ground vehicles by learn-ing in real-time to interpret overhead imagery data to predictterrain traversal costs generated from an on-board perceptionsystem. We demonstrated this approach through actual fieldtests on-board a large robot in complex natural environments.Both online and offline results were given to demonstrateseveral applications of the algorithm. While performance couldbe hampered because of limitations in the available featuresor availability of representative training examples, the use ofthis algorithm was shown to improve the quality of robotnavigation performance. Allowing robots to adapt to andimprove their performance in diverse environments withouthuman involvement through such techniques greatly expandseffectiveness and potential applications of robotic systems.

ACKNOWLEDGMENT

This work was sponsored by the Defense Advanced Re-search Projects Agency (DARPA) under contract ”UnmannedGround Combat Vehicle - PerceptOR Integration” (contractnumber MDA972-01-9-0005). The views and conclusions con-tained in this document are those of the authors and shouldnot be interpreted as representing the official policies, eitherexpressed or implied, of the U.S. Government.

We would also like to thank all the members of the UPIproject team, without whom this work would not be possible.

REFERENCES

[1] D. Lieb, A. Lookingbill, and S. Thrun, “Adaptive road following usingself-supervised learning and reverse optical flow,” inProceedings ofRobotics: Science and Systems, June 2005.

[2] A. Stentz, A. Kelly, P. Rander, H. Herman, O. Amidi, R. Mandelbaum,G. Salgian, and J. Pedersen, “Real-time, multi-perspective perceptionfor unmanned ground vehicles,” inAUVSI, 2003.

[3] D. Silver, B. Sofman, J. A. Bagnell, N. Vandapel, and A. Stentz, “Ex-perimental analysis of overhead data processing to support long rangenavigation,” in Proceedings of the IEEE/RSJ International Conferenceon Intelligent Robots and Systems, 2006.

[4] D. Blei, J. Bagnell, and A. McCallum, “Learning with scope, withapplication to information extraction and classification,”in Proceedingsof the 2002 Conference on Uncertainty in Artificial Intelligence, June2002.

[5] M. I. Jordan and Y. Weiss, Eds.,Graphical models: Probabilisticinference. MIT Press, 2002.

[6] S. M. Kakade and A. Y. Ng, “Online bounds for bayesian algorithms,”in Advances in Neural Information Processing Systems 17, L. K. Saul,Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp.641–648.

[7] A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin,Bayesian DataAnalysis. Chapman and Hall/CRC, 2004.

[8] M. E. Tipping, “Bayesian inference: An introduction to principles andpractice in machine learning,” inProceedings of the Advanced Lectureson Machine Learning, 2003.

[9] T. Minka, “A family of algorithms for approximate bayesian inference,”2001.

[10] A. Stentz, “Optimal and efficient path planning for partially-knownenvironments,” inProceedings of the 1994 IEEE Conference on Roboticsand Automation, May 1994.