Robust Combination of Local Controllers

Carlos Guestrin

Dirk OrmoneitStanford University

Planning

Planning is central in real-world systems;

However, planning is hard: Motion planning is PSPACE-

hard [Reif 79]; State and Action spaces are

often continuous; Uncertainty is ubiquitous:

Imprecise actuators; Noisy sensors.

Global versus Local Controllers

Designing a global controller is hard, but… Many real-world domains allow us to design

good local controllers with no global guarantees:

How can we combine local controllers to obtain a global solution ?

Combining Local Controllers

Randomized algorithm: Nonparametric combination of local

controllers; Generalizes probabilistic roadmaps: [Hsu et

al.99] stochastic domains; Discounted MDPs;

Theoretical analysis: Characterizing local goodness of controllers; polynomial number of milestones is sufficient.

Motion Planning Case

Deterministic motion planning: Given some start and goal configurations,

find a collision free path; Stochastic motion planning:

Given some start and goal configurations, find a high probability of success path.

Path

Start Goal

Nonparametric Combination of Local Controllers

i

j

Use simulation to estimate quality of local controllers

Quality: prob. controller reaches neighbor without collisions

Nonparametric Combination of Local Controllers

i

jpi

j

Finding a high success probability path Sample milestones uniformly at random:

X1, …, XN-1 ; Set start as X0 and goal as XN;

Simulation to estimate local connectivity: Estimate pij for j in the K nearest neigbors of i;

Shortest path algorithm to find most

probable path from X0 to XN:

Edge weights become –log pij .

Example: Maximum Success Probability Path

Example: Maximum Success Probability Path

What About Costs ? MDPs find path with lowest expected

cost: Implicit trade-off: cost of hitting obstacles

and reward for goal; In Robotics, a successful path often more

important than a short path: Robotic museum guide; Manufacturing;

Thus, we make the trade-off explicit: What is the lowest cost path with success

probability of at least pmin ?

Restricted Shortest Path Lowest cost path with success prob. at least

pmin: Restricted shortest path problem; NP-hard, however, FPAS algorithms [Hassin 92];

Dynamic programming algorithm: Discretize [pmin,1] into S+1 values;

q(s) = (pmin)s/S, s = 0, …, S;

V(s,xi): minimum cost-to-go starting at xi, reaching

goal with success probability at least q(s).

Examples:Restricted Shortest Paths

Success prob.: 0.99Path length: 1.75

Success prob.: 0.51Path length: 1.08

Examples:Restricted Shortest Paths

Success prob.: 0.99Path length: 1.75

Success prob.: 0.51Path length: 1.08

Theoretical Analysis:Characterizing quality of local controllers

Probabilistic roadmaps (PRMs): [Hsu et al. 99] Deterministic motion planning; Characterize space as (,,)-good; Bound number of milestones;

Extension to stochastic domains: Characterize space and controller as (,,,pp)-

good.XRX

RX – points reachable using controller from X with probability of success pp

Space is (,pp)-good if:Volume(RX) . Volume(free space)

Theorem For any >0, a roadmap with

N=28ln(8/)/+3/+2 milestones, with probability at least 1-, will contain a path between any two milestones in the same connected component and this path will have success probability of at least pp

3/+1.

Complete with probability at least 1-; Number of milestones poly(ln(1/), 1/, 1/, 1/); Final path has success probability of at least pp

3/+1.

In words:

Related Work Macro actions in discrete discounted

MDPs: Hauskrecht et al. 1998, Parr 1998;

Probabilistic Roadmaps (PRMs) for deterministic motion planning: Hsu et al. 1999;

Continuous state, discrete actions discounted MDPs: Rust 1997.

Centralized Control of Two Holonomic Robots

Centralized Control of Two Holonomic Robots

Success prob.: 0.99Total path length: 3.53

Success prob.: 0.13Total path length: 1.53

Success prob.: 0.54Total path length: 2.79

5 dof Robot Arm

Success prob.: 0.95Path length: 10.07

Success prob.: 0.60Path length: 7.81

7 dof Snake

Success prob.: 0.96Path length: 27.0

Success prob.: 0.11Path length: 15.4

Shortest: Most Success Probaility:

Conclusions Algorithm for planning in stochastic domains

with continuous state and action spaces: Nonparametric combination of local controllers;

Motion planning: Theoretical analysis quantifies local quality of

controllers; Proposed alternative objective function; Qualitative and quantitative properties demonstrated;

Also applicable for discounted MDPs: Describe methods for robustly combining local

controllers.

http://robotics.stanford.edu/~guestrin/Research/RobustLocalControl/