problem space analysis for plan library generation and algorithm selection in real-time systems...

Problem Space Analysis for Plan Library Generation and Algorithm Selection in

Real-time Systems

Robert H. Holder, III

Dissertation Proposal Defense

August 26, 2009Committee:Dr. Tim Finin, ChairDr. Marie desJardinsDr. Tim OatesDr. R. Scott Cost

2

``...it was not obvious what you had to do next so you had to think two, six, ten moves ahead. Scenario planning is about having twelve plans, so if one does not work you go to the next. The fun is figuring out a backup for whatever could go wrong.'‘

-Seth Godin, entrepreneur, describing his strategy for success in games and entrepreneurship. From The Red Rubber Ball At Work by Kevin Carroll

-Scott Adams, “Dilbert” comic, June 1, 2008

3

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Domain-Based Hints Sampling Interpolation

Research Directions Summary

4

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Domain-Based Hints Sampling Interpolation

Research Directions Summary

5

Motivation

Real-time Planning Problems Shipboard Computing Resource Allocation Mobile Sensor Scheduling Unmanned Vehicle Routing Wireless Sensor Network (WSN) Reconfiguration

Motivation For planning problems in a real-time environment,

6

Plan Library: Problem space analysis can inform the efficient creation of a plan library.

Problem Space Analysis: A sampling of problem instances and their solutions can lend insight into the underlying structure of the domain.

A plan library is a means of rapidly adapting to a new environment.

Hypothesis:

Motivation

7Additional benefits of problem space analysis

Contributions

Framework to model and reason about the topological structure of the solutions of related plans

Techniques to predict and leverage the effect of problem instance characteristics and attributes on the topological structure of the solution space

Novel algorithms that exploit solution space structure to generate plan libraries, select & configure algorithms, and decompose large problems

Evaluation of these algorithms by comparison to competing techniques for a set of sample problems

8

9

Dynamic Vehicle Routing Problem

Three vehicles to visit twelve destinations

10

Dynamic Vehicle Routing Problem

Static solution for known destinations

11

Dynamic Vehicle Routing Problem

Consideration for potential additional destination

12

Dynamic Vehicle Routing Problem

X

X

X

Contingency solution for additional destination

13

Dynamic Vehicle Routing Problem

Implementation of contingency solution

14

Dynamic Vehicle Routing Problem

How to plan contingencies for arbitrary possibilities?

15

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Domain-Based Hints Sampling Interpolation

Research Directions Summary

16

Related Work –Technique Comparison

Plan Library generation case-based reasoning (created after runtime) reactive planning (subset of input, localized) conditional & contingency planning (localized) decision-theoretic (relies on probabilities)

Real-time Planning plan repair (runtime, works for small changes) anytime & contract algorithms (runtime)

17

Related Work – Case-Based Reasoning

Smyth & McKenna (2001)

18

Related Work – Domain Space Analysis

State Space Analysis Bulka (2009) – Backbone planning Kondaris (2008) – Automated skill learning Hoffman (2001) – State space “benches” and “exits”

Solution Space Analysis Miner (2009) – Solution gradient lines Rosen, et. al. (2005) – Medical plan comparison Gopal & Starkschall (2002) – Medical plan topology

Plan-Space Planning Trinquart (2003) – Plan space reachability Hoffman & Nebel (2001) – Uses plan space structure to estimate

distance to goal state

Related Work – Gopal & Starkschall (2002)

19

20

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Domain-Based Hints Sampling Interpolation

Research Directions Summary

Approach Overview

21

22

Traveling Salesperson Problem

Four fixed cities Central starting point Unknown fifth city

1

3

4

2

23

Approach: Problem-Solution Map

1

3

4

2

Each point represents a potential location of the fifth city. The color of the point represents the optimal solution for the resulting 5-city TSP.

24

Approach:Problem-Solution Map

1

3

4

2

0-1-4-2-3-5

0-1-3-2-5-4

0-1-3-5-2-4

0-1-5-4-2-3

0-1-5-3-2-4

0-3-2-4-1-5

0-1-3-2-4-5

0-5-1-3-2-4

Each point represents a potential location of the fifth city. The color of the point represents the optimal solution for the resulting 5-city TSP.

25

Approach

Problem-Solution Map (PS Map) contiguous regions – only need to store one

solution per region complexity – higher interaction of regions

indicates a more complex space

26

Approach: Solution-Problem-Utility Map

Optimal library requires 8/120 = 6.7% of possible solutions. Is this too many?

27

Approach: Solution-Problem-Utility Map

Tolerating solution degradation and extending the scope of neighboring solutions can reduce library size

Approach: Solution-Problem- Utility Map

28

Discrete and continuous SPU Maps showing global competency of one solution

29

Approach

Solution Problem Utility Map (SPU Map) reduce solution regions (and thus library size) by

tolerating utility degradation gradient of solution degradation

30

0-1-4-2-3-5

0-1-3-2-5-4

0-1-3-5-2-4

0-1-5-4-2-3

0-1-5-3-2-4

0-3-2-4-1-5

0-1-3-2-4-5

0-5-1-3-2-4

Approach:Solution-Similarity Map

high similarity low similarity

31

Approach Solution Similarity Map (SS Map)

reduce library size by relying on run-time adaptation creating parameterized solutions

suggests regions where regular world assumption does not hold

32

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Domain-Based Hints Sampling Interpolation

Research Directions Summary

33

Algorithm Selection & Configuration

Initial solution – use fast, aggressive heuristics

34

Algorithm Selection & Configuration

Initial solution – use more precise heuristics, emphasize exploration over exploitation, use less aggressive hill-climbingAdaptation – if solutions are similar, use hill-climbing, else use genetic algorithm

less contiguous

35

Informed Problem Decomposition To decompose a DVRP or DTSP, a system can fix one of the

unknown city locations. Choosing cities such that the subproblem yields a more contiguous

map will be advantageous for planning.

more contiguous

36

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Sampling Interpolation Domain-Based Hints

Research Directions Summary

5-city TSP (1 unknown city) small problem with two degrees of freedom 12k problem instances naive solver fast runtime

5-city TSP (2 unknown cities) larger problem with four degrees of freedom 311k problem instances naive solver ~20 minutes runtime

Map Generation

37

38

Map Generation

Complete map generation is not feasible (would make algorithm selection irrelevant) how can we approximate the map efficiently?

Map approximation sampling interpolation domain-based hints

example: fixed city locations lie on regional borders

39

Map Generation

Initial view of solution space topology

40

Map Generation

Ideal view of solution space topology

41

Map Generation

Preliminary experiment with random sampling and nearest neighbor classification

Map Generation

Ideal PS Map Approximated PS Map

42

Preliminary experiment with random sampling and nearest neighbor classification

Map Generation

43

Preliminary experiment with random sampling and nearest neighbor classification demonstrates

75% accuracy from a 0.5% sample rate.

44

Sampling & Classification

Sampling uniform/random/NOLHS/Rapidly expanding Random Trees (RRT) strategic sampling (active learning) schemes biased by domain hints

Classification Nearest neighbor

k nearest neighbors vs. radius of nearest neighbors weighting neighbors by distance

Support Vector Machine (linear, non-linear) Bayesian Network Neural Network

45

Domain-Based Hints

All the fixed points lie on regional boundaries

46

Domain-Based Hints

All the fixed points lie on regional boundaries

47

SPU and SS Map Generation

SPU Map calculate utility of solution for

sample of problem instances regression to find limits of

solution competence does optimal region shape

inform tolerated region shape? SS Map

for each solution, find similarity to each neighbor

look for similarities to non-neighbors?

algorithm selection - can we characterize how quickly solution is changing?

48

Regression

Regression depends on a function form, i.e. a “kernel” Can we determine the appropriate kernel based on

problem characteristics? Piecemeal regression may allow local customization of

regression Support Vector Regression Machines (Drucker, 1996)

Approach Overview

49

50

Outline

Motivation Related Work Approach

Problem-Solution (PS) Map Solution-Problem-Utility (SPU) Map Solution Similarity (SS) Map

Map Utilization Plan Library Generation Algorithm Selection and Configuration Informed Problem Decomposition

Map Generation Domain-Based Hints Sampling Interpolation

Research Directions Summary

Domains

Dynamic Traveling Salesman Problem Dynamic Vehicle Routing Problem Wireless Sensor Network (Re)Configuration

51

Research Plan Goals

Investigate inferring a problem space analysis and solution topology from a sample set of problem instance and solution pairs

Apply problem space analysis and other domain characteristics to the creation of an efficient plan library

Apply problem space analysis to the selection of algorithms suited for a particular region of the problem space

Apply problem space analysis to strategic decomposition of large planning problems

52

Research Plan

Fall 2009 Generate ideal set of PS and SPU Maps for various TSP, VRP, and WSN problems Test PS Map generation using various sampling and classification schemes Test SPU Map generation using various sampling and regression schemes

Spring 2010 Confirm usefulness of domain-based hints Apply bias to sampling and classification schemes

Summer 2010 Library generation algorithm Evaluation

Fall 2010 Algorithm selection technique Problem decomposition algorithm Evaluation

Spring 2011 Final experiments and evaluation Writing and defense

53

Evaluation

System performance Comparison to other DTSP, DVRP algorithms Comparison to known optimal solutions (TSP/VRP) Comparison to baseline communication throughput and network life

metrics (WSN) Characterize performance as function of library size Offline computation time vs. online performance

Map approximation Raw accuracy Function of proximity to optimal solution region Expected utility degradation due to inaccuracies Characterize performance as function of sampling and interpolation

schemes54

Contributions

Framework to model and reason about the topological structure of the solutions of related plans

Techniques to predict and leverage the effect of problem instance characteristics and attributes on the topological structure of the solution space

Novel algorithms that exploit solution space structure to generate plan libraries, select & configure algorithms, and decompose large problems

Evaluation of these algorithms on a sample problems by comparison to competing techniques

55

56

Summary

Use of problem space analysis to facilitate real-time planning plan library creation algorithm selection problem decomposition

Use of sampling and interpolation techniques to create problem space maps traditional techniques domain-based hints informed kernel selection

References

Bulka (2009) “Efficient Planning Using Plan Libraries to Capture the Structure of the State Space”

Gopal & Starkschall (2002) “Plan space: representation of treatment plans in multidimensional space”

Hoffman (2001) “Local Search Topology in Planning Benchmarks: An Empirical Analysis”

Hoffman & Nebel (2001) “The FF Planning System: Fast Plan Generation Through Heuristic Search”

Onder & Pollack (1996) “Contingency Selection in Plan Generation” Kondaris (2008) “Autonomous Robot Skill Acquisition” Miner (2009) “Rule Abstraction: Understanding Emergent Behavior in Swarm

Systems” Smyth & McKenny (2001) “Competence Models and the Maintenance Problem” Rosen, et. al. (2005) “Interactively exploring optimized treatment plans” Trinquart (2003) “Analyzing Reachability within Plan Space”

57

58

Thank you

Questions…

Suggestions…

Comments…

Backup

59

Related Work –Contingency Planning

60

Onder & Pollock (1996)

Related Work – Miner (2009)

61