a hierarchical approach to model-based reactive planning in large state spaces artificial...
TRANSCRIPT
![Page 1: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/1.jpg)
A Hierarchical Approach to Model-based Reactive Planning
in Large State Spaces
Artificial Intelligence & Space Systems Laboratories
Massachusetts Institute of Technology
Brian C. WilliamsJoint with Seung H. Chung
![Page 2: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/2.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Outline
• Model-based programming• A Simple model-based executive (Livingstone)• The need for model-based reactive planning• The Burton model-based reactive planner
![Page 3: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/3.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Objective: Embedded languages that reason from hardware models.
(Reactive Model-based Programming)
Polar Lander Leading Diagnosis:
• Legs deployed during descent.
• Noise spike on leg sensors latched by software monitors.
• Laser altimeter registers 50ft.
• Begins polling leg monitors to determine touch down.
• Latched noise spike read as touchdown.
• Engine shutdown at ~50ft. Mars Mission Failures, 2000:•Climate Orbiter•Polar Lander
![Page 4: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/4.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Model-based Programs Interact Directly with State
Embedded programs interact withplant sensors and actuators:
• Read sensors
• Set actuators
Model-based programs interact with plant state:
• Read state
• Write state
Embedded Program
SPlant
Obs Cntrl
Model-basedEmbedded Program
SPlant
Problem: Programmer must must map between state and sensors/actuators.
Solution: Model-based executive maps between state and sensors/actuators.
S’Model-based Executive
Obs Cntrl
![Page 5: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/5.jpg)
![Page 6: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/6.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Orbital Insertion Example
EngineA EngineB
Science Camera
Turn camera off and engine on
EngineA EngineB
Science Camera
![Page 7: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/7.jpg)
Programmer specifiesabstract state evolutions
Model
Temporal plannerTemporal planner
Model-based ExecutiveModel-based Executive
Command
goals
Observations Flight System Control
RT Control Layer
State
Thrust Goals
Attitude Point(a)
Engine OffOff
Delta_V(direction=b, magnitude=200)
Power
Model-based ProgramModel-based ProgramEvolves Hidden StateEvolves Hidden State
ClosedClosed
ValveValve
OpenOpen StuckStuckopenopen
StuckStuckclosedclosed
OpenOpen CloseClose
0. 010. 01
0. 010. 01
0.010.01
0.010.01
inflow = outflow = 0
OrbitInsert()::
(do-watching ((EngineA = Firing) OR (EngineB = Firing))
(parallel
(EngineA = Standby)
(EngineB = Standby)
(Camera = Off)
(do-watching (EngineA = Failed)
(when-donext ( (EngineA = Standby) AND (Camera = Off) )
(EngineA = Firing)))
(when-donext ( (EngineA = Failed) AND (EngineB = Standby) AND (Camera = Off) )
(EngineB = Firing))))
Programmer specifies plant model
Model specifies•Mode transitions•Mode behavior
Reactive Model-based Programming Language: Asserts state Queries state Executes conditionally Preempts Iterates Executes concurrently
![Page 8: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/8.jpg)
Model
Temporal plannerTemporal planner
Model-based ExecutiveModel-based Executive
Commands
State Goals
Observations Flight System Control
RT Control Layer
Thrust Goals
Attitude Point(a)
Engine OffOff
Delta_V(direction=b, magnitude=200)
Power
Model-based Executive Model-based Executive Reasons from Plant ModelReasons from Plant Model
State Estimates
State Estimates
Reconfigure & Repair
Estimate & Diagnose
State Goals
s
Observations Commands
Goal: Achieve Thrust
Open fourvalves
Engine Off
![Page 9: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/9.jpg)
Model
Temporal plannerTemporal planner
Model-based ExecutiveModel-based Executive
Command
goals
Observations Flight System Control
RT Control Layer
State
Thrust Goals
Attitude Point(a)
Engine OffOff
Delta_V(direction=b, magnitude=200)
Power
Model-based Executive Model-based Executive Reasons from Plant ModelReasons from Plant Model
State Estimates
Reconfigure & Repair
Estimate & Diagnose
State Goals
s
Goal: Achieve Thrust
Diagnose:Valve fails
stuck closed Switch to
backup
![Page 10: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/10.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Outline
• Model-based programming• A Simple model-based executive (Livingstone)• The need for model-based reactive planning• The Burton model-based reactive planner
![Page 11: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/11.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
A simple model-based executive (Livingstone) commanded NASA’s Deep Space One probe
courtesy NASA JPL
Started: January 1996Launch: October 15th, 1998Remote Agent Experiment: May, 1999
![Page 12: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/12.jpg)
Livingstone [Williams & Nayak, AAAI96]
State estimate
ModeReconfiguration
ModeEstimation
CommandObservations
Model
Flight System Control
RT Control Layer
State goals
s
![Page 13: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/13.jpg)
Thrust
State estimate
ModeSelection
ModeEstimation
CommandObservations
Model
Flight System Control
RT Control Layer
State goals
s
Estimate current likely Modes Reconfigure modes to meet goals
![Page 14: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/14.jpg)
State estimate
ModeSelection
ModeEstimation
CommandObservations
Model
Flight System Control
RT Control Layer
State goals
s
Mode Selection:
Select a least cost set of allowed component modes that entail the current goal, and are consistent
Mode Estimation:
Select a most likely set of component mode transitions that are consistent with the model and observations
arg max Pt(m’)
s.t. M(m’) ^ O(m’) is consistent
arg min Ct(m’)
s.t. M(m’) entails G(m’)
s.t. M(m’) is consistent
![Page 15: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/15.jpg)
ModeSelection
ModeEstimation
CommandObservations
Model
Flight System Control
RT Control Layer
s
OpSat:
arg min f(x)
s.t. C(x) is satisfiable
D(x) is unsatisfiable
State estimate State goals
arg max Pt(m’)
s.t. M(m’) ^ O(m’) is satisfiable
arg min Ct(m’)
s.t. M(m’) entails G(m’)
s.t. M(m’) is satisfiable
![Page 16: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/16.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Outline
• Model-based programming• A simple model-based executive (Livingstone)• The need for model-based reactive planning• The Burton model-based reactive planner
![Page 17: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/17.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
DS 1 Attitude Control System
z facing thrusters x facing thrusters
1553 bus
Com
mands
Data N2H4
He
PDE
SRU
PDU
GDE
PASM
DSEU
PEPE
BC
FlightComputer
FlightComputer
BC
PDE
Livingstone reconfigured modes using one step commands. But How does the flight computer really open a valve?
• Requires turning on device drivers• Requires repairing bus controllers• Sending commands• Powering down devices . . .
![Page 18: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/18.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
RemoteTerminal
RemoteTerminal
Driver
BusControl
ComputerValve
Driver
Valve• Device modes are changed through indirect commanding.
• Communication paths are established by reconfiguring other devices.
• The task of reconfiguring devices in the proper order generalizes state-space planning to handle indirect effects.
• to achieve reactivity the all possible plans for all possible goal states should be pre-compiled (a generalization of universal plans).
• To achieve compactness we decompose these universal plans according to a goal/sub-goal hierarchy.
How do we reconfigure a valve?
![Page 19: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/19.jpg)
Model-based Execution & Reactive Planning
Burton [Williams & Nayak, IJCAI97]
State estimate
ModeSelection
ModeEstimation
CommandObservations
s goalsReactivePlanner
Model
Flight System Control
RT Control Layer
State goals
s
![Page 20: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/20.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Example: Driver Valve Command Sequence
Valve Driver dr Valve vlv
vcmdindcmdin
Commands Driver State Valve StateME: dr = off, vlv = openMS: dr = off, vlv = closed
MRP dcmdin = onME: dr = on, vlv = openMRP dcmdin = closeME: dr = reset failure, vlv = openMRP dcmdin = resetME: dr = on, vlv = openMRP dcmdin = offME: dr = off, vlv = open
Goal: No thrust
![Page 21: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/21.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
To achieve reactivity we eliminate all forms of search.
![Page 22: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/22.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Model-based Reactive Planning
Achieved by:
1. Eliminate Indirect Control
. . . through Compilation
2. Eliminate Search for Goal Ordering
. . . through Reversibility and Serialization
3. Eliminate Search to find Suitable Transitions
. . . by Constructing Hierarchical Polices
![Page 23: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/23.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Model-based Reactive Planning
Achieved by:
1. Eliminate Indirect Control
. . . through Compilation
2. Eliminate Search for Goal Ordering
. . . through Reversibility and Serialization
3. Eliminate Search to find Suitable Transitions
. . . by Constructing Hierarchical Polices
![Page 24: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/24.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
To Handle Indirect Control . . .dcmdout= vcmdin
off
on
failed
resettable
dcmdin = offdcmdin = on
dcmdin = reset
dcmdin = off
dcmdin = dcmdout
closed
open
stuck closed
stuck open
vcmdin = closevcmdin = open
inflow = outflow
vcmdindcmdin
flowin
flowout
![Page 25: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/25.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
. . . Compile Out Constraints
closed
open
stuck closed
stuck open
vcmdin = closevcmdin = open
off
on
failed
resettable
dcmdin = offdcmdin = on
dcmdin = reset
dcmdin = off
dcmdin = dcmdout inflow = outflowinflow = outflow
dcmdout = vcmdin
driver = ondriver = on
dcmdin = closedcmdin = open
vcmdindcmdin
flowin
flowout
![Page 26: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/26.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
. . . Compile Out Constraints
closed
open
stuck closed
stuck open
off
on
failed
resettable
dcmdin = offdcmdin = on
dcmdin = reset
dcmdin = off
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
![Page 27: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/27.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
To Compile Out Constraints• Eliminate intermediate variables.
Transitions are conditioned on mode and control variables
• Generate transitions as prime implicates:
i next(yi = ei)
where i is a conjunction of mode and control variable assignments.
• Prime implicates for transitions enumerated using OpSAT
– 40 seconds on SPARC 20 for 12,000 clause spacecraft model.
![Page 28: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/28.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Model-based Reactive Planning
Achieved by:
1. Eliminate Indirect Effects
. . . through Compilation
2. Eliminate Search for Goal Ordering
. . . through Reversibility and Serialization
3. Eliminate Search to find Suitable Transitions
. . . by Constructing Hierarchical Polices
![Page 29: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/29.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
ValveDriver
command
• Example– Current State: driver = on, valve = closed– Goal State: driver = off, valve = open– Achieving (driver = off) and then (valve = open) clobbers (driver = off)
Why Search is Needed
1) An achieved goal can be clobbered by a subsequent goal.
Achieve Valve goal before Driver goal
![Page 30: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/30.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Note: Component schematics tend not to have loops
RemoteTerminal
RemoteTerminal
BusControl
ComputerValve
Valve
Driver
Driver
Work conjunctive goals upstream from outputs to inputs
– Define: Causal Graph G of compiled transition system S • vertices are state variables.
• edge from vi to vj if vj’s transition is conditioned on vi.
dcmdin
Driver
Valve
– Requirement: The causal graph is acyclic.
![Page 31: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/31.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
• The only variables used to set some variable (y7) is its ancestors,
y7 can be changed without affecting its descendants.
Solution
13
12
9
11
8
10
7
4
6
3
5
2
1
UnaffectedAffected
• Safe to achieve goals in an upstream order.
• Simple check: – Number causal graph depth first – achieve goals in order of increasing depth first number.
![Page 32: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/32.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Latch1
Switch
data
• Example– Latch1 and Latch2 compete for the position of Switch if achieved
concurrently.
Why Search is Needed
2) Two goals can compete for the same variable in their subgoals.
Latch2
1
2
![Page 33: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/33.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
• Sibling goals (7,4) may both need shared ancestors.
13
12
9
11
8
10
7
4
6
3
5
2
1
UnaffectedNot Shared
Shared
13
12
9
11
8
10
7
4
6
3
5
2
1
Unaffected
Not Shared
• But ancestors no longer needed once goal (7) is satisfied.
• Solution: Solve one goal before starting next sibling (Serialization).• Feature: Generates first control action of plan first!
![Page 34: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/34.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Latch1
Switch
data
• Example– Assume Switch can be used once, – Then Latch1 must be latched before Latch2.
Why Search is Needed
3) A state transition of a subgoal variable has irreversible effect.
Latch2
• But irreversible effects aren’t desirable for reactive planners
Don’t allow irreversible actions. . . Except to repair failure modes
1
2
![Page 35: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/35.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution: Mark Allowed Transitions/Assignments
off
on
failed
resettable
dcmdin = offdcmdin = on
dcmdin = reset
dcmdin = off
closed
open
stuck closed
stuck open
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
DriverValve
• Mark all control variable assignments allowed:
123
![Page 36: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/36.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution: Mark Allowed Transitions/Assignments
off
on
failed
resettable
dcmdin = offdcmdin = on
dcmdin = reset
dcmdin = off
closed
open
stuck closed
stuck open
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
DriverValve
• Mark all control variable assignments allowed:
123
• For each mode variable v, in decreasing order of DF number:
• Select each transition of v, whose guard has only allowed assignments.
![Page 37: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/37.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution: Mark Allowed Transitions/Assignments
off
on
failed
resettable
dcmdin = offdcmdin = on
dcmdin = reset
dcmdin = off
closed
open
stuck closed
stuck open
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
DriverValve
• Mark all control variable assignments allowed:
123
• For each mode variable v, in decreasing order of DF number:
• Select each transition of v, whose guard has only allowed assignments.
• Given current assignment v = I for v:
• Mark assignments and transitions in SCC allowed.
• Find strongly connected component of selected transitions that contains I.
![Page 38: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/38.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution: Mark Allowed Transitions/Assignments
off
on
dcmdin = offdcmdin = on
closed
open
stuck closed
stuck open
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
DriverValve
• Mark all control variable assignments allowed:
123
• For each mode variable v, in decreasing order of DF number:
• Select each transition of v, whose guard has only allowed assignments.
• Given current assignment v = I for v:
• Mark assignments and transitions in SCC allowed.
• Find strongly connected component of selected transitions that contains I.
![Page 39: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/39.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution: Mark Allowed Transitions/Assignments
off
on
dcmdin = offdcmdin = on
closed
open
stuck closed
stuck open
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
DriverValve
• Mark all control variable assignments allowed:
123
• For each mode variable v, in decreasing order of DF number:
• Select each transition of v, whose guard has only allowed assignments.
• Given current assignment v = I for v:
• Mark assignments and transitions in SCC allowed.
• Find strongly connected component of selected transitions that contains I.
![Page 40: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/40.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution: Mark Allowed Transitions/Assignments
off
on
dcmdin = offdcmdin = on
closed
open
driver = ondriver = on
dcmdin = closedcmdin = open
dcmdin
DriverValve
• Mark all control variable assignments allowed:
123
• For each mode variable v, in decreasing order of DF number:
• Select each transition of v, whose guard has only allowed assignments.
• Given current assignment v = I for v:
• Mark assignments and transitions in SCC allowed.
• Find strongly connected component of selected transitions that contains I.
![Page 41: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/41.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Model-based Reactive Planning
Achieved by:
1. Eliminate Indirect Effects
. . . through Compilation
2. Eliminate Search for Goal Ordering
. . . through Reversibility and Serialization
3. Eliminate Search to find Suitable Transitions
. . . by Constructing Hierarchical Polices
![Page 42: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/42.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Solution
• Convert automata into hierarchical policies, one per automaton
closed
open
cmd = closecmd = open
fail
Goal
fail
driver = oncmd = open
idle
idledriver = on
cmd = close
Current
Open
Closed
Stuck
Open Closed
driver = ondriver = on
– Policy selects first transition towards achieving each automata goal state, given current state.
– Policy maps goals to subgoals and commands, in proper order– Ensures only reversible transitions are taken,
by only using transitions marked allowed.
![Page 43: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/43.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idledriver = on
cmd = close
Current
Open
Closed
Stuck
Open Closed
Goal
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
cmd = reset cmd = off
Current: Driver = off, Valve = open
12
![Page 44: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/44.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idledriver = oncmd = close
Current
Open
Closed
Stuck
Open Closed
Goal
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
cmd = reset cmd = off
Current: Driver = off, Valve = open
12
![Page 45: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/45.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
Send:cmd = on
fail
Goal
fail
driver = oncmd = open
idle
idledriver = oncmd = close
Current
Open
Closed
Stuck
Open Closed
Goal
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
cmd = reset cmd = off
Current: Driver = off, Valve = open
12
![Page 46: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/46.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
12
Current: Driver = resettable, Valve = open
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idle
Current
Open
Closed
Stuck
Open Closed
Goal
idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
cmd = reset cmd = off
driver = oncmd = close
FailedResettable
cmd = on
![Page 47: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/47.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idle
Current
Open
Closed
Stuck
Open Closed
Goal
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
cmd = reset cmd = off
driver = oncmd = close
Current: Driver = resettable, Valve = open
12
![Page 48: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/48.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idle
Current
Open
Closed
Stuck
Open Closed
Goal
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
Sendcmd = reset
cmd = reset cmd = off
driver = oncmd = close
Current: Driver = resettable, Valve = open
12
![Page 49: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/49.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idledriver = on
cmd = close
Current
Open
Closed
Stuck
Open Closed
Goal
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
Sendcmd = close
cmd = reset cmd = off
Current: Driver = on, Valve = open
12
![Page 50: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/50.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idledriver = on
cmd = close
Current
Open
Closed
Stuck
Open Closed
cmd = reset
Goal
cmd = off
cmd = on idle
idle cmd = off
Current
On
Off
Resettable
On Off
Goal: Driver = off, Valve = closed
Sendcmd = off
Current: Driver = on, Valve = closed
12
![Page 51: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/51.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Plan by passing sub-goals up causal graph
ValveDriver
fail
Goal
fail
driver = oncmd = open
idle
idledriver = on
cmd = close
Current
Open
Closed
Stuck
Open Closed
cmd = reset
Goal
cmd = off
cmd = on idle
idle
Current
On
Off
Resettable
On Off
cmd = off
Goal: Driver = off, Valve = closedSuccess
Current: Driver = off, Valve = closed
12
![Page 52: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/52.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Hierarchical, Model-based Reactive Planning• Compile-time Analysis:
– Compile-out interactions– Confirm schematics are loop free.– Depth first number variables.
• Periodic, Run-time Analysis:– Given initial state
• Identify allowed transitions and assignments
– Given autonomous jump to failure state• Identify allowed transitions and assignments
• Run-time Plan Execution:– Work conjunctive goals from outputs to inputs.– Achieve goals serially.– Only perform reversible transitions.– Lookup control actions and sub-goals in policies
![Page 53: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/53.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Complexity of Reactive Planning
• Worst Case per action: Depth * Sub-goal branch factor• Average Cost per action: Sub-goal branch factor
Valve1 = open Valve2 = open Driver1 = off Driver2 = off
Driver1 = on
CU = on
CU = on
Driver2 = on
CU = on
CU = on
CU = on CU = on
![Page 54: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/54.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
What If Plan is Not Serializable?
– compose each cycle into a single component.
BusControl
Computer
Antenna
Antenna
AmplifierK-bandTransmitter
AmplifierK-bandTransmitter
• What if causal graph G contains cycles?• Solution:
– Isolate the cyclic components (compute SCCs)
• New causal graph G’ is acyclic, • Goals of G’ are serializable
![Page 55: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/55.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Composing Cyclic Components
off
on
cmdT = off
Transmitter Amplifier
cmdT = onA = offA = off
off
on
cmdA = offcmdA = on
T = on
onT onA
onT offA
offT offA
offT onA
cmdT = offcmdT = on
cmdA = off
cmdA = on
cmdA = off
![Page 56: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/56.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Policy for Composed Components
onT onA
onT offA
offT offA
offT onA
cmdT = offcmdT = on
cmdA = off
cmdA = on
cmdA = off
cmdT = on
Goal
cmdT = on
cmdA = on idle
idle cmdA = off
Current
OnT, OnA
OnT, OffA
OffT, OffA
OnT, OnA OnT, OffA
idle
cmdT = off
cmdA = off
OffT, OffA
fail
fail
fail
OffT, OnA
fail fail cmdA = off idleOffT, OnA
• Problem: Composition grows exponential in space usage.
• Solution: Use BDD encoding
(in progress).
![Page 57: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/57.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Model-based Reactive Planning
1. Compile away constraints from the model
2. Compile away cyclic components
3. Plan serially pursuing causal graph upstream
4. Generate actions using hierarchical policies
Only performs reversible actions
Responds to failure at each step
Average cost per step = subgoal branching factor
![Page 58: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/58.jpg)
Artificial Intelligence & Space Systems Laboratories Massachusetts Institute of Technology
Current Demonstration Testbeds• Air Force Tech Sat 21 flight• NASA NMP ST-7 Phase A• NASA Mercury Messenger
on ground.• MIT Spheres on Space Station• NASA Robonaut, X-37, ISPP
• Multi-Rover Testbed• Simulated Air Vehicles
![Page 59: A Hierarchical Approach to Model-based Reactive Planning in Large State Spaces Artificial Intelligence & Space Systems Laboratories Massachusetts Institute](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649e565503460f94b4d870/html5/thumbnails/59.jpg)
Model-based Programming of Embedded Systems
• To survive decades embedded systems orchestrate
complex regulatory and immune systems.
• Future systems will be programmed with models,
describing themselves and their environments.
• Runtime kernels will be agile, deducing and planning by
solving optimization problems with propositional
constraints. • Model-based reactive planners respond quickly to failure,
while using compile-time analysis of structure to respond quickly and concisely to indirect effects.