Solving Timed Games with Variable Observations:

Proof of ConceptPeter BulychevFranck Cassez

Alexandre DavidKim G. Larsen

Jean-François RaskinPierre-Alain Reynier

GASICS Workshop 2

Timed Game Automata Timed Game Automata is

a Timed Automata where transitions are split into controllable and uncontrollable

We support safety objectives: control: AG (not Bad)

Memoryless strategy: state action

UPPAAL Tiga can be used to solve safety timed games

a

b

GASICS Workshop 3

Timed Game Automata

x≤1 : a

True : DELAYStrategy

x≤1 : b

True : DELAY

control: AG (not Bad)

a

b

GASICS Workshop 4

Controller synthesis with partial observation

Consider that controller doesn’t have full information about the current state of a system

Observation is a valuation of a finite number of state-based boolean predicates (sensors)

We allow predicates of the form: (L1 or L2 or L3) and (1≤x<2)

Controller makes its decisions based on history of the observations seen so far

Controller sees only changes on observations => stuttering-invariant strategy

GASICS Workshop 5

Controller synthesis with partial observation: the algorithm

Partition the state-space w.r.t. values of the predicates.Predicates p1, p2Losing is observable.

p1p2p1p2

p1p2p1p2

LOSING

aa

a b

DELAYb

Running example (LH boxes)

GASICS Workshop 6

Possible sets of observations: {H, L}

{H, L, y≥3}

control: AG (not Bad)

{y ≥ 1} {H, L, y≥5}

full information

{H, L, y≥1}EJECT RESET

GASICS Workshop 7

Controller synthesis with partial observation: the algorithm

Partition the state-space w.r.t. observations.Observations O1 O2 O3.Winning/losing is observable.

Algorithm, described in F. Cassez et al., 2007: Symbolic On-the-fly Subset construction-based Implemented in UPPAAL Tiga

Running example (LH boxes)

GASICS Workshop 8

{}

control: AG (not Bad)Available observations: {H, L, y ≥ 5}

{H}

{}DELAY DELAY {y ≥ 5}

DELAY

EJECT{} {y ≥ 5}

DELAY

{L}

{}DELAY {y ≥ 5}

DELAYEJECT

E0,x==y==0

H,x==y==0 E1\/E2,x==y==0 E1\/E2,x==y==5

RESET

E1\/E2,x==5, y==0 E1\/E2,x==10, y==5

H,x==y==0 E3\/E4,x==y==0 E3\/E4,x==y==5

Problem statement Assume a finite set of available sensors and

each sensor has some cost We want to synthesize a controller that will

achieve its goal by using a set of sensors with a minimal cost

Input: Timed Game Automata A Safety property φ A set of predicates Pred = {p1, …, pn} Cost function ω = {p1->c1, …, pn->cn}

Goal: To find a set of predicates P with a minimal total

cost such that A,P|=φ is trueGASICS Workshop 9

Basic algorithm

Consider a lattice of all possible predicates sets

GASICS Workshop 10

{φ}

{φ} U Pred

Basic algorithm1. Check if φ is controllable on A with full information

GASICS Workshop 11

{φ}

{φ} U Pred

Full information

Basic algorithm1. Check if φ is controllable on A with full information2. Check A,P|=φ for some set of predicates P

GASICS Workshop 12

{φ}

{φ} U Pred

P

Basic algorithm1. Check if φ is controllable on A with full information2. Check A,P|=φ for some set of predicates P3. If A,P|=φ is true, then we

remove from further consideration all sets P’ s.t. P⊆P’

GASICS Workshop 13

{φ}

{φ} U Pred

P

Basic algorithm

GASICS Workshop 14

{φ}

{φ} U Pred

1. Check if φ is controllable on A with full information2. Check A,P|=φ for some set of predicates P3. If A,P|=φ is true, then we

remove from further consideration all sets P’ s.t. P⊆P’

remove from further consideration all sets P’ s.t. ω(P’) ≥ ω(P)

P

Basic algorithm1. Check if φ is controllable on A with full information2. Check A,P|=φ for some set of predicates P3. If A,P|=φ is true, then we

remove from further consideration all sets P’ s.t. P⊆P’

remove from further consideration all sets P’ s.t. ω(P’) ≥ ω(P)

4. Otherwise, we remove from further

consideration all sets P’ s.t. P’⊆P

GASICS Workshop 15

{φ}

{φ} U Pred

Basic algorithmThe set of possible observation sets is finite, so the algorithm will converge

GASICS Workshop 16

{φ}

{φ} U Pred

Basic algorithmOptimizations: Which exploration strategy to

use? Random Top-bottom Bottom-top Midpoint

What information to reuse? Losing states from below Winning states from above State space from below

GASICS Workshop 17

{φ}

{φ} U Pred

Basic algorithmOptimizations: Which exploration strategy to

use? Random Top-bottom Bottom-top Midpoint

What information to reuse? Losing states from below Winning states from above State space from below

GASICS Workshop 18

{φ}

{φ} U Pred

Basic algorithmOptimizations: Which exploration strategy to

use? Random Top-bottom Bottom-top Midpoint

What information to reuse? Losing states from below Winning states from above State space from below

GASICS Workshop 19

{φ}

{φ} U Pred

Basic algorithmOptimizations: Which exploration strategy to

use? Random Top-bottom Bottom-top Midpoint

What information to reuse? Losing states from below Winning states from above State space from below

GASICS Workshop 20

{φ}

{φ} U Pred

Basic algorithmOptimizations: Which exploration strategy to

use? Random Top-bottom Bottom-top Midpoint

What information to reuse? Losing states from below Winning states from above State space from below

GASICS Workshop 21

{φ}

{φ} U Pred

Basic algorithmOptimizations: Which exploration strategy to

use? Random Top-bottom Bottom-top Midpoint

What information to reuse? Losing states from below Winning states from above State space from below

GASICS Workshop 22

{φ}

{φ} U Pred

State space reusage

GASICS Workshop 23

{φ}

{φ} U Pred

aaab

a

b

L1, x≥4

L2, x≥5

L3, x<2

L4, x≥8

L5, x≥7

L6, x<2

(L1, x≥4) ∨(L2, x≥5) ∨(L3, x<2)

(L4, x≥8) ∨(L5, x≥7) ∨(L6, x<2)L6, x<2

State space reusage

GASICS Workshop 24

{φ}

{φ} U Pred

aaab

a

b

L1, x≥4

L2, x≥5

L3, x<2

L4, x≥8

L5, x≥7

L6, x<2

(L1, x≥4) ∨(L2, x≥5) ∨(L3, x<2)

(L4, x≥8) ∨(L5, x≥7) ∨(L6, x<2)L6, x<2

Implementation details

25

EfficientStable

Ready for industry applications

Has a nice GUI

Easy to prototype newvery specific features

Python framework for timed automata manipulation

PyDBM – Python wrapper for UPPAAL DBM library

pyuppaal – syntactic parser of UPPAAL models

dbmpyuppaal – parses a model using pyuppaal and replaces all guards and invariants by their DBMs

opaal – model checker for timed automata

More information at: http://cs.aau.dk/~adavid/python

GASICS Workshop 26

Results

GASICS Workshop 2727

EJECT RESET

Possible observations and their cost:{H -> 1, L ->1, y≥1 -> 10, y≥2 -> 9, …, y≥10 -> 1}Optimal solution: {H, y≥5}

Results (average running time)

GASICS Workshop 28

Rando

m explo

ra...

Middle

point

Top-b

ottom

Botto

m-up0:00:000:00:080:00:170:00:250:00:340:00:430:00:510:01:000:01:09

with state space reusagew/o state space reusage

Questions?

GASICS Workshop 29