approximate reasoning for probabilistic real-time processes

Approximate reasoning for probabilistic real-time processes

Radha Jagadeesan DePaul University

Vineet Gupta Google Inc

Prakash Panangaden McGill University

Outline of talk

Beyond CTMCs to GSMPs The curse of real numbers Metrics Uniformities Approximate reasoning

Real-time probabilistic processes

Add clocks to Markov processes

Each clock runs down at fixed rate

Different clocks can have different rates

Generalized Semi Markov Processes: Probabilistic multi-rate timed automata

Generalized semi-Markov processes.

Each state is labelledwith propositional Information

Each state has a setof clocks associated with it.

{c,d}

{d,e} {c}

s

tu

Generalized semi-Markov processes.

Evolution determined bygeneralized states <state, clock-valuation>

<s,c=2, d=1>

Transition enabled when a clockbecomes zero

{c,d}

{d,e} {c}

s

tu

Generalized semi-Markov processes.

<s,c=2, d=1> Transition enabled in 1 time unit

<s,c=0.5,d=1> Transition enabled in 0.5 time unit

{c,d}

{d,e} {c}

s

tu

Clock c

Clock d

Generalized semi-Markov processes.

c. This need not be exponential.

{c,d}

{d,e} {c}

s

tu

Clock c

Clock d

0.2 0.8

Transition determines:

a. Probability distribution on next states

b. Probability distribution on clock values for new clocks

Generalized semi Markov processes If distributions are continuous and states are

finite:

Zeno traces have measure 0

Continuity results. If stochastic processes from <s, > converge to the stochastic process at <s, >

The traditional reasoning paradigm

Establishing equality: Coinduction Distinguishing states: HM-type logics Logic characterizes the equivalence (often

bisimulation) Compositional reasoning: ``bisimulation is

a congruence’’

Labelled Markov Processes

PCTL Bisimulation [Larsen-Skou,

Desharnais-Edalat-P]

Markov Decision Processes

Bisimulation [Givan-Dean-Grieg]

Labelled Concurrent Markov Chains

PCTL [Hansson-Johnsson]

Labelled Concurrent Markov chains (with tau)

PCTLCompleteness: [Desharnais-

Gupta-Jagadeesan-P]

Weak bisimulation [Philippou-Lee-Sokolsky,

Lynch-Segala]

With continuous timeContinuous time Markov chains

CSL [Aziz-Balarin-Brayton-

Sanwal-Singhal-S.Vincentelli]

Bisimulation,Lumpability

[Hillston, Baier-Katoen-

Hermanns,Desharnais-P]

Generalized Semi-Markov processes

Stochastic hybrid systems

CSL

Bisimulation:?????

Composition:?????

The curse of real numbers: instability

Vs

Vs

Problem!

Numbers viewed as coming with an error estimate.

Reasoning in continuous time and continuous space is often via discrete approximations.

Asking for trouble if we require exact match

Idea: Equivalence metrics

Jou-Smolka90, DGJP99, …

Replace equality of processes by (pseudo) metric distances between processes

Quantitative measurement of the distinction between processes.

Criteria on approximate reasoning

Soundness Usability Robustness

Criteria on metrics for approximate reasoning Soundness

Stability of distance under temporal evolution: “Nearby states stay close” through temporal evolution.

``Usability’’ criteria on metrics

Establishing closeness of states: Coinduction.

Distinguishing states: Real-valued modal logics.

Equational and logical views coincide: Metrics yield same distances as real-valued modal logics.

``Robustness’’ criterion on approximate reasoning The actual numerical values of the

metrics should not matter too much. Only the topology matters? Our results show that everything is defined

“up to uniformities.’’

What are uniformities?

In topology open sets capture an abstract notion of “nearness”: continuity, convergence, compactness, separation …

In a uniformity one axiomatises the notion of “almost an equivalence relation”: uniform continuity, …

Uniform continuity is not a topological invariant.

Uniformities: definition

A nonempty collection U of subsets of SxS such that:

Every member of U contains If X in U then so is If X in U, there is a Y s.t. YoY is contained

in X Down closed, intersection closed

Two apparently different Uniformities which are actually the same

m(x,y) = |x-y| m(x,y) = |2x + sinx -2y – siny|

Uniformities (different)

m(x,y) = |x-y|

Our results

Our results

A metric on GSMPs based on Wasserstein-Kantorovich and Skorohod

A real-valued modal logic Everything defined up to uniformity

Results for discrete time models

Bisimulation Metrics

Logic (P)CTL(*) Real-valued modal logic

Compositionality Congruence Non-expansivity

Proofs Coinduction Coinduction

Results for continuous time models

Bisimulation Metrics

Logic CSL Real-valued modal logic

Compositionality ??? ???

Proofs Coinduction Coinduction

Metrics for discrete time probabilistic processes

Defining metric: An attempt

Define functional F on metrics.

Metrics on probability measures

Wasserstein-Kantorovich

A way to lift distances from states to a distances on distributions of states.

Metrics on probability measures

Not up to uniformities

If the Wasserstein metric is scaled you get the same uniformity, but when you compute the fixed point you get a different uniformity because the lattice of uniformities has a different structure (glbs are different) then the lattice of metrics.

Variant definition that works up to uniformities

Fix c<1. Define functional F on metrics

Desired metric is maximum fixed point of F

Reasoning up to uniformities

For all c<1 we get same uniformity

[see Breugel/Mislove/Ouaknine/Worrell]

Metrics for real-time probabilistic processes

Generalized semi-Markov processes.

{c,d}

{d,e} {c}

s

tu

Clock c

Clock d

Evolution determined bygeneralized states <state, clock-valuation>

: Set of generalized states

The role of paths

In the continuous time case we cannot use single actions: there is no notion of “primitive step”

We have to talk about a “timed path” of one process matching a “timed path” of another process.

Generalized semi-Markov processes.

{c,d}

{d,e} {c}

s

tu

Clock c

Clock d

Path:

Traces((s,c)): Probability distribution on a set of paths.

Accomodating discontinuities: cadlag functions

(M,m) a pseudometric space. cadlag if:

Countably many jumps, finitely many jumpshigher than any fixed “h”.

Defining metric: An attempt

Define functional F on metrics. (c <1)

traces((s,c)), traces((t,d)) are distributions on sets of cadlag functions.

What is a metric on cadlag functions???

Metrics on cadlag functions

Not separable!

are at distance 1 for unequal x,y

x y

Skorohod’s metrics on cadlag

Skorohod defined 4 metrics on cadlag: J1,J2

M1 and M2 with different convergence

properties.

All these are based on “wiggling” the time.

The M metrics “fill in the jumps”.

The J metrics do not.

Skorohod metric (J2)

(M,m) a pseudometric space. f,g cadlag with range M.

Graph(f) = { (t,f(t)) | t \in R+}

t

fg

(t,f(t))

Skorohod J2 metric: Hausdorff distance between graphs of f,g

f(t)g(t)

Skorohod J2 metric

(M,m) a pseudometric space. f,g cadlag

Examples of convergence to

Example of convergence

1/2

Example of convergence

1/2

Examples of convergence

1/2

Examples of convergence

1/2

Non-convergence in J2:

Sequences of continuous functions cannot converge toa discontinuous function.

In general, the number of jumps can decrease in the limit,but they cannot increase.

Non-convergence

Non-convergence

Non-convergence

Non-convergence

Summary of Skorohod J2

A separable metric space on cadlag functions

Allows jumps to be nearby Allows jumps to decrease in the limit. Not complete.

Defining metric coinductively

Define functional on 1-bounded pseudometrics (c <1)

Desired metric: maximum fixpoint of F

a. s, t agree on all propositions

b.

Results

All c<1 yield the same uniformity. Thus, construction can be carried out in lattice of uniformities.

Real valued modal logic which gives an alternate definition of a metric.

For each c<1, modal logic yields the same uniformity but not the same metric.

Proof steps

Continuity theorems (Whitt) of GSMPs yield separable basis.

Finite separability arguments yield the result that the closure ordinal of the functional F is omega.

Duality theory of LP for calculating metric distances.

Summary

Metric on GSMPs defined up to uniformity. Real valued modal logic that gives the

same uniformity. Approximating quantitative observables:

Expectations of continuous functions are continuous.

Might be worth looking at the M2 metric.

Real-valued modal logic

Real-valued modal logic

Real-valued modal logic

Real-valued modal logic

h: Lipschitz operator on unit interval

Real-valued modal logic

Base case for path formulas??

Base case for path formulas

First attempt:

Evaluate state formula F on stateat time t

Problem: Not smooth enough wrt time sincepaths have discontinuities

Base case for path formulas

Next attempt:

``Time-smooth’’ evaluation of state formula F at time t on path

Upper Lipschitz approximation to evaluatedat t

Real-valued modal logic

Non-convergence

Illustrating Non-convergence

1/2

1/2