bisimulation and metrics for labelled markov processesprakash/talks/aalborg_2012.pdf · the...

Bisimulation and Metrics for Labelled MarkovProcesses

Prakash Panangaden1

1School of Computer ScienceMcGill University

13th June 2012, University of Aalborg

Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 1 / 46

Outline

1 Introduction

Outline

1 Introduction

2 Labelled transition systems

Outline

1 Introduction

3 Ordinary bisimulation

Outline

1 Introduction

4 Discrete probabilistic transition systems

Outline

1 Introduction

5 Labelled Markov processes

Outline

1 Introduction

6 Probabilistic bisimulation

Outline

1 Introduction

7 * Proof of the logical characterization theorem

Outline

1 Introduction

8 Metrics

Outline

1 Introduction

8 Metrics

9 Continuous-state systems

Outline

1 Introduction

8 Metrics

9 Continuous-state systems

10 Conclusions

Introduction

Summary of Results

Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]

Introduction

Summary of Results

Logical characterization. [LICS98,Info and Comp 2002]

Introduction

Summary of Results

Metric analogue of bisimulation. [CONCUR99, TCS2004]

Introduction

Summary of Results

Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]

Introduction

Summary of Results

Weak bisimulation. [LICS02,CONCUR02]

Introduction

Summary of Results

Real time. [QEST 2004, JLAP 2003,LMCS 2006]

Introduction

Summary of Results

Event Bisimulation [I and C, 2006]

Introduction

Summary of Results

Event Bisimulation [I and C, 2006]

Abstract MPs [ICALP 2009]

Introduction

Collaborators

Josée Desharnais

Introduction

Collaborators

Josée DesharnaisGheorghe Comanici

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-Côté

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe Chaput

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent Danos

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas Edalat

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm Ferns

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet Gupta

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha Jagadeesan

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim Larsen

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois Laviolette

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu Mardare

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu MardareGordon Plotkin

Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu MardareGordon PlotkinDoina Precup

Labelled transition systems

The definition

A set of states S,

The definition

A set of states S,

a set of labels or actions, L or A and

The definition

A set of states S,

a transition relation ⊆ S ×A× S, usually written

→a⊆ S × S.

The transitions could be indeterminate (nondeterministic).

The definition

A set of states S,

a transition relation ⊆ S ×A× S, usually written

→a⊆ S × S.

The transitions could be indeterminate (nondeterministic).

We write s a−−→ s′ for (s, s′) ∈→a.

Ordinary bisimulation

Bisimulation

s and t are states of a labelled transition system. We say s is bisimilarto t – written s ∼ t – if

s a−−→ s′ ⇒ ∃t ′ such that t a

−−→ t ′ and s′ ∼ t ′

andt a−−→ t ′ ⇒ ∃s′ such that s a

−−→ s′ and s′ ∼ t ′.

Bisimulation relations

Define a (note the indefinite article) bisimulation relation R to bean equivalence relation on S such that

sRt means ∀a, s a−−→ s′ ⇒ ∃t ′, t a

−−→ t ′ with s′Rt ′

and vice versa.

We define s ∼ t if there is some bisimulation relation R with sRt .

and vice versa.

We define s ∼ t if there is some bisimulation relation R with sRt .

This is the version that is used most often.

An example

t4 t5P1 P2

An example

t4 t5P1 P2

Here s0 and t0 are not bisimilar.

An example

t4 t5P1 P2

Here s0 and t0 are not bisimilar.

However s0 and t0 can simulate each other!

How do we know that two processes are not bisimilar?

Define a logic as follows:

φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ

s |= 〈a〉φ means that s a−−→ s′ and t |= φ.

φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ

We can define a dual to 〈〉 (written []) by using negation.

φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ

We can define a dual to 〈〉 (written []) by using negation.

s |= [a]φ means that if s can do an a the resulting state mustsatisfy φ.

Examples of HM Logic

T is satisfied by any process, F is not satisfied by any process.

s |= 〈a〉T means s can do an a action.

s |= ¬〈a〉T or s |= [a]F means s cannot do an a action.

s |= 〈a〉(〈b〉T ) means that s can do an a and then do a b.

The logical characterization theorem

Two processes are bisimilar if and only if they satisfy the sameformulas of HM logic.

Basic assumption: the processes are finitely-branching (otherwiseyou need infinitary conjunctions).

To show that two processes are not bisimilar find a formula onwhich they disagree.

The role of negation

Consider the processes below:

s0 |= 〈a〉¬〈b〉T but t0 does not.

s0 and t0 agree on all formulas without negation.

Note that [a] has an implicit negation.

Discrete probabilistic transition systems

Just like a labelled transition system with probabilities associatedwith the transitions.

(S,L,∀a ∈ L Ta : S × S −→ [0,1])

The model is reactive: All probabilistic data is internal - noprobabilities associated with environment behaviour.

Bisimulation for PTS: Larsen and Skou

Consider

t0a[ 1

s0a[ 1

a[ 13 ]

Consider

t0a[ 1

s0a[ 1

a[ 13 ]

Should s0 and t0 be bisimilar?

Consider

t0a[ 1

s0a[ 1

a[ 13 ]

Should s0 and t0 be bisimilar?

Yes, but we need to add the probabilities.

The Official Definition

Let S = (S,L,Ta) be a PTS. An equivalence relation R on S is abisimulation if whenever sRs′, with s, s′ ∈ S, we have that for alla ∈ A and every R-equivalence class, A, Ta(s,A) = Ta(s′,A).

The notation Ta(s,A) means “the probability of starting from s andjumping to a state in the set A.”

Two states are bisimilar if there is some bisimulation relation Rrelating them.

Labelled Markov processes

What are labelled Markov processes?

Reactive systems: Larsen and Skou

Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.

All probabilistic data is internal - no probabilities associated withenvironment behaviour.

We observe the interactions - not the internal states.

In general, the state space of a labelled Markov process maybe a continuum.

Motivation

Model and reason about systems with continuous state spaces orcontinuous time evolution or both.

hybrid control systems; e.g. flight management systems.

Motivation

telecommunication systems with spatial variation; e.g. cell phones

Motivation

performance modelling,

Motivation

continuous time systems,

Motivation

continuous time systems,

probabilistic process algebra with recursion.

The Need for Measure Theory

Basic fact: There are subsets of R for which no sensible notion ofsize can be defined.

More precisely, there is no translation-invariant measure definedon all the subsets of the reals.

Actually there is if you only require finite additivity.

Stochastic Kernels

A stochastic kernel (Markov kernel) is a function h : S × Σ−→ [0,1] with (a) h(s, ·) : Σ −→ [0,1] a (sub)probability measureand (b) h(·,A) : X −→ [0,1] a measurable function.

Stochastic Kernels

Though apparantly asymmetric, these are the stochasticanalogues of binary relations

Stochastic Kernels

Though apparantly asymmetric, these are the stochasticanalogues of binary relations

and the uncountable generalization of a matrix.

Formal Definition of LMPs

An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S × Σ −→ [0,1] isa transition probability function such that

Formal Definition of LMPs

An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S × Σ −→ [0,1] isa transition probability function such that

∀s : S.λA : Σ.τα(s,A) is a subprobability measureand∀A : Σ.λs : S.τα(s,A) is a measurable function.

Probabilistic bisimulation

Larsen-Skou Bisimulation

Let S = (S, i ,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S,we have that for all a ∈ A and every R-closed measurable setA ∈ Σ, τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.

Larsen-Skou Bisimulation

Let S = (S, i ,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S,we have that for all a ∈ A and every R-closed measurable setA ∈ Σ, τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.

Can be extended to bisimulation between two different LMPs.

Logical Characterization

L ::== T|φ1 ∧ φ2|〈a〉qφ

We say s |= 〈a〉qφ iff

∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).

L ::== T|φ1 ∧ φ2|〈a〉qφ

We say s |= 〈a〉qφ iff

∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).

Two systems are bisimilar iff they obey the same formulas of L.[DEP 1998 LICS, I and C 2002]

That cannot be right?

Two processes that cannot be distinguished without negation.The formula that distinguishes them is 〈a〉(¬〈b〉⊤).

But it is!

s0a[p]

We add probabilities to the transitions.

But it is!

s0a[p]

If p + q < r or p + q > r we can easily distinguish them.

But it is!

s0a[p]

If p + q < r or p + q > r we can easily distinguish them.

If p + q = r and p > 0 then q < r so 〈a〉r 〈b〉1⊤ distinguishes them.

* Proof of the logical characterization theorem

Digression on Analytic Spaces

An analytic set A is the image of a Polish space X (or a Borelsubset of X ) under a continuous (or measurable) function f : X−→ Y , where Y is Polish. If (S,Σ) is a measurable space where Sis an analytic set in some ambient topological space and Σ is theBorel σ-algebra on S.

Analytic sets do not form a σ-algebra but they are in thecompletion of the Borel algebra under any measure. [Universallymeasurable.]

Regular conditional probability densities can be defined onanalytic spaces.

Amazing Facts about Analytic Spaces

Given A an analytic space and ∼ an equivalence relation suchthat there is a countable family of real-valued measurablefunctions fi : S −→ R such that

∀s, s′ ∈ S.s ∼ s′ ⇐⇒ ∀fi .fi(s) = fi(s′)

then the quotient space (Q,Ω) - where Q = S/ ∼ and Ω is thefinest σ-algebra making the canonical surjection q : S −→ Qmeasurable - is also analytic.

Amazing Facts about Analytic Spaces

Given A an analytic space and ∼ an equivalence relation suchthat there is a countable family of real-valued measurablefunctions fi : S −→ R such that

∀s, s′ ∈ S.s ∼ s′ ⇐⇒ ∀fi .fi(s) = fi(s′)

then the quotient space (Q,Ω) - where Q = S/ ∼ and Ω is thefinest σ-algebra making the canonical surjection q : S −→ Qmeasurable - is also analytic.

If an analytic space (S,Σ) has a sub-σ-algebra Σ0 of Σ whichseparates points and is countably generated then Σ0 is Σ! TheUnique Structure Theorem (UST).

The Quotient

Given (S,Σ, τa) an LMP, we define s ≃ s′ if s and s′ obey exactlythe same formulas of L0.

The Quotient

The functions I[[φ]] : S −→ R defined by I[[φ]](s) = 1 if s |= φ and 0otherwise are a countable family of measurable functions suchthat s ≃ s′ if and only if all the functions agree on s and s′. Thusthe quotient space (Q,Ω) is analytic.

The Quotient

The functions I[[φ]] : S −→ R defined by I[[φ]](s) = 1 if s |= φ and 0otherwise are a countable family of measurable functions suchthat s ≃ s′ if and only if all the functions agree on s and s′. Thusthe quotient space (Q,Ω) is analytic.

We define an LMP (Q,Ω, ρa) where ρa(t ,U) := τa(s,q−1(U));s ∈ q−1(t).

ρ is well defined - I

Easy to check that q−1(q([[φ]])) = [[φ]]:s ∈ q−1(q([[φ]])) implies that q(s) ∈ q([[φ]]), i.e. ∃s′ ∈ [[φ]].s ≃ s′, so s |= φ so s ∈ [[φ]].

Thus q([[φ]]) is measurable.

Thus the σ-algebra generated -say, Λ - by q([[φ]]) is asub-σ-algebra of Ω.

Λ is countably generated and separates points so by UST it is Ω.Thus q([[φ]]) generates Ω.

ρ is well defined - II

The collection q([[φ]]) is a π-system (because L0 has conjunction)and it generates Ω; thus if we can show that two measures agreeon these sets they agree on all of Ω.

If q(s) = q(s′) = t then τa(s, [[φ]]) = τa(s′, [[φ]]) (simpleinterpolation).

Thus τa(s,q−1(q([[φ]]))) = τa(s′,q−1(q([[φ]]))) and hence ρ is welldefined. We have ρa(q(s),B) = τa(s,q−1(B)).

Finishing the Argument

Let X be any ≃-closed subset of S.

Then q−1(q(X )) = X and q(X ) ∈ Ω.

If s ≃ s′ then q(s) = q(s′) and

τa(s,X ) = τa(s,q−1(q(X ))) = ρa(q(s),q(X )) =

ρa(q(s′),q(X )) = τa(s′,q−1(q(X ))) = τa(s′,X ).

Metrics

A metric-based approximate viewpoint

Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).

Metrics

Formalize distance as a metric:

d(s, s) = 0,d(s, t) = d(t , s),d(s,u) ≤ d(s, t) + d(t ,u).

Quantitative analogue of an equivalence relation.

Metrics

Formalize distance as a metric:

d(s, s) = 0,d(s, t) = d(t , s),d(s,u) ≤ d(s, t) + d(t ,u).

Quantitative analogue of an equivalence relation.

Quantitative measurement of the distinction between processes.

Metrics

Criteria on Metrics

Soundness:d(s, t) = 0 ⇔ s, t are bisimilar

Metrics

Criteria on Metrics

Stability of distance under temporal evolution:“Nearby states stayclose forever.”

Metrics

Criteria on Metrics

Stability of distance under temporal evolution:“Nearby states stayclose forever.”

Metrics should be computable (efficiently?).

Metrics

Bisimulation Recalled

Let R be an equivalence relation. R is a bisimulation if: s R t if:

(s −→ P) ⇒ [t −→ Q,P =R Q]

(t −→ Q) ⇒ [s −→ P,P =R Q]

where P =R Q if

(∀R − closed E) P(E) = Q(E)

Metrics

A putative definition of a metric analogue ofbisimulation

m is a metric-bisimulation if: m(s, t) < ǫ ⇒:

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

Metrics

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

Problem: what is m(P,Q)? — Type mismatch!!

Metrics

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

Problem: what is m(P,Q)? — Type mismatch!!

Need a way to lift distances from states to a distances ondistributions of states.

Metrics

A detour: Kantorovich metric

Metrics on probability measures on metric spaces.

Metrics

M: 1-bounded pseudometrics on states.

Metrics

d(µ, ν) = supf

fdµ−

fdν|, f 1-Lipschitz

Metrics

d(µ, ν) = supf

fdµ−

fdν|, f 1-Lipschitz

Arises in the solution of an LP problem: transshipment.

Metrics

An LP version for Finite-State Spaces

When state space is finite: Let P,Q be probability distributions. Then:

m(P,Q) = max∑

(P(si)− Q(si))ai

subject to:∀i .0 ≤ ai ≤ 1∀i , j . ai − aj ≤ m(si , sj).

Metrics

The Dual Form

Dual form from Worrell and van Breugel:

Metrics

The Dual Form

min∑

lijm(si , sj ) +∑

xi +∑

subject to:∀i .

j lij + xi = P(si)

∀j .∑

i lij + yj = Q(sj )∀i , j . lij , xi , yj ≥ 0.

Metrics

The Dual Form

min∑

lijm(si , sj ) +∑

xi +∑

subject to:∀i .

j lij + xi = P(si)

∀j .∑

i lij + yj = Q(sj )∀i , j . lij , xi , yj ≥ 0.

We prove many equations by using the primal form to show onedirection and the dual to show the other.

Metrics

Example

Let m(s, t) = r < 1. Let δs(δt) be the probability measureconcentrated at s(t). Then,

m(δs, δt) = r

Metrics

Example

m(δs, δt) = r

Upper bound from dual: Choose lst = 1 all other lij = 0. Then

lijm(si , sj ) = m(s, t) = r .

Metrics

Example

m(δs, δt) = r

Upper bound from dual: Choose lst = 1 all other lij = 0. Then

lijm(si , sj ) = m(s, t) = r .

Lower bound from primal: Choose as = 0,at = r , all others tomatch the constraints. Then

(δt (si)− δs(si ))ai = r .

Metrics

The Importance of the Example

We can isometrically embed the original space in the metric space ofdistributions.

Metrics

Return from Detour

Summary of detour: Given a metric on states in a metric space, can liftto a metric on probability distributions on states.

Metrics

Metric “Bisimulation”

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

Metrics

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

The required canonical metric on processes is the least such: ie.the distances are the least possible.

Metrics

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

The required canonical metric on processes is the least such: ie.the distances are the least possible.

Thm: Canonical least metric exists. Usual fixed-point theoryarguments.

Continuous-state systems

What about Continuous-State Systems?

Develop a real-valued “modal logic” based on the analogy:Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ

Define a metric based on how closely the random variables agree.

We did this before the LP based techniques became available.

Real-valued Modal Logic

f ::= 1 | max(f , f ) | h f | 〈a〉.f

1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ

s′∈S f (s′)τa(s, ds′) a-transition

where h 1-Lipschitz : [0,1] → [0,1] and γ ∈ (0,1].

f ::= 1 | max(f , f ) | h f | 〈a〉.f

d(s, t) = supf |f (s)− f (t)|

f ::= 1 | max(f , f ) | h f | 〈a〉.f

d(s, t) = supf |f (s)− f (t)|

Thm: d coincides with the canonical metric-bisimulation.

Finitary syntax for Real-valued modal logic

1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunction(1 − f )(s) = 1 − f (s) Negation

⌊fq(s)⌋ =

q , f (s) ≥ qf (s) , f (s) < q

Cutoffs

〈a〉.f (s) = γ∫

q is a rational.

Conclusions

bisimulation and metrics for labelled markov processesprakash/talks/aalborg_2012.pdf · the...

Documents

bisimulation and simulation

approximating labelled markov...

labelled markov processesprakash/talks/lecture2.pdf ·...

model theory of xpath on data trees. part i: bisimulation

information theory and security prakash panangaden mcgill...

labelled markov processesprakash/talks/lecture3.pdf ·...

metrics for real time probabilistic processes radha...

rdf graph alignment with bisimulation

bisimulation collapse and the process taxonomy

process algebra (2if45) probabilistic branching...

on the origins of bisimulation and...

bisimulation -based abstraction of sodium-channel dynamics...

weak bisimulation for action-type coalgebras - tu/e · pdf...

bisimulation by unification

characteristic invariants in hennessy–milner...

bisimulation, model checking and other...

on bisimulation proofs for the analysis of distributed

approximating markov processes,...

deciding bisimulation and trace equivalences for systems...

eindhoven university of technology master bisimulation