making prophecies with decision predicatesejk/slides/dpredicates.pdf · 2017. 8. 11. · toy...

Making Prophecies with Decision Predicates

Eric KoskinenUniversity of Cambridge

Joint work with Byron Cook

Tuesday, 1 February 2011

Goal: prove LTL properties of real software


TraditionalTraditional Our ApproachOur ApproachProgram Property Time(s) Result Time(s) ResultExample from Sec. 2 FGp 2.32 1.98 Example from Fig. 8 of [15] G(p⇒Fq) 209.64 27.94 Toy acq/rel G(p⇒Fq) 103.48 14.18 Toy lin. arith. 1 p⇒Fq 126.86 34.51 Toy lin. arith. 2 p⇒Fq timeouttimeout 6.74 PostgreSQL strsrv G(p⇒FGq) timeouttimeout 9.56 PostgreSQL strsrv+bug G(p⇒FGq) 87.31 χ 47.16 χPostgreSQL pgarch FGp 31.50 15.20 PostgreSQL dropbuf Gp timeouttimeout 1.14 PostgreSQL dropbuf G(p⇒Fq) 53.99 27.54 Apache child G(p⇒GFq) timeouttimeout 197.41 Apache child accept liveness G(p⇒(Fa ∨ Fb)) 685.34 684.24 Windows frag. 1 G(p⇒Fq) 901.81 539.00 Windows frag. 2 FGp 16.47 52.10 Windows frag. 2+bug FGp 26.15 χ 30.37 χWindows frag. 3 FGp 4.21 15.75 Windows frag. 4 G(p⇒Fq) timeouttimeout 1,114.18 Windows frag. 4 (Fp) ∨ (Fq) 1,223.96 100.68 Windows frag. 5 G(p⇒Fq) timeouttimeout timeouttimeoutWindows frag. 6 FGp 149.41 59.56 Windows frag. 6+bug FGp 6.06 χ 22.12 χWindows frag. 7 GFp timeouttimeout 55.77 Windows frag. 8 FGp timeouttimeout 5.24


How did we do it?

Traditional ApproachAutomata theoretic, trace-based strategy(trace based. reason over sets of traces)

Our ApproachUse state-based reasoning, with auxilliary state to track history/future(as per Abadi/Lamport)


How did we do it?

Our ApproachUse state-based reasoning, with auxilliary state to track history/future(as per Abadi/Lamport)

Traditional ApproachAutomata theoretic, trace-based strategy(trace based. reason over sets of traces)

prophecy variables


How to decide what prophecy variables are needed?

How did we do it?

Open Problem:


How to decide what prophecy variables are needed?

In this paper: Automatically discover and characterize what prophecies are needed with decision predicates

How did we do it?

Open Problem:


l1

l2

l3

l4

x = true

x := true

x := false

Example

G[¬x ⇒ (F x)]


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x] LTL1 2 3 4 4 4 4

Example

x=true

x=false

G[¬x ⇒ (F x)]


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x] LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

Example

x=true

x=false

G[¬x ⇒ (F x)]


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x] LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

. . .1 2 2 2 3 4 4

Example

x=true

x=false

G[¬x ⇒ (F x)]


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x] LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

. . .1 2 2 2 3 4 4

1 2 2 2 2 2 2

Example

x=true

x=false

G[¬x ⇒ (F x)]


l1

l2

l3

l4

x = true

x := true

x := false

. . .

LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

Example

. . . but not a scalable tool.Try using a state-based approach . . .

This LTL property holds

x=true

x=false

G[(F x) ∨ x]

G[¬x ⇒ (F x)]


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x]

. . .

LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

Example

η : ϕL → ϕC

η(α) = αη(ϕL ∧ ψL) = η(ϕL) ∧ η(ψL)η(ϕL ∨ ψL) = η(ϕL) ∨ η(ψL)η(GϕL) = AG η(ϕL)η(FϕL) = AF η(ϕL)η(ϕLWψL) = A[η(ϕL) W η(ψL)]

x=true

x=falseTuesday, 1 February 2011

l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x]

. . .

LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

Example

η : ϕL → ϕC


For any ϕL,s C η(ϕL) ⇒ π L ϕL

x=true


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x]

. . .

LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

Example

η : ϕL → ϕC



PROVE (M,φL) Ω := ∅; let φC = η(φL) in while true do let MΩ = DETERMINIZE(M,Ω) in match (PROVE∀CTL(MΩ,φC)) with | Succeed -> return Succeed | Fail(χ) -> let Ω′ = REFINE(χ) in if (Ω′ = ∅) then let π ∈ χ in return Fail(π) else Ω := Ω ∪ Ω′; done

x=true


l1

l2

l3

l4

x = true

x := true

x := false

G[(F x) ∨ x]

. . .

LTL1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

Example

η : ϕL → ϕC




x=true

x=false

Usually it just works!


LTL

l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

AG[(AF x) ∨ x] ∀CTL

1

G[(F x) ∨ x]

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

x=true


LTL

l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2


1

G[(F x) ∨ x]

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4[(AF x) ∨ x]

x=true


LTL

l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2


1

G[(F x) ∨ x]

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4[(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x]

x=true


LTL

l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2


1

G[(F x) ∨ x]

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4


[(AF x) ∨ x]

x=true


LTL

l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2


1

G[(F x) ∨ x]

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4


[(AF x) ∨ x][(AF x) ∨ x]

x=true


LTL

l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2


1

G[(F x) ∨ x]

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4


[(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x][(AF x) ∨ x]

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2


1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

G[(F x) ∨ x]

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example

. . .

1 2 3 4 4 4 4

1 2 2 3 4 4 4

1 2 2 2 3 4 4

1 2 2 2 2 2 2

AF (AG x) ∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example

AF (AG x)Counterexample

∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

AG x

AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

AG x

AG x?AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

AG xAG x

AG x?AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

AG xAG x

AG x?AG x?AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

AG xAG x

AG x?AG x?AG x?

AG x

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

AG x?

AG xAG x

AG x?AG x?AG x?AG x?AG x?

AG x

AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

?

AG x?

AG xAG x

AG x?AG x?AG x?AG x?AG x?

AG x

AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

?

AG x?

x=true


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

3

x=true ∧ pc=l1

x=false ∧ pc=l31

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

?What if we knew the future?

What if we could look at the current state (i.e. “now”)and know what the program’s behavior will be in the future.

You can solve this with prophecy variables (e.g. Abadi/Lamport)

But what do we need to know about the future?


l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

?

PROVE (M,φL) let φC = η(φL) in match (PROVE∀CTL(M ,φC)) with | Succeed -> return Succeed | Fail(χ) ->


x=true


(REFINE())

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


(REFINE())Decision Predicates

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


(REFINE())

s

t

t’

Decision Predicates

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


(REFINE())

s

t

t’

a

b

¬b

Decision Predicates

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


(REFINE())

s

t

t’

a

b

¬b

Decision Predicates Prophecy Variables

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


(REFINE())

s

t

t’

a

b

¬b


s t

t’

a b

¬bsa

F

T

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


(REFINE())

adecision predicate

pair (a,b) characterizes

nondeterminism

s

t

t’

a

b

¬b


s t

t’

a b

¬bsa

F

T

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

Decision Predicates


Decision Predicates

adecision predicate

pair (a,b) characterizes

nondeterminism

s

t

t’

a

b

¬b


s t

t’

a b

¬bsa

F

T

Counterexample

1

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

a ≡ (pc = l2)

b ≡ (pc = l2)

a b

¬b


asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true

x := false

Example


∀CTL

1

F (G x)

3

2 2 2 2 2 2

3 4

3 4 4

. . .

4

4

4 4

4

4

LTL

?

a ≡ (pc = l2)

b ≡ (pc = l2)

ρ ∈ ⊥ ∪ N

asm(ρ = 0)

a b

¬b

x=true


asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true

x := false

Example

AF (AG x) ∀CTL

1,⊥

F (G x)

3,2

2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥

3,1 4,1

3,0 4,0 4,0

. . .

4,0

4,1

4,2 4,2

4,1

4,2

LTL

ρ ∈ ⊥ ∪ N

asm(ρ = 0)

1,0 2,0

1,1 2,1

1,2 2,2

2,1

2,2 2,2

x=true


asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true

x := false

Example

AF (AG x) ∀CTL

1,⊥

F (G x)

3,2

2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥

3,1 4,1

3,0 4,0 4,0

. . .

4,0

4,1

4,2 4,2

4,1

4,2

LTL

ρ ∈ ⊥ ∪ N

asm(ρ = 0)

1,0 2,0

1,1 2,1

1,2 2,2

2,1

2,2 2,2x=true


asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true

x := false

Example

AF (AG x) ∀CTL

1,⊥

F (G x)

3,2

2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥

3,1 4,1

3,0 4,0 4,0

. . .

4,0

4,1

4,2 4,2

4,1

4,2

LTL

ρ ∈ ⊥ ∪ N

asm(ρ = 0)

1,0 2,0

1,1 2,1

1,2 2,2

2,1

2,2 2,2

x=true


asm(ρ = 0)

ρ--

l1

l2

l3

l4

x = true

x := true

x := false

Example

AF (AG x) ∀CTL

1,⊥

F (G x)

3,2

2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥ 2,⊥

3,1 4,1

3,0 4,0 4,0

. . .

4,0

4,1

4,2 4,2

4,1

4,2

LTL

ρ ∈ ⊥ ∪ N

asm(ρ = 0)

1,0 2,0

1,1 2,1

1,2 2,2

2,1

2,2 2,2


x=true


ρ ∈ ⊥ ∪ N

Determinize((S, R, I),Ω) = (SΩ, RΩ, IΩ) where

SΩ = S ×−→N⊥ denoted s, ρ

IΩ = I ×−→N⊥

RΩ = (s, ρ, s, ρ) | (s, s) ∈ R ∧ ∀0 ≤ i ≤ Ω.

[ai(s) ∧ ρi = ⊥ ⇒ bi(s) ∧ ρi = ⊥]∧[ai(s) ∧ ρi > 0 ⇒ bi(s) ∧ ρi = ρi − 1]∧[ai(s) ∧ ρi = 0 ⇒ ¬bi(s) ∧ ρi ∈ N⊥]

∧[¬ai(s) ⇒ ρi = ρi]

asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true

x := falseasm(ρ = 0)

DETERMINIZE(M,Ω)


ρ ∈ ⊥ ∪ N



IΩ = I ×−→N⊥

RΩ = (s, ρ, s, ρ) | (s, s) ∈ R ∧ ∀0 ≤ i ≤ Ω.


∧[¬ai(s) ⇒ ρi = ρi]

(a0, b0), (a1, b1), ...

asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true


DETERMINIZE(M,Ω)


ρ ∈ ⊥ ∪ N



IΩ = I ×−→N⊥

RΩ = (s, ρ, s, ρ) | (s, s) ∈ R ∧ ∀0 ≤ i ≤ Ω.


∧[¬ai(s) ⇒ ρi = ρi]

(a0, b0), (a1, b1), ...

asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true


DETERMINIZE(M,Ω)

Theorem 1. For any Ω, MΩ ∼M


ρ ∈ ⊥ ∪ N



IΩ = I ×−→N⊥

RΩ = (s, ρ, s, ρ) | (s, s) ∈ R ∧ ∀0 ≤ i ≤ Ω.


∧[¬ai(s) ⇒ ρi = ρi]

(a0, b0), (a1, b1), ...

asm(ρ = 0)ρ--

l1

l2

l3

l4

x = true

x := true


DETERMINIZE(M,Ω)

Theorem 1. For any Ω, MΩ ∼M

Proof is based on (a modified version of)Refinement Mappings [Abadi/Lamport ’88]


REFINE



REFINE REFINE



REFINE REFINEREFINE



REFINE REFINEREFINE REFINE



REFINE(χ) = ∅



REFINE(χ) = ∅


All prefixes of CTL c.e.x.represent the same trace.So it is a valid LTL c.e.x.


ExamplePROVE (M,φL) Ω := ∅; let φC = η(φL) in while true do let MΩ = DETERMINIZE(M,Ω) in match (PROVE∀CTL(MΩ,φC)) with | Succeed -> return Succeed | Fail(χ) -> let Ω′ = REFINE(χ) in if (Ω′ = ∅) then let π ∈ χ in return Fail(π) else Ω := Ω ∪ Ω′; done


ExamplePROVE (M,φL) Ω := ∅; let φC = η(φL) in while true do let MΩ = DETERMINIZE(M,Ω) in match (PROVE∀CTL(MΩ,φC)) with | Succeed -> return Succeed | Fail(χ) -> let Ω′ = REFINE(χ) in if (Ω′ = ∅) then let π ∈ χ in return Fail(π) else Ω := Ω ∪ Ω′; done

• Usually, yes.

• In general, no.

Does this terminate?


Why does this work so well?

• Apply state-based reasoning

• Not determinizing (prophecizing)the entire state space

• Only making propheciesabout problematic nondeterminism(characterized by decision predicates)


Experiments


Experiments• Implemented in CIL

• Our novel infinite-state ACTL verifier:

Reduces branching-time verificationto a program analysis problem

(use known tools for safety & termination)

PROVE∀CTL

Come to my talk tonightin the student session!


Experiments

• Benchmarks from Apache, PostgreSQL, and Windows kernel code.

• Heap commands abstracted away[via Magill et al. POPL 2010]

• Compared against traditional trace-based automata theoretic approach [Gotsman et al. POPL 2007]


PreviousPrevious Our ApproachOur ApproachOur ApproachProgram Property Time(s) Result Time(s) D.P.s ResultExample from Sec. 2 FGp 2.32 1.98 1 Example from Fig. 8 of [15] G(p⇒Fq) 209.64 27.94 0 Toy acq/rel G(p⇒Fq) 103.48 14.18 0 Toy lin. arith. 1 p⇒Fq 126.86 34.51 0 Toy lin. arith. 2 p⇒Fq timeouttimeout 6.74 0 PostgreSQL strsrv G(p⇒FGq) timeouttimeout 9.56 0 PostgreSQL strsrv+bug G(p⇒FGq) 87.31 χ 47.16 0 χPostgreSQL pgarch FGp 31.50 15.20 0 PostgreSQL dropbuf Gp timeouttimeout 1.14 0 PostgreSQL dropbuf G(p⇒Fq) 53.99 27.54 0 Apache child G(p⇒GFq) timeouttimeout 197.41 2 Apache child G(p⇒(Fa ∨ Fb)) 685.34 684.24 0 Windows frag. 1 G(p⇒Fq) 901.81 539.00 0 Windows frag. 2 FGp 16.47 52.10 3 Windows frag. 2+bug FGp 26.15 χ 30.37 0 χWindows frag. 3 FGp 4.21 15.75 1 Windows frag. 4 G(p⇒Fq) timeouttimeout 1,114.18 0 Windows frag. 4 (Fp) ∨ (Fq) 1,223.96 100.68 0 Windows frag. 5 G(p⇒Fq) timeouttimeout timeouttimeouttimeoutWindows frag. 6 FGp 149.41 59.56 0 Windows frag. 6+bug FGp 6.06 χ 22.12 0 χWindows frag. 7 GFp timeouttimeout 55.77 0 Windows frag. 8 FGp timeouttimeout 5.24 0


Conclusions

• Prophecy variables enable state-based reasoning for trace properties

• But you need to know what to make prophecies about (decision predicates)

• Obtained a scalable tool for proving trace properties of real software


On the job market• Technically deep and broad

• Formal Methods and Analysis(e.g. decision predicates, coarse-grained txns, Speed)

• Systems (e.g. Transactional Boosting, Dreadlocks)

• Publications

• POPL’11, POPL’10, PLDI’09,PPoPP’08, SPAA’08, SPAA’08, EuroSys’08, Transact x3

• Industry experience: developer at Amazon.com

[email protected]


mailto:[email protected]

mailto:[email protected]

making prophecies with decision predicatesejk/slides/dpredicates.pdf · 2017. 8. 11. · toy...

Documents