abstraction for falsification

Abstraction for FalsificationAbstraction for Falsification

Thomas Ball

Orna Kupferman

Greta Yorsh

Microsoft Research, Redmond, US

Hebrew University, Jerusalem, Israel

Tel Aviv University, Israel

CAV’05

Abstraction for VerificationAbstraction for Verification

• Goal: prove properties

• Sound abstraction for verification– properties of abstract system hold for

corresponding concrete system

: C A– if abstract state a satisfies property P then all

concrete states represented by a satisfy P

: C A a A if a P

then c C . (c)=a c P

: C A a A if a P

FalsificationFalsification

detect errors

• Sound abstraction for verification– errors of the abstract system exist in

: C A a A if a P

falsificationdetect errors

• Sound abstraction for verification– errors of the abstract system exist in

: C A a A if a P

falsificationdetect errors

c C . (c)=a c P

MotivationMotivation

• An abstraction that is sound for falsification need not be sound for verification.

• Existing frameworks for abstraction for verification – Modal Transition System (MTS)– MTS, PKS,KMTS - equivalent in expressive

power [ Godefroid,Jagadessan – VMCAI’03 ]

– can be too restrictive for falsification

Main ResultsMain Results

• New framework for abstraction – Ternary Modal Transition System (TMTS)– TMTS is stronger than MTS– Semantics of -calculus for TMTS

• Weak reachability– TMTS with parameterized transitions gives

tighter underapproximation– TMTS with assume-guarantee transitions for

complete reasoning

Modal Transition SystemsModal Transition Systems

underapproximation

overapproximation

Concrete Abstract

c. (c) = a c’ . (c’) = a’ c c’

MAY(a,a’)MAY(a,a’)

MUST+(a,a’)MUST+(a,a’)

MUSTMUST––(a,a’)(a,a’)

c, c’ . c c’ (c) = a (c’) = a’

(existential abstraction)

must may

underapproximation

c’. (c’) = a’ c. (c) = a c c’onto

[ T. Ball - FMCO’04 ]

must maymust+ and must– are incomparable

TMTS strictly more expressive than MTSTMTS strictly more expressive than MTS

MTS • may and must+ transitions• precision preorder is logically characterized by PML

::= p | AX | |

TMTS• may, must+ and must– transitions• precision preorder is logically characterized by full-PML

::= p | AX | AY | |

• full-PML is strictly more expressive than PML [Pinter,Wolper - PODC’84] [Kupferman,Pnueli - LICS’95]

TMTS: what does it buy us?TMTS: what does it buy us?

• Verifying specifications with past operators

• Reasoning about specifications in falsification setting– must+ for verification and must- for falsification

• Tighter weak reachability in abstract system – combine must+ and must- along the path

Semantics of Semantics of -calculus for TMTS-calculus for TMTS

: C A• (C, c1)

• [ (A, a1) ] - the value of the -calculus formula in state a1 of TMTS A

• [ (A, a) ] = T – for all concrete state c with (c) = a, (C, c)

• [ (A, a) ] = T – there exists a concrete state c with (c) = a and (C, c)

• [ (A, a) ] = F– for all concrete state c with (c) = a, (C, c)

• [ (A, a) ] = F

– there exists a concrete state c with (c) = a and (C, c) • [ (A, a) ] = M

– there exist concrete states c and c’ such that

(c) = (c’) = a and (C, c) and (C, c’) • [ (A, a) ] =

Information Information LatticeLattice

Truth Truth LatticeLattice

• [ (A, a) 1 2 ]

• [ (A, a) EX ]• [ (A, a) ]

[ (A, a) 1 2 ] =

[ (A, a) 1 ] # [ (A, a) 2 ]

6-valued Semantics of 6-valued Semantics of 11 22

# F F M T T

F F F F F F F

M F F ? F M F

T F F F ? T

T F F M ? T

# F F M T T

F F F F F F F

M F F ? F M F

T F F F ? T

T F F M T T

# F F M T T

F F F F F F F

M F F ? F M F

T F F F T

T F F M T T

# F F M T T

F F F F F F F

M F F ? F M F

T F F F T

T F F M T T

# F F M T T

F F F F F F F

M F F F F M F

T F F F T

T F F M T T

[ (A, a) EX ] =

Semantics of EXSemantics of EX

F if for all a’, if may(a,a’) then [(A, a’) ] = F

T if exists a’ s.t. must+(a,a’) and [(A,a’) ] = T

T if exists a’ s.t. must–(a,a’) and [(A,a’) ] T

otherwise

a EX = T

must–

• [ (A, a) EX ] = T

• exists a’ s.t. must–(a,a’) and [(A,a’) ] = T

• exists c’ such that (c’)=a’ and c’ • for all c’ with (c’)=a’ there is c

with (c)=a such that cc’

if [ (A, a) EX ] = T then there exists c with (c) = a and c EX

Semantics of Semantics of

• The semantics of PML operators is monotonic

– Least fixpoint operator can be computed by iterations from F is the usual way:

– [(A,a) Z . (Z) ] = [ (A, a) *(F) ]

• The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS

• [(A,a) ] = – 3-valued abstraction refinement of must+ transitions

[Shoham,Grumberg – CAV’03] adapt for must-

• Hypermust transitions– [Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04]– adapt for must– – MTS with hypermust+ is incomparable with TMTS

EX(x>6) T EX(x>6) F EX(x>6) = T

EX(x>6) = ?

must –

x = 7x = 10

x:=x–3

7 8 9 ...

• The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS

• [(A,a) ] = – 3-valued abstraction refinement of must+ transitions

[Shoham,Grumberg – CAV’03] adapt for must-

• Hypermust transitions– [Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04]– adapt for must– – MTS with hypermust+ is incomparable with TMTS

Weak ReachabilityWeak Reachability

• a’ is weakly-reachable from a c, c’ . (c)=a (c’)=a’ c * c’

c’ a’

ainitial state

error state

error trace

Related to testing

L1: TF L0: FT L0: FF

L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

Predicates: (x < 6) (x > 7)

ExampleExample

L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

ExampleExample

Underapproximation of Underapproximation of Weak ReachabilityWeak Reachability

• if [must+]*(a,a’) then a’ is weakly reachable from a

• Arbitrary combinations of must+ and must– transitions do not preserve weak reachability

• Find a tighter underapproximation of weak-reachability

L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

ExampleExample

must – ?must + ?

Underapproximation of Underapproximation of Weak ReachabilityWeak Reachability

• if [must+]*(a,a’) then a’ is weakly reachable from a

• Arbitrary combinations of must+ and must– transitions do not preserve weak reachability

• Find a tighter underapproximation of weak-reachability

ObservationsObservations

• a3 is weakly reachable from a1

if there exists a2 such that

must–(a1,a2) and must+(a2,a3)

• Onto nature of must– is preserved by [must-]*

• Total nature of must+ is preserved by [must+]*

must–

[T.Ball – FMCO’04]

UnderapproximationUnderapproximation

If there exists a1, a2, a3 such that

[must–]*(a1,a2) and

[must+]*(a2,a3)

then a3 is weakly-reachable from a1

[must+]*

[must–]*

[T.Ball – FMCO’04]

L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

ExampleExample

( total from a? )MUST+ ?MUST+ ?

( onto a’ ?)MUSTMUST– – ??

MAYMAY

Parameterized TransitionsParameterized Transitions

must+()

total from

c. (c) = a c c’ . (c’) = a’ c c’

MUST+(MUST+())

Parameterized TransitionsParameterized Transitions

must–()

MUSTMUST–(–())

c’. (c’) = a’ c’ c. (c) = a c c’

if is TRUE then must+() is must+ and must–() is must–

ObservationObservation

– must–(1)(a1,a2)

– must+(2) (a2,a3)

– 1 2 a2 is satisfiable

must+(2)

must–(1)

ObservationObservation

– must–(1)(a1,a2)

– must+(2) (a2,a3)

– 1 2 a2 is satisfiable

• Strongest parameters 1 and 2

must–(1)

must+(2)

MUST+ ( WP(s,a’) )MUST+ ( WP(s,a’) )

Strongest ParametersStrongest Parameters

Generated automatically as part of the construction of TMTS

c. (c) = a c c’ . (c’) = a’ c c’

if must+() then a ( WP(s,a’))

MUSTMUST– – ( SP (s,a) )( SP (s,a) )

c’. (c’) = a’ c’ c. (c) = a c c’

if must–() then a ( SP(s,a))

L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

ExampleExample

SP(x:=x+3, x<6) = x < 9

WP(x:=x-3, x<6) = x < 9

L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

ExampleExample

SP(x:=x+3, x<6) = x < 9

WP(x:=x-3, x<6) = x < 9

must–(x<9)

must+(x<9)

must– (x < 9)

must+ (x < 9)

Tighter UnderapproximationTighter Underapproximation

If there exists a1,...,a5 s.t.

[must–]*(a1,a2)

must–(1)(a2,a3)

must+(2) (a3,a4)

[must+]*(a4,a5)

1 2 a3 is satisfiable

then a5 is weakly-reachable from a1

must+(2)

must–(1)

[must+]*

[must–]*

Complete Reasoning Complete Reasoning

– a’ is reachable by a certain sequence of abstract transitions from a

– a’ is weakly-reachable from a

• Assume-guarantee transitions– another type of parameterized transitions:

<> must+ <’>

<>must+<‘ > c. (c) = a c

c’ . (c’) = a’ c’ ’ c c’

< < > MUST+ > MUST+ < < ’ ’ >>

Assume-Guarantee TransitionsAssume-Guarantee Transitions

Which and ’ predicates do we need?

c’. (c’) = a’ c’ ’

c . (c) = a c c c’

< < > MUST> MUST–– < < ’ > ’ >

<>must–<‘ >

The idea...The idea...

is satisfiable

<1>must– <2>

<2>must– <3>

1 = a1

2 = SP(s1, 1) a2

3 = SP(s2, 2) a3

<4>must+ < 5>

<3>must+ < 4>

3 = WP(s3,4) a3

4 = WP(s4,5) a4

5 = a5

Assume-guarantee transitionsAssume-guarantee transitions

• Complete Reasoning about Weak Reachability– a’ is reachable by a certain sequence of

assume-guarantee transitions from a– a’ is weakly-reachable from a

• Finding right parameters ~ computing loop invariants

Weak Reachability: SummaryWeak Reachability: Summary

[must–] * [must+]*must–(1) must+(2)

[must–] * [must+]*

• Previous work [T.Ball – FMCO’04]:

• Parameterized transitions

• Assume-guarantee transitions – complete reasoning

ApplicationsApplications

• Falsification of properties in CTL, LTL

• Abstraction-guided test generation– tighter underapproximation of weakly-

reachable states improves coverage of the generated tests

– example of QuickSort’s partition function

SummarySummary

• Ternary Modal Transition System (TMTS)– onto and total must transitions– full-PML logical characterizes precision

preorder on TMTS

• 6-valued semantics of -calculus for TMTS

• Tighten underapproximation of weak reachability with parameterized transitions– completeness result using assume-guarantee

transitions

abstraction for falsification

Documents

phy abstraction for lte

using gps to detect and prevent falsification

trotsky: the stalin school of falsification - index · leon...

medicine falsification

falsification of documents case study

falsification of medicines - bsu.by

abstraction elimination for kappa calculus

five classes of falsification

rudder_the falsification fallacy.pdf

a critique on the falsification of free

navy submarine em/tempest falsification facility

1 abstraction (cont’d) defining an abstract domain...

abstraction how much abstraction ? abstractionrealism

technical solutions to address falsification of unique

falsification - langara college

a global falsification of history

popper falsification

any omission, falsification, misstatement or ... · any...

the dark matter crisis: falsification of the current...

confirmation, falsification, and...