the yogi project software property checking via static analysis and testing aditya v. nori, sriram...

The Yogi ProjectSoftware property checking via static analysis

and testing

Aditya V. Nori, Sriram K. Rajamani, Sai Deep Tetali, Aditya V. Thakur

Microsoft Research India

Synergizing Testingand Static Analysis

Idea: Combine testing and static analysis

through a “synergy” of both

New algorithms (Synergy, Dash, SMASH) to do scalable proofs using testing

Industrial strength tool (3 person years)

Synergy: Gulavani, Henzinger, Kannan, N., Rajamani, FSE 2006 Dash: Beckman, N., Rajamani, Simmons, ISSTA 2008SMASH: Godefroid, N., Rajamani, Tetali, MSR-TR-2009

The problemQuestionIs error() unreachable for all possible inputs?

Input

Testing: finds bugs, but can’t prove their absenceStatic analysis: can prove the absence of bugs, but can result in false errors

a↦true b↦falselimit↦2×

Testing×

a↦true b↦falselimit↦2i↦0lock↦1x=0y=0


Testing×

×

×

×

×

×

×

×

×

××

×

×


Testing×

×

×

×

×

×

×

×

×

××

×

×

test

Abstraction1

2:ρ×

×

s1s2

Abstraction 0

23 4

5

1

9

6 78

11 10

Proof

2:ρ

0

2: ¬ρ3:ρ 4: ¬ρ

5: ¬ρ

1:ρ

9:ρ

6:ρ 7: ¬ρ8: ¬ρ

11 10

9: ¬ρ

1: ¬ρ

4:ρ3: ¬ρ5:ρ

7:ρ6::¬ρ8:ρ

ρ = (lock != 1)

Key Ideas

Frontier: Boundary between tested and untested regions

WPα: New refinement operation that does not depend on whole program alias information

01234

789

×

××

××

××

× ×

56××

××

10

frontier

Key IdeasStep 1: Try to generate a

test that crosses the frontier– Perform symbolic

simulation on the path until the frontier and generate a constraint 1

– Conjoin with the condition 2 needed to cross frontier

– Is 12 satisfiable?

01234

789

×

××

××

××

× ×

56××

××

10

1

2

frontier

Key Ideas01234

789

×

××

××

××

× ×

56××

××

10

frontier

Step 1: Try to generate a test that crosses the frontier– Perform symbolic


– Conjoin with the condition needed to cross frontier 2

– Is 12 satisfiable? [YES]

Step 2: run the test and extend the frontier

×

×

×





– Is 12 satisfiable? [NO]

01234

789

×

××

××

××

× ×

56××

××

10

1

2

frontier





– Is 12 satisfiable? [NO]

Step 2: use WPα to refine so that the frontier moves back!

0123

4:¬ρ789

×

××

××

××

× ×

56××

××

10

frontier

4:ρ

DASH: Proofs from Tests– Algorithm uses only test case generation

operations– Maintains two data structures:

• A forest of reachable concrete states (tests)– Under-approximates executions of the

program• A region graph (an abstraction)

– Over-approximates all executions of the program

– Our goal: bug finding and proving• If a test reaches an error, we have found

bug• If we refine the abstraction so that there is

*no* path from the initial region to error region, we have a proof

– Handles the richness of C• New operator WPα uses only aliases α that

are present along concrete tests that are executed

• Algorithm uses recursive invocations to handle inter-procedural analysis

Can extend test beyond frontier?

Refine abstraction

Construct initial abstractionConstruct random tests

Test succeeded? Bug!

Abstractionsucceeded?

τ = error path in abstraction f = frontier of error path

yes

no

yes

no

Proof! yes

no

Input:Program P

Property ψ

The DASH Algorithm

Example


Refine abstraction





yes

no

yes

no

Proof! yes

no

Input:Program P

Property ψ

τ=(0,1,2,3,4,7,8,9)

Example

Symbolic execution +

Theorem proving

frontier


Refine abstraction





yes

no

yes

no

Proof! yes

no

Input:Program P

Property ψ

01234

56

789

×

× ×

× ×

× ×

× ×

×

×

× ×

×

10×

y = 1

Refinement 01234

56

789

×

× ×

× ×

× ×

× ×

×

×

× ×

×

10×

8:¬ρ 8:ρ9

89

ρ= (lock.state != L)

× ×refine

Refinement

8:¬ρ 8:ρ9

89

ρ= (lock.state != L)

× ×

01234

56

78 :¬ρ

9

×

× ×

× ×

× ×

× ×

×

×

× ×

×

10×

8:ρ

refine

Example

τ=(0,1,2,3,4,7,<8,p>,9)

01234

56

78 :¬ρ

9

×

× ×

× ×

× ×

× ×

×

×

× ×

×

10×

8:ρ Can extend test beyond frontier?

Refine abstraction





yes

no

yes

no

Proof! yes

no

Input:Program P

Property ψ

frontier

Correct, the program is0123

4⋀¬s5⋀¬s6⋀¬r

9

×

× ×

× ×

× ×

××

×

×

7⋀¬q×

8⋀¬p×

4⋀s5⋀s6⋀r7⋀q

8⋀p×

10

Handling procedure calls

Key ideaPerform a recursive Dash query on the called procedure and usethe result to either generate a test or compute WPα

Sk-1

Sk

×

Sk-2 T×

CALL(foo(i, j)) frontier

Interprocedural analysis

Sk-1

Sk

×

Sk-2 T×

CALL(foo(i, j)) Dash[assume(φ1), foo(i, j), assert(¬φ2)]- pass: perform refinement- fail: generate test

12

Correctness• Theorem. If Dash terminates on (P, φ), then

either of the following is true:– If Dash returns (“pass”, Σ≃), then Σ≃ is a proof that P

cannot reach ¬φ– If Dash returns (“fail”, t), then t certifies that P reaches ¬φ

• Theorem. Every Dash iteration entails exactly one theorem prover call

• However … Dash need not terminate! Thus, usefulness is based on empirical evidence

Locked

do { //get the write lock

KeAcquireSpinLock(&devExt->writeListLock);nPacketsOld = nPackets; request = devExt->WriteListHeadVa;

if(request && request->status){devExt->WriteListHeadVa = request->Next;

KeReleaseSpinLock(&devExt->writeListLock);

irp = request->irp;if(request->status > 0){

irp->IoStatus.Status = STATUS_SUCCESS;irp->IoStatus.Information = request->Status;}

else{irp->IoStatus.Status = STATUS_UNSUCCESSFUL;irp->IoStatus.Information = request->Status;

}SmartDevFreeBlock(request);IoCompleteRequest(irp, IO_NO_INCREMENT);nPackets++;

}} while (nPackets != nPacketsOld);

KeReleaseSpinLock(&devExt->writeListLock);

LL

Windows Device Drivers

Unlocked

Error

UU

Source Code

TestingDevelopment

API Usage Rules(SLIC)

Software Model Checking

Read forunderstanding

New API rules

Drive testingtools

Defects

100% pathcoverage

Rules

Static Driver Verifier

Architecture of Yogi

Yogi IR (yir)

C Program slamcl

Slam.li database

sliccSLIC property

Instrumented Slam.li database

li2yir

Test case that exposes a

bug

Proof that program satisfies property

Alias and mod-ref information

Z3 theorem prover

YAbsManYSim

polymorphic

region graphs

initialfunctions

summaries

Implementation

• ~6K lines of F#• Z3 theorem prover• Integrated with SDV for Windows• Engineering effort: 3 person years

Static Driver Verifier Runs• #drivers = 69, #properties = 85• largest driver = ~30 KLOCs• #KLOCs = ~350 KLOCS• Yogi works on 129 runs where SLAM “gives up”• Yogi takes ~12 hours whereas SLAM takes ~42 hours

Thanks to …

• Nels Beckman (CMU)• Bhargav Gulavani (IIT

Bombay)• Thomas Henzinger

(EPFL)• Yamini Kannan

(Berkeley)• Robert Simmons

(CMU)

• Leonardo de Moura (MSR Redmond)

• Nikolaj Bjorner (MSR Redmond)

http://research.microsoft.com/yogi

the yogi project software property checking via static analysis and testing aditya v. nori, sriram...

Documents