ieee transactions on computers, vol. 56, no. 4, april 2007 p roperties i ncompleteness e valuation...

1

IEEE TRANSACTIONS ON COMPUTERS, VOL. 56, NO. 4, APRIL 2007

PROPERTIES INCOMPLETENESS EVALUATION BY FUNCTIONAL VERIFICATION

2MAIN

CONTRIBUTION

A coverage methodology based on a combination of static and dynamic verification that allows us to reduce the evaluation time with respect to pure formal approaches

3 INTRODUCTION

Simulation-based techniques

Lack of exhaustiveness

Formal verification

Overcome the exhaustiveness problem

Properties are derived from informal design specifications.

Model checking: prove the presence of bugs, but not their absence

4VERIFICATION FLOW BASED

ON MODEL CHECKING

5INTRODUCTION – MODEL CHECKING

To increase the effectiveness of model checking

Vacuity detection: look for properties that hold in a model and can be strengthened

without causing them to fail

Property coverage: address the question of whether enough properties have been

defined

How many properties should be defined to completely check the implementation? Coverage metric!

6

Mutation-based

ACTL, LTL, and CTL

State coverage path coverage transition-

based coverage

Implementation-based

State explosion problem

Cannot precisely reflect the

completeness of properties

INTRODUCTION – PREVIOUS WORK

•How about use mutation coverage jointly with dynamic verification to address the quality of the model checking process?

7 BACKGROUND

Kripke structure K = {S, S0 , R, L}

FSM M = {I, O, S, s0 , R}

Product machine MP = M1 XP M2

Retroactive network

Ιε

8METHODOLOGY

OVERVIEW

9GENERATION OF FAULTY

IMPLEMENTATIONS

The proposed methodology is independent of the adopted fault model Different fault models can provide different estimations of

the property completeness

Functional fault model Bit coverage has been proved to be related to design errors

Bit coverage fault model assumptions

Bit failure: stuck-at 0 or stuck-at 1

Condition failure: stuck-at true or stuck-at false

Single fault: A faulty implementation is generated for each fault

10GENERATION OF FAULTY

IMPLEMENTATIONS(CONT.) Detectable faults

11GENERATION OF FAULTY

IMPLEMENTATIONS(CONT.) A non-optimized algorithm

If fail then f is ε-detectable

Time-consuming and very likely state explosion

12ESTIMATION OF GOLDEN

MODEL INCOMPLETENESS

Ƥ-detectable and Ƥ-det

Property coverage

13

ESTIMATION OF GOLDEN MODEL

INCOMPLETENESS(CONT.)

CP = 1 formal properties are complete w.r.t. a particular fault model

Non-optimized algorithm

14

ESTIMATION OF GOLDEN MODEL

INCOMPLETENESS(CONT.)

Witnesses and counterexamples Tools can provide witnesses and counterexamples for

CTL and LTL properties

Input witness and input counterexample

15 WITNESS COVERAGE

Property coverage can be estimated by using input witnesses

Under some conditions, CP = Cw

16 PROOF OF CP = CW

Consider the safety and liveness properties separately

17 PROOF OF CP = CW (CONT.)

18 PROOF OF CP = CW (CONT.)

19INCREMENTAL PROPERTY

COVERAGE COMPUTATION

20COVERAGE ACCURACY

COMPARISON

Combining static and dynamic verification makes this methodology can deal with real industrial circuits.

The methodology presented in this paper covers faults rather than states.

Can estimate coverage more accurate (compare with previous works)

21EXPERIMENTAL

RESULTS

22INSPIRE TO IC/CAD

CONTEST

Functional fault model

Estimate coverage by fault instead of properties