stephan berg track f

All you need to know about chip design, testing & manufacturing

Israel, May 4, 2010 1

Using Algorithmic Test Generation to improve Functional Coverage in existing Verification Environments

Staffan BergVerification SpecialistMentor Graphics (Europe)


Israel, May 4, 2010

What’s the Problem?

2

Current Simulation Stimuli Generation Techniques Can’t Keep Up With Design

Complexity

Current Simulation Stimuli Generation Techniques Can’t Keep Up With Design

Complexity

How Can We Take Advantage of New Technology Without Breaking Existing

Flows/Environments?

How Can We Take Advantage of New Technology Without Breaking Existing

Flows/Environments?


Israel, May 4, 2010

Limitations of C-R Stimuli Generation

3

To Produce N unique Vectors C-R Requires

N * ln(N) Vectors

To Produce N unique Vectors C-R Requires

N * ln(N) Vectors

For many of today’s complex designs, it is Impossible to reach Coverage goals

within the project schedules

For many of today’s complex designs, it is Impossible to reach Coverage goals

within the project schedules


Israel, May 4, 2010

Current Test Generation Technology

• Explicitly describe every test procedurally• addr <= “0000”; wait 10; addr <= “0001”;

Directed Tests

• Implicitly define the stimuli space by declaration of Constraints

• Constraint c1 (addr inside {2, 4, [80:100]};

C-R Tests

• Explicitly Define the whole space by defining a Grammar through a set of Rules

• Transaction = (req read | write wait_ack);

Algorithmic Tests

4


Israel, May 4, 2010

Algorithmic Test Generation Basics

5

Rule_graph simple_engine {

action init;action wait_rdy, setup_rd, setup_wr, ack;action rw_1, rw_2, rw_4;

symbol rw_opts, rw_size;rw_opts = setup_rd | setup_wr;rw_size = rw_1 | rw_2 | rw_4;

simple_engine = init repeat { wait_rdy rw_opts rw_size ack }

}

Rule_graph simple_engine {

action init;action wait_rdy, setup_rd, setup_wr, ack;action rw_1, rw_2, rw_4;

symbol rw_opts, rw_size;rw_opts = setup_rd | setup_wr;rw_size = rw_1 | rw_2 | rw_4;

simple_engine = init repeat { wait_rdy rw_opts rw_size ack }

}


Israel, May 4, 2010

Real Example: AMBA AHB Master


Israel, May 4, 2010

Key Concepts

The Rules are compiled into an NDFSM (non-Deterministic Finite State Machine) Representation

Action Functions are written in Verilog, SystemVerilog, VHDL or C++

The Rule Graph is then traversed during simulation and

the Action functions are called to produce stimuli

Without coverage goals, the traversal will be random

The Rules are compiled into an NDFSM (non-Deterministic Finite State Machine) Representation

Action Functions are written in Verilog, SystemVerilog, VHDL or C++

The Rule Graph is then traversed during simulation and

the Action functions are called to produce stimuli

Without coverage goals, the traversal will be random

7


Israel, May 4, 20108

The Role of CoverageStimulus model describes valid inputsCoverage model describes interesting/priority stimulus

Directed TestsAccurate but low productivity

Difficult to produce enough tests

Constrained RandomAutomation boosts productivity

But, difficult to targetRedundancy slows coverage closureRequires manual analysis and constraining to close coverage

Algorithmic Test GenerationEliminates redundant stimulusEfficiently targets coverage modelProduces stimulus outside coverage model after coverage achieved


Israel, May 4, 2010

Using Coverage to drive Stimuli Generation

9

Path Coverage is used to define the Coverage goalsPath Coverage is used to define the Coverage goals

A single Path Coverage Object can cover all legal paths in a graph….

A single Path Coverage Object can cover all legal paths in a graph….

Or you could use multiple PC Objects to cover specific goals and cross products


Israel, May 4, 2010

ATG and OVM

10

DUTDUT

Fundamentals of OVM•Highly Modular Testbench Components•TLM-Based Communication•High Degree of Configurability & Re-Use

Fundamentals of OVM•Highly Modular Testbench Components•TLM-Based Communication•High Degree of Configurability & Re-Use

What Is OVM?•Open Verification Methodology•Joint Development by Cadence and Mentor•7400+ Registered Users

What Is OVM?•Open Verification Methodology•Joint Development by Cadence and Mentor•7400+ Registered Users


Israel, May 4, 2010

sequencer(simple_sequencer)

driver(simple_driver)

DUT

simple_seq_sub_seqs(sequence)

test_env

test

simple_item

sequencer(simple_sequencer)

driver(simple_sequencer)

DUT

test_env

test

simple_item

inFactScheduler

Sequence

(infact_simple_sequence)

Integration in Existing Testbench Environment: SV OVM

11

OVM Testbench (Partial)

Rules

Meta-Actions selects values from ranges or

sets

Meta-Actions selects values from ranges or

sets


Israel, May 4, 2010

Integration in Existing Testbench Environment: SV OVM

12

Declare Sequence

Item

Declare Sequence

Item

Create a Sequence

Item

Create a Sequence

Item

Assign ValuesAssign Values

Send Item to Sequencer

Send Item to Sequencer

Modify Environment

Modify Environment


Israel, May 4, 2010

Rule Graph

C objC obj

C apiC api

Sn_compile

‘e’ Unit or Struct

‘e’ Verify Unit

graph_1 : infact_e_comp is instance; graph_1 : infact_e_comp is instance;

Verify_1() @posedge_clk is{ var req: trans_struct = new; while (TRUE) do { graph_1.fill_item(req);

Verify_1() @posedge_clk is{ var req: trans_struct = new; while (TRUE) do { graph_1.fill_item(req);

Integration in Existing Environment: ‘e’ Testbench

13

‘e’ struct


Israel, May 4, 2010

‘e’ Integration Specifics

14

Testengine is UntimedThe Action Functions can be Completely

Auto-Generated‘e’ Enumerated Types are mapped to Action

Functions:type instruction_t : [add, sub, mult];

Minimal Changes to existing ‘e’ environment:

Testengine is UntimedThe Action Functions can be Completely

Auto-Generated‘e’ Enumerated Types are mapped to Action

Functions:type instruction_t : [add, sub, mult];

Minimal Changes to existing ‘e’ environment:


Israel, May 4, 2010

Case Study: Wireless Infrastructure

Complex Interface with 1000’s of configurations

Complex Interface with 1000’s of configurations

15

Complex ‘e’ testbench representing 100’s of man-years

Complex ‘e’ testbench representing 100’s of man-years

Impossible to achieve Coverage goals within reasonable time using C-R

Impossible to achieve Coverage goals within reasonable time using C-R

Seq. Driver(s) Action functionAction functionAction functions

TBuVC


Israel, May 4, 2010

Case Study: Wireless Infrastructure

16

inFact vs Specman coverage

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0

100000

200000

300000

400000

500000

600000

700000

800000

900000

Vectors

Co

vera

ge

inFact

Specman (igen)

Results: ATSG Achieved Coverage Goals in 1/10th of the simulation time*(less than 100K tests vs. 850K tests)

Results: ATSG Achieved Coverage Goals in 1/10th of the simulation time*(less than 100K tests vs. 850K tests)

*Using Algorithmic Test Generation in a Constrained Random Test EnvironmentHåkan Askdal DAC Conference 2009


Israel, May 4, 2010

Case Study: AXI Bus Bridge

Existing Design and Verification EnvironmentDUT OverviewParameterizable N-Channel Bus BridgeAXI bus control interfaceArbitrates requests from proprietary interface, performs splits or aggregation as appropriate, and generates AXI bus calls

Verification EnvironmentSimulator - VCSTestbench - SystemVerilogStimulus - Constrained Random

Current TestbenchGeneration of 20 Random VariablesInterdependencies described with constraintsCover-points on all variablesCrosses between select cover-points

ProprietaryI/F VIP

AXI Bus VIP

CoverageScoreboard

SequenceGenerator

Testbench

ParameterizableBus Bridge


AXI BusInterfaceAXI BusInterface

N RequestChannelsN RequestChannels

DUTDUT


Israel, May 4, 2010


Existing Verification Objectives

Functional Domain Space (White Column)•Too many legal combinations to simulate•Not all are important anyway

Coverage Domain Space (Green Column)•Reduced to manageable number•Number of crosses will be limited•But still too many combinations

Verification Goals•Run constrained random tests until

test conditions are achieved•Goal # 1 - Cover each value of each variable one time•Goal # 2 - Cover a cross of all combinations

of the bytes and addr variables

VariableFieldtransphysaddrid1id2

bytespri

wrapstartendseq1seq2

offset1offset2

rescachetype1ctrl1type2ctrl2

FunctionalDomain

52

2^36256256

65,5362222

2^322^321616424844

CoverageDomain

52

2556464776

2222

64641616424844


Israel, May 4, 2010

Verification Goal # 1•Cover each value of each variable one time•Total of 20 cover-bins•Largest cover-bin contains 776 cover-points•Total of 1360 cover-points

Verification Goal # 2•Cover a cross of all combinations of the bytes and addr

variables - minus a few select unimportant cases•bytes cover-bin contains 776 cover-points•addr cover-bin contains 255 cover-points•Total of 196,608 cross cover-points

VariableField

FunctionalDomain

CoverageDomain

transphysaddrid1id2

bytespri

wrapstartendseq1seq2

offset1offset2

rescachetype1ctrl1type2ctrl2

52

2^36256256

65,5362222

2^322^321616424844

52

2556464776

2222

64641616424844

13601360 #1

#2


Existing Verification Objectives


Israel, May 4, 2010


Results with Existing Testbench

Verification Results # 1•Total cover-points - 1360

•Largest cover-bin - 776•Coverage achieved - 100%

•Testcases required - 475,500

Verification Results # 2•bytes cover-bin - 776

•addr cover-bin - 255•Total crosses - 196,608

•Coverage achieved - 79%•Testcases required - 26,315,000

100%

Testcases

Co

vera

ge

0 475,5000

1360

CRT

0%

100%

Testcases

Co

vera

ge

0 26,315,0000

196,608

CRT

0%

79%


Israel, May 4, 2010


Updated Verification EnvironmentDesign Under Test OverviewNo Change

Verification EnvironmentSimulator - Replace VCS with Questa/inFactTestbench - Stay with SystemVerilogStimulus - Add Graph at the top level

New TestbenchAlmost No Change (see next slide)

ProprietaryI/F VIP

AXI Bus VIP

CoverageScoreboard

SequenceGenerator

Testbench



AXI BusInterfaceAXI BusInterface

N RequestChannelsN RequestChannels

DUTDUT


Israel, May 4, 2010


ATSG Testbench StepsGraph DevelopmentDescribe the domain of each variableDescribe constraints on variable relationshipsDescribe variables and combinations to cover

Graph IntegrationReplace call to SV “randomize( )” functionwith call to inFact “fill ( )” method

Testbench Development EffortTotal time required - less than 1 dayinFact code written - approximately 100 linesSystemVerilog code written - less than 10 linesReuse of existing testbench code - 99.9%


Israel, May 4, 2010


Coverage Closure Results Verification Results # 1Total cover-points - 1360Largest cover-bin - 776Coverage achieved - 100%Testcases required - 776Coverage closure acceleration - 612x

Verification Results # 2bytes cover-bin - 776addr cover-bin - 255Total crosses - 196,608Coverage achieved - 100%Testcases required - 196,608Coverage closure acceleration - >>170x

iTBA

100%

Testcases

Co

vera

ge

0 475,5000

1360

CRT

0%776

612 x

iTBA100%

Testcases

Co

vera

ge

0 26,315,0000

196,608

CRT

0%

79%

1.15%

196,608


Israel, May 4, 2010

Conclusions

24

ATG can significantly shorten Time-to-CoverageATG can significantly shorten Time-to-Coverage

ATG can be introduced while still preserving existing verification IP

ATG can be introduced while still preserving existing verification IP

More VerificationLess CyclesMore VerificationLess Cycles

THANK YOU!THANK YOU!

stephan berg track f

Education

testcases coverage

coverage atg

functional coverage

coverage goals

existing verification

r stimuli generation

algorithmic test generation

existing flowsenvironments