managing verification data with the unified coverage ...mentorgraphics.pdfunified coverage format...

Darron May

Managing Verification Data with the Unified Coverage

Interoperability Standard

Product Manager (VM & Coverage)Product Manager (VM & Coverage)

DVT

January 2014

Agenda

� Verification Management— Importance of Data Management

� UCIS Overview

2© 2013 Mentor Graphics Corp. Company Confidential

www.mentor.com

� UCIS Overview— Quick look at the database and API

� Solutions emerging— Enabling the sharing of data

Typical VM Requirements

BuildDesign Spec

� Process Management— Automated closed loop— Full Visibility— Turn-around time

� Data Management— Data Overload, capacity


www.mentor.com

Verify

Analyze

Testplan

Refine

— Data Overload, capacity— Handling complex relationships— Immediate & Historical Analysis

� Tool Management— Multiple Verification methods— Throughput— Repeatability, Control &

Automation

Tests

Rank

Testplans

Debug

System

Block

Data Management Infrastructure

� In 2005, Mentor saw need for solution— Differing types and producers— Volume size, design size— Storage, handling & visibility

� Unified Coverage DataBase— Data unification— Optimized for …… Unified Unified Unified


www.mentor.com

Rank

Merge

Queries

AnalysisReports

Assertions

Coverage

— Optimized for ……– Capacity– Performance– Platform Independence

— Test analysis— Native Testplan tracking

� Developed with Verification Management vision— Open Read/Write API— Donated to UCIS in 2008, selected 2009

Unified

Coverage

DataBase

Unified

Coverage

DataBase

Unified

Coverage

DataBase

Unified Coverage Format

Test Records and History Section

TESTNAME MYTEST#4

DATE 20091122120000

CPUTIME 324.3

USER darronm

TESTSTATUS 2

SEED 20203

VSIMARGS +acc=af –assert&.

HOSTNAME debian

HOSTOS linux

RUNCWD /work/SCRATCH

&&&&&&&&&&&&..

MYATTR NoDataPresent

TESTNAME MYTEST#3

DATE 20091122120000

CPUTIME 324.3

USER darronm

TESTSTATUS 2

SEED 20203


HOSTNAME debian

HOSTOS linux


&&&&&&&&&&&&..

TESTNAME MYTEST#2

DATE 20091122120000

CPUTIME 324.3

USER darronm

TESTSTATUS 2

SEED 20203


HOSTNAME debian

HOSTOS linux

TESTNAME MYTEST#1

DATE 20091122120000

CPUTIME 324.3

USER darronm

TESTSTATUS 2

SEED 20203


HOSTNAME debian

Testbench

Design#1

Design#2

INST#1

Receiver

DU#2

DU#1

Design & Coverage Section

2 Receiver

2.1 Mode 1

2.2 Mode 2

2.2.1 Shutdown

2.2.1.1 Power-up

2.2.1.1.1 Polling

1 Transmitter

Testplans Section

Description

Manager

Priority

Description

Manager

Priority

Description

Manager

Priority

Scopes


www.mentor.com

&&&&&&&&&&&&..


HOSTOS linux


&&&&&&&&&&&&..


HOSTNAME debian

HOSTOS linux


&&&&&&&&&&&&..


ORGFILE coverage1.ucdb

COMMAND vcover merge &

RUNCWD /work/SCRATCHORGFILE testplan.ucdb

COMMAND xml2ucdb –format

RUNCWD /work/SCRATCHORGFILE tp_coverage.ucdb

COMMAND vcover merge &


FSM

States

State1Bin

State2Bin

Transitions

Assertion#1

PassBin

FailBin

2.2.1.1.1 Polling

3 Register Tests

3.1 Reading

3.2 Writing

Description

Manager

Priority

Description

Manager

Priority

Questa Native Format

Testplan Traceability

Test Association with Coverage

Merge &Import

Scopes

Bins

Bins

Test Attributes

Rank

Testplan import

HTML

Diff Tool

History

� Binary database can be accessed by either API

� Comprehensive Command Line Interface based on API— Merge/Rank

Core Data Utilities, Native UCIS support


www.mentor.com

Rank

Merge

Coverage Queries

Create & edit

Text Reports

Exclude

Code Coverage Analysis

UCDB

UCISDB

UCDB

UCISDB

UCDB

UCISDB

— Merge/Rank— Text/HTML/XML reports— Coverage Query— Exclude/tag— Attribute/Test access— Create/edit

� Verification Management— Process and Tool

Management Features

Questa Verification Management

Automation

Testplan Tracking

Trend Analysis

� Process Management— Automated closed loop— Immediate & Historical Analysis— Turn-around time

� Data Management— Performance & capacity


www.mentor.com

Core Utilities

Results Analysis

UCDB

UCISDB

UCDB

UCISDB

UCDB

UCISDB

— Performance & capacity— Unified Coverage

� Tool Management— Multiple Verification methods— Throughput efficiency— Repeatability, Control &

Automation

Accellera and the UCIS

� UCIS Technical Sub-Committee— Formed Nov 2006, Chairman Richard Ho, D.E. Shaw— Accellera Policy, transfer standards to the IEEE

– Verilog 1995/2001, SDF, PDEF, SPEF– DCL, ALF, Verilog/VHDL Synthesizable Subset– VHDL/VITAL

— Approx 130 members, 18 member Companies


www.mentor.com

— Approx 130 members, 18 member Companies— 2013 Chairman, Alan Hunter, ARM

� Initial TSC goals— Identify interoperability opportunities— Define standard coverage models— Define an operability standard for exchange— Encourage user and EDA technology advancement

UCISDB and the UCIS API

� UCISDB 1.0 Overview, released June 2012— Generic database, scopes, bins, attributes, flags— Exclusions, test association, tagging— Defined coverage data models— Open API with extendable capabilities— Streaming and in-memory access modes— Implementations not binary compatible


www.mentor.com

— Implementations not binary compatible

� Original UCDB routines 60% unchanged ucdb_ to ucis_— Change to iterators— Introduced API optimizations to property interface— Additions

– Universal Object Recognition (recommendation)– Unique IDs (recommendation)– Formal coverage extension, data & new routines – An XML interchange format (recommendation)

— Excluded testplan scope, total and test coverage calculations

SV Functional Coverage Example

Standard SV

covergroup

Generates UCIS

Covergroup Scope

Generates UCIS

Coverpoint Scopes

� Viewing using in-memory access to UCISBD (UCDB)


www.mentor.com

Standard SV

coverpoint

Standard

SV bins

Standard UCIS

bins or coveritem

Weights, goals,

attributes

Bin

counts

In-Memory API Access to data

� UCISDB to CSV creation— Application has random access— Use of Iterators— Performance trade-off


www.mentor.com

Open

database Data that will

be extractedCall user

routine on DB




www.mentor.com

Iterate over

hierarchyTest for

required objects

Recursive call

of routine




www.mentor.com

Data that will

be extractedCall user

routine on DB

Iterate over

hierarchyTest for

required objects

Recursive call

of routine

Streaming API Access to data

� UCISDB to CSV creation— Set-up Call-backs— Stream all objects— Memory efficient for reporting


www.mentor.com

Open

database

Setup callback

routine



Callback

reason


www.mentor.com

reason

Called on

each scope

Called on each

coveritem




www.mentor.com

Open

database

Creating and running the application

ucis.h

ucis2csv.c


www.mentor.com

gcc -lz -lm -ldl ucis2csv.c -o ucis2csv $(QUESTA)/linux_x86_64/libucis.a -I $(QUESTA)/include

ucis2csv test.ucdb > test.csv

Changing the vendor

ucis.h

ucis2csv.c


www.mentor.com

gcc -lz -lm -ldl ucis2csv.c -o ucis2csv $(VENDB)/linux_x86_64/shared.a -I $(VENDB)/include

ucis2csv test.ucdb > test.csv

UCIS Tool/Data Landscape

Tool 1

UCIS API

UCISDBUCIS

API

Tool 4

UCIS API

UCISDB

UCIS

API


www.mentor.com

UCIS

API

UCIS

API

Tool 2

UCIS XMLSchema

UCIS XML

XM

L tools

Identical schema models by

standardizationTool 3

customdata

format

Cust

om

API?

Data can be exchanged via an app

Tool 1

UCIS API

UCISDBUCIS

API

Tool 4

UCIS API

UCISDB

UCIS

API

COMPRESSXML

UCIS

2XM

L

XM

L2UCIS


www.mentor.com

UCIS

API

UCIS

API

XML

UCIS

2XM

L

XM

L2UCIS

ucis2xml.c

donation

Identical schema models by

standardization� Exchange Data

� Compressed XML

� Data Manipulation

UCDB enabling verification solutions

Testplan tracking

� Multiple verification engines

� Requirements Traceability

� Common management

Build

Design Spec


www.mentor.com

UCDB API

UCIS API

CLI

EmulationADMS/SpicePower AwareFormalCDC

3rd Party

Verify

Analyze

Refine

Testplan trackingTrendingResults AnalysisRun Management

Testplan

UCIS API

Questa Code

Coverage

Simulation

Write Write

Traditional Manual Code Coverage

Closure

Enabling use of complimentary technologies

-12+14

Covered


www.mentor.com

82.79%UCIS API

Review

Coverage

Holes

Write

Exclusions

Uncoverable

Write

Tests

Coverable

Covered

Objects

UCIS API

Questa Code

Coverage

Simulation

Enabling use of complimentary technologies

Auto-Generate

Write

Tests

Identical front-ends


www.mentor.com

UCIS API

Uncoverable

Auto-Generate

Exclusions

Review Results

Coverable

View

WaveformsQuesta

CoverCheck

Configuration

front-ends100% Match

Save Man Months on Coverage Closure

Summary

� Mentor committed to standards

� UCIS is enabling solutions


www.mentor.com

� UCIS is enabling solutions

� Questa VM delivers data, process and tool management


www.mentor.com

w w w . m e n t o r . c o m

2DKM, Verification Management

managing verification data with the unified coverage ...mentorgraphics.pdfunified coverage format...

Documents