m9 freescale - test and verification

TM

November 2013

TM 2

MotivationSetupActivationDetectionCompletion Conclusion

TM 3


TM 4

• Code and toggle coverage provides incomplete verification closure information

• Missing tool to perform an incoming VIP acceptance test

• Number of test cases along with runtime at SoC level causes workflow bottleneck

• Time to tape out gets shorter and requires efficient flow setup

VIP = Verification IP

TM 5


TM 6

• Extraction of information from the design system into the tool setup files

• Design gets compiled for the Activation Phase of the Tool

• Compilation of the design for the Detection Phase of Certitude

Environment Setup

Setup Activate

Setup Detect

TM 7

• Extraction of information from the design system into the tool setup files− Regression test cases and test category gets

converted into certitude_testcase.cer

− Design and test bench files get written into certitude_hdl_file.cer

− Initial certitude configuration file gets created certitude_config.cer

− Certitude regression execution files get created � batch_script.sh

� certitude_compile

� certitude_execute

Environment Setup

TM 8





− Certitude regression execution files gets created � batch_script.sh



Environment Setup

TM 9








Environment Setup

addtestcase -testcasegroup=BATS \-testcase=ns_ram@TOP_BATS_bats_stop.tcm

addtestcase -testcasegroup=BATS \-testcase=ss_ram@TOP_BATS_bats_ssctest.tcm

addtestcase -testcasegroup=GDU \-testcase=ns_ram@TOP_GDU_gdu_register_stim.tcm

addtestcase -testcasegroup=GDU \-testcase=ns_ram@TOP_GDU_gdu_amp0_stim.tcm

TM 10








Environment Setup

addsystemverilog -qualify -file=/top/rtl_v/top.svaddsystemverilog -file=/block_a/rtl_v/block_a.svaddverilog -file=/block_b/rtl_v/block_b.vaddverilog -file=/block_c/rtl_v/block_c.v

block_b o2

i5

i6

i7

TM 11








Environment Setup

setconfig -TopName=s12zvm_mcusetconfig -Simulator=vcssetconfig -FaultSet=Connectivitysetconfig -ConnectivityTopName=[list s12zvm_mcu s12 zvm_core]

setconfig -MaxJobsInParallel=40setconfig -ReentrantTests=false

setconfig -XLIWatchdogTimeout=10000setconfig -DetectionAutoStop=false

setconfig -HeartBeatPeriod=10000

setconfig -BatchCommand=./batch_script.sh

TM 12


TM 13

• This step in the flow is similar to run toggle coverage on the design

• Goal is to get 100% analyzed activation coverage

cer> activate –verbose

cer> resettestcase –testcaselist=[list … ]

cer> activate –testcaselist=[list …]

TM 14


TM 15

• Faults from the activated fault list get inserted

• Goal is to get to 100% analyzed detection

Same as Activation

Different approachespossible

TM 16

Faults activated by pattern suite of module block_a

Faults activated

by pattern suite of module block_b

Faults for module instance block_b

Faults for module instance block_a

Running Certitude on SoC level differs from module level

• Runtime of test cases

• Number of faults

• Number of test cases

• Test case do activate a lot of faults,but do not check them

TM 17

• Brute Force method− Running all test cases on all faults

0

1

2

3

4

5

6

7

45 75 112 151 193 234 273 314 355 394 435 493 533 574 614 643 685 728 770 814 865 940 972

Detected Faults [%]

Detected Faults [%]

cer> detect –verbose

2332 faults considered by Certitude; 232 test cases available

TM 18

• Instance Scope− Running subset of pattern on one

instance

0

20

40

60

80

100

15 30 45 60 75 90 105 120

[min.]

Detected Faults [%]

Environment Setup

block_b o2

i5

i6

i7

cer> detect -verbose faulttestcasepairlist=[create_testcase_pair_list_for_instance work.s12zvm_core.SPI SPI0]

TM 19

# Create Testcasepair list for instanceproc create_testcase_pair_list_for_instance {instan ce

testcase_group} {set testcase_list [testcaselist -testcasegroup=$tes tcase_group]set fault_list [faultlist -activatedby=$testcase_li st \

-connectivityoninstance=$instance]set fault_testcase_pair_list [list ]foreach fault $fault_list {

set fault_activation_list [faultinfo -fault=$fault -activatedby]# Use certitude "and" tcl command to calculate inte rsectionset fault_testcase_group_activation_list \

[intersection $testcase_list $fault_activation_list ]set fault_testcase_pair_list \

[lappend fault_testcase_pair_list \[list $fault

$fault_testcase_group_activation_list]]}return $fault_testcase_pair_list

}

TM 20

# Ensure testcases are part of the current activation setting as otherwise the tests won’t be usedcer> activate –testcaselist=[testcaselist –testcase group=block_b]

# Configure relevant faults (module specific) cer> faultdisable -allcer> faultenable -faultlist=[faultlist –unitname=bl ock_b]

# Configure relevant test cases (module specific)cer> testcasedisable -allcer> testcaseenable -testcaselist=[testcaselist

–testcasegroup=block_b]

cer> detect -verbose

Supported commands since 2012

TM 21

• Signal scope− Detection granularity at signal level

• Further guidance for Certitude to use the best suited test case to qualify certain signal faults− Example would be a qualification of a late ECO change

ECO = engineering change order

#MapPortToTestcase <module> <signal> <testcases>MapPortToTestcase work.block_b i7 reg_test_block_b

TM 22

MotivationSetupActivationDetectionCompletionConclusion

TM 23

• Goal is to get the report to 100% analyzed

• Waiving can be done in GUI− Can not be captured in a file that

is under version control

• Waiving can be done through TCL commands

2991 Faults are injected in the design2332 Faults are considered for status185 Faults are disabled by Certitude20 Faults are disabled by user0 Faults are not yet qualified0 Faults are dropped

0 Faults are non-activated0 Faults are non-propagated

2127 Faults are detected0 Faults are non-detected

# OUTPUTS:# ahb_m0_hprot - NA : tied to zerofaultdisable -faultlist=[faultlist -subunitname=work.enet_macahb.ahb_m0_hprot -type=OutputPortStuckAt0]

TM 24


TM 25

• Keep test cases short and focused. Test not everything in one test case.

• Test cases should fail on first error. Testbench environment should be setup to terminate with an error on the first fail.Example: VCS can be configured to fail after a certain number of offending assertions:“-assert global_finish_maxfail=1”

• Test case timeouts should be set close to the actual passing test case runtime.

• Improve the detection probability;Interface assertions, bus monitors and checkers are helpful here.

TM 26

• With using Certitude in the verification Certitude found− Test cases that have accidently not been run

− Test cases had part of the code commented

− Problems in the verification infrastructure

− Test cases did not cover all promised faults

• Using the instance based detection concept leads to a reasonable turn around time

• The waiving of non-activated and non-detected faults is not as smooth as working with the DVE coverage waiving mechanism

m9 freescale - test and verification

Documents