1

Test and Verification Solutions

Achieving ISO 26262

Compliance in Silicon

(And Beyond?)

Mike Bartley, CEO @ TVS

mike@testandverification.com

2 2 7 July 2008

Agenda

• Some background on

– asureSign

• ISO26262

– And the implication for requirements

• Advanced verification

– Requirements sign off is not so obvious

• Supporting hierarchy and cross-discipline

• And the supply chain?

3 3 7 July 2008

Background

• TVS was established in 2008

• The TVS headquarters are in Bristol, UK with – Offices in India, France and Germany

– 80 verification engineers and software testers engaged around the world

• TVS developed a verification management tool in 2010 • Aimed at verification data management

• Linking verification activities to requirements

4 4 7 July 2008

ISO 26262 and Requirements Management D

o w

n s t r e

a m

U p s t r e

a m

Intent to implement

Intent to verify

Stakeholder Requirements (Customers and internal)

Product Requirements

Safety Requirements

SOC and Module Specs

Verification & Test Plans

Proof of implementation

Verification & Test Results

Requirements

ISO 26262 Stipulates “The management of safety requirements includes managing requirements, obtaining agreement on the requirements, obtaining commitments from those implementing the requirements, and maintaining traceability.”

5 5 7 July 2008

Why do we need a new tool?

• Good tool support for requirements tracing – Doors, Reqtify, Enterprise Architect, Jira, …

• But users report limited support for testing

• What do they need for requirements signoff? – Capture the mapping of requirements

• to signoff activities (not just directed tests)

• ability to manage changes

– Automate • recording of tests results

• recording of the configuration data

• reporting of requirements signoff status

– Document the sign off of the requirements

6 6 7 July 2008

Functional Verification Trends Industry evolving its functional verification techniques

72%

72%

64%

69%

40%

48%

41%

37%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Functional coverage

Code coverage

Constrained-Random

Simulation

Assertions

2007

2010

Wilson Research Group and Mentor Graphics 2010 Functional Verification Study, Used with permission

19%

29%

0%

5%

10%

15%

20%

25%

30%

35%

2007 2010

2007

2010

Me

dia

n p

ea

k n

um

be

r o

f ve

rifi

ca

tio

n e

ng

ine

ers

The adoption of formal property

checking has grown by 53%

Listen to the

2012 survey

Harry Foster at

DVClub April

8th

7 7 7 July 2008

The mechanics of an advanced test bench

Driver Stimulus generator

Test Checker Functional

Coverage

Design

Under

Test

Monitor

assert

constraint

addr data Assertions

Coverage

Coverage

Active Passive Code Coverage

8 8 7 July 2008

What are the implications for Requirements Signoff?

• Just mapping a requirement to a directed test is NOT sufficient

• Requirements need to map to – Tests

• Directed

• Constrained random with a particular seed

– Coverage • Code, functional and assertion

– Checkers • Dynamic and Static

• Need to automate – Test pass and fail

– Coverage collection and reporting

– Checker pass and fail

• All linked to configuration management data

9 9 7 July 2008

asureSign

Metrics can be:

• From HW verification

• From Silicon validation

• From SW testing

Export Reqs Status as XML:

Req1 [,]

Req2 [,]

Complex Bi-Directional Mappings

Req1 Feat1 Feat1.1 Goal1 Metric1

Metric2

Metric3

Feat1.2 Goal2

Feat2 Req2 Metric6

75%

50%

0%

Metric4

Feat1.3 Goal3

Goal4

Metric5

10 10 7 July 2008

asureSign – Requirements Signoff

Reqs

1. _____

2. _____

3. _____

4. _____

5. _____

Bug Tracking

Configuration

System

asureSign

UCIS

Regression Tests

Test Holes Reqs Signoff

Reqs History

Resources 1. Staff

2. Hardware

Test History 1. Versions

2. Pass/Fail

1.1 ___

1.2 ___

1.3 ___

1.4 ___

Reqs

1. _____

2. _____

3. _____

4. _____

5. _____

Regression

Scripts

Do not disturb

your current

regression flow

11 11 7 July 2008

Cross Domain Considerations

• Requirements can be signed off via a range of

different disciplines

• Pre-Silicon Verification

– Across hierarchy

• Software testing

– Pre-Silicon

– Post Silicon

• Post-Silicon

– Functional verification of features

– Validation of electrical features

12 12 7 July 2008

Some features can only be verified at SoC

• Low power features

– Voltage islands

– Power down and power up

– Frequency scaling

• The interrupt system

• HW/SW co-verification

• Digital and Analog integration

• Chip wide ECC

• Lock step CPUs

Cost Saving: Remove Over Engineering Across Hierarchy

and Across Disciplines

• Requirements can

be tested at

– Block, Subsystem and

SoC level

– Across multiple

disciplines

• Manage

requirements

signoff at those

levels and

disciplines

14 14 7 July 2008

Up the supply chain?

• asureSign supports

– Software testing

– Lab testing

– ???

• Can we pass asureSign databases up the

supply chain?

– SQL databases?

– XML files?

Summary

• Electronics in cars – More of it and more complex!

• ISO26262 mandates requirements signoff

• Hardware verification is a complex task – Complex mappings from requirements to signoff

– Needs high levels of automation

• Requirements management tools not good at – Supporting complex signoff

– Cross domain signoff

• Single view required across all domains – Identify over/under engineering & risk based testing

• And Beyond?