Verification Techniques

for Better Code and

Higher Productivity

mark.richardson@ldra.com

© 2015 LDRA Ltd 1

Agenda

• Introduction

– The risk of Software Failure

– Who are LDRA?

– Why are LDRA interested in this?

• Certification/Qualification

• Code Quality Drivers

• Productivity Drivers

• Summary

2

Introduction

3

Risk of Software Failure

• Consider the European

Space Agency’s Ariane 5

flight 501 on Tuesday,

June 4 1996

• Due to an error in the software design (inadequate

protection from integer overflow), the rocket veered

off its flight path 37 seconds after launch and was

destroyed by its automated self-destruct system

4

LDRA Ltd

• Liverpool Data Research Associates

• Founded 1975

• Provider of Test Tools & Solutions

• Metrics Pioneer ex: LCSAJ

• LDRA Certification Services

• Active participation in standards

ex: DO-178C, MISRA C/C++, CERT

5

LDRA:

Standards Experience & Pedigree

Professor Mike Hennell

• Member of SC-205 / WG-71 (DO-178C) formal methods subgroup

• Member of MISRA C committee and MISRA C++ committee

• Member of the working group drafting a proposed secureC annex for the C language

definition (SC 22 / WG14)

Bill St Clair

• Member of SC-205 / WG-71 (DO-178C) Object-Oriented Technology subgroup

Chris Tapp

• Member of ISO software vulnerabilities working group (SC 22 / WG 23)

• Member of MISRA C committee

• Member of the working group drafting a proposed secureC annex for the C

language definition (SC 22 / WG14)

Liz Whiting

• Member of MISRA C

committee

Dr Clive Pygott

• Member of MISRA C++ committee

• Member of MISRA C committee

7

Standards

8

Leading Safety Critical Standards

IEC 61511 (First published 2003)

ISO 26262 (Published 2011)

IEC 62304 (First published 2006)

IEC 61513 (First published 2001)

CENELEC EN 50128 (First published 2001)

DO-178B (First published 1992) / DO-178C

IEC 61508 (First published 1998, Updated 2010)

Avionics

Industrial

Railway

Nuclear

Automotive

Medical

Process

…

9

Level E to Level A

SIL Level 0 to SIL Level 4

Class A to Class C

ASIL A to ASIL D

SIL Level 1 to SIL Level 4

Safety Integrity Levels

IEC 61508 (Industrial)

ISO 26262 (Automotive)

IEC 62304 (Medical)

CENELEC EN 50128 (Railway)

DO-178B / DO-178C (Avionics)

10

DO-178C & Supplementary Documents

DO-331: "Model-Based Development and Verification Supplement to DO-178C and DO-278"

Airborne

(DO-178C)

FM

(DO-333)

MBDV

(DO-331)

OOT/RT

(DO-332)

Ground

(DO-278A)

TOOLS

(DO-330)

FAQ,DP

(DO-248C)

11

Safety Objectives: Example DO-178C

Design Assurance Level Objectives Objectives that must be

verified with independence

A - Catastrophic 71 30

B - Hazardous 69 18

C - Major 62 5

D - Minor 26 2

E – No Effect - -

12

Safety Objectives: Example DO-178C

13

Safety Objectives: Example IEC 61508

14

Requirements

15

Downstream Traceability

16

Upstream Traceability

17

Code Quality Drivers

18

Software Errors

• There are two types of errors that can be found:

• Technical errors

– These are due to language and programming and can

be found by performing static analysis

• Application errors

– These are due to the code not implementing correctly

the requirements

– These can only be found by performing dynamic

analysis on the actual hardware and measuring

structure coverage

19

Coding Standard

20

ACME Standard

21

Verify Coding Standard Compliancy

22

Satisfied Objectives

23

Structural Coverage

• First of all, Structural Coverage is often mandated

by standards such as DO-178C, IEC 61508,

ISO 26262, …

• Secondly the more coverage

we obtain, the higher the

confidence, that no code

remains that will do something

unexpected when it is deployed

• It also helps us understand

when we have done enough

testing

24

Structural Coverage

• Depending on the Safety Integrity Level (SIL) of the

project, there are different types of Structural

Coverage that need to be achieved, for example:

– Entry points coverage

– Statement coverage

– Branch decision coverage

– Modified Condition / Decision Coverage (MC/DC)

– Data coupling and control coupling coverage

– Object code coverage

– Linear Code Sequence And Jump coverage (LCSAJ)

25

Measure Structural Coverage

26

Productivity Drivers

27

Productivity is Crucial

• Certification/Qualification and Code Quality are

important

– Time consuming

– Resource intensive

– Costly: Time, resources, financially

– Requires traceability and analysis

• How do you get this time back?

– Automation

– Acceleration

– Integration

28

Automation:

Static Analysis

• A view into your code

29

Automation:

System Dynamic Analysis

30

Automation:

Structural Coverage Analysis

31

Automation:

Requirement Verification

32

Automation:

Continuous Integration

33

Acceleration:

Unit / Integration Testing - Host

34

Acceleration:

Unit / Integration Testing - Target

35

Integration:

Model-Based Testing

• MathWorks® Simulink®

– LDRA integrates into SIL/PIL mode

• IBM® Rational® Rhapsody®

– LDRA integrates into Rhapsody

• Esterel® SCADE®

– Test Cases generated using formal methods

– LDRA integrates to independently verify test cases

36

Integration:

Eclipse IDE

37

Integration:

Visual Studio

38

Integrations …

39

Summary

• Code Quality & Certification/Qualification are becoming increasingly important

• To satisfy safety/security objectives while minimising cost, we focus on maximising developer productivity

• LDRA can help you achieve these goals through: – Automation

– Acceleration

– Integration

40

@ldra_technology LDRA Software Technology LDRA Limited

For further information:

www.ldra.com info@ldra.com

42