© LDRA 2017 LDRA tool suite v9.7.0 August 2017

SimplifyingFunctional Safety

Certificationwith the ARM® DS-5 Development Studio

and the LDRA tool suite®

1

2

Goals

▪Starting with some simple requirements written in Word & Excel, show how a project can be developed using the ARM® DS-5 IDE with the Cortex-M3 VFP

▪Verify the traceability between all the requirements and the code

▪Verify every high level and low level requirement by running tests on the target

▪Verify that the code is MISRA C:2012/AMD1 compliant and also clear, maintainable and testable

▪Verify that 100% structural coverage has been attained

3

Pre-requisites

▪ LDRA tool suite V9.7.0 or above

▪ARM DS-5 IDE (Tested with v5.27.1)

Preparation

4

5

ARM DS-5 Development Studio

▪ It is assumed that ARM DS-5 Development Studio is installed and a license obtained

6

LDRA tool suite

▪ It is assumed that the LDRA tool suite is installed and licensed

7

Environment Variables

▪ In order to use ARM DS-5 via the command line, it may be necessary to configure the following Environment Variables:

▪ This can be done by running a batch file similar to the following:

setx ARM_PRODUCT_PATH "C:\Program Files\DS-5 v5.27.1"

setx ARM_TOOL_VARIANT ult

setx ARMLMD_LICENSE_FILE L:\LDRA_Flexlm\Ltype\DS-5_license.dat

8

Install the TLP (Target License Package)

▪ The TLP allows the LDRA tool suite to work with ARM DS-5 Development Studio

▪Run the ARM DS-5 TLP installer

▪Select the following options

9

ARM DS-5 Installation Path

▪Select the path of where ARM DS-5 is installed

10

ARM Compiler Installation Path

▪Select the path of where the ARM compiler is installed

11

Eclipse Configuration

▪Select LDRA

12

Upload Method

▪Select SCRIPT

Additional Tools

▪Run the LDRA Launcher and then select “Additional Tools”

13

Additional Tools

▪Select the LDRA Eclipse Plug-in

14

Eclipse Plugins

▪Select the DS-5 plugins directory

15

16

ARM_DS5_Workspace

▪Once installed, locate the following workspace

Load the ARM_DS5_Workspace

▪Start ARM DS-5 and select the provided workspace

17

Import

▪ Import the provided Projects

18

Import Project: DS5_Safe_Utilities

▪ Import the following project

19

20

Cortex-M3

▪ This simple project targets the Cortex-M3

21

LDRA Configuration

▪Select the LDRA Configuration

22

LDRA Configuration

▪An LDRA configuration is added to the project to be tested to allow any necessary options such as increased stack size, or suppression of warnings to be set

23

Build

▪Build the project

24

DS-5 Debugger Configuration

▪Create a DS-5 Debugger Configuration

25

DS-5 Debugger

▪Create a new Debug Configuration

26

DS-5 Debugger Configuration

▪Configure the DS-5 Debugger Configuration

27

Debug

▪Now the program can be debugged

28

target.cli

▪ Finally export the DS-5 Debugger Configuration to a file called target.cli

▪ This is a file that will be used by LDRA in order to run on the target

Requirements

29

30

Requirements

▪Open the Requirements folder

31

System Level Requirements

▪ This is a very simple document that contains just one requirement that starts with SYS_

32

High Level Requirements

▪ This document contains High Level Requirements that all start with HLR_ and which have a link to a System Level Requirement

33

Styles

▪Note how in this document, styles have been used to help identify requirements ex:

34

Low Level Requirements

▪ The Low Level Requirements are in an excel document and contain links to High Level Requirements

35

Restore_Initial_State.bat

▪Run the Restore_Initial_State.bat file, this will create a new TBmanager project and in case the tutorial has already been started or completed, will delete any existing results

36

TBmanager

▪Open the newly created TBmanager project, by double-clicking on the .tbp file

37

UniView

▪By default the UniView should be shown

38

Groups

▪ The following groups have been added▪ SLR: System Level Requirements

▪ HLR: High Level Requirements

▪ LLR: Low Level Requirements

▪ HLT: High Level Tests

▪ LLT: Low Level Tests

39

Import System Level Requirements

▪We want to import the System Level Requirements from the Word document into the Group SLR

▪ First switch to the documents view, where we can see that the Requirements documents have already been added

40

Import from Word Document…

▪Select the Safe_Utilities_SLR.docx file, right-click and select “Import from Word Document…”

41

Regular Expressions

▪Note how regular expressions have previously been created, to extract the System Level Requirements

42

Preview

▪Click on Refresh Preview to see that the regular expressions have correctly identified the requirement: number, name and body

▪Note also that the requirements will be imported into the group SLR

43

Import

▪Click on OK to first view the requirements, then on OK again to import them

44

Imported Requirement

▪We should now be able to see the imported requirement in the Project Tree

45

Import High Level Requirements

▪Next we want to import the High Level Requirements from the Word document into the Group HLR

▪Select the Safe_Utilities_HLR.docx file and “Import from Word Document…”

46

Rule 1: Requirement_ID Style

▪ This time since styles have been used, it is much easier to identify the requirements. Different rules have been created for each style ex:

47

Rules 2 & 3

▪Similarly rules have been created for the styles Requirement_Text and Traceability_Data

48

Preview

▪As before, refresh the preview to check the rules

▪Check also that the requirements will be imported into the group HLR

49

Import

▪Click on OK to first view the requirements, then on OK again to import them

50

Imported Requirements

▪We should now be able to see the imported requirements in the Project Tree, nested below SYS_100, since they all have a link to that requirement

51

Import Low Level Requirements

▪Next we want to import the Low Level Requirements from the Excel document into the Group LLR

▪Select the Safe_Utilities_LLR.xlsx file, right-click and select “Configure .xlsx Format…”

52

Columns

▪Each column is identified as a specific attribute ex: requirement: number, name, body, …

53

Group LLR

▪ The first row is ignored and the requirements will be imported into the group LLR

54

Get Requirements from File

▪Click OK to close this menu, then to import the requirements, select “Get Requirements from File”

55

Imported Requirements

▪We should now be able to see the imported requirements in the Project Tree

56

Relationships View

▪Switch to the relationships view

57

SLR Requirements

▪Right-click in the 1st column and add all the requirements from the group: SLR

58

Traceability

▪Now we can observe the upstream and downstream traceability by clicking on any requirement

59

UniView

▪Switch to the UniView view

60

Traceability Matrix

▪Right-Click on the HLR group and select “Traceability Matrix Report to Requirement”, then select Group: LLR

61

High level Requirement <Not Covered>

▪We can see clearly that one High Level Requirement is not covered!

Traceability to Code

62

63

Source Code

▪We now need to look at the traceability between the source code and the Low Level Requirements

▪ First we need to analyse the source code, the simplest way to do this is via the LDRA Eclipse plug-in in ARM DS-5

▪Start ARM DS-5

▪Select the LDRA Perspective

64

Set As Active Project

▪Ensure that the Active Configuration is LDRA

▪Right-click on the project and set as LDRA Active Project

65

Analysis

▪Now perform the analysis

66

Source Code

▪We now need to look at the traceability between the source code and the Low Level Requirements

▪ First we need to analyse the source code

▪ In the Project Tree, right-click on Source Code and select “Add Compiler Project…”

67

Code Violations

▪As can be seen, there are a number of detected violations, we will look at these later

▪But for now, close ARM DS-5

68

TBmanager

▪Back in TBmanager, right-click on Source Code and select “Add Multiple Files as Set…”

69

DS5_Safe_Utiltiies

▪Select the set that was created by the LDRA plug-in called DS5_Safe_Utilties

70

Map Source View

▪Switch to the Map Source View and expand each file

71

Map Source Code

▪Map each function (except main) to the appropriate Low Level Requirement by dragging and dropping

72

Relationships View

▪Switch to the Relationships view and note that every function should now trace back to a requirement

Tests

73

74

Tests

▪Before testing against the requirements, we also want to perform the following tests:▪ Check that the code is compliant to MISRA C:2012/AMD1

▪ Check that the code is clear, maintainable and testable

▪ Then after testing against the requirements, we will want to perform the following test:▪ Check that we have 100% Structural Coverage

75

TCI Grid View

▪Switch to the TCI Grid View

76

Map TCIs to Set

▪Map each TCI (Test Case Identifier) to the System Set by dragging and dropping ex:

77

Map Source View

▪Switch to the Map Source View and the TCIs should be shown

78

Code Review

▪Right-click on the TCI_CodeReview and select “Verify with LDRA tool suite…”

79

Code is Not Compliant

▪ The Green dots indicate that there are no Mandatory or Advisory violations, but there are some Required violations

80

Callgraph

▪Drag the System Set onto the Output Callgraph placeholder

81

Callgraph – Programming Standards

▪Double-click to open the callgraph

82

Violations

▪Clicking on a function, highlights the coding standard violations

83

Code Review Report

▪Alternatively double-click on the Code Review Report

84

View Results with LDRA tool suite

▪Or view the results with the LDRA tool suite

85

Quality Review

▪Next verify the quality of the code

▪Right-click on the TCI_QualityReview and select “Verify with LDRA tool suite…”

86

Code is Clear, Maintainable and Testable

▪ The Green dots indicate that the quality of the code is good, it is clear, maintainable and testable. All the measured metrics are within the specified thresholds

87

Callgraph

▪Drag the System Set onto the Output Callgraph placeholder, then double-click to open the Callgraph

88

Maintainability View

▪Select the Maintainability View

89

Sort Metrics

▪Clicking on the column title, sorts the metrics by value, making it easy to locate the most complex function

▪ The flowgraph for each function can be viewed ex:

90

Flowgraph

High Level Tests

91

92

High Level Tests

▪Next we want to verify the High Level Requirements

▪ The Safe_Utilities_HLT.xslx file contains just a single High Level Test

93

High Level Tests

▪ For this test, we are simply going to execute the main which exercises every function and afterwards check the structural coverage

94

Import High Level Tests

▪Switch to the Documents View, select the Safe_Utilities_HLT.xslx file and “Get Test Cases from File”

95

Set External Task

▪Set the External Task to be the following

96

Build Instrumented

▪As the code executes, we need to measure the structural coverage. In order to do this, we first need to instrument the source code and then to perform a build

▪ This can be done by executing the batch file Build_Instrumented.bat

▪Click on the following menu to do this

97

Verify with External Task

▪Now we can right-click on the HLT and verify with External Task

98

Execution History

▪At the end of the execution, the compressed execution history is uploaded to the host

▪ This can now be processed by running the batch file Get_Coverage.bat

▪Click on the following menu to do this

99

Code Coverage

▪Now switch to the Map Source View and verify the code coverage

100

Callgraph

▪Drag the System Set onto the Output Callgraph placeholder

▪Double-click to view the pass/fail coverage Callgraph

101

Callgraph

▪As expected, every function has been invoked, but of course we don’t have 100% coverage

102

Flowgraph

▪View the Coverage Pass/Fail Flowgraph for the function safe_uncompress

103

Defensive Programming

▪Since the code checks that the parameters aStringand anArray are not NULL, we don’t have 100% structural coverage

▪ In order to test this “defensive programming”, we will need to perform unit testing

Low Level Tests

104

105

Low Level Tests

▪Next we want to verify the Low Level Requirements

▪ The Safe_Utilities_LLT.xslx file contains the Low Level Tests

106

Import Low Level Tests

▪Switch to the Documents View, select the Safe_Utilities_LLT.xslx file and “Get Test Cases from File”

107

Associated Test Case File (.tcf)

▪With each Low Level Test, there is an associated Test Case File which contains a sequence of test cases

▪ TCF files can be regressed using TBrun

▪Select the following Low Level Test and “Verify Test Interactively in TBrun...”

108

TBrun

▪ For each Test Case, we can see the list of inputs and expected outputs

109

Run

▪ The Test Cases can now be compiled, linked and executed on the target

▪ The Test Cases all pass with 100% coverage

110

Test Passes

▪Exiting TBrun will update the status of the Low Level Tests in TBmanager

▪Now verify the next Low Level Test

111

Failed Test Case

▪ This time the Test Case fails

112

Failed Test Case

▪ This is exactly why we unit test, the function safe_uncompress does not work correctly and must be modified

113

Regression Report

▪Viewing the Regression Report shows why the test case failed

114

Fail

▪Since the Test failed, it has a red dot

115

TCI Grid

▪Switch to the TCI Grid

116

Filter

▪Press “Control + L” to create the following filter

117

2nd Filter

▪Press “+” to create a second filter as shown

118

Regress All Unverified Tests

▪Press “Control + A” to select all the unverified Test Cases, then regress them

119

One Unverified Test Case Identifier

▪Once the filter is refreshed, there should now be just the single unverified Test Case Identifier

120

Code Coverage

▪Now that all the unit tests have been run, check that we have 100% structural coverage

121

Objectives

▪Next import the objectives from a standard

122

ISO 26262

▪Any standard can be imported, but for the moment, select ISO 26262 and click OK

123

Standards

▪ The existing standards can be customised or additional standards added ex:

124

Objectives View

▪Switch to the Objectives View

125

Placeholders

▪ For each objective, it is possible to add placeholders for all the various artifacts / assets that need to be created or produced in order to satisfy it

126

Placeholders

▪Expand the objective “Part 6: Section 5: Table 1: 1a”

▪Note the placeholders for Artifacts that need to be created/produced in order to satisfy this objective

127

Realising Objectives

▪As the Artifacts/Assets get produced, they can be used to satisfy the placeholders

▪Right-click on each placeholder and associate the appropriate document ex:

128

Satisfied Placeholders

129

Fulfilled Objective

▪Once the associated documents, have been reviewed, the status of the objective can be changed to Fulfilled

130

Objectives Report

▪Generate an Objective Summary Report and navigate from it to view the documents

131

Reports

▪ Finally, reports such as the Project Coverage Detailed Report can be generated

132

Command Line

▪Everything that was performed manually in this tutorial can be automated from a batch file

▪Close TBmanager and try running the following batch file

Summary

133

134

Summary

▪ In this simple example, we have seen:▪ How the traceability from requirements to code can be

verified

▪ How the code in a ARM DS-5 project can be analysed and checked for MISRA C:2012/AMD1 compliance as well as ensuring that it is clear, maintainable and testable

▪ How the High Level Requirements and Low Level Requirements can be verified by executing tests on the target

▪ How the Structural Coverage can be measured

▪ How everything can be automated from a batch file

135

For More Information

▪ For more information, please view the following tutorials

.com

Need more information?

info@ldra.com

Contact Us

136