10 analysis

8/6/2019 10 Analysis

1/57

SSW 540

Foundations of Software

Engineering

Week 10

Analysis of Software Systems


2/57

OutlineOutline

1. NASA Case Study

2. Analysis Methods and Tools

Week 10 2


3/57

Case Study:Case Study:

Toyota Unintended Acceleration (UA)Toyota Unintended Acceleration (UA)

National Highway Traffic Safety Administration (NHTSA)

opened an investigation into Unintended Acceleration

(UA) problems with Toyota automobiles in March 2004

Toyota recalls in October 2009 and January 2010

Claimed that faulty floor mats were a cause of UA

Congressional hearings held in early 2010

Much skepticism about floor mats

Speculation that throttle control was a factor

Week 10 3


4/57

Reported ProblemsReported Problems

About 3000 Vehicle

Owners Questionnaire

(VOQ) reports of UA in

Toyota automobiles in2000-2010 time period

Many reports submitted

after media coverage of

NHTSA investigationannouncement and

congressional hearings

Week 10 4


5/57

Toyota Acceleration Control SystemToyota Acceleration Control System

Week 10 5


6/57

Toyota SoftwareToyota Software

Developed by Toyota and DENSO

More than 280 KLOC of C

Must meet real-time performance requirements Employs well-known practices for safety-critical

systems

Coding practices to avoid troublesome issues

Fail-safe states

Watch-dog timers and heartbeat monitoring

Week 10 6


7/57

Toyota Software DevelopmentToyota Software Development

Toyota and their subcontractor DENSO use a traditional waterfall

lifecycle model:

1. Specification (Toyota)

2. High-level Design (Toyota)3. Detailed Design (DENSO)

4. Implementation (DENSO)

5. Unit test (DENSO)

6. Acceptance test (Toyota)

7. Integration test (Toyota)

Week 10 7


8/57

Toyota Software Quality Assurance ToolsToyota Software Quality Assurance Tools

QA-C, a static analysis tool from the British company Programming

Research

CAST, an in-house TMC tool to verify type conversions and

potential cases of value overflow and some common cases ofcoding defects

Careless, another in-house TMC tool for verifying different coding

patterns and coding style

A Mode Transition checker, to verify correct transitions between

normal and failure modes

A Stack overflow checker

A Task interference check, mostly manual check, assisted by a

tool-generated list of cases to be inspected

Week 10 8


9/57

NASA StudyNASA Study

NHTSA asked the NASA Engineering and Safety

Center (NESC) to conduct an independent

assessment of the design and implementationof Toyota's Electronic Throttle Control System

interactive (ETCS-i)

Started mid April 2010 and ran through mid

August 2010 Toyota provided access to engineers, hardware

and software

Week 10 9


10/57

Fault Tree AnalysisFault Tree Analysis

Week 10 10

OR node

AND node


11/57

Fishbone Diagram ofFishbone Diagram of

Potential Software Causes of UAPotential Software Causes of UA

Week 10 11


12/57

Coding DefectsCoding Defects

Array bound violations

Nil/bad pointer

references

Unchecked parameter

dereference

Unexpected interleaving

Uninitialized data

Use after free

Unbounded loop

Side effects in expression

evaluaion

Integer overflow

Loss of precision in cast

Unchecked return value

Unreachable code

Redundant conditions

Missing break statements

Week 10 12


13/57

Algorithmic FlawsAlgorithmic Flaws

Cruise control design

Pedal learning algorithm

Throttle learning

algorithm

Brake overrides

Throttle angle

computation

Throttle motor drive

commanding

PID control and PWM

computation

Wrong value learned

Incorrect value stored

Non-learning

Unintended

(de)activation

Week 10 13


14/57

Task InterferenceTask Interference

Missing locks or interrupt masks

Unprotected access to shared data

Week 10 14


15/57

Insufficient Fault ProtectionInsufficient Fault Protection

Bit flips, memory corruption

No mirroring

Incorrect mirror values

CAN network problems

Missing, delayed or corrupted data

Babbling nodes

Insufficient margins

CPU overload

Missed time bounds

Unanticipated computer resets

Week 10 15


16/57

Software Analysis MethodsSoftware Analysis Methods

1. Static analysis

Detect unsafe coding

practices

Discover potentialvulnerabilities

2. Logic model checking

Conversion of desired

throttle position into

transistor signals Generic processing of the

sensor inputs

Processing of accelerator

pedal position sensor input

3. Algorithm design analysis

Model-based design

techniques used to create

high-fidelity models of the

software

Source code analysis

increased fidelity of

models

Testing upon the Camry

simulators and actual

Camry vehicles confirmed

the accuracy of the models

Week 10 16


17/57

Tools UsedTools Used

Static Analysis

Coverity

CodeSonar Uno

Logic Model Checking

Spin

Swarm

Algorithm Design

Analysis

MATLAB Simulink

Stateflow

SystemTest

aiT

Week 10 17


18/57

CoverityCoverity

One of the leading static source code analysis

tools on the market

Excels at finding common coding defects andsuspicious coding patterns in large code bases,

taking a relatively short amount of time

Also supports custom-written checkers that can

verify compliance with user-defined additionalcoding rules

Week 10 18


19/57

CodeSonarCodeSonar

Strong static source code analysis tool from

Grammatech that uses a different technology

for detailed inter-procedural source code

analysis

Codesonar analysis typically takes longer to

complete than comparable tools, but can reveal

more subtle types of defects and suspect codingpatterns, requiring deeper path analysis

Week 10 19


20/57

CodeSonarCodeSonar ReportingReporting

Week 10 20


21/57

UnoUno

Static analysis tool designed to check for:

Use of uninitialized variables

Nil-pointer references Out-of-bounds array indexing

May also specify user-defined propertiesvoid uno_check(void) {

if (select("assert", FCALL, NONE))if (select("", DEF|FCALL, NONE))

if (unselect("assert", ANY, NONE))error("side effect or fct in assert"); }

Week 10 21


22/57

SpinSpin

Free model checker developed by Gerard

Holzmann while at Bell Labs (now at NASA)

Models are expressed in Promela, a processalgebra language

Properties are expressed in Linear Temporal

Logic

Graphical front-ends have been developed tosimplify use of Spin

2011 Stevens Institute of Technology 22


23/57

Model CheckingModel Checking

Check properties of a finite-state model using

brute-force exploration (and state-space

reduction)

Used by Intel in chip design

Increasing use in software design



24/57

SwarmSwarm

Generates a script that performs many small

Spin verification jobs in parallel

Swarm uses parallelism and searchdiversification to reach its objectives

Week 10 24


25/57

MATLABMATLAB

High-level language and environment for

constructing and analyzing design models

Mathematical functions for linear algebra,statistics, Fourier analysis, filtering,

optimization, and numerical integration

2-D and 3-D graphics functions for visualizing

data Functions for integrating MATLAB based

algorithms with external applications

Week 10 25


26/57

SimulinkSimulink

Graphical tool for construction of models

Integrated with MATLAB



27/57

StateflowStateflow

Finite state

machine models

Week 10 27


28/57

SystemTestSystemTest

Provides

automated testing

of Simulink

models

Week 10 28


29/57

aiTaiT

Statically computes tight bounds for the worst-

case execution time (WCET) of tasks in real-

time systems

aiT directly analyzes binary executables and

takes the intrinsic processor cache and pipeline

behavior into account

Week 10 29


30/57

Study ConclusionsStudy Conclusions

"Proof for the hypothesis that the ETCS-i caused

the large throttle-opening UAs as described in

submitted VOQs could not be found with the

hardware and software testing performed."

A couple minor potential problems were found,

but releasing the accelerator or overriding withthe brake would cancel them.

Week 10 30


31/57

Analysis Methodsand ToolsAnalysis Methodsand Tools

Week 10 31


32/57

Static AnalysisStatic Analysis

Good:

Tools perform checks thoroughly and consistently

By examining code tools can point to the root cause

Tools can find errors early, before the code is run

Tools can recheck software for new vulnerabilities

Bad:

False positives: tool reports a flaw that isn't

False negatives: tool misses a flaw

Unsolvable:

Any interesting question cannot be reliably answered

by a tool in all cases 32


33/57

Types ofstatic analysisTypes ofstatic analysis

Type checking

Style checking

Program understanding

Program verification

Property checking

Bug finding

Security review

33


34/57

Type checkingType checking

Most compilers perform type checking

Finds many errors

Still prone to false positives and false negatives

Programmers can override compiler complaints

with explicit type casting

34


35/57

Style checkingStyle checking

May enforce rules about whitespace, naming

lint set the standard for style checkers

Many lint checks now done routinely bycompilers

New tools allow more customization

Many tools will check for compliance with astandard (e.g., MISRA)

Recent focus is on new strategies for bug

finding

35


36/57

MISRA:MISRA:

Motor Industry Software Reliability AssociationMotor Industry Software Reliability Association

Collaboration between vehicle manufacturers,

component suppliers and engineering

consultancies Promotes best practice in developing safety-

related electronic systems in road vehicles and

other embedded systems.

36


37/57

MISRA GuidelinesMISRA Guidelines

MISRA-C:2004 Guidelines for the Use of the C

Language in Critical Systems

MISRA-C++:2008 Guidelines for the Use of the

C++ Language in Critical Systems

Other guidelines specific to particular

technologies, such as automatic code

generation

37


38/57

MISRAMISRA--C Example Rules (1/4)C Example Rules (1/4)

Rule 8.1 (required): Functions shall have

prototype declarations and the prototype shall

be visible at both the function definition and

call. Rationale:

The use of prototypes enables the compiler to check

the integrity of function definitions and calls.

Without prototypes the compiler is not obliged topick up certain errors in function calls (e.g. different

number of arguments from the function body,

mismatch in types of arguments between call and

definition). 38


39/57


Rule 10.1 (required): The value of an

expression of integer type shall not be

implicitly converted to a different underlying

type if:a) it is not a conversion to a wider integer type of the

same signedness, or

b) the expression is complex, or

c) the expression is not constant and is a functionargument, or

d) the expression is not constant and is a return

expression39


40/57


Rule 14.1 (required): There shall be no

unreachable code.

Rationale: Dead code causes confusion during code review, and

it may reveal mistakes made by the programmer.

This rule refers to code which cannot under any

circumstances be reached, and which can beidentified as such at compile time. Code that can be

reached but may never be executed is excluded

from the rule (e.g. defensive programming code).

40


41/57


Rule 20.4 (required): Dynamic heap memory

allocation shall not be used.

Rationale: This precludes the use of the functions calloc,

malloc, realloc and free.

There is a whole range of unspecified, undefined

and implementation-defined behaviour associatedwith dynamic memory allocation, as well as a

number of other potential pitfalls. Dynamic heap

memory allocation may lead to memory leaks, data

inconsistency, memory exhaustion, non-

deterministic behaviour.41


42/57

Power of Ten Rules for Safety Critical CodePower of Ten Rules for Safety Critical Code

Short list of rules for safety-critical applications

"The Power of Ten -- Rules for Developing Safety

Critical Code,'' IEEE Computer, June 2006, pp.

93-95

http://spinroot.com/p10/

Week 10 42


43/57

Power of Ten Rules (1/2)Power of Ten Rules (1/2)

1. Restrict to simple control flow constructs.

2. Give all loops a fixed upper-bound.

3. Do not use dynamic memory allocation afterinitialization.

4. Limit functions to no more than 60 lines of

text.

5. Use minimally two assertions per function onaverage.

Week 10 43


44/57

Power of Ten Rules (2/2)Power of Ten Rules (2/2)

6. Declare data objects at the smallest possible level of

scope.

7. Check the return value of non-void functions, and

check the validity of function parameters.8. Limit the use of the preprocessor to file inclusion and

simple macros.

9. Limit the use of pointers. Use no more than two levels

of dereferencing per expression.10. Compile with all warnings enabled, and use one or

more source code analyzers.

Week 10 44


45/57

Program understandingProgram understanding

IDEs include functionality for understanding

find all uses of a method or a variable

may support refactoring Some tools provide reverse engineering

Fujaba allows you to move between code view and

UML view

45


46/57

Program verificationProgram verification

Some tools check that a program agrees with a

formal specification

Problems: Difficult to construct formal specifications of large

systems

Difficult to simplify all expressions that need to be

checked for equivalence

46


47/57

Property checkingProperty checking

Could use a tool to check for some of the properties

that it might satisfy, rather than all of them

Properties of interest are often temporal safety

properties

A particular sequence of events will never occur

If the system reaches a given state, it will eventually

reach another good state

Property checking often done with model checkers

Exhaustively explore (almost) the entire state space

Use clever tricks to reduce the size of the state

space47


48/57

Bug findingBug finding

Look for patterns in code

Often try to produce a low rate of false

positives, even if that allows false negatives FindBugs is a good tool for Java

Similar tools are available for C and C++

Coverity

Microsoft VS \analyze

48


49/57

Static Analysis Algorithms (1/3)Static Analysis Algorithms (1/3)

Local analysis

Examine only the code within a single function or

method

Global analysis Examine code that spans functions or methods

Many useful analyses can be done by checking the

validity of assertions

taint propagation checks on assertions abouttrustworthiness of values

range analysis checks on size expressions

type state checks on valid states of an asset

(allocated or freed memory) 49


50/57


Symbolic execution

follow flow of execution and use symbolic values for

unknown values

Program slicing remove all parts of code that are not relevant to the

question asked

Abstract interpretation

abstract away aspects of code that are not relevant

to analysis

could do flow-insensitive analysis of statements

50


51/57

Program SlicingProgram Slicing

Original

int i;

int sum = 0;

int product = 1;for(i = 0; i < N; ++i) {

sum = sum + i;

product = product *i;

}

write(sum);

write(product);

Slice on (write(sum),{sum})

int i;

int sum = 0;

for(i = 0; i < N; ++i) {

sum = sum + i;

}

write(sum);



52/57


Predicate transformation

Calculate weakest preconditions

Model checking

Exhaustively execute a model of the program to

check for assertion violations

Function summaries

For global analysis can summarize behavior of

functions with specifications of their pre- and post-

conditions

Java Modeling Language (JML) could be used to

describe behavior in Java52


53/57

Static AnalysisRulesStatic AnalysisRules

Rules describe the assertions that tools will

check

Best case is to let the user of the tool createand edit all the rules that are used

Users could also annotate code with rules

JML

Microsoft Standard Annotation Language (SAL)

53


54/57

Example Annotation in JMLExample Annotation in JML

/*@ public normal_behavior

@ requires valid;

@ assignable state;

@ ensures -1


55/57

Taint Propagation RulesTaint Propagation Rules

Used to discover security vulnerabilities caused by

invalid input

Source rules define locations where tainted data

enters the system

Sink rules define locations that should not receive

tainted data

Pass-through define the way a function processes

tainted data

Cleanse pass-through that removes taint

Entry point similar to sink but invoked by attacker

55


56/57

Example of Taint PropagationExample of Taint Propagation

if (fgets(buf, ... {

strcpy(othr, buf);

system(othr);

}

1. Source rule forfgets() taints buf

2. Dataflow analysisconnects uses of buf

3. Pass-through rule forstrcpy() taints othr

4. Dataflow connects usesof othr

5. Sink rule for system()

reports commandinjection because othr

is tainted

56


57/57

Dynamic AnalysisDynamic Analysis

Purify

uninitialized memory access

buffer overflow and improper freeing of memory

memory leak detection

Valgrind

can be used to create your own dynamic analyses

comes with dynamic memory checking

2011 Ste ens Instit te of Technolog 57

10 analysis

Documents