Practical Experience of Applying Agile Techniques to the

Development of a Safety-Critical System

David Nicoll

Project Realization

When do we get value from software?

When we get the software into…

Business Benefit

Typical Waterfall Development

Requirements

Design

Code

Test

Traceability between phases

Completion of a Phase is often a contract payment milestone.

Review

Review

Review

End of Phase reviews form a stage-gate When do we get the ROI?

Right at the end…..

But will it deliver what we want ?

Reputation & Outcomes

• Software is always late

• It doesn’t work

• Costs too much

• Quality is poor

• Changes are slow and costly

Model of Traditional Development

Quality

Time Cost

Functionality

Fixed

Variable

Status of Agile

• Is by far the fastest growing methodology today

• Is being taken up across all sectors both public and private

• Is becoming the de-facto standard approach

• Originally for small development teams – Typical team of 7 people (± 2)

– Is now being scaled to teams of 1000+

• Number of approaches for organisational levels – Development level (includes Scrum, XP)

– Project & Programme level (includes DSDM, SAFe)

Basis of Agile and ROI

• Agile is concerned with getting the fastest ROI • Continuous iterative development • Progressive incremental delivery

– to provide Business Benefit throughout the development

• Driven by costs and timescales – Functionality is removed or deferred

• Assumes not everything is known – Anticipates Change will happen

• Fast feedback supports continuous improvement • Collaborative working between

– between Client and Supplier – Development teams

Deliveries are Fixed In Time

Time

Incremental Delivery

#1

Incremental Delivery

#2

Incremental Delivery

#3

Planned Delivery Dates based on Timescale NOT content

Increment #1 Increment #1

Incremental Functionality

Time

Incremental Delivery

#1

Incremental Delivery

#2

Incremental Delivery

#3

Build incrementally on firm foundations

Increment #1 Increment #1

Increment #2 Increment #2

Increment #3

Increment #1

Increment #2

Increment #2

Increment #1

Timescale Takes Precedence

Time

Incremental Delivery

#1

Incremental Delivery

#2

Incremental Delivery

#3

Continuous delivery – functionality deferred

PRIORITISE

PRIORITISE

Increment #3

Increment #1

Increment #2

Increment #3

Completeness of Increments

Time

Incremental Delivery

#1

Incremental Delivery

#2

Incremental Delivery

#3

Design

Code

Test

Design

Code

Test

Design

Code

Test

Review

Plan

Review

Plan

Review

Plan

Incremental Safety Assurance

Time

Increment #1

Increment #2

Increment #3

Increment #2

Increment #1 Safety Audit

#1

Safety Audit

#2

Increment #1 Safety Audit

#1 Increment #1

Safety Audit

#1

Increment #2 Safety Audit

#2

Increment #1 Safety Audit

#1

Increment #2 Safety Audit

#2

Increment #1 Safety Audit

#1

Increment #3 Safety Audit

#3

Model of Agile Development

Quality

Time Cost

Functionality

Fixed

Variable

Agile Development

No change to existing best practice

• Full traceability (requirements, design, code, test)

• Coding standards – Static analysis

– Complexity

– Module size

• Unit Test – full path coverage

• Independent reviews

• Test Driven Development (TDD)

• Automated overnight build and test

• Strict configuration control and change control

14

Management of Risk

• Agile provides early tangible working product – Evidence based progress

– Avoids the “90% complete” syndrome

– Provides for re-prioritisation

• Overall risk is progressively reduced throughout the development

• Risk exposure is limited to the cost of the current increment

• Lessons Learned from one increment are passed to the next

Risk Over Time

Time

Risk

Delivery Deadline

Waterfall

Agile

Increment #1

Increment #2

Increment #3

Progress Monitoring

• Traditional Gantt Charts are only useful at a high level

• Daily Stand-Up meetings – Provides an environment for communication and team building – Each team member provides a verbal update to the rest of the

team

• Wall boards show – The Workflow – Who is doing what – Where the progress blocks are

• Burn-down charts – Shows how fast work is being performed (velocity) – Provide a forecast completion date

Progress Monitoring: Burn-Down Charts

To Do

Complete

In Progress

Estimated Finish #1

Estimated Finish #2

Date

Number of

Reqmts

Estimated Finish #3

Actual Finish

Agile for Safety-Critical Rail

• Autonomous Underground Train Control system • Real-Time Safety-Critical System

– CENELEC 50128 (SIL4) – Loss of Life

• Automatic control of – train, signals, points

• Radio based communication between – Trains – Timetable (including local speed restrictions)

• Doppler radar – Provides speed, distance & direction

• Axle counters – to determine train position in station

FBP: System Layout

Radio

Interlocking

Control Centre

ATP ATO

Radio Links

Communications

Signals & Points

Radio

BP

BP Fixed Block Processor

Interlocking

ATP ATO

Train driving

Initial Development Process

Detailed Design

Code

Review & Safety Audit

Review & Safety Audit

Requirements Definition

High-Level Design

Unit Test

Integration Test

Acceptance Test

System Test

Review & Safety Audit

Review & Safety Audit

Review & Safety Audit

Review & Safety Audit

Review & Safety Audit

Development Process

Detailed Design

Code

Requirements Definition

High-Level Design

Unit Test

Integration Test

Acceptance Test

System Test

Code

Detailed Design

Unit Test

Detailed Design

Code

SPARK

Unit Test

Increments

SPARK

Code

Detailed Design Detailed Design

Code

SPARK

Unit Test

Code

Detailed Design

SPARK SPARK

Unit Test

Code

Detailed Design

Unit Test Unit Test

Detailed Design

Code

Unit Test

SPARK

Detailed Design

Code

Unit Test

SPARK

Detailed Design

Code

Unit Test

SPARK

Incremental Development

Time

Review Previous Increment

Plan this Increment

Overall Requirements

Backlog

Design

Code

Test

SPARK

Increment Requirements Backlog

Process Improvements

Incremental Development

Design

Code

Test

SPARK

• Full Traceability – Requirements – Design – Code – Test

• Independent Unit test – 100% path coverage – MC/DC testing – Boundary values

• Independent Formal Reviews

• Incremental Hazard Analysis

25

Conventional Safety Analysis Process System Hazards & Safety Constraints

New Hazards

New Hazards

Vertical Slice Analysis

Known Hazards

Known Hazards

Design Verification Safety Analysis

Code Verification Safety Analysis

Safety Requirements Verification Analysis

Safety Audit Report

Phase Specific Safety Reports

Safety Analysis

Safety Analysis

Design

Code

Test

Safety Analysis

FBP: Increment based Safety Analysis System Hazards & Safety Constraints

Vertical Slice Analysis

Increment

Unit Test Safety Analysis

Code Verification Safety Analysis

Design Verification Safety Analysis

Safety Audit Report

(Increment)

Design

Code

Test

New Hazards Safety

Analysis

New Hazards

Phase Specific Safety Reports

Safety Analysis

New Hazards

FBP Burn-Down: Testing

0

100

200

300

400

500

600

700

800

900

No

. o

f M

od

ule

Tests

Weeks

FBP: Module Testing (Formal)

Complete

Progressing

To Go

Total Tests

28

FBP: Retrospective Project Analysis

Development Team Size = 70+ (35 UK, 35 India, 4 Spain)

Primary project objective = Timescale

Crude industry standard = 22 ↔ 24 months duration

Actual development = 18 months

Cost (Effort)

Time

FBP

Approaching Agile

• Fundamentally Agile is a mind-set

• It is about managing project risk in order to deliver business benefit

• Agile is not proscriptive

– Best practice in all activities

– No conflict with current industry practice

• Agile advocates a number of methods, techniques and approaches that deliver business benefit

• It is up to you to tailor these to your need

Any Questions?

David Nicoll

www.project-realization.com