Practical Experience of Applying Agile Techniques to the
Development of a Safety-Critical System
David Nicoll
Project Realization
When do we get value from software?
When we get the software into…
Business Benefit
Typical Waterfall Development
Requirements
Design
Code
Test
Traceability between phases
Completion of a Phase is often a contract payment milestone.
Review
Review
Review
End of Phase reviews form a stage-gate When do we get the ROI?
Right at the end…..
But will it deliver what we want ?
Reputation & Outcomes
• Software is always late
• It doesn’t work
• Costs too much
• Quality is poor
• Changes are slow and costly
Model of Traditional Development
Quality
Time Cost
Functionality
Fixed
Variable
Status of Agile
• Is by far the fastest growing methodology today
• Is being taken up across all sectors both public and private
• Is becoming the de-facto standard approach
• Originally for small development teams – Typical team of 7 people (± 2)
– Is now being scaled to teams of 1000+
• Number of approaches for organisational levels – Development level (includes Scrum, XP)
– Project & Programme level (includes DSDM, SAFe)
Basis of Agile and ROI
• Agile is concerned with getting the fastest ROI • Continuous iterative development • Progressive incremental delivery
– to provide Business Benefit throughout the development
• Driven by costs and timescales – Functionality is removed or deferred
• Assumes not everything is known – Anticipates Change will happen
• Fast feedback supports continuous improvement • Collaborative working between
– between Client and Supplier – Development teams
Deliveries are Fixed In Time
Time
Incremental Delivery
#1
Incremental Delivery
#2
Incremental Delivery
#3
Planned Delivery Dates based on Timescale NOT content
Increment #1 Increment #1
Incremental Functionality
Time
Incremental Delivery
#1
Incremental Delivery
#2
Incremental Delivery
#3
Build incrementally on firm foundations
Increment #1 Increment #1
Increment #2 Increment #2
Increment #3
Increment #1
Increment #2
Increment #2
Increment #1
Timescale Takes Precedence
Time
Incremental Delivery
#1
Incremental Delivery
#2
Incremental Delivery
#3
Continuous delivery – functionality deferred
PRIORITISE
PRIORITISE
Increment #3
Increment #1
Increment #2
Increment #3
Completeness of Increments
Time
Incremental Delivery
#1
Incremental Delivery
#2
Incremental Delivery
#3
Design
Code
Test
Design
Code
Test
Design
Code
Test
Review
Plan
Review
Plan
Review
Plan
Incremental Safety Assurance
Time
Increment #1
Increment #2
Increment #3
Increment #2
Increment #1 Safety Audit
#1
Safety Audit
#2
Increment #1 Safety Audit
#1 Increment #1
Safety Audit
#1
Increment #2 Safety Audit
#2
Increment #1 Safety Audit
#1
Increment #2 Safety Audit
#2
Increment #1 Safety Audit
#1
Increment #3 Safety Audit
#3
Model of Agile Development
Quality
Time Cost
Functionality
Fixed
Variable
Agile Development
No change to existing best practice
• Full traceability (requirements, design, code, test)
• Coding standards – Static analysis
– Complexity
– Module size
• Unit Test – full path coverage
• Independent reviews
• Test Driven Development (TDD)
• Automated overnight build and test
• Strict configuration control and change control
14
Management of Risk
• Agile provides early tangible working product – Evidence based progress
– Avoids the “90% complete” syndrome
– Provides for re-prioritisation
• Overall risk is progressively reduced throughout the development
• Risk exposure is limited to the cost of the current increment
• Lessons Learned from one increment are passed to the next
Risk Over Time
Time
Risk
Delivery Deadline
Waterfall
Agile
Increment #1
Increment #2
Increment #3
Progress Monitoring
• Traditional Gantt Charts are only useful at a high level
• Daily Stand-Up meetings – Provides an environment for communication and team building – Each team member provides a verbal update to the rest of the
team
• Wall boards show – The Workflow – Who is doing what – Where the progress blocks are
• Burn-down charts – Shows how fast work is being performed (velocity) – Provide a forecast completion date
Progress Monitoring: Burn-Down Charts
To Do
Complete
In Progress
Estimated Finish #1
Estimated Finish #2
Date
Number of
Reqmts
Estimated Finish #3
Actual Finish
Agile for Safety-Critical Rail
• Autonomous Underground Train Control system • Real-Time Safety-Critical System
– CENELEC 50128 (SIL4) – Loss of Life
• Automatic control of – train, signals, points
• Radio based communication between – Trains – Timetable (including local speed restrictions)
• Doppler radar – Provides speed, distance & direction
• Axle counters – to determine train position in station
FBP: System Layout
Radio
Interlocking
Control Centre
ATP ATO
Radio Links
Communications
Signals & Points
Radio
BP
BP Fixed Block Processor
Interlocking
ATP ATO
Train driving
Initial Development Process
Detailed Design
Code
Review & Safety Audit
Review & Safety Audit
Requirements Definition
High-Level Design
Unit Test
Integration Test
Acceptance Test
System Test
Review & Safety Audit
Review & Safety Audit
Review & Safety Audit
Review & Safety Audit
Review & Safety Audit
Development Process
Detailed Design
Code
Requirements Definition
High-Level Design
Unit Test
Integration Test
Acceptance Test
System Test
Code
Detailed Design
Unit Test
Detailed Design
Code
SPARK
Unit Test
Increments
SPARK
Code
Detailed Design Detailed Design
Code
SPARK
Unit Test
Code
Detailed Design
SPARK SPARK
Unit Test
Code
Detailed Design
Unit Test Unit Test
Detailed Design
Code
Unit Test
SPARK
Detailed Design
Code
Unit Test
SPARK
Detailed Design
Code
Unit Test
SPARK
Incremental Development
Time
Review Previous Increment
Plan this Increment
Overall Requirements
Backlog
Design
Code
Test
SPARK
Increment Requirements Backlog
Process Improvements
Incremental Development
Design
Code
Test
SPARK
• Full Traceability – Requirements – Design – Code – Test
• Independent Unit test – 100% path coverage – MC/DC testing – Boundary values
• Independent Formal Reviews
• Incremental Hazard Analysis
25
Conventional Safety Analysis Process System Hazards & Safety Constraints
New Hazards
New Hazards
Vertical Slice Analysis
Known Hazards
Known Hazards
Design Verification Safety Analysis
Code Verification Safety Analysis
Safety Requirements Verification Analysis
Safety Audit Report
Phase Specific Safety Reports
Safety Analysis
Safety Analysis
Design
Code
Test
Safety Analysis
FBP: Increment based Safety Analysis System Hazards & Safety Constraints
Vertical Slice Analysis
Increment
Unit Test Safety Analysis
Code Verification Safety Analysis
Design Verification Safety Analysis
Safety Audit Report
(Increment)
Design
Code
Test
New Hazards Safety
Analysis
New Hazards
Phase Specific Safety Reports
Safety Analysis
New Hazards
FBP Burn-Down: Testing
0
100
200
300
400
500
600
700
800
900
No
. o
f M
od
ule
Tests
Weeks
FBP: Module Testing (Formal)
Complete
Progressing
To Go
Total Tests
28
FBP: Retrospective Project Analysis
Development Team Size = 70+ (35 UK, 35 India, 4 Spain)
Primary project objective = Timescale
Crude industry standard = 22 ↔ 24 months duration
Actual development = 18 months
Cost (Effort)
Time
FBP
Approaching Agile
• Fundamentally Agile is a mind-set
• It is about managing project risk in order to deliver business benefit
• Agile is not proscriptive
– Best practice in all activities
– No conflict with current industry practice
• Agile advocates a number of methods, techniques and approaches that deliver business benefit
• It is up to you to tailor these to your need
Any Questions?
David Nicoll
www.project-realization.com