Page 1
1
Systems-of-Systems Assurance
Taz Daughtrey
Cyber Security and Information Systems Information Analysis Center
10 June 2014 webinar System of Systems Engineering Collaborators
Information Exchange
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Page 2
2
Let’s consider ….
What systems? What assurance?
Challenges
Responses
The Way Forward
Page 3
3
Air Traffic Control System
Page 4
4
Satellite Communication System
Page 5
5
Satellite Imaging System
Page 6
6
Military Transportation System
Page 7
7
Need to “stitch” these systems together
ad-hoc system of systems
Page 8
8
Search-and-Recovery System
Page 9
9
Beware of the “stitches”
Page 10
10
Beware of those doing the “stitching”
Page 11
11
Communication and Decision Making are the “stitches”
Page 12
“Take the red pill … See how deep the rabbit hole goes”
Page 14
14 directed
acknowledged
collaborative
virtual
Page 15
15
Software Engineering Institute:
“Mission Threads”
Page 16
16
Also need:
“Assurance Threads”
Page 17
17
Scenario-based
and
Page 18
Assurance
It’s all a “confidence” game.
Providing adequate confidence that …
… product requirements are being satisfied.
… project plans are being actualized.
… stakeholders’ interests are being honored.
Page 19
19
critical
moderate
low
Requirements Assurance
Page 20
20
Let’s consider ….
What systems? What assurance?
Challenges
Responses
The Way Forward
Page 21
Meeting stakeholders’ expectations
Page 31
System shall do …
System shall not
do …
Acceptable behavior Unacceptable behavior
System might do …
Page 32
Security Requirements
confidentiality
32
Page 33
33
Let’s consider ….
What systems? What assurance?
Challenges
Responses
The Way Forward
Page 34
“We must run as fast as we can, just to stay in place.
And if you wish to go anywhere, you must run twice as fast as that.”
Page 35
static
inspections
walkthroughs
audits
reviews
assessments
35
Page 36
prototyping
simulation
unit testing
integration testing
system testing
acceptance testing
dynamic assessments
36
Page 38
Testing
Lifecycles
-- IEEE 1012
38
Page 40
40
Let’s consider ….
What systems? What assurance?
Challenges
Responses
The Way Forward
Page 41
41
effective
efficient
satisfactory risk-free
contextual
Quality in Use
Quality in Use
Page 42
42
functional
efficient
compatible
usable
reliable
secure
maintainable
portable
Product
Quality
Page 43
43
functional
efficient
compatible
usable
reliable
secure
maintainable
portable
Page 46
Set measureable dependability targets.
Design. Implement. Build in dependability.
Conduct appraisals. Identify opportunities. Release? Rework?
Improve processes.
Standards Best Practices
Professional Communities
46
Page 49
Costs of meeting requirements
COST OF QUALITY
Costs of not meeting requirements
Prevention
Appraisal
Internal failures
External failures
49
Page 50
COST OF QUALITY
Prevention Planning
Training
Tools
Appraisal Inspections
Audits
Tests
Page 51
COST OF QUALITY
Internal failures Scrap
Rework
External failures Warranty
Liability
Loss of reputation
Page 52
consequence
Likelihood
2 3 4 4 4
1 3 1 1 3
negligible catastrophic
infrequent
reasonable
Page 54
54
Systems-of-Systems Assurance
Taz Daughtrey [email protected]
434 841 5444
Cyber Security and Information Systems Information Analysis Center
Page 55
"Everybody has won and all must have prizes.“