safety and security considerations for software and ... · safety and security considerations for...
TRANSCRIPT
![Page 1: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/1.jpg)
Safety and Security Considerations for Software and
Digital Hardware Verification
John Colley, Michael Butler
Verification Futures Conference
Reading, 6 April 2017
![Page 2: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/2.jpg)
Acknowledgements
2
This work was made possible by the ATI SECT-AIR project funded by
Innovate UK. The participants in the SECT-AIR project are
![Page 3: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/3.jpg)
OUR VISION OF AN AUTOMATED FUTURE
Zero accidents. Mobility for all.
www.intel.com
3
![Page 4: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/4.jpg)
Introduction
• Motivation
• Standards and Guidelines
• System Theoretic Process Analysis (STPA)
• A Case Study: applying STPA and Formal Analysis
• Safety vs Security Considerations
• Organisational Concerns
• Summary
4
![Page 5: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/5.jpg)
Motivation
• Safety and Security are emergent properties of the SYSTEM
– Must be verified at the system level
• Traditional verification techniques do not scale well to the system level
• Abstraction and Formal Analysis are increasingly being adopted
– Clock-Domain Crossing Verification
– C-RTL Formal Equivalence Checking
5
![Page 6: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/6.jpg)
Where to Begin?
Formal Analysis
Abstraction
Safety
Security System Level Verification
6
![Page 7: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/7.jpg)
Standards and Guidelines
• DO-326A (2014) Airworthiness Security Process Specification – Tackles Integration of Security Engineering with
Safety Engineering – Links the Security Process, the Safety Assessment
Process (ARP-4761) and the Systems Engineering Process (ARP-4754A)
• ISO/AWI 21434 (under development) Road Vehicles – Automotive Security Engineering
• NIST 800-160 (2016) Systems Security Engineering Considerations for a Multidisciplinary Approach in the Engineering of Trustworthy Secure Systems – Complements ISO/IEC/IEEE 15288:2015, Systems and
software engineering – System life cycle processes 7
![Page 8: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/8.jpg)
NIST 800 160
“security is defined as freedom from those conditions that can cause loss of assets with
unacceptable consequences”
8
“the system protection capability is a system control objective
and a system design problem ”
![Page 9: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/9.jpg)
NIST 800 160 “Trustworthiness”
“Trustworthiness requirements can include, for example, attributes of safety, security,
reliability, dependability, performance, resilience, and survivability under a wide
range of potential adversity in the form of disruptions, hazards, and threats.”
ASSURANCE CASE
“Effective measures of trustworthiness are meaningful only to the extent that the
requirements are sufficiently complete and well-defined, and can be accurately assessed.”
9
![Page 10: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/10.jpg)
• Provides guidance to software developers wishing to use formal methods in the certification of airborne systems
• Identifies the
– Modifications
– Additions
to DO-178C
– Objectives
– Activities
– Software Lifecycle Data
including the artifacts expressed in formal notation and the verification evidence (Section 6 DO-178C) that could be derived from them
RTCA DO-333 Formal Methods Supplement to DO-178C
10
![Page 11: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/11.jpg)
DO-333 Table FM.A-3
11
NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
![Page 12: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/12.jpg)
• Hazard Analysis Technique
– Identify potential causes of accidents: scenarios that can lead to losses
– Eliminate or Control hazards in design or operations before damage occurs
• Causal factors
– Design errors including software flaws
– Component interaction accidents
– Human decision-making errors
– Social, organisational and management factors contributing to accidents
• Also applicable for Security Analysis
System Theoretic Process Analysis (STPA)
Engineering a Safer World, Leveson MIT 2011 12
![Page 13: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/13.jpg)
The Two Phases of STPA
1. Identify Potentially Hazardous Control Actions and derive the Safety Constraints
2. Determine how Unsafe Control Actions could occur
Engineering a Safer World, Leveson, 2012 13
![Page 14: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/14.jpg)
The Two Phases of STPA
1. Identify Potentially Vulnerable Control Actions and derive the Trustworthiness Constraints
2. Determine how Untrustworthy Control Actions could occur
Engineering a Safer World, Leveson, 2012
14
![Page 15: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/15.jpg)
Compliance Traceability System Requirements to High-Level Requirements
focusing on the objectives of DO-333 Table FM.A-3
We use System Theoretic Process Analysis
(STPA) and
Formal Modelling to demonstrate compliance
15
![Page 16: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/16.jpg)
CASE STUDY
16
![Page 17: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/17.jpg)
“.. FGS system has two physical sides, or channels, one on the left side and one on the right side of the aircraft. These provide redundant implementations that communicate with each other over a cross-channel bus.” NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
Flight Guidance System Function
17
![Page 18: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/18.jpg)
R1. At least one side shall be the pilot flying side.
R2. At most one side shall be the pilot flying side.
R3. Pressing the Transfer Switch shall always change the pilot flying side.
R4. The system shall start with the Primary Side as the pilot flying side.
R5. The system shall not change the pilot flying side unless the Transfer Switch is pressed.
System-level Requirements (stated informally)
NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
18
![Page 19: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/19.jpg)
System Level Model
19
![Page 20: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/20.jpg)
Synchronous Implementation
20
NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
![Page 21: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/21.jpg)
• FGS has two physical sides, or channels, one on the left side and one on the right side of the aircraft. These provide redundant implementations that communicate with each other over a cross-channel bus.
• Each bus in the synchronous Pilot Flying example transfers its input to its output with a one-step delay
• Only the pilot not flying side listens for the Transfer Switch • If a side believes itself to be the Not_Pilot_Flying side, it will
become the Pilot_Flying side when it sees the Transfer_Switch pressed (i.e. a rising edge).
• If a side is the Pilot_Flying side, it will become the Not_Pilot_Flying side when it sees the other side become the pilot flying side.
Synchronous Implementation – Informal Requirements
21
NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
![Page 22: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/22.jpg)
Control Action Not Providing Causes Vulnerability
Providing Causes Vulnerability
Wrong Timing or Order (too early, too late)
Stopped too Soon or Applied too Long
Assert pilot flying V1 Request for control not issued when the transfer switch has been pressed
V3 Request for control issued when the transfer switch has not been pressed
V5 Too Early: As V3 V6 Too Late: Delay of requested switch
V9 Stopped too Soon: Other side does not see request for control V10 Applied too Long: As V2
De-assert pilot flying
V2 Side does not acknowledge that it has relinquished control when requested to
V4 In control but signal issued to the contrary
V7 Too Early: As V4 V8 Too Late: Delay of requested switch
V11 Stopped too Soon: As V2 V12 Applied too Long: As V6
STPA Step1: Identify Vulnerabilities
22
Vulnerabilities can lead to Failure • Loss of required function • Mode Confusion: the crew do not know what state the system is in,
which could lead to loss of the aircraft
![Page 23: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/23.jpg)
C1. pilotflying must not be asserted by a controller unless
a rising edge has been received on transferswitch
and the controller’s state is not_flying V3, V5
C2. pilotflying must not be de-asserted by a controller unless
a rising edge has been received on the bus
and the controller’s state is flying V4
C3. pilotflying must be asserted by controller in the same cycle
when a rising edge is received on transferswitch
and controller state is not_flying V1,5,6,9,10
C4. pilotflying must be de-asserted by controller in the same cycle
when a rising edge is received on the bus and
controller state is flying V2,7,8,11,12
C5. controller l state and controller r state never the same V3-12 R1, R2
Controller Constraints
R5
R3
23
![Page 24: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/24.jpg)
Refine System Model to two Synchronous Processes
⊆ Safety Constraint C5
lstate ≠ rstate
24
![Page 25: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/25.jpg)
STPA Step 2: Determine how Untrustworthy Control Actions could occur
• In Step 1, we considered the Vulnerabilities
• In Step 2, we analyse the Threats that expose/exploit these Vulnerabilities
– Unintentional (safety)
– Intentional (security)
25
![Page 26: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/26.jpg)
STPA Step 2: Determine how Untrustworthy Control Actions could occur
NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
Spurious request to take control
Spurious response to relinquish control
SAFETY: Consider each vulnerability in isolation SECURITY: A concerted attack could exploit both vulnerabilities in sequence
26
![Page 27: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/27.jpg)
Safety-Related Security The Policing Function
NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014
Spurious request to take control
Spurious response to relinquish control
High Integrity Formal Model
Developed and Verified With Independence
27
![Page 28: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/28.jpg)
Organisational Concerns
Project Development
Verification Function
Safety Function Security Function
Only the Verification Function can determine that the Safety AND Security requirements have been met
28
![Page 29: Safety and Security Considerations for Software and ... · Safety and Security Considerations for Software and Digital Hardware Verification ... •NIST 800-160 (2016) Systems Security](https://reader035.vdocuments.us/reader035/viewer/2022063013/5fcc75d949dea5175020a0b1/html5/thumbnails/29.jpg)
Summary • NIST 800-160 and DO-178C / DO-333 drive the
Development and Verification Processes • STPA provides a System-Level method for
– Identifying the Vulnerabilities – Determining the Safety and Security Constraints – Determining how unsafe control actions could occur
• The Policing Function – Is developed Formally from the Requirements – Can be used to monitor control processes which are too
complex to be verified formally – Considers multiple vulnerabilities to mitigate against concerted
attacks – Ensures that the Safety and Security Constraints are observed – Mitigates against mode confusion
• An Independent Verification Function – Ensures that Safety and Security Requirements have been met
29