sas_08_automation_for_system_safety_analysis_malin 1 automation for system safety analysis jane t....

SAS_08_Automation_for_System_Safety_Analysis_Malin1

Automation for System Safety Analysis

Jane T. Malin, Principal Investigator

Project: Automated Tool and Method for System Safety Analysis

Software Assurance SymposiumSeptember, 2008

Complex systems typically fail because of the unintended consequences of their design, the things they do that were not intended to be done. - M. Griffin, System Engineering and the “Two Cultures” of Engineering, March 28, 2007


Overview

• Problem/NASA Relevance• Technical Approach and Overview• 2008 Target Capability • Crew Exploration Vehicle (CEV) Launch Abort System Case• Data – Constellation (Cx) failure modes and effects

analysis/critical items lists (FMEA/CILs)• Technical Challenges

– Information Extraction – Semi-Automated Model Construction– Analysis and Test Case Generation

• 2009 Planned Capability• Potential Applications


Problem and NASA Relevance• NASA needs early evaluation of software (SW)

requirements and design, to reduce software-system integration risks– Assess system failures and anomalous conditions that may

challenge software in system integration testing– Identify robustness issues early (and often)– Identify requirements gaps early (and often)

• Project test case: NASA Constellation (Cx) Launch Abort System (LAS) for Pad Abort

PLANT and Environment SOFTWARE‘Activate’ Faults and

Influence Failures

‘Activate’ Faults and Influence Failures

Operations and Stresses

FAULTS/ReliabilityFAULTS/Reliability


Technical Approach

Systematic semi-automated extraction and analysis for early evaluation and rapid update– Capture model of the controlled system architecture

• Abstract physical architecture models with subsystems, functions, interfaces, connections

• Extract directly from requirements and design text and data– Capture risks and hazards in model

• Constraints, hazards, risks from requirements and design • Risk and failure libraries

– Analyze and simulate to identify risks and constraints• Analyze and simulate hazard/risk propagation in the system• Use operational and off-nominal scenarios and

configurations– Identify possible test scenarios for virtual system

integration testing


Technology Overview

Extract and Model

Analyze, Simulate and Test Early

- Identify interaction-propagation paths

- Investigate influence of timing

- Perform Virtual Tests

Requirements Text

Hazard Identification Tool (HIT)

CONFIG Hybrid Simulation

Virtual System Integration Laboratory (VSIL)

Information Extractor Identify Test Cases

Aerospace Ontology


Modeler: Architecture Model and Visualization of a Set of Requirements

[C.1] Telecommunication Subsystem (TeleSub)• [C.1.1] The CDHC sends the TeleSub a compressed

picture. [FG.1] [TeleSub C.1.4]• [C.1.2] The CDHC sends the TeleSub telemetry. [FG.2]

[FR.1] [FR.5] [TeleSub C.1.5] • [C.1.3] The CDHC sends In View of Ground alerts to the

TeleSub. [DP.5.6] [TeleSub C.1.6]• [C.1.4] The CDHC receives plan files from the TeleSub.

[FR.3] [TeleSub C.1.3]• [C.1.5] The CDHC receives ground commands from the

TeleSub. [FR.3] [TeleSub C.1.2]

• [C.1.6] The CDHC receives the TeleSub operating state

from the TeleSub. [DP.5.5] [TeleSub C.1.1] …

[C.2] Camera Subsystem• [C.2.1] The CDHC sends the Camera a "take picture"

command. [FG.2] [FR.1] [FR.3] • [C.2.2] The CDHC sends the Camera x, y and z gimballing

coordinates. [FG.2] [FR.1] [FR.3] • [C.2.3] The CDHC sends a turn on command to the

Camera. [DP.5.3] [H Constraint 1.1.4]• [C.2.4] The CDHC sends a turn off command to the

Camera. [DP.5.3] • [C.2.5] The CDHC receives a compressed picture file from

the Camera. [FG.1] [FG.2] [FR.1]

…

[C.4] Attitude Determination Subsystem (ADS)• [C.4.1] The CDHC receives an In View of Ground alert from

the ADS. [DP.5.6] [ADS]• [C.4.2] The CDHC receives the ADS operating state from

the ADS. [DP.5.5] [ADS]

Note: CDHC is Command and Data Handling ComputerPhysical/Functional Architecture Visualization


CONFIG Simulation: Assess Timed Scenarios

CONFIG simulation tool used for software virtual validation testing for NASA 1997 90-day manned Lunar Life Support Test

• Software: Intelligent control for gas storage and transfer• Models: Gas volumes and processing systems controlled by software; mixed fidelity, discrete and continuous • Testing: Simulated failures and imbalances that would not be tested in hardware-software integration

• Too slow to develop, too expensive, too destructive• Results: Identified software requirements deficiency due to unintended consequences of integrating gas processing systems


Virtual System Integration Lab (VSIL)

• Triakis has used VSIL in >25 avionics verification projects

• Project Output to VSIL: Models and test definitions

Models and Test Definitions

DE: detailed executable, the simulation of the embedded controller hardware RAM/ROM: memories

ES: executable specifications I/O: input/output

V&V: verification and validation CPU: processor


2008 Target Capability• Integration: Information extraction, architecture modeling and test

generation– Model parts extracted from requirements and FMEA/CIL texts

• XML output, including reference traces• Components, physical hierarchy, connections, interface components,

flows/resources, time or phase context• Functions, vulnerabilities, limits, failures, causes

– Ontology for model extraction and semi-automated modeling• Identify types of components, functions, problems, resources• Paths: A provides power to B; C receives command data from B• Functions and failures: B processes command data; B failure mode is No Output

command to D; cause of no output is B does not receive power.– Semi-automated model development from extracted model parts

• Component model library: Resource producer; Data processor…• Generic functions, failures and influences: Resource problem, Stressor, Data rate

problem, Data Integrity problem…– Model visualization for overview and completeness checking– Simulation and path analysis to identify hazardous configurations, scenarios

and test cases • Where failure or degradation of required functions results from unintended system

interactions

• Project Participants– CEV Flight Software Engineering, Abort Decision Logic, Abort Sequence– Orion Software Safety and Mission Assurance


CEV Launch Abort System (LAS) Case

Crew Exploration Vehicle (CEV) Pad Abort Sequence - notional


CEV Launch Abort System Case

• CEV Crew Module (CM) software controls the Pad Abort Sequence – LAS events, trajectory– No direct command feedback

• Components for the case– Paths and interactions for commands firing separation pyros during

pad abort sequence• CM computer → Remote Interface Unit → LAS Pyros• Possible Addition Case: Inertial measurement unit (IMU) → GN&C →

Abort motor– Summer Systems Engineering Intern manually built TEAMS models

of pyros and Remote Interface Unit (RIU) from FMEA/CILs• Need to perform simulation rather than pure path analysis on

the LAS case– Timing is important in aborts– Hazard Identification Tool (HIT) path analysis models do not

capture timing– CONFIG simulations can use timing


Data – ConOps, Requirements, Safety Analyses

• Developing tools and methods using documents and data from CEV sources, both NASA and Orion contractor

• Met with NASA expert on Orion software that controls Launch Aborts– Identified key CEV documents and confirmed analysis

approach• Orion Contractor’s Concept of Operations

– Best guess at the Abort Sequence• Interface Requirements Documents• Interface Control Documents, when they become available• Project Orion Flight System Safety Hazard Analysis• FMEA/CILs (preliminary now)

– Determined that much key information (sensors, feedback) is TBD, and being defined by the Cx Integrated Abort Team


Technical Challenges

• Limitations of early life-cycle requirements, design, hazard analysis and FMEA/CIL as sources for– Automatic extraction of model information from requirements

and design text– Semi-automatic construction of models from extracted

information, for simulation and visualization

• Combining graph analysis and simulation to identify possible hazard paths and off-nominal test scenarios for complex system interaction models

• Maturation Challenge: Develop mature software prototypes that can be used to develop products for broader use


Information Extraction• Objective: Extract information from CEV sources for semi-automated model

construction • Information Extraction Evaluation

– Success Criterion: % of available model information extracted, compared to % of model information available

• Types of Extractions (from text to XML)– Interface Requirements → Components, connections, flows/resources,

reference trace • Some interface components, vulnerabilities, functions, limits, context (time or phase)

– FMEA/CILs → Components, system to component hierarchy, interfaces, subcomponents, functions, failures, causes, effects

– Architecture Descriptions → Components, interfaces, hierarchy, functions; some design parameters, acronyms

• Challenges– Multiple document formats require definition of data structures for each

document, some difficult sentence parsing– Indirect access to Cradle requirements via PDF documents

• Progress: – Parser improved by incorporating NESC-funded parser from Univ. Central

Florida– Experience coding multiple document data structures, leading to format

specification approach– Successful extraction from Cradle-based PDF docs.


Example Information Extraction Benchmarking

LAS ComponentsComponent: Nose cone

Component: Canard Section Function: ? Component Enables Function: LAS: reorient the CM Function Enables Function: LAS?: deploy

parachute (following an abort) Component Group: Three propulsive motors Component: attitude control motor

Enables Function: LAS?: control attitude … Component: A bi-conic adapter Connection

Connector: bi-conic adapterFrom: LASTo: CMType: Structural

Component: Boost protective cover Acronym: Boost protective cover = BPC Design parameter: size

Determined by: ascent heating Function: protect CM thermal protection system coatings

Agent: Boost protective coverAction: protectOperand: CM thermal protection system coatings

Other: CM Component: Thermal protection system

Acronym: thermal protection system = TPS

LAS Description: The LAS consists of a nose cone, a canard section which enables the LAS to reorient the CM for parachute deployment following an abort, three propulsive motors (attitude control, jettison, and abort), a bi-conic adapter which provides the structural interface to the CM, and a boost protective cover (BPC) sized for ascent heating to protect CM thermal protection system (TPS) coatings.

Ideal Model Extraction Benchmark: Top Level: LAS Function: reorient the CM Agent: LAS

Action: reorientOperand: CM

Function: control attitude Agent: LAS?

Action: controlOperand: ?

Variable: attitude … Function: deploy parachute Agent: LAS?

Action: deployOperand: parachute


Model Construction• Objective: Use extracted model information

for semi-automatic model construction • Challenges

– LAS case with timing issues requires CONFIG simulation, not just HIT architecture model

– Missing information in pre-PDR documents• Operating modes, vulnerabilities, side effects etc.

• Progress– CONFIG provides visualization and supports

libraries of generic components– Concept for use of Aerospace Ontology

hierarchies with CONFIG library for generic components, operating modes, functions, side effects, problems


Analysis and Test Case Generation• Objectives:

– Identify and evaluate failure and hazard propagation in the system model• Unintended system interactions and unanalyzed propagation of failure

effects– Generate corresponding off-nominal test cases (configurations, scenarios)

• Evaluations – Hazard Analysis Success Criterion: Are new hazards and failures identified,

compared to standard method?– Test Case Success Criterion: Does model-based hazards and failures

analysis make test generation easier than current methods?• Challenges

– Path analysis algorithm for HIT models needs to be adapted, because LAS case will use CONFIG simulation rather than HIT

– Extracting information for operational and failure scenarios has not yet been addressed

• FMEA/CIL effects information can provide parts of failure scenarios• Progress

– Concept for combining HIT and CONFIG models, using CONFIG Inner Models for subsystem details


2009 Planned Capability

• Capabilities should be valuable from pre-PDR through operations

• Continue tool enhancements focusing on – Off-nominal test scenario discovery and evaluation– Component model library and generic defaults

• Use on a new CEV case – more complex interactions and more complete system information

• Deliver– Tool prototype files – Information extraction tool, Aerospace

ontology, HIT graph modeling and analysis, CONFIG simulation modeling, model libraries

– Documentation – methods, tools, user manuals


Future Applications

• Improve efficiency and repeatability of system and software risk analysis – Reduce time spent reanalyzing when specifications and designs

change• Visualize integrated requirements

– Combined success and failure spaces– Combined system and operation/event spaces

• Validate requirements and perform integration tests early with low-fidelity and multi-fidelity simulation

• Validate FMEAs and fault trees• Evaluate completeness and consistency of requirements

and risk– Support requirements traceability evaluations– Enhance analysis with reliability and event probability

information