assignment of design assurance levels - microway … steve...assignment of design assurance levels....

August 20, 2008 FAA Software & AEH Conference 1

SAE S18 & EUROCAE WG-63

Presented bySteve Beland

Associate Technical FellowBoeing Commercial Airplanes

Assignment of Assignment of Design Assurance LevelsDesign Assurance Levels


Status of New Draft GuidanceSAE S-18 Airplane Safety Assessment Committee updating SAE ARP 4754

Subcommittee: Fellowship of the DAL (FotDAL)

Regular coordination with EUROCAE WG-63Mature text inserted as Sec 5.2 in Draft G2, S18 meeting Jan ’08

Delicate consensus, WG-63 participated remotelyEstablished baseline

Draft H2 contains April joint meeting effortFlowchart addedConsensus strengthened, FotDAL done


FotDAL Problem StatementInitial DAL has not always been based on rigorous safety analysisDelineation of the architectural containment boundaries were not always properly definedInconsistency between guidance sourcesDifficult to delineate the subtleties between “independence” and “dissimilarity”Probabilities have often been improperly linked to development assurance levelsPresent ARP 4754 Section 5.4 does not clearly distinguish between loss vs. erroneous output of a function (but it is there “between the lines” of Table 4)


SituationUse of ARP4754 is becoming more common

ARP4754 was published in 1996Referenced in AMC 25.1309, AC 23.1309-1C and draft 25.1309 (Arsenal) published in 2002

FAA Policy Memo allows ESF for Part 25Section 5.4 permits reduced level of process rigor for development assurance activities provided proper safety analysis shows architectural containmentSome perceived inconsistencies between ARP4754 and DO-178B and DO-254 exist (Ref. FAA Policy Statement ANM-10-117-09, Jan 2004)


Related Changes to ARP4754Revision A restructured to include both the aircraft and system development life cycle process modelsDocument outline has a smoother flow Sections 4 through 10 combined into 2 sections

Section 4 outlines the aircraft/system development process modelSection 5 addresses the integral processes used during aircraft/system development

New Section 5.2 is influenced by existing ARP 4754 Table 4 and the Functional Failure Path Analysis concept in RTCA DO-254 Appendix B

Includes constraints to preclude extreme FFPA interpretations


Development vs. Design AssuranceDevelopment Assurance:

All of those planned and systematic actions used to substantiate, at an adequate level of confidence, that errors in requirements, design and implementation have been identified and corrected such that the system satisfies the applicable certification basis. Note: The development assurance of an item has sometimes been called design assurance, such as in RTCA/EUROCAE DO-254/ED-80.

Development Assurance Level (DvAL)Applies to aircraft and systemsV&V Process defined in SAE ARP4754AIncludes Safety AnalysisInfluences Architectures


Development vs. Design AssuranceDesign Assurance:

All of those planned and systematic actions used to substantiate, at an adequate level of confidence, that design errors have been identified and corrected such that the items (hardware, software) satisfy the applicable certification basis.

Design Assurance Level (DsAL) Applies to items (software / hardware)V&V Process defined in RTCA DO-178 & DO-254


n Sy

stem

s in

Airp

lane

m It

ems

in n

Sys

tem

sA

irpla

ne S

yste

ms

Layered Development Life Cycle

Aircraft Function

DvAL

System Function

DvAL

Item DsAL


Assurance Activities

Assurance ActivityAt system level ARP 4754/ED-79

At Hardware Level DO-254/ED-80

At Software Level DO-178B/ED-12B

FHA Yes None None

PSSA/SSA Yes None None

FMEA Yes Implied via Functional Failure Path Analysis.

None

Common Cause Analysis (including SW and complex HW common mode failures)

Yes None None

Requirement Capture Yes Yes YesRequirement Validation Yes Yes, for HW specific

requirementsYes, for SW specific

requirements

Implementation Verification

Yes Yes Yes

Configuration management

Yes Yes Yes

Process assurance Yes Yes Yes


FotDAL Approach to Assigning LevelsExisting ARP explicitly addresses system level (with only some mention of airplane level)

Level assignment/reduction applied to items defined from the system architecture

New ARP addresses airplane & system levels explicitlyDvAL is effectively new for the new ARP and should be assigned to the systems from the aircraft architecture using the PASA/PSSADsAL assignment should be similar to existing ARP

Discusses “independence” rather than “dissimilarity”Emphasize assigning levels rather than reducing levels

“Reduction” is a misnomer, but arises when a function has a level lower than its parent function


ARP4754A Draft Outline

4 AIRCRAFT & SYSTEM DEVELOPMENT PROCESS4.1 Conceptual Aircraft/System Development Process4.2 Aircraft Function Development4.3 Allocation of Aircraft Functions to Systems4.4 Development of System Architecture4.5 System Implementation

5 INTEGRAL PROCESSES5.1 Safety Assessment5.2 Development Assurance Level (DvAL) Assignment5.3 Requirements Capture5.4 Requirements Validation5.5 Implementation Verification5.6 Configuration Management5.7 Process Assurance5.8 Certification and Regulatory Authority Coordination


System Development Process Model

CONCEPTDESIGN

FUNCTIONDEVELOPMENT

FUNCTIONALLOCATION

ARCHITECTUREDEVELOPMENT

REQUIREMENTALLOCATION

SYSTEMIMPLEMENTATION

PLANNING

- 5.1 SAFETY ASSESSMENT- 5.2 DEVELOPMENT ASSURANCE- 5.3 REQUIREMENTS CAPTURE- 5.4 REQUIREMENTS VALIDATION- 5.6 CONFIGURATION MANAGEMENT- 5.7 PROCESS ASSURANCE- 5.8 CERTIFICATION & REGULATORY GUIDANCE COORDINATION

AIRCRAFT/SYSTEMDEVELOPMENT PROCESS

4.1

4.2 4.3 4.4 4.5 4.6

3.0

DATA &DOCUMENTATION

6.0

5.5 IMPLEMENTATION VERIFICATION

INTEGRAL PROCESSES

Figure 4-2


5.2 Assurance Level Assignment

5.2.1 Introduction 5.2.2 General Principles5.2.3 Independence Attributes5.2.4 DvAL & DsAL Assignment Guidelines


DvAL/DsAL Assignment Process

Figure 5-2


TerminologyFunctional Failure Set:

A single member, or a specific group of members (not necessarily limited to one system) whose anomalous behavior (random failure or systematic error) leads to a top level Failure Condition. An FFS member can be related to a function, a sub-function, or an item.


DvAL Assignment General PrinciplesCatastrophic Failure Condition:

At least 1 development process is Level A, or at least 2 independent development processes are Level B, but none any lower than their individual hazard and no lower than Level COverall integration process is Level A

Hazardous Failure Condition:At least 1 development process is Level B, or at least 2 independent development processes are Level C, but none any lower than their individual hazard and no lower than Level DOverall integration process is Level B


Independence AttributesFunctional: members with different fcns & reqts

Common requirements errorsRequirements interpretation errors

Design: members with different designsHardware or software design errorsSoftware language or HDL errorsDesign tool errors

Others: do not influence DvAL/DsAL assignmentPhysical

Redundancy, installationProcess

Between independent designs or functionsBetween development/design vs. verif/validation


Independence AttributesDvAL considers the functional independence of the aircraft (or system) functionsDsAL considers the design independence of itemsOnce the DsALs are assigned to items, they should be fed back to the system and aircraft processes to ensure that no common mode is inadvertently introduced that violates any claimed functional independence.The assertion of independence needs to be substantiated & address potential common-modes One type of independence does not necessarily imply the other


DvAL/DsAL Assignment ProcessDevelopment Assurance Level (DvAL)

Aircraft and system levels assigned within Aircraft Level FHA & PASAValidated with Aircraft & System level Safety Analysis

Design Assurance Level (DsAL)Assigned from System Level FHAs & PSSAsValidated per System level Safety Analysis and Component Functional Failure AnalysisMust trace up to upper level functions’ DvAL so that it is not decomposed/assigned more than once (e.g. 4 Level D items assuring a Level A aircraft function).Non-complex items that are fully and deterministically tested and analyzed may be considered Level A


Table 5-2: DvAL Assignments (paraphrased)

Two or more Independent FFS MembersOne FFS

Members Option 1 Option 2

CAT A A for 1 member, more members assigned per its failure cond (but ≥C).

B for 2 members, more members assigned per its failure cond (but ≥C).

HAZ B B for one member, more members assigned per its failure condition (≥D)

C for 2 members, more members assigned per its failure condition (≥D)

MAJ C C for one member, more members assigned per its failure condition

D for 2 members, more members more per its failure condition

MIN D D for one member, more members assigned per its failure condition

members assigned per its failure condition

None E E E

Top Event


DEVELOPMENT ASSURANCE LEVEL

functional failure sets with multiple independent members(note 2)

functional failure sets with a single member or with non-independent members

Option 1 (NOTE 3)

Option 2

Catastrophic DvAL A(NOTE 1)

DvAL A for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level C for the additional members).

DvAL B for two of the members leading to top level failure condition. The other member(s) at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level C for the additional member(s)).

Hazardous DvAL B DvAL B for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level D for the additional members).

DvAL C for two of the members leading to top level failure condition. The other members at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level D for the additional members).

Major DvAL C DvAL C for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions.

DvAL D for two of the members leading to top level failure condition. The other members at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions.

Minor DvAL D DvAL D for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions.

DvAL of each member at the level associated with the most severe individual effects of an error in their development process

No Safety Effect DvAL E DvAL E DvAL E

Top Event / Failure Condition

Classification


Notes after DvAL/DsAL Table 1. When a FFS has a single member and the mitigation strategy for

systematic errors is to be DvAL A alone, then the applicant may be required to substantiate to the Certification Authorities that the applicant’s development assurance activities have the rigor to provide for an acceptable level of safety.

A single Level A member is OK, but may get more scrutiny2. When decomposing an aircraft function it is necessary to stay in the

same row no matter the number of functional decompositions performed (e.g. any degree of decomposition from a level A function should include at least one level A or 2 level B functions

3. If there is a large disparity on the numerical availability of the members in the functional failure set, the higher level DvAL should generally be assigned the higher availability

4. Some classes of 14CFR Part 23 /CS-23 aircraft will allow DvALslower than derived in this process. See the current FAA AC23.1309 and EASA policy for guidance.


DsAL Assignment Cases: 0 No independence:

Assign DvAL & DsAL for top FC (DvAL = DsAL)1 Both functional & design independence:

Assign DvAL using Table 2, then DsAL using Table 2 staying in the same row as the top failure conditionWatch cross-products of one member’s DvAL and another member’s DsAL; re-assess assignment for FFS not meeting general principles

2 Functional but no design independence:DsAL of common items per top-level failure conditionEnsure partitioning ensure functional independence? May lead to re-evaluation of DvAL if not

3 Design but no functional independence:Assign DsAL using Table 2, staying in the same row as the top-level failure condition


Level Assignment

Flow Fig 5-5

(1 of 2, DvAL)CAN

FUNCTIONAL INDEPENDENCE

BE CLAIMED? (5.2.4.1)

FAILURE CONDITIONS FROM

THE FHA

SELECT ONE (OR NEXT) FC AND ITS FFS

CAN DESIGN

ASSIGN DvAL = TOP DvAL PER COL 2 OF

TABLE 2

ASSIGN DvAL PER OPTION 1 OR 2 OF

TABLE 2

ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO

HAVE ALL FCs FOR

THIS FUNCTION BEEN

ASSESSED?

FUNCTION DvAL

ITEM DsAL

COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs.

NO

NO

YES

YES

ASSIGN DvAL TO AIRCRAFT-LEVEL FUNCTION (TOP DvAL) PER TABLE 1

Function DvAL

Item DsAL


Level Assignment

Flow Fig 5-5

(2 of 2, DsAL)

CAN DESIGN

INDEPENDENCE BE CLAIMED?

(5.2.4.2)

ASSIGN ITEM DsAL = TOP DvAL (5.2.4.2.2)

ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO

DETERMINE DvAL; CHECK CROSS-PRODUCTS OF DvAL

& DsAL IF DsAL DIFFERS FROM ITS DvAL (5.2.4.2.1, .3)

THIS FUNCTION BEEN

ASSESSED?

HAVE ALL FCs FOR THIS

ITEM BEEN ASSESSED?

FUNCTION DvAL

ITEM DsAL

COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs.

FINAL DsAL MUST SATISFY ALL FCs (AND MAY NEED TO BE REASSIGNED FOR COMBINATION OF ALL APPLICABLE FCs)

NO

NO

NO

YES

YES

YES

Function DvAL

Item DsAL


CA

N

FUN

CTI

ON

AL

IND

EP

EN

DE

NC

E

BE

CLA

IME

D?

(5.2

.4.1

)

FAIL

UR

E

CO

ND

ITIO

NS

FR

OM

TH

E F

HA

SEL

EC

T O

NE

(OR

NEX

T) F

C

AN

D IT

S F

FS

CA

N

DES

IGN

IN

DE

PE

ND

EN

CE

B

E C

LAIM

ED

? (5

.2.4

.2)

ASS

IGN

DvA

L =

TOP

D

vAL

PE

R C

OL

2 O

F TA

BLE

2

ASS

IGN

DvA

L P

ER

OP

TIO

N 1

OR

2 O

F TA

BLE

2

AS

SIG

N IT

EM

DsA

L =

TOP

DvA

L (5

.2.4

.2.2

)

ASS

IGN

DsA

L P

ER O

PTIO

N 1

O

R 2

OF

TAB

LE 2

US

ING

SA

ME

RO

W A

S U

SE

D T

O

DE

TER

MIN

E D

vAL;

CH

EC

K

CR

OS

S-P

RO

DU

CTS

OF

DvA

L &

DsA

L IF

DsA

L D

IFFE

RS

FR

OM

ITS

DvA

L (5

.2.4

.2.1

, .3)

HA

VE

ALL

FC

s FO

R

THIS

FU

NC

TIO

N

BE

EN

ASS

ESS

ED?

HA

VE

ALL

FC

s FO

R T

HIS

IT

EM

BE

EN

ASS

ESS

ED?

FUN

CTI

ON

DvA

L

ITE

M D

sAL

CO

MP

ILE

A L

IST

OF

ALL

FU

NC

TIO

NS

AN

D T

HEI

R D

vALs

FI

NA

L D

vAL

MU

ST

SAT

ISFY

ALL

APP

LIC

ABLE

FC

s.

FIN

AL

DsA

L M

US

T S

ATIS

FY A

LL F

Cs

(AN

D M

AY

NE

ED

TO

BE

RE

ASS

IGN

ED

FO

R

CO

MB

INA

TIO

N O

F A

LL A

PP

LIC

ABL

E F

Cs)

NO

NO

NO N

O

YE

S

YES

YES

YES

ASS

IGN

DvA

L TO

AIR

CR

AFT

-LE

VEL

FUN

CTI

ON

(TO

P D

vAL)

PE

R T

AB

LE 1

Whole Figure 5-5 For Your Handouts


Special Cases:Non-complex items (e.g. mechanical parts, relays, electro-mechanical devices, electro valves, servo valves, simple logic devices, etc.) when the requirements that are applicable to the design have been validated and the analysis and testing of the design for verification with those requirements are considered to provide a level of confidence equivalent to DsAL A. However, this is only when the non complex item is fully tested or fully analyzed relative to the identified failure conditions. Independent functions using common resources based on the same COTS designs (e.g. computers, networks, interfaces) are likely to require careful consideration if assigned a DsAL of A. A practical case of functional and design independence is whereby independent functions are implemented in designs that are independent from one another.


DvAL Considerations for Conditional Events

Arises for protection systems for environmental or intrinsic threats

e.g. windshear, fire

DvAL should be consistent with reduction in safety marginsDoes not apply to operations intentionally planned (e.g. autolands, ETOPS missions)


DvAL Assignment as a Function of an External Event

A

B

C

10-910-710-510-3

DvAL

Probability of the external event

Legend: CAT Top Level Failure Condition HAZ Top Level Failure Condition

10-4 10-6

Figure 5-5


Outputs of DvAL/DsAL AssignmentDescribe FFS members and explain their DvALs in Cert Plan and PASA/PSSADescribe items and explain their DsALs in applicable plans and PSSADescribe interactions between FFS members in PASA/PSSA

Functional interactionsSubstantiate claimed functional & design independenceCapture assumptions made in the supporting analysesDerive requirements needed to support assumptions and analyses.

Substantiate that all planned activities are successfully accomplished in the ASA/SSA and Cert Summary.


CAST Comments on Draft HComments received are in-work by S18 & WG-63Some clarifications are neededA couple significant misunderstandings arising:

2 Level Bs don’t easily make a Level A functionEven if the item (DsAL) can be Level B, the system & aircraft functions still need to be developed & integrated at DvAL ANot intended to be endorsement of n-version programming

Concern of entering Table 5-2 more than once may lead to 4 Cs to meet Level A, etc.

Text disallows moving from row to row, each use must stay in the same row.

S18/WG-63 will disposition comments & reply


More Difficult Than Algebra

First we had Algebra in math classThen Boolean Algebra in collegeIntroducing……..


DALgebraA = A*C = B*B B = B = B*D = C*CSubstituting…….B*D*C*C = A?


ClosingDvAL and DsAL are based on safety analysis; do it early (and revisit it often as the system evolves)!Emphasize assigning levels rather than reducinglevelsPASA and PSSA should be used to derive requirements including DvAL/DsALDevelopment/Design Assurance can be an enabler to focus resources on aspects that matter most. Aim is to have consistent guidance in one place for use across a systemThe DsAL assigned to software or hardware should be about the same as determined when using existing ARP 4754.

assignment of design assurance levels - microway … steve...assignment of design assurance levels....

Documents