eurocontrol safety case development manual ed 2.1 13-oct-2006

7/25/2019 Eurocontrol Safety Case Development Manual Ed 2.1 13-Oct-2006

1/66

EUROPEAN ORGANISATION FOR THE SAFETY OFAIR NAVIGATION

EUROCONTROL

EUROPEAN AIR TRAFFIC MANAGEMENT

Safety Case Development Manual

DAP/SSH/091

Edition : 2.1Edition Date : 13 Oct 2006Status : Released IssueClass : EATM Stakeholders


2/66


Page ii Released Issue Edition: 2.1

DOCUMENT CHARACTERISTICS

TITLE


EATMP Infocentre Reference:

Document Identifier Edition Number: 2.1

DAP/SSH/091 Edition Date: 13 Oct 2006

Abstract

This Safety Case Development Manual provides the reader with an overview of the methodologybeing proposed for the construction and development of Safety Cases. Version 2.0 is a completerewrite of Version 1.3 which was published in 2003, taking into consideration user needs and recentexperience with Safety Case developments.

This Manual includes the concept of a Safety Case as presenting the entirety of argument andevidence needed to satisfy oneself and the regulator with respect to safety. It does not provideguidance on the generation or documentation of the evidence itself. Where separate guidance isavailable, such as the ANS Safety Assessment Methodology (SAM) then this is referencedaccordingly. For further guidance on Safety Assessment see the SAM, or contact DAP/SSH.

KeywordsESARR 3 Safety Argument Safety Case Lifecycle Safety ObjectivesGoal Structured Notation Safety Case Safety Management Safety Requirements

Contact Person(s) Tel Unit

Dr. Bernd TIEMEYER 95038 DAP/SSH

STATUS, AUDIENCE AND ACCESSIBILITY

Status Intended for Accessible via

Working Draft General Public Intranet

Draft EATMStakeholders

Extranet

Proposed Issue RestrictedAudience

Internet (www.eurocontrol.int)

Released Issue

ELECTRONIC SOURCEPath:

Host System Software Size

Windows_XP Microsoft Word 2002 1180 Kb


3/66

Edition: 2.1 Released Issue Page iii

DOCUMENT APPROVAL

The following table identifies all management authorities who have successively approvedthe present issue of this document.

AUTHORITY NAME AND SIGNATURE DATE

Co-ordinatorAgency & EATM SMS

DAP/SSH Dr. Bernd TIEMEYER

Chairman of theSAMTF

Patrick MANA

Chairman ofSafety Team

Dr. Erik MERCKX

DirectorDAS

Bo REDEBORN

DirectorDAP

Guido KERKHOFS


4/66


Page iv Released Issue Edition: 2.1

DOCUMENT CHANGE RECORD

The following table records the complete history of the successive editions of the presentdocument.

EDITIONNUMBER

EDITIONDATE

REASON FOR CHANGEPAGES

AFFECTED

1.3 07.07.03 Proposed Issue Including editorial corrections All

1.41 29.11.04 Initial Draft of Revised Version All

1.42 13.12.04 Working Draft of Revised Version All

1.43 23.12.04 Further Draft of Revised Version All

1.44 01.02.05 Update following DAP/SAF internal review All

1.45 11.02.05 Chapter 5 and Appendix updated following DAP/SAFinternal review

Chap 5and

Appdx C

1.5 25.02.05 Minor changes following final DAP/SAF internal review All

2.0 28.09.05 Proposed Issue following external stakeholder review All

2.1 13.10.06 Released Issue, incorporating final stakeholdercomments

4, 6, 8,19


5/66


Edition: 2.1 Released Issue Page v

CONTENTS

PREFACE .............................................................................................................................. 1

CHAPTER 1 INTRODUCTION ............................................................................................ 3

1.

Purpose of the Manual .................................................................................. 3

2. Structure of the Manual ................................................................................. 4

CHAPTER 2 ESSENTIALS GETTING STARTED........................................................... 5

1.

What is a Safety Case for? ............................................................................ 5

2.

Which Kind of Safety Case? ......................................................................... 6

3.

Safety Cases and the Project Safety Lifecycle ............................................ 7

4.

Contents of a Safety Case ............................................................................. 9

5.

Defining the Scope and Boundaries for the Safety Case .......................... 10

6.

Setting the Context ...................................................................................... 10

CHAPTER 3 ESSENTIALS ARGUMENT AND EVIDENCE.......................................... 13

1.

Introduction.................................................................................................. 13

2.

Safety Argument .......................................................................................... 13

3.

Safety Evidence ........................................................................................... 14

CHAPTER 4 GUIDANCE PROCESS AND TECHNIQUES.......................................... 17

1.

Overview ...................................................................................................... 17

2.

Determining the Safety Criteria .................................................................. 18

3.

Constructing a Safety Argument ................................................................ 19

4.

Gathering, Assessing and Presenting Safety Evidence ........................... 23

5.

Evidence Safety Requirements Determination .................... ................... 24

6.

Evidence Safety Requirements Satisfaction ........................................... 26

7. Developing a Safety Plan ............................................................................ 29

8.

Format, Structure and Layout of the Safety Case ..................................... 30

9.

Verifying the Safety Case ............................................................................ 32

10.

ESARR Compliance .................... ................................................................. 33

CHAPTER 5 GUIDANCE - GSN SAFETY ARGUMENT EXAMPLES................................. 35


6/66


Page vi Released Issue Edition: 2.1

1.

Example Application of GSN A Project Safety Case ........................... 35

2.

Example Application of GSN Unit Safety Cases ..................................... 45

3.

Example Application of GSN Preliminary Safety Cases ........................ 48

APPENDIX A

REFERENCES ......................................................................................... 51

APPENDIX B GLOSSARY AND ABBREVIATIONS ....................................................... 53

GLOSSARY .................................................................................................................... 53

ABBREVIATIONS ........................................................................................................... 55

APPENDIX C

SAFETY CASE DEVELOPERS AND REVIEWERS CHECKLIST ............ 57


7/66

Safety Case Development Manual Preface

Edition:2.1 Released Issue Page 1

PREFACE

Background

Related Documents

EATM SMS Context

User Community

Feedback

Health Warning !!

Version 2.0 of the Safety Case Development Manual has been

developed based on recent experience, user feedback andlessons learned since Version 1.3 was published in July 2003.

The new Manual tries to explain in simple terms the Essentialsof how to construct a Safety Case and then provides supportingGuidance and Examples. It is intended primarily for use withinthe EUROCONTROL Agency but may also be used (whereappropriate) by the Member States.

The provisions of the Manual are intended to be consistent withESARRs and the EUROCONTROL ANS Safety AssessmentMethodology (SAM), to which numerous references are madeherein. In particular, the Manual as a whole is intended to

satisfy the requirements of paragraph 5.3 of ESARR 4concerning the documentation of the arguments and evidenceassociated with the risk assessment and mitigation processes.

The Manual provides a product description for the Safety Casedescribed in Element 5 (Risk Assessment and Mitigation) of theEATM Safety Management Handbook. Organisational issuesare dealt with in Element 3 (Organisation and Structure).

The Manual is intended primarily for safety professionals whohave received training in SMS and the EUROCONTROL SAM.

We believe that it will be of most immediate use to the willingdeveloper but accept that it could present some difficulties for

the uncertain starter; we also believe that it has something tooffer to the confident adopter 2.

Therefore, the contents of the Manual will be supported byrelated training, and readers (particularly, but not exclusivelywilling developers and uncertain starters) are urged toundertake this training and the related EUROCONTROL SAMtraining before embarking on the development of a Safety Casefor an operational application.

The users of this Manual are invited to provide any feedbackand suggestions for improvement on this current version to me,which will be taken into consideration when a future updated

Version is produced.

Finally, this Manual cannot give a complete insight into allaspects of Safety Assessment and Safety Case developmentnor provide ready made solutions to fit every situation.

1Where appropriate is inserted here because there is no explicit requirement in ESARRs for ANSPsto produce Safety Cases. EUROCONTROL has chosen the Safety Case approach for EATMProgrammes and Domain Activities because experience has shown this approach to be effective forthese applications.2See EUROCONTROL SAM [5] for explanation of these terms


8/66

Preface Safety Case Development Manual

Page 2 Released Issue Edition: 2.1

PAGE INTENTIONALLY LEFT BLANK


9/66

Safety Case Development Manual Introduction


CHAPTER 1 Introduction

1. Purpose of the Manual

What does it do?

Who is it for?

What is it for?

What does it not do?

This Safety Case Development Manual provides guidance onthe development of Safety Cases as a means of structuring anddocumenting the demonstration of the safety of an ATM serviceor new / modified System3.

The Manual is intended for use by those, employed on projects

or in service-provider organisations, who have to:

Produce Safety Cases eg safety practitioners;

Approve Safety Cases eg programme managers andheads of ATSUs;

Review Safety Cases eg safety department staff.

The aim is to achieve sound, well-presented Safety Casesthrough the adoption of a logical, rigorous, consistent andaccurate approach that is based on good safety practice.

Whereas this Manual should aid the process of developing and

presenting a Safety Case, it cannot give assurance of thevalidity of the end product, and it does not, therefore, relieve itsusers of their responsibility to provide such assurance.

The Manual does not provide guidance on how to carry out asafety assessment see the EUROCONTROL SAM [5]for that rather it describes how to present the results of a safetyassessment, in the context of a Safety Case.

3The term System is used throughout this manual to include airspace, equipment, people andprocedures.


10/66

Chapter 1 Safety Case Development Manual


As the Manual is intended to be used within the framework of aSafety Management System (SMS), it does not addressquestions, such as what is a change and when should a SafetyCase be produced these are assumed to be addressed by the

SMS4. Rather, the starting point for the Manual is the point atwhich it has been decided to produce a Safety Case for a

particular ATM service or change (a so-called substantialchange in this document).

2. Structure of the Manual

Essentials - Getting

Started

Essentials Argumentand Evidence

Guidance Processand Techniques

Guidance - Examples

References

Glossary

Checklists

The remainder of this Manual is presented in a further 4Chapters, covering the Essentialsof Safety Case Developmentand supporting Guidance, as follows.

Chapter 2explains the background to Safety Cases and

provides the key points in planning the development of a SafetyCase.

Chapter 3presents the essentials of the Safety Case itself.

Chapter 4provides guidance in support of Chapters 2 and 3,including the use of Goal-structuring Notation (GSN). TheGuidance may be accessed directly or via links embedded inthe relevant parts of those Chapters.

Chapter 5provides generic examples of structured SafetyArguments using GSN. The main example is the introduction ofa substantial change to an ATM service / system. Variations ofthat Safety Argument for an on-going ATM service and for atypical limited-scope Safety Case are also explained.

Appendix Aprovides a list of the references called up in theManual.

The EATMP Glossary document [4]facilitates the search anduse of acronyms, abbreviations, terms and definitions mostfrequently used within the EATM Programme. As the EATMPGlossary document includes more than 5,800 acronyms and2,000 definitions, a summary of the terms commonly used inSafety Case development is provided at Appendix B.

Appendix Ccontains checklists for use by Safety Casedevelopers, reviewers and approvers in assessing the quality ofthe argument structure and presentation of the Safety Case.

4For guidance on what is a change please see the SAM [5] Guidance Material, Part IV, Appendix H.


11/66

Safety Case Development Manual Getting Started


CHAPTER 2 Essentials

Getting Started

1. What is a Safety Case for?

A Lesson from History

ICAO Obligation

The Burden of Proof

In the investigation into the Piper Alpha accident [14] LordCullen wrote:

Primarily the Safety Case is a matter of ensuring that everycompany produces a formal safety assessment to assureitselfthat its operations are safe.

Only secondarily is it a matter of demonstrating this to aregulatory body. That said such a demonstration both meets alegitimate expectation of the workforce and the public andprovides a sound basis for regulatory control.

ICAO Annex 11 places an obligation on the providers of AirTraffic Management services to ensure the safety of air traffic,in respect of those parts of the ATM System and supportingservices within their managerial control.

Implicit to this obligation is the requirement on those withmanagerial control to demonstratepositively that the relevantSafety Regulations are satisfied. In essence there is a burdenof proof on Air Navigation Service Providers to show thatacceptable levels of safety5are, and continue to be, achieved.

5As defined by the Safety Criteria see Chapter 4, section 2


12/66



Primary Purpose

Relationship withRegulatory Approval

Relation to SafetyManagement System

Relation to SafetyAssessment

Broadly, the Safety Case is the documentedassurance (ieargument and supporting evidence) of the achievementand maintenance of safety. It is primarily the means bywhich those who are accountable for service provision orprojects6assure themselvesthat those services orprojects are delivering (or will deliver), and will continue to

deliver, an acceptable level of safety.

As the main objective of safety regulation is to ensure thatthose who are accountable for safety discharge theirresponsibilities properly, then it follows that a Safety Casewhich serves the above primary purpose should alsoprovide an adequate means of obtaining regulatoryapproval for the service or project concerned.

In the context of a Safety Management System, the SafetyCase can be a means of documenting and recording thesafety of a service or system. Conversely, theimplementation of a Safety Management System would

provide evidence to support a Safety Case.

The development of a Safety Case is not an alternative tocarrying out a Safety Assessment, in accordance with, forexample, the EUROCONTROL SAM [5]; rather, it is ameans of structuring and documenting a summary of theresults of a Safety Assessment, and other activities (egsimulations, surveys etc), in a way that a reader can readilyfollow the logical reasoning as to why a change (or on-going service) can be considered safe.

2. Which Kind of Safety Case?

Types Safety Cases may come in many forms but most, if not all, canbe thought of as falling into one of two categories, as follows:

those which are used to demonstrate the safety of an on-going service these are known herein as Unit SafetyCases; and

those which are used to demonstrate the safety of asubstantial change to that service(and/or underlying

system) these are known herein as Project Safety Cases.

Unit Safety Case

The two categories are interrelated, as explained below.

An ATSU (or other major, safety-related service / facility) maydecide to produce, and maintain, a (Unit) Safety Case in orderto show that the on-going, day-to-day operations are safe andthat they will remain so indefinitely.

6The distinction between services and projects / systems is to emphasise the difference between Unitand Project (or System) Safety Cases see section 2. The generic term project should be taken toinclude EATM Programmes and Domain Activities.


13/66



Project Safety Case

A Unit Safety Case would include typically an a priori safetyassessment (to show that service / system is predicted to besafe), together with the results of safety audits, surveys andoperational monitoring (to show that, up that point in time, itactually has been safe). It should also demonstrate thatprocesses are in place to ensure that all future changes to theATSUs system will be managed safely through, inter alia,Project Safety Cases.

An ATSU (or other responsible organisation) may also decide toproduce a Project Safety Case when a particular substantialchange to an existing safety-related service / system (includingthe introduction of a new service / system) is to be undertaken.

A Project Safety Case would normally consider only those riskscreated or modified by the change and rely on an assumption(or evidence from the corresponding Unit Safety Case) that thepre-change situation is at least tolerably safe.

Project Safety Cases are used to update, and are usually

subsumed into, Unit Safety Cases7

.

Further details of both Unit Safety Cases and Project SafetyCases are given in section 3 below and in Chapter 5.

3. Safety Cases and the Project Safety Lifecycle

A simplified view of a typical project lifecycle is shown in thediagram overleaf.

Safety Considerations This is an EATM SMS process to identify the main safety issuesassociated with a project as soon as possible after anOperational Concept has been developed and to help indeciding whether a full Safety Plan and Safety Case arerequired. It provides an initial assessment of the safetyimplications of the project, as the basis for developing a SafetyPlan in which the detailed safety activities will be specified. Itshould address, inter alia, what the project is seeking to achieve(eg to deliver benefits in capacity, efficiency and/or safety), thepossible impact on safety (in general terms only, since a safetyassessment would not have been started at this stage), thecriteria for deciding what is safe in the context of the Projectand, in broad terms, the strategy for demonstrating safety.

7However, this is not meant to imply that a Unit Safety Case is merely a collection of project SafetyCases!


14/66



SafetyConsiderations

OperationalConcept

InitialSafety

Argument

FHA

PSSA

Implementation

Transfer intoOperation

SafetyPlan

Project

Safety

Case

UnitSafetyCase

Evidence

Approval

Evidence

Evidence

Evidence

Evidence

Update, if required

SafetyMonitoring

Reports

Update

UpdateEvidence

SSA

Integration

Operation &Maintenance

SafetyConsiderations

OperationalConcept

InitialSafety

Argument

FHA

PSSA

Implementation

Transfer intoOperation

SafetyPlan

Project

Safety

Case

UnitSafetyCase

Evidence

Approval

Evidence

Evidence

Evidence

Evidence

Update, if required

SafetyMonitoring

Reports

Update

UpdateEvidence

SSA

Integration

Operation &Maintenance

Initial Safety Argument

Safety Plan

Safety Assessment:

FHA

PSSA

SSA

Building on the Safety Considerations, the initial SafetyArgument should be as complete as possible and at leastsufficient to provide a set of goals for the Safety Plan toaddress. It also provides the starting point, and framework, forthe development of the Project Safety Case, although it needsto be recognised that the initial view of what the SafetyArgument should look like may need to change, depending onthe results of the subsequent safety assessment.

Specifies the safety activities to be conducted throughout theproject lifecycle and the responsibilities for their execution.

The three main phases of safety assessment (FHA, PSSA andSSA) provide much of the Evidence needed for the ProjectSafety Case, as follows:

o FHA produces Safety Objectives to limit the frequency ofoccurrence of hazards, such that the associated risk wouldbe acceptable, and the external means of mitigating theeffects of the hazards, including those means that are notpre-existing and need to be captured subsequently asSafety Requirements in the PSSA.

o PSSA produces Safety Requirements and Assurance Levelsfor the system elements.

o SSA produces the assurance that the Safety Requirementsand Safety Objectives are met in the implemented system


15/66



Implementation &Integration

Transfer into Operation

Operational Service

Unit Safety Case

and that risk is acceptable.

For further information on safety assessment, see the SAM [5].

This phase covers all the preparation needed in order to bringthe new / modified system the subject of the Safety Case into operational service.

Transfer into Operation of the new/modified system wouldnormally be subject to a risk assessment and mitigation for thisphase itself (part of the Project Safety Case) and be concludedby finalisation and regulatory approval of the Project SafetyCase.

Because most, if not all, of the preceding safety assessmentwork is predictive in nature, it is important that further assuranceof the safety is obtained from what is actually achieved inoperational service. If the operational experience differssignificantly from the results of the predictive safetyassessment, it may be necessary to review and update theProject Safety Case.

Once a satisfactory steady state has been achieved, it would beappropriate to update the Unit Safety Case (if one exists) withthe information from the Project Safety Case thus establishing anew safety baseline for the on-going operational service.

4. Contents of a Safety Case

Aim

Purpose

Scope

System Description

Justification

Argument

Evidence

Caveats

A good Safety Case (of whichever type) should include, at least:

what the Safety Case is trying to show - this should bedirectly related to the Claim that the subject of the SafetyCase is acceptably safe;

why is the Safety Case being written and for whom;

what is, and is not, covered - see section 5below;

a description of the system / change and its operational /physical environment, sufficient only to explain what theSafety Case addresses and for the reader to understand theremainder of the Safety Case see section 6below;

for project Safety Cases, the justification for introducing thechange (and therefore potentially for incurring some risk);

a reasoned and well-structured Safety Argument showinghow the Aim is satisfied see Chapter 3, section 2 below;

supporting Safety Evidence to substantiate the SafetyArgument see Chapter 3, section 3 below;

all Assumptions, outstanding safety Issues, and anyLimitationsor restrictions on the operation of the system;


16/66



Conclusions a simple statement to the effect that the Aim has beensatisfied, subject to the stated Caveats.

Chapter 4, section 8provides further guidance on the content,structure and layout of a Safety Case

5. Defining the Scope and Boundaries for the Safety Case

Scope Definition

Lifecycle Limitations

Defining the scope and boundaries of the Safety Case is anessential first step in the development of the Safety Case. Itshould explain clearly:

what the Safety Case covers (and does not cover);

boundaries of responsibility with respect to managerialcontrol and other stakeholders;

relationship with other Safety Cases, if applicable;

applicability and compliance with safety regulations andstandards;

any assumptions made in defining the scope, boundaries orsafety criteria.

A Safety Case may be (temporarily) restricted to the safety of a

new concept, and therefore be conditional on the subsequentcomplete and correct implementation of that concept by theresponsible organisation. This is the situation on those EATMProgrammes (and similar activities) for which EUROCONTROLis not responsible for implementation; the output would then bea validated set of Safety Requirements (from the PSSA).

The term Preliminary Safety Caseis used herein to cover suchsituations, and it should be supported by guidance material forthe subsequent implementation of the Safety Requirements andfor the development of a full Safety Case. An example of aPreliminary Safety Case, and the sort of guidance that shouldaccompany it, are given in Chapter 5, section 3.

6. Setting the Context

Rationale

Content

It is vital to fully describe the operational environment to whichthe Safety Case applies and the system configuration on whichthe Safety Case (and underlying Safety Analysis) is based.

The description of the Context should include:


17/66



the purpose of the system from a safety perspective;

the interfaces with other systems including people,procedures and equipment;

the operational environment including all characteristicsthat may be affected and elements that are relied upon,when assessing acceptable levels of safety;

A reference to (together with a summary of) Concept ofOperations that explain how the system, and the service thatit supports, are intended to operate


18/66





19/66

Safety Case Development Manual Argument and Evidence


CHAPTER 3 Essentials

Argument and Evidence

1. Introduction

Overview

Non-prescriptive

This Chapter presents the essential points of:

the construction of Safety Arguments;

the collation, review and presentation of Safety Evidence.

The information presented draws on current good practicewithout prescribing a particular methodology, and is supportedby references to the Guidance in Chapter 4.

2. Safety Argument

What is a SafetyArgument?

Making the Claim

Supporting the Claim

A Safety Argument is a statement (or a set of statements) that isused to assert that the service or system concerned is safe, andshould be developed as follows.

The Safety Argument must start with a top-level statement(Claim) about what the Safety Case is trying to demonstrate inrelation to the safety of the service or system.

The Claim must be supported by:

Safety Criteria,which define what is safein the context ofthe Claim;

for Project Safety Cases, the Justificationfor introducing


20/66



Structuring the

Argument

Guidance

the change to the service or system concerned;

the Operational Context for the Claim;

any fundamental Assumptions on which the Claim relies.

The decomposition of the Claim into lower-level Arguments

provides the essential links between the Claim and the wealth ofEvidence needed to show that the Claim is valid.

In performing this decomposition, it is important that:

each Argument in the structure is expressed as a simplepredicate ie a statement that can be only true or false;

the Argument structure does not contain any negative orinconclusive Arguments8;

the set of Arguments at each level of decomposition isnecessary and sufficient to show that the parent Argument istrue;

a valid counter-Argument, which would negate the parentArgument, does not exist9;

where the rationale for decomposition of an Argument intolower-level Arguments is not self evident, it is explained bysupporting text;

the number of levels of decomposition is appropriate to thecomplexity of the Safety Case and/or supporting Evidence;

each branch of the Safety Argument structure is terminatedin supporting Evidence;

there is a clear distinction between, and correct use of,Direct(product-based) and Backing(process-based)Arguments and related Evidence.

Further guidance on the structuring of Safety Arguments isgiven in Chapter 4, section 3, and generic ATM examples arepresented in Chapter 5.

3. Safety Evidence

What is SafetyEvidence?

Safety Evidence is information, based on established fact orexpert judgement, which is presented to show that the SafetyArgument to which it relates is valid (ie true).

The essential rules of Evidence are as follows:

Necessity Evidence must be presented only to the degree and extent

8The main point here is that lack of Evidence of risk is not Evidence of lack of risk!9This point is concerned only with the sufficiency of the Argument sufficiency of the Evidence iscovered in section 3.


21/66

Safety Case Development Manual Argument and Evidence


Sufficiency

Appropriateness

Rigour

Relevance

Guidance

necessary to support the related Argument;10

Evidence must show that the related Argument is true in away that is clear, unequivocal, conclusive and, whereverpossible, objective;

the type of Evidence from safety analysis, design,

simulation, test, previous usage, compliance with standardsetc must be appropriate to the Argument;

the rigour of the Evidence must be appropriate to theassociated risk;

Evidence must actually relate to the correct configuration ofthe system under consideration.

Further guidance on the gathering, assessing and presentingEvidence is given in Chapter 4, section 4.

10The point here concerns only irrelevant Evidence. Clearly any Evidence which actually counters theArgument must not be ignored; on the contrary, the validity (or at the very least the phrasing) of anArgument must be reconsidered in the light of such Evidence


22/66





23/66

Safety Case Development Manual Process and Techniques


CHAPTER 4 Guidance

Process and Techniques

1. Overview

Context

Scope

This Chapter provides guidance in support of the requirementsstated in Chapters 2 and 3 above. Whereas this guidance isbased on experience gained on the development of Safety

Cases by EUROCONTROL on a wide range of EATMProgrammes, it is intended to be applicable also to otherenvironments eg service provision.

The guidance covers the following areas:

determining the Safety Criteria - section 2;

constructing a Safety Argument, using a recognisednotation that is considered to be good practice ie Goal-structuring Notation (GSN) - section 3;

general issues concerning gathering, collating,assessing and presenting Safety Evidence - section 4;

specific issues concerning Evidence of SafetyRequirements determination -section 5;

general issues concerning Safety Requirementssatisfaction -section 6;

developing a Safety Plan - section 7;

deciding the format, structure and layout of a SafetyCase - section 8;


24/66



verifying the Safety Case - section 9;

ESARR compliance - section 10.

2. Determining the Safety Criteria

Absolute

Relative

Reductive

Selecting Criteria

Safety Criteria are essential to the definition of what is safeinthe context of the top-level Safety Claim

Basically, they fall into three categories as follows:

Compliance with a defined target eg the ESARR 4 RiskClassification Scheme (RCS) or ICAO Target Level of Safety(TLS) or portion thereof. Such criteria are usuallyquantitative;

Relative to an existing (or previous) level of safety. Suchcriteria may be quantitative or qualitative;

Where the risk is required to be reduced as far asreasonably practicable. Such criteria are usually qualitative.

In general, absolute criteria are preferred since satisfaction ofthem does not depend on proof of past safety achievement andsuch proof may be difficult if a suitable baseline does not existor sufficient historical data is not available.

However, in some cases, there may be a problem inestablishing what would be a suitable target on which to basethe criterion because either:

a regulatory target has not been set for the operationalenvironment concerned11; or

for Project Safety Cases, it may not be feasible to determinewhat portion of the overall target it would be reasonable toallocate to the system concerned.

As an alternative to the absolute approach, a relative SafetyArgument (ie based on a relative criterion) could be use for aProject Safety Case12if:

a well-defined baseline, prior to the introduction of (orchange to) a system, could be established; and

it can be shown, or at least reasonably be assumed, that thebaseline situation was safe.

The justification for a relative approach is the ATM 2000+ [1]Safety Objective which requires that risk shall not increase andpreferably decrease, relative to historical achievement. The

11This is being addressed by the current (2005) EUROCONTROL TLS study12For Unit Safety Cases an absolute approach should always be the primary criterion.


25/66



Multiple Criteria

ESARR 4 RCS, although normally considered to be an absolutemeasure, is actually a quantified interpretation of the ATM2000+ Safety Objective13.

A reductive approach is called for by ESARR 3 (paragraph5.1.4), which requires ANSPs to reduce risk as far asreasonably practicable. It is an important criterion for in-servicesafety monitoring especially regarding incident investigationand corrective action.

It is usual to specify more than one type of criterion, andsometimes all three. In ATM, reducing risk as far as reasonablypracticable is rarely adequate on its own14but it is often useful insupport of one (or both) of the other two criteria.

Use of RiskClassification Schemes

Risk classification schemes (RCS)15are often used as criteriaon which to base absolute Arguments. However, experiencehas shown that a lack of understanding by the user as to howthe RCS was originally derived can lead to inappropriate use. IfRCS are used, it is important that the user understands:

at what level in the system hierarchy the values are intendedto be applied;

where the probability/frequency values used in the schemecame from and whether they are (still) valid;

to what operational environment the values apply eg typeof airspace, traffic patterns, traffic density, spatial dimension,phase of flight etc;

how the aggregate risk, as specified in ESARR 4 forexample, can be deduced from analysis of individualhazards, in restricted segments of the total system.

The last three bullets should not be a problem for the user in anorganisation that has a single, well-founded RCS, applicable toall operational environments.

All other RCS users should be aware of, and address, the issueraised in the first bullet.

For further guidance on RCS, please see SAM FHA Chapter 3GM E, based on ED-125 [15].

3. Constructing a Safety Argument

Requirements Since the Safety Argument forms the framework of a Safety

13See ESARR 4 [9], Appendix A, Endnote (2)14Both ATM 2000+ and ESARR 4 require, as a minimum, that risk must not increase reducing riskas far as reasonably practicable on its own does not ensure that this minimum requirement is met.15The comments here are aimed at the use of Risk Classification Schemes to determine what is atolerable level of risk, not at the use of Severity Classification / Categorisation Schemes proposed in,inter alia, ESARR 2, ESARR 4 and the EUROCONTROL ANS Safety Assessment Methodology.


26/66



Case, it is important that the Argument is set out in a rigorous,

GSN Solution

hierarchical and well-structured and easily-understood way.

Goal-structuring Notation (GSN), developed by the University ofYork, provides a graphical means of setting out hierarchicalSafety Arguments, with textural annotations and references to

supporting Evidence.

The logical approach of GSN, if correctly applied, brings somerigour into the process of deriving Safety Arguments andprovides the means for capturing essential explanatory material,including assumptions, context and justifications, within theargument framework.

The diagram below shows, in an adapted form of GSN, aspecimen Argument and Evidence structure to illustrate theGSN symbology most commonly used in EUROCONTROL ATMSafety Cases.

Argument

Arg 1

Fig n

OverallArgument /Claim

Arg 0

Argument

Arg 2

Argument

Arg n

Continuationpage

A0001Assumption

J0001Justification

C0001Context

St0001Strategy

Argument

Arg 1.2

Argument

Arg 1.1

Ref

Evidence

Ref

Evidence

Cr001

Criteria

Argument

Arg 1

Fig nFig n


Arg 0


Arg 0

Argument

Arg 2

Argument

Arg n

Continuationpage

A0001Assumption

J0001Justification

C0001Context

St0001Strategy

Argument

Arg 1.2

Argument

Arg 1.1

Ref

Evidence

Ref

Evidence

Cr001

Criteria

An Argumentshould take the form of a simple predicate - ie astatement which can be shown to be only true or false.

GSN provides for the structured, logical decomposition ofArguments into lower-level Arguments. For an Argumentstructure to be sufficient, it is essential to ensure that, at eachlevel of decomposition:

Argument

Arg 1.1

Argument

Arg 1.1


27/66



the set of Argumentscovers everything that is needed inorder to show that the parent Argumentis true;

there is no valid (counter) Argumentthat could underminethe parent Argument.

In the above diagram, for example, if it can be shown that Arg 1is satisfied by the combination of Arg 1.1 and Arg 1.2, then weneed to show that Arg 1.1 and Arg 1.2 are true in order to showthat Arg 1 is true.

If this principle is applied rigorously all the way down throughand across a GSN structure, then it is necessary to show onlythat each Argument at the very bottom of the structure issatisfied (ie shown to be true) in order to assert that the top-level Claim has been satisfied. Satisfaction of the lowest-levelArgumentsis the purpose of Evidence.

Unnecessary (or misplaced) Arguments do not in themselvesinvalidate an Argument structure; however, they can seriouslydetract from a clear understanding of the essential Argumentsand should be avoided. The cover-up method illustrated in thethree GSN diagrams below can be used to determine identifyunnecessary and misplaced Arguments.

A=True

B=True C=True

E=TrueD=True

A=True

B=True C=True

E=True

D=True

A=True

B=True C=True

E=True

F=True

If this is complete

and correct

then D is incorrectlypositioned

and F isunnecessary

D=True

A=True

B=True C=True

E=TrueD=True

A=True

B=True C=True

E=True

D=True

A=True

B=True C=True

E=True

F=True

If this is complete

and correct

then D is incorrectlypositioned

and F isunnecessary

D=True

It follows from the above that, for an Argumentstructure to beconsidered to be complete, every branch must be terminated ina reference to the item of Evidencethat supports the Argumentto which it is attached.

Evidencetherefore must be:

appropriate to, and necessary to support, the relatedArgument- spurious Evidence(ie information which is notrelevant to an Argument) must be avoided since it wouldserve only to confuse the picture;

sufficient to support the related Argument- inadequate

Ref

Evidence


28/66



evidence undermines the related Argumentandconsequently all the connected higher levels of the structure.

Strategies are a useful means of adding comment to thestructure to explain, for example, how the decomposition willdevelop. They are not predicates and do not form part of thelogical decomposition of the Argument; rather, they are there

purely for explanation of the decomposition.

An Assumption is a statement whose validity has to be reliedupon in order to make an Argument.

Assumptions may also be attached to other GSN elementsincluding Strategiesand Evidence.

Context provides information necessary for an Argument (orother GSN element) to be understood or amplified.

Contextmay include a statement which limits the scope of anArgumentin some way.

A Justification is used to give a rationale for the use orsatisfaction of a particular Argument or Strategy. Moregenerally it can be used to justify the change that is the subjectof the Safety Case.

Criteriaare the means by which the satisfaction of an Argumentcan be checked.

Numbering It is recommended that Argumentsbe numbered hierarchically(eg, Arg 1.1) to reflect their logical structure.

Strategies, Assumptions, Context, and Criteria should be

numbered sequentially (eg, St0001) since they elaborate, but doNOT form part of, the logic of the structure.

It is recommended that Evidencebe numbered according to itssource reference and that the Evidencebubble contains a briefindication of the form that the Evidencetakes.

Other Symbology

A Choice can be used while a Safety Argument is beingdeveloped to show a decision point between alternativeStrategies. However, they must be removed before a SafetyArgument is finalised.

A Model is some representation of the system, sub-system orenvironment eg Simulations, Data Flow Diagrams, CircuitLayouts, State Transition Diagrams etc.

A Stakeholder is the person or role responsible for ensuringsatisfaction of an Argument, Strategy, or Choice.

Constraintsare used to restrict the way in which an Argumentcan be solved; they are restrictions imposed on theinterpretation of the parent Argument.

St0001Strategy

A0001

Assumption

C0001Context

J0001Justification

Cr001

Criteria

Cr001

Criteria


29/66



A Problemis attached to an Argument to indicate that there is apossible obstacle to showing that it is true. Problemscan alsobe attached to other GSN elements.

Examples of the application of GSN to generic SafetyArguments are presented in Chapter 5.

4. Gathering, Assessing and Presenting Safety Evidence

Importance

Direct and Backing

General Attributes

Necessity

Evidence is the heart of every case and ultimately it is on thequality and completeness of the Evidence that the validity of aSafety Case depends. Of course, a well-structured SafetyArgument is very important but only insofar as it provides thecontext for, and thus facilitates interpretation of, the Evidence.

In decomposing the Safety Arguments, the following two maintypes of Argument (and related Evidence) are used:

that which shows that a particular objective has beenachieved (ie that a higher level Argument or Claim has beensatisfied) this is referred to as DirectArgument andEvidence;

that which shows that the Direct evidence is trustworthy (iethat it can be relied upon) this is referred to as BackingArgument and Evidence.

DirectEvidence may be thought of as being that which reliesdirectly on the observable properties of a product (ie the outputof a process), supporting a logical Argument as to how theproduct satisfies its safety objectives or requirements, asappropriate.

Backing Evidence is obtained from the properties of theprocessesby which DirectEvidence was obtained, and showsthat those processes, tools and techniques, human resourcesetc were appropriate, adequate and properly deployed.

The points below expand upon the essential rules outlined inChapter 3, section 3 above.

Evidence must be presented only to the degree and extentnecessary to support the related Argument.

The issue here is that, in the context of an Argument-basedapproach, any Evidence which is unrelated to a part of thatArgument is not only of no value but could also serve as adistraction from those aspects of the Safety Case that arerelevant. On the other hand, any Evidence which actuallyundermines the validity of an Argument must not be ignored the existence of such Evidence must be acknowledged andexplained fully in the Safety Case.


30/66



Sufficiency:

o Clarity, andConclusiveness

o Objectivity

Appropriateness

Rigour

Relevance

Evidence must be sufficient, as follows.

It is bad (but unfortunately not uncommon!) practice to presentan element of a structured Argument and then refer to a mass ofinformation as Evidence to substantiate the Argument.

It is vital to the integrity of the Safety Case that the Evidence bepresented in such a way that is clear to the reader that theEvidence does actually show the related Argument to be true,beyond all reasonable doubt.

Where Evidence is contained in appendices or externaldocuments, a summary justifying the adequacy of the Evidenceshould be presented (in the Safety Case) along with theassociated Argument. It is not sufficient to merely reference theEvidence with statements such as Evidence to support theArgument is presented in .

Wherever possible, Evidence should consist of proven facts eg, the results of a well-established process such as simulations

and testing. Only where such objective Evidence is notavailable should Evidence based on expert opinion be used,and then only when the credentials of the expert(s) and themeans of eliciting the opinion are adequate and have beenpresented as Backing Evidence.

The type of Evidence, from safety analysis, design, simulation,test, previous usage etc, must be appropriate to the Argument see sections 5 and 6below.

The rigour of the Evidence must be appropriate to theassociated risk. This is the principle behind the AssuranceLevel concept in ESARR 6 [11](for software) and theEUROCONTROL ANS Safety Assessment Methodology [5] forsoftware, procedures and human aspects.

Evidence must relate to the configuration of the system andoperational environment under consideration - eg a correct andknown:

version of the equipment, procedures, training, etc.; documentation used in the production of that version; range of configuration data.

Application How the above should be applied specifically to the two mainstages of the safety development lifecycle requirementsdeterminationand requirementssatisfaction- is discussedbelow, in sections 5 and 6respectively.

5. Evidence Safety Requirements Determination

What are SafetyRequirements

To paraphrase ESARR 4 [9], Safety Requirements are meansby which the necessary risk reduction measures identified in thehazard and risk analysis are formally16specified. Necessaryin

16In the normal English meaning of the word.


31/66



this context means necessary in order to achieve the requiredsafety levels, as defined by the Safety Criteria (seesection 2above) and translated into specific Safety Objectives during theFunctional Hazard Assessment [5]

Role of SafetyRequirements in ATM

Types of SafetyRequirement

Direct Evidence ofSafety RequirementsDetermination

Backing Evidence ofSafety RequirementsDetermination

The primary purpose of ATM is to reduce the risk of accident toair traffic that would otherwise exist. The amount of risk

reduction is determined primarily by the functionality andperformance of the ATM systems elements, includingequipment, people and procedures. However, failure within theATM system can cause risk to increase again, either byreduction in functionality or performance, or by the introductionof new risk caused by corruption of the outputs of ATMfunctions.

Therefore, in order to achieve a net safety benefit from ATM, thereduction in risk afforded by the desired functional andperformance properties of ATM needs to be substantially greaterthat any increase due to failure17. It follows therefore that SafetyCases are critically dependent on the determination andsatisfaction of a complete and correct set of SafetyRequirements in which system functionality and performanceare appropriately considered alongside system integrity.18

DirectEvidence of Safety Requirements Determination isconcerned with the requirements themselves and should show,inter alia, that:

all relevant Hazards have been identified;

the potential outcomes of the Hazards have beencategorised correctly;

Safety Objectives have been specified to control the

frequency of occurrence of the Hazards such that anacceptable level of risk (as defined by the Safety Criteria) isachieved;

Safety Requirements have been specified to control thecauses of the Hazards such that the Safety Objectives aresatisfied, and to capture the external means of mitigation ofthe Hazard effects.

The key issue here is to ensure that the Safety Requirementsare complete ie that all risks are taken into account. It wouldnot be sufficient to show that the Safety Requirements satisfythe Safety Criteria if those Safety Requirements were based onan incomplete / incorrect Hazard assessment.

BackingEvidence of Safety Requirements Determination isconcerned with the process of deriving the requirements andshould show, inter alia, that:

the Safety Requirements were determined using anestablished and appropriate process;

17SAM [5]expresses the distinction in terms of the success approach and the failure approach18This point is emphasised here because of a popular misconception that safety is dependent mainlyon integrity. Neglect of functionality and performance can lead to systems that are reliably unsafe.


32/66



Relationship toEUROCONTROL SafetyAssessmentMethodology

the techniques and tools used to support the SafetyRequirements Determination were verified and validated;

the Safety Requirements Determination process wasexecuted by suitably competent and experienced personnel.

The Functional Hazard Assessment (FHA) and PreliminarySystem Safety Assessment (PSSA) stages of theEUROCONTROL, Air Navigation System Safety AssessmentMethodology [5]provides an appropriate and sound process forthe determination of ATM Safety Requirements demonstrationof adherence to the FHA and PSSA processes could thereforebe used as BackingEvidence as in the first bullet point above.

6. Evidence Safety Requirements Satisfaction

Sources of Evidence

Service Experience9

Direct Evidence from-Service Experience

Evidence of Safety Requirements satisfaction may be used fromthree main sources, as follows:

Service Experienceof previous usage

Verification and Validation

Compliance with Standards

Service Experience is data from previous operational use of theproduct concerned. DirectEvidence is concerned with analysisof data from Service Experience and what the results of thatanalysis showed in terms of satisfaction of the safety

requirements. BackingEvidence is concerned with showingthat the environment from which the data was obtained issufficiently similar to that to which the re-used product will besubjected, that adequate performance-assessment and fault-recording processes were in place when the product wasoriginally deployed, and that the analysis of the outputs of thoseprocesses was adequate and properly carried out.

In assessing and presenting Direct Evidence from ServiceExperience, it is important to ensure that:

an analysis process, with pass/fail criteria, was specified foreach aspect of the product safety requirement whosesatisfaction is being justified using service experience;

analysis of the service records shows that the criteria foreach product safety requirement, whose satisfaction is beingjustified using service experience, have been met;

all of the details relevant to the argument being made (eg oflength of service, history of modifications, list of users) areincluded in the Evidence;

19Further guidance on Service Experience, specific to CNS/ATM software, may be found in ED-109[13]. Note, however, that ED-109 makes no distinction between Direct and Backing evidence.


33/66



Backing Evidence fromService Experience

any product capabilities that are not necessary to satisfy theSafety Requirements cannot have an adverse effect on thesafe operation of the system.

In assessing and presenting Backing Evidence from ServiceExperience, it is important to ensure, inter alia, that:

Verification andValidation

the subject of the Safety Case and the product for which theService Experience Evidence is available are identical orsufficiently similar;

the conditions of use of the product for which the ServiceExperience is available is taken into account in the analysis;

the proposed operational environment and the operationalenvironment for which the Service Experience Evidence isavailable are identical or sufficiently similar;

any changes made to the operational environment,conditions of use, or product during the period of the Service

Experience are analysed to determine whether thosechanges alter the applicability of the data obtained fromService Experience for the period preceding the changes;

all aspects of those product functions whose safetyrequirements that are being justified from ServiceExperience have been exercised in the (previously)deployed product;

the extent of the Service Experience is sufficient todemonstrate that each aspect of the product safetyrequirement has been met;

a Defect Reporting, Analysis and Corrective Action System(DRACAS) is in place for the deployed product, and isoperated in a reliable manner, and is adequate to supportthe Service Experience Evidence;

the procedures and tools used to support the creation andanalysis of Service Experience Evidence were verified andvalidated;

for all reported failures of an aspect in the productcomponent, the underlying fault has been corrected, or ithas been shown that the fault is not relevant because it hasno safety impact;

the collection and analysis of Service Experience Evidencewas done by suitably competent and experienced personnel.

Evidence from system Verification and Validation (V&V) may bebased on, inter alia, analysis and/or testing.

Analysis, in this context, covers any proof of requirementssatisfaction that is obtained from the design or otherrepresentation of the product, including models, prototypes,software source code etc. It includes, for example, simulationformal proof, hardware reliability prediction, inspection, andsoftware static and dynamic code analysis.


34/66



Direct Evidence -V&V

Backing Evidence -V&V

Testingis restricted largely to tests of the final product in anenvironment which is as close as possible to the operationalenvironment. Its purpose, broadly, is to demonstrate that whathas been built satisfies the requirements, and it is used tosupplement (sometimes replace) Analysis.

It is beyond the scope of this Manual to discuss the relativemerits of analysis and testing, or of the various techniqueswithin those two broad categories. Suffice it to say that a SafetyCase should set out clear justifications of the selectedtechniques according to the nature and integrity required of thesystem to which the Safety Case applies. The followingguidance is however given concerning the principalrequirements of Directand BackingV&V Evidence.

However obtained, Directevidence is concerned with the outputof the V&V processes, and should include, as a minimum: 20

specifications of what V&V activities were carried out;

evidence that the V&V activities and pass/fail criteria weresufficient to demonstrate that the related requirements weresatisfied;

the results of the V&V activities;

analysis of the results to shows that all the specifiedpass/fail criteria were met;

explanation and justification of any discrepancies in theresults.

Whether obtained from analysis or testing, Backingevidence isconcerned with the V&V processes themselves, and shouldinclude, as a minimum Evidence that:

the processes were specified and performed independentlyfrom design;

the methods and techniques used are appropriate andadequate, for the properties of the product underconsideration;

the tools used to support the processes were verified andvalidated to a level appropriate for the assigned assurancelevel and were properly used;

the V&V processes were properly and completely executed,and the guidance, procedures, and standards were adheredto;

for previously existing V&V evidence, obtained for COTS orre-used products, the evidence is entirely valid for the newsystem application;

any differences between the operational and V&V

20The list is intended to provide only an overview of the main issues. Further detail on V&V, specificto CNS/ATM software, may be found in section 3 of ED-109 [13].


35/66



Compliance withStandards

Product Standards

Process Standards

Relationship toEUROCONTROL SafetyAssessmentMethodology

environments were identified, and the impact on the resultswas assessed and justified.

Evidence of compliance with standards can be a significantcontribution to the safety case. However, the way in whichadherence to a particular standard can be used to demonstratecompliance with Safety Requirements will depend on the nature

of the standard itself.

Product standards specify precisely what is required of aspecific item of equipment in terms of function, performance,integrity and, in some cases, form and fit. A good example isthe Arinc 700 series of standards, which define digital avionicssystems and equipment installed on civil aircraft. Currently,product standards are not common in ATM.

Compliance with product standards could be used as DirectEvidence of system safety, subject to it being shown that thestandard was appropriate to the particular application and to theprovision of sufficient BackingEvidence concerning theadequacy of the process by which compliance wasdemonstrated.

At the other end of the spectrum, are standards which addressthe processes of development and manufacture examplesrange from the very broadly based ISO 9000 series to the morespecific ED-78A (Guidelines for the Approval of the Provisionand Use of ATS Supported by Data Communications) and ED-109 ( Guidelines for CNS/ATM System Software IntegrityAssurance). In none of these cases would it appropriate tocertify a product against them, from a safety viewpoint;however, compliance with such standards, especially the morespecific ones, could provide excellent Backing Evidence forsafety requirements determination and/or satisfaction.

The distinction between product- and process-based safetyassurance is clearly fundamental since the former is concernedwith getting the right product and the latter with getting theproduct right.

The System Safety Assessment (SSA) stage of theEUROCONTROL, Air Navigation System Safety AssessmentMethodology [5]provides further guidance on the application ofthe above approaches to requirements-satisfaction.

7. Developing a Safety Plan

Introduction

Basic Requirements

Safety Activities

A Safety Plan specifies, inter alia, the safety assurance activitiesthat are to be carried out in order to create necessary andsufficient Evidence for the production of a Safety Case.

The SRC-EATM Interface process [6] specifies the followingcontents for a Safety Plan:

the Safety Activities needed to meet the (high-level) safetyobjectives, as well as the links and relationships between


36/66



Resources

Roles andResponsibilities

Safety Deliverables

The Safety / ProgrammeLifecycle

Safety Activity /Deliverables Mapping

Schedule

SpecificRecommendations

Safety Argument

Safety CaseDevelopment

Reviews and Approvals

Handover

Safety Regulation

Safety CaseMaintenance

safety activities and safety objectives;

the means and resources to carry out safety activities withinthe Programme;

responsibilities and accountabilities for Safety Activities;

the safety deliverables associated with the Safety Activities;

the allocation of Safety Activities and Safety Deliverables inthe progression of the Programme;

the relationships and dependencies between successiveSafety Activities and associated Safety Deliverables;

the detailed schedule and milestones for conducting SafetyActivities and releasing associated Safety Deliverables.

It is recommended that the following items, specific to thecreation of a Safety Case, also be included in the Safety Plan:

an initial version of the Safety Argument. The rationale forthis is that most of the Safety Activities (see above) shouldbe directed at the collection and assessment of Evidence tosupport the Safety Argument;

planned development stages of the Safety Case and theirrelationship to the overall Programme milestones;

requirements for review and approval of the Safety Case;

arrangements for the proper handover of Safety Caseactivities or obligations;

arrangements for the approval of the Safety Case by theregulatory authorities;

arrangements for the maintenance of the Safety Case duringoperations.

8. Format, Structure and Layout of the Safety Case

Executive Summary

Introduction:

Background

This section presents guidance on Safety Case layout.

Examples, relating to EATM can be found in theEUROCONTROL Pre- and Post-Implementation Safety Casesfor RVSM, [2]and [3] respectively. More-recent examples canbe provided on request.

This should provide the reader with an overview of what theSafety Case is about, what it is trying to show and for whom, asummary of the conclusions and caveats (see below) andrecommendations (if any).

The Introduction should include:

an outline of, for example, the circumstances which led to


37/66



Aim

Purpose

Scope

Layout

Service / SystemDescription

Overall SafetyArgument

Claim

Criteria

Context

Justification

Principal SafetyArguments

High-level Assumptions

Safety Argument andEvidence sections

the need for, and development of, the Safety Case;

a simple statement of the aim ie whatthe Safety Caseseeks to demonstrate. It should be related directly to thetop-level Claim (see below);

the purpose of the Safety Case ie why, and for whom, it

has been produced;

the scope and boundary of the Safety Case. It is importantto explain what is included and what is not included;

the purpose of each of the sections of the document. Ingeneral, the main part of the document should be structuredalong the lines of the Safety Argument.

Provide a description of the system to which the Safety Caseapplies, including its operational environment, interfaces andboundaries of responsibility.

This section should describe and explain the highest levels ofthe Safety Argument structure, including:

the Claim ie the top-level statement which asserts that theservice / system (etc) is safe;

the Safety Criteria which define what is meant by safe in thecontext of the Claim;

a description of the operational context to which the SafetyCase applies;

the justification for the change, where the Safety Caseaddresses a change to a service and/or system that is not

being made mainly for reasons of improving safety, andtherefore potentially for incurring some risk;

the principal Safety Arguments ie the first level ofdecomposition of the top-level Claim these should bereasoned and well structured, showing how the SafetyCriteria are satisfied and the rationale for the approachtaken in the decomposition;

the key Assumptions on which the highest levels of theSafety Argument critically depend for example, the level ofrisk prior to the introduction of a change is acceptable.Other Assumptions, applicable to the lower levels of theSafety Argument structure should be included in the

Assumptions section see below.

These sections should present each of the principal SafetyArguments (see above) in turn, together with the supportingEvidence which shows that each of the Arguments is valid. It isrecommend that, where applicable, each section be structuredas follows:

Objective (of the section) related directly to the principalSafety Argument;

Strategy (breakdown of the principal Safety Argument into


38/66



Assumptions

Issues

Limitations

Conclusions

Recommendations

lower-level arguments);

Rationale (for the Strategy);

Lower-level Arguments / Evidence;

Conclusions (of section).

Present directly, and/or by reference, all the Assumptions onwhich the Safety Case depends, including the high-levelAssumptions mentioned above. Assumptions usually relate tomatters outside of the direct control of the organisationresponsible for the Safety Case but which are essential to thecompleteness and/or correctness of the Safety Case. EachAssumption must be shown to be valid or at least reasonableaccording to the circumstances.

List any outstanding safety issues that must be resolved beforethe Claim can be considered to be valid, together with theresponsibilities and timescales for clearing them.

State and explain any Limitations or restrictions that need to beplaced on the deployment and/or operation of the system.

Do not merely repeat the conclusions from each section here.The main conclusion should refer to the original Claim and, ifapplicable, reassert its validity, subject to the following caveats:

the Scope especially what the Safety Case does notcover;

the operational Context to which the Safety Case applies;

the Assumptions that have had to be made;

the outstanding Issues;

any Limitations placed on the deployment and/or operationof the service / system.

Recommendations are not mandatory and any that are madeshould not be temporary in nature. For example, it might beappropriate to make recommendations on the use of the SafetyCase by its recipients, but not concerning its approval.

Recommendations must not contain any statements that wouldundermine, or add further caveats to, the Conclusions.

9. Verifying the Safety Case

Further guidance on what to look for in developing andreviewing a Safety Case can be found in the detailed checklistsin Appendix C.


39/66



10. ESARR Compliance

Overview

ESARR 2

ESARR 3

Compliance with ESARRs may be used in support of a SafetyCase. However, as indicated below, ESARRs are largelyconcerned with processes and, therefore, compliance withESARRs should normally be used as BackingEvidence, not as

Direct Evidence21.

Reporting and Assessment of Safety Occurrences in ATM

The rationale for ESARR 2 is that the achievement of consistenthigh levels of aviation safety and the management of safety inATM require, as a priority, the successful implementation ofharmonised occurrence reporting and assessment schemes.Such schemes will lead to more systematic visibility of safetyoccurrences and their causes, and will allow identification ofappropriate corrective actions as well as areas where flight safetycould be improved by changes to the ATM system.

Compliancewith ESARR 2 can be used as (Backing) Evidenceof having an adequate in-service safety monitoring process insupport of:

a Project Safety Case see for example Arg4 / St005 inFigure 11 of Chapter 5;

a Unit Safety Case see for example Arg2.5 in Figure 12 ofChapter 5.

Use of Safety Management Systems by Service Providers

The rationale for ESARR 3 is that an ATM service provider has aresponsibility to ensure that all relevant safety issues have been

satisfactorily dealt with, and to provide assurance that this hasbeen done. Safety management is that function of serviceprovision, which ensures that all safety risks have been identified,assessed and satisfactorily mitigated. A formal and systematicapproach to safety management will maximise safety benefits in avisible and traceable way.

Because a typical SMS includes a very wide range of safetyprocesses in support of ATM operations, compliance with ESARR3 can be used in many areas of a Safety Case - for example:

in a Project Safety Case, SMS processes could be used asBackingEvidence under Arg2.1 and Arg2.2, in Chapter 5,

Figure 8 andFigure 9 respectively;

in a Unit Safety Case, SMS processes could be used asDirectEvidence under Arg2.2 / St003, in Figure 14 ofChapter 5.

In both cases, the SMS-process Evidence would be strengthenedif it could be shown that the SMS was compliant with ESARR 3 ie ESARR 3 compliance would provide BackingEvidence.

21A possible exception to this statement could be a Safety Case which is based mainly on thesatisfaction of the requirements of the ESARR 4 Risk Classification Scheme.


40/66



ESARR 4 Risk Assessment and Mitigation in ATM

ESARR 4 concerns the use of Risk Assessment and Mitigation,including hazard identification, in Air Traffic Management whenintroducing and/or planning changes to the ATM System. In this

ESARR 5

ESARR 6

requirement, Risk Assessment and Mitigation are addressed via atotalaviation-system approach.

There are two aspects of ESARR 4 which can be used in a SafetyCase, as follows:

subjective to the provisos of Chapter 4, section 2above, theRisk Classification Scheme in Appendix A to ESARR 4 couldbe used as the basis of quantitativeSafety Criteria see, forexample, Cr001 / #1 in Chapter 5, Figure 1;

compliance with the qualitative(process) requirements ofsection 5 of ESARR 4 could be used as Backing Evidence as,for example, in Arg 1.10 inChapter 5, Figure 6.

ATM Services Personnel

ESARR 5 documents general safety regulatory requirements forall ATM services personnel responsible for safety related taskswithin the provision of ATM services across the ECAC area,including the specific safety regulatory requirements for air trafficcontrollers and engineering/technical personnel.

Compliance with ESARR 5 could be used in a Unit Safety Caseto show that the human aspects of the on-going operation arebased on, inter alia, competent personnel. This would appear, forexample in the lower levels of decomposition of Arg2.1 andArg2.2 in Chapter 5, Figure 14.

Software in ATM Systems

ESARR 6 deals with the implementation of software safetyassurance systems to ensure that the risks associated with theuse of software in safety-related ground-based ATM systems arereduced to a tolerable level.

Compliance with ESARR 6 could be used, for example, in:

a Project Safety Case - software processes could be used asBackingEvidence under Arg2.1 / St011 and Arg2.2 / St015,in Figure 8and Figure 9respectively of Chapter 5 ;

a Unit Safety Case - software processes could be used asDirectEvidence under Arg2.2 / St003, in Figure 14ofChapter 5.

22DirectEvidence would be the outputs of those processes


41/66

Safety Case Development Manual GSN Safety Argument Examples


CHAPTER 5Guidance -

GSN Safety Argument Examples

1. Example Application of GSN A Project Safety Case

Relationship to theEUROCONTROL SAM

Figure 1to Figure 11overleaf show a structured SafetyArgument for a hypothetical substantialchange (SGxy) to anATM service which could form the basis of what is known herein

as a ProjectSafety Case.

The structure is intentionally not complete in all areas of thedecomposition; however, it is intended to be sufficient, inbreadth and depth, to illustrate the use of the GSN notation. Acommentary on the development of the Safety Argument is alsoprovided below. This commentary is also not exhaustive but isintended to bring out all the main points concerning theapplication of GSN.

Where applicable references are given to those process(es) inthe SAM [5]that would generate the required Evidence.

Claim

Justification

Context

The Safety Argument starts, in Figure 1, with the top-levelClaim(Arg 0)that the ATM service, following the change, willbe acceptably safe.

J001indicates that the change is justified operationally and thisjustification would need to be elaborated in the Safety Case.

C001provides an essential marker that the change itself needsto be defined in terms of the ATM service / system andaccompanying operational concept such descriptions wouldneed to be provided in the related Safety Case.


42/66



.

Change_SGxy willbe acceptably safe

in operationalservice

Arg 0

St 001

Specify safety criteria for each of the4 main life-cycle stages and show thateach stage is / will be acceptably safe ie the safety criteria are sufficient toachieve the required level of safety,and are satisfied

C001Operational concept

(SAM-FHA ch1-GM A)

Change_SGxyImplementationis acceptably safe

Arg 2Change_SGxyConcept isacceptably safe,in principle

Arg 1

A001Current ATM serviceis accepted as being safe

J001Change_SGxy is being

introduced to meet alegitimate operational need

Cr001

The risk of an accident followingChange_SGxy shall be:

1.Within the regulatory requirements eg:

a. such that the whole ATM servicemeets ESARR 4 Design SafetyTargets (SAM-FHA ch3 GM E); OR

b. no greater (and preferably lower)than currently exists.

AND

2. reduced as far as reasonablypracticable.

Migration toChange_SGxywill beacceptably safe

Arg 3

On-going Operationof Change_SGxy willbe shown to beacceptably safe

Arg 4

Fig 2 Fig 7

St004

Show that risk during(and immediatelyfollowing) Migration willsatisfy Cr001 item 2

Fig 11

St005

Safety Monitoringwill satisfy Cr001items 1 and 2

Fig 10

St002

Show that SafetyRequirements satisfy Cr001:

item 1 (Args 1.1 & 1.2);

Item 2 (Arg 1.3)

St003

Show that the Safety

Requirements are satisfied:1. Initially in system design

2. Subsequently in therealisation of that design

C002Subject to declaredAssumptions, Limitationsand outstanding Issues

.



Arg 0



Arg 0

St 001


St 001


C001Operational concept

(SAM-FHA ch1-GM A)


Arg 2


Arg 2Change_SGxyConcept isacceptably safe,in principle

Arg 1

Change_SGxyConcept isacceptably safe,in principle

Arg 1

A001Current ATM serviceis accepted as being safe

J001Change_SGxy is being

introduced to meet alegitimate operational need

Cr001





AND


Cr001





AND



Arg 3


Arg 3

On-going Operationof Change_SGxy willbe shown to beacceptably safe

Arg 4On-going Operationof Change_SGxy willbe shown to beacceptably safe

Arg 4

Fig 2Fig 2 Fig 7Fig 7

St004


St004


St004


Fig 11Fig 11

St005


St005


St005


Fig 10Fig 10

St002


item 1 (Args 1.1 & 1.2);

Item 2 (Arg 1.3)

St002


item 1 (Args 1.1 & 1.2);

Item 2 (Arg 1.3)

St003




St003




St003




C002Subject to declaredAssumptions, Limitationsand outstanding Issues

Figure 1 Arg 0: Safety Argument

Safety Criteria

Choice of Criteria

Assumption

Arg1 to 4

Acceptably safe is defined in terms of the three criteriasummarised in Cr001. These criteria reflect the three mainways of expressing a Safety Argument ie:

Absolutely: as compliance with a (numerical) target level of

safety eg ESARR 4 (or local regulatory interpretationthereof), OCP, ICAO, JAR 25 etc;

Relatively: in relation to a current / previous (usuallyqualitative) level of safety;

Reductively: showing risk to be further reduced as far asreasonably practicable;

The first two bullets are alternative ways of expressing a typicalregulatory minimum safety level23and specify what issometimes known as tolerablerisk. In the further developmentof the Safety Argument for Change SGxy, only the absolute

criterion is actually used24

, and is supported by the reductivecriterion in order to specify what is sometimes known as anacceptablelevel of risk.

If a relative Argument, were to be used it would be necessary toestablish that the pre-change baseline is safe. This is addressedin A001 which is shown on Figure 1 for illustration only.

As indicated in StrategySt001, ClaimArg 0is decomposed into

23It would not normally be necessary to comply with both.24For examples of developing Safety Arguments using relative criteria, please contact DAP/SAF


43/66

Safety Case Development Manual GSN Safety Argument Examples


four principal Arguments which, in this case, relate to the fourmain, contiguous stages of the lifecycle of the Change. Theoutcome of each stage is argued to be acceptably safe andSt002to St005are used to indicate, by reference to Cr001whatis defined as acceptably safe for each stage:

Arg 1(through St002)asserts that the Change is acceptably

safe in principle ie subject to subsequent complete andcorrect implementation of the Safety Requirements;

Arg 2 (through St003)asserts that the Implementation ofthe Change is acceptably safe, through satisfaction of theSafety Requirements, and that the rigour of the Assurance(ie lower-level Argumentsand Evidence) to support this isappropriate to the risk associated with the Change;

Arg 3(through St004)asserts, in effect, that the Migrationfrom the current state to the po