s2.sys.pl.1001 software pa plan issue 2 -...

Doc No S2.SYS.PL.1001 Issue : 2 Date : 12.07.2005

LISA Pathfinder Sheet : 2

The contents of this document are the copyright of SciSys Limited and shall not be copied in whole, in part or otherwise reproduced (whether by photographic, reprographic or any other method) and the contents thereof shall not be divulged to any other person or organisation without the prior written

consent of SciSys Limited.

S2.SYS.PL.1001 Software PA Plan Issue 2

INTENTIONALLY BLANK






CONTENTS

1 SCOPE..................................................................................................................................................6 1.1 Applicability........................................................................................................................................6 1.2 Applicable and Reference Documents ..............................................................................................7

1.2.1 Applicable Documents ...........................................................................................................7 1.2.2 Standards Documents ...........................................................................................................7

1.3 Reference Documents.......................................................................................................................8

2 ACRONYMS..........................................................................................................................................9

3 SOFTWARE PRODUCT ASSURANCE MANAGEMENT...................................................................10 3.1 Organisation and Responsibilities ...................................................................................................10 3.2 Software Product Assurance Reporting ..........................................................................................11 3.3 Risk Management and Critical Item Control ....................................................................................11

4 PROBLEM REPORTING AND CORRECTIVE ACTION ....................................................................12 4.1 Software Non-Conformance and Problem Reporting......................................................................12 4.2 Waivers and Deviations...................................................................................................................14 4.3 Alerts................................................................................................................................................14

5 SOFTWARE LIFE CYCLE DEFINITION.............................................................................................15 5.1 Overview..........................................................................................................................................15 5.2 System Requirements Engineering Process...................................................................................15 5.3 Software Requirements Engineering Process.................................................................................15 5.4 Design Engineering and Coding Process........................................................................................16

5.4.1 Automatic Code Generation ................................................................................................16 5.5 Software Verification and Validation Process..................................................................................16 5.6 Software Installation and Acceptance Process ...............................................................................16 5.7 Operations and Maintenance Process ............................................................................................17

6 SOFTWARE PRODUCT AND PROCESS QUALITY APPROACH....................................................18 6.1 Software Quality Objectives and Metrication...................................................................................18

6.1.1 Definition of the Software Quality Model .............................................................................18 6.1.2 Definition of Software Metrics ..............................................................................................20 6.1.3 Definition of Metrication Process .........................................................................................23

6.2 Software Dependability for Critical Functions..................................................................................23 6.2.1 Criticality Definition ..............................................................................................................23 6.2.2 Critical Function Identification and Tracking........................................................................24 6.2.3 Dependability Activities........................................................................................................24 6.2.4 Independent Software Verification and Validation (ISVV) ...................................................25 6.2.5 Hardware/Software Interaction Analysis..............................................................................25

7 REVIEWS, AUDITS AND INSPECTIONS ..........................................................................................26 7.1 General ............................................................................................................................................26 7.2 Software Audits................................................................................................................................26 7.3 Software Reviews ............................................................................................................................26

7.3.1 General Review Criteria.......................................................................................................26






7.3.2 System Requirements Review (SRR) .................................................................................27 7.3.3 Preliminary Design Review (PDR).......................................................................................27 7.3.4 Critical Design Review (CDR)..............................................................................................28 7.3.5 Qualification Review (QR) ...................................................................................................28 7.3.6 Acceptance Review (AR).....................................................................................................28 7.3.7 Incremental Delivery Reviews (DRB) ..................................................................................29

7.4 Software Inspections .......................................................................................................................29 7.4.1 Incoming Inspection of delivered Software..........................................................................29 7.4.2 Product Assurance Inspections at every Milestone.............................................................29

8 SOFTWARE DOCUMENTATION .......................................................................................................30 8.1 General ............................................................................................................................................30 8.2 Development Documents ................................................................................................................30 8.3 Product Assurance Documents, Plans, and Procedures ................................................................31 8.4 Software Verification and Validation Documents ............................................................................31

9 SOFTWARE CONFIGURATION MANAGEMENT..............................................................................33 9.1 Change Control................................................................................................................................33 9.2 Protection and Marking....................................................................................................................33 9.3 Incremental / Early Delivery of Software .........................................................................................34

10 SUBCONTRACTOR CONTROL.........................................................................................................35 10.1 Subcontractor Selection...............................................................................................................35 10.2 Software Product Assurance Requirements for Subcontractors.................................................35 10.3 Subcontractors Statement of Work..............................................................................................35 10.4 Review of Subcontractor Plans ...................................................................................................35 10.5 Supplier Monitoring......................................................................................................................35

11 COTS AND REUSED SOFTWARE ....................................................................................................36 11.1 COTS Software............................................................................................................................36 11.2 Reused Software .........................................................................................................................36

11.2.1 Definition..............................................................................................................................36 11.2.2 Assessment Approach.........................................................................................................36 11.2.3 Selection Criteria .................................................................................................................37

ANNEX 1 LISA-PATHFINDER COMPLIANCE MATRIX FOR ECSS Q-80B ..............................................38






Table of Figures

Figure 3.1-1 LISA Pathfinder SQA Quality Organisation...............................................................................10 Figure 4.1-1 Handling of SPR/OBS, NCRs, and RFW/RFDs between contractors – showing SPR/OBS

details ......................................................................................................................................................13






1 SCOPE

This Software Product Assurance Plan (SWPAP) defines the Software Product Assurance organisation, responsibilities, and activities for the Implementation phase of the LISA Pathfinder programme. The SWPAP is applicable to LISA-Pathfinder Prime and it’s subcontractors.

On-Board Software

It is not planned to develop any on-board software in the frame of Astrium’s prime function for LISA Pathfinder but to procure such software from LISA Pathfinder core team members and subcontractors. Therefore, Astrium through the support of SciSys SQA function will only perform a control function regarding the SciSys avionics on-board software and subcontractors’ software quality and development processes. A dedicated Software Product Assurance Requirements for Subcontractors document supports and complements the software product assurance programme laid out herein.

Other Software

Experience with other satellite projects has shown that particular emphasis will be given on software quality aspects for Commercial-Off-The-Shelf (COTS) and reused software. The software quality activities specifically related to these types of software are defined in section 11 of this plan.

This plan has been prepared in response to the requirements defined in ECSS-Q-80B [SD10] and the LISA Pathfinder Product Assurance and Safety Requirements [AD1]. This plan is concerned only with the software related elements of the programme, and complements the overall LISA-Pathfinder Product Assurance Plan [AD2].

The prime quality objectives of the software product assurance programme defined in this plan are the following:

- Reliability, functional and temporal correctness of the software, - Efficiency for real-time embedded software, - Maintainability, - Correctness, completeness, and consistency of the software documentation.

Section 5 of this plan defines a software quality model supporting these objectives.

This Software Product Assurance Plan is maintained throughout the project life cycle such that for each milestone the activities to be undertaken during the following development phase are fully defined.

1.1 Applicability

The Software Product Assurance programme defined here is applicable to the LISA Pathfinder project Implementation phase activities.

The Software Product Assurance programme defined in this plan applies to following types of software:

- On-board software; - ASIC/FPGA designs; - EGSE, MDVE; - On-Board Control Procedures (OBCP);

If not otherwise stated the term “software” comprises all types of software listed above. The term “subcontractor” in this plan refers to those suppliers developing any kind of software for and/or within the LISA Pathfinder project.






1.2 Applicable and Reference Documents

1.2.1 Applicable Documents

The following documents are applicable to the Software Product Assurance Plan, and are hierarchically superior to it:

[AD1] S2.EST.RS.1004 SMART-2 / LISA Pathfinder Product Assurance & Safety Requirements

[AD2] S2.ASU.PL.1003 LISA Pathfinder Product Assurance and Safety Plan

[AD3] SCI-PJ-EST-SOW-1001 LISA-Pathfinder SoW

[AD4] S2.ASU.PL.1009 LISA Pathfinder Risk Management Plan

[AD5] S2.ASU.PL.1017 LISA Pathfinder Audit & Surveillance Plan

[AD6] S2.ASU.PL.1014 LISA Pathfinder Configuration & Data Management Plan

[AD7] S2.SYS.PL.1002 Software Development Plan

1.2.2 Standards Documents

The following standards documents (latest issue at contract signature) will be applicable to the extent and with the modifications specified in this document.

[SD1] ECSS-Q-00 Policy and principles

[SD2] ECSS-Q-20 Quality Assurance

[SD3] ECSS-Q-30 Dependability

[SD4] ECSS-Q-40 Safety

[SD5] ECSS-M-40A Configuration Management

[SD6] ECSS-Q-20-09 Non-Conformance Control System

[SD7] ECSS-E-10-05 Functional Analysis

[SD10] ECSS-Q-80B Software Product Assurance

[SD11] ECSS-E-40B Software Engineering Standard






1.3 Reference Documents

The following documents provide reference material for understanding of this plan.

[RD1] IEEE Std 730-1998 IEEE Standard for Quality Assurance Plans

[RD2] ECSS-Q-30-02A FMECA

[RD3] ECSS-Q-40-02 (draft) Hazard analysis

[RD4] ISO 9001:2000 Quality management system: Requirements

[RD5] SSRL/QMS/PRC/25 SciSys Quality Manual: Purchasing and Receiving Procedure

[RD6] S2.ASU.LI.1006 LISA-Pathfinder Acronyms List






2 ACRONYMS

Refer to [RD6]

Note also that an Observation (OBS) is a term used in the SciSys Quality Manual, [RD5], for a Software Problem Report (SPR).






3 SOFTWARE PRODUCT ASSURANCE MANAGEMENT

3.1 Organisation and Responsibilities

A software product assurance representative from SciSys is assigned as Software Product Assurance Manager, responsible on behalf of Astrium Ltd. for all activities described in this plan. This person has overall responsibility for the preparation, maintenance, and implementation of this Software Product Assurance Plan.

The Software PA manager has responsibility for approving the initial issue and all subsequent changes to the SWPAP, and has responsibility for ensuring that all software related activities are completed according to the PA model defined within this plan. The Software PA Manager will participate in the preparation, detailed review, and negotiation of the product assurance provisions of software contracts and in the assessment and review of all changes to the contractual requirements.

The Software PA Manager is a member of SciSys PA line organisation and reports functionally to the Group Quality Manager of SciSys and Astrium Ltd. The PA Managers have unimpeded access to higher management.

The Figure 3.1-1 below gives an overview of the LISA Pathfinder quality organisation and shows the scope of the software product assurance programme defined in this plan.

Figure 3.1-1 LISA Pathfinder SQA Quality Organisation






3.2 Software Product Assurance Reporting

The status and progress of the software product assurance programme is regularly reported (see LISA Pathfinder Product Assurance Plan [AD2], to the project manager and the customer via the overall product assurance reporting. This reporting also includes a synthesis of the software product assurance reports provided by the core team and subcontractors.

In addition, software product assurance reports submitted for milestone reviews contain an assessment of the current quality of the software products and processes, addressing verifications undertaken, problems detected and resolved. This assessment is made with reference to the software metrication programme defined in this plan. The software product assurance reports contain an overview of software process and product metrics relevant to the reporting period. The purpose of the reporting is to provide the customer with an insight into the level of quality obtained for the software product.

Furthermore, a summary of verification activities undertaken for each phase and corrective actions implemented are included in the regular software product assurance reporting. The results of the reported verification activities are summarised in the respective review minutes or in a verification report.

3.3 Risk Management and Critical Item Control

See LISA Pathfinder Risk Management Plan [AD4].






4 PROBLEM REPORTING AND CORRECTIVE ACTION

4.1 Software Non-Conformance and Problem Reporting

Non-conformances (NCRs) and Software Problem Reports (SPRs) (or OBS – Observations in the Scisys terminology) related to software items delivered by subcontractors are handled according to the non-conformance procedure provided in the LISA Pathfinder Product Assurance Plan [AD2].

An SPR/OBS can be any comment concerning a system, the design or documentation relating to a system, or any product, component or module. An SPR/OBS may be a reported or a perceived fault, or non-conformity, or it may be a suggestion of a better way of doing something. It sometimes reflects a failure to convey the system’s requirements from the author to the reader.

The definition of NCRs and SPRs/OBSs that will be used is that any problem found during any stage of the software development process will be raised as a SPR/OBS, whereas an NCR will only be raised by a higher level in response to a specific requirement not being met. Where an NCR is raised in response to an SPR/OBS then the NCR will reference the associated SPR/OBS, and the NCR will not be closed until the associated SPR/OBS has been closed.

If the problems raised by locally generated SPRs/OBSs, or those passed down via NCRs from a higher level contractor, cannot be fixed then a Waiver or Deviation will be raised.

Requests for Waivers or Requests for Deviations from subcontractors, if applicable, are submitted to the LISA-Pathfinder Prime CRB and are reviewed and accepted or rejected by them.

The relationship between LISA-Pathfinder Prime and any of it’s subcontractors is summarised below. The general handling of SPRs/OBSs, NCRs and RFWs/RFDs between a higher and lower level contractors is shown as follows:






Figure 4.1-1 Handling of SPR/OBS, NCRs, and RFW/RFDs between contractors – showing SPR/OBS details

Approve or Reject

RFW/RFD

Raise SPR/OBS in response to: - Requirements - Design - Analysis - Inspection - Test - Errors - Changes

Is an RFW/RFD

appropriate?

Is the problem related to a

Subcontractor?

Report all SPR/OBS to the

Customer

Fix the Problem

No No Yes Raise an

RFW/RFD and submit to the

Customer

Yes

NCR with SPR/OBS Reference to

Subcontractors

Close SPR/OBS

Raise NCR and submit to

Subcontractor (with

SPR/OBS Reference)

SPR/OBS Related to a

Subcontractor

SPR/OBS references from a higher level

Customer or Higher Level Contractor

Subcontractor or Lower Level Contractor






All software anomalies or errors detected during the system-level AIT activities will be recorded as non-conformances and distributed to the responsible subcontractor as shown for analysis and close-out. On prime level this procedure applies to each subcontracted software item during its development, starting with the subcontractor’s first delivery of the software item.

Each software subcontractor will be required to maintain their own problem tracking and reporting system, which will be used by them to track problems and Non Conformances reported by the prime, and also to track problems discovered during their own verification activities. They will be required to maintain records of the problem reports so that these can be reported as part of the project metrics. All problems discovered by a subcontractor will be reported to the prime immediately. The LISA Pathfinder prime ensures and monitors that subcontractors apply procedures on their level starting with successful completion of software unit tests.

The Software Review Board includes, at least, representatives from software product assurance and software engineering.

The use of the ESA NCTS tool for the reporting and tracking of software non-conformances is currently proposed to be used within the prime team and with all software subcontractors.

4.2 Waivers and Deviations

A Waiver is a written authorisation to use or release a product which does not conform to the specified requirements, identified during or after design, coding or test, including non-conformances affecting function, mission objectives, safety, performance, reliability or maintainability and, if agreed, will not be rectified thereby rendering the item permanently non-conforming. When approved, waivers are entered into the ‘As Built’ Configuration in the Software Configuration File (SCIDL).

A Deviation is a written authorisation to depart from a requirement prior to production and test of a configured item. When approved, deviations are entered into the ‘As Designed’ Configuration in the Software Configuration File (SCIDL).

Each subcontractor will notify the customer through the use of a Request for Waiver (RFW) or Request for Deviation (RFD) as shown above in: Figure 4.1-1 Handling of SPR/OBS, NCRs, and RFW/RFDs between contractors – showing SPR/OBS details.

The status of all RFWs and RFDs will be included in the periodic reporting in the Software Configuration File (SCIDL).

4.3 Alerts

Software alerts will be handled in the same manner as all other alerts, see LISA Pathfinder Product Assurance Plan [AD2].






5 SOFTWARE LIFE CYCLE DEFINITION

5.1 Overview

This section defines the software development life cycle processes applicable all the LISA-Pathfinder software development activities. This includes the interface between the overall software development activity and the platform engineering, and also the interface between SciSys and the software subcontractor.

The scope of the processes on LISA Pathfinder prime level listed below is limited to monitoring and evaluating the activities and performance of the corresponding subcontractor processes. • Software requirements engineering process; • Design engineering and coding process; • Software Verification and Validation process; • Operations and Maintenance Process.

The full details of the Software Lifecycle are described in the LISA Pathfinder Software Development Plan [AD7], which includes the tailoring of ECSS E-40B [SD11] for the project.

The monitoring and control function is mainly achieved by means of software reviews and via the interface between the prime’s SPA representative and its counterpart on the core team and subcontractor level.

5.2 System Requirements Engineering Process

As far as software is concerned the system requirements engineering process consists of the following main activities: • System requirements analysis; • System and functional criticality analysis; • System partitioning with definition of items; • Definition of system level requirements for software integration, verification and validation; • Definition of requirements for software operations and maintenance.

For each software item (e.g. electronic equipment containing software) identified by the partitioning of the system, the system requirements analysis consists of the following tasks:

• Establish and document software requirements in the Requirements Baseline (RB), including non-

functional requirements, e.g. requirements related to quality, dependability, portability, and resources. • Define dependability requirements for critical software functions as derived from the criticality analysis of

the system; • Identify each requirement uniquely within the system in order to allow for traceability to higher-level

requirements; • Evaluate the software requirements for consistency, completeness, understandability, and testability.

The System Requirements Review (SRR) for the software concludes the software system requirements engineering process, resulting in an agreed set of requirements for the software. The scope and verification criteria for this review are described in Section 7.3.2. The Requirements Baseline produced by this process serves as an input to the Software Requirements Engineering Process at subcontractor level.

5.3 Software Requirements Engineering Process

The software requirements engineering process on LISA Pathfinder prime level comprises the monitoring and evaluation of the subcontractor’s software requirements engineering and architectural design activities.

The Preliminary Design Review (PDR) concludes the software requirements engineering process. The scope and verification criteria for this review are described in Section 7.3.3






5.4 Design Engineering and Coding Process

The design engineering and coding process on LISA Pathfinder prime level comprises the monitoring and evaluation of the subcontractor’s software design, coding, testing, and integration activities. The software design will be monitored against the agreed design standard and all coding will be monitored against the agreed coding standard. The design and code standards will be agreed with the prime prior to the start of the design and coding process at the PDR.

The Critical Design Review (CDR) concludes the design engineering and coding process. The scope and verification criteria for this review are described in Section 7.3.4

5.4.1 Automatic Code Generation

For all software produced by Automatic Code Generation (ACG) techniques, the baseline is that the same PA processes will be applied as for normally generated code to ensure the code quality. However, the results of the current ESA study into the PA aspects of ACG will be taken into account when available.

5.5 Software Verification and Validation Process

This process comprises the monitoring and evaluation of the subcontractor’s software verification and validation activities.

The verification activities will include unit and integration level tests against the software design, and validation tests against the technical specification (TS). For each level of testing, the tests will be reviewed to ensure that they have clear goals and that the actual tests performed meet these goals. For any requirements that are not covered by tests, a suitable V&V approach will be agreed, and the subcontractor will provide appropriate evidence that the required activity has been successfully completed (ie by analysis, review of design etc).

The validation activities will commence after successful completion of the CDR, and will be performed in 2 stages. For ‘SW only’ subcontracts, the supplier will perform validation of the software against the TS and will be completed prior to the QR, and then the customer will then performs second stage of validation against the RB (Acceptance testing) which will be performed prior to the AR, as defined in Section 5.6 below.

The Verification and Validation process described here is a tailoring of the process defined in ECSS E-40B [SD11], as stated in the Software Development Plan [AD7].

The Qualification Review (QR) concludes the supplier’s software verification and validation process. The scope and verification criteria for this review are described in Section 7.3.5.

5.6 Software Installation and Acceptance Process

When a subcontractor is ready to deliver the qualified software item, the LISA Pathfinder prime judges whether or not the software is acceptable according to criteria and in a manner specified in the contract or Statement of Work (SoW). The acceptance of a software item is approved by means of a formal acceptance test campaign. It is the responsibility of the prime, supported by subcontractor, to define the acceptance tests.

For ‘SW only’ subcontracts, the acceptance test campaign will be combined with the customer validation of the software against the RB.

The software installation and acceptance process consists of the following tasks: • Acceptance test planning; • Generation and installation of executable code from the software configuration file; • Execution of the acceptance tests on the target environment; • Conducting the Acceptance Review against the applicable requirements baseline (RB), with support

from the supplier of the software item;






• Evaluation and documentation of the acceptance test results.

Any problems discovered during generation, installation, or acceptance testing of the software are documented in non-conformance reports. On completion of the review the SPA produces a report, which is signed by representatives of all parties involved (supplier, Astrium prime, ESA). The purpose of this report is to certify conformance to the applicable requirements baseline and state the conclusion concerning the test result for the software item (accepted, conditionally accepted, rejected).

The Acceptance Review (AR) concludes the software installation and acceptance process. The scope and verification criteria for this review are described in Section 7.3.6.

5.7 Operations and Maintenance Process

The operations and maintenance process begins after the Acceptance Review has been completed successfully and lasts until the end of the contract.

The inputs to this process are the accepted software items and the Acceptance Review reports for each software item.

The outputs from this process are the maintenance plans and procedures.

The LISA Pathfinder prime ensures that the organisation responsible for maintenance establishes and maintains plans and procedures for performing maintenance activities and support of the operation of the software product. The maintenance organisation will specify the assurance, verification and validation activities applicable to maintenance interventions by the end of Initial Development Phase. (SW Maintenance File).

The maintenance plans and procedures include the following: • Scope of maintenance; • Identification of the initial status of the software product; • Support organisation; • Maintenance activities; • Maintenance records and reports.

Maintenance records include the following information • List of requests for assistance or problem reports that have been received and the current status of

each; • Organisation responsible for responding to requests for assistance or implementing the appropriate

corrective actions; • Priorities that have been assigned to the corrective actions; • Results of the corrective actions; • Statistical data collected on failure occurrences and maintenance activities.

Furthermore, the LISA Pathfinder prime ensures that: • Plans and procedures are verified against specified requirements for maintenance of the software

product; • All changes to the software product are documented in accordance with the procedures for document

control and configuration management.






6 SOFTWARE PRODUCT AND PROCESS QUALITY APPROACH

6.1 Software Quality Objectives and Metrication

The software quality objectives that are applied to guide the development of software items are expressed as a set of software quality factors. These factors are further subdivided into quality criteria, which may be supported by a set of software product metrics.

6.1.1 Definition of the Software Quality Model

The following table identifies the quality factors of the LISA Pathfinder software quality model:

Factor Description

Functionality (see 6.1.2.1)

The capability of software to provide functions which meet stated and implied needs when the software is used under specified conditions.

Reliability (see 6.1.2.2)

The capability of the software to maintain its level of performance when used under specified conditions.

Maintainability (see 6.1.2.3)

The capability of the software to be modified. Modifications may include corrections, improvements, or adaptation of the software to changes in environment, and in requirements and functional specifications.

Documentation Quality Those attributes of the software that determine the adequacy of the documentation related to software development, maintenance, and operation. This quality factor is not further split down into supporting quality criteria/metrics.

Documentation quality will be assured by means of reviews and inspections.

Software Development Effectiveness (see 6.1.2.4)

The degree of success/quality of the software development process to which the software has been subjected, which provides valuable indications of the product quality.






The table below defines for each quality factor a set of criteria contributing to the factor. Factor Criterion Criterion Description

Completeness The capability of the software to provide full implementation of the functions required.

Correctness The degree to which a system or component is free from faults in its specification, design, and implementation.

Functionality

Efficiency The capability of the software product to provide appropriate performance, relative to the amount of resources used, under stated conditions.

Reliability Evidence The capability to show that software reliability analyses and assessments have been performed during the software development process.

Reliability

Robustness The extent to which a software continues to perform its function in presence of hardware failures, operating system and software faults, non-planned input, and faults in interfacing software.

Testability Characteristics of the source code which are used to assess the effort necessary to validate the modified software.

Stability Those characteristics of the source code that are used to assess the risk of unexpected effects resulting from modifications.

Changeability Characteristics of the source code which are used to assess the effort necessary to modify the source code, correct errors, or change environment.

Analysability Characteristics of the source code which are used to assess the effort necessary to diagnose the causes of the deficiencies and/or failures, or used to identify the portions to be modified.

Maintainability

Modularity Those attributes of the software that provide a structure of highly cohesive modules with optimum coupling.

SW Development Effectiveness

Process Effectiveness

Evidence that shows which activities have been performed during the development process to adhere to standards, conventions and regulations considered relevant for the evaluation of the software quality.






6.1.2 Definition of Software Metrics

6.1.2.1 Metrics for Quality Factor “Functionality”

Criterion Metric Metric Definition Limit

RB_COV Ratio between number of correctly implemented requirements confirmed by tests and total number of requirements defined in requirements baseline.

100% Completeness

TS_COV Ratio between number of correctly implemented requirements confirmed by test and total number of requirements defined in technical specification.

100%

C0_COV Ratio between number of statements covered by tests and the total number of executable statements.

100%

C1_COV Ratio between number of branches covered by tests and the total number of branches (per software component).

100%

Correctness

FAULT_DENS Ratio between number of faults detected and total number of executable statements (in KLoC).

-

MEM_UTIL Ratio between the memory size used and the total memory size available.

75% Efficiency

CPU_UTIL Processor utilisation. 75%

6.1.2.2 Metrics for Quality Factor “Reliability”

Criterion Metric Metric Definition Limit

Reliability Evidence REL_ADEQ Scores achieved in checklist to be proposed by the contractor.

-

Robustness MTTF Mean-Time-To-Failure -






6.1.2.3 Metrics for Quality Factor “Maintainability”

The software product metrics described in this Section support the quality factor “Maintainability”. The entities of the call graph, i.e. functions and procedures, form the basis for the product metrics assessment. Key metrics listed below in bold type are mandatory for contractors. All other metrics are advisory.

Value RangeMetric

Test

abili

ty

Stab

ility

Cha

ngea

bilit

y

Ana

lysa

bilit

y

Mod

ular

ity

Min

.

Max

.

Description

comf

comr

0.2

0.2

3

∞

Number of blocks of comments per statement in the function; OR

Number of comment lines per executable statement in the function.

ct_goto 0 0 Number of GOTO statements.

ct_nest 0 8 Maximum number of control structure nesting in the function.

dc_vars 0 10 Total number of variables declared in a function (Local VARiables).

dc_calling 0 10 Number of other functions calling the designated function.

ic_params 0 5 Number of function parameters.

ct_path 1 80 Number of non-cyclic execution paths.

ct_exit 0 1 Number of nodes associated with an explicit exit from a function (RETURN, EXIT).

lc_stat 1 100 Number of statements in a function’s body.

ct_vg 1 15 McCabe cyclomatic complexity number of the function.

cg_hiercpx 1.00 5.00 Average number of components per level (number of components divided by number of levels).

cg_levels 1 15 Number of relative call-graph levels.

cg_strucpx 0.00 3.00 Average number of calls per component (number of calls between components divided by the number of components).

Overview of Code Complexity Metrics






A static source code analysis tool provides the means for collecting, evaluating, and reporting software product metrics related to the quality factor “Maintainability”. A description of these software product metrics is available electronically (Logiscope Audit Basic Concepts Manual, Section 6).

Each quality criteria is assigned a rating level according to the table below. This rating is performed separately for each software component.

Criteria Rating Description Weight

EXCELLENT All metrics within range 3

GOOD One metric out of range 2

FAIR Two metrics out of range 1

POOR All metrics out of range 0

The overall rating of the quality factor “Maintainability” is calculated from the individual ratings of its contributing criteria by using the scheme defined in the table below:

Maintainability Factor Rating

Avg. Weight Range of Contributing Criteria

EXCELLENT x = 3

GOOD 2 ≤ x < 3

FAIR 1 ≤ x < 2

POOR 0 ≤ x < 1

The scheme above provides an overall classification for each software component. Components classified as “POOR” are considered not acceptable in general and require further assessment, for example, by means of manual code inspection. This also applies if any of the key metrics are out-of-range for a particular software component.

6.1.2.4 Metrics for Quality Factor “Software Development Effectiveness”

The following list of software process metrics is used to assess the effectiveness of the software development processes: • Number of RIDs raised during a review, including:

• Minor versus major RIDs; • RIDs per reviewed document; • RIDs per reviewing organisation

• Number of open and closed non-conformances or Software Problem Reports; • Number of open and closed change requests; • Number of open and closed requests for waiver; • Number of open and closed actions (including actions initiated by software product assurance and

reviews). • Development duration against schedule • Development effort against schedule






6.1.3 Definition of Metrication Process

The following processes will be applied for each of the different classes of metrics that are collected.

Factor Collection Process Remedial Actions

Functionality These metrics will be collected and reviewed at the CDR, QR, and AR. .

For any units that are found not to meet the target values then a justification will be provided or a suitable corrective action proposed

Reliability These metrics will be collected and reviewed at the CDR, QR, and AR. .


Maintainability These metrics will be collected and reviewed at the CDR, QR, and AR. .


Documentation Quality The documentation quality will be reviewed at all reviews (SRR, PDR, CDR, QR, and AR).

Actions will be performed in response to RIDS agreed at the review.

Software Development Effectiveness

These process metrics will collected per month and subsystem and summarised in the regular product assurance reporting.

The results and overall trends will be reviewed at each progress meeting and suitable corrective actions proposed .

In addition to the ongoing collection and assessment of metrics, the overall metrics from the project will be used as part of a ‘lessons learnt’ report at the end of the project as part of on-going process improvement.

6.2 Software Dependability for Critical Functions

6.2.1 Criticality Definition

To categorise the severity of a failure relative to its impact on the mission, a criticality level classification will be defined in accordance with the following basic classifications:

1. Catastrophic • loss of life, life threatening or permanently disabling injury to personnel or occupational

illness • loss of or damage to launcher, or launch site facilities • loss of spacecraft with safety related consequences • propagation of failure to launcher or other launcher payloads • long-term detrimental environmental effects

2. Critical • loss of LISA Pathfinder mission • short-term detrimental environmental effects

3. Major • mission degradation

4. Minor • other






This classification is taken from the LISA-Pathfinder PA Plan [AD2], and will be used to classify all software items in terms of their potential impact of the system.

6.2.2 Critical Function Identification and Tracking

The system functional architectural model is analysed and evaluated to support the identification of potential critical functions as part of a system-level FMECA. The output of this will be used to produce a Software Criticality Analysis report, which will identify the specific critical functions within the software. The RB will include the list of critical software requirements, identified either as specific functions or as requirements on functions as appropriate.

Critical functions related to software are listed and traced in the overall system critical items list (CIL). Procedures for critical item control are described in the LISA Pathfinder Product Assurance Plan [AD2].

The mitigation, control, and elimination of risks (risk management) associated with critical functions is guided by the “As Low As Reasonably Practicable (ALARP)” principle, i.e. taking into account the cost and schedule constraints of the project.

6.2.3 Dependability Activities

The LISA Pathfinder prime ensures that: • Subcontractors design critical software components as simply as possible, to facilitate dependability and

safety analysis and software testing; • The results of the functional analysis concerning criticality are reflected in the Requirements Baselines

and are further broken down by subcontractors to identify the critical software modules; • The subcontractor identifies appropriate measures for handling critical software components; • The Hardware Software Interaction Analysis (HSIA) as part of the subcontractor’s FMECA takes into

account the interaction of the software with its environment (system hardware, human intervention, etc.) jointly with the system level analysis;

• The list of critical software functions on system-level is verified for continuing validity at each development milestone;

• The system-level functional analysis is updated for each development milestone, if necessary; • Independent software verification and validation is performed for software components according to and

following the criteria stated in Section 6.2.4.

Furthermore, the subcontractor is requested to define in its Software Product Assurance Plan measures enforcing the reliability of critical software components, e.g.: • Use of software designs or methods that have performed successfully in a similar application; • Defensive programming techniques, such as input verification and consistency checks; • Prohibiting the use of language commands and features which are known to be unpredictable or difficult

to programme, by using a coding standard; • Full inspection of source code; • Removal of dead code1 identified by means of inspection and dynamic test coverage analysis; • Demonstrate that the means by which deactivated code could be inadvertently executed are prevented,

isolated, or eliminated. This should be achieved by a combination of analysis and testing; • Performing a structural test coverage analysis in order to demonstrate that full branch coverage on

source code level has been achieved.

1 ECSS-Q-80B, clause 6.2.3.6, uses the term “unreachable code”. For the purpose of this plan, this term comprises “dead code” (unintentionally unreachable code, e.g. because of a coding error) as well as “deactivated code” (intentionally unreachable code).






6.2.4 Independent Software Verification and Validation (ISVV)

• Independent Software Verification and Validation is performed based on the criticality classification of the software item or its functions.

The detailed scope of work, deliverable documents, work-packages, and associated milestones for ISVV will be described in the ISVV plan (Ref. TBD), which will be prepared by the LISA Pathfinder prime. In the frame of the LISA Pathfinder project all ISVV activities are performed by a contractor to be selected by ASU, to maintain independence from SciSys. The main activities related to ISVV are subdivided into the following four phases:

• Requirements Verification: This activity ensures that all the originating requirements are expressed

correctly and fully in the equipment specification and that they are consistently decomposed and allocated to subordinate software (and hardware) requirements.

• Design Verification: This activity verifies all the requirements cross-referencing between the software requirements baseline and the software design baseline.

• Code Verification: This activity includes the evaluation of the code itself, consistency and correctness between the code and the design baseline, and an assessment of how well the code follows applicable standards and practices (e.g. as defined in the Software Development Plan). A static code analysis tool should assist the assessment of code.

• Validation: This activity, in addition to the validation activities performed by the subcontractors, concentrate on testing special error cases and worst-case scenarios.

The software quality assurance activities related to and in support of ISVV are: • Review of the ISVV plans and ISVV reports submitted to Astrium by the ISVV contractor; • Define the set of functions that are considered to be critical and thus, are subject to ISVV; • Participate in reviews, audits, etc related to ISVV; • Insure that the handling of software problems and non-conformances detected by the ISVV contractor is

in accordance with the requirements defined in this plan; • Ensure the ISVV contractor does not only conduct a simple repetition of S/W tests, but defines a

different set of test cases ("added-value"). The LISA Pathfinder prime ensures that the ISVV contractor puts particular emphasis on testing special error cases, stress testing, worst-case scenarios, contingency, deadlock situations, failure modes and handling.

All software subcontractors will be required to provide support to the ISVV activity.

6.2.5 Hardware/Software Interaction Analysis

The purpose of an HSIA is to verify that the software specifications as expressed in the requirements baseline or technical specification cover the hardware failures according to the applicable FDIR requirements. HSIA will be performed by each software supplier, and will be updated and reviewed at each milestone. The HSIA will be performed in accordance with the relevant section of [SD3]; ECSS-Q-30-02A.

The review criteria for a subcontractor’s HSIA are: • The HSIA considers all software that interacts with the hardware analysed in the relevant FMECA; • The HSIA will only consider the impact of hardware failures on the software; the impact of software

failures on the hardware will not be considered as part of this analysis. • Particular attention has been paid to each failure mode of hardware that is controlled by software and/or

involved in compensatory provisions (redundancy, protection); • The results obtained by the HSIA are included in the list of critical items in case the analysis has

identified failure modes that are not or insufficiently handled.

The results of the HSIA by each supplier will be analysed and used by the prime to update the overall Software Criticality Report if additional critical functions are identified.






7 REVIEWS, AUDITS AND INSPECTIONS

7.1 General

This section defines reviews, audits, and inspections that are conducted in order to evaluate the status and quality of each software item. Reviews occur at various phases during the project and conclude the activities of a software development process. The successful completion of reviews provides assurance that requirements and design integrity is maintained, technical deficiencies are identified, and necessary changes have been identified and implemented by means of Review Item Discrepancies (RIDs). A RID identifies the reviewed item, its author, the reviewer, the conclusion and agreed corrective actions.

For each review and inspection a written procedure is prepared. This procedure defines: • the items subject to review or inspection; • the participants and the organisation in charge; • the schedule and place of the review or inspection; • the means of inspection; • who reports and maintains the results and corrective actions.

Software reviews and inspection are carried out according to the criteria defined in the following Sections, by someone other than the author of the reviewed item. Review and inspection records are subject to configuration management.

7.2 Software Audits

Software audits are conducted internally and/or at subcontractor level whenever necessary to overcome failure, consistent poor quality, or other problems. The purpose of such audits is to assess the implementation and effectiveness of the activities defined in the Software Product Assurance Plan.

Audits will be performed in accordance with the LISA Pathfinder Audit and Surveillance Plan [AD5].

7.3 Software Reviews

The specific reviews for the software are all described individually in the following sections, although it is proposed that for the DH SW, a single combined software SRR and PDR will be performed.

The scheduling of the validation activities in relation to the reviews is subject to the tailoring of ECSS E-40B as stated in [AD7], with the validation of the software against the TS starting after the CDR, and being completed by the QR. The validation of the software against the RB will be performed by the as part of the acceptance test process, and will be completed by the AR.

7.3.1 General Review Criteria

The following general criteria are used when evaluating the adequacy of a review data package: • Were all deliverables subject to review available? • Was the review agenda distributed to all parties involved prior to the review? • Were the physical facilities for the review adequate? • Does the milestone schedule as presented match the Software Development Plan? • Have software documents been produced according to the standards and procedures defined in the

relevant plans? • Do the software documents conform to the results from previous development phases? • Have agreed changes been properly implemented?

The output of a review (review report) documents evidence that identifies: • The project and software documents being reviewed; • The members of the review team;






• The specific input to the reviews, i.e. all software documents subject to review; • The objectives of the review and whether they were met; • Whether the review items meet the applicable standards, guidelines, plans, and procedures without

deviations; • A list of identified discrepancies; • Any recommendations made by the review team on how to dispose of unresolved issues and

anomalies; • Action item status (open, closed), responsible and due date (if open), or completion date (if closed).

The successful completion of a review, including the rework to address the discrepancies identified in the reviews, is used as a completion criterion for the associated development process.

The documents subject to review for each review milestone are identified in section 7 of this plan.

7.3.2 System Requirements Review (SRR)

The SRR concludes the System Requirements Engineering Process related to a specific software item. A dedicated SRR is foreseen for each software item except for the DH SW, where the SRR and PDR will be combined. Participants of this review are ESA, the LISA Pathfinder prime, and the subcontractor responsible for the software item under review.

The Requirements Baseline is the primary input to the SRR review process. The contents of the Requirements Baseline is verified considering the criteria listed below: • The criticality classification of software components is derived from a criticality analysis on system level; • Software requirements are traceable to system partitioning and system requirements; • Software requirements are internally and externally consistent; • Software requirements are verifiable. This means that it will be possible to: • Check that the requirement has been incorporated in the design; • Prove that the software will implement the requirement; • Test that the software does implement the requirement.

The contents of the subcontractor’s Product Assurance File is verified against the following criteria: • The subcontractor has prepared the Software Product Assurance Plan and a compliance matrix against

the applicable product assurance requirements for approval by the LISA Pathfinder prime; • The subcontractor has defined or referenced a software development life cycle in its Software Product

Assurance Plan. The LISA Pathfinder prime reviews the software life cycle against the contractual software engineering and product assurance requirements;

• The subcontractor has defined quality objectives that are appropriate for the criticality classification of the software and product assurance activities have been defined to evaluate these objectives;

• The subcontractor has defined a software metrication programme in its Software Product Assurance Plan;

• Critical functions identified in the Requirements Baseline are analysed; • ISVV reports are provided for critical software components.

7.3.3 Preliminary Design Review (PDR)

The PDR concludes the Software Requirements Engineering Process related to a specific software item. A dedicated PDR is foreseen for each software item except for the DH SW, where the SRR and PDR will be combined. Participants of this review are ESA, the LISA Pathfinder prime, and the subcontractor responsible for the software item under review.

The subcontractor’s Technical Specification and Product Assurance File are the primary inputs to the PDR review process.

These sets of documents are verified against the following criteria:






• The Software Product Assurance Plan defines the activities related to software product assurance for the next development phases;

• Procedures for software problem reporting are defined or referenced; • Procedures and standards for design and coding are defined or referenced. The design and coding

standards reflect the software quality and dependability requirements; • The choice of development and ground equipment are described and a justification for its selection is

provided; • The subcontractor has submitted the final Reuse Software Document for approval by the LISA

Pathfinder prime and ESA; • Software verification and testing activities are defined; • ISVV reports are provided for critical software components.

7.3.4 Critical Design Review (CDR)

The CDR concludes the Design Engineering Process related to a specific software item. A dedicated CDR is foreseen for each software item. Participants of this review are ESA, the LISA Pathfinder prime, and the subcontractor responsible for the software item under review.

The objective of the CDR is to verify that the Product Assurance File and the Design Justification File contain or reference the following aspects: • The Software Product Assurance Plan defines the activities related to software product assurance for

the next development phases; • The software is properly decomposed into individual components; • The requirements are fully and correctly implemented in the design; • Design and coding standards are followed; • The conducted tests are compliant with the test plans and procedures. The tests results are properly

recorded and evaluated; • Discrepancies identified during tests are documented and traced via the non-conformance and software

problem control system. • Evidence of full branch coverage is provided by means of structural test coverage analysis; • Software product metrics are collected and a report with identified discrepancies and remedial actions is

provided in the software product assurance reports; • ISVV reports are provided for critical software components.

7.3.5 Qualification Review (QR)

The purpose of the Qualification Review is to verify that the software product meets all of its specified requirements. A summary of test reports and the operations manual are reviewed and the consistency of all software documentation is verified. Participants of this review are ESA, the LISA Pathfinder prime, and the subcontractor responsible for the software item under review.

The validation of the software against the TS will be reviewed at this milestone.

7.3.6 Acceptance Review (AR)

The purpose of the Acceptance Review is to verify whether the acceptance of the software with respect to the intended operational environment can be stated.

The configuration management verifies the correctness and completeness of the software items delivered. Prior to acceptance the software item under review is verified against the following aspects: • The delivered software complies with the contractual requirements; • The source and object code correspond to each other and also to the status provided in the SCIDL; • All agreed changes have been implemented; • All non-conformances are either resolved or declared;






• The acceptance tests have been conducted in accordance with the approved acceptance test plan.

The acceptance of the software includes the generation of the executable code from configuration managed source code components and its installation on the target environment. Any problems or discrepancies detected during the Acceptance Review are documented in non-conformance reports and brought to the attention of the subcontractor responsible for the software.

7.3.7 Incremental Delivery Reviews (DRB)

In order to improve the control of incremental software development each increment will be subject to a Delivery Review Board (SW-DRB), which will review the test results, and will consent to ship individual staggered software deliverables.

7.4 Software Inspections

The inspections described below are expected to be performed during the process of the software development. In addition to this, ESA and Astrium are invited to inspect any stage of the development process, at 5 days notice, for SciSys or any software subcontractor.

7.4.1 Incoming Inspection of delivered Software

Incoming inspections are performed following the review objectives stated for Acceptance Reviews.

7.4.2 Product Assurance Inspections at every Milestone

The subcontractor’s Product Assurance File is verified at each review milestone against the following aspects to ensure that: • The subcontractor’s software product assurance report contains an assessment of the current quality of

the software product and development processes, based on measure properties, verifications undertaken, problems detected and problems resolved;

• The Software Product Assurance Plan defines quality criteria for the assessment of the software product and development processes;

• The correct use of methods and tools is verified and documented in the software product assurance reports;

• The software product assurance reports provide results of all assurance, verification, and validation activities.






8 SOFTWARE DOCUMENTATION

8.1 General

The terminology used for review milestones and software documentation used in the following tables is based on ECSS-Q-80B [SD10] with some minor modifications. The software documents to be produced by the software item suppliers are combined in a number of files. These files are briefly described in the following:

• Product Assurance File (PAF): The PAF contains documents related to planning and definition of requirements and activities related to software product assurance activities and processes (see [SD11], annex A.9).

• Requirements Baseline (RB): The RB expresses the customer’s requirements that are to be implemented by the subcontractor (see [SD11], annex A.2).

• Technical Specification (TS): The TS contains the subcontractor’s response to the requirements baseline, and is the primary input to the PDR review process (see [SD11], annex A.3).

• Design Definition File (DDF): The DDF is a subcontractor-generated file that documents the result of the design engineering processes (see [SD11], annex A.4).

• Design Justification File (DJF): The DJF contains the documents that describe the trade-offs, design choice justifications, verification plan, validation plan, validation testing specification, test procedures, test results, evaluations and any other documentation called for to justify the design of the subcontractor’s software product (see [SD11], annex A.5).

• Management File (MGT): The MGT is a subcontractor-generated file that describes the management features of the software project (e.g. the organisational breakdown and responsibilities, work activities breakdown, selected life cycle, delivery milestones and risks) (see [SD11], annex A.6).

• Maintenance File (MF): The MF is a maintainer-generated file that describes the planning and status of the maintenance, migration and retirement activities (see [SD11], annex A.7).

8.2 Development Documents

The table below identifies the software development documents provided by subcontractors that are subject for review at the individual review milestones described in section 6 “Reviews and Audits”.

The documents required for each review are listed separately, although it is proposed that for the DH SW, the SRR and PDR will be combined. In this case, all of the documents for both reviews will be required for the combined review, with the goal of the combined review to ensure that the software is in a specifed state according to ECSS E-40B [SD11].

Review Milestones Document File SRR PDR CDR QR AR

Software System Specification (SWSS) (from prime)

RB

Software Requirements Document (SWRD) TS

Software Reuse Document (SRUD) PAF







Interface Control Document (ICD) TS

Software Development Plan (SWDP) MGT

Software Maintenance Plan (SMP) MF

Software Design Document (SDD) DDF

Software User Manual (SUM) DDF

Software Configuration File (SCIDL) DDF

Memory and CPU Budgets DJF

The Software Design Document (SDD) includes the justification of design choices. The contents of each software version will be included in the SCIDL as part of the DDF.

8.3 Product Assurance Documents, Plans, and Procedures

The table below identifies the software product assurance documents provided by subcontractors that are subject for review at the individual review milestones.


Software Product Assurance Plan (SWPAP) PAF

SW Configuration Management Plan (SCMP) MGT

Software Design Standard (SDES) PAF

Software Coding Standard (SCOS) PAF

Software Criticality Analysis (from prime)

PAF

Software Failure-Mode Analysis/HSIA PAF

Software Product Assurance Reporting PAF regularly

Software Problem Report Status PAF

Note, The Software Criticality Analysis report will be produced by the prime for the SRR, and will be maintained and re-issued for each subsequent review.

8.4 Software Verification and Validation Documents

The table below identifies the software verification, validation and testing documents provided by subcontractors that are subject for review at the individual review milestones.







Software Verification and Validation Plan (SVVP)

DJF

Software Unit Test Plan (SUTP) DJF

Software Unit Test Report (SUTR) DJF

Software Integration Test Plan (SITP) DJF

Software Integration Test Report (SITR) DJF

Software Validation Test Report (SVTR) (against TS)

DJF

Software Acceptance Test Plan (SATP) (validation against RB, from prime)

DJF

Software Acceptance Test Report (SATR) DJF

The Software Acceptance Test Plan (SATP) is under the responsibility of the LISA Pathfinder prime. However, the software supplier contributes to its preparation.






9 SOFTWARE CONFIGURATION MANAGEMENT

The purpose of Software Configuration Management is to establish and maintain the integrity of the software products of the project throughout the project’s life cycle.

The requirements for Software Configuration Management are described in the standard ECSS–M–40A [SD5]. This section contains requirements on product assurance activities related to software and documentation configuration management, which are complementary to the requirements stated in [SD5].

For any in-house software development the configuration management implementation document explicitly addresses the configuration management of software, with respect to the requirements of [SD5]. The software configuration management system will allow any reference version to be re-generated from backups. Procedures for configuration management of software will be described in the relevant Software Development Plans.

The LISA Pathfinder prime performs configuration management as defined in the Configuration and Data Management Plan [AD6] and ensures that subcontractors submit the software configuration file for approval at acceptance testing. Furthermore, it is verified that the subcontractors’ software configuration file is available and up-to-date for each project milestone.

In case of incremental/early delivery the delivered version and its related documentation will be put under configuration control and the software change control will be started. Activities related to incremental and/or early delivery of software items are described in Section 9.3 of this plan.

9.1 Change Control

The LISA Pathfinder prime ensures that all authorised changes are implemented according to the requirements of the Configuration and Data Management Plan and that only appropriately authorised changes are made. The following aspects are verified: • The configuration baseline of a software item is identified by approved documents; • Document changes are subject to the same authorisation and circulation conditions as the initial

document; • Formal actions are defined for the approval of configuration baselines; • A change procedure is defined in the Software Configuration Management Plan and implemented in the

project.

All software documents produced or used within the LISA Pathfinder project are subject to formal configuration control. The relevant procedures for documentation control are described in the Configuration and Data Management Plan [AD6].

The use of a computer-based configuration management tool for control of documents and software is foreseen. The tool is identified in the Configuration and Data Management Plan.

9.2 Protection and Marking

The LISA Pathfinder prime ensures that: • A mechanism will be applied to protect all supplied software (e.g. source, executable, data) against

corruption; • SPA and/or Configuration Control use a checksum calculation and checking software for each

executable binary and support file; • The checksum value is documented in the software configuration file; • The identification key will be used prior to each delivery and at reception to check identification.

If the protection mechanism used is based on a specific tool, this tool is supplied to ESA along with the delivered software products. However, selection of tools that are available publicly should be preferred.






The software media deliverable to ESA are marked during the preparation of each delivery, indicating the following minimum information: • The name of the software item; • The version number; • The reference to the software configuration file.

9.3 Incremental / Early Delivery of Software

The LISA Pathfinder prime ensures that in case an incremental/early delivery of software items is specified in the contract, the scope of each delivery is defined and the subcontractor reflects that appropriately in its software documents. For each incremental delivery the following documents are requested for review: • Software User Manual; • Test specifications and reports; • SCIDL; • Software on electronic media.






10 SUBCONTRACTOR CONTROL

10.1 Subcontractor Selection

The selection of subcontractors developing software for and/or within the LISA Pathfinder project will be on the basis of customer certification, recent development experience, demonstrated continued capability, or pre-award audit as defined in ECSS-Q-20B [SD2].

10.2 Software Product Assurance Requirements for Subcontractors

The Software Product Assurance Requirements for Subcontractors (SWPARS) document will be generally applicable to all software subcontractors as appropriate to their engagement during the implementation phase. The SWPARS is based on the requirements defined in ECSS-Q-80B [SD10] and [AD1].

Each software subcontractor is requested to provide a Software Product Assurance Plan that defines how the subcontractor will implement the requirements.

10.3 Subcontractors Statement of Work

The SPA representative verifies that the statements of work (SoW) sent towards subcontractors take into account the applicable standards and project-specific requirements. It is also ensured that the subcontracted software is correctly classified for dependability and safety criticality, if this classification forms part of the subcontract.

10.4 Review of Subcontractor Plans

The software product assurance representative is involved in the assessment of the following subcontractors’ plans and standards: • Software Project Management Plan; • Software Development Plan; • Software Product Assurance Plan; • Software Configuration Management Plan; • Software design and coding standards and procedures

The LISA Pathfinder prime ensures that: • The verification activities planned by the subcontractors are adequate to ensure the products of each

development phase are compliant to the relevant requirements and standards; • The verification activities are performed according to the plans; • The planned verification activities include verification of critical software according to the requirements

of the SWPARS.

10.5 Supplier Monitoring

The monitoring of suppliers will be performed through the reviews and audits as defined in Section 0. This monitoring will be performed against the set of plans produced by the subcontractor and agreed with the prime.






11 COTS AND REUSED SOFTWARE

This section describes general criteria for the selection and assessment of COTS and reused software.

11.1 COTS Software

The choice of purchased software (COTS – Commercial Off The Shelf software) takes into account constraints associated with development, future use, and exportability constraints.

The following aspects are considered: • Assessment of the product with respect to quality requirements and objectives • Available support documentation • Acceptance and warranty conditions • Conditions of installation, preparation • Maintenance conditions, including possibility of evolutions (portability by example) • Copyright constraints

The choice of purchased software is described and submitted for acceptance by the LISA Pathfinder prime and ESA in the form of a software component list. The software component list includes specification of, at least: • Ordering criteria (versions, options, extensions, etc.) • Arrangements for maintenance and upgrades to new releases • Back-up solutions if the product becomes unavailable • Contractual arrangements for the development and maintenance phases • Receiving inspection criteria

All purchased software that is used in the operational system is identified and registered by configuration management.

All purchased software undergoes a planned incoming inspection and an incoming inspection report (including identification of detected problems) is generated accordingly. The purchased software package will be checked for completeness and for damage, and all delivered electronic media will be scanned for the presence of viruses.

The acquisition process for COTS software will be performed according to the SciSys Quality Maunal [RD5].

11.2 Reused Software

11.2.1 Definition

Re-used software includes software from previous developments intended to be used for the project development as is or with adaptation. This also includes any software supplied by the customer for use in the project development.

11.2.2 Assessment Approach

The LISA Pathfinder prime takes care that each supplier reusing a software product completely or partially, produces a Software Reuse Document (SRUD) in order to provide evidence of the software’s suitability for its re-use within the project. The document is submitted to the LISA Pathfinder prime and ESA for approval at the SRR and PDR. In case the project’s quality requirements are not fully met the subcontractor is requested to define actions to meet these requirements. The SPA representative reviews the Software Reuse Document by taking into account the aspects listed below: • Qualitative summary review of the reused components and an assessment of the possible level of reuse

taking into account the intended operational context in LISA Pathfinder;






• Evidence of fitness-for-use and operational correctness, e.g. by means of test campaign results and/or a proven operational record;

• Identification of the configuration status of each software component intended for reuse together with basic software product metrics, such as lines of code, number of instructions, reuse and comments rate;

• Description of the assumptions and the method of calculating the level of reuse including an assessment of the differences between the old and new operational environment for the reused software;

• Description of the standards, methods and tools (with issue) that have been used for software design, coding, test, management, and configuration control during the previous development life cycle;

• Description of corrective actions foreseen regarding the reused components design, coding, tests/regression;

• Statement of conformity to quality requirements by means of a compliance matrix to the PA requirements applicable to the supplier;

• Action plan defining how and when open issues identified during the reuse assessment will be closed.

11.2.3 Selection Criteria

The LISA Pathfinder prime ensures that the identification process is finalised at the PDR and the choice of reused software takes into account the following selection criteria: • Assessment of the product with respect to requirements; • Acceptance and warranty conditions (demonstration of correct operation); • Conditions of installation, preparation, training and use; • Identification and registration by configuration management; • Maintenance conditions, including the possibilities of changes; • Copyright constraints (licence, modification rights); • Criticality of the functions provided.

In addition, for each reused software component the following aspects are assessed: • Validation level or operational behaviour; • Documentation status; • Quality status, i.e. residual non-conformances, complexity analyses, waiver, etc.






ANNEX 1 LISA-PATHFINDER COMPLIANCE MATRIX FOR ECSS Q-80B

Section ECSS-Q-80B Compliance Comments

5 Software Product Assurance Process Implementation (heading only)

5.1 Introduction (Contains no requirements)

5.2 Organisation and Responsibility (heading only)

5.2.1 Organisation C SWPAP §3.1

5.2.2 Responsibility and authority C SWPAP §3.1

5.2.3 Resources C SWPAP §3.1, §7.1

5.2.4 Software product assurance manager (heading only)

5.2.4.1 C SWPAP §3.1

5.2.4.2 C SWPAP §3.1, §1, §3.2, §7.2

5.2.5 Training (heading only)

5.2.5.1 C

5.2.5.2 C 5.2.5.3 C 5.2.5.4 C 5.3 Contractual aspects C SWPAP §3.1

5.4 Software product assurance programme management (heading only)

5.4.1 Software product assurance planning and control (heading only)

5.4.1.1 C SWPAP §3.1

5.4.1.2 C SWPAP §All

5.4.1.3 C SWPAP §8.3


5.4.1.5 C Applicable at AR only

5.4.1.6 C SWPAP §App A

5.4.2 Software product assurance reporting (heading only)

5.4.2.1 C SWPAP §8.3

5.4.2.2 C SWPAP §3.2

5.4.2.3 C

5.4.2.4 C

5.4.3 Audits C SWPAP §7.2

5.4.4 Alerts C SWPAP §3.3

5.4.5 Nonconformances (heading only)

5.4.5.1 C SWPAP §4.1

5.4.5.2 C SWPAP §4.1

5.4.5.3 C SWPAP §8.3

5.4.6 Software problems (heading only)







5.4.6.1 C SWPAP §4.1

5.4.6.2 C SWPAP §4.1

5.4.6.3 C SWPAP §4.1

5.5 Risk management and critical item control (heading only)

5.5.1 Risk management C SWPAP §3.3

5.5.2 Critical item control C SWPAP §3.3

5.6 Supplier selection and control (heading only)

5.6.1 Supplier selection C SWPAP §10.1

5.6.2 Supplier requirements (heading only)

5.6.2.1 C SWPAP §10.2

5.6.2.2 C SWPAP §10.2

5.6.3 Supplier monitoring (heading only)

5.6.3.1 C SWPAP §10.4, §8.3

5.6.3.2 C SWPAP §10.4, §8

5.6.3.3 C SWPAP §10.4, §8.2

5.6.3.4 C SWPAP §8.3

5.6.4 Criticality classification C SWPAP §10.3

5.7 Procurement (heading only)

5.7.1 Requirements C SWPAP §11.1

5.7.2 Selection C SWPAP §11.1

5.7.3 Approval C SWPAP §11.1

5.7.4 Procurement details C SWPAP §11.1

5.7.5 Identification C SWPAP §11.1

5.7.6 Inspection C SWPAP §11.1

5.7.7 Exportability C SWPAP §11.1

5.8 Tools and supporting environment (heading only)

5.8.1 Development computer selection C 5.8.2 Choice description C 5.8.3 Methods and tools (heading only)

5.8.3.1 C 5.8.3.2 C 5.8.3.3 C 5.8.3.4 C 5.8.3.5 C 5.8.4 Tool selection (heading only)

5.8.4.1 C







5.8.4.2 C 5.9 Assessment and improvement process (heading only)

5.9.1 Assessment process C SWPAP §6.1.2.4

5.9.2 Assessment procedure C SWPAP §6.1.2.4

5.9.3 Assessment records C SWPAP §6.1.2.4

5.9.4 Assessment data C SWPAP §6.1.2.4

5.9.5 Quality data C SWPAP §6.1.2.4

5.9.6 Process improvement C SWPAP §6.1.2.4

5.9.7 Process or project documentation C SWPAP §6.1.2.4

6 Software process assurance (heading only)

6.1 Software development life cycle (heading only)

6.1.1 Life cycle definition C SWPAP §5

6.1.2 Life cycle definition review C SWPAP §5

6.1.3 Life cycle resources C SWPAP §5

6.1.4 Quality objectives C SWPAP §6.1

6.1.5 Phase outputs C SWPAP §8

6.1.6 Special characteristics C SWPAP §8

6.1.7 Milestones C SWPAP §7.3

6.1.8 Role of customer C SWPAP §7.3

6.1.9 Validation process schedule C SWPAP §5.5, SWPAP §7.3

Note, Validation against TS started at CDR, and completed by QR. Validation against RB performed by customer as part of acceptance testing, and completed by AR

6.2 Requirements applicable to all software engineering processes

(heading only)

6.2.1 Documentation management process (heading only)

6.2.1.1 C SWPAP §5


6.2.1.3 C SWPAP §3.1

6.2.1.4 C SWPAP §8

6.2.1.5 C SWPAP §5

6.2.1.6 C SWPAP §6.1.2.4, §6.2

6.2.1.7 C SWPAP §6.2




6.2.2 Software dependability and safety analysis (heading only)







6.2.2.1 C SWPAP §6.2.2

6.2.2.2 C SWPAP §6.2.3

6.2.2.3 C SWPAP §6.2.2

6.2.2.4 C SWPAP §6.2.3

6.2.2.5 C SWPAP §6.2.3

6.2.2.6 C SWPAP §6.2.3

6.2.2.7 C SWPAP §6.2.3

6.2.3 Handling of critical software (heading only)

6.2.3.1 C SWPAP §6.2.3

6.2.3.2 C SWPAP §6.2.3

6.2.3.3 C SWPAP §6.2.3

6.2.3.4 C SWPAP §6.2.3

6.2.3.5 C SWPAP §6.2.3.

6.2.3.6 C SWPAP §6.2.3

‘unreachable’ code expanded to include ‘dead’ code and ‘deactivated. code.

6.2.3.7 C SWPAP §6.2.3

6.2.3.8 C SWPAP §6.2.3

6.2.3.9 C SWPAP §6.2.3

6.2.4 Software configuration management (heading only)

6.2.4.1 C SWPAP §9

6.2.4.2 C

6.2.4.3 C SWPAP §9

6.2.4.4 C SWPAP §9

6.2.4.5 C

6.2.4.6 C SWPAP §9.1

6.2.4.7 C

6.2.4.8 C SWPAP §9.2

6.2.4.9 C SWPAP §9.2

6.2.4.10 C SWPAP §9.2

6.2.4.11 C SWPAP §9.2

6.2.4.12 C SWPAP §9.2

6.2.5 Process metrics (heading only)

6.2.5.1 C SWPAP §6.1.2.4

6.2.5.2 C SWPAP §6.1.2.4

6.2.5.3 C SWPAP §6

6.2.5.4 C SWPAP §6.1.2.4

6.2.5.5 C







6.2.5.6 C SWPAP §6.1.2.4

6.2.6 Verification (heading only)

6.2.6.1 C 6.2.6.2 C 6.2.6.3 C 6.2.6.4 C 6.2.6.5 C 6.2.6.6 C 6.2.6.7 C 6.2.6.8 C 6.2.6.9 C 6.2.6.10 C 6.2.6.11 C 6.2.6.12 C 6.2.6.13 C 6.2.7 Reuse of existing software (heading only)

6.2.7.1 C 6.2.7.2 C 6.2.7.3 C 6.2.7.4 C 6.2.7.5 C 6.2.7.6 C 6.2.7.7 C 6.2.7.8 C 6.2.7.9 C 6.2.7.10 C 6.2.7.11 C 6.3 Requirements applicable to individual software

engineering processes or activities (heading only)

6.3.1 Software requirements analysis (heading only)

6.3.1.1 C 6.3.1.2 C 6.3.1.3 C 6.3.1.4 C







6.3.1.5 C 6.3.2 Software architectural design and design of software

items (heading only)

6.3.2.1 C 6.3.2.2 C 6.3.2.3 C 6.3.2.4 C 6.3.2.5 C 6.3.2.6 C 6.3.2.7 C 6.3.2.8 C 6.3.2.9 C 6.3.3 Coding (heading only)

6.3.3.1 C 6.3.3.2 C 6.3.3.3 C 6.3.3.4 C 6.3.3.5 C 6.3.3.6 C 6.3.3.7 C 6.3.3.8 C 6.3.3.9 C 6.3.3.10 C ‘unreachable’ code expanded to

include ‘dead’ code and ‘deactivated. code.

6.3.4 Testing and validation (heading only)

6.3.4.1 C 6.3.4.2 C 6.3.4.3 C 6.3.4.4 C 6.3.4.5 C 6.3.4.6 C 6.3.4.7 C 6.3.4.8 C 6.3.4.9 C







6.3.4.10 C 6.3.4.11 C 6.3.4.12 C 6.3.4.13 C 6.3.4.14 C 6.3.4.15 C 6.3.4.16 C 6.3.4.17 C 6.3.4.18 C 6.3.4.19 C 6.3.4.20 C 6.3.4.21 C 6.3.4.22 C 6.3.4.23 C 6.3.4.24 C 6.3.4.25 C 6.3.4.26 C 6.3.4.27 C 6.3.4.28 C Concerns will be addressed in

Acceptance Test Plan (SATP)

6.3.4.29 C 6.3.4.30 C 6.3.4.31 C 6.3.4.32 C 6.3.5 Software delivery and acceptance (heading only)

6.3.5.1 C 6.3.5.2 C 6.3.5.3 C 6.3.5.4 C 6.3.5.5 C

6.3.5.6 C 6.3.5.7 C SWPAP §4.1

6.3.5.8 C 6.3.5.9 C







6.3.6 Operations (heading only)

6.3.6.1 C 6.3.6.2 C 6.3.6.3 C 6.3.7 Maintenance (heading only)

6.3.7.1 C 6.3.7.2 C 6.3.7.3 C 6.3.7.4 C 6.3.7.5 C 6.3.7.6 C 6.3.7.7 C 7 Software product quality assurance (heading only)

7.1 Product quality objectives and metrication (heading only)

7.1.1 Assurance activities for product quality requirements C SWPAP §7

7.1.2 Deriving of requirements C SWPAP §6

7.1.3 Quality models (heading only)

7.1.3.1 C SWPAP §6.1

7.1.3.2 C SWPAP §6.1

7.1.3.3 C SWPAP §6.1 7.1.4 Product metrics (heading only)

7.1.4.1 C SWPAP §6.1

7.1.4.2 C SWPAP §6.1

7.1.5 Measurement C

7.1.6 Measurement results C SWPAP §8.3

7.1.7 Basic metrics C SWPAP §6.1

7.1.8 Metrication process C 7.1.9 Metrics analysis C 7.1.10 Numerical accuracy C 7.1.11 Analysis of software behaviour C 7.1.12 Metrics trend C 7.1.13 Improvement actions C 7.2 Product quality requirements (heading only)

7.2.1 Technical specification (heading only)

7.2.1.1 C SWPAP §6.1







7.2.1.2 C SWPAP §6.1

7.2.1.3 C SWPAP §6.1

7.2.1.4 C SWPAP §6.1

7.2.2 Design and related documentation (heading only)

7.2.2.1 C 7.2.2.2 C 7.2.2.3 C 7.2.3 Software intended for reuse (heading only)

7.2.3.1 C

7.2.3.2 C

7.2.3.3 C

7.2.3.4 C

7.2.3.5 C

7.2.3.6 C

7.2.3.7 C

An initial assessment of the potential for this will be performed prior to the SW SRR, and specific requirements for any software components for which re-use is planned will be included. For these components, the requirements of this section will apply. If no components are identified that can realistically be re-used then this section will become not-applicable.

7.3 Supporting documentation (heading only)

7.3.1 Test and validation documentation (heading only)

7.3.1.1 C 7.3.1.2 C 7.3.1.3 C 7.3.1.4 C 7.3.1.5 C 7.3.1.6 C 7.3.2 Reports and analysis C SWPAP §3.2

7.4 Standard hardware for operational system (heading only)

7.4.1 Procurement C

7.4.2 Constraints C

7.4.3 Selection C

7.4.4 Maintenance C

7.4.5 Documentation C

7.5 Firmware (heading only)

7.5.1 Device programming C 7.5.2 Marking C 7.5.3 Calibration C Annex A

Software documentation C SWPAP §8






DOCUMENT CHANGE DETAILS

ISSUE DATE RELEVANT INFORMATION/INSTRUCTIONS

1 18.05.2004 Initial issue

1.1 13.07.2004 Updated in response to RIDS following system-level SRR

1.2 08.09.2004 Reference to SciSyS Quality Manual added to Section 11.1

ANNEX 1: update of SWPAP Section references in the comments column to correspond with this issue of the document.

Reference to ECSS E-40B tailoring in Software Development Plan for the software validation process added.

HSIA definition and criticality classification updated

Minor typos corrected

2 12.07.2005 Updated with the following SRR RID disposition:

S2-RI-ESA-SW-064/69 : Add figure 4.1-1 and associated explanations and descriptions in Sections 4.1 and 4.2

Other changes to the document for issue 1.1 are as follows:

Section 1.3: [RD6] added, S2.ASU.LI.1006, LISA-Pathfinder Acronyms List and Section 2 text deleted and ‘Refer to [RD6]’ added. Minor text updates.

DISTRIBUTION LIST

INTERNAL

EXTERNAL

LISA PF Team ESA

Configuration Management

Library