tma-413 technical review meeting.pdf

Engineering Manual Common

Engi

neer

ing

Man

ual TMA 413

TECHNICAL REVIEWS MANUAL

Version 3.0

Issued September 2011

Owner: Manager, Engineering Standards and Configuration

Approved by:

Jagath Peiris Manager Engineering Standards and Configuration

Authorised by:

Jim Modrouvanos General Manager Chief Engineers Division

Disclaimer This document was prepared for use on the RailCorp Network only. RailCorp makes no warranties, express or implied, that compliance with the contents of this document shall be sufficient to ensure safe systems or work or operation. It is the document user’s sole responsibility to ensure that the copy of the document it is viewing is the current version of the document as in use by RailCorp. RailCorp accepts no liability whatsoever in relation to the use of this document by any party, and RailCorp excludes any liability which arises in any manner by the use of this document. Copyright The information in this document is protected by Copyright and no part of this document may be reproduced, altered, stored or transmitted by any person without the prior consent of RailCorp.

UNCONTROLLED WHEN PRINTED of 68

RailCorp Engineering Manual — Common Technical Reviews Manual TMA 413

© RailCorp of 68 Issued September 2011 UNCONTROLLED WHEN PRINTED Version 3.0

Document control

Version Date Summary of change 1.0 Nil - First issue – RSA Version (AM 0022PM) 2.0 RIC Version (AM 0022PM) 3.0 September 2011 See table below

Summary of changes from previous version

Summary of change Section Application of TMA 400 format. Re-organising and numbering of sections 1 to 6 to align with TMA 400. Editorial changes to suit TMA 400. Update of cross references to other documents General editing and updating of text

General

Post Completion Review description removed Description of background documents expanded App A



Contents 1 Introduction .............................................................................................................................5 2 Purpose....................................................................................................................................5 3 Scope........................................................................................................................................5 4 Reference documents.............................................................................................................5 5 Terms and definitions.............................................................................................................6 6 Purpose of technical reviews ................................................................................................7 6.1 Purpose.....................................................................................................................................7 6.2 General considerations .............................................................................................................8 7 Structure of technical reviews...............................................................................................9 7.1 Life cycle application .................................................................................................................9 7.2 System Concept Review.........................................................................................................11 7.3 System Definition Review .......................................................................................................11 7.4 Preliminary Design Review .....................................................................................................11 7.5 Critical Design Review ............................................................................................................12 7.6 System Verification Review.....................................................................................................12 7.7 Physical Configuration Audit ...................................................................................................13 8 System and Subsystem Reviews ........................................................................................13 9 Tailoring .................................................................................................................................13 10 Concurrent Engineering Practice........................................................................................14 11 Conduct of reviews...............................................................................................................14 11.1 Responsibilities .......................................................................................................................15 11.2 Scope of individual reviews.....................................................................................................15 11.3 Technical review planning.......................................................................................................15 11.4 Review planning checklist .......................................................................................................16 12 Technical reviews cross reference .....................................................................................18 13 System Concept Review.......................................................................................................19 13.1 Purpose...................................................................................................................................19 13.2 Scheduling the review.............................................................................................................19 13.3 Responsibilities .......................................................................................................................19 13.4 Participants .............................................................................................................................19 13.5 Prerequisites ...........................................................................................................................20 13.6 Exit criteria and objectives ......................................................................................................20 13.7 Specific outcomes ...................................................................................................................20 13.8 General guidelines ..................................................................................................................20 13.9 Detailed guidelines..................................................................................................................21 14 System Definition Review ....................................................................................................29 14.1 Purpose...................................................................................................................................29 14.2 Scheduling ..............................................................................................................................29 14.3 Responsibilities .......................................................................................................................29 14.4 Participants .............................................................................................................................29 14.5 Prerequisites ...........................................................................................................................29 14.6 Exit criteria and objectives ......................................................................................................29 14.7 Specific outcomes ...................................................................................................................29



14.8 General guidelines ..................................................................................................................30 14.9 Detailed guidelines..................................................................................................................31 15 Preliminary Design Review ..................................................................................................36 15.1 Purpose...................................................................................................................................36 15.2 Scheduling ..............................................................................................................................36 15.3 Responsibilities .......................................................................................................................36 15.4 Participants .............................................................................................................................37 15.5 Prerequisites ...........................................................................................................................37 15.6 Exit criteria and objectives ......................................................................................................37 15.7 Specific outcomes ...................................................................................................................37 15.8 General guidelines ..................................................................................................................38 15.9 Detailed guidelines..................................................................................................................39 15.10 Systems Engineering ..............................................................................................................40 16 Critical Design Review..........................................................................................................49 16.1 Purpose...................................................................................................................................49 16.2 Scheduling ..............................................................................................................................49 16.3 Responsibilities .......................................................................................................................49 16.4 Participants .............................................................................................................................50 16.5 Prerequisites ...........................................................................................................................50 16.6 Exit criteria and objectives ......................................................................................................50 16.7 Specific outcomes ...................................................................................................................50 16.8 General guidelines ..................................................................................................................51 16.9 Detailed guidelines..................................................................................................................52 17 System Verification Review .................................................................................................60 17.1 Purpose...................................................................................................................................60 17.2 Scheduling ..............................................................................................................................60 17.3 Responsibilities .......................................................................................................................60 17.4 Participants .............................................................................................................................61 17.5 Prerequisites ...........................................................................................................................61 17.6 Exit criteria and objectives ......................................................................................................61 17.7 Specific outcomes ...................................................................................................................61 17.8 General guidelines ..................................................................................................................61 17.9 Detailed guidelines..................................................................................................................62 18 Physical Configuration Audit...............................................................................................64 18.1 Purpose...................................................................................................................................64 18.2 Application...............................................................................................................................64 18.3 Scheduling ..............................................................................................................................64 18.4 Responsibilities .......................................................................................................................64 18.5 Participants .............................................................................................................................65 18.6 Prerequisites ...........................................................................................................................65 18.7 Exit criteria and objectives ......................................................................................................65 18.8 Specific outcomes ...................................................................................................................65 18.9 General guidelines ..................................................................................................................65 Appendix A Background documents........................................................................................67 Appendix B List of Tables..........................................................................................................68



1 Introduction Technical reviews form an essential element of the assurance process for the acceptance of designs and assets for RailCorp. They represent an important element in the design verification and validation processes required under AS/NZS ISO 9001 and a means of managing risk within the acquisition process.

Reviews represent a means of managing risk during acquisition projects, by providing opportunities for the progressive review of products, facilities, infrastructure and support before and during the formal contract phase of a project. They serve as a means of verifying that a design or product to be supplied under a contract is consistent with the customer specification and of recognising potential deviations from the intent of the contract as early as possible in the process, so that corrective action can be taken.

The scope of technical reviews can vary considerably, depending on the type of project or product to be acquired and the contractual arrangements which apply. However, they are based on a common set of objectives and processes which should be adapted or tailored to fit the individual circumstances of each contract.

The set of reviews included in this Manual is consistent with Engineering Procedure EPD 0013 Technical Reviews. Information contained within the Manual is structured to cover high complex projects, but is designed to be readily adaptable by tailoring for projects of lesser complexity, without diminishing the integrity or essential purpose of the process.

2 Purpose The primary purpose of the Technical Reviews Manual is to communicate the means of assessing progress toward achievement of operating, engineering and support objectives, at each stage in the life of a project.

3 Scope This Manual provides guidelines to assist in planning and designing technical reviews. A set of six technical reviews is described within this Manual. These have been designed to provide a basis for assessment at critical stages in high complex projects.

This Manual should be read in conjunction with Procedure EPD 0013 Technical Reviews.

Due to the changing nature of procedures and other requirements associated with projects and their management, the guidelines and descriptions within this Manual may need to be adapted to suit various conditions.

4 Reference documents AS/NZS ISO 9001 Quality management systems – Requirements EPD 0013 Technical Reviews EPD 0014 Managing Configuration Change RPMM Program Management Guidelines from the suite of guidelines included in: RailCorp Project Management Methodology Rail Asset Enhancement Guidelines (RPMM RAE)

Background documents

Documents used in the development of this Manual are described in Appendix A



5 Terms and definitions availability the measure of the percentage of time that an item or system is available to perform its designated function. (Refer to EPD 0009 Reliability, Availability and Maintainability (RAM) for mathematical formulae for calculation of availability.)

configuration interrelated functional and physical characteristics of a product defined in ’product configuration information’ (see definition below) (from AS ISO 10007 Quality management systems - Guidelines for configuration management)

configuration baseline approved product configuration information that establishes the characteristics of a product at a point in time that serves as reference for activities throughout the life cycle of the product. Note that during the in service phase the approved configuration represents a configuration baseline. (From AS ISO 10007 )

configuration management coordinated activities to direct and control configuration (from AS ISO 10007)

customer the organisation or position requesting or initiating a design task.

design (noun) the product of the process of designing that describes the solution (conceptual, preliminary or detailed) of the system, system elements or system end items

design personnel those involved in the design process making decisions affecting the design and includes designers, checkers, verifiers and acceptors of designs, and others that provide information and recommendations on which designs are based

engineering change change to a technical requirement that results in a design variation

FMEA Failure Modes and Effects Analysis

FMECA Failure Modes, Effects and Criticality Analysis

hazard a condition that is a potential source of harm (from SMS-06-GD-0031 Hazard Identification and Safety Risk Assessment)

infrastructure see railway infrastructure

integrated support a complete process for identifying the support requirements for a new or existing rail system asset

interface common boundary or points of connection between two or more items or systems

interface specification describes the essential functional, performance and design requirements and constraints at a common boundary between two or more system elements. This includes interfaces between humans and hardware or software, as well as interfaces between humans themselves.

maintainability a characteristic of design and is essentially a measure of the ease with which the item can be maintained. The commonly used measure of maintainability is the mean time to repair (MTTR).

may indicates the existence of an option

railway infrastructure the facilities that are necessary to enable a railway to operate safely (other than rolling stock and any facility, or facility of a class, that is prescribed by the regulations not to be rail infrastructure) (from NSW Rail Safety Act 2008)

reliability probability that a specified item will perform a specified function within a defined environment, for a specified length of time.



risk see ‘safety risk’

safety risk combination of the likelihood of a hazard being realised and its consequence (from SMS-06-GD-0031 Hazard Identification and Safety Risk Assessment)

shall indicates that a statement is mandatory

should indicates a recommendation

specification a document that fully describes a design element or its interfaces in terms of requirements (functional, performance, constraints, and design characteristics) and the qualification (validation) conditions and procedures for each requirement

supportability the inherent quality of a system – including design for reliability and maintainability, technical support data, and maintenance procedures – to facilitate detection, isolation and timely repair or replacement of system anomalies.

validation the process of ensuring that the final product conforms to defined user (customer) needs and requirements (see EPD 0012 Design Validation)

verification the process carried out to ensure that the output of a design stage (or stages) meets the design stage input requirements. (see EPD 0011 Design Verification)

Abbreviations

CDR Critical Design Review CI Configuration Item CMP Configuration Management Plan COTS Commercial-off-the Shelf CSCI Computer Software Configuration Item ECR Engineering Change Request PCA Physical Configuration Audit PDR Preliminary Design Review RFT Request for Tender SCR System Concept Review SDR System Definition Review SEMP Systems Engineering Management Plan SOW Statement of Work SVR System Verification Review WBS Work Breakdown Structure

6 Purpose of technical reviews

6.1 Purpose Technical reviews provide the means of assessing progress toward achievement of operating, engineering and support objectives, progressively throughout the life of a project. They represent an important element in the design verification and validation processes required under AS/NZS ISO 9001 and a means of managing risk within the acquisition process.

Each review provides the means to assess aspects appropriate to a specific stage of the task. This in turn provides opportunities to identify and to correct deviations from the



specified requirement and to highlight potential problems and risks in meeting project objectives.

The six technical reviews described within this Manual have been designed to provide a basis for assessment at critical stages in high complex projects.

Not all reviews are appropriate for all contracts and tasks. Because each task and contract is different, the review process to be applied should be considered and adapted to meet the circumstances in each case. Selection of the reviews to be applied and the scope of each is known as “tailoring”, and is further described in later sections within this Manual.

Technical reviews are not confined to an examination of the detail design aspects of an asset. They cover the full range of considerations necessary for integrated support of the asset, once the project has been completed and the asset or installation has entered the use and maintain phase of the asset life cycle. Later units in this section cover the general scope of each major review and subsequent sections cover the application within individual reviews.

The type of information to be reviewed remains substantially common from one review to the next, which provides traceability between reviews. However the level of detail and focus of the reviews changes significantly, reflecting the evolution of the project task.

6.2 General considerations A number of general considerations are important in understanding the purpose and application of the technical review process.

6.2.1 Contractual obligations For the review process to be effective, the scope and purpose of the reviews should be defined as part of the contract or by other arrangements in place for the task. Inclusion of a review of a specific type within the contract creates obligations for both the supplier and for the customer.

The supplier or contractor clearly has the obligation to support the review process and to present or provide information necessary for an informed assessment of the approach being proposed to meet the specifications and other provisions of the contract.

The technical reviewers, as the representatives of RailCorp, have the obligation to review the information and the approach presented and to assess whether it appears to be consistent with the requirement specified in the contract.

In some cases this involves agreement to specific proposals, either within the scope of the existing contract or through a contract variation, so that work can proceed. This aspect of the process is no different to that normally experienced in managing a contractor task, where unexpected or unanticipated circumstances should be considered and resolved during the course of the task. However, the review process does not represent a mechanism for transfer of risk within the project.

6.2.2 Review outcomes The general outcome of each review is acceptance by the customer that the project should proceed to the next stage. This implies that the proposed design and supporting aspects appear to be capable of meeting the specified requirement, or that any aspect which does not conform has been identified and corrective actions determined.



Determining that the proposed design appears to be capable of meeting the requirement does not require full and detailed checking of design calculations or comprehensive assessment of other deliverable material. Assurance that the task is proceeding in a direction and at a rate likely to meet the requirement is gained by reviewing the approach, as well as evidence provided by the contractor, to determine that the development effort is consistent with requirements, specifications and standards specified in the contract.

Specific outcomes to be achieved as part of each review are covered within the relevant sections for each type of review, included later in this manual. However each review normally covers a range of aspects, introduced within this section and further described in later sections.

7 Structure of technical reviews Six types of review are defined within this Manual. Each serves a specific purpose and each is designed to provide a different level of information as part of the total process.

The review processes defined are as follows:

• System Concept Review (SCR) • System Definition Review (SDR) • Preliminary Design Review (PDR) • Critical Design Review (CDR) • System Verification Review (SVR) • Physical Configuration Audit (PCA).

EPD 0013 provides requirements on the above reviews. The purpose and scope of each of the reviews is outlined in the next section and described in detail in later sections.

Each section contains Tables providing guidelines for questions and considerations for the technical reviews. A list of the tables is in Appendix B.

7.1 Life cycle application As previously noted, the set of reviews extends over the life of a project and covers the evolution of the project task, from initial definition of the concept to the point that the project is complete and the asset has either been accepted and commissioned or full scale production has begun. The set of reviews closely parallels the systems engineering approach to development and is also closely linked to definition and acceptance of configuration baselines.

Figure 1 provides a typical representation of the timing of the review process, as well as the relationship of the reviews to configuration baselines, as a basis for explanation of the purpose of each review.


Figure 1 Application of Technical Reviews




7.2 System Concept Review The System Concept Review (SCR) is carried out during feasibility stage after the preparation of a detailed statement of requirement for the project, but prior to a Request for Tender (RFT) being released. It is normally only applicable for large-scale high complexity projects such as a new line or major reconfiguration of existing infrastructure.

The primary purpose of the SCR is to determine whether the operational, engineering and support requirements for the asset have been fully defined and that the requirements of the proposed RFT provide a sound and comprehensive statement of the product and services required within a contract.

The requirements included in the proposed RFT provide the basis for formulation of a contract against which all subsequent reviews should be carried out. The operating concept and related technical specification, as well as the specifications and standards included in the contract for detailed design of the asset, governs the approach to be taken by the Contractor and the basis for evaluation of the result. Similarly, the requirements of the contract for supporting services and documentation, such as spares, training, drawings and manuals, governs not only the data to be reviewed as part of successive reviews but the level of support available during the utilisation phase.

The SCR provides a forum for both operators and support staff to influence the specifications and contract performance requirements before the RFT is released. This is essential if the resultant contract is to properly reflect a total set of requirements capable of being met with an acceptable level of risk. It is also essential if the acquisition project is to deliver an asset which will meet the business needs of RailCorp.

7.3 System Definition Review The System Definition Review (SDR) is normally conducted a short period after contract award. The exact period depends on the nature and complexity of the contract task, but a nominal period of eight to twelve weeks is generally appropriate.

The period chosen should be sufficient to permit the contractor to fully assess the task, to complete a preliminary design synthesis and to develop a preliminary concept design for the item or installation. The period may be reduced where the RFT calls for a detailed concept proposal to be submitted as part of the tender response, but the SDR is essential even under those circumstances.

One of the primary purposes of the SDR is to verify that the detailed requirements of the contract task have been assessed and understood by the contractor, subsequent to the contract being let, and that any significant areas of uncertainty are resolved, before work commences on the detailed design. This process also provides a means of mitigating risk.

RFT responses are often prepared under severe time constraints and demand that the respondents prepare a large amount of data over a short period of time. Review following contract award may lead to the need to resolve detailed aspects, which were not fully considered in the pre-contract review or which emerge as the design synthesis task begins.

7.4 Preliminary Design Review The Preliminary Design Review (PDR) takes place as early as possible in the design phase, to enable the overall direction of the design effort to be assessed and compared to the requirements of the specification. This typically involves some form of requirements traceability process to establish the proposed means by which the requirements of the functional specification will be reflected in the final product.



The PDR also provides the first real opportunity to review and assess the proposed means of fulfilling other contractual obligations, such as the provision of integrated support elements and initial proposals for test and verification of the final product.

While support elements may be reviewed at the top level of detail at the SDR, many aspects can only be considered once a preliminary or concept design has been prepared. Aspects such as reliability and maintainability features, testability and the level and extent of operating and maintenance documentation fall within this category.

The PDR provides the basis for determining that the preliminary design accurately reflects the functional and performance requirements established in the technical specifications and in the Functional Baseline initially documented at the conclusion of the System Definition Review. Successful completion of the PDR is necessary prior to proceeding with the detailed design phase.

7.5 Critical Design Review The Critical Design Review (CDR) takes place at or close to the completion of the detailed design phase and prior to commencing construction or fabrication of hardware items or coding of software. In some cases, where separate design and construct contracts are involved, the CDR may form the major process for determining whether the requirements of the design contract have been met.

The CDR is an extension of the process begun at the PDR, but involves the review of detailed design and support proposals. It covers all aspects of the specification and support requirements of the contract and represents the most comprehensive review in the project cycle. The results of preliminary testing to demonstrate that the design offered complies with the requirements of the contract and plans for final verification and test of the design or product should be reviewed as part of the CDR process.

The CDR leads to definition of the preliminary product baseline, which establishes the configuration of the product for construction or manufacture during the next phase. The detailed interface specifications between major elements of the project or between new works and existing infrastructure are an important element of this baseline.

7.6 System Verification Review The System Verification Review (SVR) takes place when all testing and verification processes required under the contract have been completed and results are available. This may not occur until very late in the project, since the hardware or software to be tested should be fully representative of the final product, and usually includes the results of commissioning tests where these are applicable.

The SVR does not involve testing processes, but rather a review of the tests carried out and the results of these, to verify that performance and other requirements set out in the specification have been achieved. The actual testing or verification effort should have taken place beforehand and in most cases the results reviewed progressively, in advance of the formal SVR. The review process may include the adequacy of delivered design documents and support documents including publications and manuals.

The SVR also provides the basis to review any aspects of performance which have not been met and to establish the requirements for and timing of corrective action necessary for final acceptance of the project or product.



7.7 Physical Configuration Audit Physical Configuration Audit (PCA) is formally included as part of the configuration management process (see EPD 0014 Managing Configuration Change), but also forms an important part of the technical review process.

PCA covers a comparison of the “as-built” configuration of the product developed under the project, with the design documentation, to ensure that the product or software conforms to the documentation or that any differences can be reconciled.

The process serves a dual purpose within the set of technical reviews. It provides the formal means to ensure that the product to be delivered as part of the project does conform to an approved set of drawings and thus provides the basis for formal acceptance of the product. The PCA also provides the means of formally reviewing the documentation used to produce the product, to ensure that it is accurate and complete and forms the basis for establishing the Product Baseline or “as-built” configuration of the asset.

8 System and Subsystem Reviews Technical reviews may take place at either the system or the sub-system level, dependent on the structure of the project and the timing of specific elements of the task.

The initial reviews, covering the SCR and the SDR, should take place at system level if they are to be effective in ensuring that the project definition properly recognises the total set of requirements, as well as the integration and interfaces between elements of the task. The SDR may be preceded by sub-system level reviews if this is necessary due to the complexity of the task.

Sub-systems or individual items may be scheduled for the PDR, CDR and elements of the SVR and PCA as appropriate. However, there is a need to review the outcomes of these sub-system reviews at the system level in all cases, to ensure that the proposed approaches remain consistent with the overall objective. The system phase should, as a minimum, focus on the interfaces and interoperability between individual sub-systems.

PCA is particularly well suited for application at the sub-system or individual item level, since the item to be reviewed is effectively self contained and the focus of the review is on the physical aspects of the design and documentation. The system level PCA is required only to ensure that sufficient coverage has been achieved and this aspect can be included as part of the SVR.

The purpose of the technical review process remains the same, irrespective of whether it is conducted at the system or the sub-system level. The detail of the process varies, because of the different range and complexity of the items to be reviewed, but in most cases this is a tailored subset of the total process.

9 Tailoring Tailoring was earlier described as the process of adapting the review process to the circumstances prevailing for a specific task or contract.

The extent of tailoring is generally a function of two aspects. The nature and complexity of the task is the primary consideration prior to the contract award and in particular during the process of developing a statement of work for inclusion in an RFT.

While a structured and thorough review process provides evident advantages as a means of ensuring that the objectives of a project are met, the conduct of reviews is inherently time consuming and expensive. Proper preparation by the contractor requires a



considerable amount of time and effort, which should be reflected in the contract price. Review of delivered material by the customer as well as attendance at formal review meetings also requires a great deal of effort and incurs considerable cost.

This means that the scope and extent of the reviews should be carefully considered during the pre-contract phase, so that the intended approach is both appropriate to the contract task and so that it is likely to deliver the expected benefit. Guidelines for tailoring are covered in more detail within the sections on individual reviews, but generally they require an assessment of each aspect of the review to determine its applicability for a specific task.

The scope of the review process included in the contract statement of work is the primary determinant for planning during the project performance phase. Once the contract has been let there is limited scope for defining or altering the scale or scope of planned reviews, even though some flexibility remains for detailed planning and tailoring of the process to fit the developing approach and progress on the contract. This in no way diminishes the responsibilities of either the contractor or the customer for proper execution of the designated reviews, which is ultimately in the best interest of both parties.

10 Concurrent Engineering Practice The standard approach to the review process is oriented toward the more traditional approach to projects, where the design is developed, verified and in some cases tested before the manufacturing or construction phase begins.

Technical reviews are equally applicable and in many respects are even more critical for projects which employ the concurrent engineering approach to development. In such projects the design and construction phases are proceeding in parallel and there is a substantial challenge in ensuring that diverse elements of the task remain in synchronisation.

The application of the technical review process within this environment is a special case of tailoring. In this circumstance, it is crucial that reviews take special account of the integration and interface aspects of the design and support requirements, to ensure that the final product meets the project objectives.

11 Conduct of reviews Technical reviews may be conducted in a number of ways. For relatively minor tasks, or some elements of a sub-system review process, it may be appropriate to complete the main part of the review through small group discussion and examination of the available data.

For more significant projects the reviews normally involve a formal meeting supported by presentations by the contractor. The review should be attended by representatives of all major stakeholders in the project, including specialists from the relevant support functions.

This approach provides the very important benefit of reviewing the development effort in an environment where operators, design personnel and staff who are responsible for maintenance and continuing support are all represented. This provides the opportunity for stakeholders to influence the outcomes within the terms of the specification and contract and to ensure that specialist requirements are being met.

The interaction generated through a multi-disciplinary approach to the review process is an important means of ensuring that the technical and business objectives of the project are met. However, the technical review process should not be regarded as an opportunity



to redefine project requirements. This should have been accomplished at the pre-contract stage and in particular during the system concept review.

The reviews do provide the opportunity to identify any aspect of the project which is not consistent with wider business objectives and to introduce changes and variations to the contract where appropriate, to correct deficiencies before the product or installation enters the utilisation phase.

11.1 Responsibilities Responsibility for management of technical reviews is generally vested in the RailCorp Project or Program Manager appointed for the project. This responsibility includes scheduling of the individual reviews, arranging for the appropriate specialist support and the conduct of the review.

Specialists appointed to participate in specific reviews are responsible for assessing those aspects of the development set out in the sections covering each type of review.

All participants in the review process have a responsibility for taking a broad view of the development effort. This requires that the aspects for which they are responsible be considered in the context of total system performance and against the business objectives for the project.

11.2 Scope of individual reviews Table 1 provides an outline of the major topics which may be appropriate for inclusion in the technical review process, together with a cross reference to the applicability of each to the individual reviews. Each section then provides more detail on the scope of each review and the type of information which may be included in the review.

11.3 Technical review planning Planning of the technical review process is integral to the schedule planning process for the overall project. Reviews represent major technical milestones for transition from one phase of the task to the next, have a direct linkage to configuration baselines and may represent defined hold points under some contracts.

Planning for the actual review process also crosses several boundaries within the project.

The reviews form a major element in the systems engineering process and the intent and tailoring intended for each review may be included in the Systems Engineering Management Plan (SEMP), if a SEMP is required as part of the contract.

The linkage with configuration baselines also means that the technical review planning process should be integrated with the Configuration Management Plan (CMP), if specified. Typically the CMP covers the creation of baselines and policies for management and approval of Engineering Change Requests (ECRs), as different baselines are established.

The System Verification Review is closely related to the management of test activity within the project, especially the creation of a qualification database, and may be included as an integral part of the Master Test Plan if a separate test plan is included in the project requirement.

Both the CMP and the Master Test Plan are typically subsidiary to the SEMP and in this respect the SEMP is the primary document for technical review planning details.



Where none of the preceding plans have been specified as part of the contract requirement it may be necessary to develop a separate Technical Review Plan, to define the approach within a specific project. Where such a plan is prepared it should cover details of the reviews to be conducted as part of the contract statement of work, proposed schedule, tailoring, required outcomes and responsibilities in accordance with the guidelines contained in this Manual.

11.4 Review planning checklist Each section provides information relating to the detailed planning and conduct of individual reviews. Table 1 provides a more general checklist of aspects which should be considered in formulating a plan for technical reviews as part of the project.

Table 1 Technical Reviews Planning Checklist

PLANNING ACTIVITY GENERAL CONSIDERATIONS 1. Establish contract requirement. Determine which reviews are called up as

part of the contract Statement of Work (SOW).

2. Determine responsibilities. Contractor responsibilities for presentation of information should be specified as part of the SOW, as should the level of support required of subcontractors. Internal responsibilities should also be defined as part of this process, to ensure that nominated officers may become familiar with the project and their responsibilities as reviewers.

3. Establish standards to be followed.

If the SOW does not specify a standard covering the scope and conduct of each review, it will be necessary to agree to a standard. EPD 0013 should normally be specified.

4. Establish the level of tailoring to be applied.

Tailoring will be needed to adapt the requirements of this manual to the circumstances and type of project involved. Tailoring should normally define the range and extent to which each topic will be covered, with reference to the appropriate section in this manual.

5. Establish subsystem review structure.

Determine which subsystems or configuration items require separate reviews and ensure that these can be integrated into the overall project plan. Establish a schedule for the reviews. Establish phasing of individual reviews, if necessary.

6. Verify data delivery requirements.

Ensure that the contract requirement for delivery of data is consistent with the review structure and that sufficient time is available for evaluation in advance of the formal review.

(continued)



Table 1 Technical Reviews Planning Checklist (continued)

PLANNING ACTIVITY GENERAL CONSIDERATIONS 7. Confirm and establish review

outcomes. Ensure that the outcomes of each review are clearly stated and understood. This should cover: • Configuration baselines to be established. • Approval to proceed with the next phase. • Corrective action processes and nominal

timescales. Specific timescales can only be determined when the actions have been defined and agreed.

8. Establish tentative schedule. The relationship of each review to other contract milestones should normally be defined in the SOW. This should be confirmed, or established if necessary, before the plan is finalised. Ensure that the planned reviews are properly integrated with the master program schedule.

9. Confirm general administrative arrangements and responsibilities.

This needs to include locations, representation and responsibilities for arranging the review, recording and distribution of actions and minutes and so on.

10. Document the plan. The contract may require that the prime contractor is responsible for detailed planning of the review process and that this plan be incorporated in another plan, such as the SEMP. An internal planning document will still be useful as a means of ensuring that all participants and stakeholders are aware of the planned process and their responsibilities.



12 Technical reviews cross reference Notes for application

1. This table provides a general listing of the topics which may be considered for inclusion in one or more of the technical review processes.

2. The topics are grouped according to major discipline or area of performance. This is for convenience in using the manual and does not imply that the topics should be considered independently in the review process.

3. The cross reference column in the table provides a general indication of the primary applicability of the topic to a specific review. Refer to the specific section for additional detail.

Table 2 Technical reviews cross reference

PRIMARY APPLICABILITY MAJOR TOPIC SUBJECT SCR SDR PDR CDR SVR PCA Functional Requirement • • • • • Operating Concept • • • Maintenance and Support Concept • • •

Service Life • • • • Environmental Specification • • • • •

Operational Requirements

Operational Interfaces • • • Program Schedule • • • • Contract Milestones • Designated Reviews • Quality Assurance • • • Configuration Management • • • • • •

Program Requirements

Documentation • • • • • Design Specifications • • • • • Design Standards • • • • Requirements Traceability • • • • • Design Synthesis • • • Standardisation • • • • Interface Definition and Management • • • •

Drawings and Documentation

• • •

Systems Engineering

Software • • • • • Maintenance / Inspection • • • Spares Support • • • Training • • • Support Equipment • • • Facilities • • • Operating and Support Documentation • • •

Integrated Support Concept

Packaging • • • Planning • • • • • Integration • • • Qualification • • • First Article • • •

Verification and Test

Commissioning • • •



13 System Concept Review

13.1 Purpose The purpose of the System Concept Review (SCR) is to determine whether the operational, engineering and support requirements for the proposed acquisition project have been fully defined and that the resultant specifications and Request for Tender (RFT) provide a sound basis for ensuring that customer requirements can be met.

The specifications and the requirements of the contract arising from an RFT form the primary basis for all technical review activity carried out as part of the project. It follows that the success of the review process depends directly on the effort invested in bringing these documents to a high standard prior to letting a contract.

13.2 Scheduling the review Review of the requirements to be included in the SCR may take place progressively as part of the lead-up effort to preparing a final version of the RFT. However it is important that a final, formal, SCR should take place, to provide the opportunity for multi-disciplinary review of the final requirements and the structure of the total contract task. This review often reveals inconsistencies in the requirement which may not be evident from independent “small group” reviews.

Scheduling of the formal SCR should provide sufficient time for corrective action to be taken and for changes to be made to the documentation prior to final release. There is little purpose in conducting the review unless this outcome can be assured.

Ideally the review should be scheduled two to three weeks ahead of the planned release date for the RFT and at the stage where the specification and technical requirements of the Statement of Work exist in at least advanced draft form. Any shorter period imposes major constraints on the process and planning for the preparation and release of the RFT should include a firm milestone for conduct of the SCR.

13.3 Responsibilities The designated Program or Project Manager is normally responsible for planning and scheduling the SCR.

This includes preparation of an agenda which reflects the proposed tailoring of the SCR process for the project and distribution of the draft specification and RFT documentation, in sufficient time for participants to thoroughly review the complete requirement.

Designated participants in the SCR are responsible for reviewing the documentation in advance of the meeting, to ensure that they are fully prepared for discussion within their area of specialty and to assess the overall program proposed for the project.

13.4 Participants Participants in the SCR process should include, but also be limited to, those appointments or individuals having a direct stake in the outcome.

Specialist contract staff may not have a direct role, since the review is focused on technical aspects rather than contract terms and conditions. However, there is frequently a close relationship between the two and contract staff should be available to participate if required. Where possible contract representatives should be chosen on the basis of



their probable future involvement in the contract task, to provide an understanding of the task for the contract negotiation phase.

13.5 Prerequisites Prerequisites for the SCR have previously been covered under the responsibilities heading.

13.6 Exit criteria and objectives As already established, the purpose of the SCR is to establish that the project documentation is accurate and complete prior to the release of the RFT.

The only exit criterion from the SCR is that the group formed to review the documentation are satisfied that the review questions have been answered satisfactorily and that a proposed contract based on the RFT and specifications would be likely to result in a product meeting the business needs of the customer.

13.7 Specific outcomes Required outcomes of the SCR are:

• Corrective actions arising from the review process • Concurrence from the review group that the technical aspects of the specification

and the draft RFT are suitable for release, following the incorporation of any changes identified during the review

13.8 General guidelines The primary focus of the SCR is to ensure that the requirements specified for the proposed acquisition are realistic, complete and cohesive.

In most cases this does not require minute examination of specification requirements but rather assurance that the total set of requirements has been specified in a way which meets operating and performance needs by the most efficient means.

Support provisions are an essential element in meeting the operating and business need throughout the asset life cycle. This requirement influences aspects of the technical specification for design and development as well as support provisions, such as accurate maintenance and spares data and training support for new equipment to be included in the SOW.

Most importantly, the SCR provides a final opportunity to review and assess the statement of requirement which represents the performance requirement for a binding contract. Decisions taken at this point have the potential to lead to a contract which provides a sound basis for contractor performance and customer satisfaction, or one which leads to an inferior product and/or to a large number of variations and the attendant increase in total project cost.

Table 3 to Table 7 provide a set of detailed guidelines for use in structuring the SCR process. The questions are generic, in that they represent the most significant issues to be addressed as part of the process for any item, and should be adapted to the circumstances which prevail for each project.



13.9 Detailed guidelines

Table 3 Guidelines for System Concept Review – Operational Requirements

SUBJECT OR REQUIREMENT REVIEW TOPICS QUESTIONS AND CONSIDERATIONS

Status of Functional Specification

• Has a functional specification been prepared and what is its current status?

• How were the specified functions determined?

• Do they represent functional statements rather than design solutions? This question should be reviewed and answered for each major group of functions.

• Have alternatives been considered? If so, why were they rejected?

Performance Requirements

• What are the major performance requirements for the system or installation? These should be listed and reviewed.

• Do they relate to the specified functions?

• Are they achievable within the current technologies available within the industry?

• If the technology required represents “state of the art”, what are the risks? Have they been reviewed as part of a risk study and what were the results?

Functional Requirement

Growth Potential • Is growth potential required as part of the specification or design brief? How has this been provided for?

• Have growth requirements been specified on the basis of average or peak utilisation?

NOTE: This aspect is particularly significant for software development.

Operating Concept

Usage Factors • Have usage factors been defined? Do they cover average, peak and maximum overload conditions? How do they relate to growth potential requirements? Usage factors should be listed and reviewed at the meeting.

• Are defined usage factors realistic in terms of current and forecast future demand?

• Has an Availability requirement been specified? Is it applicable? How is availability to be measured?

(continued)



Table 3 Guidelines for System Concept Review – Operational Requirements (continued)


Control Systems • Identify control systems to be used in conjunction with the project under review e.g. signalling computer, supervisory system.

• Are they capable of supporting the type of technology expected for the project under consideration?

• Are they properly specified, in terms of operating characteristics and interfaces?

• (See later questions on these aspects)

Operating Concept (continued)

Performance Indicators • Have performance indicators been specified for the equipment during the utilisation phase? (See earlier question re availability)

• If so, are they consistent with the requirements of the specification?

Support Responsibilities • Has the maintenance and support concept been defined?

• Does the requirement provide sufficient flexibility to change the support concept at some later stage? Asset ownership and likely long term availability of support should be considered here.

Levels of Maintenance • Have levels of maintenance been defined? Are the divisions of responsibility e.g. between in-house and contractor support clearly defined?

• Detailed considerations are also covered under the integrated support concept.

Maintenance and Support Concept

Support Contract Requirements Defined

• Does the requirement for any support contract included as part of the RFT cover other aspects such as performance guarantees, response times, quality system requirements, staff qualifications, configuration management requirements?

• Are the above requirements defined in a way which permits the potential contractor to submit an unambiguous response?

(continued)



QUESTIONS AND CONSIDERATIONS

Table 3 Guidelines for System Concept Review – Operational Requirements (continued)

SUBJECT OR REQUIREMENT REVIEW TOPICS

How Specified • Has the required service life been specified?

• Is it based on a maintenance free period or is some form of upgrade/refit process to be assumed? If so, after what interval?

• What form of verification is required?

Service Life

Basis for Design • Has sufficient information been provided about the service life requirement to provide a firm basis for design?

How Specified • Have environmental requirements been specified?

• Have normal operating parameters, such as temperature, humidity, soil condition as well as unusual factors such as high wind loading, corrosive environment been specified?

Standards Used • What standards have been called up? • Are these appropriate for this

application?

Environmental Specification

Verification Requirements (General)

• Have verification methods i.e. environmental testing requirements, been specified? Are they required?

NOTE: Verification methods are also considered under test and verification heading.

Related Equipment and Services

• Have operational interfaces been specified for existing related equipment and services?

How Specified • How well are these defined and how are they specified?

Operational Interfaces

Verification Requirements

• Is verification of interface specifications required as part of the contract task? Should they be?

NOTE: The accuracy and completeness of operating interfaces for existing equipment i.e. the equipment with which the new acquisition is required to operate is a customer responsibility.



Table 4 Guidelines for System Concept Review – Program Requirements


Work Breakdown Structure

• Has a Work Breakdown Structure (WBS) been defined as part of the contract SOW?

Program Schedule

Scheduled Defined • Is it accurate and complete? Does it reflect the major items and services included in the specification and the RFT?

• Is the schedule traceable to the WBS?

• Are major milestones defined as part of the schedule?

Specific Reviews Nominated

• What technical reviews are included in the RFT?

• Are they appropriate for this project? Defined Scope • Has the scope of the reviews been

defined in the RFT? What standard has been invoked, if any? Has the outcome of each review been specified in terms of baselines to be established, milestone achievement?

• To what extent has tailoring been applied for this project? Is the approach to tailoring explained in the SOW?

Designated Reviews

Responsibilities Defined • Have the responsibilities of the contractor and the customer been defined in the RFT?

Specified Standard • What standard has been specified for compliance by the contractor? Has the same standard been specified for sub-contractors?

Alignment with Reviews and Audits

• Has any provision been made for the contractor to attain certification? If so, is the period consistent with the review and audit schedule?

• Has the requirement for audits and on-site inspection and verification been defined?

Quality Assurance

Test Articles • Have any special requirements been specified for test articles and first article inspection? Is this required for this project?

(continued)



Table 4 Guidelines for System Concept Review – Program Requirements (continued)


Specified Requirements • What standard and/or requirements have been placed on the contractor to maintain a configuration management system?

• What change control methods have been specified?

Configuration Management

Baseline Definition • Has the requirement for baselines been clearly defined?

• What baselines are required? • Are these appropriate for the task?

Documentation Specified Requirement (General)

• Has the requirement for design documentation and drawings been specified?

• If yes, what standard? Is this compatible with rail system requirements

• Is documentation required in hard copy or electronic format, or both?

• Has any form of maintenance or update service been requested? Is it necessary for this project?



Table 5 Guidelines for System Concept Review – Systems Engineering



• Covered by Functional requirement at SCR.

Design Specification

Development of Additional Specifications

• Has the requirement for additional specifications been included in the SOW?

• If so, what standards have been called up?

• Is approval process defined? Specified Requirements • Have the appropriate design

standards been specified for the equipment? The main standards should be listed and reviewed at the meeting.

Design Standards

Applicability • Does the specification include requirements for Reliability, Maintainability, Accessibility, Inspection, Compatibility and similar?

• Has the applicability of each standard, or section thereof, been defined?

Contract Requirement • Does the RFT include a requirement for requirements traceability as part of the design synthesis?

Requirements Traceability

Documentation Method • Has a specific format been defined? If so, will likely respondents be able to comply? If not, is the standard really necessary?

Design Synthesis Status of Development Task

• Not applicable to this review, except where developed equipment is proposed for the project.

Standardisation Requirement Justification

• What level of standardisation has been specified?

• Is this requirement justified on economic grounds? Does it prevent the potential contractor from using better, more advanced solutions?

Standardisation

Application • What approval process is required if the contractor suggests a departure from specified standardisation requirements?

Interface Definition and Management

Interface Specifications Responsibility for Control

• Covered by similar question under operational requirements heading for this review.


Standards Delivered Documents

• Covered by similar heading under program requirements for this review.

Requirements • How have requirements been developed and how are they specified?

Software

Development Environment

• Has a specific development environment been nominated? Why?



Table 6 Guidelines for System Concept Review – Integrated Support Concept


Maintenance / Inspection Requirements

Responsibility Analysis • Are maintenance and inspection requirements to be developed as part of the contract task? If not, how will these be determined?

• Has a standard or methodology been specified for FMEA/FMECA? If so, which standard?

• In what format is analysis data to be delivered?

Spares Support Requirements Analysis • Does the statement of work require spares support analysis?

• Has a methodology been specified?• Are spares required? If not, how will

the requirement be met? Training Requirements Analysis • Is training required as part of the

contract task? • Are numbers of trainees and types

of courses identified? • Is training material needed to

conduct follow up training? Is it specified in the SOW?

• Is any form of training analysis needed and specified?

Support Equipment

Requirements Analysis • Have any support equipment requirements been identified?

• Is it to be supplied as part of the contract?

• Have preferred types been nominated, where this is appropriate?

Facilities Requirements Analysis • As above, for facilities. • Are facilities to be provided as part

of the contract? Operating and Support Documentation

Specifications Development

• Is new documentation to be developed?

• If so what standard has been specified? Is this appropriate? Will existing commercial standard documentation be satisfactory?

• Has a validation process or standard been nominated?

Packaging Requirement • Is any special packaging needed? If so, has the standard been specified?



Table 7 Guidelines for System Concept Review – Verification and Test

SUBJECT OR REQUIREMENT REVIEW TOPICS QUESTIONS AND

CONSIDERATIONS Planning Requirement Schedule • At SCR stage the main requirement

is to ensure that the forms of verification and testing required under the contract SOW are clearly identified and that the processes for planning, conducting, approving and documenting test plans and results are clearly identified.

• The further requirement is to ensure that any schedule nominated for test activity is consistent with the overall development program.

• Specific test requirements, standards and methods (if appropriate) may be included and reviewed as part of the specification or may be separately defined.

• Each type of testing identified in this table should be reviewed separately, to ensure that the specified requirement is both necessary and appropriate for the project task.

Integration Requirement As above Qualification Requirement As above First Article Requirement As above Commissioning Requirement As above



14 System Definition Review

14.1 Purpose The purpose of the System Definition Review (SDR) is to verify that the requirements of the contract task have been assessed and understood by the contractor, subsequent to contract award. The review is carried out to ensure that possible uncertainties and misunderstandings are resolved as early as possible in the contract and before work commences on the detail design process.

14.2 Scheduling The SDR should be scheduled as soon as practicable after the contract has started, but after a sufficient time to permit the contractor to have completed a preliminary design synthesis. This period varies between projects and the actual time should be set by agreement with the contractor.

14.3 Responsibilities The Project Manager is responsible for arranging the SDR and for establishing the agenda for the meeting. The SDR meeting should be chaired by the Project Manager.

Agenda items raised by the contractor should be advised to participants in advance of the meeting, to allow time for review and internal resolution of any requirements.

14.4 Participants Participants in the SDR are nominated by the Project Manager or Principal Engineer as appropriate and should be selected on the basis of their responsibility for some aspect of the project task.

14.5 Prerequisites The prerequisite for the review is the delivery of design and documentation as required by the contract (refer EPD 0013 Appendix A).

14.6 Exit criteria and objectives Exit criteria should be established by the Project Manager in advance of the SDR, to reflect specific circumstances of the contract.

14.7 Specific outcomes Specific outcomes of the SDR are a direct function of the issues raised at the review. Under normal circumstances the review identifies a number of aspects requiring clarification which should be accepted as action items, either by the Project Manager or the contractor.

A specific target date should be agreed for completion of each action item, to ensure that the project is not delayed and to avoid any potential for later claims for excusable delay, due to lack of clarity in the requirement.

The SDR should establish the preliminary Functional Baseline for the project, for confirmation at the PDR.



14.8 General guidelines The SDR process is intended primarily as an opportunity to review, discuss and resolve any aspect which may lead to unnecessary conflict and delays in the development process later in the project.

It is important that the review should take place in a spirit of cooperation and mutual resolve, to establish the best possible basis for a successful outcome to the project. Many aspects which have the potential to develop into major problems can be resolved at this stage with minimal disruption to the project. This is an important element in the management of risk within the project.

The guidelines presented at Section 14.9 follow the same pattern as for the SCR, with the focus of the review being to understand the contractor’s interpretation in each case and to resolve any resultant differences.

The guidelines are structured for review of the program requirements with the Prime contractor and each review item is intended to apply to the total project task. For some projects, particularly those involving major infrastructure works, the task may need to be considered as a series of sub-programs because of the diversity of requirements and industry approaches.

This can be accomplished through a single review involving the Prime and major subcontractors, if this can be arranged. This approach has the advantage of exposing all major participants to the initial process and of highlighting any inconsistencies in the understanding of the task or areas requiring coordination, particularly interfaces.

An alternative approach is to conduct the review with the Prime contractor alone, and to rely on that input for major subcontracts.

Irrespective of the approach deemed best for a specific set of circumstances, it is essential that the review should cover the total scope of the project task. This is particularly important for aspects of requirements traceability, for the identification of configuration items and lower level specifications, and for interface management aspects.




Table 8 Guidelines for System Definition Review – Operational Requirement



Status of Functional Specification Performance Requirements Growth Potential

• Establish mutual agreement as to the issue status of the functional specification.

• Review system performance requirements, including reserve capacity for specified growth requirements.

• Review contractor assessment of requirements based on analysis of the specification.

• Review any preliminary functional allocation by the contractor, to ensure that allocated functions are consistent with intended use.

Operating Concept

Usage Factors Control Systems Performance Indicators

• Review contractor’s assessment of specified usage factors, on same basis as used for the SCR.

• Review contractor’s initial assessment of interoperability with existing equipment.

• Review contractors proposed approach to Availability computation, if specified.


Support Responsibilities Levels of Maintenance Support Contract Requirements Defined

• Establish contractors understanding of maintenance concept. Detailed discussion of proposed approach is premature at SDR.

Service Life How Specified Basis for Design

• Establish contractor’s assessment of service life requirement. Seek information on proposed approach to meeting requirement in design phase.


How Specified Standards Used Verification Requirements (General)

• Seek advice from contractor on preliminary analysis of environmental design requirements. Establish whether any difficulties have been encountered in obtaining, understanding or applying specified standards.


Related Equipment and Services How Specified Verification Requirements

• Review contractors understanding of interface specifications approach to management of interfaces. Review approach to verification, if required as part of the contract. Provide any additional information available on interface definition. This may need to be included in contract data list if it is to be used as baseline information for design.



Table 9 Guidelines for System Definition Review – Program Requirements


Program Schedule

Work Breakdown Structure Scheduled Defined

• Review further development of WBS, as indication of proposed product structure. Schedule to be reviewed at Project Manager’s option, may be reserved for contract progress meeting.

• Discussion of contractors progress in letting any subcontracts is significant for the SDR process in that the approach by subcontractors in some areas have a major bearing on the outcome of the project

Designed Reviews

Specific Reviews Nominated Defined Scope Responsibilities Defined

• Review scope of review process as applicable. May be more appropriately discussed in contract progress meeting but review of the information needed at each stage is significant issue.

Contract Milestones


• As above.

Quality Assurance

Specified Standard Test Articles Alignment with Reviews and Audits

• As above.


Specified Requirements Baseline Definition

• Review initial proposals for designation of configuration items.

• Review initial proposals for specification hierarchy, if appropriate for the program.

• Agree basis for establishing the Functional Baseline.

NOTE: The above aspects are closely associated with discussion of the status of the functional specification and also preliminary design synthesis. These are covered under systems engineering and may be combined in a single topic.


• Requirement for additional specifications and progress toward defining specification hierarchy.

• Requirements for inclusion in each is a continuation of discussion on functional specification.

• Review proposed approach, if appropriate.



Table 10 Guidelines for System Definition Review – Systems Engineering



Status of Functional Specification Development of Additional Specifications

• Topic covered by previous review of functional and derivative specifications. Objective is to review the contractors approach to requirements allocation and to resolve any areas of the specification which are unclear.

NOTE: The relationship between this topic and the specification development aspects are covered under configuration management.

Design Standards

Specified Requirements Applicability

• Review understanding and proposed application of specified design standards.

• Review any further tailoring proposed by the contractor.

• Resolve any questions regarding issue status of designated specifications. Note that it is the contractor’s obligation to review the designated specifications and to raise any concerns.


Contract Requirement Documentation Method

• Contractor to present proposed approach to requirements traceability process (if required by the contract) including method of documentation.


• At SDR this is likely to be limited to review of requirements allocation covered in earlier topics, but the status of any partly or previously developed designs may be reviewed if applicable.

Standardisation Standardisation Requirement

• Contractor to present assessment of standardisation requirements included in the specification and contract, including preliminary information on areas where alternatives may be proposed.



• Review contractors understanding and assessment of defined interface specifications and documents.

• Review proposed means of interfacecontrol, particularly where major subcontractors are involved in the project.


Standards Delivered Documents

• Not applicable to this review unless any formal documents are required as part of contract data requirements list.

(continued)



Table 10 Guidelines for System Definition Review – Systems Engineering

(continued)


Software Requirements Development Environment

• Review contractors approach to software development effort, if applicable to the project.

• Establish status of requirements allocation and specification development.

• Confirm details of development tools to be used, if appropriate.

Table 11 Guidelines for System Definition Review – Integrated Support Concept


Maintenance / Inspection Requirements

Responsibility Analysis • Review contractors proposed approach to FMEA/FMECA if specified in the contract.

• Establish linkages between FMECA and any reliability estimation or modelling required by the contract.

• Review general approach to incorporation of maintainability aspects in the design phase.

Spares Support Requirements Analysis • Review proposed approach to analysis of spares support requirements, including the use of any computer based models, if applicable.

Training Requirements Analysis • Review contractors proposed approach to training analysis, if required by the contract.

Support Equipment

Requirements Analysis • Identify proposals for any major support equipment development. Dependent on scope of task this may need to be considered in the same way as other configuration items within the contract.

Facilities Requirements Analysis • As for support equipment. Major facilities may be treated as a separate major project dependent on contract arrangements.

Operating and Support Documentation


• Review of operating and support documentation is premature at the SDR stage, except to ensure that appropriate specifications and standards are recognised.

Packaging Requirement Proposed Methods

• Only applicable if contract includes the development of any specialised packaging or transportation equipment, in which case the item may need to be considered as a separate configuration item.



Table 12 Guidelines for System Definition Review – Verification and Test


Planning Requirement Schedule • Review of the verification and test effort at the SDR is primarily intended to establish the planned approach to management of the test program.

• This should include linking of test planning with the requirements traceability process, planned use of facilities if the testing of individual items is involved, and proposed documentation methods.

• Detailed discussion of any of the specific forms of test is unlikely to be necessary, unless this testing is associated with site verification or background environmental conditions and is needed as a basis for further design synthesis. Background electronic or audible noise are examples.

Integration Requirement Schedule As above. Qualification Requirement Schedule As above. First Article Requirement Schedule As above. Commissioning Requirement Schedule As above.



15 Preliminary Design Review

15.1 Purpose The purpose of the Preliminary Design Review (PDR) is to confirm that the proposed approach to the detailed design satisfies the functional requirements for the project, that risks associated with the design development have been identified and are being actively addressed and that logistics support elements are proceeding in line with the design development task.

The PDR provides the first formal opportunity to review the concept design for the complete project and to assess whether the directions being taken appear to be in accordance with the specified requirement. Thus the PDR is more oriented toward detail issues than the previous reviews in the set.

Concepts and data established during the SDR form an input to the PDR process. This is particularly relevant in the case of product definition aspects, in the form of lower level specifications and requirements traceability. At the same time the concept established as a result of the PDR forms an input to the CDR later in the design and development process.

15.2 Scheduling Timing of the PDR is critical to the success of a project. The review should be scheduled to take place after sufficient time has elapsed to thoroughly establish the requirements allocation process and the contractor has had time to translate this into a well-founded design concept

Allowing too short a period can result in a need to establish the design concept without adequate consideration of alternatives and a final result which is not the best available. Undue delay means that the customer does not gain visibility of the proposed concept until relatively late in the process, with the possibility of wasted effort and delays if the concept does not meet the requirement. Delays in the PDR also reduces the equally critical period between PDR and presentation of the detail design at the CDR.

The schedule established for the PDR should attempt to strike the best balance between the two extremes. It can be reduced to the minimum if the project technology is relatively well established and the development requires adaptation rather than innovation. A longer period may be allowed where the technology is new or the project involves complex development effort or interfaces.

There is no single “right” solution. Each project should be judged on its merits and the PDR scheduled as early as possible, given the need to be able to conduct a meaningful review if the process is to serve the intended purpose.

15.3 Responsibilities The Project Manager is responsible for scheduling the PDR, to a timetable which normally established in the contract. Unless otherwise agreed in the contract, this responsibility includes:

• Arranging a suitable venue for the review. • Coordinating attendance by the contractor and internal staff who provide specialist

review effort. • Ensuring that data and documentation has been delivered in sufficient time to allow

proper evaluation prior to the review. • Establishing an agenda for the meeting.



• Chairing the meeting. • Arranging for a record of the proceedings and actions arising from discussion to be

maintained, either by the contractor or by internal staff.

Staff having responsibility for providing specialist review effort should ensure that they are well prepared to play an active role in the review. This includes being familiar with the specification and contract requirements and having reviewed delivered data relevant to their area of specialisation in advance of the meeting.

15.4 Participants Participants in the review normally include the Project Manager and other project staff nominated by him, Principal Engineers or their delegates, representatives from the customer organisation if appropriate and contractor and subcontractor staff.

Contractor representation depends on the provisions of the contract and in particular whether it has been agreed that subcontractors will be represented.

15.5 Prerequisites Prerequisites for the review are:

• Delivery of such design and other documentation as is required under the contract data requirements, refer EPD 0013 Appendix A.

• Demonstrated progress on the project task to permit a meaningful review to take place.

15.6 Exit criteria and objectives Exit criteria should be established by the Project Manager in advance of the PDR, to reflect the specific circumstances of the contract. However, in general, the review process should demonstrate the existence of a sufficiently mature preliminary design to permit confirmation of the Functional baseline and to establish a preliminary version of the Development baseline.

This evidence should exist in the form of well developed concept drawings, additional specifications if required, evidence that the concept design reflects the requirements of the functional specification, preliminary plans for support requirements and preliminary test and verification plans.

If the review does not provide satisfactory evidence of appropriate progress in all areas, then the review objectives have not been met and the status of the project shall be reassessed. Corrective action may include a requirement to repeat specific aspects of the design and development, requirements for additional data or, in the worst case, consideration of whether the project should proceed in the existing form.

15.7 Specific outcomes Specific outcomes of the PDR should be:

• Formal definition and agreement to the Functional Baseline for the project. • Preliminary definition of the Development Baseline. • Actions to correct deficiencies and residual risks noted during the review. • Agreement that the detailed design and development task should proceed to the

next stage.



15.8 General guidelines The purpose of the PDR is to confirm that the approach being taken to the design and development process is likely to result in a product which meets the functional and performance requirements for the project.

At the preliminary design stage this can only effectively be achieved by ensuring that the developing design has taken account of specified requirements and standards and that the approach being taken by the contractors is both systematic and reflects recognised engineering practice for the type of project or task involved.

Similarly, the purpose of the review is not to ensure that the result is consistent with the preconceived solutions of the reviewers, but that the product meets or is likely to meet the specified requirement.

To some extent this level of confidence can only be achieved through exposure to some of the detailed considerations and results of the design effort. This may involve review of a sample of detailed design calculations in specific areas, or review and discussion of assumptions used in the preliminary design and synthesis stage. However, the purpose is not to verify that the task has been accurately completed, i.e. that the sums are right, but to ensure that the right considerations have been taken into account.

The Tables included in Section 15.9 provides guidelines for the PDR process. The aspects listed are not necessarily the only questions which may be raised as part of the review, but are intended to provide a guide as to the more important considerations in each case. Specific issues should be determined for each review, to suit the circumstances of the review and the project.

The questions generally apply to either a sub-system or a system level PDR, with appropriate tailoring to ensure that the scope of the question and response is appropriate to the level of the review.

Review questions also need to be tailored to the contract statement of work as far as specific deliverables and topics are concerned. For example, there is little purpose or justification in pursuing questions about qualification test requirements or reliability allocation models if neither aspect is required as part of the contract. Questions about the design assumptions and approach, material properties, durability and characteristics, and other aspects which may be regarded as legitimate considerations by a competent designer are completely valid topics for review.




Table 13 Guidelines for Preliminary Design Reviews – Operational Requirements



Status of Functional Specification Performance Requirements Growth Potential

• Contractor to present. The PDR should review the current status of the functional specification, including the status of any Engineering Change Requests (ECRs) raised since the SDR and establish the extent to which the effect of the ECR is reflected in the concept design.

• Contractor to present details of allocation of performance requirements to individual CIs.

• Contractor to demonstrate how the growth requirements have been applied in the design effort.

Operating Concept


• Contractor to provide evidence of how the usage factors and interface requirements have been incorporated in the design. (Detail discussion in systems engineering segment).

• Contractor to show how availability requirement is likely to be met, if this is required in the contract. May be combined with reliability presentation.



• Considered as part of maintenance and reliability aspects in systems engineering.


• Covered in review of detail design.


How Specified Standards used Verification Requirements (General)




• Contractor to present details of interface specifications and other interface documents.

• Contractor to provide details of verification effort and results if required as part of the contract.

• Review to consider whether the defined interfaces are complete and whether the basis for the definition is accurate. May include aspects such as physical attachment, power supply voltage and stability, data rates and formats, earthing, material compatibility, duty cycles and related operating characteristics.



Table 14 Guidelines for Preliminary Design Reviews – Program Requirements


Program Schedule

Work Breakdown Structure Schedule Defined

• Contractor should present further definition of WBS, as a means of ensuring that the review covers all aspects of the project.

• Discussion of schedule is informative only, as a means of assessing risks associated with achieving nominated schedule. Contractor performance in this area is not a primary objective of the technical review.

Designated Reviews

Specific Reviews Nominated

• Not applicable to this review.

Contract Milestones



Quality Assurance

Specified Standard • PDR may consider contractors progress in achieving required accreditation status, if appropriate to the review. Provides visibility of status of contractors systems as a basis for general understanding of capability.



• Contractor to present outline of proposed Functional Baseline and preliminary Development Baseline. Details of both baselines are normally required as part of data delivery prior to the review.

• Both baseline statements are considered by the review and agreement represents a major outcome of the review process.

• Status of functional specification and ECR status covered under an earlier heading form part of the total configuration management content of the PDR and may be combined.


• Consider whether all necessary documentation has been provided, as a general outcome of the review process.

15.10 Systems Engineering The review questions included in the preceding tables are focused on the system level, even though they may be applied at Configuration Item (CI) level in sub-system reviews.

The questions and considerations included in this table apply both at Configuration Item and the system level. Each of the questions should be applied for the individual CI and where appropriate at the system level.



For example, specifications and standards apply to the design approach for CIs; Reliability and maintainability aspects apply for the CI, as do specific reliability allocations to lower level assemblies within the CI. Interface requirements apply for the CI, both with other CIs and possibility at subassembly level within the CI.

Each of these aspects also applies at the system level but with a different focus. For example, discussion of specifications is more likely to be applicable to interface definition, as well as the functional specification for the total project. Reliability data should be focused on allocation between CIs if a total system reliability has been specified.

Table 15 Guidelines for Preliminary Design Reviews – Systems Engineering




• General status is covered by an earlier question under the operational requirements heading. This segment of the review should focus on the functional requirements for individual CIs and the flow down of the requirement to development specifications for the CI.

• The objective is to ensure that specific functional requirements applicable to a CI are properly reflected in the specification for that item.

Design Standards Specified Requirements

• Design standards should be reviewed or readily identified for the item under review.

• Contractor to demonstrate the application of the relevant standard to the concept design presented.

• Review to consider whether the standard has been correctly applied, especially for specifications which cover a range of applications. Primary question is whether the appropriate factors have been chosen for the application and in some cases the justification for the selection.


Contract Requirement

• If the contract requires that the traceability from the functional specification to lower level specifications be demonstrated, the review may expect to be provided with details of this traceability on a requirement by requirement basis. This can generally only be traced accurately to the lower level specification. Traceability from the functional specification to the concept design level is more difficult and is of marginal value.

(continued)



Table 15 Guidelines for Preliminary Design Reviews – Systems Engineering (continued)


Requirements Traceability (continued)

Specific Functional Allocation

• Whether or not formal definition of traceability is required, the review should consider how major requirements of the specification have been reflected in the preliminary design. This may be accomplished through the use of functional flow diagrams and schematics at the higher level for a CI and by review of detail design features at the lower level.


• This segment of the review normally consists of presentations by the contractor to explain and to demonstrate how the preliminary design meets the requirements of the specification AS WELL AS areas where there is some difficulty or residual risk in being able to meet the specified requirement. NOTE that the latter may be related to single requirements or to groups of requirements i.e. where individual requirements can be met but not in combination with others.

• The PDR should not normally take the form of a line by line review of drawings, even though these should have delivered prior to the review and should be available during the review.

• Detailed aspects which may be considered during the PDR process are presented separately in Table 17.

Standardisation Standardisation Requirement

• See Table 18.



• Have appropriate interfaces been defined for the CI and for the system?

• Are these compatible with the interfaces defined at project level?

• Has control of the interfaces been assigned to specific individuals or organisations?


Standards Delivered Documents Design Detail

• This aspect covers a general assessment of the quality and completeness of the drawings and other data presented as part of the PDR. If formal approval is required, this would normally take place as a project management function outside of the scope of the review

(continued)




Table 15 Guidelines for Preliminary Design Reviews – Systems Engineering (continued)


Software Requirements Development Environment Design Detail

• At the PDR level, the evaluation of software elements of a project closely parallel those applicable to hardware items. Specific aspects which may be considered are as follows.

• Status of specifications, including soft-ware requirements specifications if appropriate.

• Allocation of software functions and hardware / software interfaces.

• Structure proposed for software elements, including the designation of computer software configuration items (CSCIs) as appropriate.

• Functional flow and timing analysis. • Proposed programming language, if not

included in the specification. Proposed development environment.

• Estimated sizing and growth provisions • Designated safety elements of the

proposed design.



Table 16 Guidelines for Preliminary Design Reviews – Integrated Support Concept


Maintenance and Inspection Requirements

Requirements Analysis • At the PDR the review of integrated support elements should focus on the requirements and the way in which the analysis is being developed in conjunction with the design development task.

• The review should seek evidence that supportability considerations are being taken into account in the design process. The contractor should be in a position to provide an outline of the proposed approach to detailed analysis tasks planned as the design concept matures.

• A similar level of detail should be expected for each of the aspects listed under the integrated support heading. Specific detail which may be presented by the contractor in each case is as follows.

• Specific requirements and how these are to be integrated with design approach.

• Outline of procedures to be used for analysis in each case.

• Proposed models and computer based tools to be used, including status of contractors system. Proposed documentation methods and formats.

• Preliminary results of analysis. • Validation methods to be used, if

applicable. Spares Support Requirements Analysis As above. Training Requirements Analysis As above. Support Equipment

Requirements Analysis As above.

Facilities Requirements Analysis As above. Operating and Support Documentation


As above.


As above.



Table 17 Guidelines for Preliminary Design Reviews – Verification and Test


Planning Requirement Schedule • Contractor to present outline of each element of Master Test Plan or equivalent showing the following:

• Requirement for each test. • Proposed approach to meeting each

element of the program, including responsibility for the test, if appropriate.

• Proposed schedule for each series of tests.

• Proposed documentation of test plans and test results.

• Results of any development or proving tests completed to that stage in the program.

• Customer support or representation planned or required.

Integration Requirement Schedule As above Qualification Requirement Schedule As above First Article Requirement Schedule As above Commissioning Requirement Schedule As above Software Requirement Schedule As above



The following headings and major considerations are not necessarily exhaustive, nor are all applicable to all projects. The information presented is intended to provide guidance as to the general scope of review topics and may be added to, or aspects may be deleted, as the needs of the project dictate.

Table 18 Detail Design Aspects – Preliminary and Critical Design Review

REVIEW TOPIC MAJOR CONSIDERATIONS Accessibility • Does proposed design provide suitable accessibility

for maintenance or replacement? Clearance • Have specified clearances for other installations and

for traffic been incorporated in design? Corrosion and Surface Protection

• What is susceptibility of proposed materials to corrosion?

• Does design incorporate dissimilar metal junctions or joints?

• What form of protective treatment has been proposed? Can it readily be applied and repaired?

Drainage • Is drainage provision appropriate for proposed design and conditions (as distinct from environmental protection requirements)?

Earthing • Does the earthing and grounding design conform to requirements?

• Is it appropriate for the terrain and area of installation?

• Is it compatible with other equipment? • Has grounding of individual components been

considered in the design? Electromagnetic Design Criteria

• Have electromagnetic compatibility requirements been specified for the design?

• Is the design presented compatible with the intended operating environment?

Environmental Protection • Is project covered by environmental impact statement?

• Does proposed design conform to environmental protection requirements for noise, waste disposal, RF emissions, drainage, appearance etc.?

Fixing and Fastening • Do fasteners and fastening methods proposed rely on high tolerance joints or fits? Have alternatives been considered in such cases?

Heating and Cooling • Is heating or cooling required for operation in the defined environment?

• What provisions have been made? • How has capacity been calculated? Thermal models

prepared? (continued)



Table 18 Detail Design Aspects – Preliminary and Critical Design Review (continued)

REVIEW TOPIC MAJOR CONSIDERATIONS Human Interfaces • Have human factors for operation and maintenance

been considered in the design of equipment? (See also accessibility and maintainability headings)

• Is the design of consoles or other control equipment and the proposed layout of the installation compatible with monitoring of critical functions and proposed staffing arrangements?

• Have lifting and handling considerations been taken into account?

Interchangeability • Is interchangeability required? To what level? • How is it provided for in the detail design?

Loading • Are loading factors and factors of safety appropriate for the defined operating conditions and required growth capability?

• Have most severe design loading cases been considered in design calculations?

• Have stress calculations or models been prepared and presented?

Maintainability and Inspection Provisions

• Does the equipment incorporate facilities for easy access and replacement of sub-assemblies?

• Will special handling equipment be required? Is it provided for?

• Will special test facilities be necessary? Are they available or included?

• Does the design incorporate appropriate test points? • Is lubrication necessary? Have appropriate points

been included? • Can adjustments be carried out if required? What

form of tooling or test equipment is needed? • Does the design incorporate built-in test or

monitoring provisions? Materials • Do any of the materials proposed for use carry a

high risk in terms of durability, susceptibility to damage or fatigue?

• Have alternative materials been considered? Power Distribution • Does the power distribution arrangement presented

meet the specified requirement and does it conform to the relevant standards?

• Does it incorporate the necessary protective and isolation devices?

Produceability • Does design incorporate any aspects which are considered to be high risk for manufacturing or fabrication process?

• Is required dimensional accuracy likely to be achieved by proposed methods?

• Is final assembly feasible on site (if applicable)? • Are special fixtures or jigs needed?

(continued)



Table 18 Detail Design Aspects – Preliminary and Critical Design Review (continued)

REVIEW TOPIC MAJOR CONSIDERATIONS Reliability • Has reliability been allocated to CIs from system

level? • Has a reliability model been prepared? Does it

appear to be complete for the System or CI being considered?

• What derating factors have been assumed (if applicable)?

Safety • Has any form of safety assessment been carried out of the item or installation?

• What specific safety considerations and hazards have been identified?

• How have they been considered? • Does any aspect identified as a safety or protective

feature incorporate single point failure mechanisms? • Has redundancy been considered as a means of

mitigating failure effects? Soil Conditions • Have soil conditions and other features of terrain

been adequately considered in concept design? Sourcing • Are items and materials to be used in the design

readily available from local sources? • If not, have other alternatives been considered? • Is proposed vendor likely to provide stable source of

supply? • Have items and materials proposed by the contractor

been superseded or are they obsolete? Staff Facilities • Do facilities comply with appropriate legislative

requirements? Standardisation • Does design employ standard items, to the extent

required by specification?



16 Critical Design Review

16.1 Purpose The purpose of the Critical Design Review (CDR) is to ensure that the detailed design developed under the contract is complete and has reached the stage and level of maturity necessary to proceed with construction or fabrication and final coding.

In many respects the CDR is a continuation of the PDR process. The objective is to review the results of the detailed design effort and the evolution of the concept design in more detail, to establish whether the design appears to be capable of meeting the specified requirement.

The CDR generally takes place prior to the completion of detail tests required under the contract and when the ability of the design to meet specified conditions has not been demonstrated. However, the purpose is consistent with that of the PDR, where the review effort is oriented toward establishing a high level of confidence in the proposed product and to ensuring that all significant design and development risks have been identified and addressed

16.2 Scheduling Scheduling of the CDR should be set to coincide with the expected completion of the detail design effort, so as not to introduce any delay into the next phase of construction, fabrication or coding.

Under some circumstances it is necessary for construction, fabrication or coding of some elements to proceed in advance of the CDR. This is generally a function of the overall schedule for the project, as well as the need to complete the work on some items in order to provide the necessary level of confidence in the design. In addition, it is necessary to order some material or equipment in advance of the review because of long lead times.

These practical and commercial considerations should be taken into account in both the scheduling of the review and in applying the general guideline that completion of the CDR is necessary prior to construction, fabrication or coding effort. The timing of the review should be established to minimise the amount of work to be completed in advance, while attempting to complete the CDR at a stage where the available data provides a sound basis for assessing the design and integration aspects of the project.

Early or progressive authorisation for work to proceed involves minimal risk in most cases, because of the overriding obligation of the contractor to meet performance requirements and standards established in the contract. A phased approach represents the best alternative, in circumstances where an early start on some aspects is essential to meet project deadlines.

16.3 Responsibilities The Project Manager is responsible for scheduling the review, to a timetable which is normally established in the contract. Unless otherwise agreed in the contract, this responsibility includes:

• Arranging a suitable venue for the review. • Coordinating attendance by the contractor and internal staff who provide specialist

review effort. • Ensuring that data and documentation has been delivered in sufficient time to allow

proper evaluation prior to the review. • Establishing an agenda for the meeting.



• Chairing the meeting. • Arranging for a record of the proceedings and actions arising from discussion to be

maintained, either by the contractor or by internal staff. The minutes should be published and provided to the contractor and other participants, as soon as possible after conclusion of the review.

Staff having responsibility for providing specialist review effort should ensure that they are well prepared to play an active role in the review. This includes being familiar with the specification and requirements and having reviewed delivered data relevant to their area of specialisation in advance of the meeting. Participants should also be familiar with the outcomes of the PDR and any relevant sub-system CDRs held on the items.

16.4 Participants Participants in the review normally include the Project Manager and other project staff nominated by him, Principal Engineers or their delegates, representatives from the customer organisation if appropriate and contractor and subcontractor staff.

Contractor representation depends on the provisions of the contract and in particular whether it has been agreed that subcontractors will be represented.

16.5 Prerequisites Prerequisites for the review are:

• Delivery of design and other documentation as required under the contract data requirements.

• Demonstrated progress on the project task to permit a meaningful review to take place. In most cases this is evident from delivered documentation in the form of drawings and related data but in many cases it is also possible to review prototype hardware and equipment at or close to the intended configuration for the final product.

16.6 Exit criteria and objectives Exit criteria should be established by the Project Manager in advance of the CDR, to reflect the specific circumstances of the contract. However, in general, the review process should demonstrate the existence of a complete design to permit confirmation of the Development Baseline and to establish a preliminary version of the Product Baseline

This evidence should exist in the form of completed drawings and specifications, supporting design data and development test results where applicable, detailed analyses and recommendations covering all integrated support requirements of the contract and well developed plans for validation and testing of the product or installation.

If the review does not provide satisfactory evidence of appropriate progress in all areas, then the review objectives have not been met and the status of the project shall be reassessed. Corrective action may include a requirement to repeat specific aspects of the design, requirements for additional data or, in the worst case, consideration of whether the project should proceed in the existing form or to the current schedule.

16.7 Specific outcomes Specific outcomes of the CDR should be:

• Final definition and agreement to the Development Baseline for the project.



• Preliminary definition of the Production Baseline. This may still evolve as the equipment or installation is manufactured or fabricated but changes from this point should be closely controlled via the ECR system, in accordance with the configuration management plan for the project.

• Actions to correct deficiencies and residual risks identified during the review. • Agreement that remaining aspects of the project should proceed to the next stage

of construction, fabrication or coding.

16.8 General guidelines The format of the CDR is very similar to that established for the PDR and the same general observations apply, even though the criteria for the review are much more specific.

Whereas the PDR should be based on an assessment of the approach being taken to fulfil the requirements of the contract, the focus of the CDR is toward the result of the detail design effort. In many cases this should be in the form of prototype product, supported by test data demonstrating various aspects of performance.

The detailed review questions presented at Section 16.9 incorporate the same general questions covered within the PDR, but should be pitched at a different level and require more specific information in response. This is particularly so for the detail design aspects covered by Table 17 in Section 15.10, which applies to both the PDR and the CDR. In each case the contractor may be expected to provide actual results, covering detailed calculations, or test results, or physical data about the design.

The same observations apply to support aspects, where it should be possible to review actual maintenance analysis results, spares listings and publications, for example, and to compare these to the detail design for the product.




Table 19 Guidelines for Critical Design Review. – Operational Requirements




• Review any ECRs raised since the PDR and establish approval status.

• Confirm Functional Baseline definition in terms of original specification plus changes.

Operating Concept


• Contractor to confirm that usage factors and interface requirements have been incorporated in the design. Detail discussion in systems engineering segment.

• Contractor to show how availability requirement will be met, based on actual reliability predictions for the design (if this is required in the contract). May be combined with reliability presentation.



• Considered as part of maintenance and reliability aspects in systems engineering.




How Specified Standards used Verification Requirements (General)

• Covered in review of detail design, verification and test.



• Contractor to present details of updated interface specifications and documents.

• Contractor to provide details of any unresolved interface aspects and to provide assessment of impact on any outstanding design aspects.

• Review to consider whether the defined interfaces are accurate and complete. (May include aspects such as physical attachment, power supply voltage and stability, data rates and formats, earthing, material compatibility, duty cycles and related operating characteristics, including results of PDR consideration in this area).



Table 20 Guidelines for Critical Design Review. – Program Requirements


Program Schedule

Work Breakdown Structure (WBS) Scheduled Defined

• Contractor to present final WBS allocation to least the level of indenture required by the contract. Note that this provides a useful means of ensuring that the review has covered the total project scope.

• Presentation by contractor of planned schedule for completion of remaining major milestones is appropriate for the CDR.

Designed Reviews

Specific Reviews Nominated Defined Scope Responsibilities Defined


Contract Milestones



Quality Assurance

Specified Standard Test Articles

• Presentation of current status of contractor and subcontractor accreditation is useful for the review, as are details of any reviews and audits carried out on facilities or first article hardware.

• Separate coverage is optional, but the information is required in some part of the review.



• Confirmation of Functional Baseline, at beginning of the review (covered at operational requirements table, Table 19).

• Review status of development specifications, leading to confirmation and further definition of the Development Baseline from PDR status. The Development Baseline should be finalised as a result of the CDR effort.

• Establish preliminary version of the Production Baseline.

• Review general plans for conduct of PCA process.

• Confirm and agree to the process for approval and control of any changes to the baselines following completion of the CDR.


• Contractor to advise progress in development of all documentation necessary as part of the statement of work. Some aspects of the documentation task, such as operating and maintenance manuals are reviewed separately in addition to this general status report.



Table 21 Guidelines for Critical Design Review. – Systems Engineering




• General status covered under operational requirements, baseline definition under configuration management.

• Systems engineering segment of the review may need to examine current status of functional and development specifications, including detailed status of ECRs, in more detail as a basis for review of the detailed design.

• Contractor should provide specific information on the basis for the detail design to be presented to the review i.e. which issue of the specifications or changes are reflected in the design.

Design Standards Specified Requirements Applicability

• Included in the review of design aspects for individual CIs and system. Major specifications used should be presented for each item reviewed.


Contract Requirement Specific Functional Allocation

• Presentation should reflect the result of the contract requirement for traceability.

• Contractor should present results of functional allocation process i.e. what functions are to be performed by what part of the hardware or software in the proposed final design.

• This should be an update from the PDR but can be important, even where it appears to be self evident, as a means of ensuring that all requirements have been incorporated in the design solution.

Design Synthesis Status of Development Task Design Detail

• Contractor should present an overall assessment of the status of the design and development task for each CI. Detailed design aspects at Table 17 in Section 15 are applicable to the CDR.

• This part of the review should include an overall assessment of the progress made toward integration of individual CIs and the compatibility of vendor items to be incorporated into the design.

• Details of the test program are covered separately in the review but inclusion of details of development test data provides the means of establishing the level of maturity of the design.

(continued)



Table 21 Guidelines for Critical Design Review. – Systems Engineering (continued)


Standardisation Standardisation Requirement Application



Interface Specifications Status and Detail

• Status of interface specifications should be presented as part of the earlier review of specifications.

• This segment of the review is intended to focus on specific interface design issues, including unresolved aspects.

• Review should consider whether defined interface requirements have been met within the design, including such aspects as power input and stability requirements, earthing or grounding, timing, resonance, bogie dynamics.


Standards Delivered Documents Design Detail

• This aspect covers a general assessment of the quality and completeness of the drawings and other data presented as part of the CDR.

• Assessment may cover aspects such as the format and layout of the drawings, dimensions and tolerancing, the use of parts lists and the existence of a well defined and controlled numbering system.

• The format and means of integrating vendor data into the data package should also be considered as part of this topic.

• Formal approval of delivered data would normally take place as a project management function.

(continued)




Table 21 Guidelines for Critical Design Review. – Systems Engineering (continued)



The evaluation of software elements follows the same pattern as that established at the PDR. Specific aspects which may be considered are as follows: • Status of specifications, including

software requirements specifications and other software detailed design documents, as appropriate.

• Structure defined for software elements, including the designation of computer software configuration items (CSCIs) and the current status of each element of the design.

• Allocation of software functions and hardware and software interfaces. This is the detailed extension of the requirements allocation and traceability process, for software elements.

• Functional flow and timing analysis. • Estimated sizing and growth

provisions, particularly where the software is to be integrated with specific target hardware, or implemented as firmware.

• Elements of the design designated as safety critical and action taken during the development process to verify the correct operation of these parts of the design..


• Built in test and diagnostic routines, including the extent of coverage of these tests

• Recovery characteristics after unscheduled interruption, such as power failure.

• Status and results of development testing.

• The requirement for any special support facilities and the status of these.



Table 22 Guidelines for Critical Design Review. – Integrated Support Concept


Maintenance and Inspection Requirements

Analysis Recommendations

• Contractor to present the results of FMEA or FMECA or maintenance analysis completed for each item and the system, including recommendations for maintenance.

• The review should consider the general approach and whether the recommendations appear to have been developed through a systematic appraisal of the requirements of the equipment. Detailed review and decisions as to adoption take place outside of the review process.

Spares Support Analysis Recommendations

• Contractor to present details of spares support analysis process, including recommended spares lists where these are required as part of the contract task.

Training Requirements Analysis Recommendations

• Contractor to present results of training analysis completed, including the following:

• Requirement for specialist courses. • Progress toward developing training

courses, if this is required as part of the contract, otherwise the source and availability of appropriate training.

• Development of training material, where required.

• Validation of training courses and material.

Support Equipment

Requirements Recommendations

• Contractor to present recommendations for any special support equipment needed to install, remove, handle or maintain the equipment covered by the design.

• Compatibility of the design of the prime equipment, with major test or support equipment specified as “standard” within the contract specification, should also be reviewed as part of this section of the presentation.

(continued)



Table 22 Guidelines for Critical Design Review. – Integrated Support Concept (continued)


Facilities Requirements Analysis Recommendations

• Contractor to present details of any special facilities required to maintain the equipment.

• Note that this excludes major new facilities, such as new buildings, which may be considered separately within the project but covers aspects such as special power supplies for maintenance, air conditioning and other services.


• Contractor to present details and status of any special or reusable packaging required to support the equipment.

Operating and Support Documentation

Specifications Development Product

• Contractor to present details of any operating, support or maintenance instructions being developed or supplied under the contract, including vendor handbooks and data where applicable. This should include an outline of the content of the documents, the status of development, examples of completed parts of the documents and proposed methods of validating the finished product.

Table 23 Guidelines for Critical Design Review. – Verification and Test


Planning Requirement Schedule

• Contractor to present an overview of the Master Test Program, including major tests planned, facilities to be used (if appropriate), timing and proposed method of managing the test process.

• The presentation should cover all forms of testing, including development testing carried out as part of the design process..

• The reviewers should consider whether the proposed test program meets the objectives of the contract task, with further detail to be presented within individual test programs

(continued)



Table 23 Guidelines for Critical Design Review. – Verification and Test (continued)


Integration Requirement Schedule Results

• The contractor should identify requirements for integration testing, to demonstrate that major elements of the design are compatible and to test and verify interfaces.

• This should include details of the test configuration, specific functions and parameters to be tested and the sequence of tests where progressive integration is to take place.

• The results of any integration testing conducted prior to CDR should be presented for review.

Qualification Requirement Schedule Results

• The contractor should present details of planned qualification test programs, including the schedule for completion and test facilities to be used. This may include aspects such as material compatibility, durability, EMI and EMC, environmental and service testing, if required in the contract.

• These should be pitched at major item level, to cover the range of tests to be completed on each item and the traceability of these tests to specified requirements. Details of individual test plans and test methods are subject to separate review outside of the CDR process.

• Presentation to include details and results of any preliminary or final tests completed prior to the CDR.

First Article Requirement Schedule Results

• Contractor to identify plans for completion of any first article tests or examinations required under the contract task, including results of any testing carried out prior to the CDR.

Commissioning Requirement Schedule • Contractor to present plans for commissioning tests, including the scope and purpose of each aspect of the test program, facilities required and customer participation.



17 System Verification Review

17.1 Purpose The purpose of the System Verification Review (SVR) is to demonstrate that all verification and testing effort required under the contract has been satisfactorily completed, prior to acceptance of the product or project. An SVR as defined within this section fulfils the purpose of a Functional Configuration Audit. (A functional configuration audit (FCA) is conducted to determine whether a configuration item has the performance and functional characteristics specified in its functional baseline.)

The SVR is a very focused review. Requirements for specific tests should have been progressively identified during the project, either directly as a result of contract requirements, e.g. commissioning, or should have been developed from more general requirements. Examples of the latter include aspects such as the need to demonstrate the ability of the design to withstand specified environmental conditions, or the functional performance of software developed or supplied as part of the contract task.

Similarly, detailed plans for the testing should have been developed and approved as part of the normal project management process and the results of specific tests should have been reviewed on a case by case basis, as they are completed. The SVR serves to confirm that all verification and test requirements have been completed and to identify any areas in which the product does not meet specified performance requirements.

17.2 Scheduling The SVR is normally scheduled when all required verification and test programs have been completed and the results are available. In cases where it is not possible to finalise all required tests by the project completion date the SVR may proceed, on the basis that a final assessment of completion status will not be given until the outstanding tests have been completed.

17.3 Responsibilities The Project Manager or Principal Engineer is responsible for ensuring that the SVR is conducted where required for a project. Completion of the SVR process may be delegated to the Test Engineer for a major project, or to a suitably experienced engineer for smaller projects.

The manager responsible for conducting the review is to make suitable arrangements for the review, including:

• Arranging a suitable venue and agenda for the meeting. • Arranging for test plans and results to be available, including test databases if

maintained. • Arranging for attendance by the prime and subcontractors, as appropriate. • Arranging for a record of the meeting and for distribution of the minutes as soon as

possible after completion of the review.

The review normally includes a presentation by the prime and subcontractors as appropriate, covering the aspects identified in Section 17.9.



17.4 Participants Participants in the review are the Project manager or his delegate, specialist engineers from each area to be covered by the review and representatives of the prime and subcontractors as applicable.

17.5 Prerequisites Completion of all designated verification and test activity required as part of the project scope is necessary for conduct of the review, except as provided for under the scheduling heading.

17.6 Exit criteria and objectives The review is to ensure that all verification and test requirements required as part of the project scope of work are identified and acquitted.

17.7 Specific outcomes Specific outcomes of the SVR are:

• Ensure that traceability exists from the requirements of the functional specification and any development specifications, to the tests completed as part of the project.

• Identify other essential testing, such as integration, commissioning and software testing, carried out as part of the project.

• Ensure that each test and series of tests has been completed and that the results have been documented and approved as meeting the functional requirements of the specification, or the requirements of the contract statement, as appropriate.

• Identify tests which have not been completed, or those where the required level of performance has not been met.

• Establish corrective action where the specified level of performance has not been met.

17.8 General guidelines The SVR is essentially an audit of the performance characteristics of an item or installation, comparing demonstrated performance to the specified requirement.

Several steps are essential as part of this process. These are shown in the Table in Section 17.9.




Table 24 Guidelines for System Verification Review



Status of Functional Specification Performance Requirements Requirements Traceability

• The review is to ensure that the Functional and Development Specifications used as a basis for the SVR are to the latest approved standard, and include reference to all ECRs which have been approved and implemented.

• Performance requirements included in these specifications establish the baseline for the review.

• The review is also to consider performance related test requirements arising from contract requirements, such as integration and commissioning.

• Requirements should have been separately identified for traceability purposes and possibly as part of a Master Test Plan. These records provide a checklist for the purposes of verifying compliance.


Standards Used Verification Requirements (General)

• Environmental specifications are used for reference purposes only as part of the SVR.

• Detailed applicability and use of the specifications should have been verified at the time of the test.



• Status of functional and development specifications is covered under the functional requirements heading.

• The SVR may need to verify that the configuration of items tested represents the latest approved configuration, or that any differences have been reconciled.

• This should have been done at the time the test was performed but the SVR should check approved ECRs, to determine whether they have an impact on the verification process.

• Approved Deviations and Waivers may also impact on performance aspects and should be checked if any discrepancies are noted.

(continued)



Table 24 Guidelines for System Verification Review (continued)


Planning • Review the Master Test Plan for the project, to determine the range of tests required as part of the project.

• The accuracy and completeness of the test plan should have been separately managed as part of the project and verification of the plan should be confined to ensuring that changes arising from ECRs are incorporated.

Qualification • Review each element of the qualification test plan or qualification database if one has been maintained.

• Review the results of each test carried out.

• Identify and record any deficiencies. Integration • Review the results of any formal

integration tests required under the contract.

• Identify and record any deficiencies. First Article • Review the results of first article tests,

if required by the contract. • Identify and record any deficiencies.

Commissioning • Review completion records for commissioning tests.

• Identify and record any discrepancies.

Test Results

Software • The review should verify that all required tests have been completed and that the results were satisfactory.

• This record is normally be found in a separate software test plan, which should also contain results.

• Identify and record any deficiencies.



18 Physical Configuration Audit

18.1 Purpose The purpose of the Physical Configuration Audit (PCA) is to examine the “as-built” configuration of an item against its configuration documentation, to ensure that the item conforms to the documentation.

Completion of a PCA also provides the basis to formally establish the Production Baseline for the equipment or installation. This baseline consists of specifications, drawings, plans and other documentation describing the “as-built” standard of the item or items concerned.

18.2 Application A PCA is required for every project involving design and development of new equipment or infrastructure, irrespective of whether previous technical reviews have been carried out.

18.3 Scheduling The PCA should be scheduled after completion of the design and development phase of a project and either:

At the completion of infrastructure construction projects, where the installation is essentially unique, or

Prior to delivery of the first article, for projects involving manufacture or fabrication of a number of products to the same design.

The PCA may be scheduled to take place progressively and this is often the most effective approach. Phased PCAs are necessary for projects involving a mix of products and infrastructure, or for those involving a number of separate configuration items. Planning for the review is also influenced by the level of effort involved and the phasing of different configuration items in the design and manufacturing cycle.

Progressive reviews are also necessary in some construction projects because of the difficulty in access at later stages in the project. Specific milestones should be established in such cases, usually coinciding with other mandatory inspection points e.g. inspection of footings for buildings or other structures or closure in other manufacturing tasks.

Application of the PCA process for software relies on review of other records likely to be available as part of the development process, such as code walkthroughs or the results of Independent Verification and Validation reviews, if these are included as a requirement of the project.

18.4 Responsibilities The Project Manager or Principal Engineer responsible for the project is to ensure that an effective PCA is carried out.

Responsibility for the completion of the audit may be delegated to the Quality or Configuration Managers. The results of inspections carried out by licensed building inspectors may also be used as part of the PCA record, provided that satisfactory evidence is available of the status of plans used for the inspection.



18.5 Participants The PCA is normally be carried out by suitably qualified engineers and quality assurance staff assigned by the project Manager or Principal Engineer, assisted by the contractor as required.

18.6 Prerequisites Completion of the audit requires the following:

• Access to a set of drawings or plans for the item or installation to be audited which conform to the current approved configuration.

• Access to hardware which is representative of the configuration defined in the drawings, or site access for construction projects.

• Access to manufacturing and quality records, including approved deviations and waivers and production test records for the item.

18.7 Exit criteria and objectives The audit is required to verify that the item inspected conforms to the documentation or that any differences have been reconciled. Any differences which cannot be reconciled should be recorded for resolution following the PCA.

18.8 Specific outcomes Specific outcomes of the PCA process are:

• A record of non conforming items. • Formal definition of the Production Baseline once any discrepancies have been

resolved.

18.9 General guidelines The PCA involves detailed comparison of hardware or construction work with the drawings or plans.

Several basic decisions are needed as to the level and extent of the process but, in general, it should seek to establish confidence that the product is both consistent with the documentation and that methods and processes called up in the documentation have been used in the manufacturing or construction phase. This is an important consideration, where additional items may need to be built at some later stage, but may also have an impact on the durability of the product.

PCA does not require a one hundred per cent comparison, but may be based on a sample of the most significant subassemblies or characteristics. Selection of the range of items or characteristics to be audited is the initial decision and may be made on the basis of design or manufacturing complexity or the level of technology represented in the design.

For example, audit of a Configuration Item comprising a number of subassemblies may cover only selected assemblies and the final assembly and test process. Audit of electronic circuit boards or modules would normally be confined to the module level, even though manufacturing records and receipt inspection data held within the manufacturer’s quality system may be reviewed to verify the quality and identity of component parts or materials.

Similar techniques may be applied to structural items and assemblies, where sampling or access to material or production records may be useful in establishing compliance with



the drawing. Verification of bought in items is normally confined to review of purchasing and inspection records, as well as physical inspection of the item and comparison of data plate information with that specified in the design.

Conduct of the audit follows several steps, as follows:

• Selection of the item or area of the product or installation to be audited. • Assembly of the design and manufacturing data for the item. • Verification that the data is at the current approved status for the item and that any

outstanding ECRs have been identified and are available. • Assembly of selected manufacturing records for the item. This may include parts

lists, process records, inspection records, deviations and waivers and test procedures and results.

• Physical examination of the item or installation against the data. • Recording and resolution of discrepancies.

Physical examination should focus on several major aspects, as follows:

• The physical identification of individual items, where this is possible. This may need to be accomplished through examination of material markings and codes or through build records. In other cases the identity of the item can be verified through manufacturers’ part numbers or model identification.

• The layout and physical assembly details of the item, i.e. does it look the same as it does in the drawing.

• Manufacturing method, for example, is the item made from a casting when the drawing calls for fabrication.

• Evidence that the item has been manufactured using tooling, jigs or fixtures called up in the design.

• Dimensions, clearances and tolerances as per drawing. • Use of test procedures and equipment as specified in the design.

As with previous reviews the underlying purpose of the PCA is to establish confidence in the design. In this case the review is seeking verification that the item is capable of being constructed or fabricated to the approved design, that the process is repeatable and that the “as-built” documentation accurately reflects the final product. The level of detail to be covered by the audit should be tailored accordingly.



Appendix A Background documents In the initial development of this Manual, reference was made to Electronic Industries Association Interim Standard EIA/IS-632, Systems Engineering, 1994, (now issued as ANSI/EIA-632-1999 Processes for Engineering a System) on which was based the concept of technical reviews and the general principles included in this Manual. Descriptions of the reviews, including the title and content of each, were tailored to the specific applications of RailCorp. The reviews described within this Manual are not directly aligned with those described in ANSI/EIA-632, but the purpose and intent of the total set is equivalent.

While the EIA Standard forms the primary reference source, the processes are generic to any form of engineering development or acquisition project and further reference material was drawn from a range of sources.

Other background documents are:

ANSI/EIA-632-1999 Processes for Engineering a System AS/NZS 15288 Systems engineering – System life cycle processes ISO/IEC 26702 (IEEE Std 1220-2005) Systems engineering – Application and management of the systems engineering process INCOSE. Systems Engineering Handbook NASA. Systems Engineering Handbook SMS-09-PR-1473 Application of Network Design Authority for New and Altered Assets

Australian Standard AS 4292.1-1995, Railway Safety Management, Part 1: General is used as a further source, particularly with regard to the close link between the technical review process and safety.

The following internal publications are also used as references in establishing the purpose and conduct of individual reviews:

RPMM Program Management Guidelines from the suite of guidelines included in RailCorp Project Management Methodology Rail Asset Enhancement Guidelines (RPMM RAE) EPA 280 Design Acceptance BMS-051-ST-002 RTAM Strategic Overview The following internal publications are no longer available but were used as references in establishing the purpose and conduct of individual reviews in the original version of this Manual (Technical Reviews Policy Manual, AM 0022 PM):

• Asset Management Policy Manual, State Rail Authority of NSW. • Configuration Management Policy Manual, CSR 0002 • Specification and RFT Policy Manual, AM 9998 PM, RSA



Appendix B List of Tables Section 11 Conduct of reviews

Table 1 – Technical Reviews Planning Checklist

Section 12 Technical reviews cross reference

Table 2 – Technical reviews cross reference

Section 13 System Concept Review

Table 3 – Guidelines for System Concept Review – Operational Requirements

Table 4 – Guidelines for System Concept Review – Program Requirements

Table 5 – Guidelines for System Concept Review – Systems Engineering

Table 6 – Guidelines for System Concept Review – Integrated Support Concept

Table 7 – Guidelines for System Concept Review – Verification and Test

Section 14 System Definition Review

Table 8 – Guidelines for System Definition Review – Operational Requirements

Table 9 – Guidelines for System Definition Review – Program Requirements

Table 10 – Guidelines for System Definition Review – Systems Engineering

Table 11 – Guidelines for System Definition Review – Integrated Support Concept

Table 12 – Guidelines for System Definition Review – Verification and Test

Section 15 Preliminary Design Review

Table 13 – Guidelines for Preliminary Design Reviews – Operational Requirements

Table 14 – Guidelines for Preliminary Design Reviews – Program Requirements

Table 15 – Guidelines for Preliminary Design Reviews – Systems Engineering

Table 16 – Guidelines for Preliminary Design Reviews – Integrated Support Concept

Table 17 – Guidelines for Preliminary Design Reviews – Verification and Test

Table 18 – Detail Design Aspects – Preliminary and Critical Design Review

Section 16 Critical Design Review

Table 19 – Guidelines for Critical Design Review. – Operational Requirements

Table 20 – Guidelines for Critical Design Review. – Program Requirements

Table 21 – Guidelines for Critical Design Review. – Systems Engineering

Table 22 – Guidelines for Critical Design Review. – Integrated Support Concept

Table 23 – Guidelines for Critical Design Review. – Verification and Test

Section 17 System Verification Review

Table 24 – Guidelines for System Verification Review