tq final digital 4

Guidance Notes forTechnology Qualification

December 2014

Lloyds Register

Lloyds Register is a trading name of Lloyds Register Group Limited and its subsidiaries.

For further details please see http://www.lr.org/entities Lloyds Register Group Limited, its subsidiaries and affiliates and their respective officers, employees or agents are, individually and collectively, referred to in this clause as Lloyds Register. Lloyds Register assumes no responsibility and shall not be liable to any person for any loss, damage or expense caused by reliance on the information or advice in this document or howsoever provided, unless that person has signed a contract with the relevant Lloyds Register entity for the provision of this information or advice and in that case any responsibility or liability is exclusively on the terms and conditions set out in that contract.

A guide to the Guidance Notesand published requirements

Guidance Notes forTechnology Qualification

IntroductionThese Guidance Notes are intended to be a live document and are subject to change without notice.

A comprehensive List of Contents is placed at the beginning of these Notes.

Numbering and Cross-ReferencesA decimal notation system has been adopted throughout. Five sets of digits cover the divisions, i.e., Part, Chapter, Section, sub-Section and paragraph. The textual cross-referencing within the text is as follows, although the right hand digits may be added or omitted depending on the degree of precision required:(a) In same Chapter, e.g., see 2.1.3 (i.e., down to paragraph).(b) In same Part but different Chapter, e.g., see Ch 3,2.1 (i.e., down to sub-Section).(c) In another Part, e.g., see Pt 5, Ch 1,3 (i.e., down to Section).

The cross-referencing for Figures and Tables is as follows:(a) In same Chapter, e.g., as shown in Fig. 2.3.5 (i.e., Chapter,Section and Figure Number).(b) In same Part but different Chapter, e.g., as shown in Fig. 2.3.5 in Chapter 2.(c) In another Part, e.g., see Table 2.7.1 in Pt 3, Ch 2.

Lloyds Register 1

Contents

Technology Qualification

Section 1 Introduction 2

2 Abbreviations and Definitions 4

3 Technology Qualification Principles 4

4 TQ Stage 1: Technology Appraisal 6

5 TQ Stage 2: Technology Qualification Plan and Execution 11

6 TQ Stage 3: Performance Review and Certification 14

Appendix 1 Existing Guidance on Technology Qualification 16

2 Technology Qualification Roadmap 17

3 Technology Maturity Level 18

4 Integration Maturity Level 19

5 IML Adjusted TML Assignment Methods 19

Guidance Notes for Technology Qualification, December 2014

Lloyds Register Group Limited 2014. All rights reserved.

Except as permitted under current legislation no part of this work may be photocopied, stored in a retrieval system, published, performed in public, adapted, broadcast, transmitted, recorded or reproduced in any form or by any means, without the prior permission of the copyright owner. Enquiries should be addresse Lloyds Register Group Limited 2014Published by Lloyds Register Group LimitedRegistered office (Reg. no. 08126909)71 Fenchurch Street, London, EC3M d to Lloyds Register Group Limited, 71 Fenchurch Street, London, EC3M 4BS.



Section

1 Introduction

2 Abbreviations and Definitions

3 Technology Qualification Principles

4 TQ Stage 1: Technology Appraisal

5 TQ Stage 2: Technology Qualification Plan and Execution

6 TQ Stage 3: Performance Review and Certification

n Section 1 Introduction

1.1 GeneralThe increasing complexity and innovation in new technology can often lead to the development of novel designs, concepts or applications not covered by existing rules, normative standards and industry practices.

Innovative engineering concepts and/or the application of exist-ing technology to new environments challenge the certification bodies, regulators and standard procedures. Therefore the unknown associated with the deployment of novel technology needs to be assessed to prevent undesired outcomes.

1.2 PurposeThese Guidance Notes present Lloyds Registers Technology Qualification (TQ) methodology to quantify the risks brought in by novel technologies, which should be applied prior to deployment of the new technology. The Technology Qualification is a robust and systematic risk assessment process that demonstrates to interested parties that the uncertainties introduced by a novel technology, or new application of an existing technology have been considered and any associated risks mitigated.

The Technology Qualification is a goal risk-based process that uses the readiness level framework, total system perspective and whole lifecycle approach to qualify new technologies, unconventional designs and new ways of applying existing technology.

TQ is a methodology that provides assurance to owners, operators, suppliers and investors at the different stages of novel technology development.

The TQ process is a methodology developed with specific acceptance criteria for technology under review. This is focused on, but not limited to, the following areas: Safety Environmental Functionality Performance Reliability Availability

This includes, but is not limited to, the following activities: Identifying technology elements and their readiness levels Assessing risks posed by the elements Establishing acceptance criteria and goals Evidence collection Further analysis and testing including test failure analysis Performance review.

The TQ process can be applied at any stage during the lifecycle of development of systems or technology.

The outcomes of the process at one stage of the TQ process will be used as inputs for the next stage. Traceability from qualification requirements, qualification activities, testing results, qualification results (including recommendations to mitigate identified risks) is ensured throughout the whole Technology Qualification process.

1.3 Overview of the ProcessFigure 1.2 shows the stages in the TQ process and is followed by a summary of the stages in Sections 4, 5 and 6.

Section 1

Figure 1.1 Lifecycle of development of systems or technology

Businessfeasibility Concept Design

DetailedEngineering Construction

Operation & Maintenance Decommission

Lloyds Register2

Lloyds Register 3


1.4 Summary of TQ stages:

1.4.1 Appraisal (Section 4)There are two steps within the Appraisal Stage; System Decomposition and Technology Assessment & Risk Assessment (as two separate activities).

1.4.1.1 System DecompositionThe first stage in the Technology Appraisal is to break down the technology into distinct elements that can be individually assessed. System Decomposition is a structural and functional breakdown of the overall system into subsystems, equipment, and components down to the smallest element that can be assessed. The outcome of System Decomposition step will provide the basis for review and assessment. It defines, but is not limited to, the scope for the Technology Qualification review, document system and operat-ing conditions, functions and sub-functions requirements, process sequences and operations and capabilities with respect to qualitative and quantitative claims and goals.

1.4.2.1 Technology AssessmentTechnology Assessment uses the Technology Readiness Level (TRL) approach to screen the maturity and integration of the technol-ogy or items of equipment. Based on the assigned TRL and the operating experience for each technology or each item of equipment, a screening level from 0 to 7 is determined. This score dictates how thorough and intensive the qualification activities should be. Integration of technology or equipment into a system can provide challenges even for matured technology or equipment; therefore, an integration readiness review is performed as part of the TRL assessment.

1.4.2.2 Risk AssessmentThe analysis identifies which technological elements drive the risk and issues risk reducing measures. It is recommended that the risk assessment uses methods acceptable to LR to analyze the risks of failure, which means estimating the consequence and frequency of occurrence.

1.5.1 Plan & Execute (Section 5)At this stage qualification methods are selected based on inputs from the previous stages. LR can prepare the TQ plan or support its development or provide an independent review of the plan. A Statement of Endorsement can be provided upon satisfactory review of the TQ.

The TQ plan is implemented and outcomes of qualification activities according to the plan are documented.

1.6.1 Review and Certification (Section 6)For those TQ activities which are not undertaken by LR, the results are submitted to LR for review and assessment. Qualification activities on critical technologies that govern safety systems or provide safety functions are witnessed by LRs approved personnel.

Once the evidence, analysis and reasonable arguments have been documented the content may be submitted to LR for review whereupon LR will issue a Certificate of Technology Qualification, approving the readiness of the technology for application or as regulatory compliance documents, if the documentation is found to be in accordance with this TQ process specified in this guidance note.

In the scenario where the documentation does not satisfy the process specified in this Guidance Note the documentation (including all the gathered evidence and analyses) may be developed further for those areas that were not satisfied.

Section 1

Figure 1.2 TQ process stages

Stage 1: Appraisal (see Section 4)

Stage 2: Plan & Execute (see Section 5)

Stage 3: Review & Certification (see Section 6)

System Decomposition

TechnologyAssessment

RiskAssessment



n Section 2 Abbreviations and Definitions

The following abbreviations are applicable to these Guidance Notes unless otherwise stated.

*Lloyds Register Technology Qualification Team (LR TQ Team)

LR can provide a LR TQ Team (independent third party) comprised of all required, authorized competencies in representative design, manufacturing, operation and maintenance (end-users). The LR TQ Team will also have or be able to access the necessary technical expertise in risk assessment, implementation, integration and installation of the technology and systems. The LR TQ Team will be assigned to evaluate the TQ Program and analyse the independent tests, the evidence, claims, and arguments which confirm compliance to this TQ procedure and specified Rules & Standards through witnessing and assessing implementation of the TQ Process or specific tests and analyses. Proven technology elements can be verified to be in compliance with standards which are acceptable to LR or to be required to provide relevant evidence of their performance records. Participation of the LR TQ Team at each TQ stage is advised, as stated in the guidance. Where applicable, the managerial and/or organizational relationship between the facilities where testing and/or validation activities are being performed and the LR TQ Team may be documented. The Technology Qualification Team might also comprise of members from other stakeholders In the TQ process.

n Section 3 Technology Qualification Principles

3.1 GeneralAt early stages of TQ and/or early development stages of the technology elements, expert opinions contribute a significant role to TQ. The opinion includes identification of elements to be qualified, safety and reliability evidence to be developed, and interpretation of the evidence provided. It is recommended to use a structured process for drawing expert opinions, and demonstration of systematically analysing this information to the satisfaction of the technology elements safety and reliability claims and goals.

TQ is an iterative process and before deployment, a technology concept may be reviewed and improved several times using observations and the results at different steps of the TQ process at different development stages of the technology. Traceability to demonstrate the link among identified technology elements, identified risks, and collected evidence is hence required throughout the TQ process for the whole development lifecycle of the technology. TQ cost is mainly accounted for by evidence collection (mainly testing); therefore resources should be focused on building and improving the relevant data and information of the right evidence. The structured and systematic process and approach should be able to determine which aspects of the evidence are required to be enhanced and to easily identify main factors contributing to the uncertainty about the satisfaction of TQ requirements.

In this procedure, a systematic and structured assessment approach built on the notion of goal-based assurance cases might be applied for modelling goals and claims, expert opinion elicitation, structuring arguments and evidence, identifying and decomposing qualification acceptance criteria, qualification plans, and arriving at conclusions. Several goal modelling languages exist, such as i* and KAOS.

The basic objectives of the Technology Qualification process should be established and agreed before the start of the TQ process. These objectives should be documented as measurablecriteria covering all aspects of the technology such as already mentioned; Safety, Environmental, Functionality, Performance, Reliability and Availability.

3.2 Basic Inputs to TQThroughout the TQ process, documentation needs to be clear, transparent, traceable and well defined to avoid misinterpretations and where specifically required by LR, documentation should be recorded in the standard format approved by LR.

The information submitted for TQ review by LR should aim to satisfy and confirm that performance requirements, goals and claims of the technology in its target operating environment are met.

Sections 2 & 3

CTE Critical Technology Element

FHA Functional Hazard Assessment

FMECA Failure Mode, Effects and Criticality Analysis

HAZID Hazard Identification

HAZOP Hazard & Operability Analysis

IML Integration Maturity Level

KAOS Knowledge Acquisition in Automated Specification

LR Lloyd's Register

LR TQ Team* Lloyd's Register Technology Qualification Team*

MDR Master Document Register

MODU Mobile Offshore Drilling Unit

MOU Mobile Offshore Unit

RCA Root Cause Analysis

SME Subject Matter Expert

SWIFT Structured What-If Technique

TQ Technology Qualification

TQP Technology Qualification Plan

TML Technology Maturity Level

TRL Technology Readiness Level


Lloyds Register4


In addition, the following information would need to be submitted: Requirements definition List of Codes and Standards applied General descriptions Context of use

If a Master Document Register (MDR) is available then this should also be submitted.

3.3 Requirements DefinitionDefine the functional behavior and performance requirements of the technology in the environments to which the technology is in-tended to be operating in and identify the source of requirements.

3.4 General DescriptionIntroduce the technology, its intended functionality, capability and performance when operating in the target environment. The gen-eral description should be supported by the following information as applicable for the Technology under consideration:

Technology: Basis of design: Documentation of justifications to applied codes and boundary

limits if exceeding recognised scope of standard, standards, and regulations for design criteria

Capacity and performance requirements, reliability philosophy, operation and maintenance and repair philosophies, etc.

Reference to other criteria Functional descriptions and drawings: Equipment list General arrangement drawings System block diagrams Single line diagrams Description of operating modes Description of safety related arrangements Description of interfaces to other machinery, equipment and

systems Plans of physical arrangements Changes or modifications from original approved qualified design or context of use Software-architecture, functional specifications, user interface,

data flow, control logic sequences, etc. Detailed drawings and descriptions of technology elements to

be qualified Records and content of document revisions

Manufacturing and installation: Manufacturing, installation, and software development process, procedures, certification, records Specification of tools, machinery, and materials (including relevant certificates) Other certificates as relevant

Operation and Maintenance: Relevant operating and maintenance manuals

Allowable values of operating, maintenance, and, environmental conditions Specification and procedures of maintenance, inspection, and repair, upgrade, or modification in the intended context of use.

3.5 ConfidentialityConfidentiality agreements should be established among involved parties where applicable. However, this requirement also needs to take account of legislative requirements or when disclosure is authorised by the involved parties. Availability of the required documentation for TQ review should not be restrained by confidentiality agreements. More thorough testing and analysis may be expected in the scenario that the original documents are unavailable or inaccessible.

3.6 Context of useThe context of the equipment or system undergoing Technology Qualification needs to be defined. The context should refer to the operating environment, the integration with existing systems, the human interactions, and the operation and maintenance. A comprehensive statement describing in detail the operating environment where the technology is deployed should cover the following points, as a minimum: Fully and clearly describe the manner and purpose of use for

the technology Identify existing key points of reference Address any important criteria regarding the circumstances under which the technology is qualified Define the boundaries within which the available data adequately justifies deployment in operating environment Potential value outside these boundaries Outline of the goals of the technology and the context of use

in which the technology is intended Data from additional studies obtained over time may be submitted to expand the qualified context of use

Section 3

Lloyds Register 5



n Section 4 TQ Stage 1: Technology Appraisal

This stage describes the principles to derive the list of technology elements through decomposition analysis, a goal-based structured and systematic approach to handle expert opinions elicitation, TQ cost, traceability, hazard identification, risk assessment and the input to the next TQ stage.

4.1.1 System decomposition

4.1.1 System Decomposition PrinciplesA system is defined as a group of interacting elements or technologies having a functional relationship that when grouped or integrated, provide some processes or services. System performance is often as strong as its weakest technology element(s).

The outcome of the system decomposition provides the basis for review and assessment. It defines, but is not limited to, the scope for Technology Qualification review, document system and operating conditions, functions and sub-functions requirements, process sequences and operations, control-logic sequences, and capabilities with respect to qualitative and quantitative claims and goals.

The decomposition methodologies should be selected and agreed upon by the LR TQ team with respect to the complexity of the integration and the development stage of the systems and/or technology elements under qualification. If the system is software-intensive, the selected methodologies are applicable to such systems.

The decomposition process should start with the functions

at the highest level, which is usually at the system-level. The decomposition process stops at components and/or technology elements whose maturity and operating experience is identifiable and assessable and their novelty or uncertainties, including the interfaces and integration technology between them, allow the technology to be assessed by codes or good practice. The decomposition process should also apply to both hardware and software elements.

There are several possible approaches which can be used together or alone. A structure-based breakdown of hardware or architecture-based approach for software-based systems is one avenue. Another method is a function-based approach to sub-divide main functions into sub-functions in order to identify which (sub-) functions are performed by which elements and then to derive the physical and logical location and/or connection with other components/sub-systems/systems.

These approaches are to determine how the functions should be accomplished, the expected performance level by which technology elements need to perform those functions.

4.1.1.1 Approach to System DecompositionSystem engineering principles and processes should be adopted in the decomposition analysis starting from system-level functions and structure, human-machine interaction, roles and responsibilities of organizations and personnel participating in the system development. A goal-based analysis should be applied in the decomposition analysis to identify relevant functions, boundary conditions, interfaces and sub-goals for each identified element. Descriptions of the purpose of use, operating modes, and integration issues for each decomposed element should be documented. A goal-based scheme might be applied to structure the goals, claims, arguments, and evidences for all elements.

High level goals and/or claims of the system or technology elements should be documented together with the context of use in the target operating environment which is identified and confirmed by SMEs in the LR TQ team.

The decomposition can be either full or partial. Full decomposition means that a parent goal has been completely refined and that no more sub-goals will be added to the decomposition; whereas partial decomposition means that more sub-goals may be added in the future.

4.1.2 Critical Technology ElementsA technology element is critical if Its application or the element itself is either new or novel, or in an area that poses major technological, business, environmental or safety risk for each phase of its development

life cycle, such as detailed design, manufacture, installation, commissioning, operation, or decommissioning; and

The LR TQ team will determine if each technology element identified in the decomposition step is a CTE. All CTEs should be identified and qualified. The CTEs are labelled using a

decision matrix, a criticality rating scheme and a criticality

Section 4




Lloyds Register6




RiskAssessment


assessment approach which is fully documented and is approved by LR. The following are documented: How the CTEs are identified Why the CTEs are important (criticality assessment) Maturity assessment of CTEs Novelty, areas of major risks, and adequate performance in the

appropriate environment of CTEs

It is allowable, after review by and where acceptable to LR, for a non-critical technology element to avoid subsequent qualification steps. In that scenario, the assumptions and justifications for the decision of skipping the qualification of those elements and all information related to the skipped elements should be documented and reported, and the elements should also be listed in the goal-based analysis.

4.1.2.1 Measurable criteria for the selection of CTEIt should be ensured that the decision on CTE-related aspects such as selection or ranking choice(s), traceability of recommendations, solutions or opinions is documented and verified by SMEs and re-viewed LR. Documentation should be unambiguous.

For example, available supporting evidence documented in a data-base containing different sets of numerical data and using different datasets of the same database can result in different qualification states/levels. Hence, expert opinions should be gathered and uti-lized as the basis to deciding the most relevant set of data for a specific goal/function used in a specific application in a specific en-vironment.

4.2 Technology AssessmentThe technology assessment considers the technology readiness level and its operating environment, and also requires an integration review for the areas which are recognized as well proven in the same application but are connected to a novel system.

Technology assessment is performed through a structured workshop with participation of SMEs in the systems design, technologies, maintenance and operations, and third party independent technical authorities.

In this stage, the following items are considered: Equipment List Readiness level for each item Operational, reliability and maintainability aspects for each item Integration issues Review of relevant philosophies Report that identifies the technology readiness for each item of

equipment, and overall systems. Established areas to be considered during a risk assessment

4.2.1 Technology Initial ScreeningAll elements identified in the system decomposition step should be screened and assigned an uncertainty rating colour-coded rating of red, green or yellow. Screening of technology elements for

further review to reduce uncertainty are to be based on the technology readiness level and the operating maturity level (see 4.2.2.1). This screening step is to reduce economic cost of qualification through eliminating the identified proven technology elements; hence the qualification resources and scope are to focus only on the unproven elements.

The color-coded category matrix depicted in Fig.2 with respect to the experience level in the target operating environment and the technology readiness level should be used to assign uncertainty level for each element.

Qualification review requirements for each uncertainty level are briefly described in Fig.3

The elements in the red category are highly likely to introduce new technical uncertainties and should be subjected to the high qualification review comprising of the most intensive review requirements in the subsequent qualification steps (see 5.3.1) High qualification review attempts to reduce technical uncertainties and/or bring up the TRL of these elements. Requirements and recommendations about (including but not limited to) testing, risk mitigation, and design improvements aiming at moving those elements to categories of a higher certainty in the same or subsequent technology and/or system development stages should be determined and provided.

Section 4

Figure 2 Initial screening matrix

Figure 3 Qualification review levels

Proven

Limited experience

New

02 35 67

Technology Readiness Level

No new technical uncertainties

Technology Qualification review medium

Technology Qualification review high

Lloyds Register 7



The elements in the yellow category should be subjected to the medium qualification review whose qualification activities and requirements are a subset of the high qualification review. Medium qualification review should include required processes and requirements of risk assessment.

The elements in the green category should be considered as introducing no new technical uncertainties and screened out in order not to go through the remaining qualification process. They should be subjected to review by established codes and standards.

4.2.2 Technology Readiness Level AssessmentThe technology readiness level (TRL) is a function of the Technology Maturity Level (TML) and the Integration Maturity Level (IML)

4.2.2.1 Technology Maturity LevelAgreed by the LR TQ team as acceptable to LR, the ranking of TML of technologies should be chosen to be relevant to the industry in which their applications are specifically required by the regulations. This procedure aligns with TML ranking in API 17N. A comparison between different TML ranking regimes such as NASA is provided in Appendix 3.

TML assessment of any CTE is tied to the intended application through the performance specifications provided. It is possible that the same technology element will be assigned for different TML in different applications or in different operating environments. For example, the TML of a coating technology may be 7 in a non-critical/low spec application, but could be 3 in a critical/high spec application.

Where applicable, the technology maturity level review for an element or set of elements should include the review of and account for the novelty aspects of the integration of the element(s). (See 4.2.2.2)

A scheme and/or a method to adjust TML assigned to an element with respect to different IMLs of the sub-elements should be determined and agreed with the LR TQ team as acceptable to LR. The scheme and method might be qualitative or quantitative or combined by both and should be consistent throughout the TQ review process. Appendix 5 provides an illustration to qualitative and quantitative aspects of the IML-adjusted TML assignment methods.

4.2.2.1.1 Technology Maturity Level Assignment ProcessSupporting tools and models to assist TML calculation might be employed; in that case the calculation steps and methods are to be documented in the technology assessment report and be agreed upon by the LR TQ team as acceptable to LR.

Where data is limited for concluding a calculable TML assignment, expert opinions should be considered and applied in assigning TML for the elements under review.

Expert opinions might be referred to in order to determine the TML for integrated elements whose TML and IML are not identical, in case of limited data available to be used to assess

the TML quantitatively.

It should be noted that the TML of an element at a higher level in the hierarchy should not be higher than the lowest TML among the integrated CTEs of the element if the element and those integrated CTEs are operating in an identical environment (see Appendix 4).

4.2.2.1.2 Software-based Technology ElementsSoftware is continually changing and a new release/modification of software could introduce new faults or break down the architectural integrity of software. Therefore software degradation might alter previously confirmed maturity level. Assessment of TML for software-based elements might adopt a different philosophy from mechanical or non-software-based technologies. The TML of the software could be assessed at any development stage without waiting until the actual code is produced. For example, the maturity level of critical algorithms could be assessed through analysing their pseudo code in addition to their implemented code.

Software-based technology elements should be categorized as either standalone or non-standalone where applicable. The standalone category is to include elements such as operating system and computer-based process control system. The non-standalone category is to include elements that are required to operate on non-PC hardware such as PLC-based software, network, and device drivers. Issues related to the integration of software-based elements are often challenging and costly to resolve at the later development stage. The maturity level of the standalone elements might be assessed through techniques, models, and methods in the existing standards, which are acceptable to LR, governing software development lifecycle in the target operating environment.

A procedure to assess TML of software-based technology elements should be established and be acceptable to LR.

4.2.2.2 Integration Maturity LevelTechnology elements could pose technology challenges and uncertainties when integrated into a system, even if the standalone element(s) is a matured technology. Review of integration issues should be performed for each element identified at the system decomposition step to determine the novelty of the integration.

This review attempts to determine if elements within systems have been previously integrated and to review documented experience or performance of the integration and issues. Information and data used to assess the IML should be provided. For example, process-related integration issues might be reviewed through process flow diagrams (PFDs), while software-related integration issues might be reviewed through data flow diagrams. More attention might be paid to the integration aspects of elements whose qualification outcomes could affect the critical claims and goals.

4.2.3 Technology Operating Maturity LevelAll of the elements identified in the system decomposition step should be assigned to a maturity level of operating in the target operating environment in one of three categories: proven, limited experience and new.

Section 4


Lloyds Register8


A technology element could have the highest TML, but still be new to or have a limited experience in a target environment.

The operating maturity of a technology element is relatively determined by the technologys operating performance. The body of knowledge about the operating performance of technology is therefore confirmable, accessible and verifiable.

The maturity in an operating environment is determined by the time period the technology has been continuously used in the operating environment, and also by the technical uncertainties posed by the technology as perceived by the LR TQ team, SMEs, third party technical independent assessors and/or determined by the specific guidelines/requirements from regulators and end-users.

Where applicable, a consensus-based decision should be applied to determine the operating maturity level in the target operating environment for all elements identified in the system decomposition step.

4.3 Risk AssessmentThe intended scope of this step is to provide the risk management principles and processes determined via examination by SMEs throughout the lifecycle for each of the technology elements identified at System Decomposition (see 4.1.1) and screened in the technology assessment (see 4.2). The elements include CTEs, equipment and systems where novel technology is used. Each technology element is identified and reviewed individually, and recommendations for qualifications or mitigations are identified, recorded and traceable throughout the qualification process. It is an iterative process until all mitigation and qualification activities are satisfactorily identified. It is possible that the list of technology elements and/or their screening assessments identified at earlier qualification steps might be adjusted during the detailed Risk Assessment steps.

The assessment process aims to ensure that all reasonably foreseeable hazards associated with a particular technology are identified and adequately controlled, such as hazards related to novel aspects of system integrity failure, and control/isolation failure. Adequate risk control measures should follow the risk control hierarchy acceptable to LR and/or recommended in respective international standards (e.g. see ISO 17776, ISO/IEC 31010). The adequate measures could be, but are not limited to, such as elimination, prevention, reduction or mitigation, and can be realised in the forms such as physical equipment, process control systems, operating or maintenance procedures for all aspects. Where a technology undergoing qualification is derived from a technology previously qualified by LR or other qualified assessment organization, the risk assessment to be undertaken may refer to the relevant parts of risk assessment resulting from the previous qualification and also include the differences in integration and/or in the application of its use.

Risk Assessment is established to ensure any risks stemming from the technology are addressed, to ensure risks are eliminated

wherever possible and risks which cannot be eliminated are mitigated.

4.3.1 Risk Assessment TeamRisk assessment is not a substitute for good engineering judgment and expertise. In order for risk assessment to be effective the persons involved in the process must have the knowledge and experience in the technology area and/or target operating environment to provide a high quality, robust evaluation of the technology and systems being assessed.

Independent third party expert(s) should be involved in the risk assessment process for high ranking hazards identified at the high-level hazard review, for example HAZID/HAZOP (see 4.3.3).

The risk assessment team should include:a. Key Stakeholder (e.g. owner, operator/end-user, designer/ developer, third party assessor)b. Technical SMEs (pertaining to technology area and target operating environment)c. Regulators or the jurisdictional authorities, if deemed necessaryd. A risk SME should chair the team during the risk assessment

process

4.3.2 Assessment MethodsThis document does not mandate any particular risk assessment method and does not offer acceptance criteria. It defines the principles and processes which LR requires to ensure that the risk assessment is performed to an acceptable standard. The methodology of risk assessment should meet the intent of international, national and industry standards acceptable to LR, such as LR risk assessment guidelines.

The selection of methods and techniques should take into consideration the nature of the CTE, the system and equipment to be reviewed, safety and environmental factors, the risk profile, regulatory requirements and other pertinent variables.

The selected risk assessment technique and risk management procedure should be approved by LR. Acceptable risk assessment approaches should be identified and dependent on engineering complexity, availability of data regarding sub-systems/components integrity and reliability. The selected method should consider complexity and maturity of the technology being assessed. Comprehensive descriptions of the risks should be provided.

ISO/IEC 31010 provides guidance on selection and application of systematic techniques for risk assessment.

ISO 17776 identifies some of the tools and techniques that can be used to identify hazards and risks in the offshore exploration and production industry and provides guidance on how they may be applied to particular activities.

Risk assessments may be built on a foundation of applying the appropriate codes and standards where appropriate.

Section 4

Lloyds Register 9



4.3.3 Hazard IdentificationA prerequisite of risk assessment is hazard identification. A complete list of hazards related to a technology, subcomponent, equipment and system should be identified by means of a structured and systematic approach. HAZID and HAZOP are possible techniques for identification of all significant hazards.

Each identified hazard is reviewed to determine whether it is significant and requires further evaluation by applying hazard evaluation techniques.

It is important to record all identified hazards and the reasons for classifying some as not significant. This ensures traceability as the qualification progresses into later stages. Hazards to personnel, environment, and asset should be identified and potential associated risks should be evaluated. A formal hazards regis-ter should be prepared, detailing each hazard together with appropriate data such as potential cause, potential consequence, and actions for risk control measures.

More information on HAZID should be referred to ISO 17776 Annex B and Annex D.

4.3.3.1 HAZOP - Hazard & Operability studyThe basis of HAZOP is a systematic examination of deviations from expected operational boundary conditions. It is performed by using a series of guidewords and parameters to identify if any of the combinations can occur, and if so the possible causes and consequences. In addition, existing measures to minimize causes and consequences are listed together with any recommendations to eliminate the deviation or improve upon the existing measures.

IEC 61882 might be referenced to as a guide for HAZOP studies of systems, providing guidance on application of the technique and on the HAZOP study procedure, including definition, preparation, examination sessions, resulting documentation and follow-up.

4.3.3.2 SWIFT - Structured What-IF TechniqueSWIFT is similar to HAZOP as it is a structured brainstorming session; it uses a set of questions to stimulate discussion on consequences, safeguards and recommendations. Instead of using the guidewords and parameters in the HAZOP, standard what-if type phrases and a set of prompt words associated with the system are used to facilitate the discussion.

The SWIFT technique is described in ISO 31010.

4.3.3.3 FHA - Functional Hazard AssessmentFunctional Hazard Assessment is a top-down safety assessment technique defined in SAE ARP4761. FHA is a predictive technique that attempts to explore the effects of functional failures of parts of a system. FHA is first carried out for the whole system working from a description of system functions. Then, following allocation of functions to the systems, FHA is performed again for each subsystem.

4.3.3.4 Risk IdentificationIdentification and assessment of the failure modes of the systems

should be performed. The selected technique should be relevant and adequate for this purpose. FMECA, FTA and ETA are possible methods.

4.3.3.5 FMECA - Failure Modes, Effects and Criticality AnalysisThe function of each decomposed element of the system is analysed and for each element, consideration is given to the failures or incorrect performance that may occur. The associated cause of each failure and the corresponding effects of the failures are listed alongside each failure mode.

Each failure mode identified is ranked according to its importance or criticality. There are several ways this may be conducted. Common methods include the measure of the probability that the mode being considered will result in failure of the system as a whole, level of risk by combining the consequences of a failure mode occurring with the probability of failure or a semi-quantitative measure of criticality obtained by multiplying numbers from rating scales (usually between 1 and 10) for consequence of failure, likelihood of failure and ability to detect the problem.

IEC 60812 provides the procedure for the FMECA technique.

4.3.3.6 FTA - Fault Tree AnalysisThe FTA is used to qualitatively identify the potential causes and pathways to a failure. It can also be used quantitatively to calculate the probability of the failure given the probabilities of the causes.

IEC 61025 describes the process of a Fault Tree Analysis and IEC 60300-3-9 gives guidelines on dependability management which is used to develop the fault tree of the system.

4.3.3.7 ETA - Event Tree AnalysisThe ETA is used to illustrate and quantify all possible outcomes from an initiating event by considering what can happen next. The tree is used to map the different ways the initiating event can escalate and the effectiveness of the control measures at each juncture.

IEC 62502 gives guidance on carrying out the ETA technique.

4.3.4 Risk Evaluation and ReportingAn LR-acceptable risk ranking methodology should be determined and carried out at each critical milestone/gate/phase for qualifying and managing the process of the whole Technology Qualification lifecycle.

4.3.4.1 Risk Likelihood & Severity and ClassificationThe selection of method (quantitative vs. qualitative) should take into consideration the complexity and maturity of the technology being assessed. Likelihood and severity ratings should be discussed and agreed upon at the beginning of the risk assessment workshop by all SMEs.

An appropriate risk classification methodology should be agreed upon and established at the beginning of the risk assessment process by all SMEs to assess how new technology has been qualified and managed through the qualification process. A Risk Matrix is one possible method for classifying risks.

Section 4


Lloyds Register10


4.3.4.2 Risk Criteria DevelopmentThe risk acceptance criteria can be either qualitative or quantitative (preferred) and fully agreed upon by the risk assessment team. The acceptance criteria which are used to judge the acceptability of the risks should account for the likelihood and severity of the consequences of each risk and have appropriate safety margins. This can be based on equivalence with current arrangements such as international standards.

Any risk acceptance criterion, be it qualitative or quantitative may be more stringent than what is required by regulations.

The qualification methods and risk control measures such as elimination, prevention, and reduction are identified as basis to support the next development phase, if applicable.

4.3.4.3 Risk assessment reportThe format of risk assessment report may adopt forms as provided in the respective international standards (e.g., see ISO 17776) and/or industry standards relevant to the target operating environment of the technology element(s) (e.g., see API RP 17Q). The report needs to address the following, as a minimum: Justification on selected methodology Scope and objective of assessment Competency of personnel involved Results of risk assessment including risk reduction through mitigation and qualification are recorded and reported correctly for the intended audience Verification and validation procedure for identified risks

4.3.4.4 Review Risk Assessment resultsThe work performed related to risk assessments is documented and submitted to LR.

A comprehensive review of the risk assessment process will be carried out by LR to determine if risks are analysed using acceptable techniques and that the risks are eliminated wherever possible, with adequate risk management measures, such as mitigation plans, in place for risks which are not eliminated.

The results of the risk assessment will identify the critical components or sub-systems which will be reviewed in detail during the design review to have adequate risk control for the hazards and risks identified.

4.3.5 Refinements to the Risk AssessmentThe risk assessment process is iterative and might include reviews and revisions of the design concept and details when applicable. Change(s) to the design of the technology and/or to the intended context of use of the technology may require further assessment.

It is possible that there will be no existing reference for the evaluation criteria for a truly novel design; in this case LR will work with the client to agree on the strategy for defining the evaluation criteria.

During the TQ process, the technology might have an increasing number of details and further engineering analyses used to verify

the designs proposed functionality and operation. In this case, the risk assessment might need to be revisited.

The reassessment of the risks should be carried out after each revision to the design until it is agreed that the risks are controlled to satisfactory levels.

n Section 5 TQ Stage 2: Technology Qualification Plan and Execution

This stage contains the process related the TQ plan and execution of that TQ plan.

5.1 TQ Plan

5.1.1 PurposeA TQ Plan contains the details of the activities that are developed through an iterative process and in order to be implemented to provide the evidence needed to support goals and claims and to manage the critical failure modes. Both the risk assessment and TQ plan will be reviewed by LR and may be revised by clients as the project moves forward to the next development phase.

5.1.2 Technology ScopeThe following aspects of the technology should be included in the scope of the TQ plan with sufficiently accurate details for LR review: All relevant functions, performance requirements and acceptance criteria Life-cycle considerations Interfaces and integration between different technology elements, between human and machines, and between new technology and existing systems Selected and specified qualification methods The technology development process together with major milestones.

Sections 4 & 5





RiskAssessment

Lloyds Register 11




Where necessary, expert-based recommendations, assumptions and actions should be recorded for TQ Plan revision, traceability and follow-up.

Section 5

Figure 2 Qualification Plan (see 4.1-4.5)

5.1.3 Qualification Methods, Activities and Tests

5.1.3.1 Qualification Methods and ActivitiesQualification methods for each identified element or identified risk are selected and specified in the Technology Qualification Plan. Activities for each qualification method are to be specified in the plan. These activities should provide evidence of document compliance with the requirements of the System Decomposition and Technology Assessment steps.

In specifying qualification activities, the following aspects should be considered, but not be limited, to reduce the uncertainty: Analysis to assess the performance of individual elements or the system evaluated. Detailed risk analyses where applicable Modelling and numerical simulation might be used to predict

the potential performance of the novel technology. Prototyping can be used to provide confidence that the technology can function according to requirements Manufacturability Material (destructive or non-destructive) testing Function testing to demonstrate the performance of the technology Proof testing which will demonstrate the limits of the technology Audits (e.g. personnel, manufacturing facilities, software,)

In the selection of the qualification activities the TRL should be considered (e.g. Technology Qualification review high is expected to result in more detailed qualification activities)

Tests are an essential part of most qualifications and aim to provide experiential evidence on which qualification can be based on. Elements assigned as high at the technology screening step (see 4.2.1) should have qualification test plans in place as part of the qualification activities.

The test plans should be linked to the failure mechanisms identified during the risk assessments and test procedures and acceptance criteria should be approved by LR. The procedures should show how the functionality is to be achieved by the technology. It should also be possible to verify a staged process of integrating the subcomponents and subsystems of the entire system to be qualified. LR might survey the tests with experts within the test areas as deemed required and witness the test activities for the high items.

The types and extent of the tests required depend on the technology type, confidence in analyses and the extent of experience with similar concepts. The objectives of the tests are to verify function and reliability, as well as to obtain data for further analyses and verify the results obtained from analytical methods. Operational data might also be considered.

Test plans should explain the validation of the performance of CTE materials and systems. Details of laboratories performing tests on the materials or systems should be supplied. The laboratories will either be accredited to ISO 17025 for the required tests, or be subject to an audit by LR of the systems as applied to the specific tests.

Test could be undertaken to confirm whether the assumptions used during the risk assessments are valid and could be used to identi-ty safety margins compared to the limits defined in the designs. Expectation of test results is also given.

Confidence in the experimental evidence should be ensured by specifying an independent review of test specifications, including instrumentation and quality assurance of the test activities, and independent witnessing of tests as appropriate.

Qualification Method,

Activities & TestsSee 4.3

Acceptance CriteriaSee 4.4

Remarks / Recommendations

See 4.4

Initial Screening Level 2 items Require Test Plan & Review

See 4.3.2

ISSUE STATEMENT OF ENDORSEMENT

OF TQPSee 4.6


Lloyds Register12

Technology QualificationSection 5

5.1.3.2 Use of certified componentsIn establishing the TQ plan activities recognition can be given to components which are certified to LR rules (e.g. using Type Approval Process) or other recognized codes or standards provid-ed the terms of reference of the certification are appropriate to the goals of technology and the integration has been assessed with the Technology Qualification process.

5.1.4 Acceptance Criteria, Failure and RecommendationsAcceptance criteria associated with each qualification activity and requirements are identified, agreed upon and recorded. The qualification activities need to be traceable back to the failure mode identified during risk assessment. The reasoning relating the pieces of evidence to the failure modes is to be recorded.

Expert-based recommendations and assumptions during risk assessment workshops should be taken into account to derive acceptance criteria and verification plan.

Where applicable, the use of quantitative performance measures and evidence such as confidence levels or process capabilities (for an example, statistical capability measures) should be recommended.

5.1.5 Content of TQ PlanThe TQ Plan should contain the following sections:1.Requirements definition Initial screening results and list of modes to be qualified Functional behaviour and performance claims/goals

2.Qualification activities Mitigation/Qualification Methods and Activities Responsible party TQ Plan describing clear roles and responsibility of stakeholders for each qualification activity. Acceptance criteria Testing procedures and plans

3.Planning and Execution Organization chart Communication and reporting protocol Where CTEs are developed, tested or integrated, and reviewed by different entities, a communication plan among the entities should be agreed upon and established and documented in the TQ Plan. Cost and schedule Qualification cost needed for each technology element and the whole system might be estimated and included into the TQ Plan. Qualification costs will not be reviewed by LR and as such can be excluded in any submission to LR. Qualification schedule should be part of the TQ plan and coordinated with the system development master project plan, and where applicable the schedule should consider performing parallel qualifications. Milestone/checkpoints for review Management of change process Evidence documentation

5.1.6 TQ Plan Review and EndorsementLR will issue a Statement of Endorsement for the TQ Plan upon satisfactory assessment and review if LR has not prepared the TQ plan. It is recommended that this is issued prior to execution of the TQ Plan.

5.2 ExecutionQualification Execution should strictly follow the endorsed TQ Plan. The results of each activity are documented and traceable to the endorsed plan for that element. Unanticipated failure modes and margins of performance are also recorded.

Deviation of TQ execution from the endorsed TQ plan should follow an approved management change of process and be reviewed, justified and not undermine the quality and accuracy of evidence to be collected. The TQ plan should then be updated accordingly.

Figure 3 Qualification Execution (see 5.1)

ExecutionSee 5

Review ResultSee 5.3

Report & Follow UpSee 5.1.2

ISSUE CERTIFICATE OF TECHNOLOGY QUALIFICATION

See 4.6

Manufacturing Audit

See 5.2

Lloyds Register 13



n Section 6 TQ Stage 3: Performance Review and Certification

This stage contains the process related the performance review and describes what certification might be issued..

6.1 Performance Review

6.1.1 Performance Review and AcceptanceAll documented outcomes from performing qualification activities, and testing are be reviewed, analysed, and interpreted by LR and the client. Where applicable, the analysis by the use of quantitative performance measures and evidence should be recommended. The evidence collected from these activities are used to support the performance requirements and reduce the identified risks previously assessed in the qualification development process.

Arguments are derived upon the analyzed and interpreted results in order to support qualification claims and goals.

The review process confirms that all required activities are completed, recommendations and assumptions have been addressed, and deviations are identified.

Evidence and arguments to support claims, goals, and acceptance criteria are reviewed for approval by LR.

6.1.2 Unsuccessful TQ activitiesIf during a qualification activity a failure occurs that was not expected, then the cause and mechanism of the failure needs to be identified. The method for identifying the cause and mechanism of the failure should be a recognized approach such as RCA. This might then require the TQ plan to be revisited or elements of the TQ process to be reanalyzed.

6.1.3 TQ documentationThe TQ documentation should include and/or refer all outcomes from all steps in the TQ process.

The TQ documentation should contain all the evidence and reports from the analysis, testing and performance review and recommendation for the next phase of technology development. Routine reports and results from the milestone or checkpoint reviews should also be included. Where applicable, LR will approve the completion of qualification goals where reasonably supported by evidence and arguments that have been recorded in the TQ documentation. 6.2 Certificate of Technology QualificationUpon the satisfactory review and assessment of manufacturing audit(s) and the final TQ documentation confirming that all acceptance criteria are met and qualification goals and claims are sufficiently supported, Lloyds Register may issue a Certificate of Technology Qualification for the whole system or technology under qualification. Description of performance for the target operating environment may be referred to in the Certificate. Qualified states and performance associated with parameters in the technology elements are recorded, managed and traceable to configuration management when the technologies are deployed.

Section 6





RiskAssessment


Lloyds Register14


Appendix

1 Existing Guidance on Technology Qualification

2 Technology Qualification Roadmap

3 Technology Maturity Level

4 Integration Maturity Level

5 IML Adjusted TML Assignment Methods

Technology QualificationAppendix


Lloyds Register 15


n Appendix 1 Existing Guidance on Technology Qualification (TQ)

A number of recognized standards and practices have been published with regard to Technology Qualifications. The subsections below provide a general overview of some of those used most prominently throughout industry, including:

ISO 16290:2013 Definition of the Technology Readiness Levels (TMLs) and their criteria of assessment API RP 17N Recommended Practice for Subsea Production

System Reliability and Technical Risk Management API RP 17Q Subsea Equipment Qualification-Standardized

Process for Documentation IEC 61508 Functional safety of electrical/electronic/ programmable electronic safety-related systems (all parts ) IEC 60300-9 Dependability management - Part 3: Application guide BS 5760 part 5 - Reliability of systems, equipment and components. Guide to failure modes, effects and criticality

analysis (FMEA and FMECA) IEC 60812 Analysis techniques for system reliability - Procedure for failure mode and effects analysis (FMEA) ISO 17776 Petroleum and natural gas industries - Offshore

production installations Guidelines on tools and techniques for hazard identification and risk assessment

ISO/IEC 31010 Risk management Risk assessment techniques IEC 61882 Hazard and operability studies (HAZOP studies) - Application guide IEC 61025 Fault Tree Analysis (FTA) IEC 62502 Analysis techniques for dependability - Event tree

analysis (ETA) SAE ARP 4761 Guidelines and Methods for Conducting

the Safety Assessment Process on Civil Airborne Systems and Equipment

INCOSE SE Handbook Version 3.2.2 ISO 20815 Petroleum, petrochemical and natural gas industries Production assurance and reliability management BS 5760 Part 2 - Reliability of systems, equipment and components. Guide to the assessment of reliability ISO 17065 Conformity assessment - Requirements for bodies

certifying products, processes and services LR Type Approval Test Specification Number 1, 2, 3, 4, and

GT04 Jul-2013 IEC 61511 Functional safety Safety instrumented systems

for the process industry sector IEC 62061 Safety of machinery: Functional safety of electrical, electronic and programmable electronic control systems ISO 11161:2007 Safety of machinery - Integrated manufacturing systems Basic requirements ISO/IEC 15288:2008 Systems and software engineering System life cycle processes

ISO/IEC 15026-4:2012 Systems and software engineering - Systems and software assurance - Part 4: Assurance in the life cycle ISO 12100:2010 Safety of machinery - General principles for

design risk assessment and risk reduction ISO 9001:2008 Quality management systems Requirements IEC 60721-2-1 Classification of environmental conditions -

Part 2: Environmental conditions appearing in nature Temperature and humidity ISO 2394:1998 - General principles on reliability for structures IEC 62023:2000 - Structuring of technical information and

documentation IEC 61082-1:1991 - Preparation of documents used in electrotechnology Part 1: General requirements IEC 60204-1 Safety of machinery Electrical equipment of

machines Part 1: General requirements IEC 60445:1999 Basic and safety principles for man-machine

interface, marking and identification Identification of equipment terminals and of terminations of certain designated

conductors, including general rules for an alphanumeric system IEC 60034 Rotating electrical machines (all parts) IEC 61000-6-1 Electromagnetic compatibility (EMC) Part 6-1: Generic standards Immunity for residential, commercial and light industrial environments IEC 61000-6-2 - Electromagnetic compatibility (EMC) Part 6-2: Generic standards Immunity for industrial environments IEC 61000-6-4, Electromagnetic compatibility (EMC) Part 6-4: Generic standards Emission standard for industrial

environments ISO 13849-1:2006 Safety of machinery Safety-related parts

of control systems Part 1: General principles for design ISO 13849-2:2003 Safety of machinery Safety-related parts

of control systems Part 2: Validation ISO 13850:2006 Safety of machinery Emergency stop Principles for design ISO 6336-2 - Calculation of load capacity of spur and helical

gears Part 2: Calculation of surface durability (pitting) ISO 6336-3:2006 Calculation of load capacity of spur and

helical gears Part 3: Calculation of tooth bending strength ISO 76:2006 Rolling bearings Static load ratings ISO 281 Rolling bearings Dynamic load ratings and rating life IEC 61024-1 Protection of structures against lightning Part 1: General principles IEC/TR2 61662 Assessment of the risk of damage due to

lightning ISO/IEC 17025 General requirements for the competence of

testing and calibration laboratories

Appendix 1


Lloyds Register16


n Appendix 2 Technology Qualification Roadmap

Start

Technology Appraisal (Section 4)


System Structure Decomposition System Function Decomposition Claims/Goals Critical Technology Elements

Technology Assessment

Technology Readiness Level Operating maturity Functional criticality Integration Readiness Level Assessment Initial Screening level

Risk Assessment

Assessment Team Assessment Methods & Acceptance Criteria Hazard Identification Failure Mode, Mechanism, Effect Severity, Likelihood, Risk Criticality Concept, Design Reviews

Technology Qualification Plan and Execution (Section 5)

Qualification plan

Qualification Method, Activities & Tests Acceptance Criteria Remarks/Recommendations

Issue statement of Endorsement of Technology Qualification Plan

Technology Qualification, Performance Review and Certification (Section 6)

Qualification execution

Execution Review Result Report & follow up Manufacturing Audit

Issue Certificate of Technology Qualification

End

Appendix 2


Lloyds Register 17


n Appendix 3 Technology Maturity Level

The API 17N technology maturity model is adopted for use in this procedure. The table below defines each stage of the TML and shows a comparison between the API and ISO 16290/NASA TML definition.

Appendix 3

API TML

Development Stage Definition of Development Stage NASA TML

Co

nce

pt

0Unproven concept (Basic R&D, concept on paper)

Basic scientific/engineering principles observed and reported; concept on paper; no analysis or testing completed no design history. 1

1

Proven concept (Proof of concept as a paper study or R&D experiments)

a) Technology concept and/or application formulated b) Concept and functionality proven by analysis or reference to features common with/to existing technology c) No design history; essentially a paper study not involving physical models but may include R&D experimentation

2

2

Validated concept (Experimental proof of concept using physical model tests)

Meets all requirements of TML 1; concept design or novel features of design is validated by a physical model, a system mock up or dummy and functionally tested in a laboratory environment; no design history; no environmental tests; materials testing and reliability testing is performed on key parts or components in a testing laboratory prior to prototype construction

3

3

Prototype tested (System function, performance and reliability tested)

Meets all requirements of TML 2.a) Item prototype is built and put through (generic) functional and performance tests; reliability tests are performed including: reliability growth tests, accelerated life tests and robust design development test programme in relevant laboratory testing environments; test are carried out without integration into a broader systemb) The extent to which application requirements are met are assessed and potential benefits and risks are demonstrated

4

4Environment tested (Pre-production system environment tested)

Meets all Requirements of TML 3; designed and built as production unit (or full scale prototype) and put through its qualification programme in simulated environment (e.g. hyperbaric chamber to simulate pressure) or actual intended environment (e.g. subsea environment) but not installed or operating; reliability testing limited to demonstrating that prototype function and performance criteria can be met in the intended operating condition and external environment

5

5System tested (Production system interface tested)

Meets all the requirements of TML 4; designed and built as production unit (or full scale prototype) and integrated into intended operating system with full interface and functional test but outside the intended field environment

6

6System Installed (Production system installed and tested)

Meets all the requirements of TML 5; production unit (or full scale prototype) built and integrated into intended operating system; full interface and function test program performed in the intended (or closely simulated) environment and operated for less than three years; at TML 6 new technology equipment might require additional support for the first 12 to 18 months

7

7Field proven (Production system field proven)

Meets all requirements of TML 6; production unit integrated into the intended operating system, installed and operating for more than three years with acceptable reliability, demonstrating low risk of early life failures in the field

8&9

Table 1: API 17N TML table adopted by the Industry Technology Facilitator (ITF) with comparison to NASA TML


Lloyds Register18


n Appendix 4 Integration Maturity Level

An integration specific matrix is used to determine the level of maturity of integration between two or more subsystems, equipment, component or elements. The Integration Maturity Levels are shown below. (Ref: Gove, R., B. Sauser, and J. Ramirez-Marquez. 2007. Integration Maturity Metrics: Development of an Integration Readiness Level. In Stevens Institute of Technology, School of Systems and Enterprises. Hoboken, NJ)

Table 2 Integration readiness level (Gove 2007; Gove, Sauser, and Ramirez-Marquez 2007)

n Appendix 5 IML adjusted TML Assignment Methods

1. Start with identified systems, items of equipment, subsystems and components per the hierarchical product breakdown using structure-based and/or architecture-based and/or function-based approach, where applicable.

2. Assign TML basis to all CTEs and lowest non-CTE elements in the hierarchy based on the TML ranking

scheme agreed upon by the TQ team and acceptable to LR.

3. Assign TML to elements taking into account IML and the state of integration of their components/sub-systems at a lower level in the hierarchy. An IML rating scheme might be used to measure the relative state of integration. Repeat the

process until all identified elements are assigned with a TML.

4. Identify all components, subsystems and systems that are at a lower TML than required by the program (final product level). This is to provide visibility to the qualification team the areas for

improvement or mitigation in next technology development stages.

5. Provide a baseline technological maturity assessment and technology readiness assessment report.

Appendix 4 & 5

IML Definition

1An interface (i.e. physical connection) between technologies has been identified with sufficient detail to allow characterization of the relationship

2There is some level of specificity to characterize the interaction (i.e. ability to influence between technologies through their interface)

3There is compatibility (i.e. common language) between technologies to orderly and efficiently integrate and interact

4There is sufficient detail in the quality and assurance of the integration between technologies

5There is sufficient control between technologies necessary to establish, manage, and terminate the integration

6The integrating technologies can accept, translate and structure information for its intended application

7The integration of technologies has been verified and validated with sufficient detail to be actionable

8Actual integration completed and mission qualified through test and demonstration, in the system environment

9Integration is mission proven through successful mission operations

Breakdown system into element list

Assign TRL to all CTEs and lowest

non-CTE elements

Assign IRL-adjusted TRL to elements

Identify elements that fall below required TRL

Produce baseline report

Start

End


Lloyds Register 19


It is acceptable to note that the TML of an element at a higher level in the hierarchy should not be higher than the lowest TML among its integrated critical sub-elements if the element and those integrated sub-elements are operating in an identical environment; see below. In that scenario, it is adequately possible that the TML of element X might be assigned as the lowest TML among integrated sub-elements of X (element A, B, C).

Expert opinions might be referred to determine the TML for an element of the integrated sub-elements whose TML and IML are not identical, see below. The opinions and justifications are to be documented.

A quantitative approach to IML-adjusted TML, using the scenario above as example, is to:1) assign TML of element Y following the agreed TML framework

(see Appendix 2) to form the TML basis. It is reasonable to assign the basic TML of element Y to be the lowest TML among

Ys integrated critical sub-elements, 2) adjust the basis TML to be the round-up of the lowest value

among all integrated sub-elements using the mathematical expression

Adjusted TML = Min (TML x IMLsub-element / max IML)

where the max IML is the highest value in the IML rating scheme (see Appendix 3).

For the scenario in Fig.9, the IML-adjusted TML should be the round-up value of the basis TML x minimum of {5/9, 9/9, 3/9} = basis TML x 0.33. If TML basis is selected as 5, the IML-adjusted TML should be 2.

Appendix 5

Element XTML lowest TML among A, B, C = 4

Integrated sub-elements with the same IML

Element ATML = 4IML = 6

Element BTML = 5IML = 6

Element CTML = 6IML = 6

Element YTML: adjusting with the respect to TML & IML of E, F, G

Integrated sub-elements with different IMLs

Element ETML = 6IML = 5

Element FTML = 5IML = 9

Element GTML = 7IML = 3


Lloyds Register20

Lloyds Register Group Limited 2014Published by Lloyds Register Group Limited

Registered office (Reg. no. 08126909)71 Fenchurch Street, London, EC3M

Working togetherfor a safer world

Lloyds Register EMEA71 Fenchurch StreetLondon EC3M 4BS, UK

T +44 (0)20 7709 9166F +44 (0)20 7488 4796

www.lr.org

tq final digital 4

Documents

lloyds registerlloyds

relevant lloyds

addresse lloyds

different chapter

technology qualification

guidance notesand

existing guidance

technology appraisal