aeo self-assessment checklist for

T MU MD 00009 F2 AEO Self-Assessment Checklist Template

Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7

AEO Self-Assessment Checklist for <Insert Organisation>

T MU MD 00009 F2

1. Instructions – Self-assessment checklist 1.1. Who fills it in?

The individual(s) within the prospective AEO who are preparing for the authorisation assessment

The individual(s) completing the self-assessment should call on SME support internally to support them

1.2. When is it used? It is completed after the scoping meeting with the ASA facilitator and as part of evidence gathering in preparation to begin the formal assessment process

It is to be used by the applicant to gauge their organisation's readiness for the full formal assessment as an AEO

1.3. What documents accompany it? • T MU MD 00009 ST AEO Authorisation Requirements (which includes guidance on applicant interpretation against mandatory requirements)

• T MU MD 00009 F3 AEO Self-Assessment Tool Template

• Applicant organisations engineering management process documentation

• Applicant organisations evidence records of deployment of engineering management processes

© State of NSW through Transport for NSW of 29


Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7

1.4. Explanation on completing key fields • read these instructions and the maturity levels set out in Table 1

• read each requirement carefully, as it reflects the key requirements to become an AEO

• use the completed Engineering Services Scoping Matrix for the organisation to provide the context for self-assessment

• identify evidence for system documentation and deployment for your engineering management function which supports each requirement and list it in the

"Evidence available" column

• insert comments supporting the context of your evidence in the "Comments supporting the available evidence:" row for each requirement

• for evidence that documented arrangements and processes are in place, enter a maturity rating (0 – 5) in the "Documented system" column. This number

indicates a self-assessment of the organisations confidence level in its own documented processes

• for evidence that the documented arrangements are deployed, enter a maturity rating (0 – 5) in the "Deployed System" column. This number indicates a self-

assessment of the organisations confidence level in the deployment of its own documented processes

• insert comments supporting the context of your rating in the "Comments supporting the rating:" row for each requirement

Table 1 - Levels of maturity and description of rating levels

Rating level Rating level description

0 Our organisation does not currently have or do this.

1 This is created or deployed unpredictably, with poor control and is reactive.

2 This is created or deployed on a project or service basis and is often reactive

3 This is created or deployed at the organisational level and is tailored for projects or services accordingly.

4 This is created or deployed at the organisational level and is actively measured and controlled within the organisation.

5 This is created or deployed at the organisational level with a focus on process improvement.

N/A Not applicable



AEO Self-Assessment Checklist for <Insert Organisation>

Item No

Requirements (For guidance and rationale see "TS 10502 AEO Authorisation Requirements)

Evidence available (Document title / number)

Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

8 Engineering management

8.1 General engineering management

ENM1 An AEO shall have arrangements for executing engineering services in a competent and systematic way. An AEO shall have engineering management methodologies appropriate to its engineering services

Comments supporting the available evidence:

Comments supporting the rating:

ENM2 An AEO shall ensure that its engineering services map to relevant phases of the TfNSW asset life cycle model




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

ENM3 An AEO shall establish and maintain an engineering assurance process that aligns to the range of engineering services and activities that the organisation intends to provide



ENM4 An AEO shall demonstrate engineering assurance based on progressive stage gateway reviews



ENM5 AEOs shall apply a risk-based approach to engineering assurance




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

ENM6 Design AEOs shall have capability to provide design support during construction, inspection, test and commissioning stages



ENM7 Design AEOs shall have 'safety in design' as part of producing engineering designs



ENM8 An AEO shall incorporate sustainability in design as relevant to the scope of the authorised engineering services




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

ENM9 An AEO shall consider environmental impact as relevant to the scope of the authorised engineering services



8.2 Asset management

ENM10 An AEO shall have management arrangements in place that align with the requirements of ISO 55001: 2014 Asset management - Management systems – Requirements, or PAS 55-1:2008 Asset Management, as applicable to the scope of engineering services provided




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

8.3 Standards management

ENM11 An AEO shall establish and maintain a repository of the latest relevant engineering standards, guidance, codes of practice, and similar documents that are applicable to the engineering service areas and disciplines for which it seeks and obtains authorisation. Standards relevant to TfNSW include:

• TfNSW standards (including legacy RailCorp standards) • Australian standards (referred to indirectly in TfNSW standards, or in their own right) • international standards (ISO, IEC, EN, and similar) • other standards that may be appropriate to the transport industry



8.4 Judgment of significance

ENM12 An AEO shall establish arrangements for assessing the significance of proposed engineering changes arising from the delivery of its engineering services




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

ENM13 An AEO shall ensure that a competent person has accountability for assessing and approving the engineering change



9 Engineering management

9.1 General systems engineering management

SEM1 An AEO shall have a systems engineering approach to the planning and delivery of its engineering services or products




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

9.2 Requirements management

SEM2 An AEO shall have requirements management arrangements that set out process, responsibilities, structure, tools and deliverables for management of stakeholder requirements applicable to the scope of engineering services provided across the system life cycle



SEM3 An AEO shall establish and maintain a requirements management tool/system that is capable of managing the categorisation, allocation, changes, traceability and verification of all requirements within its scope of control




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM4 The requirements management tool/system shall be able to easily exchange all requirements information using a common interchange format with other organisations



9.3 Interface management

SEM5 An AEO shall have interface management arrangements that set out the process, responsibilities, structure, tools and deliverables




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM6 An AEO shall ensure that all interface requirements under the control of its engineering services are identified, captured and managed



SEM7 An AEO shall ensure that interface design reviews and checks are conducted at appropriate stages of the design process by competent subject matter experts



SEM8 An AEO shall identify and manage interface risks and outcomes that may have a safety or other undesired impact




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM9 An AEO that intends to offer rail systems integration services shall demonstrate that it has suitable management arrangements to plan and carry out the integration of all the declared systems



9.4 System architecture management

SEM10 A design AEO shall demonstrate that it has arrangements to manage the synthesis and development of system level requirements into a credible system architecture



9.5 RAM management


Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM11 An AEO shall demonstrate that it has RAM management arrangements in place, relevant to the engineering services or products provided

•



9.6 Verification and validation

SEM12 An AEO shall have arrangements for verification and validation management of the engineering services or products provided



9.7 Human factors integration


Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM13 An Authorised Engineering Organisation shall manage all human factors relevant to the scope of the authorised engineering services



SEM14 Human factors integration tasks shall be done by a competent resource. The competency requirements of the resource shall be determined by the needs of the specific project




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

9.8 Electromagnetic compatibility

SEM15 An AEO engaged by TfNSW to undertake engineering activities involving the specification, design, integration or testing of electrical or electronic systems involving electromagnetic interference emitters (threats) or receivers (victims) shall have arrangements for managing electromagnetic compatibility



SEM16 An AEO engaged by TfNSW to undertake engineering activities involving the specification, design, build, integration or modification of electrically conductive or magnetically permeative structures shall ensure that arrangements are in place for managing electromagnetic interference and electromagnetic compatibility



Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

9.9 Systems safety assurance

SEM17 The AEO shall have system safety assurance arrangements in place, relevant to the engineering services or products provided. These arrangements must include suitable planning activities and deliverables. They must also demonstrate suitable and sufficient integration into the engineering services



SEM18 The AEO's safety assurance arrangements must provide progressive assurance through the project or system life cycle




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM19 The AEO shall have arrangements for the identification and management of safety risks associated with the changes to be introduced. This process should follow a life cycle approach such that the granularity of risks and the level of analysis aligns with the progression through the engineering life cycle. It shall also support risk-based decision-making with records to show traceability of all decisions made



SEM20 The AEO shall have arrangements for delivering safety assurance arguments and supporting evidence (or input to such documentation) that describes how it has ensured safety SFAIRP and managed safety risks to tolerable and SFAIRP. The content of such documents should be aligned with the requirements of the System Safety Standard TS20001 so that they meet the requirements of the configuration management committee process




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

SEM21 The AEO shall have its safety engineering and assurance arrangements subject to independent safety assessment (ISA), where it is responsible for the introduction of new or novel systems that affect on the operational safety of the network, or where the general scope and complexity of the project requires it. There shall be arrangements in place to support the appointment of an ISA and to engage with an ISA at all stages of the engineering activities being undertaken



10 Configuration management

CFM1 An AEO shall have a documented approach, including identifiable roles and responsibilities for the management of all proposed or existing configuration items under its control




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CFM2 An AEO shall have arrangements to identify and manage all the configuration items under its control



CFM3 An AEO shall have configuration change control arrangements that are appropriate for the engineering services offered



CFM4 An AEO shall have arrangements for identifying stakeholders and managing their engagement during changes




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CFM5 An AEO shall have appropriate tools for managing the change control process



CFM6 An AEO shall have arrangements in place for assuring that configuration items match documentation



CFM7 An AEO shall have arrangements to manage and appropriately deliver the necessary configuration information about configuration items under its control




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CFM8 An AEO shall have suitable tools for managing configuration information under its control



11 Competence management

CPM1 An AEO shall demonstrate that it employs people who have the necessary knowledge, skills, and attitude to competently and safely discharge their duties in providing engineering services




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM2 An AEO shall have comprehensive arrangements and systems for managing the competence of its staff relevant to the engineering services provided. A system for managing staff competence must consist of an unambiguous series of documented arrangements of the organisations roles, plans, processes, tools, and records in relation to managing competence



CPM3 An AEO shall define the skills and proficiency levels that are relevant to the tasks and activities used in the provision of engineering services to TfNSW




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM4 An AEO shall set minimum acceptable competence requirements for posts carrying out ‘key roles’ identified in the organisation



CPM5 An AEO shall identify and assess its engineering work activities and associated risks to determine those that could affect safe railway operations, or that may affect occupational health and safety of staff, passengers or the general public



CPM6 An AEO shall provide a job description containing roles, function, and abilities before starting assessment of a candidate




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM7 An AEO shall consider relevant external qualification standards to benchmark the skills to be assessed



CPM8 An AEO shall maintain evidence that relevant industry competence requirements, including TfNSW standards, have been analysed and interpreted for the appropriate engineering services offered



CPM9 An AEO shall have arrangements in place to train, develop, and assess the competence of staff using established methods and competence standards




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM10 An AEO shall provide for the planning, implementation, recording, assessment and recognition of relevant continuing professional development activities to enhance the knowledge and skills of staff and the organisation as a whole



CPM11 An AEO shall establish training and development needs and competence assessment requirements for staff that will deliver the engineering services



CPM12 An AEO shall establish timescales for periodic re-assessment of staff competence




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM13 An AEO shall maintain competence management records that contain appropriate and up-to-date information about all competence aspects of a candidate



CPM14 An AEO shall maintain evidence for audits, of periodic reviews of it competence records for its engineering staff and other engineering-related service provision staff



CPM15 An AEO shall establish and maintain a register of all engineering and other engineering-related services provision staff and their competences




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM16 An AEO shall keep all engineering staff competence records in a secure location for the duration of the AEO certification validity period



CPM17 An AEO shall maintain the competence of those managers and assessors executing the competence management system, and ensure that the managers and assessors understand their responsibilities



CPM18 An AEO shall define competence management roles and responsibilities for its engineering-related services




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

CPM19 An AEO shall assign ownership and responsibility of competence management for its engineering-related services to suitably qualified and experienced individuals



CPM20 Assessors shall have relevant subject matter expertise, qualifications and assessment experience in the relevant engineering or related services being assessed



12 Quality management

QAM1 An AEO shall establish and maintain a management system, which complies with the requirements of ISO 9001




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7


Item No



Rating level 1 to 5

Doc

umen

ted

syst

em

Dep

loye

d sy

stem

12.1 Knowledge management

QAM2 An AEO shall demonstrate the development and maintenance of knowledge management networks and the sharing of industry-relevant lessons learned and experience with the ASA



12.2 Record keeping

QAM3 An AEO shall establish and maintain as a minimum, records of the following: • competence management records, for example, assessments, development, logbooks, and CVs • engineering assurance records applicable to the authorised services, for example, construction,

manufacturing, integration, testing, commissioning, maintenance and disposal • configuration data records, for example as built drawings, manuals and training materials in a common

format that can be exchanged with other parties • quality assurance records, for example, inspections, audits, and reports • safety assurance records, for example, hazard logs, safety plans and safety reports




Sup

erse

ded

by T

MU

MD

000

09 F

2 v2

.0, 0

8/05

/201

7

aeo self-assessment checklist for

Documents