effective 9101 aerospace auditing breakout session · purpose •to provide audit conduct criteria,...

Effective 9101 Aerospace Auditing

Breakout Session

Dale Gordon, ANAB

Tim Lee, The Boeing Company

Registration Management Committee – July, 2019 - Torrance, CA 1

Purpose

• To provide audit conduct criteria, guidance, tips, recommendations, and discuss how to improve Industry Controlled Other Party (ICOP) audits.

• Focus on audit conduct and not just 9101 report writing

• Effective auditing versus just recording evidence found

• Adding value to the Client, the IAQG, and all stakeholders that utilize audit results recognize the ICOP certification scheme.


Presenters

• Dale Gordon– AS&D Advisor


• Tim Lee– Supplier Quality Manager, The

Boeing Company

Agenda

• 9101 Audit Criteria

• Audit Elements

• Sufficient objective evidence to support Audit conclusion

• Evaluation of process effectiveness

• Auditor competence

• Tips for adding value

• Lessons learned from the audit trail

• Resources


AS9101E Audit Criteria• 9101:2016 Audit Criteria

– The audit team shall pursue relevant audit trails

» Gather sufficient information to support conclusions

– Surveillance & Recertification audits shall include review of:

» changes to the QMS

» requirements from new customers

» customer satisfaction information

» requested corrective actions and associated responses;

– They shall also include:

» an interview with top management;

» audit of the organization's processes, including their performance and

effectiveness, as identified in the audit plan

» audit of the continual improvement of the QMS;

» follow-up actions from previous audits

» the purchasing process



Audit Conclusions

4.2.2.6

Closing Meeting

4.2.2.6

• 9101:2016 Audit Elements

– In addition to ISO 17021-1,

Section 9.4 “Conducting Audits”

Opening Meeting

4.2.2.2

Site Tour

4.2.2.3

Audit Conduct

4.2.2.4

Special Processes

4.2.2.4.1

Audit Findings

4.2.2.5

Process Results

4.2.2.5.1

Process

Realization

4.2.2.5.2

Process

Effectiveness

4.2.2.5.3

Sufficient Objective Evidence


Audit - “Process for obtaining audit evidence and evaluating it objectively to determine the extent to which

the audit criteria are fulfilled”

Audit Evidence - Collect audit evidence in the places where the activities are taking place

• Real time

• Samples of all shifts

• From representative sites and processes

• Provides enough detail to create a clear link from the audit report to a specific issue

• Builds the foundation for audit process credibility

Lessons Learned:

• Lack of sampling:

– There may not be more than a sample of one item for a small organization. But that’s usually the

exception rather than the rule.

– The amount of sampling may be based more on the time allotted than the volume of activity the

process executes.

– Sampling three purchase orders issued in an organization that issues 10 per week on average may

seem sufficient but how about an organization that issues 30 per day? Is only looking at three still

adequate?

– The same holds true for manufacturing processes: How many product lines, and how many

operations? While we use employees as a gauge for time allocation, we need to look at process

volume for audit time application, and we don’t always see that happening.

Evaluating Process effectiveness


• Evaluate the effectiveness of each audited operational process

utilizing the data recorded on each PEAR

– process realization - the extent to which planned activities are realized

– process results - the extent to which planned results are achieved

• Items to consider:

– Process nonconformances

– KPI’s not being met

– Actions taken – Appropriate?

• Lessons Learned

– KPI’s that are not process related

– Superficial actions taken

– Lack of process documentation

Auditor Competence

9

• Criteria

– The CB shall determine the competences

it needs to include in its audit team (ISO

17021-1, Clause 9.1.2.3)

– Annex A, Table A.1, defines the required

knowledge and skills

– Lesson Learned

• Effective Auditors will demonstrate:

Registration Management Committee – July 2019, Torrance, CA

Adding Value

• Auditors can add value when conducting an AQMS audit by:

– Effective Audit Planning

– Process Approach to Gather Evidence

– Gathering sufficient evidence

– Clear concise reporting

• Lessons Learned

– Auditors are driven to gather minimal evidence just to complete the 9101 report

– Rush by poor time management

– Reliance on Client’s documented information versus visual and verbal evidence

– Lack of addressing changes and the impacts on organization performance

10Registration Management Committee – July 2019, Torrance, CA

Tips for Adding Value

• Auditors can add value by:

Making sure that the organization’s customers are satisfied and that the organization is consistently self correcting and improving the efficiency of it’s processes, product and procedures.

That the organization is focused on the customer and aligns its processes

around conforming to the customer’s requirements and that the standard is a

method of requirements for mapping a way for the organization to assure

customer satisfaction.

11Registration Management Committee – July 2019, Torrance, CA

Lessons from the Witness Audit Trail

Registration Management Committee – Month, Year- Meeting Location 12

Observations about the audit conduct

The Scope of the Organization

• What is the applicability of the scope?

1. Everything on the Internet is Fake

2. They don’t have design approval so theydon’t design anything

3. Purchasing is outsourced so clause 8.4 is NA

4. Value added has to be 9100 not 9120ISO 17021-1 9.5.2 Actions prior to making a decision

The certification body shall have a process to conduct an effective review prior to making a decision for granting certification, expanding or reducing the scope of certification, renewing, suspending or restoring, or withdrawing of certification, including, that

a) the information provided by the audit team is sufficient with respect to the certification requirements and the scope for certification

ISO 17021-1 9.2.3.3 Communication of audit team tasks

The tasks given to the audit team shall be defined, and require the audit team to:

a) examine and verify the structure, policies, processes, procedures, records and related documents of he client relevant to the management system standard;

b) determine that these meet all the requirements relevant to the intended scope of certification



Interviewing TOP MANAGEMENT

• What’s the purpose for interviewing theOwner, President, Sr. Official of theorganization being audited?

1. Discussing the local weather

2. Checking on good places to eat

3. Seeing if he/she can spell QMS

4. Discussing whether or not the businesshas a strategic plan or purpose

ISO 9001 5.1 & 5.2Top management shall demonstrate leadership and commitment with respect to the quality management system by taking accountability for the effectiveness of the quality management system;

AS9100D 5.3Top management shall appoint a specific member of the organization’s management, identified as the management

representative, who shall have the responsibility and authority for oversight of the above requirements.



The Factory or Shop Tour

• When or why should there be a tourof the facilities?

1. Need steps for my fitness tracker

2. The conference room smells from thetuna fish for lunch

3. Need to verify that there are fire exitsin case of fire

4. No need for a tour we can wander around during the audit

AS9101 4.2.2.3 Site Tour The audit team leader may conduct a site tour to address any changes in scope or facilities, since the last visit, or to familiarize audit team members with the organization's activities.

That means that the competent auditor will prepare for the audit in order to know what kind of objective evidence he or she needs to look for, rather than requiring the auditee to “serve up the evidence on a plate”.



Following the Audit Trail

• What’s the proper way to followan Audit Trail?

1. Evaluate at least 1 item for each product

2. Evaluate at least 1 item for each PEAR

3. Evaluate at least 1 item for each sub-clauseof clause 8

4. No need to evaluate anything there are noAS&D parts currently being produced

ISO 9001 APG Guidance on Audit Trial

The starting point for the audit is to use the chosen samples and identify the process path and the controls that were applied. It is vital that the samples are linked and come from the same trail. Too frequently, audit samples are taken at different stages of the process and are not related or linked to the initial sample chosen, which means that an auditor is unable to verify that the process is working. He will only be able to check if that particular document is filled in correctly.


THE BODY COUNT

• How many NCRs can be written duringan Audit?

1. Somewhere between 10 and 25

2. We need to add-value to the audit sothere as to be a lot of NCRs

3. The last audit was a Recert and there wereonly 2 NCRs so we need to make up some more

4. I am being watched so I should write up everything I see to prove I am a good auditor

17021-1 - 9.4.3.2Where the available audit evidence indicates that the audit objectives are unattainable or suggests the

presence of an immediate and significant risk (e.g. safety), the audit team leader shall report this to the client

and, if possible, to the certification body to determine appropriate action. Such action may include

reconfirmation or modification of the audit plan, changes to the audit objectives or audit scope, or

termination of the audit. The audit team leader shall report the outcome of the action taken to the

certification body.



Resources

Registration Management Committee – July 2019, Torrance, CA 18

IAQG Deployment Support Auditing Practices

https://www.sae.org/iaqg/organization/requirements.htm

https://committee.iso.org/sites/tc176sc2/home/page/iso-9001-auditing-practices-grou.html

Questions?

19Registration Management Committee – July, 2019 - Torrance, CA

effective 9101 aerospace auditing breakout session · purpose •to provide audit conduct criteria,...

Documents