9101 – ask the experts

Company Confidential

Registration Management Committee (RMC)

1

9101 – Ask the Experts

Boston, MAJuly 21-22, 2011

Tony Marino, Tom Gallagher, Larry Beck

Auditor WorkshopBoston, MA

July 21-22, 2011



Agenda– Summary of Changes – Tony Marino (Boeing)– Sector/CB Feedback – Tom Gallagher (UTC)– Future Changes – Larry Beck (Lockheed)




9101 Team Membership• 13 Members on the 9101 Team• Representing:

– Americas, Europe, and Asia-Pacific IAQG Sectors– 6 Different Countries– 8 IAQG member companies– Certification Body

community (4 members)




9101 Stakeholders• Civil Aviation Industry and Authorities• Defense Industry and Authorities• Space Industry and Authorities• Certification/Accreditation Bodies• Auditors• Suppliers• IAQG Member Companies• IAQG Strategy Streams and Teams




9101 Revision Objectives• Ensure alignment with the IAQG strategy (on-time, on-quality

performance, improved control of Other Party certification)• Incorporate the 9100:2009-series changes and align with the ISO 17021

content (new ISO assessment standard)• Support the process approach for quality management systems as

described in the 9100-series standards • Incorporate the ICOP (Industry Controlled Other Party) resolutions and

improve the value of OP auditing• Define the requirements for all activities of the audit program• Provide an adequate audit report to stakeholders on Performance of

Organizational Processes. Note: product performance/results driven




9101 Process Approach target• Definition of the audit process phases with audit planning, execution,

analysis and reporting:(Pre-audit, Stage 1, Stage 2, Surveillance, Re-Certification)

• Audit of the organization’s processes and continual improvement approach, as required in 9100-series standards

• Development of performance-based, process-orientedaudit methods and techniques.

• Recording of objective evidence of process conformity and effectiveness

• Reporting of audit results in a uniform, transparent and standardized way




9101 Major Revision Summary• 9101 completely rewritten reflecting now an enhanced audit process

for evaluating “process based management systems”• One audit standard covering 9100/9110/9120 QMS standards. • Elimination of “Scoring” and “Key Requirements”.• Use of Customer feedback concerning Organization QMS

performance as an input for process oriented audits (e.g., OASIS, customer satisfaction/performance scores).

• In addition to conformity, also determination of Effectiveness included.

• More emphasis on performance measuring.• Introduction of a new Objective Evidence Record• Introduction of a new Process Effectiveness Assessment Report




AuditRecords Methods/Tools

Process Approach

Process-Based Auditing

Inputs Outputs

Performance Focused

1 Auditing Company Name Nonconformity Report (NCR)

2 Auditing Company

Logo 3 Organization: 5 Site/OIN:

4 Audit Report ID/type:

4 NCR Number:

6 Issue Date: 7 SECTION 1 - DETAILS OF NONCONFORMITY: 8 Process/Area/Department: 9 Requirement/Clause No.(s): 10 Classification (ma/mi):

11 Statement of Nonconformity:

12 Objective Evidence:

13 Due Date:

14 Auditor 15 Auditee Representative Acknowledgement NAME: Signature: Name: Signature:

7SECTION 2- AUDITEE PLANNED ACTIONS (Attach separate sheet, as needed.) 16 Containment Action(s), including correction, with supporting completion date(s):

17 Planned Completion Date:

18 Actual Completion Date:

19 Root Cause: 20 Cause Code:

21 Corrective Action(s), including supporting completion date(s): 22 Planned

Completion Date:

23 Actual Completion Date:

15 Auditee Representative Name and Signature:

Date:

24 Auditor Signature for Acceptance of C/A(s):

Date:

7/25 SECTION 3 - DETAILS OF AUDITOR VERIFICATION OF ACTION:

7/26 SECTION 4 - NCR CLOSURE (Auditor Name /Signature/Date):

27 Approved by Audit Team Leader (Name/Signature/Date):

NCRs PEARs

OER

Evidence of conformity Evidence of effectiveness

Controls

Resources

3Site: 4Issued Da te:

7Support Process(es) Assessed:

69100/9110/9120 Section Referenc e:

5□ Core Process Assessed:

□ Support Process Assessed:

□ Foundation Process Assessed:

Effectiveness Assessment Record1Organiza tion: 2Report Number:

8Statement of Effectiveness:

□The assessed core/support/foundation process has been determined to be effective.

□The assessed core/support/foundation process has been determined to be ineffective.

9How was process effectivene ss determined (methodology)?

10Objective Evidence, Observed Conditions, Data , and Comments to Support Effectiveness Determination

11Assessment Team Leader 12Signature and Date

Signature indicates that all relevant requirements (including requirements for support processes identified in Block #7) have been assessed to determine effectiveness of the core or support process identif ied in Block #5.

Model for Process Based Auditing - 9101




9101: Core principles• Customer driven• Process based approach• Organization Performance focus• Audit methods included• Quality Management System effectiveness, in

addition to conformity• Improved reporting of audit results




• 5 Areas of Sector/Certification Body Feedback– Audit Planning

» Increased preparation time» Need to clarify and confirm expectations for 4.2.1

with respect to timing and 9104/1 auditor days» Some clients refusing to provide proportion of business

as a percentage for each customer» Audit plans must be written in terms of client’s QMS

processes – not departments or generic ISO/AS clauses» There must be continuity between the terminology (e.g.

process names) used in audit plan to those in Process Matrix and PEARs




• 5 Areas of Sector/Certification Body Feedback– Stage 1 Audit

» A good way to determine readiness on process approach/measurement of process effectiveness. Should be mandatory for transition audits as well

» Send Stage 1 Report to client 1-2 months prior to audit. Have them send back to auditor partially completed, this will help with time management

» Auditors say that 1 auditor day (for small size organizations) for Stage 1 audit is sometimes not enough

» In the text of 9101D at Stage 1 audit areas of concern might be identified. But in audit report (stage 1) “Has organization been requested to submit a corrective action plan prior to the stage 2 audit?” Needs more explanation.




• 5 Areas of Sector/Certification Body Feedback– Stage 2 Surveillance / Recertification

» Better focus on process effectiveness through the PEAR +» Better focus on Customer Satisfaction +» Lead auditors spend a lot of time to explain changes in the audit

methodology. Help (communication) from industry seems necessary.

» When based on PEAR results, closing meeting generates higher interest from top management +

» A more detailed audit team meeting is needed than before. For example, to prepare PEAR, detailed communication about assessing the process is needed.

» Effectiveness of the audit itself is increasing +




• 5 Areas of Sector/Certification Body Feedback– Reporting

» Forms are difficult and time consuming (e.g. format issues, pagination, redundant information, etc.) Suggest modification of header to include ‘common’ information for copying/cut & pasting.

» Suggest combining all forms (w/exception of NCR) with OER and call it Audit Report

» Headcount (required for OASIS entry)only appears on Stage 1 report form

» PEAR , QMS Process Matrix, OER, NCR, Stage 1 Audit Report – various comments & suggestions for improvement

» Signature requirements is ‘19th century’




• 5 Areas of Sector/Certification Body Feedback– Time

» Audit preparation time has increased» Time required to complete forms is onerous and takes

away from actual audit time» Technical review and OASIS entry time has increased» More time needed for audit planning and post-audit

activities than before» … must mandate minimum time for reporting in our new

9104/1 document




What’s next for 9101?• Kick-off meeting for 9101 re-write April 2011 in New

Orleans, LA• Expected topics include:

– Feedback Discussion– OER Changes– Forms Simplification/Consolidation– Efficiency Discussion




9101 Revision plan:• Input for Next Revision

– ISO/IEC 17021:2011 – FAQ list Notes or Requirements– “Left overs”

» Risk based auditing» “Biased auditing”» “Intelligent auditing”

– Team Evaluation– “Survey” Feedback from auditors/CB/Client.– Feedback from TP trainers (combine with feedback to the training

package)– 9104/1 input




Completed by the appropriate IAQG Document Representative (IDR)

Date Revision Initiated: 19-10-2010

IAQG-issued Standard Number: 9101 Current Issue (letter & date): Rev , 2009

IAQG-issued Standard Title: Quality Management Systems Audit Requirements for Aviation, Space and Defense Organizations

IAQG-issued Standard Scope: Currently published in America (AS9101D), Europe (prEN9101 Edition P6) and Asia-Pacific (SJAC 9101D)

IAQG Document Representative (IDR): Hans Luijt / Masahiro Kawamoto (After York meeting) Strategy Focus Stream Leader (SFSL): Requirements – Alan Daniels

AAQG Sector Document Representative (SDR): Tony Marino APAQG Sector Document Representative (SDR): Masahiro Kawamoto

EAQG Sector Document Representative (SDR): Hans Luijt / Stuart Anthony (After York meeting)

Reason(s) for revision: To incorporate the changes of referenced ISO/IEC 17021 standard (currently FDIS stage), and to reflect input from FAQ, feedback from auditors/CBs/clients/TPs by survey, left-overs and other related standard (e.g. 9104/1).

Initiative deliverables: Revised 9101 standard and relating deployment support materials, such as FAQ, Changes Overview presentation, etc.

Estimated Publication Date: 01-08-2012

9101 Revision Form:




Task Who Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

IAQG MeetingsYork Meeting Exisiting teamNew Orleans Meeting New teamBordeax Meeting New teamXXXX Meeting New team

9101 Team Make UpProject concurrence with RequirementsStream (Alan Daniels) Luijt, Kawamoto

Design & agree team make up Existing teamContact stakeholders (CBs & Ind) SDRsConfirm team members SDRsConfirm 9101 revision scope SDRsProject concurrence with RequirementsStream (Alan Daniels) Kawamoto

17021Review FDIS 17021 Bonnel, AnthonyReview IS 17021 Bonnel, AnthonyIdentify 9101 changes Bonnel, Anthony

FAQsReview FAQs Flink, MarinoIdentify 9101 changes Flink, Marino

9104

9101/9104 coordination meeting Jacquaniello, Luijt,Kawamoto, Anthony, Flink

Detailed 9104 review Jacquaniello + otherIdentify 9101 inputs/changes Jacquaniello

Left OversCapture left overs Kawamoto, LuijtAnalyse initial 9101 survey results KawamotoIdentify 9101 changes Kawamoto

SurveysDesign client survey Ind Rep + CB RepDesign CB survey Ind Rep + CB RepDesign Auditor survey Ind Rep + CB RepDesign TP survey Ind Rep + CB RepCoordinate & launch via SAE MarinoConduct surveys SAEAnalyse results Ind Rep + CB RepIdentify 9101 changes Ind Rep + CB Rep

Review new 9101 audit reportsAnalayse Stg 1 / 2 reports (sample) Bonnel + CB RepIdentify 9101 changes Bonnel + CB Rep

Revise 9101 in readiness for ballot Team

BallotPre-ballot SDRsRevise 9101 (post pre-ballot)Final ballot - AAQG Sector ResponsibleFinal ballot - EAQG Sector ResponsibleFinal ballot - APAQG Sector Responsible

Revise 9101 (post ballot) Team

9101 re-issue Sector Responsible

201120109101 Revision 2012

• ECD for balloting: March, 2012• ECD Publication: July, 2012

• 9101 Revision Milestone:




9101 Questions


9101 – ask the experts

Documents

audit process phases

enhanced audit process

audit planning

audit programprovide

members boston

quality management systems

process oriented audits

processorientedaudit