quality: the basic tool kit quality audits 102, alex klein asq illiana section and calumet college...

Quality: The Basic Tool Kit Quality Audits 102, Alex Klein

ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

1

Quality Audits 102 Objectives

• “Quality (environmental, safety, food safety, financial, IT, …) audits are required.”

• Audits can enable quality control and quality assurance.– Product or service conformance– Meet customer requirements– System conformance– Process conformance– Process capability– Continual improvement

• Share personal experience and• Explore recent information from others



2

Review about Audits

• A systematic, independent and documented process for obtaining records, facts or other information and evaluating it objectively to determine the extent to which policies, procedures or requirements are fulfilled. ISO 19011

• Audit Requirements:– Planned: audit criteria, scope frequency and methods– Select auditors who are objective and impartial; shall not audit own work.– Procedure for conducting, recording and reporting– Management must take action

• Goals:– Analyze processes– Review conformance– Confirm information– Evaluate risks



3

Quality Control and Quality Assurance

• Fulfilling quality requirements (ISO 9000:2005)• Providing confidence that quality requirements will be fulfilled (ISO

9000:2005)• Concepts/ requirements:

– Quality objectives (ISO 9001, clause 5.4.1)– Standardized processes (7.5.1, 7.5.2)– Process monitoring and control (7.5, 8.2.3)– Statistical process control, SPC (ISO/TS 16949, clauses 8.1.1 and 8.1.2)– Measurement system analyses, MSA (TS 7.6.1)– Inspection, measurement and testing (8.2.3, 8.2.4)– Calibration (7.6)– Measurement traceability (7.6.a)– Process capability (TS 8.2.3.1)– Product or service conformance (8.1.a, 8.2.4)



4


• A process is a sequence of activities or steps which transforms inputs into outputs (ISO 9000:2005).

• Processes should be standardized, monitored, controlled, measured and continually improved.

• Every individual participates in at least one process that impacts the product, service or customer.



5


• Annual internal audits of the system, every process and products/ services.– Document, record, process and information control systems

– Manufacturing processes (including order management, R&D, scheduling, purchasing, inspecting, testing, shipping, transportation)

– Support processes (personnel services, training, maintenance, testing, safety, utilities) and

– Other processes (control of nonconforming product or service, billing and invoicing, internal audits, CA, PA) that impact customers.

– Top management processes (resource management, management review)

– Product coming to and leaving from each processing unit and in storage



6

A process is a set of interrelated or interacting activities which transform inputs into outputs.

• Supplier Inputs Process Outputs CustomerMaterialsInformationInquiries, OrdersVOC, voice of the customerPeopleRequirements, SpecificationsProcedureEnergyMoney

Activities/ StepsProduct

(Material, report, plan, schedule)Service

(Inspection, testing, maintenance)ReportPlan (business,

production, maintenance)Results



7

The SIPOC model may be used to understand processesSupplier Inputs Process Output CustomerCustomer Inquiry Review inquiry Contract Customer

Customer Order Enter order Accepted order Order management

Order management Accepted order Manufacturing limits

Prepare manufacturing order

Manufacturing order

Scheduling

Materials producer Order for material Procurement Material Production unit #1

Scheduling Manufacturing order Prepare line-ups Line-ups Production units

Production unit #1 MaterialLine-up

Production process #1 Product #1 Production unit #2

Production unit #2 Product #1 Production process #2 Product #2 Production unit #3

Production unit #3 Product # 2 Production process #2 Product #3 Final production unit

Audit team Audit schedule Audit procedure

Internal audit Audit report Auditee

Final production unit Product #n Inspect, test Results Packaging unit

Packaging unit Packaging specs Package Packaged coil Shipping

Shipping Packaged product Deliver packaged product

Delivery of product Processor or customer



8

Lessons Learned

• Train auditors about the standard(s) and process approach to auditing; use ISO 19011.

• Train all auditors annually.– Revised standard requirements– New interpretations/ understanding about requirements– Discuss focus areas (e.g., performance training, calibration) for the

current audit cycle.– Review feedback from previous audits.– Provide aids to auditors:

• Checklists (e.g., manufacturing, non-manufacturing, maintenance processes

• Checklists for layered process audits– Practice

• Review findings from previous audits



9

Lessons Learned

• Include process/ technology experts on audit teams.

• Be efficient and effective:– Time management – audits are an intrusion in the audited area.– It may not be necessary to audit what has been audited by Internal

Affairs and SOx auditors and financial consultants. Verify that action was taken to correct any findings.

– Interview small groups of individuals performing the same activity.– Observe the evidence for effective implementation of actions taken to

correct nonconformances from previous audits– Representative/ random sampling – enough to reach comfortable

conclusions



10

Lessons Learned• Observe measures of performance for each process

– Not just output provided, also consider,– Was process effective and efficient?

• Allow correction and corrective action– Allow RCA and correction of minor nonconformances.– Expect formal corrective action for systemic problems.

• Expect continual improvement for some, not all measures of performance.

• Remember the organization– Exists for its customers– Must run the business



11

Lessons Learned• Recognize good performance

– Processes capable– Processes improving

• Share information– About successful implementation in another area– From benchmarking– Offer opportunities for improvement

• How to audit mature systems– Consider the audit team – rotate the members, include different

persons.– Consider the audited areas – have the systems, processes or people

changed since the previous audit?– Are measure of performance being met? Is performance improving?



12

Lessons Learned

• Inform/ train all employees about the benefits of auditing:– Understanding about the audit process– Employees’ role in the audit– Reduce the fear about an audit– Enables participation in continual improvement activities.

• See “Improving the Internal Audit Experience”, Theresa Wasche and Nancy Sciortino, QP, November 2007



13

Lessons Learned

• Explore/ benchmark– Consult consultants– Don’t allow them to dictate– What does the standard require?– How can we most efficiently and effectively meet requirements?– How can the management system enable meeting customer

requirements and continual improvement?– Read and evaluate– Network– Stay current with rapidly advancing technology and use it to improve

audit efficiency



14

“Word Power, 12-point style guide for the quality professional”, Burjor Mehta, QP (Quality Progress), October 2010

• Use words and phrases with positive connotations.• Be brief and concise.• Use consistent terms.• Use consistent formats.• Use white space.• Be specific.• Don’t use jargon or clichés.• Use complete words and sentences.• Use “I” to ask and “you “ to give.



15

“Brace for Impact, Modified approach improves outcome of internal audits”, James Wells, QP, October 2010

• Observe measures of performance (metrics) and determine which are not meeting target performance.

• Audit processes that support these metrics.• Analyze finding for impact on the business (reputation, scrap,

rework, wasted time and effort).• For many findings a direct or indirect connection may be found to a

potential financial risk associated with a poor practice found during the audit:– Past due calibration/ daily verification– Closure time of a customer complaint

• Also see “All Ears, Use financial terms to get management to take notice of your quality message”, Peter J. Sherman and James G. Vono, QP, July 2009



16

“Dealing with DISCONNECT”, Joseph D. Conklin with Michael J. Mazu, QP, October 2010

• Raise the awareness about significant nonconformances.

• Same recommendation made each year:– Trained person to back-up lone knowledgeable person– Funding needed to improve system

• Question upper management and external auditors– Clearly understood expectations– Poorly written findings



17

“Trust, but Verify”, J.P. Russell, QP, November 2009

• There are systems audits.

• There are more product, service and process audits.

• Combined product, service and process audits = verification audit.

• Verification should be performed when there are routine changes (suppliers, equipment, process settings, methods, requirements, personnel).

• This audit helps minimize risk and ensure sustainability.



18

“Think Again, Change your view of audits to improve their effectiveness”, Sonny Sahi, QP, October 2009

• Auditors and management must accept nonconformances and present them and receive them as opportunities to investigate how the process or system failed not how the person(s) failed.



19

“Gauge Audit Program Value”, J.P. Russell, QP, June 2009• Effectiveness of a system or process:

– Achieves desired result consistent with objectives for the product– Is capable, efficient and consistent with objectives for the process

• Effectiveness of audit process and follow-up actions:– Counting audits performed, nonconformances cited, closed corrective

actions are typical performance measures.– Consider:

Objectives How? Performance measuresMeet customer requirements Obtain feedback Internal satisfaction

Meet management system requirements

Identify and report CAs that improved performanceUpgrade competence of auditors

No serious findings from external auditsRoot cause(s) identifiedCA effective, no reoccurrences

Maintain supplier and outsourcing requirements

Verify claimed improvements Quality and delivery met



20

“Gauge Audit Program Value”, J.P. Russell, QP, June 2009• Effectiveness of audit process and follow-up actions:

– Counting audits performed, nonconformances cited, closed corrective actions are typical.

– Consider:Objectives How? Performance measuresImprove effectiveness and efficiency of management system

Prompt audit reporting Findings linked to objectives or customer requirementsReport terminology understoodRepeat nonconformancesAddress OFIs = PAs

Determine management objectives for lowering expenses

Mini audits Reduced cost of audits

Implement processes to ensure sustainability

Schedule audits based on process changesTrain QS resources in areas

Repeat nonconformances

Validate management system design to meet future needs

? ?



21

“Can Do, Effective Overseas Food Safety Audits”, Steve Wilson, QP, May 2009

• These audits must/ can be successful when considering and planning for– Language and social barriers– Varying requirements in other countries.



22

“Putting Your Internal Audit System on Steroids”, Mike Micklewright, presented at NIQC, May 2008

• Audit for conformance• Audit for effectiveness

• Teach auditors about wastes– Overproduction → PA– Excess inventory → PA– Transportation → PA– Delays → PA (If there was objective/ target, then CA)– Excess processing → PA– Wasted motion → PA– Defective product → CA– Underutilized employees → PA



23

“Putting Your Internal Audit System on Steroids”, Mike Micklewright, presented at NIQC, May 2008

• Teach auditors about – Value stream mapping and process mapping– RCA– Mistake proofing

• Evaluate auditors:– Preparation– Auditing– Report– Relevance of nonconformances



24

“Drill Deep for Strategic Alignment”, Joe Kausek, QP, October 2008

• Provide auditors the organization’s:– Vision, mission and values– Overall business plan– Balanced scorecard– Process for establishing metrics and reporting performance.

• Audit for strategic alignment– How does process contribute to strategic objectives?– How have process owners aligned processes with strategic objectives?– Were resources provided for successful accomplishment of the

objectives?– Were process metrics aligned with metrics for strategic objectives?– Is the process contributing to the success of the strategic objective?



25

“10 Auditing Rules”, Joe Kausek, QP, July 2008• Make auditees feel like members of the audit team.• Start by asking general, open-ended questions, then use clarifying

questions to fill the gaps.• Be an active listener.• Never let the auditee pick the samples.• Always try to identify the financial effects of findings, when possible.• Always confirm your findings with the auditee.• Don’t look for findings that could never have an impact on the

performance of the management system.• Provide sufficient information about the requirement,

nonconformance, and evidence found.• When citing areas of strength, be specific.• Feed the auditor.



26

“The Great Debate, Can one process approach apply to QMS and EMS audits?”, Terry A. Mors, QP, April 2008• (And Safety management system and Food Safety management and

other management system audits?– OHSAS 18001 does require determining effectiveness of the OHSAS

management system.)• ISO 9001 and ISO/TS 16949 support the process approach.• Although an EMS (and SMS) is a system of processes, some

adjustments are required due to language, system and standard differences.– ISO 14001 (and OHSAS 18001) was not written based on the process

approach.– ISO 14001 does not require measuring effectiveness of each process.

• Using inputs and outputs (interaction) terminology can steer organizations toward a process approach.



27

“Sharpen Your Auditing Skills”, Joe Kausek, QP, February 2008

• Use checklists to guide planning, auditing and reporting.– Identify the process, inputs, procedures, instructions, outputs, recent

performance.– Prepare an audit planning checklist.– Review procedures, diagrams and other information.– Select items to verify.– Mark up the documents to serve as a checklist.– After initial interviews were completed, enter the objective evidence.– Identify additional questions or items to be audited.



28

Other resources• “The Right Approach, An effective QMS audit requires a process-

based strategy”, Sandford Leibsman, QP, September 2009

• ISO 9004, Managing for the sustained success of an organization

• Other periodicals:– “Quality” – “Quality Digest”

• The Process Auditing and Techniques Guide, J.P. Russell, second edition, 2010

• Websites:– ASQ, www.asq.org

http://www.asq.org/



29

Your Comments and Questions are desired.

Thank you!

29

quality: the basic tool kit quality audits 102, alex klein asq illiana section and calumet college...

Documents

quality control

quality requirements

basic tool kit quality

quality objectives iso

process approach

documented process

statistical process

alex klein asq illiana