in-tolerance non-conformance investigations webinar slides

In ToleranceNon-Conformance Investigations

Presented by

Better by Your Every Measure • 800.828.1470 • www.transcat.com

NCSLI LA/OC Regional Meeting May 12, 2015

To understand:

1. The familiar concept of OOT Measurement Risk

2. The less-familiar concept of In Tolerance Measurement Risk

3. The purpose of Guard Band

4. Compare to your organization’s current definitions• Create definitions for your organization if one doesn’t exist

• Enhance definitions for your organization if one already exists

5. How to apply these concepts to control of your measurement processes in order to reduce/eliminate product false accept/reject situations

Learning Objectives

Measurement Quality Assurance

• Metrological Traceability is important, but . . .

• It means nothing if the industrial processes don’t maintain a good measurement quality assurance program

• MQA is critical to industrial manufacturing processes:– To keep product costs at a minimum

– To maintain safety for employees and product consumers

– To keep product lead times from being delayed

– To make the traceability chain a worthwhile effort (and its costs)

• Am important part of using calibration data to control product decisions is to understand that even an In-Tolerance calibration result can cause False Accept situations on product

Where does Risk Creep into Your Process?

Measurement Quality Assurance encompasses a number of assumptions within a manufacturing process:

1. Product Acceptance You made the right decision about the acceptance or rejection of your product (or a component of your product) during the manufacturing and quality acceptance process.

2. Basis of Decision The decision was based (partially or completely) on quantitative information provided by one or more measuring instruments.

3. Suitability for Intended Purpose

You made the right choice in selecting instrument(s) that are appropriate for the measurements in your process.

4. Instrument Application The instrument(s) were used correctly in the manufacturing process when making decisions about product quality.

Where does Risk Creep into Your Process?

Measurement Assurance encompasses a number of assumptions (con’t):

5. Beginning of Period Reliability

The calibration of the instrument prior to your decision about the product indicated it met its performance expectations.

6. End of Period Reliability

The subsequent calibration of the instrument following your decision about the product indicates that it continues to meet its performance expectations.

7. Calibration Process Calibrations are executed correctly and support the application(s).

8. Non-Conformance Review (NCR)

If a calibration indicates that the instrument failed to meet its performance expectations, your quality system reviewed any impact to the decision(s) made about your product(s) that were based on the quantitative values presented by the instrument. Caution: This may include In Tolerance results in some circumstances

9. NCR Accuracy This impact study was thorough, followed an unbiased, forensic approach, and did not miss the mark on determining any detrimental affect on product.

10. Risk mitigation Any negative result from the impact study was properly mitigated to remove or reduce product risk.

• You’ve got to be kidding, right?

• Sorry but, no –

• But . . . This revelation of another ‘quality blind spot’ is important in protecting product decisions from resulting in a false accept or false reject situation

• Evaluation requires semi-complex concepts relating to sequences of events over time in order to see how this quality blind spot occurs

• An understanding of this particular quality blind spot also requires knowledge of measurement ratios (TAR, TUR, PAR, PUR, etc.)

• Let’s examine this using an example in Medical Device Manufacturing

• We’ll look at product measurements over a one year period

• As we review the measurements, we’ll logically step through the reasoning behind the commonly applied 4:1 ratio


• Heart-Lung Bypass Machine (perfusion equipment)

– Used to handle blood flow through the patient’s body,

bypassing the heart and lungs

– Patient is maintained in a hypothermic state between 28 ⁰C

and 32 ⁰C during the surgery



– Measuring the product influent and effluent temperatures

– Process tolerance for the effluent is 30 °C ± 2.0 °C

– 30 products measured over a 12 month period



– “As Produced” measurements taken on dates shown


• Example: Temperature measurement

– Process measurement is 30° ± 2.0° C

– Determine Inspection, Measuring, and Test Equipment (IMTE)

that is suitable for this measurement

LL:

Process

28°C

UL:

Process

32°C30°C


• Process measurement

– An instrument with an accuracy equal to the process limits?

– This is a 1:1 ratio

– Not good because the instrument can drift the full amount over its

cal interval, directly impacting previous process measurements


LL: IMTE

28°C

UL: IMTE

32°C30°C

• Process measurement: 1:1 Ratio

– IMTE was calibrated on 5/12/2014 and adjusted to nominal

– IMTE is used to measure effluent temperature the same day

– Effluent temperature is right at nominal: 30° C

28°C 32°C30°C

LL:

Process

UL:

Process


• Process measurement: 1:1 Ratio

– Instrument is on its last day of use before it will be recalibrated

– Instrument is used to measure effluent temperature 5/11/2015

– Effluent temperature is right at nominal: 30° C

28°C 32°C30°C

LL:

Process

UL:

Process


• IMTE Calibration: 5/12/2015

– IMTE is recalibrated

– As Found reading shows the IMTE drifted to the upper limit: 32°C

– This drift occurred over its calibration interval; instrument adjusted

28°C 32°C30°C

LL: IMTE UL: IMTE


• IMTE Calibration

– Did it move all at once or a little over time?

– How does this impact the measurements of the process since

the last time this IMTE was calibrated?

– Error = IMTE – Standard = 32 – 30 = +2 °C; Correction = -2 °C

28°C 32°C30°C

LL: IMTE UL: IMTE


• OOT-NCR Evaluation

– It must be assumed this shift occurred immediately following the

last calibration because that is the only supporting evidence to

show the last known good condition of its temperature values

– Returning to the product measurement made on 5/12/2014:

28°C 32°C30°C

LL:

Process

UL:

Process

x


• OOT-NCR Evaluation

– The process measurement taken on the day the instrument was

previously calibrated could be incorrect if the instrument shifted in

transportation or handling after the cal was performed

– Applying the correction of 30 °C - 2 °C = 28 °C

– The same applies to the process readings at the end of the cal cycle

28°C 32°C30°C

LL:

Process

UL:

Process

x


• In Tol-NCR Evaluation

– If one process reading had shown the product was at its lower limit:

28 °C, this would have been acceptable for the product test, but . . .

– The In Tolerance cal would not have flagged this situation as ‘at risk’!!

28°C 32°C30°C

LL:

Process

UL:

Process

x


• Here’s a look at the impact to the “As Produced” readings:


Falsely Rejected

Correctly Rejected

Falsely Accepted

• IMTE with better accuracy: 2:1 Ratio

– What if an instrument was selected with an accuracy twice as good?

– If it drifted to its limit but was still In Tolerance, process readings

would only be off by half of the process tolerance

28°C 32°C30°C

LL:

Process

UL:

Process

29°C

LL:

IMTE

31°C

UL:

IMTE

x




Falsely Rejected

Correctly Rejected

Falsely Accepted

• Instrument with better accuracy: 4:1 Ratio

– What if an instrument was selected with an accuracy 4X as good?

– If it drifted to its limit, process readings would only be off one quarter

of the process tolerance

28°C 32°C30°C

LL:

Process

UL:

Process

29.5°C

LL:

IMTE

30.5°C

UL:

IMTE

x




Falsely Rejected

Correctly Rejected

Falsely Accepted

• IMTE with better accuracy:

– How far can we take this? 10X better? 100X better?

– There are limits of technology

– Eventually it becomes cost prohibitive to go too far with this concept

– A 4:1 ratio generally produces about a 2% consumer risk

– This is widely accepted and practiced

– Depending on the criticality of the product, this may still be too much


• Lesson Learned:– Even “In Tolerance” results can impact your process

measurements!– All cal data must be reviewed against process measurements to

understand the impact to the product (IT-NCR evaluation)– Impact studies are expensive; consumes valuable resources– Can cost thousands of dollars per evaluation event– Parts or product likely have already been released/shipped– Could require product recall– If not released, could require rework– This risk can be reduced– Guard Band the process limits!


• What is Guard Band?

– Determine realistic tolerance limits for the process measurement

– Reduce these limits by the tolerance of the instrument used in the

process to arrive at Lower/Upper Acceptance Limits

30°C28°C

LL:

Process

32°C

UL:

Process

29.5°C 30.5°C

LL:

IMTE

UL:

IMTE

x27.5°C

28.5°C

LAL:

Process

31.5°C

UAL:

Process

x


• Guard Band result:

– Now if the instrument cal reveals a drift up to the maximum tolerance

there is no need to perform IT-NCR evaluation

– If instrument error is greater than this (i.e. it is OOT), then OOT-NCR

28°C 32°C30°C

LL:

Process

UL:

Process

29.5°C 30.5°C

LL:

UUT

UL:

UUT

x

28.5°C

LAL:

Process

31.5°C

UAL:

Process




Product will be

corrected BEFORE

it is shipped

Product will be

corrected BEFORE

it is shipped

• Summary:

– Implement a good Measurement Assurance Program

– Understand and exercise good Suitability selection

– Guard Band the process tolerance limits to reduce costly

Non-Conformance investigations

– Understand both the process and the calibration to ensure

the intent of preserving good measurements on the product is

not lost

– Be thorough in your NCR evaluations

– Get help if you’re in over your head!


• In the end, running your business is not only about international, federal, customer, or internal requirements, policies, or procedures – it’s about making a safe, reliable, superior product that fills a need/desire in the market place and is profitable for your company.

• Measurement Quality Assurance should be designed to help you minimize risks in your decision making process about your product’s safety and quality.

• Guard Banding processes is a key part of guaranteeing good measurements!

• Too often MQA is not fully implemented (all 10 categories), causing the reliability of your Measurement Quality Assurance Program to lose value and become ineffective.

• If you’re going to put the effort and money into only some parts of this program, or you simply do not recognize all of the factors, it will likely cost you elsewhere through rework, recall, or consumer perceptions.

• Make your Measurement Quality Assurance Program robust so that it works for you to keep cost and safety issues to a minimum and profits up!

Measurement Quality Assurance/Risk

Thank you for attending!

800.828.1470www.transcat.com

Questions?

in-tolerance non-conformance investigations webinar slides

Technology