practical solutions to traceability and uncertainty in accreditation

American Association for Laboratory Accreditation

Practical Solutions to Traceability and Uncertainty in

AccreditationPresented to CITAC-NCSLI Joint Workshop

“Traceability and Uncertainty:

Key Technical Issues and Laboratory Accreditation”

PITTCON 2002, New Orleans

Sunday, 17 March 2002

Warren Merkel, A2LA Technical Manager


Presentation Overview

• Relevant ISO/IEC 17025 Requirements

• Accreditation Body Policies

• Practical Approaches for Laboratories

• Future Developments

• Questions


Measurement Traceability - 17025 Requirements

– Calibrations & reference materials traceable to SI where possible

– Where not possible, traceable to certified reference materials, agreed methods and/or consensus standards

– …Unless it has been established that the associated contribution from the calibration contributes little to the total uncertainty of the test result


Traceability- Definition• Relation to stated references through an

unbroken chain of comparisons…– Traceability to some stated reference available

with most RMs– Normally not SI unit for amount of substance, but

often other SI units

• All having stated uncertainties– Little evidence of verification of uncertainty

claims


Traceability in Practice

• In many cases, traceability to the test method

is all that is required by clients

• Uncertainty statements on RMs are

important, but usually not a significant

contributor to uncertainty of test

• Performance in proficiency tests can serve as

an indicator of traceability problems


Measurement Uncertainty - 17025 Requirements

• 5.4.6.2 - Testing laboratories shall have and apply procedures for estimating uncertainty of measurement

• Nature of the test method may preclude rigorous, metrologically and statistically valid, calculation of uncertainty of measurement

• Laboratory shall at least attempt to identify all uncertainty components, make a reasonable estimation, and ensure that the form of reporting of result does not give a wrong impression of uncertainty


Measurement Uncertainty - 17025 “Loopholes”

• Note 1 - Degree of rigor depends on:– Requirements of the method

– Requirements of the client

– Existence of narrow limits on specification conformance

• Note 2 - In cases where a well-recognized test method specifies limits to the values of the major sources of uncertainty and specifies the form of presentation of calculated results, the laboratory is considered to have satisfied this clause by following the test method and reporting instructions


Measurement Uncertainty - 17025 Reporting Requirements

• 5.10.3.1 c) Information on uncertainty is needed in test reports when it is relevant to the validity or application of the test results, when a client’s instruction so requires, or when the uncertainty affects compliance to a specification limit

• 5.10.1 - In the case of a written agreement with the client, results may be reported in a simplified way


A2LA Interim Policy on Measurement Uncertainty for Testing Laboratories

• Five categories of test methods• Intended to facilitate transition- pragmatic

approach, not ideal in all cases• Results from review at annual meeting:

– Some modification of language

– Publish list of example methods for each category by field of testing

– Publish guidance on determining uncertainty in testing based on ISO 5725



• Five categories of test methods:I. Qualitative

No uncertainty calculations required

Examples: Ignitability; Microbiological screening

II. Well-recognized methods that specify limits to uncertainty contributions (Note 2)

No further uncertainty calculations required

Examples: Flash point; Hardness

Problems: Modification of method

What if a client wants uncertainty?



• Five categories of test methods, continued:III. Published methods that do not specify limits to

uncertainty sources and/or reporting format

Uncertainty estimated using standard deviation of laboratory control samples

Examples: Alloy analysis by OES; VOA

Problems: Normal process for analyzing control samples may lead to an

underestimate of uncertainty

Quality of control sample



• Five categories of test methods, continued:IV. Methods requiring identification of major

uncertainty components and reasonable estimate of uncertainty

Examples: PBMS; One-off tests

V. Methods requiring full uncertainty analysis consistent with ISO Guide to the Expression of Uncertainty in Measurement

Example: Reference material value assignment


A2LA Interim Policy - Chemical Laboratories

• Majority of methods classified as Category II, III, IV

• Classification can vary by laboratory and use

• Use of RMs critical to demonstrating process control and evaluating bias

• More practical guidance required


17025 Requirements: Quality Control and Proficiency Testing

• Laboratory shall have procedures for monitoring validity of tests, including:– regular use of CRMs or internal QC using secondary

reference materials

– participation in interlaboratory comparison or proficiency testing programs

• Record data so trends are detectable

• Laboratories in most cases already have sufficient data for estimating uncertainty


Practical Approach - Category II• Laboratories performing tests in Category II

can utilize precision data published with method as uncertainty estimate if:– Laboratory has data demonstrating that its

repeatability is comparable to the method data– Material used for precision estimate is similar to

materials tested by lab– Method not modified– Basis for estimate clearly stated


Practical Approach - Category III

• Intermediate measures of precision provide adequate estimate of uncertainty, if:– Measurement method standardized– Measurement process is in control– Control sample well characterized (preferably CRM)– Process for collecting data designed to vary all

significant uncertainty components


Steps Laboratories Can Take• Identify major uncertainty components

(App. D of EURACHEM/CITAC Guide)• Establish control charts (ISO 8258)• Attempt to design QC process to ensure representative

variation of inputs• Fewer points more emphasis on design• May need to record additional data related to

precision as objective evidence


Steps Laboratories Can Take• When required to report uncertainty, clearly

define the basis for the estimate– Clarify during contract review process– If possible, determine with client end use of data

• Make use of PT study data– Compare internal precision data to spread of results

of participants– If study based on reference value, compare lab

result/uncertainty with reference


Future Developments• Consensus method development

– More rigor in precision data – Additional detail regarding uncertainty sources

• Accreditation of RM producers

• Increased focus on PT based on reference values vs. consensus values

• Increased awareness of traceability uncertainty issues in user community


Conclusions

• Accreditation bodies and laboratories reconciling 17025 requirements that are ahead of the state of development in many industries

• Pragmatic approach to requirements is necessary

• Goal: provide data that is fit for purpose


Contact InformationWarren Merkel

A2LA5301 Buckeystown Pike Suite 350

Frederick, MD, USA 21704

Direct line: 301 644 3204Main: 301 644 3248Fax: 301 662 2974

[email protected]

practical solutions to traceability and uncertainty in accreditation

Documents

uncertainty claimstraceability

uncertainty of testperformance

uncertainty contributions

calculation of uncertainty

total uncertainty

uncertainty components

major sources of uncertainty

test results