astm international interlaboratory study program (ils) · astm international interlaboratory study...
TRANSCRIPT
![Page 1: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/1.jpg)
ASTM International
Interlaboratory Study Program
(ILS)
September 2014
Officers’ Training Workshop
Phillip Godorov & Caitlin Farrell
![Page 2: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/2.jpg)
The Interlaboratory Study (ILS) Program
Who we are and what we do …
We were originated following approval by the Board of
Directors to create a unit that would help to strengthen the
perceived quality of ASTM Test Methods by:
• facilitating the production of data to develop Precision & Bias
statements and Research Reports;
• providing administrative support to the committees;
• offering financial support to the committees;
• helping to ensure the confidentiality of participating labs.
www.astm.org // 2
![Page 3: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/3.jpg)
Administrative Support
• Review of Experimental Design
• Assistance identifying volunteer laboratories
• Identification of sample vendors
• Coordination of sample distribution
• Data collection
• Statistical processing
• Generation of reports
www.astm.org // 3
![Page 4: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/4.jpg)
Strengths of the ASTM Interlaboratory
Study Program
• Input encouraged from active Committee volunteers, as
well as non-members, broadening the range and diversity
of the study participants, allowing the study to most
accurately demonstrate expected “real-world” precision
• Scientific neutrality of ASTM in reviewing test data
• Provides value-added Quality Assurance Programs to
participating laboratories
www.astm.org // 4
![Page 5: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/5.jpg)
Potential Benefits of Participation
• Statistical program to monitor strengths and
weaknesses of lab testing when compared to peers
• Assess testing performance and adherence to written
procedures by lab technicians
• Recognition in the final Research Report
• Upon publication of the results of an ILS, all participating
labs receive a free 12 month listing in ASTM’s Lab
Directory
www.astm.org // 5
![Page 6: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/6.jpg)
Benefits to the Committees
• Obtain valuable feedback on
methods, leading to the
correction of errors and
omissions, as well as
highlighting the need for
technical updates
• Resource for increased
membership
• Database of laboratories for
test method validation
• Database of suppliers for
common study materials
www.astm.org // 6
![Page 7: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/7.jpg)
Administering Programs
• Conference Calls or Virtual Meetings with the participants
to discuss specific study instructions
• Coordination of acquisition and distribution of study
material
– Physical
– Electronic
• Collection of data
• Analysis of data
• Compilation of Precision Statement and Draft Research
Report (from registered information)
www.astm.org // 7
![Page 8: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/8.jpg)
Precision
• To calculate precision, we need usable data from at least
six laboratories.
– The closer that we can get to 30 participants, the better.
• The precision statement in an ASTM test method is not
meant to qualify it as good or bad.
– The published precision is there to help a user of the
standard understand what can be expected based on the
real world results of others.
www.astm.org // 8
![Page 9: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/9.jpg)
Repeatability (r) ranges
• With 95% confidence, the same operator, in the same
laboratory, using the same equipment, under the same
conditions, should obtain results when testing the same
material that agree within this range.
– Example: repeatability range = 2.4 ppm
• Result 1: 79.1 ppm Result 2: 81.6 ppm
• Results differ by 2.5 ppm, therefore: Suspect
– Internal laboratory investigation may be
advisable
www.astm.org // 9
![Page 10: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/10.jpg)
Reproducibility (R) ranges
• With 95% confidence, two operators, in different
laboratories, using different equipment, under conditions
meeting those specified in the standard, should obtain
results when testing the same material that agree within
this range.
– Example: reproducibility range = 3.2 ppb
• Result from lab 1: 50.8 ppb
• Result from lab 2: 47.9 ppb
– Results differ by 2.9 ppb, therefore: As
Expected
www.astm.org // 10
![Page 11: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/11.jpg)
The Statistics (in a nutshell)
~ ASTM E691
• E691 is useful for estimating the precision of different materials, at
varying levels, with a repeatability and reproducibility range being
calculated for each.
• Within laboratory precision is evaluated against a k-statistic.
• Variability among replicates in any one lab
• Between laboratory precision is evaluated against an h-statistic.
• Lab averages compared between all participants
www.astm.org // 11
![Page 12: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/12.jpg)
The Statistics (for Committee D02)
~ ASTM D6300 Standard Practice for Determination of Precision and Bias Data for
Use in Test Methods for Petroleum Products and Lubricants
~ ASTM D7778 Standard Guide for Conducting an Interlaboratory Study to
Determine the Precision of a Test Method
• Subcommittee D02.94 - Coordinating Subcommittee on
Quality Assurance and Statistics
www.astm.org // 12
![Page 13: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/13.jpg)
Bias
• To calculate bias, we need to include a reference
“standard” among the sample specimens distributed to
the participating laboratories.
– Bias may be determined as the average discrepancy
between the “known” value and the reported values.
www.astm.org // 13
![Page 14: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/14.jpg)
Precision and Bias
www.astm.org // 14
![Page 15: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/15.jpg)
Remember …
• You can correct for bias.
• You cannot correct for imprecision.
• An ILS can be used to demonstrate improvement as
standards are modified.
– Compare results from Method 1 with those from
Method 2
• The presence of a Precision Statement can make a
standard stronger, but it doesn’t make it more repeatable.
www.astm.org // 15
![Page 16: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/16.jpg)
ILS informational Web pages
www.astm.org // 16
http://www.astm.org/ILS/
![Page 17: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/17.jpg)
Establishing New Programs
• Concept registered as a Work Item
• Program registered online as an ILS
• Initial conference call, with the a technical contact from
the committee, to establish the basic study parameters
• Experimental design (with input from the committee’s
statistical support person, if available)
• Identification of study materials, suppliers, a distributor,
and volunteer laboratories
www.astm.org // 17
![Page 18: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/18.jpg)
www.ASTM.org
www.astm.org // 18
![Page 19: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/19.jpg)
www.astm.org // 19
![Page 20: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/20.jpg)
www.astm.org // 20
![Page 21: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/21.jpg)
www.astm.org // 21
![Page 22: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/22.jpg)
www.astm.org // 22
When you
are
starting a
brand new
ILS
If you are
updating
an ILS
that
you’ve
already
started
![Page 23: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/23.jpg)
First Page of the Registration
www.astm.org // 23
![Page 24: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/24.jpg)
Technical Contact
www.astm.org // 24
![Page 25: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/25.jpg)
Analyses
www.astm.org // 25
![Page 26: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/26.jpg)
Replicates
www.astm.org // 26
![Page 27: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/27.jpg)
Materials
www.astm.org // 27
![Page 28: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/28.jpg)
Suppliers
www.astm.org // 28
![Page 29: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/29.jpg)
Suppliers Continued…
www.astm.org // 29
![Page 30: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/30.jpg)
Distributor
www.astm.org // 30
![Page 31: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/31.jpg)
Laboratories
www.astm.org // 31
![Page 32: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/32.jpg)
Statistical Support
www.astm.org // 32
![Page 33: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/33.jpg)
ILS Registration Summary
![Page 34: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/34.jpg)
Committee Week Reports
www.astm.org // 34
![Page 35: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/35.jpg)
Research Reports
• The ASTM Form and Style Manual (Section A29.1)
states, "Where numerical data have been generated to
establish the precision and bias of a test method, a
research report is required.”
• ASTM Research Report Template
• The draft research report should be made available to
committee members while the associated precision
statement is on ballot.
• Research report numbers are assigned after ballot
approved.
www.astm.org // 35
![Page 36: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/36.jpg)
Parts of a Research Report
• List of participating laboratories
• Description of samples with their suppliers
• A copy of the laboratory instructions
• Description of equipment/ apparatus used
• All of the raw data (lab name’s hidden)
• A statistical summary
• A copy of the precision and bias statement
www.astm.org // 36
![Page 37: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/37.jpg)
Research Report Assistance
• ILS can assist with the following research report tasks:
– Answer any research report inquiries
– Create research reports from ILS studies
– Reformat older research reports to fit the ASTM
research report Template
– Assign research report numbers
www.astm.org // 37
![Page 38: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/38.jpg)
Research Report Numbers
• A research report number will be assigned by ASTM
when all of the following have been completed:
• the research report is submitted to ILS;
• it has been reviewed for completeness;
• and the ballot item to include the corresponding
precision and bias statement is approved for
publication.
www.astm.org // 38
![Page 39: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/39.jpg)
Questions
www.astm.org // 39
![Page 40: ASTM International Interlaboratory Study Program (ILS) · ASTM International Interlaboratory Study Program (ILS) // 40 Phillip Godorov, Director Phone: 610-832-9715 Fax: 610-834-7011](https://reader030.vdocuments.us/reader030/viewer/2022040615/5f0e60f17e708231d43ef550/html5/thumbnails/40.jpg)
ASTM International
Interlaboratory Study Program
(ILS)
www.astm.org // 40
Phillip Godorov, Director
Phone: 610-832-9715
Fax: 610-834-7011
E-mail: [email protected]
Caitlin Farrell, Project Manager
Phone: 610-832-9746
Fax: 610-834-7076
Email: [email protected]
Interlaboratory Study Program
ASTM International
100 Barr Harbor Drive
West Conshohocken, PA
19428-2959