calibration fundamentals
DESCRIPTION
Calibration FundamentalsTRANSCRIPT
-
Calibration
Fundamentals
An
Introduction to
Metrology
Presented by Jol Rameaux
Service & Support Unit
Agilent Technologies Europe
October 24, 2013
Calibration Fundamentals
1
-
October 24, 2013
Calibration Fundamentals
2
Calibration Fundamentals Todays topics
Theres much more beneath the surface !
Contents
Metrology definition Measurements
confidence
What is calibration? Cal comparison Iceberg ahead ! How to calibrate Calibration of Agilent
equipment
Errors and uncertainty Summary
An
Introduction
to Metrology
-
The Importance of Measurement Metrology is the science of measurement
October 24, 2013
Calibration Fundamentals
3
British scientist Lord Kelvin (William Thomson
1824-1907) is quoted from his lecture to the
Institution of Civil Engineers, 3 May 1883...
"I often say that when you can measure what you are
speaking about and express it in numbers you know
something about it;
but when you cannot express it in numbers your
knowledge is a meagre and unsatisfactory kind"
You cannot manage what you cannot measure. HP co-founder Bill Hewlett
-
Metrology : Wiki Definition
October 24, 2013
Calibration Fundamentals
4
Metrology is the science of measurement.
Metrology includes all theoretical and practical aspects
of measurement. Applied metrology concerns the application of measurement
science to manufacturing and other processes and their use in
society, ensuring the suitability of measurement instruments,
their calibration and quality control of measurements. Although
the emphasis is on the measurements themselves, traceability
of the calibration of the measurement devices is necessary to
ensure confidence in the measurements.
Metrologist : Perform metrology
-
The Need for Good Measurement
October 24, 2013
Calibration Fundamentals
5
We all make and use measurements
every day How much fuel will I get? What weight of fruit does the pie need? Will this nut fit the bolt? Does your mobile phone reliably connect to any network? Which drug is most effective? What is the time?
We have expectations
about the correctness of
these measurements
-
What are Specifications? Define a products expected performance
October 24, 2013
Calibration Fundamentals
6
An example using a familiar product
Manufacturers view
Frequency: 900 MHz
Channel spacing: 200 kHz
Max power: 2 W
Rate/Modulation: 270 kbps/GMSK
Sensitivity: -97 dBm
etc.
Reason: Conforms with legal criteria (safety, interference); compatible with
operators networks.
Consumers view Reception quality
Reason: Can hear clearly when driving(*) a
car, while in an elevator
or walking through an
underpass.
(*) Via Bluetooth hands-free, of course!
-
Why Measure?
October 24, 2013
Calibration Fundamentals
7
We take measurements to get information about the
manufactured or used product so that we can make
good decisions.
Measure the product
to see if it conforms
with specs or
requirements
We have a
good product Yes
But how do we know our
measurement tools are
giving good results?
No
Does it
comply?
Measurement decisions
are made during:
Research Product development Production Final testing Operation
-
Measurement Confidence
October 24, 2013
Calibration Fundamentals
8
Measurements are made using a wide variety of
instruments, ranging from very simple to extremely
sophisticated.
We must have confidence that all those measuring
instruments are giving valid, useful data.
Because measured values are the basis of decisions.
That confidence is provided by an
effective program of calibration
-
Calibration Confidence
October 24, 2013
Calibration Fundamentals
9
Reference Equipment used during calibration should be
calibrated and should match or exceed accuracy expectation.
Procedures used during calibration should be clear, validated
and repeatable.
Measurement results should be given at minimum as a
Pass/Fail result and preferably with measurement data and
uncertainties associated
Existing Traceability toward reference standards
People should know what they are doing
Preference should be given to automated calibration as it
increases confidence in overall calibration procedure
-
So, what is calibration ? (1)
October 24, 2013
Calibration Fundamentals
10
A graphic illustration search Google Images for meal and see the variety
Are they all identical ?
-
So, what is calibration ? (2) Definition
October 24, 2013
Calibration Fundamentals
11
Operation that, under specified conditions, in a first step establishes a relation between the quantity values with measurement
uncertainties provided by measurement standards and corresponding
indications with associated measurement uncertainties and, in a second
step, uses this information to establish a relation for obtaining a
measurement result from an indication.
From the ISO/IEC Guide 99 : 2007 -- International Vocabulary of Metrology
Or more simply...
The process of comparing a device with unknown performance to measurement standards traceable to the SI units and with determined
uncertainties to enable obtaining thereafter useful measurement results
from the device.
There is no detail on what should be done during calibration.
-
Calibration Provides Confidence
October 24, 2013
Calibration Fundamentals
12
Confidence that a devices past performance was valid and that its future performance will continue to be good within
recommended calibration interval.
Unacceptable past performance may necessitate customer alert and product recall.
Due to drift, measuring equipment needs to be regularly calibrated to contain error to
acceptable limits.
Knowledge of the value of a calibrated parameter becomes less certain as time
elapses since calibration.
Some products require annual optimisation (adjustment) for high confidence of
compliance in future.
Time P
ara
mete
r valu
e
Performance
drift
Calibration &
adjustment
Spec limit
Calibration &
adjustment
-
Verification, Calibration, Adjustment, Repair
October 24, 2013
Calibration Fundamentals
13
Specification Limit of normal equipment operation, defined by original manufacturer
Verification Test versus specifications. Result is Pass or Fail. Verification should include a data report with failed points, if any, and possibly measurement uncertainty.
Adjustment Mechanical or electrical tuning of equipment to bring it back within specifications.
Adjustment can be done in case of Failed verification or Passed verification (adjust always equipment). Adjustment should include a as-received data report and a post-
adjusted data report.
Calibration Verification + Adjustment combined (if needed)
Repair If equipment fail verification and cannot be adjusted, then unit is faulty and should be
repaired. Adjustment is not a repair !
-
Calibration Performance
Real data comparison MXG RF Digital Signal Generator
No
rma
lize
d M
eas
ure
d V
alu
e
Po
wer
An
alo
g
Po
we
r
Dig
ita
l
Ha
rmo
nic
Sp
uri
ou
s
Su
b
Ha
rmo
nic
No
n
Ha
rmo
nic
Dig
ita
l
Mo
du
lati
on
10 M
Hz
Fre
qu
en
cy
W-C
DM
A
Po
we
r
Ou
tpu
t
Po
we
r
Max
Min
Max
Min
Ag
ile
nt
Ca
l N
on
-Ag
ile
nt
Cal
N5182A Dig
ita
l M
od
Po
we
r
820 POINTS TESTED
257 POINTS TESTED
-
Calibration Performance
Real data comparison MXG RF Digital Signal Generator
No
rma
lize
d M
eas
ure
d V
alu
e
Po
wer
An
alo
g
Po
we
r
Dig
ita
l
Ha
rmo
nic
Sp
uri
ou
s
Su
b
Ha
rmo
nic
No
n
Ha
rmo
nic
Dig
ita
l
Mo
du
lati
on
10 M
Hz
Fre
qu
en
cy
W-C
DM
A
Po
we
r
Ou
tpu
t
Po
we
r
Max
Min
Max
Min
Ag
ile
nt
Ca
l N
on
-Ag
ile
nt
Cal
N5182A Dig
ita
l M
od
Po
we
r
Not measured (long or difficult tests)
-
Calibration -- The Iceberg Analogy
October 24, 2013
Calibration Fundamentals
16
The calibration certificate and
measurement data report
What the customer sees
-
Calibration -- The Iceberg Analogy More to it than meets the eye
October 24, 2013
Calibration Fundamentals
17
The calibration certificate and
measurement data report
What the customer sees
What the customer doesnt see Calibration procedure, software
Measurement adequacy
Traceability
Equipment recall system
Records
Labelling for identity & status
Interval management
Environment
Supplier evaluation
Technical competence
-
How to deliver Calibration? (1)
Calibration is verifying instrument specifications by measuring actual performance
using external lab standards [instruments] that in turn have better performance, and
are traceable to International Standard (NIST, NPL, PTB and NMIJ).
Agilent measures the actual
performance of every warranted
specification, for every installed
option, every time using same
procedure and including adjustments
You need :
- Systems/ETEs used as reference
- To follow OEM recommendations
- Appropriate Procedures
- Trained People
- Cal Laboratory
- To archive results
-
How to deliver Calibration ? (2)
October 24, 2013
Calibration Fundamentals
19
Use OEM Calibration Labs
Built your own cal lab
Bring Cal Lab on your site(s)
Mix of all above
-
How to deliver Calibration ? (3)
October 24, 2013
Calibration Fundamentals
20
TME Software Calibration Platform
Check ETEs (reference) needed
Check ETEs cal due date
Compute uncertainties as a function of
ETEs used
Define tests to be run as a function of configuration (option, frequency, etc), test operator setup
Give Pass/Fail summary and data reports with points out of specs
Print clean data report
Archive data and traceability
Validated by Agilent (operation, data points, uncertainties)
-
Calibration of Agilent Equipment?
October 24, 2013
Calibration Fundamentals
21
Automatic Calibration with commercialy available software Reliable & Repeteable Automatic configuration check and uncertainty measurement Most new equipment
Automatic Calibration with proprietary software Reliable & Repeteable Automatic configuration check and uncertainty measurement Older equipment
Manual Calibration Rely on strickly following recommended test procedure Highly operator dependant Very old equipment, no computer interface
No-Calibration required Accessories, cables, adapters
30 %
40 %
20 %
10 %
-
What Needs Calibrating?
October 24, 2013
Calibration Fundamentals
22
Any equipment used in incoming, final inspection, in-process or quality control (SPC), product calibration/testing.
Equipment used to perform lengthy or costly machine set-up.
Items whose indications impact upon controlled conditions (e.g. environment).
Any situation where the failure costs are too high (safety, one off testing, etc).
Equipment used in R&D where design decisions are made.
Used during failure or returned material analysis, where critical determinations could be affected (including repair troubleshooting).
Whenever someone acts upon an instrument's reading.
Equipment used for calibration (traceability) of any of the equipment used in the above situations.
In general, all equipment who are used as reference for other items
-
Error & Uncertainty
October 24, 2013
Calibration Fundamentals
23
Error is the result of a measured
value minus the expected or
nominal value of the quantity. The
purpose of calibration is to
determine the magnitude of (and
possibly correct for) errors.
Error is by how much we miss the
target (i.e. the red bulls-eye).
The dispersion (spread of shots)
represents repeatability a component of uncertainty. Shots at a target
another illustration of error
Outlier
Average
error
Negative
error
Positive
error
Expected
value Measured
value
Measured
value
-
Error, Precision & Accuracy
October 24, 2013
Calibration Fundamentals
24
Accurate and precise
Precise but inaccurate Inaccurate and imprecise
Quite accurate but imprecise
Accurate = True
Precise = Repeatable
Note: In reality we need lots of
data to determine the distribution
(5 shots is not enough).
Tru
e v
alu
e
Tru
e v
alu
e
Tru
e v
alu
e
Tru
e v
alu
e
Bad reference equipment
Bad methodoloy
Bad approach
State of the art
-
Concepts of Uncertainty Example : How long is a piece of string?
October 24, 2013
Calibration Fundamentals
25
Contributors
Tape measure
Manufacturing tolerance
Bent or stretched
Temperature / humidity
Graduation resolution
String
Lying straight / flat
Under / Over-stretched
Temperature / humidity
Frayed ends
Method & Personnel
Lining-up string & tape measure
Horizontal or vertical
Repeatability
Eye report on graduation
-
Concepts of Uncertainty Sources of error & uncertainty
October 24, 2013
Calibration Fundamentals
26
Measuring equipment
Specifications inc. imported uncertainty (traceability)
Readability
Properties of the thing being measured
Stability
Sensitivity to input change
Readability
Measurement method
Use of correctional data
Configuration
Data processing
Operator skill
Reaction time
Manual dexterity
Visual ability (meter parallax)
Environment
Temperature
Humidity
Altitude / Barometric pressure
EMI
Impedance effects
Mismatch
Loading (e.g. local / remote sensing)
Cables and connections
Torque
Materials
Cleanliness
Mechanical stress
Measurement non-repeatability
Due to properties of the test equipment and unit-under-test and also the variable influences
of test environment and operator.
Mistakes !
Cant be accounted for but can be statistically identified (outliers).
-
Uncertainty Example : Power sensor cal factor
October 24, 2013
Calibration Fundamentals
27
Erse
te
Mi
Mt
Ms
MiKsKsKb
2
2
))(1(
))(1(
2
1)(
Measurement equation
-
Uncertainty calculation for a device
October 24, 2013
Calibration Fundamentals
28
One spreadsheet/calculation to be created for each measurement setup and adjusted (uncertainty) for each measured point
-> Give uncertainty per measured point
-> Imply Pass/Fail criteria
Same analysis to be done for each measurement point
Same analysis to be done for each setup (alternate replacement)
Remember the MXG : 820 measurement points for 10 performance
setup ! Not including alternate ETEs analysis.
-> This analysis and validation is the pure role of the metrologist
-
Basic Model for Product Specification Where does uncertainty fit in?
October 24, 2013
Calibration Fundamentals
29
Mean Production line
test limit
Production
process
distribution
(manufacturing
reproducibility)
Production
margin
Calibration
uncertainty
Delta environment
and drift
Specification
Guardband
Time
Para
mete
r valu
e
Performance
drift
Calibration &
adjustment
Spec limit
-
Calibration Intervals
October 24, 2013
Calibration Fundamentals
30
May be expressed as a calendar period, usually months or years
between calibrations.
Or might be an interval of usage time such as 1000 hours of operation.
Or possibly the number of usage operations, especially mechanical
items subject to degradation through
wear.
Although a manufacturer may recommend an interval (the spec
implicitly has time dependency
owing to expected drift), the
equipment user is responsible for
assigning the appropriate
interval. Most of users follow
manufacturers recommended CI
Aim is to balance the cost of calibration (how often and the
extent of the testing) with the
risk of bad measurement (cost
of product recall, safety, etc.)
Better do a complete calibration every 3 years than a bad
calibration every year
-
Calibration Summary What you need to remember
October 24, 2013
Calibration Fundamentals
31
Only manufacturer can define specs and what need to be tested.
All calibration are not equivalent. Compare what is really done to have a complete picture : function, number of points and accuracy.
Calibration certify past performance was correct and future performance is expected to be good
Some calibrations can be done by yourselves, some cannot.
Agilent is ready to help, in performing on-Site calibration for reduced down time and minimize risk in shipment
You can tune calibration interval. It is your decision, but you should be able to justify in front of auditors.
Metrology is really a specific and full time job reserved to specialists
Any question : [email protected] or contact Boris at IBIS
-
Calibration Summary
October 24, 2013
Calibration Fundamentals
32
Questions ?
-
Calibration Summary
October 24, 2013
Calibration Fundamentals
33
Thanks for your attention !
-
Metrological [measurement] Traceability Definition
October 24, 2013
Calibration Fundamentals
34
Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of
calibrations, each contributing to the measurement uncertainty.
From the ISO/IEC Guide 99 : 2007 -- International Vocabulary of Metrology
Or more simply as...
A series of calibrations of known accuracy that link measurements to recognised standards through an
unbroken chain of comparisons.
The need for records is to link a product or service event to a process, status or time.
If the process is measurement, detail about the equipment, procedure, test conditions and personnel are
all relevant to demonstrating traceability.
-
Traceability International System of Units (SI units)
October 24, 2013
Calibration Fundamentals
35
http://physics.nist.gov/cuu/Units/index.html
Every measurement
is linked to one or
more of just 7 base
units !
The kilogramme is the
only unit defined by a
prototype (a specific physical item held at
BIPM in Paris, France).
Definitions (at NIST site)
-
Measurement Traceability A representation of the national measurement system
October 24, 2013
Calibration Fundamentals
36
NATIONAL
METROLOGY INSTITUTE
STANDARDS
LABORATORY
CAL FACILITY
MANUFACTURING
PRODUCT SPEC
Uncertainty is the potential inaccuracy.
The levels do not represent consistent values of uncertainty.
Test uncertainty ratio (TUR) is the ratio of the unit-under-test's
specification to the measurement uncertainty.
Spec = 15%
TUR 5:1
Unc = 3%
TUR 10:1
Unc = 0.3%
TUR 3:1
Unc = 0.1%
TUR 2.5:1
Unc =
0.04%
NATIONAL
REALISATION OF
SI DEFINITION Unc =
0.008% International
comparisons
via BIPM
International
comparisons
via regional
accreditation
organisation
Another
stds lab
Another
cal lab And another
Another
production line
Another
factory And another
-
Traceability Why Bother? Summary
Common measurement references are critical to the world-wide
exchange and acceptance of
products, services and technology.
The aim is to transfer these references in a controlled manner
to the billions of individual
measurements made daily.
Traceable measurements ensure uniformity of manufactured goods
and industrial processes.
It supports equity in trade, compliance to regulatory laws and
standards.
If the calibration paper trail was followed from the unit under test up
through the working, reference and
primary standards, it would
eventually end at the record of an
experiment made to establish the
quantity in terms of one or more of
the SI base units.
It is essential to science and the development of technology.
October 24, 2013 37
Calibration Fundamentals
-
Recommended Resources (Free) Find out more
October 24, 2013
Calibration Fundamentals
38
NPL Good Practice On-line Modules on:
Measurement http://www.npl.co.uk/publications/good-practice-online-modules/measurement/
Uncertainty http://www.npl.co.uk/publications/good-practice-online-modules/measurement-uncertainty/
NPL -- A Beginner's Guide to Uncertainty of Measurement http://resource.npl.co.uk/cgi-bin/download.pl?area=npl_publications&path_name=/npl_web/pdf/mgpg11.pdf
UKAS M3003 -- The Expression of Uncertainty and Confidence in
Measurement http://www.ukas.com/library/Technical-Information/Pubs-Technical-Articles/Pubs-List/M3003.pdf
-
In or Out-of-Spec?
October 24, 2013
Calibration Fundamentals
39
But it doesnt define how.
Most accreditation bodies (all of Europes) stipulate ILAC-G8 which requires conformity limits are set at spec minus uncertainty.
The US national implementation of 17025 is ANSI/NCSL-Z540.3 which
accepts a producer/consumer shared-risk approach:
The probability of false acceptance associated with any test point labelled Pass shall not exceed 2 %.
Learn more at: http://www.agilent.com/metrology/ncsli2009_dobbert.shtml
Measured values
with plus and minus
uncertainty bars
Ideal
value
Specification
limit A
B C
D E
ISO/IEC 17025 clause 5.10.4.2 :
When statements of compliance [with a specification] are made, the
uncertainty of measurement shall be
taken into account.
A= Pass
B,C,D = Indeterminate
(Agilent treats D as Fail)
E= Fail
International Laboratory Accreditation Cooperation (ILAC) G8 : Guidelines on the Reporting of Compliance with Specification