Download - Analyzers Accuracy
-
8/8/2019 Analyzers Accuracy
1/15
Description of Analyzerperformance
(Accuracy & Precision)
-
8/8/2019 Analyzers Accuracy
2/15
Accuracy & Precision
What do our customers expect from our analyzers?
How will they convince themselves?
0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25 30 35 40 45
Accepted Standard Laboratory test
FT-NIRA
nalyzerresult
Lab error
Analyzer error
Measure a single sample many times
-
8/8/2019 Analyzers Accuracy
3/15
Testing the performance with a single sample
0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25 30 35 40 45
Accepted Standard Laboratory test
F
T-NIRA
nalyzerresult
Lab error
Analyzer error
Error in repeated measurements= Repeatability
Below we see different results for the FT-NIR analyzer and the AcceptedStandard laboratory test
We can see that the analyzer agrees with the Accepted Standard
Laboratory test because both are centered at 25.
-
8/8/2019 Analyzers Accuracy
4/15
Accuracy
Common (old) Definition of accuracy Accuracy is measure of deviation from true value.
Who knows the true value!!
Replace Accuracy with Agreement (NIST directive)
Error in agreement between analyzer
result and result obtained with acceptedstandard reference method.
-
8/8/2019 Analyzers Accuracy
5/15
Agreement
How do we determine it?
Measure a sample or process product with FT-NIR analyzer and in
laboratory using accepted standard method
Compare the results
For one measurement result can be offset or error
Repeat procedure many times
Many = at least 20 times
20 comparisons using the same sample 20 comparisons tested by same operator on same day
20 comparisons with different samples on different days tested in (different)
lab by different operators
-
8/8/2019 Analyzers Accuracy
6/15
Agreement
Test Agreement between analyzer and Accepted Standard Laboratory test
with many samples
y = 0.999x + 0.014
20
22
24
26
28
30
32
34
36
38
20 22 24 26 28 30 32 34 36 38
Accepted Standard Laboratory test
FT-NIR
analyzer
-
8/8/2019 Analyzers Accuracy
7/15
Estimation of agreement error
0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25 30 35 40 45
Accepted Standard Laboratory test
FT-N
IRA
nalyzerresult
Combined error
The agreement error between analyzer and Accepted Standard Method is the vector
sum of the repeatability of each.
The agreement error is always greater than the repeatability of the the analyzer or theAccepted Standard Method
When repeatability of both are same, agreement= 1.41x error of Accepted Standard
Method
-
8/8/2019 Analyzers Accuracy
8/15
Summary on Agreement
(guaranteed) specification for agreement between our analyzer and the AcceptedStandard Method
We must know the repeatability of the Accepted Standard Method.
As estimated in the protocol
As used by the customer
For several operators, different samples, different labsetups etc.
It is best to obtain real up to date statistical data from the customer Use work sheet for determination of standard error of duplicate determinations
(Pooled standard deviation)
We must also know the repeatability of the analyzer.
Often much better than Accepted Standard Method. Rule of Thumb1: Check performance of customer lab carefully
Rule of Thumb2: Leave some margin in guaranteed spec.
Quote between 2 and 1.5 times the Standard error of duplicate determinations of
Accepted Standard Method.
-
8/8/2019 Analyzers Accuracy
9/15
Summary on Agreement (cont.)
Agreement specification is needed when product is tested with Accepted
Standard Method.
By Government inspector to comply with regulatory quality
By customer to provide custody transfer certification By customer to insure product in process quality
You can see with term agreement, we are not bound by Accepted
Standard Method.
We can establish agreement specification between any 2 methods of
analysis as long as we know their repeatability spec.
-
8/8/2019 Analyzers Accuracy
10/15
Repeatability and Reproducibility
Repeatability is specification of error for repeat determinations with allconditions the same
Formerly called precision
Reproducibility is specification of error for repeat determinations wheresome conditions may be different
Different operators
Different labs
Different Spectrometers
Our specification of absorbance reproducibility fromspectrometer to spectrometer gives us the possibility ofcalibration transfer
-
8/8/2019 Analyzers Accuracy
11/15
Summary on Repeatability & Reproducibility
Repeatability indicates how precise an analyzer can analyze theproduct
The repeatability error is commonly much smaller than the agreement
error In process monitoring it may be more important than agreement
It insures that the product is always the same to within a small error.
Reproducibility of our analyzers is important for customers who want toproduce the same product at different sites.
-
8/8/2019 Analyzers Accuracy
12/15
Agreement
OHvalue
40
45
50
55
60
65
70
75
80
85
40 45 50 55 60 65 70 75 80 85
Accepted Standard lab method
FT-NIR
analyzer
RMSD=0.7 (2 Factors)
-
8/8/2019 Analyzers Accuracy
13/15
Repeatability
OH value
70
71
72
73
74
75
76
77
78
79
80
70 71 72 73 74 75 76 77 78 79 80
Actual
Predicted
cal samples
Validation
Linear (cal samples)
Repeatability RMSD 0.05
-
8/8/2019 Analyzers Accuracy
14/15
Repeatability
control chart 10 weeks
y = 0.0002x + 57.202
56
56.2
56.4
56.6
56.8
57
57.2
57.4
57.6
57.8
58
0 20 40 60 80 100 120 140
days
O
H
value
RMSD= 0.05
-
8/8/2019 Analyzers Accuracy
15/15
Reproducibility
System to System Reproducibility
55
56
57
58
59
60
1 2 3 4 5 6
FTLA2000-160 unit
OH
value
STDEV (from unit to unit) = 0.11