applying metrology to the limitations of breath testing ... · applying metrology to the...

The 2016 Criminal Justice Institute – August 22 & 23, 2016

SESSION 301

Applying Metrology to the Limitations of Breath Testing – Using the Government

Mule to Plow Your Field

Daniel J. Koewler Charles A. Ramsay Ramsay Law Office

Roseville

Minnesota CLE’s Copyright Policy

MINNESOTA CLE is Self-Supporting

A not for profit 501(c)3 corporation, Minnesota CLE is entirely self-supporting. It receives no subsidy from State Bar dues or from any other source. The only source of support is revenue from enrollment fees that registrants pay to attend Minnesota CLE programs and from amounts paid for Minnesota CLE books, supplements and digital products.

© Copyright 2016

MINNESOTA CONTINUING LEGAL EDUCATION, INC.

ALL RIGHTS RESERVED

Minnesota Continuing Legal Education's publications and programs are intended to provide current and accurate information about the subject matter covered and are designed to help attorneys maintain their professional competence. Publications are distributed and oral programs presented with the understanding that Minnesota CLE does not render any legal, accounting or other professional advice. Attorneys using Minnesota CLE publications or orally conveyed information in dealing with a specific client's or other legal mat-ter should also research original and fully quoted sources of authority.

Minnesota Continuing Legal Education wants practitioners to make the best use of these written materials but must also protect its copyright. If you wish to copy and use our CLE materials, you must first obtain permission from Minnesota CLE. Call us at 800-759-8840 or 651-227-8266 for more information. If you have any questions about our policy or want permission to make copies, do not hesitate to contact Minnesota CLE.

All authorized copies must reflect Minnesota CLE’s notice of copyright.

Applying Metrology to the Limitations of Breath Testing – Using the Government Mule to Plow Your Field

Daniel J. Koewler & Charles A. Ramsay

Table of Contents

Fredrickson Order re Kevin Hunt Gullberg Summary Starr Order re Mahoney McCarthy & McMahon BT-023 - Estimation of Measurement Uncertainty for DataMaster DMT-G (DMT) Breath Test Results

1

Measurement Uncertainty Computed for the

Forensic Breath Alcohol Test Program in Minnesota

Our objective here is to develop and illustrate the calculation of measurement uncertainty in forensic

breath alcohol analysis for the forensic breath alcohol test program in Minnesota. We will follow the

approach outlined in the Guide to the Expression of Uncertainty in Measurement (GUM) document. [1] This

document is internationally recognized as an approach to estimating, interpreting and reporting

measurement uncertainty. We begin by identifying the principle components contributing to measurement

uncertainty. For the program in Minnesota these three components include: (1) breath sampling, (2)

reference standard traceability and (3) measurement bias. The standard deviation (standard uncertainty)

was estimated for each of these components from data received from the Minnesota Breath Alcohol Test

Program. A spreadsheet was developed in Microsoft Excel V 8.0 (Microsoft, Inc., Redmond, WA) which

allows for the entry of data for a particular breath alcohol test result and then computes the measurement

uncertainty as a 99% confidence interval.

Computing the Uncertainty Function

The state of Minnesota employs the Datamaster DMT (Intoximeters, Inc., St.Louis, MO) breath alcohol

test instrument in which the alcohol is quantified by infrared absorption. Duplicate breath test data from

actual arrested and tested subjects was received from the Minnesota program in April 2016. These included

n=35,793 sets of duplicate test results all reported to four digits. The means of duplicates ranged from 0 to

0.4261 g/210L with an overall mean result of 0.1569 g/210L. The data were from drunk driving subjects

arrested during the years 2014 and 2015. The uncertainty function was computed using the statistical

program R Ver. 3.2.1. The R code is found in the Appendix. The standard deviation was determined for all

values within each 0.010 g/210L interval. Three digit breath test data was used (truncating the fourth digit)

because that is how breath test results are reported on the test document. The standard deviation was

computed from:

1.2

1

2

Eqk

d

S

k

i

i

where: d = the difference between duplicate test results

k = the number of individuals within the specific 0.010 g/210L interval

These estimates for the standard deviation are then plotted against the midpoint for each concentration interval (0.010 g/210L) seen in figure 1. A linear regression model is developed using all duplicates including

2

those with differences greater than 0.020 g/210L. Only duplicate data having mean results ≥ 0.020 g/210L were used in this first computation (figure 1) to ensure that each interval had at least n=30 duplicates from which to estimate the standard deviation. The linear uncertainty function is then incorporated into the Excel spreadsheet for computing the combined uncertainty. The value C in the uncertainty function represents the mean breath alcohol results for the individual’s test. All duplicate test results which did not agree within 0.020 g/210L were then removed and the uncertainty function was computed for this reduced set of data as well. A total of 1.3% (n=468) of the full data set did not meet the 0.020 g/210L agreement standard and were removed for the re-analysis. The results of this uncertainty function using only those results with differences less than or equal to 0.020 g/210L is seen in figure 2. This computation of the uncertainty function based on duplicate breath test results is very important and demonstrates the increasing variation with concentration. [2,3] It will include most of the components contributing to total uncertainty. Some of these components include: (1) breath sampling (possibility of low level mouth alcohol or interfering substances, breath temperature, partition ratio, etc.), (2) analytical component, (3) operators (instructions to subject, etc.), (4) over 30,000 subjects, (5) two years of time, (6) 280 instruments, (7) environmental elements, (8) ambient conditions, (9) health conditions etc. All of these contribute to the variation observed in duplicate test differences and are captured with the computation and resulting linear function observed in figures 1 and 2.

3

Figure 1 – The uncertainty function using all duplicate data where the instrument accepted the results and differences may have exceeded 0.020 g/210L

SD = 0.0222C + 0.0010n = 35,398

4

Figure 2 – The uncertainty function using only duplicate results that complied with the 0.020 g/210L agreement criteria

From figures 1 and 2 we see that using all of the data (figure 1) yields a slightly larger estimate for the standard deviation. At a concentration of 0.085 g/210L the model in figure 1 would yield a standard deviation of 0.00287 g/210L while the model in figure 2 would yield a standard deviation of 0.00282 g/210L. The difference is negligible. However, the more conservative estimate from figure 1 will be used in the Excel spreadsheet computation of combined uncertainty.

Excel Spreadsheet and Computations

We begin by entering the duplicate breath test results for a specific individual on lines 1 and 2. The mean is computed (line 3) and then entered into the uncertainty function on line 4. The coefficient of variation squared (CV2) for this breath sampling term is then computed from the standard deviation, n and the mean of the individual results. The reference value (line 5) is the value of the gas standard used in the field and measured with each subject test. These standards are purchased from a vendor and their reference value is

SD = 0.0166C + 0.0014n = 34,962

5

assigned based on a barometric pressure correction (0.077 g/210L in our example). The uncertainty estimate from the certificate of analysis (assumed here to be 0.001 g/210L on line 5) will then be divided by the square root of the number of measurements performed by the vendor on the reference material (if known) or divided by at least the square root of two. This is finally divided by the reference value (0.077 g/210L on line 5) and squared to yield the CV2 for the standard reference material. The third term found on line 6 is for the bias component. We recommend that the maximum observed bias be the fixed value of 0.005 g/210L. This is the maximum absolute bias observed for 98.8% of the gas standard measurements in the field for the two years of 2014 and 2015. Using the field gas standard measurements for the bias estimates is preferred to those control standard measurements performed in the laboratory during annual calibration procedures because they are more representative of the large variation in environmental and analytical conditions experienced in the field throughout the year. This maximum absolute bias is assumed to follow a uniform distribution. The standard deviation for this distribution is determined from half the interval width (the maximum observed bias) divided by the square root of three. This is then divided by the control value or mean gas standard measurement associated with the duplicate test results (assumed here to be 0.076 g/210L on line 6) and squared to yield the CV2 value. These three CV2 values are then added (line 7) and the square root taken which is multiplied by the mean of the individual’s test results (line 8). This finally yields the combined uncertainty on line 8 which in our example is 0.00389 g/210L. This combined uncertainty is then used to compute the 95% (line 9) and 99% (line 10) confidence intervals. For the 95% confidence interval we have used a coverage factor of 1.96 which yields an exact 95% confidence interval. For the 99% confidence interval we have used a coverage factor of 2.575 which provides an exact 99% confidence interval. The far right column indicating Percent shows the percent that each of the three terms contributes to the combined uncertainty. Where one of the terms contributes less than one-third of the largest term, this term can be reasonably eliminated. [4] This would be the reference standard control value. However, given the forensic context it is advisable to retain all three terms in the calculations. The values that will need to be added for each uncertainty estimation are highlighted in yellow on the Excel spreadsheet.

The computations described above for computing the combined uncertainty are found in the equation 2 below:

00390.00760.0

3

005.0

077.0

2

0010.0

085.0

2

00289.0

085.0

2.Re

3

Re

222

22

Re

Re

2

c

f

f

Breath

Sampling

c

u

Eqf

Bias

f

n

u

Y

n

u

Yu

6

The first term under the square root in equation 2 comes from the uncertainty function (line 4), the second term from the reference value (line 5) and the third term from the bias component (line 6).

Plot of Differences

Figure 3 shows the plot of absolute differences against their mean. This includes all data, including those where the 0.020 g/210L agreement was not met. From this we see that most of the results not complying with the 0.020 g/210L agreement are above 0.10 g/210L. This indicates, as do the uncertainty functions in figures 1 and 2, that the variation is increasing with concentration. This is indicative of a multiplicative error model and is very common in the analytical sciences.

Figure 3 – Plot of the absolute duplicate test differences against their mean. The agreement criteria of 0.020 g/210L is shown by the horizontal line

Mean of Duplicate Results (g/210L)

0.0 0.1 0.2 0.3 0.4

Abs

olut

e D

uplic

ate

Tes

t D

iffer

ence

s (g

/210

L)

0.00

0.02

0.04

0.06

0.08

n = 35,792

7

Figure 4 shows the plot of differences against their mean which is known as a Bland-Altman plot. [5] The difference is determined from BrAC1-BrAC2. We look for trends in this plot and for systematic differences as measured by the mean difference. The data look very uniform throughout the range with no indication of a trend. The mean difference of 0.0012 g/210L does not suggest that one result is systematically different from the other. From this plot we also see the increasing variation with concentration.

Figure 4 – Plot of duplicate test differences (BrAC1-BrAC2) against their mean

Figure 5 shows the frequency distribution of the bias estimates from the full data set received. The bias estimates ranged from -0.0078 (-10.3%) to 0.0075 (+9.9%) g/210L. The distribution is quite normal in appearance.

Mean BrAC Result (g/210L)

0.0 0.1 0.2 0.3 0.4 0.5

Dup

licat

e T

est

Diff

eren

ce (

BrA

C1-

BrA

C2)

g/2

10L

-0.08

-0.06

-0.04

-0.02

0.00

0.02

0.04

0.06

0.08

n = 35,792

mean = 0.0012 g/210L

mean + 2SD = 0.0142 g/210L

mean - 2SD = -0.0118 g/210L

8

Figure 5 – Frequency distribution of the bias values observed for all data during 2014-2015

When The Confidence Interval Brackets a Critical Per Se Level

Assume that an individual arrested for DUI provides two breath samples resulting in 0.083 and 0.087

g/210L. The combined standard uncertainty associated with breath alcohol measurement at this

concentration is σ = 0.00389 g/210L determined from the example illustrated in the Excel spreadsheet. We

use the large sample Z statistic to compute the 99% confidence interval and determine the probability that

the individual’s true mean breath alcohol concentration exceeds 0.080 g/210L.

We compute the 99% confidence interval as follows:

Duplicate Test Difference (g/210L)

-0.008 -0.006 -0.004 -0.002 0.000 0.002 0.004 0.006 0.008

Fre

quen

cy

0

2000

4000

6000

8000

10000

12000

mean = -0.0007 g/210L SD = 0.0018 g/210L n = 35,793

9

0950.00750.00100.00850.000389.0575.20850.0995.0 toSZYY

The probability that the individual is over 0.080 g/210L is found by first considering the following form for

expressing the confidence interval showing the probability that µ is bracketed by upper and lower limits:

= SZ + Y SZ -Y PY/2)-(1Y/2)-(1

The value π simply represents the probability that we are interested in determining and Y represents the mean breath alcohol results for the subject. Since we are interested in determining the probability that µ exceeds the lower limit we rewrite the equation as follows:

= SZ Y P

Y/2)-(1

We now set the lower limit equal to 0.080 g/210L and solve for Z(1-α/2):

28.1000390.050 = Z .080 = Z 0.08 0.080 = SZ Y /2)-(1/2)-(1Y/2)-(1

Next, we rearrange our probability statement and introduce the value for Z(1-α/2):

0.080

0.0750 0.0950

10

899728.12/1 0.= Z P= ZZ P= ZS

-Y P= SZ Y P /2-1

Y

Y/2-1

The probability that the true mean breath alcohol results exceeds 0.080 g/210L is 0.8997. Whether this is

evidence beyond a reasonable doubt is for the jury to decide.

Based on the analyses presented above it is recommended that the uncertainty function, the maximum observed bias, the vendor’s estimate for the uncertainty of the gas standard and the gas standard target value be reviewed annually. Any changes in these values should be incorporated in the Excel computation spreadsheet. In addition, it is recommended that the program policy manual define which set of spreadsheet computations will apply to a particular individual’s test results. In addition, it is recommended that the program policy manual explain the meaning of the values found in the Excel spreadsheet.

Appendix – R Code for developing the Uncertainty Function

Routine for importing duplicate breath alcohol data and then separating into bins of 0.01 based on duplicate means and then computingestimates of the standard deviation and plotting against the bin mid-points data=read.table("c:/Minnesota Breath Test/dat8.csv", header=TRUE, sep=",") attach(data) mn=(brac1+brac2)/2 d=brac1-brac2 ds=d^2

11

conc=c(seq(.005,.325,.01)) factor=(mn<.01)+(mn<.02)+(mn<.03)+(mn<.04)+(mn<.05)+(mn<.06)+(mn<.07)+(mn<.08)+(mn<.09)+(mn<.10)+(mn<.11)+(mn<.12)+(mn<.13)+(mn<.14)+(mn<.15)+(mn<.16)+(mn<.17)+(mn<.18)+(mn<.19)+(mn<.20)+(mn<.21)+(mn<.22)+(mn<.23)+(mn<.24)+(mn<.25)+(mn<.26)+(mn<.27)+(mn<.28)+(mn<.29)+(mn<.30)+(mn<.31)+(mn<.32)+(mn<.33)+(mn<.34) table(factor) sum(ds[factor==2]) s33=sqrt(sum(ds[factor==2])/(2*(length(mn[factor==2])))) sum(ds[factor==3]) s32=sqrt(sum(ds[factor==3])/(2*(length(mn[factor==3])))) sum(ds[factor==3]) s31=sqrt(sum(ds[factor==4])/(2*(length(mn[factor==4])))) sum(ds[factor==5]) s30=sqrt(sum(ds[factor==5])/(2*(length(mn[factor==5])))) sum(ds[factor==6]) s29=sqrt(sum(ds[factor==6])/(2*(length(mn[factor==6])))) sum(ds[factor==7]) s28=sqrt(sum(ds[factor==7])/(2*(length(mn[factor==7])))) sum(ds[factor==8]) s27=sqrt(sum(ds[factor==8])/(2*(length(mn[factor==8])))) sum(ds[factor==9]) s26=sqrt(sum(ds[factor==9])/(2*(length(mn[factor==9])))) sum(ds[factor==10]) s25=sqrt(sum(ds[factor==10])/(2*(length(mn[factor==10])))) sum(ds[factor==11]) s24=sqrt(sum(ds[factor==11])/(2*(length(mn[factor==11])))) sum(ds[factor==12]) s23=sqrt(sum(ds[factor==12])/(2*(length(mn[factor==12])))) sum(ds[factor==13]) s22=sqrt(sum(ds[factor==13])/(2*(length(mn[factor==13])))) sum(ds[factor==14]) s21=sqrt(sum(ds[factor==14])/(2*(length(mn[factor==14])))) sum(ds[factor==15]) s20=sqrt(sum(ds[factor==15])/(2*(length(mn[factor==15])))) sum(ds[factor==16]) s19=sqrt(sum(ds[factor==16])/(2*(length(mn[factor==16])))) sum(ds[factor==17]) s18=sqrt(sum(ds[factor==17])/(2*(length(mn[factor==17])))) sum(ds[factor==18]) s17=sqrt(sum(ds[factor==18])/(2*(length(mn[factor==18])))) sum(ds[factor==19]) s16=sqrt(sum(ds[factor==19])/(2*(length(mn[factor==19])))) sum(ds[factor==20]) s15=sqrt(sum(ds[factor==20])/(2*(length(mn[factor==20])))) sum(ds[factor==21])

12

s14=sqrt(sum(ds[factor==21])/(2*(length(mn[factor==21])))) sum(ds[factor==22]) s13=sqrt(sum(ds[factor==22])/(2*(length(mn[factor==22])))) sum(ds[factor==23]) s12=sqrt(sum(ds[factor==23])/(2*(length(mn[factor==23])))) sum(ds[factor==24]) s11=sqrt(sum(ds[factor==24])/(2*(length(mn[factor==24])))) sum(ds[factor==25]) s10=sqrt(sum(ds[factor==25])/(2*(length(mn[factor==25])))) sum(ds[factor==26]) s9=sqrt(sum(ds[factor==26])/(2*(length(mn[factor==26])))) sum(ds[factor==27]) s8=sqrt(sum(ds[factor==27])/(2*(length(mn[factor==27])))) sum(ds[factor==28]) s7=sqrt(sum(ds[factor==28])/(2*(length(mn[factor==28])))) sum(ds[factor==29]) s6=sqrt(sum(ds[factor==29])/(2*(length(mn[factor==29])))) sum(ds[factor==30]) s5=sqrt(sum(ds[factor==30])/(2*(length(mn[factor==30])))) sum(ds[factor==31]) s4=sqrt(sum(ds[factor==31])/(2*(length(mn[factor==31])))) sum(ds[factor==32]) s3=sqrt(sum(ds[factor==32])/(2*(length(mn[factor==32])))) sum(ds[factor==33]) s2=sqrt(sum(ds[factor==33])/(2*(length(mn[factor==33])))) sum(ds[factor==34]) s1=sqrt(sum(ds[factor==34])/(2*(length(mn[factor==34])))) sd=c(s1,s2,s3,s4,s5,s6,s7,s8,s9,s10,s11,s12,s13,s14,s15,s16,s17,s18,s19,s20,s21,s22,s23,s24,s25,s26,s27,s28,s29,s30,s31,s32,s33) v=sd^2 mod1=lm(sd~conc) summary(mod1) plot(conc,sd,pch=16,xlab="Breath Alcohol Concentration",ylab="Standard Deviation",xlim=c(0,.4),ylim=c(0,.015)) abline(lsfit(conc,sd)) sd conc SCRIPT2 C:\Minnesota Breath Test\

References

13

1. International Organization for Standardization, Guide to the Expression of Uncertainty in Measurement, ISO, Geneva, 2008. Available at: http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf 2. Thompson, M. and Wood, R., Using uncertainty functions to predict and specify the performance of analytical methods, Accred Qual Assur, Vol.10, 2006, pp. 471-478. 3. EURACHEM/CITAC Guide, Quantifying Uncertainty in Analytical Measurement, 3rd Ed., 2012, pp. 117-119 Available at http://www.eurachem.org/images/stories/Guides/pdf//QUAM2012_P1.pdf 4. EURACHEM/CITAC Guide, Quantifying Uncertainty in Analytical Measurement, 3rd Ed., 2012, p. 16 Available at http://www.eurachem.org/images/stories/Guides/pdf//QUAM2012_P1.pdf

5. Altman DG, Bland JM, Measurement in medicine: the analysis of method comparison studies,

The Statistician Vol.32, 1983, pp. 307–317

This work was completed under contract with the State of Minnesota, SWIFT contract number 107618

Rod G. Gullberg, MS, PStat Clearview Statistical Consulting 20119 61st Avenue SE Snohomish, WA 98296 [email protected]

6/1/2016

http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf

BT-023

Authorization: Catherine M. Knutson

Page 1 of 8 Pages Version: Issue Date: 07/13/2016

UNCONTROLLED COPY WHEN PRINTED

MN BCA Forensic Science Service

Title: Estimation of Measurement Uncertainty for DataMaster DMT-G (DMT)

Breath Test Results

Scope:

This document provides instruction for the estimation of measurement uncertainty for DMT

breath test results.

Background Information:

Estimation of measurement uncertainty is an element of measurement traceability. Traceability

ensures consistency and comparability of test results reported by DMTs in the State of Minnesota

and between other traceable breath test results. Traceability of the measurement result is

established by using a properly certified DMT, which has calibration results that are traceable to

NIST and by establishing an Uncertainty of Measurement associated with the breath test results.

Measurement uncertainty defines a confidence interval symmetric about the average of multiple

measurement results. This uncertainty applies to the average value of the breath results obtained

from a DMT test. The uncertainty is an estimate; it is not a specific or exact number.

The measurand is the concentration of ethanol in the breath of the subject of a DMT test.

Both Type A and Type B uncertainty components must be considered:

• Type A uncertainty components can be evaluated statistically through a series of

observations. For a well-characterized measurement under statistical control, a combined

or pooled experimental standard deviation that characterizes the measurement may be

used to evaluate Type A standard uncertainty (GUM 4.2.4).

• Type B uncertainty components are evaluated by means other than statistical analysis of a

series of observations. The information used in the evaluation of Type B uncertainty may

be taken from sources including, but not limited to, manufacturer's specifications and

data provided on calibration or other certificates (GUM 4.3.1).

The approach utilized here for the estimation of measurement uncertainty for DMT subject test

results is outlined in the Guide to the Expression of Uncertainty in Measurement (GUM). [1]

This approach was applied to the Minnesota Breath Alcohol program by Rod Gullberg, MS,

PStat. [2]

BT-023





References:

1. International Organization for Standardization, Guide to the Expression of Uncertainty in

Measurement, ISO, Geneva, 2008. Available at:

http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf

2. Gullberg, R.G., Measurement Uncertainty Computed for the Forensic Breath Alcohol

Test Program in Minnesota, June 1, 2016

3. ASCLD/LAB Policy on Measurement Traceability AL-PD 3057 Ver 1.3, ASCLD/LAB-

International (2013).

4. ASCLD/LAB Guidance on Measurement Traceability AL-PD-3058 Ver 1.0,

ASCLD/LAB-International (2013).

5. ASCLD/LAB Guidance on Measurement Traceability-Measurement Assurance AL-PD-

3059 Ver 1.0 ASCLD/LAB-International (2013).

6. ASCLD/LAB Policy on Measurement Uncertainty AL-PD-3060 Ver 1.1, ASCLD/LAB-

International (2013).

7. ASCLD/LAB Guidance on the Estimation of Measurement Uncertainty- Overview AL-

PD-3061 Ver 1.0 ASCLD/LAB-International (2013).

Operation:

1. Breath sample values are saved to four digits in the Breath Alcohol Database (BrAD).

Since the DMT test record printout shows breath sample results to only three digits, all

values in the duplicate breath sample data set will be truncated to three digits prior to data

analysis.

2. To prevent loss of information, rounding and/or truncation of numbers is avoided during

all calculations performed prior to the final result. At minimum all significant figures are

conserved and additional digits may be conserved during calculations.

3. When calculating the combined standard uncertainty, the uncertainty components must

all be in the same units.

BT-023





4. A list of the major uncertainty components to be considered and characterization of the

method of evaluation of the components are maintained in the Calibration Laboratory.

5. The evaluation of the data contributing to the Type A uncertainty components has been

performed using a custom computer script written by Rod Gullberg for the R program.

6. An Excel spreadsheet will be used to facilitate calculation of combined standard

uncertainties and to produce a confidence interval table.

7. Dry gas certified reference material (CRM) is considered to have valid measurement

traceability when supplied by an accredited Reference Material Producer that is

accredited to ISO Guide 34:2009 by an accrediting body that is a signatory to a mutual or

multilateral recognition arrangement in an ILAC recognized regional accreditation

cooperation or the ILAC Mutual Recognition Arrangement, with a scope of accreditation

covering the CRM.

There are three terms that contribute to the combined uncertainty. They are:

1. CV� − Coefficientofvariationforduplicatebreathsamples 2. CV� − Coefficientofvariationforthereference 3. CV� − Coefficientofvariationforthebias

These terms are calculated and then combined as a root sum squared to estimate the combined

uncertainty.

1. Coefficient of variation for duplicate breath samples (��)

An uncertainty function will be used to estimate the standard deviation associated with the

average of two accepted breath samples in a DMT test. This standard deviation will be used

to calculate the coefficient of variation for the sampling term of the combined uncertainty

function. This is a type A uncertainty component.

Duplicate breath data is compiled and the average of each set of duplicates is calculated. The

duplicate samples are assigned to a bin corresponding to their average result. Each bin has a

width of 0.01. Bins containing fewer than 30 sets of duplicate data are disregarded. The

standard deviation of the remaining bins is calculated and then plotted against the midpoints

of the respective bins. A linear regression is then developed to model the data. This linear

regression will serve as the uncertainty function for duplicate breath samples and will

BT-023





provide an estimation of the standard deviation for the average of two breath samples

provided in a single breath test.

A. Data collection (perform steps in list order):

i. Using the sql query “Test Uncertainty Data w 02 agreement not met”, collect at

least 30,000 sets of duplicate breath sample data, with corresponding control data.

Transfer of appropriate data is verified and documented in the uncertainty folder

on the L drive.

ii. Copy and paste the data, with headers, to an Excel spreadsheet.

iii. Cut and paste Target and Control columns to a second spreadsheet. The original

sheet should now contain just the columns “brac1”, “brac2” and “mean”.

iv. Save the original sheet as a CSV file

v. Calculate the average control target and average control value on the second

sheet.

B. Evaluate the data using R

i. Open R code in a text file and edit first line of code to point to the CSV file

created in part A.iv.

ii. Copy/paste the code to the R workspace.

iii. R will run the code automatically.

C. Assemble the uncertainty function for duplicate breath samples in the following format:

�� = �!"#$% + '!

where: �� = ()*+,*-,,./0*)01+.()02*).31-,45607*).8-.*)ℎ(*256.( �! = �615.1360+.*--.:-.((01+3-12; "#$% = */.-*:.13,45607*).8-.*)ℎ(*256.( '! = '0+).-7.5)1360+.*--.:-.((01+3-12;

Calculate the coefficient of variation for duplicate breath samples using the following

equation.

BT-023





<=� =��>+�"#$%

where: <=� = 71.33070.+)13/*-0*)01+31-,45607*).8-.*)ℎ(*256.( �� = ()*+,*-,,./0*)01+.()02*).31-,45607*).8-.*)ℎ(*256.( +� = +428.-13(*256.()*?.+,4-0+:8-.*)ℎ).()@2B "#$% = */.-*:.13,45607*).8-.*)ℎ(*256.(

2. Coefficient of variation for the reference (��C)

Each complete DMT test includes a control sample drawn from a dry gas cylinder of known

alcohol concentration. The certificate of analysis for each dry gas control lists a measurement

uncertainty for the concentration of the cylinder. The uncertainty of the known cylinder

concentration is included in the combined uncertainty estimation as a type B component.

The uncertainty of the cylinder should be reduced to a single standard deviation before

calculating the coefficient of variation for the reference.

Calculate the coefficient of variation for the reference using the following equation.

<=D =��D√+DF#$%

where: <=D = 71.33070.+)13/*-0*)01+31-,45607*).8-.*)ℎ(*256.( ��D = ()*+,*-,,./0*)01+.()02*).31--.3.-.+7.

+D = +1. 137H60+,.-(*256.(*+*6HI.,8H2*+43*7)4-.-@min 2B F#$% = */.-*:.71+)-16)*-:.)3-12,*)*(.)

3. Coefficient of variation for the bias (��J)

Measurement bias is accounted for in the combined standard uncertainty. The absolute

observed bias is fixed at 0.005, which is the maximum bias observed in 98.8% of tests run

2014 – 2015. The absolute observed bias will be evaluated as described in part 6. It is

included in the uncertainty calculation as a type A component.

BT-023





The bias is assumed to have a uniform distribution and therefore should be divided by √3

when calculating the coefficient of variation for the bias.

Calculate the coefficient of variation for the reference using the following equation.

<=L =0.005√3<#$%

where: <=L = 71.33070.+)13/*-0*)01+31-80*( <#$% = */.-*:.71+)-16-.(46)3-12,*)*(.)

4. Calculating the combined uncertainty and the expanded uncertainty

A. The combined uncertainty will be calculated using the combined uncertainty function

described below:

4O = "#$%P<=�Q + <=DQ + <=LQ

where: 4O = 71280+.,4+7.-)*0+)H31-�RF8-.*)ℎ).()( "#$% = */.-*:.13,45607*).8-.*)ℎ(*256.( <=� = 71.33070.+)13/*-0*)01+31-,45607*).8-.*)ℎ(*256.( <=D = 71.33070.+)13/*-0*)01+31-,-H:*(-.3.-.+7.()*+,*-,

<=L = 71.33070.+)13/*-0*)01+31-80*(

B. The expanded uncertainty is calculated to a 95% confidence interval and a 99%

confidence interval.

i. The combined uncertainty (4O) is multiplied by a factor of 1.96 to achieve a 95%


ii. The combined uncertainty (4O) is multiplied by a factor of 2.575 to achieve a 99%


5. Reporting measurement uncertainty for a DMT breath test

A confidence interval table will be made available for customer use. The table will include

95% and 99% confidence intervals for all possible average test results and information

regarding the interpretation of the table.

BT-023





6. Annual review of measurement uncertainty

The Breath Alcohol Calibration Laboratory will calculate the test measurement uncertainty

estimation annually using a data set containing at least 30,000 sets of duplicate breath sample

data. A new confidence interval table will be created and published each year based on the

most recent uncertainty estimation. The new table will be applied to the following year’s

tests.

7. Updating and Using the Excel Uncertainty Calculator

Save a new copy of the uncertainty calculator spreadsheet in the uncertainty folder on the L

drive and name it with the year the current uncertainty calculation will be implemented. The

following cells in the uncertainty calculator should be reviewed and revised, if necessary,

after the annual review of measurement uncertainty. The combined uncertainty and

confidence interval table will automatically update if any changes are made.

A. Uncertainty Function Coefficients

i. Update the Slope and y-Intercept lines with the appropriate values from the R

result.

B. Reference Value (Line 5)

i. Column “Uncertainty”: input single standard deviation of field dry gas standards.

ii. Column “Mean”: input the mean value of the control targets within the current data

set.

iii. Column “n”: input the number of samples taken by the manufacturer to determine

the stated alcohol concentration of the field dry gas standards. (If unknown, use 2)

C. Bias Term (Line 6)

i. Column “mean”: input the mean value of the control sample readings within the

current data set.

Record requirements from the ASCLD/LAB Policy on Measurement Uncertainty AL-

PD-3060, May 2013, with the locations listed below:

(a) Statement defining the measurand (the quantity intended to be measured)1

(b) Statement of how traceability is established for the measurement1,

(c) The equipment (e.g., measuring device[s] or instrument[s]) used1, 3

BT-023





(d) All uncertainty components considered3

(e) All uncertainty components of significance and how they were evaluated1,3

(f) Data used to estimate repeatability and/or reproducibility2

(g) All calculations performed1,3

(h) The combined standard uncertainty, the coverage factor, the coverage probability and

the resulting expanded uncertainty1,3

(i) The schedule to review and/or recalculate the measurement uncertainty1,3

Locations of Required Items:

1 Breath Alcohol Calibration Laboratory Standard Operating Procedure BT-023

2 Maintained in shared network folder located at L:\Calibration Lab\Uncertainty\Test

Uncertainty

3 Gullberg, R.G., Measurement Uncertainty Computed for the Forensic Breath Alcohol

Test Program in Minnesota, June 1, 2016

**************************************************************************************

Revision and Review History: Previous version: None (New SOP)

Technical Leader/FS3(s): KK 07/12/16

Supervisor: SAB 07/12/16

Quality Manager/Quality System Coordinator: MS 07/12/16

Assistant Laboratory Director(s) St. Paul: AWH 07/12/2016

Laboratory Director: CMK 07/12/2016

**************************************************************************************

Archived:

Reason for Archiving: Quality Manager / Quality System Coordinator: Date:

applying metrology to the limitations of breath testing ... · applying metrology to the...

Documents