Ž .Clinica Chimica Acta 307 2001 37–43www.elsevier.comrlocaterclinchim

Optimal analytical performance for point of care testing

Callum G. Fraser)

Biochemical Medicine, Tayside UniÕersity Hospitals NHS Trust, Ninewells Hospital and Medical School, Dundee DD1 9SY, Scotland, UK

Abstract

Quality specifications for the reliability of performance characteristics of laboratory testing, particularly precision andbias, are necessary prerequisites for creation and control of analytical quality. Many strategies have been promulgated forsetting these specifications. Recently, the available approaches have been fixed into a hierarchical framework that has nowbeen accepted by experts in the field to be the best current approach to a global strategy for setting quality specifications inlaboratory medicine. They should be incorporated into quality planning strategies everywhere irrespective of the settings in

Ž .which laboratory medicine is practised, including the point of care testing POCT . Models higher in the hierarchy arepreferred to lower approaches but lower approaches are better than none and should be used as the minimum standard.q 2001 Elsevier Science B.V. All rights reserved.

Ž .Keywords: Point of care testing POCT ; Laboratory testing; Hierarchy; Quality specifications

1. Introduction

Every analytical method, irrespective of where itis actually performed, can be described fully in termsof its performance characteristics. These are of twotypes, practicability performance characteristics andreliability performance characteristics. The formerinclude skills required, speed of analysis, volumerequired, and type of sample required. The latterinclude precision, bias, limit of detection, and mea-suring range. It is often suggested that, for point of

Ž .care testing POCT , considerations of speed of anal-Ž .ysis expressed as total turnaround time surpass all

other requirements. However, quality specificationsfor the reliability performance characteristics of lab-oratory tests, particularly precision and bias, areabsolutely necessary prerequisites for analytical qual-ity management. Moreover, such analytical quality

) Tel.: q44-1382-660111; fax: q44-1382-654333.Ž .E-mail address: callum.fraser@tuht.scot.nhs.uk C.G. Fraser .

specifications should be firmly based upon medicalrequirements, useable in all laboratories irrespectiveof size, type or location, generated using simple tounderstand models, and widely accepted as cogentby professionals in the field.

Quality specifications are required in many facetsof the discipline, including generating specificationsfor new analytical systems, assessing available lite-rature to assist in method selection, evaluating sub-mitted tenders, assessing data generated in methodvalidation, and creating appropriate internal qualitycontrol and external quality assessment schemeswhich guarantee the specified analytical quality. Aplethora of papers, reviews, and book chapters deal-ing with the generation and application of quality

w xspecifications has been published over time 1 .However, there still seem to be real dilemmas indeciding on appropriate quality specifications, partic-ularly for precision and bias. Although there aremany very logical reasons for this situation, a crucialrecent development was that a consensus was reached

0009-8981r01r$ - see front matter q 2001 Elsevier Science B.V. All rights reserved.Ž .PII: S0009-8981 01 00429-6

( )C.G. FraserrClinica Chimica Acta 307 2001 37–4338

Table 1Hierarchical approach to classification of strategies to set quality specifications

1. Assessment of the effect of analytical performance on Quality specifications in specific clinical situationsspecific clinical decision-making2. Assessment of the effect of analytical performance on 2A. General quality specifications based ongeneral clinical decision-making biological variation

2B. General quality specifications based on medical opinions

3. Professional recommendations 3A. Guidelines from national or internationalexpert groups3B. Guidelines from expert individuals orinstitutional groups

4. Quality specifications laid down by regulation or by 4A. Quality specifications laid down by regulationw xexternal quality assessment scheme EQAS organisers 4B. Quality specifications laid down by EQAS organisers

5. Published data on the state of the art 5A. Published data from external quality assessment andw xproficiency testing PT schemes

5B. Published individual methodology

in 1999 on global strategies to set quality specifica-w xtions in laboratory medicine 2 . This consensus was

based upon a hierarchical approach published justw xprior to the consensus conference 3 .

The hierarchy and its application to the setting ofanalytical quality specifications for precision andbias are the subjects of this review. Examples usedare taken from those quantities often measured inPOCT settings. The hierarchy shown in Table 1 hasbeen accepted by experts in the field to be the bestcurrent means to classify the available strategies.

2. Assessment of the effect of analytical perfor-mance on specific clinical decision-making

Clearly, the first choice should logically be thestrategy at the top of the hierarchy. Thus, analyticalquality specifications should be derived from analy-sis of the effect of analytical quality on medicaldecision-making in specific clinical situations. A firstexample is provided by consideration of cholesterolassays.

If the POCT methodology had a positive analyti-cal bias, then the population distribution would moveto the right and Afalse positiveB results would befound irrespective of the clinical decision makingcriterion used for patient classification. If the clinicalstrategy was to treat with either lifestyle advice, diet

wor drugs which would all entail further laboratoryxtests and recall , or even to simply repeat of the test,

then additional health care resources would be spentand more of the population would be labelled as Aatgreater riskB. In contrast, if the laboratory had nega-tive bias, the distribution would shift to the left; thenumber of Afalse negativesB would increase, savingcosts on additional testing in the short term, butpossibly eventually leading to huge health care costsas the population missed at initial testing succumbedto premature coronary artery disease.

w xThis situation can be assessed more formally 4 .In Fig. 1, the distribution of serum cholesterol con-

wcentrations in a Danish population is shown upperxpanel . The effect of negative and positive biases of

10% on those of at high risk, that is, purely forillustrative purposes here, those with true serumcholesterol concentration of greater than 7.0 mmolrl,can be easily calculated. Subsequently, the calcula-tion can be done for all values of bias. The func-tional relationship between the decreases and in-creases in the percentage of the population at highrisk and analytical bias is as shown in Fig. 2. If themedical needs in terms of allowable percentage mis-classification could then be defined, the allowableanalytical bias—the analytical quality specification—can be easily calculated.

Investigation of the relationship between medicalneeds and analytical performance can be done in asimilar manner for other quantities and we have

w xexplored this in some detail 5 . For example, there isa relationship between the risk of microalbuminuriaand the blood glycated haemoglobin concentration.

( )C.G. FraserrClinica Chimica Acta 307 2001 37–43 39

w x wFig. 1. The effect of negative middle panel and positive lowerx wpanel bias on the percentage of the population at risk above 7.0

x w x Ž .mmolrl . From Ref. 4 p. 79 .

Fig. 3 shows the consequences of analytical bias onthe risk in an individual with a true glycatedhaemoglobin of 10.1%.

If negative bias was present, the reported valueswould be less than 10.1%, the clinician would imag-ine that the patient was under reasonable control andnot change therapy in any way—the patient actuallyhas a higher glycated haemoglobin concentration,less good glycaemic control and a greater risk of

Fig. 2. Functional relationship between the percentage of thew x Ž .population at high risk and analytical bias. From Ref. 4 p. 79 .

Fig. 3. Influence of analytical bias on the risk of microalbumin-w x Ž .uria. From Ref. 5 p. 200 .

wmicroalbuminuria and the other sequelae of poorxcontrol . In contrast, if the analytical method had

positive bias, the glycated haemoglobin would ap-pear lower: the clinician might congratulate the pa-tient on maintaining good control but, while the riskof microalbuminuria might be lower, the risk ofhypoglycaemic episodes might be increased. Thus,deciding the acceptable clinical outcomes could al-low clear definition of acceptable analytical perfor-mance.

However, one significant problem with this ap-proach is that only very few tests are used in single,well-defined clinical situations. Moreover, qualityspecifications calculated depend very much on theassumptions made about how the test results are usedby clinicians—even for glycated haemoglobin assaysw x6 .

3. Assessment of the effect of analytical perfor-mance on general clinical decision-making

The second strategy in the hierarchy is the cre-ation of quality specifications based on general waysin which clinicians use test results. Quality specifica-tions for precision and bias in monitoring and diag-nosis can be based on data on the components of

w xbiological variation, namely, within-subject CV andIw xbetween-subject CV variation.G

In clinical monitoring, analytical random variationmust be kept low so that changes in test results in anindividual are significant, with high probability, andthat these do not simply reflect analytical randomvariation. This is particularly important when POCT

( )C.G. FraserrClinica Chimica Acta 307 2001 37–4340

is considered because, at least traditionally, the ana-lytical performance achieved in sites other than thelaboratory were inferior and the results were intrinsi-cally more variable. It should be noted that one ofthe alleged advantages of POCT is that patients canbe monitored closely and frequently. Irrespective ofthe time scale, monitoring involves comparison oftest results from an individual over time.

In the simplest AhomeostaticB model, changes inserial results can be due to the following:

v the patient getting better,v the patient getting worse,v pre-analytical variation,v w xbiological variation within-subject ,v analytical variation—changes in bias and inher-

w xent precision CV .A

Thus, if pre-analytical sources of variation areminimised, then, to assess whether change has oc-curred, it must exceed the inherent variation due tobiological and analytical variation that is defined asthe reference change value. The reference change

w xvalue RCV can be calculated as -1r21r2 2 22 PZP CV qCVA I

where Z is the number of standard deviates appropri-wate to the probability selected for example, 1.96 forxP-0.05 and 2.56 for P-0.01 .

It is simple to demonstrate the effect of precisionwon medical decision-making. Taking cholesterol CVI

x;6% as an example, the change required for signif-w xicance at P-0.05 increases with precision as

shown in Table 2. For precision, the widely acceptedquality specification is that the analytical variationw xCV should be less than one-half the averageA

w xwithin-subject biological variation 7 . Harris showed

Table 2w xEffect of precision on reference change value RCV for serum

cholesterol at P-0.05

w x w xPrecision CV, % RCV %

2 17.54 20.06 23.58 27.7

10 32.3

Fig. 4. Percentage increase in test result variability due to analyti-wcal precision expressed as a ratio of analytical to within-subject

xbiological variation showing three possible quality specificationsw xbased on within-subject biological variation. From Ref. 9 , p. 9.

that, if CV -0.50 CV , then the amount of variabil-A Iw xity added was about 10% in reality, 11.8% , which

w xwas stated to be AreasonableB 8 . This proposal hasbeen very widely accepted by professionals. Further-more, this idea has been expanded more recently andthree classes of analytical quality, optimum, desir-able and minimum, based upon different fractions ofwithin-subject biological variation have been pro-

w xposed as shown in Fig. 4 9 .Although there are many strategies for the inter-

pretation of laboratory test results in diagnosis, manyuse population-based reference values, particularlythe less experienced. It is often the case that patientshave tests done in various locations such as theemergency room, the outpatient clinic, and the ward—in which POCT may be used—and in the labora-tory. Clearly, test results should be comparable overlocation. In consequence, the ideal is that all testingsites serving a homogeneous population should alluse the same reference values. For this to be achieved,

w xit has been shown 10 that bias should be less thanwone-quarter of the group biological variation that is,

w 2 2 x1r2 xBIAS-0.25 CV q CV . Again, three classesI G

of analytical quality, optimum, desirable and mini-mum, based upon different fractions of within- plusbetween-subject biological variation, have been pro-

w xposed as shown in Fig. 5 9 .These well-established approaches have advan-

tages in that data on components of biological varia-tion are available for more than 300 quantities. A

( )C.G. FraserrClinica Chimica Acta 307 2001 37–43 41

Fig. 5. Percentage of results outside reference limits due tow Žanalytical bias expressed as a ratio of analytical to group within-

. xplus between-subject biological variation showing three possiblew xquality specifications based on biological variation. From Ref. 9 ,

p. 9.

recent compilation in the easily available literaturew xmakes the data easy to obtain 11 , and the data seem

independent of study location, number of subjects,length of study, analytical methodology, age of sub-jects or whether they are in a state of health or havestable but chronic disease. Moreover, data on com-ponents of biological variation have been used todefine quality specifications for other characteristics

w xand in other laboratory settings 12 .Quality specifications sometimes alleged to be

based on Amedical needsB have been calculated fromthe responses of clinicians to a series of short case

w xstudies vignettes on the general interpretation oftest results. Most of these studies have significantdeficiencies in design and execution: these problemsand potential solutions have been debated again re-

w xcently 13 . However, the best example of this ap-w xproach is that of Thue et al. 14 who derived quality

specifications for the precision of analysis of onew xquantity only haemoglobin through a series of vi-

gnettes submitted to a large single specialty clinicalw xgroup general practitioners in Norway . This study

could be used as a model for future similar vignettestudies.

4. Professional recommendations

Certain national or international professionalgroups have published quality specifications. For

example, the recommendations of the Nationalw xCholesterol Education Panel US have been used

w xextensively 15 as have the detailed recommenda-w xtions of the American Diabetes Association 16 for

self-monitoring of blood glucose. The latter haveevolved over time; a major problem with these par-ticular guidelines is that they seem empirical and itis, in fact, quite difficult to interpret what theyactually mean. Moreover, the quality specificationslaid down by experts often differ quite markedly.

Additionally, certain quality specifications havebeen proposed through publication of guidelinesbased on what could be viewed as best or goodlaboratory practice. These are often presented ordeveloped at a single consensus conference withoutsignificant discussion. However, these guidelineshave the advantage that they are usually generatedfrom the very broad experience of either a singleexpert or an expert group from a single institution.

5. Quality specifications laid down by regulation[ ]or by external quality assessment scheme EQAS

organisers

The acceptable standards of analytical perfor-mance required have been laid down in a number ofcountries. The best example is the US CLIA ’88

w xlegislation 17 that documents acceptable total errorfor a number of commonly assayed analytes. Themajor disadvantage of these quality specifications isthat, although based on expert views, they tend to beempirical and are clearly influenced by what is actu-

Ž .ally achievable at the time the state of the art .EQAS use a variety of measures of location and

allowable dispersion. In Europe, some use statisticalanalysis of the data returned from the participantlaboratories but, more and more, fixed limits are

w xused 18 . Again the problem of quality specifica-tions based upon these fixed limits is that, althoughoften based on expert opinion, they tend to be sub-jective and are affected by the state of the art.

6. Published data on the state of the art

Quality specifications could be generated throughreference to the performance achieved by groups of

( )C.G. FraserrClinica Chimica Acta 307 2001 37–4342

laboratories participating in EQA and PT schemes.This has the advantage that many data are oftenavailable. However, for a number of obvious rea-sons, true analytical performance may not be accu-rately mirrored by this apparent state of the art.

Measures of the quality of analytical performancecould be obtained be comparison with attainmentdocumented in published works on similar or otherassay methods for the quantity for which qualityspecifications were required. This has some merit inthat many data are often available, but has the realdifficulty that published method performance may bethe best possible rather than that achieved in prac-tice. Again, performance achieved analytically maybear no relationship to actual medical needs. Withregard to POCT, a problem is that many evaluationsof technology are done in laboratories with well-trained staff only and are not done by the clinicalstaff who would actually do the procedures in prac-tice. Moreover, traditionally, for example in a studydone by us on cholesterol assays in the Coronary

w xCare Unit 19 , it was considered that the state of theart achieved by clinical staff was inferior to thatattained by laboratory staff. However, modern tech-nology does seem to allow results to be obtainedwhich are operator independent and after minimal

w xtraining 20 .

7. Conclusion

A hierarchy of approaches to set analytical qualityspecifications has been created and approved byexpert professionals. The hierarchy should be appliedin practice. These simple to understand models areappropriate for all settings in which laboratorymedicine is practised, including POCT, and theyshould be incorporated into quality planning strate-gies everywhere. As we have stated previously in areview on quality specifications for analyses done in

w xalternate sites including POCT 21 , there is noreason why different standards are warranted, and wehave tabled general numerical analytical qualityspecifications based on biological variation for testcommonly performed as POCT. Clearly, modelshigher in the hierarchy are preferred to lower ap-proaches but lower approaches are better than noneand should be used if all that are available. New

useful models may be developed in the future andthese should be incorporated into the hierarchicalscheme when widely approved by professionals inlaboratory medicine.

Acknowledgements

The author thanks I.M. Kennedy for the exem-plary perspicacity evidenced in the preparation of themanuscript of this paper.

References

w x1 Fraser CG. Quality specifications in laboratory medicine.Clin Biochem Revs 1996;17:109–14.

w x Ž .2 Hyltoft Petersen P, Fraser CG, Kallner A, Kenny D, Eds. ,Strategies to set global analytical quality specifications inlaboratory medicine. Scand. J. Clin. Lab. Invest. 1999;59:475–585.

w x3 Fraser CG, Hyltoft Petersen P. Analytical performance char-acteristics should be judged against objective quality specifi-cations. Clin Chem 1999;45:321–3.

w x4 Hyltoft Petersen P, Horder M. Influence of analytical qualityŽon test results. Scand J Clin Lab Invest 1992;58 Suppl.

.208 :65–87.w x5 Hyltoft Petersen P, deVerdier CH, Groth T, Fraser CG, et al.

The influence of analytical bias on diagnostic misclassifica-tions. Clin Chim Acta 1997;260:189–206.

w x6 Lytken Larsen M, Hyltoft Petersen P, Fraser CG. A compari-son of analytical goals for haemoglobin A1c assays derivedusing different strategies. Ann Clin Biochem 1990;28:272–8.

w x7 Cotlove E, Harris EK, Williams GZ. Biological and analyticcomponents of variation in long-term studies of serum con-stituents in normal subjects: III. Physiological and medicalimplications. Clin Chem 1970;16:1028–32.

w x8 Harris EK. Statistical principles underlying analytic goal-set-ting in clinical chemistry. Am J Clin Pathol 1979;374:72–82.

w x9 Fraser CG, Hyltoft Petersen P, Libeer JC, Ricos C. Proposalsfor setting generally applicable quality goals solely based onbiology. Ann Clin Biochem 1997;34:8–12.

w x10 Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Horder M.Analytical goals for the acceptance of common referenceintervals for laboratories throughout a geographical area.Scand J Clin Lab Invest 1988;48:757–64.

w x11 Sebastian-Gambaro MA, Liron-Hernandez FJ, Fuentes-Arde-riu X. Intra- and inter-individual biological variability data

wbank. Eur J Clin Chem Clin Biochem 1997;35:845–52 alsoxavailable at www.westgard.com .

w x12 Fraser CG. Quality specifications in laboratory medicine—current consensus views. Accredit Qual Assur 1999;4:410–3.

w x13 Fraser CG. Judgement on analytical quality requirementsfrom published vignette studies is flawed. Clin Chem LabMed 1999;37:167–8.

( )C.G. FraserrClinica Chimica Acta 307 2001 37–43 43

w x14 Thue G, Sandberg S, Fugelli P. Clinical assessment ofhaemoglobin values by general practitioners related to analyt-ical and biological variation. Scand J Clin Lab Invest 1991;51:453–9.

w x15 National Cholesterol Education Program Laboratory Stan-dardization Panel, Current status of blood cholesterol mea-surement in clinical laboratories in the United States. ClinChem 1988;34:193–201.

w x16 American Diabetes Association, Consensus statement onself-monitoring of blood glucose. Diabetes Care 1994;17:81–6.

w x17 US Dept. of Health and Human Services, Medicare, Medi-caid, and CLIA programs: regulations implementing the Clin-

Ž .ical Laboratory Improvement Amendment of 1988 CLIA .Final rule. Fed Reg 1992;57:7002–186.

w x18 Ricos C, Baadenhuisjen H, Libeer JC, Hyltoft Petersen P, etal. Currently used criteria for evaluating performance in EQAin European countries and a comparison with criteria for afuture harmonisation. Eur J Clin Chem Clin Biochem1996;34:159–65.

w x19 Nelson LM, Clark RS, Fraser CG. A laboratory and inwardevaluation of cholesterol assays on the Ames Seralyzer. JAutom Chem 1985;7:173–6.

w x20 Bingham D, Kendall J, Clancy M. The portable laboratory:an evaluation of the accuracy and reproducibility of I-STAT.Ann Clin Biochem 1999;36:66–71.

w x21 Fraser CG, Hyltoft Petersen P. Desirable performance stan-dards for imprecision in alternate sites. Arch Pathol Lab Med1995;119:909–13.