1 from the analytical uncertainty to uncertainty in data interpretation d. concordet, j.p. braun...

1

From the analytical uncertainty to uncertainty in data interpretation

D. Concordet, J.P. [email protected]

[email protected]

2

Biological data vary according several sources of variations

Biological- species- status (healthy, sick)- breed- format, age...- inter-individual- intra-individual

Metrological- intra- laboratory variations (imprecision)- inter-laboratories variations (bias)

controlled

of interest

not controlled

parasitic

3

Influence of imprecision on the reference interval

Pl-creatinine (mol/L)

10 30 50 70 90 110 130 150

CV=0%

Population distribution of Pl-Creatinine of healthy dogs

4



10 30 50 70 90 110 130 150

CV=0%

CV=10%


Y = X + with E (

5



10 30 50 70 90 110 130 150

CV=0%

CV=10%

CV=20%


Y = X + with E (

6

Sampling variations of the reference interval

95%

CI

of th

e re

fere

nce

inte

rval

Pl-

crea

tinin

e (

mol

/L)

10

30

50

70

90

110

130

150

170

N=100 dogs

CV=0% CV=10% CV=20%

7

Use of reference interval

Pl-creatinine (mol/L)40 135

Effect of large imprecision

8

Use of reference interval


When the precision is better

9

Replicates decrease the influence of imprecision

Y1 = X +

Y2 = X + with E (i

SD(Yi)= SD(i

Yp = X + p

p

YSD

CV(Yi) = 20 %

p CV(Y)

2 14.1%

3 11.5%

4 10.0%

10

20 40 60 80 100 120 140 160 180

Ref.

Influence of inter laboratories variability on the reference interval


11

20 40 60 80 100 120 140 160 180

Ref.

CV = 10% / Ref



12

20 40 60 80 100 120 140 160 180

Ref.

CV = 10% / Ref

CV = 20 % / Ref



13

Reference range of Pl-creatinine11 books of animal clinical biochemistry

0

25

50

75

100

125

150

175

200

225

250P

l-C

reat

inin

e (µ

mo

l/l)

[2] [6] [7] [8] [27] [28] [34] [45] [46] [55] [59]

Lefebvre HP et al. 1998.

35 µmol/l

245 µmol/l

Reference number

14

A “demographic” source of variation

GFR value

0

1

2

3

4

5

Dachshund Labrador Great Dane

GFR

(m

L/kg

/min

)M

ean

SD

8 8 6

GFR value depends on breed

Lefebvre HP unpublished results

15

Influence of the population’s structure

20 70 120 170 220 270

breed 1 = 30%

breed 2 = 30%

breed 3 = 20%

breed 4 = 10%

breed 5 = 5%

breed 6 = 5%

20 70 120 170 220 270

breed 1 = 30%

breed 2 = 30%

breed 3 = 20%

breed 4 = 10%

breed 5 = 5%

breed 6 = 5%

total

130Pl-creatinine (mol/L)

16

Influence of the population’s structure

20 70 120 170 220 270

breed 1 = 16%

breed 2 = 16%

breed 3 = 16%

breed 4 = 16%

breed 5 = 16%

breed 6 = 20%

20 70 120 170 220 270

breed 1 = 16%

breed 2 = 16%

breed 3 = 16%

breed 4 = 16%

breed 5 = 16%

breed 6 = 20%

total

210Pl-creatinine (mol/L)

17

Influence of multiplicity : case of independence


40

95% of healthy animals

95%

of

hea

lth

y an

imal

s

90% of healthy animals

18

Imprecision and multiplicity


40

19

Imprecision and multiplicity


40

CV= 15 %82 % of healthy animals

CV = 15%93% of healthy animals

20

Diagnostic tests : a better way to proceed

20 70 120 170 220 27020 70 120 170 220 270

Diseased animalsWithout the disease

Threshold

Considered as sick


Considered as healthy

Sensitivity :SeSpecificity :Sp

21

Performances of the test

Sensitivity = Se% of diseased animals with a result > threshold

Specificity = Sp% of animals without the disease with a result < threshold

22

Several definitions of specificity

Specificity = Sp

• Sp1 :% of healthy animals with a result < threshold

• Sp2 :% of animals without the disease (healthy + with any other disease) with a result < threshold

23

20 70 120 170 220 270

Several definitions of specificity

Healthy animalsSp1

Threshold

Healthy animals+ with other diseasesSp2

Sp2<<Sp1

24

The operational indices : the predictive values

Spe

ePPV

1Pr1SPr

SPr

Spe

SpNPV

Pr1S1Pr

Pr1

Positive Predictive Value (PPV) : Probability that the animal has the disease when its result >threshold

Negative Predictive Value (NPV) : Probability that the animal has not the disease when the result < threshold

25

The clinician experience : Pr

Pr = Pre-test probabilityprobability that the animal has the disease

Pr = 1 : The clinician is sure that the animal has the disease

Pr = 0 : The clinician is sure that the animal has not the disease

PPV =1

NPV =0whatever Se and Sp

PPV =0

NPV =1whatever Se and Sp

Positive diagnostic gain : PPV-Pr

Negative diagnostic gain : NPV-(1-Pr)

Pr = 0.5 : The clinician does not know (coin tossing)

26

20 70 120 170 220 270

Sp = 80%

Influence of imprecision

Threshold

Pl-creatinine (mol/L)20 70 120 170 220 270

Se = 92%

healthy1

Sp = 73%

CV = 15 %

Se = 81%

CV = 15%

27

0

0.10.2

0.3

0.40.5

0.6

0.7

0.80.9

1

0 0.2 0.4 0.6 0.8 1

PPV

NPV

Consequences on interpretation

Pre-test probability

PP

V/N

PV

0

0.10.2

0.3

0.40.5

0.6

0.7

0.80.9

1

0 0.2 0.4 0.6 0.8 1

PPV

NPV

PPV CV 15%

NPV CV 15%

28

Sampling variations of sensitivity and specificity

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 50 100 150 200 250 300

Sample size : n95%

co

nfi

den

ce in

terv

al o

f S

e=0.

92, S

p=

0.80

N

SSS

N

SSS

12;

12

Se = 0.92

Sp = 0.80

95% confidence interval of Se/ Sp :

29

Consequences on interpretation9

5% c

on

fid

enc

e in

terv

al o

f P

PV

/NP

V


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

N=50

30


5% c

on

fid

enc

e in

terv

al o

f P

PV

/NP

V


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

N=100

31


5% c

on

fid

enc

e in

terv

al o

f P

PV

/NP

V


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

N=300

32

Effects of the biological sources of variation

Increase the overall dispersion of the results

Improvement possible if Se and Sp are determined for each level of the factors of variation (e.g. breed)

decrease Se and Sp

decrease PPV and NPV for a fixed pre-test proba

33

The future ?

Individualisation

Blood sample when the animal is young and healthy

Follow-up of the evolution of the appropriate marker

Critical difference

Independent of demographic factors (breed, sex…)

dependent only on intra-individual variability

analytical errors

34

The end

1 from the analytical uncertainty to uncertainty in data interpretation d. concordet, j.p. braun...

Documents