![Page 1: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/1.jpg)
…Borrelia diagnostics – statistical aspects
Jørgen [email protected]
February 2009
Notes have been added in this file
![Page 2: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/2.jpg)
Plan of my talk
Clinicometric framework
Descriptors of diagnostic power
Displays of diagnostic power
including the ROC diagram
Simultaneous use of 2 measurements
Randomized testing of diagn. procedures
Special topics in supplementary slides
Biostatistical motto: Formalism with a human face
![Page 3: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/3.jpg)
Topics not mentioned
Systematic reviews &
meta-analyses
![Page 4: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/4.jpg)
”Clinicometrics”
…always considers a stream of cases
( statisticians say: a population of cases ):
They are the units of clinical experience
and also of clinical decision making.
They are instances of a (well-defined?)
clinical problem,
”the who-how-where-why of
a patient-doctor encounter.”
Therefore…
![Page 5: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/5.jpg)
In clinical studies the choice of sample, and of the variables on which one bases one's prediction,
must match the clinical problem as it presents itself at the time of decision making.
In particular, one mustn't discard subgroups (as ‘atypical’ or ‘impurities’) that did not become identifiable until later: ensure prospective recognizability !
Data
colle
ctio
n*
*as opposed to the ’engineering’ phases
![Page 6: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/6.jpg)
Purity vs. representativeness:
A meticulously filtered case stream ( 'proven single-agent infections', or 'meeting CDC criteria' ) may be needed for patho- and pharmaco-physiological research,
but is inappropriate as a basis for clinical decision policies
[incl. cost studies].
Data
colle
ctio
n
![Page 7: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/7.jpg)
Your job is to create decision rules that help the clinician decide, e.g.
- whether to proceed with antibiotics- when to plan clin. & serol. follow-up checks - when to apply other tests for, e.g. HSV
►ideally drawing a complete management flowchart,i.e. a bushy tree of action diagnoses, not etiological diagnoses
Don’t
fo
rget…
![Page 8: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/8.jpg)
Consecutivity as a safeguard against selection bias.
Standardization: How? Who? Where? When? Gold standard … the big problem !! w. blinding, etc.
Safeguards against change of data after the fact.
w3.consort-statement.org/Initiatives/stardClinical_Chemistry_statement.pdf
Data
colle
ctio
n
![Page 9: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/9.jpg)
Quantitative markers
A quantity holds the result of a diagnostic procedure. Histograms describe its distribution in two subpopulations.
We can interpret ordinates and areas under the two humps in terms of true and false decisions …
and get a feel for the trade-off involved, provided that the pre-test probability of disease (percentage diseased) is known.
Focussing on
![Page 10: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/10.jpg)
… principle …
Measurement
Non-disease
Each area = 1.00 = 100 % of the subpopulation
False negatives
False positives
Cutoff point
Negative rangePositive range
Diseased
Healthy
False positive
False negative
Measurement
![Page 11: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/11.jpg)
… principle …
Measurement
Non-disease
False negatives
False positives
Cutoff point
Negative rangePositive range
Diseased
Healthy
False positive
False negative
Measurement
Sensitivity( true positive
fraction )Specificity
( true negative fraction )
Note: BLACK&WHITE
paradigm !
![Page 12: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/12.jpg)
Pre-test ’case mix’ 30% 70% diseased non-diseased
I.e., 64.4 % of cases are true negatives; the other three areas are analogous.
the
probability
square’
Sensitivity,
true posit. fraction
1 – spec. = false
positive fraction
All the positives
Specificity = 0.92, say
True negatives area =
0.70 × 0.92 = 0.644
![Page 13: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/13.jpg)
Classical terminology”Positive” = suggestive of (target) disease
”Negative” = suggestive of its absence
”False / True Positive / Negative …”
Sensitivity = TP/(those diseased)
Specificity = TN/(those without it)
What is meant by PV ( ”predictive value” )?What is meant by LR ( ”likelihood ratio” )?
![Page 14: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/14.jpg)
Classical terminology
”Positive” = suggestive of (target) disease”Negative” = suggestive of its absence”False / True Positive / Negative …” Sensitivity = TP/(those diseased) Specificity = TN/(those without it)
PVpos = the ”predictive value” of a positive outcome = TP/(all positives) = Pr{ disease | pos }
…chance that the test is right when it says ”positive”
![Page 15: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/15.jpg)
Classical terminology
”Positive” = suggestive of (target) disease”Negative” = suggestive of its absence”False / True Positive / Negative …” Sensitivity = TP/(those diseased) Specificity = TN/(those without it)
PVneg = the ”predictive value” of a negative outcome = TN/(all negatives) = Pr{ non-disease | neg }
…chance that the test is right when its verdict is ”negative”
![Page 16: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/16.jpg)
pre-test odds = 3 : 7
30% 70%
diseased non-diseased
”LR” = 5 : 1 (the ratio of red arrows);
ergo post-test odds = 15 : 7.
”Likelih
ood
ratio”
principle …
Sensitivity
1 – specificity
![Page 17: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/17.jpg)
Pre-test odds low in Lyme problems diseased non-diseased
”LRpos” = 5 : 1 is fair; but
post-test odds and PVpos are still low.
Sensitivity
1 – specificity
Specificity is not bad. Yet most positives are
false positives
![Page 18: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/18.jpg)
classical terminology:
Sensitivity = TP/(those diseased)
Specificity = TN/(those without it)
LRpos = the ”likelihood ratio” occasioned
by a positive outcome =
(sensitivity) / (1 – specificity) =
Pr{ pos | disease } / Pr{ pos | non-disease }
Not quite so
![Page 19: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/19.jpg)
classical terminology:
Sensitivity = TP/(those diseased)
Specificity = TN/(those without it)
LRneg = the ”likelihood ratio” occasioned
by a negative outcome =
(1 – sensitivity) / (specificity) =
Pr{ neg | disease } / Pr{ neg | non-disease }
Not quite so
= 0.1 = 1 : 10, for instance. If the pre-test risk of Lyme Disease is low, say p = 2%, a negative outcome almost eliminates it:
(post-test odds) = (pre-test odds)(LR) = (1 : 49)(1 : 10) = (1 : 490) .
![Page 20: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/20.jpg)
”LR” principle: it’s the factor by which the observed data will change the odds
Measurement
Non-disease
False negatives
False positives
Cutoff point
Negative rangePositive range
Diseased
Healthy
False positive
False negative
Measurement
LRpos = ratio –
– ratio = LRneg*
*LRneg < 1 (!)
![Page 21: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/21.jpg)
”LR” principle: it’s still the factor by which the observed data will change the odds
Measurement
Non-disease
False negatives
False positives
Cutoff point
Negative rangePositive range
Diseased
Healthy
False positive
False negative
Measurement
LR = ratio
Cutpoint now
irrelevant
*
LRdata when data = a measurement value*
![Page 22: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/22.jpg)
A 2-gate study
50 diseased 75 non-diseased
”LRpos” = 5 : 1 but the ”predictive values” and
the post-test odds are unavailable.
Sensitivity
1 – specificity
Warning…
![Page 23: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/23.jpg)
A 2-dim. task
IgM
IgG
Confirmed infectionConfirmed non-infected case
![Page 24: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/24.jpg)
A 2-dim. task
IgM ?
IgG ?
Iso-Likelih. Ratio lines
(uphill arrow)
![Page 25: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/25.jpg)
A 2-dim. task
IgM
IgG
?
Nearest-neighboursclassification of
a new patient
![Page 26: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/26.jpg)
A 2-dim. task
IgM
IgG
etc., with decreasing influence, the farther away.
Kernel methods form aweighted averageof neighbouring
prototypes(diagnosed cases)
![Page 27: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/27.jpg)
Iso-density (iso-tætheds-)
linier
IgM
IgG
![Page 28: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/28.jpg)
IgM
IgG
Iso-Likelihood Ratio lines
(uphill arrows)
![Page 29: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/29.jpg)
IgM
IgG
Infection
Non-infected
Simulated data (100+100)
![Page 30: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/30.jpg)
![Page 31: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/31.jpg)
A ROC diagram shows the true positive fraction against the false positive fraction as a function of the choice of cutoff point
Hypothetical smooth trajectory, and two raw empirical ones [ sample sizes: 17+17 ( ), and 40+40 ( ) ]
Everyone negative
Everyone treated as positive
Liberal cutoffStrict cutoff
![Page 32: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/32.jpg)
The ROC diagram describes the nosographics
Sens, spec.LRpos, LRneg = slopes of segments.
Y = Youden’s Index = sens + spec – 1 is equivalent to
AUC [Area Under Curve] = ½(sens + spec)
in this case. ROCY = 1
Y = 0
FN
TP
FP TNBLACK&WHITE
neg
posWe are
within the Black&WhiteParadigm
nosographic
properties
![Page 33: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/33.jpg)
The ROC diagram describes the nosographics*
The slope of each outcome line is its LR; e.g.
LRpos = (TP fraction of Diseased)/(FP fraction of non-dis.)
ROCY = 1
Y = 0
FN
TP
FP TN
neg
pos
*i.e., the information obtainable from a 2-gate study
![Page 34: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/34.jpg)
Three testoutcomes
Ideal test
Om
inou
s
Almost no
evidence
either way
Reassuring
![Page 35: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/35.jpg)
Three testoutcomes
Ideal test
Positive
+/-
NegativeNeg.?
Pos.?
Ordered (ordinal) test outcomes
Ordered how?By increasing slope, i.e. LR[ concavity ! ]
![Page 36: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/36.jpg)
Three testoutcomes
Definitely positive
+/-
NegativeNeg.?
Possibly positive
Ordered (ordinal) test outcomes
Ordered how?By increasing slope, i.e. LR[ concavity ! ]
The slope reflects the medical trade-off between % sensitivity and % specificity
Those with a ”+/ – ”test result are best treated as negative
in this situation
Trade-off? Constant benefit? … Please take a look at the supplementary figures
A ’constant-benefit’ line
![Page 37: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/37.jpg)
E.g., 5 cases of disease D and 10 non-D cases:The ROC square holds 50 small rectangles, 40 of which happen to be below the ROC trajectory, because 40 times (out of 50) it so happens that a a non-D finding > a D-group finding[the desired ordering]. For an example, see patient * vs. patient **.
Area Under ROC Curve =freq{ (non-D value) > (D value) } = 0.80.
Interpretation of the area under the ROC as a rank statistic ( cf. Wilcoxon-Mann-Whitney )
![Page 38: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/38.jpg)
The 5 cases of disease D and 10 non-D cases:The ROC square holds 50 small rectangles, 40 of which happen to be below the ROC trajectory, because 40 times (out of 50) it so happens that a a non-D finding > a D-group finding[the desired ordering]. For an example, see patient * vs. patient **.
Area Under ROC Curve =freq{ (non-D value) > (D value) } = 0.80.
But where does that lead us?The AUC has no definable interpretation in terms of blood, sweat and tears(loss, benefit, utility).
It only has a soft association with decision-analyticmeasures of diagnostic power (separation, discrimination).
Its frequent use is purely a matter of being thepopular girl in the class.
![Page 39: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/39.jpg)
The primary virtues of the ROC: it allows you
(1) to compare tests regardless of scale, units, & transformations
(2) to see oddities [ which may point to a technical problem, or
call for a revised test interpretation rule ]
What!?
![Page 40: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/40.jpg)
Lesions in floating locationsSuspect
area?Red = as the imagist saw it
Green = surgical truth
How do we score diagnostic performance
in such situations ???
![Page 41: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/41.jpg)
Randomized trials of diagn. tests
… theory under development
Purpose & design: many variantsSub(-set-)randomization, depending on the
pt.’s data so far collected.
”Non-disclosure”: some data are kept under seal until analysis. No parallel in therapeutic trials!
Main purposes …
Digression…
![Page 42: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/42.jpg)
… Randomized trials of diagn. tests
1) when the diagnostic intervention is itself potentially therapeutic;
2) when the new test is likely to redefine the disease(s) ( cutting the cake in a completely new way );
3) when there is no obvious rule of translation from the outcomes of the new test to existing treatment guidelines;
4) when clinician behaviour is part of the research question…
…end of digression
![Page 43: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/43.jpg)
Statistical analysis … in the narrow sense: … is very much standard
once you knowwhat aspects to count and compare.
To know that, work backwards from (likely) consequences:
what would have happened to these patients? And
what would have happened in the alternative scenario?
Never argue ”It’s customary to calculate … (this or that)” !
![Page 44: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/44.jpg)
Thank you !
Let me add a personal maxim: Never ask
”What can the journal impact factors do for me?” Ask instead
”What can I do for the journal impact factors?”
![Page 45: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/45.jpg)
Supplementary pictures follow here
… Vassily Vlassov pixit
![Page 46: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/46.jpg)
The rôle of noise
Pure noise, independent of the patient’s true condition,flattens distributions and henceflattens the ROC; less information.
Remedies: technical & procedural standardization, duplicate measurements,
(averaging over assessors, dominance-free consensus formation) …
… may be ineffective if the noise is ”inter-patient”
![Page 47: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/47.jpg)
Three testoutcomes
Definitely positive
+/-
NegativeNeg.?
Presumably positive
Ordered (ordinal) test outcomes
Ordered how?By increasing slope, i.e. LR[ concavity ! ]
Its slope reflects the medical trade-off between % sensitivity and % specificity
Slope? Constant benefit? … Let’s first look at a
continuous test & selection of cutoff that maximizes benefit
An ”iso-benefit” line
![Page 48: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/48.jpg)
The slope chosen so as to imply constant benefit
A continuous test Cutoff at measurement x = c
maximizes benefit
x = c How do we find that critical slope?
It depends on the pre-test ’disease mix’ – and on the (human) loss associated with wrong or suboptimal treatment
- when only two courses of action are available (otherwise there will be more lines, reflecting several trade-offs).
![Page 49: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/49.jpg)
The slope chosen so as to imply constant benefit
A continuous test : Cutoff at measurement x = c
maximizes benefit
x = c
Treat
no-
one
Treat
eve
rybo
dy
![Page 50: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/50.jpg)
The slope chosen so as to imply constant benefit
A continuous test : Cutoff at measurement x = c
maximizes benefit
x = c
Treat
no-
one
Treat
eve
rybo
dy
Without the test, it’s (slightly) betterto treat everybody than to treat no-one.
With the test available, about 60 % of the’misdiagnostic burden’is eliminated;cf. purple bar.
No misdiagnoses
![Page 51: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/51.jpg)
B
A
AB
A or B
Not A
A a
nd
B
A, n
ot B
Parallelogram
Two binary tests and their 6 most important
joint rules of interpretation( positivity criterion BLACK )
Always!
Never!
Three slides that illustrate the pitfalls of combining two or more tests …
![Page 52: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/52.jpg)
Test interpretation rules in Boolean terms
Test result,
T = A OR B
vs.
T = A AND B
T = A OR B
A
B
A B
positive
positive
… but do not talk about tests in parallel
vs. tests in series (next slide)
not A not Bnegative
![Page 53: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/53.jpg)
Suppose Tests A and B both figure in the rule of interpretation adopted.
Next, choose a rule of execution,
sequential or simultaneous
– depending on cost, inconvenience and delays
A first; if needed, B
B first; if needed, A
A & B simultaneously
Beware: careless writers may describe either rule by words like ”parallel” & ”serial” without realizing the ambiguity. Distinguish!
Rule of interpretation vs. rule of execution …
![Page 54: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/54.jpg)
Three existing tests with poorly researched ROC trajectories& their conventional decision points
(cut-off values)
The point is that – unbeknownst to us –
the 3 tests are really equally good (the ROCs nearly coincide):
only historical coincidences, FDA, and other irrelevant factors
make performance look very different.
Remedy...
Sensit.
1 – specif.
![Page 55: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/55.jpg)
Estimate the entire ROC
- with focus on the local slope
- which becomes the LR estimate belonging to the test outcome concerned
- to be weighed into the pre-test odds: (post-test odds) = (pre-test odds)(LR).
Slope +/– statistical uncertainty
Sensit.
1 – specif.
![Page 56: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/56.jpg)
Sensit.
1 – specif.
Actual data (108 infected; 815 non-infected subjects)
An ordinary (mathematically formulated) statistical model, and a more closely fitting (over-fitting?) kernel procedure
![Page 57: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/57.jpg)
Lesions in floating locations
?
Red = as the imagist sees it
![Page 58: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/58.jpg)
Lesions in floating locations
?
Red = as the imagist saw it
Green = surgical truth
How do we score diagnostic performance
in such situations ???
– No neat answer !
![Page 59: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/59.jpg)
Lesions in floating locations
?
Red = as the imagist saw it
Green = surgical truth
1 true positive
1 dubious lesion confirmed apart from extention (two ½-errors)
1 false positive & 1 false negative, but due to proximity surgeons count:
1 true positive finding
An ’infinite’ number of true negative locations
![Page 60: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/60.jpg)
the region-by-region truth may remain unknown
( death from focus A leaves a suspect
location B unresolved ); and
– region-to-region interdependence is common:
– pathogenetic ( a focus at A makes focus at B more likely )
– diagnostic ( a focus at A
makes location B less visible, or
prompts surgery making B directly observable, or
sharpens the imagist’s attention to B, C, …)
– prognostic ( one verified metastatic focus incurability )
– therapeutic (a positive finding, even a false positive one,
may prompt a drug regimen that cures an overlooked
focus [cancelling a false negative blunder] )
Unfortunately,
![Page 61: …Borrelia diagnostics – statistical aspects Jørgen Hilden jh@biostat.ku.dk February 2009 Notes have been added in this file](https://reader036.vdocuments.us/reader036/viewer/2022062305/56649d4d5503460f94a2c2d1/html5/thumbnails/61.jpg)