is it true? evaluating research about diagnostic tests
TRANSCRIPT
![Page 1: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/1.jpg)
Is it True? Evaluating Research about Diagnostic Tests
![Page 2: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/2.jpg)
The Case of Baby Jeff
![Page 3: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/3.jpg)
The Case of Baby Jeff
CPK testing for Muscular Dystrophy• Sensitivity: 100%
• Specificity: 99.98%
Prevalence: 1 in 5,000 (0.02%)
![Page 4: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/4.jpg)
Sensitivity 100%
99,980 – no M.D.20 will have M.D.
20 correctly positive
0 false negative
20 false positive
99,960 correctly negative
99,960 negative tests
100% truly negative
0 falsely negative
100% NPV
40 positive tests
50% truly positive
50% falsely positive
50% PPV
Ne
gativ
e res
ultsP
osi
tiv
e re
su
lts
Prevalence = 1 in 5,000 = .02% = 20 newborn boys
100,000 newborn boys
Specificity 99.98%
![Page 5: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/5.jpg)
Why this is important
http://today.msnbc.msn.com/id/42829175
![Page 6: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/6.jpg)
Other examples
Lyme disease• Sensitivity= 95%; specificity= 95%
• High prevalence (20%): PPV =83%
• Low prevalence (2%): PPV = 28% Echocardiogram as part of executive physical
• Prevalence = 10%; PPV = 50%
![Page 7: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/7.jpg)
Technical vs. Clinical Precision
Technical precision Clinical precision
SensitivityThe percentage of patients with the disease who have a positive test
Positive predictive valueThe percentage of patients with a positive test who have the disease
SpecificityThe percentage of patients without disease who test negative
Negative predictive valueThe percentage of patients with a negative test who are without disease.
![Page 8: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/8.jpg)
Predictive Values
Positive Predictive Value• The percentage of patients with a positive test who
have the disease Negative Predictive Value
• The percentage of patients with a negative test who don’t have the disease
![Page 9: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/9.jpg)
Let’s practice
Task 1. A serum test screens pregnant women for babies with Down’s syndrome. The test is a very good one, but not perfect. Roughly, 1% of babies have Down’s syndrome. If the baby has Down’s syndrome, there is a 90% chance that the result will be positive. If the baby is unaffected, there is still a 1% chance that the result will be positive. A pregnant woman has been tested and the result is positive.
![Page 10: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/10.jpg)
Positive: 90% correctly identified
Ne
gativ
e res
ultsP
osi
tiv
e re
su
lts
Prevalence = 1% = ___ patients/1,000?
1,000 similar
Negative: 99% correctly identified
![Page 11: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/11.jpg)
Positive: 90% correctly identified
Ne
gativ
e res
ultsP
osi
tiv
e re
su
lts
Prevalence = 1% = ___ patients/1,000?
1,000 similar
Negative: 99% correctly identified
![Page 12: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/12.jpg)
Positive: 90%correctly identified
990 No Downs10 – Downs
1 false negative
10 false positive
980 correctly negative
981 negative tests
99.99% truly negative
0.001% falsely negative
19 positive tests
47.5% truly positive
52.5 falsely positive
Ne
gativ
e res
ultsP
osi
tiv
e re
su
lts
Prevalence = 1% = 10 with Downs
1,000 similar patients
Negative: 99%correctly identified
Down’s Syndrome
9 correctly positive
![Page 13: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/13.jpg)
Task 2
A 45-year-old woman presents with a sore throat and cough but without fever, tonsillar exudate, or cervical nodes. Using a clinical decision rule, you determine her likelihood of having strep throat is 1%. However, according to your office protocol, your medical assistant already has performed a rapid strep (antigen) test, which is positive. What is the likelihood the patient has strep throat now?
Antigen test -- Sensitivity: 88% Specific: 96%
![Page 14: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/14.jpg)
Sensitivity 88%
99,000 – viral1,000 – Strep
880 correctly positive
120 false negative
3,960 false positive
95,040 correctly negative
95,160 negative tests
99.87% truly negative
0.126% falsely negative
99.87% NPV
4840 positive tests
18% truly positive
82% falsely positive
18% PPV
Ne
gativ
e res
ultsP
osi
tiv
e re
su
lts
Prevalence = 1% = 1,000 with strep
100,000 similar patients
Specificity 96%
Strep throat
![Page 15: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/15.jpg)
Adopting new screening/diagnostic tests
Sensitivity/specificity not enough Testing as an intervention
• Did the authors study an outcome patients care about?
![Page 16: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/16.jpg)
Levels of “POEMness” for Diagnostic Tests
1. Sensitivity & specificity2. Does it change diagnoses?3. Does it change treatment?4. Does it change outcomes?5. Is it worthwhile (to patients and/or society)?
(examples: HbA1C for DM, CPK vs T4/PKU in newborns, electron beam tomography for CAD)
Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making 1991; 11:88-94
![Page 17: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/17.jpg)
Screening pulse oximetry for CHD Diagnostic performance of abnormal pulse oximetry for congenital heart
defects
for all major congenital defects * sensitivity 49.06% * specificity 99.016% * positive predictive value 13.33% * negative predictive value 99.86%
for critical congenital defects * sensitivity 75% * specificity 99.12% * positive predictive value 9.23% * negative predictive value 99.97%
Lancet 2011 Aug 27;378(9793):785
![Page 18: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/18.jpg)
Screening pulse oximetry for CHD
Jaundice, terminating breast-feeding, and the vulnerable child Breast-feeding was more common in the jaundiced group (61% vs
79%). By 1 month, more mothers of jaundiced infants had completely stopped breast-feeding (19% vs 42%). They were more likely to have never left the baby with anyone else (including the father) or left the baby at most one time for less than 1 hour (15% vs 31%), more well-visits, more ED visits (2% v 11%, not including bili measurements).
Thus, may increase the risk for premature termination of breast-feeding and for development of the VULNERABLE CHILD SYNDROME.
Pediatrics 1989 Nov;84(5):773-8
![Page 19: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/19.jpg)
Naming is not curing
In the 1600s, astrology dominated medicine as a healing profession. Neither worked but astrology was much more popular because it focused on fixing people's problems. Medicine, on the other hand, focused mainly on categorizing illnesses (i.e., diagnosing) and not so much on treatment.
400 years later there is still a priority on categorizing, regardless of whether it's helpful. A correct diagnosis is only useful when it results in the selection of a treatment that benefits the patient; otherwise, it's only a label.
James Burke. The day the Universe Changed. Boston: Little, Brown and Company, 1985, p. 333.
![Page 20: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/20.jpg)
Disease No disease
True Positive (TP)
a
False Positive
(FP) b
c False Negative
(FN)
d True Negative
(TN)
TEST +
TEST -
![Page 21: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/21.jpg)
Disease No disease
True Positive (TP)
a
False Positive
(FP) b
c False Negative
(FN)
d True Negative
(TN)
TEST +
TEST -
Sensitivity
![Page 22: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/22.jpg)
Disease No disease
True Positive (TP)
a
False Positive
(FP) b
c False Negative
(FN)
d True Negative
(TN)
TEST +
TEST -
Specificity
![Page 23: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/23.jpg)
Disease No disease
True Positive (TP)
a
False Positive
(FP) b
c False Negative
(FN)
d True Negative
(TN)
TEST +
TEST -
Positive Predictive Value
![Page 24: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/24.jpg)
Disease No disease
True Positive (TP)
a
False Positive
(FP) b
c False Negative
(FN)
d True Negative
(TN)
TEST +
TEST -
Negative Predictive Value
![Page 25: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/25.jpg)
Likelihood Ratios Similar to the concepts of “ruling in” and “ruling out” disease
Pre Test Odds x LR = Post Test Odds
The problem – we don’t think in terms of odds
Clinical decision rules: Do the hard math for us, be we need to enter the appropriate data and interpret results
![Page 26: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/26.jpg)
II. Are The Results Valid?II. Are The Results Valid?
Diagnostic test compared with the “Gold standard” on all patients
Blinded comparison
Independent testing
Consecutive patient enrollment (adequate spectrum of disease)
(Must have all for LOE = 1b)
![Page 27: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/27.jpg)
II. Are The Results Valid?II. Are The Results Valid?
What are the results?
• Sensitivity, specificity and predictive values
• Likelihood ratio calculation
• Prevalence of disease in the study population
• Typical?
• Similar to your practice?
![Page 28: Is it True? Evaluating Research about Diagnostic Tests](https://reader036.vdocuments.us/reader036/viewer/2022062313/56649cb85503460f9497e9bb/html5/thumbnails/28.jpg)
Levels of “POEMness” for Diagnostic Tests
1. Sensitivity & specificity2. Does it change diagnoses?3. Does it change treatment?4. Does it change outcomes?5. Is it worthwhile (to patients and/or society)?
(examples: HbA1C for DM, CPK vs T4/PKU in newborns, electron beam tomography for CAD)
Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making 1991; 11:88-94