diagnostic testing 2009

1

Medical Epidemiology

Interpreting Medical Tests and Other Evidence

2

Developmental characteristics: test parameters

2 types of Error: False Positive and False Negative

Disease Yes (D+) No (D-) Total

Positive (T+) a b a+b Test Negative (T-) c d c+d

Total a+c b+d n

3

Developmental characteristics: test parameters

Complements of error rates as desirable test propertiesSensitivity = Pr(T+|D+) = a/(a+c)

Sensitivity is PID (Positive In Disease) [pelvic inflammatory disease]

Specificity = Pr(T-|D-) = d/(b+d)

Specificity is NIH (Negative In Health) [national institutes of health]

Disease Yes (D+) No (D-) Total

Positive (T+) a b a+b Test Negative (T-) c d c+d

Total a+c b+d n

4

Typical setting for finding Sensitivity and Specificity Best if everyone who gets the new test also

gets “gold standard” What is a “gold standard”?

The perfect test, the truth Doesn’t happen Even reverse doesn’t happen Not even a sample of each (case-control

type) Case series of patients who had both tests

5

Setting for finding Sensitivity and Specificity Sensitivity should not be tested in

“sickest of sick” Should include spectrum of disease Specificity should not be tested in

“healthiest of healthy” Should include similar conditions.

6

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

Healthy

7

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

Healthy Sick

8

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

Fals pos= 20% True pos=82%

9

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

Fals pos= 9% True pos=70%

10

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

F pos= 100% T pos=100%

11

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

F pos= 50% T pos=90%

12

13

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

Receiver Operating Characteristic (ROC)

14

Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC)

Receiver Operating Characteristic (ROC)

15

Receiver Operating Characteristic (ROC)

ROC Curve allows comparison of different tests for the same condition without (before) specifying a cut-off point.

The test with the largest AUC (Area under the curve) is the best.

16

17

Clinical Interpretation: Predictive Values

Most test positives below are sick. But this is because there are as many sick as healthy people overall. What if fewer people were sick, relative to the healthy?

18

Clinical Interpretation: Predictive Values

Now most test positives below are healthy. This is because the number of false positives from the larger healthy group outweighs the true positives from the sick group. Thus, the chance that a test positive is sick depends on the prevalence of the disease in the group tested!

19

Clinical Interpretation: Predictive Values

•the chance that a test positive is sick, as well as the chance that a test negative is healthy, are what a physician needs to know.

These are not sensitivity and specificity!

The numbers a physician needs to know are the predictive values of the test.

20

Clinical Interpretation: Predictive Values

Sensitivity (Se)

Pr{T+|D+}

true positives

total with the disease

Positive Predictive Value (PV+, PPV)

Pr{D+|T+}

true positives

total positive on the test

21

Positive Predictive Value Predictive value positive The predictive value of a positive test. If I have a positive test, does that mean I have the

disease? Then, what does it mean? If I have a positive test what is the chance

(probability) that I have the disease? Probability of having the disease “after” you have a

positive test (posttest probability) (Watch for “OF”. It usually precedes the denominator

Numerator is always PART of the denominator)

22

Clinical Interpretation: Predictive Values

T+

D+

T+andD+

23

Clinical Interpretation: Predictive Value

Specificity (Sp)

Pr{T-|D-}

true negatives

total without the disease

Negative Predictive Value (PV-, NPV)

Pr{D-|T-}

true negatives

total negative on the test

24

Negative Predictive Value

Predictive value negative If I have a negative test, does that mean

I don’t have the disease? What does it mean? If I have a negative test what is the

chance I don’t have the disease? The predictive value of a negative test.

25

Mathematicians don’t Like PV-

PV- “probability of no disease given a negative test result”

They prefer (1-PV-) “probability of disease given a negative test result”

Also referred to as “post-test probability” (of a negative test)

Ex: PV- = 0.95 “post-test probability for a negative test result = 0.05”

Ex: PV+ = 0.90 “post-test probability for a positive test result = 0.90”

26

Where do you find PPV?

Table?

NO Make new table Switch to odds

27

Use This Table ? NO Disease

Test Result

+ - Total

+ 95 8 103 - 5 92 97

Total 100 100 200 You would conclude that PPV is 95/103 = 92%

28

Make a New Table

Disease Test

Result + - Total

+ 95 72 167 - 5 828 833

Total 100 900 1000

29

Make a New Table

Disease Test

Result + - Total

+ 95 72 167 - 5 828 833

Total 100 900 1000 Probability of having the disease before testing was 10%. (pretest probability prevalence) Posttest probability (PPV) = 95/167 = 57% So we went up from 10% probability to 57% after having a positive test

30

Switch to Odds

1000 patients. 100 have disease. 900 healthy. Who will test positive?

Diseased 100__X.95 =_95Healthy 900 X.08 = 72

We will end with 95+72= 167 positive tests of which 95 will have the disease

PPV = 95/167

31

From pretest to posttest odds

Diseased 100 X.95 =_95

Healthy 900 X.08 = 72 100 = Pretest odds

900 .95 = Sensitivity__ = prob. Of pos test in dis

.08 1-Specificity prob. Of pos test in hlth

95 =Posttest odds. Probability is 95/(95+72)

72

32

Remember to switch back to probability

33

What is this second fraction?

Likelihood Ratio Positive Multiplied by any patient’s pretest odds

gives you their posttest odds. Comparing LR+ of different tests is

comparing their ability to “rule in” a diagnosis.

As specificity increases LR+ increases and PPV increases (Sp P In)

34

Clinical Interpretation: likelihood ratios

Likelihood ratio

LR+ = Sensitivity/(1-Specificity)

LR- = (1-Sensitivity)/Specificity

35

Clinical Interpretation: Positive Likelihood Ratio and PV+

O = PRE-TEST ODDS OF DISEASE

POST-ODDS (+) = O x LR+ =

YSPECIFICIT - 1

YSENSITIVIT x O

ODDS(+)POST-+1

ODDS(+)POST- = PPV = PV+

36

Likelihood Ratio Negative

Diseased 100_ X.05 =_5__Healthy 900 X.92 = 828

100 = Pretest odds 900 .05 = 1-sensitivity = prob. Of neg test in dis

.92 Specificity prob. Of neg test in hlth(LR-)

Posttest odds= 5/828. Probability=5/833=0.6% As sensitivity increases LR- decreases and NPV

increases (Sn N Out)

37

Clinical Interpretation: Negative Likelihood Ratio and PV-

POST-ODDS (-) = O x LR- =

YSPECIFICIT

YSENSITIVIT-1 x O

38

Remember to switch to probability and also to use 1 minus

39

Post test probability given a negative test

= Post odds (-)/ 1+ post odds (-)

ODDS(-)POST-+1

ODDS(-)POST- -1= NPV = PV-

40

Value of a diagnostic test depends on the prior probability of disease

Prevalence (Probability) = 5%

Sensitivity = 90% Specificity = 85% PV+ = 24% PV- = 99% Test not as useful

when disease unlikely

Prevalence (Probability) = 90%

Sensitivity = 90% Specificity = 85% PV+ = 98% PV- = 49% Test not as useful

when disease likely

41

Clinical interpretation of post-test probability

Don't treat for disease

Do further diagnostic

testingTreat for disease

Probability of disease:

0 1

Testing threshold

Treatment threshold

Disease ruled out

Disease ruled in

42

Advantages of LRs

The higher or lower the LR, the higher or lower the post-test disease probability

Which test will result in the highest post-test probability in a given patient?

The test with the largest LR+ Which test will result in the lowest post-test

probability in a given patient? The test with the smallest LR-

43

Advantages of LRs

Clear separation of test characteristics from disease probability.

44

Likelihood Ratios - Advantage

Provide a measure of a test’s ability to rule in or rule out disease independent of disease probability

Test A LR+ > Test B LR+– Test A PV+ > Test B PV+ always!

Test A LR- < Test B LR-– Test A PV- > Test B PV- always!

45

Using Likelihood Ratios to Determine Post-Test Disease Probability

Pre-test probability of disease

Pre-test odds of disease

Likelihood ratio

Post-test odds of disease

Post-test probability of disease

46

47

Predictive Values

Alternate formulations:Bayes’ Theorem

PV+ =

Se Pre-test Prevalence

Se Pre-test Prevalence + (1 - Sp) (1 - Pre-test Prevalence)

High specificity to “rule-in” disease

PV- =

Sp (1 - Pre-test Prevalence)

Sp (1 - Pre-test Prevalence) + (1 - Se) Pre-test Prevalence

High sensitivity to “rule-out” disease

48

Clinical Interpretation: Predictive Values

PV+ And PV-1 Of Electrocardiographic Status2

For Angiographically Verified3 Coronary ArteryDisease, By Age And Sex Of Patient

Sex Age PV+ (%) PV- (%)

F <40 32 88F 40-50 46 80F 50+ 62 68

M <40 62 68M 40-50 75 54M 50+ 85 38

1. Based on statistical smoothing of results from 78 patients referred to NCMemorial Hospital for chest pain. Each value has a standard error of 6-7%.

2. At least one millivolt horizontal st segment depression.3. At least 50% stenosis in one or more main coronary vessels.

49

Clinical Interpretation: Predictive Values

50

If Predictive value is more useful why not reported?

Should they report it? Only if everyone is tested. And even then. You need sensitivity and specificity from

literature. Add YOUR OWN pretest probability.

51

So how do you figure pretest probability? Start with disease prevalence. Refine to local population. Refine to population you serve. Refine according to patient’s presentation. Add in results of history and exam (clinical

suspicion). Also consider your own threshold for testing.

52

Pretest Probability: Clinical Significance Expected test result means more than

unexpected. Same clinical findings have different

meaning in different settings (e.g.scheduled versus unscheduled visit). Heart sound, tender area.

Neurosurgeon. Lupus nephritis.

53

What proportion of all patients will test positive?

Diseased X sensitivity

+ Healthy X (1-specificity) Prevalence X sensitivity +

(1-prevalence)(1-specificity) We call this “test prevalence” i.e. prevalence according to the test.

54

Combination tests: serial and parallel testing

Combinations of specificity and sensitivity superior to the use of any single test may sometimes be achieved by strategic uses of multiple tests. There are two usual ways of doing this.

Serial testing: Use >1 test in sequence, stopping at the first negative test. Diagnosis requires all tests to be positive.

Parallel testing: Use >1 test simultaneously, diagnosing if any test is positive.

55

Serial Testing

Doing the tests sequentially, instead of together with the same decision rule, is a cost saving measure.

This strategy – increases specificity above that of any of

the individual tests, but – degrades sensitivity below that of any of

them singly. Serial test to rule-in disease

56

Combination tests: parallel testing

Parallel Testing Usual decision strategy diagnoses if

any test positive. This strategy – increases sensitivity above that of any of

the individual tests, but – degrades specificity below that of any

individual test.Parallel test to rule-out disease

57

Clinical settings for parallel testing Parallel testing is used to rule-out

serious but treatable conditions (example rule-out MI by CPK, CPK-MB, Troponin, and EKG. Any positive is considered positive)

58

Clinical settings for serial testing

When treatment is hazardous (surgery, chemotherapy) we use serial testing to raise specificity.(Blood test followed by more tests, followed by imaging, followed by biopsy).

59

Typical setting for finding Sensitivity and Specificity Best if everyone who gets the new test

also gets “gold standard” Doesn’t happen Even reverse doesn’t happen Not even a sample of each (case-

control type) Case series of patients who had both

tests

60

EXAMPLE

Patients who had both a stress test and cardiac catheterization.

So what if patients were referred for catheterization based on the results of the stress test?

Not a random or even representative sample.

It is a biased sample.

61

62

If the test is used to decide referral for gold standard?

Disease No Disease

Total

Test Positive

95 72 167

Test Negative

5 828 833

Total 100Sn95/100 =.95

900Sp 828/900 = .92

1000

63

If the test is used to decide referral for gold standard?

Disease No Disease

Total

Test Positive

95

85

72

65

167

167150

Test Negative

5

1

828

99

833

833 100

Total 100

86Sn85/86=.99

900

164Sp 99/164=.4

1000

64

If the test is used to decide referral for gold standard?

Disease No Disease

Total

Test Positive

85 65 150

Test Negative

1 99 100

Total 86Sn85/86=.99

164Sp 99/164=.4

250