Medical decision making

2

Predictive values

• 57-years old, Weight loss, Numbness, Mild fewer• What is the probability of low back cancer?• Base on demographic prevalence ~20%• Should he receive the

• low-cost ESR-test with sensitivity of 78% and specificity of 67%

• Expensive MRI scanning with sensitivity and specificity of 95%

• surgery at once• be send home

3

The threshold model for a patient with low back pain

• The doctor worries about cancer, should the doctor send him home, test, or treat him?

4

The accuracy of a test

• Sensitivity; The ability to detect patients with the condition• Definition: The probability of a positive result in patients who

have the condition: True Positive rate• High sensitivity ↔ few false negative

• Specificity; The ability to detect patients without the condition• Definition: The probability of a negative result in patients who

do not have the condition: True negative rate or1- False Positive rate

• High specificity ↔ few false positive

5

The accuracy of a test

• Accuracy = (TP + TN) / total N

Known truth

Positive D+ Negative D-

Test

Positive T+

TP (true-positive)

FP (false-positive)

Negative T-

FN (false-negative)

TN (true-negative)

6

The accuracy of a ECG test for myocardial infarction

• Sensitivity:• TP ratio: 6/31 = 0.19

• Specificity:• TN ratio: 59/72 = 0.82

Known truth

MI Present MI Absent

Test

ST > 5mm

TP: 6 FP: 13

ST < 5mm

FN: 25 TN: 59

Total 31 72

7

Predictive values

• Assuming 20% chance for this specific patient

• ESR test has sensitivity = 78% and specificity = 67% (Joines et al. 2001)

• TP = 78% of 200 = 156• TN = 67% of 800 = 536• FN = 200 – 156 = 44• FP = 800 – 536 = 264• 156 + 264 = 420 tested

positive• 44 + 536 = 580 tested

negative

D+ D-

Test

T+ TP: 156 FP: 264 420

T- FN: 44 TN: 536 580

Total

200 800 1000

8

Predictive values

• PV+ = 156/420 = 0.37• PV- = 536/580 = 0.92• If the test is positive we are

37% sure that it is spinal cancer

• If it is negative we are 92 % sure it is not

D+ D-

Test

T+ TP: 156 FP: 264 420

T- FN: 44 TN: 536 580

Total

200 800 1000

9

Predictive values of MRI scan is ESR tested positive

• Assuming 37% chance for this specific patient

• FMR test has sensitivity = 95% and specificity = 95% (Joines et al. 2001)

• PV+ = TP/(TP+FP) = 0.92• PV- = TN/(TN+FN) = 0.97

D+ D-

Test

T+ TP: 351.5

FP: 31.5 383

T- FN: 18.5

TN: 598.5

617

Total

370 630 1000

10

Likelihood ratio

• Assuming 20% chance for this specific patient• ESR test has sensitivity = 78% and specificity = 67% (Joines et al.

2001)• Pretest odds = Prior probability / (1 - Prior probability) = 0.2 / (1 –

0.2) = 0.25• Likelihood ratio (LR) = sensitivity / false positive rate = 0.78 / (1 –

0.67) = 2.36• Posttest odds = 0.25*2.36 = 0.59

11

Probability vs. odds

• Posttest odds = 0.59• PV+ = 0.37

1 1

0.59Posterior probability = 0.37

1+0.59

p oo p

p o

12

Negative likelihood ratio

• Positive likelihood ratio (+LR) = 2.36 • Is the likelihood ratio that he has the disease if he is tested

positive• Negative likelihood ratio (-LR)• Is the likelihood ratio that he has the disease if he is tested

negative

- FN ratio 0.22LR= 0.33

TN ratio 0.67

+ TP ratioLR=

FP ratio

13

Test with continuous outcome

• What if the test outcome is continuous? Which threshold should be chosen?

• Optimizing Specificity and sensitivity• Increasing sensitivity at to loss of specificity

14

Receiver operating characteristics (ROC) curve

• X-axis: 1-Specificity• Y-axis: Sensitivity• The ROC curve describes the test.• Poor test → large overlap → ROC curve

close to diagonal• Good test → little overlap → ROC curve close

to vertical / horizontal

15

Receiver operating characteristics (ROC) curve

From wikipedia

16

ROC curve to test beast cancer by mammography

Status Normal1

Benign2

Probably benign

3

Suspicious4

Malignant5

Total

Cancer 1 0 6 11 12 30

No Cancar

9 2 11 8 0 30

17

ROC curve to test beast cancer by mammography

• How good is the test? • Where to put the threshold?

Status Normal1

Benign2

Probably benign

3

Suspicious4

Malignant5

Total

Cancer 1 0 6 11 12 30

No Cancar

9 2 11 8 0 30

Threshold <1 1.5 2.5 3.5 4.5 >5

TPR (sensitivity)

30/30 = 1.00

29/30 = 0.97

29/30 = 0.97

23/30 = 0.77

12/30 = 0.40

0/30 = 0.00

FPR(1-specificity)

30/30 = 1.00

21/30 = 0.70

19/30 = 0.63

8/30 = 0.27

0/30 = 0.00

0/30 = 0.00

18

ROC curve to test beast cancer by mammography

• How good is the test? • Where to put the threshold?

Threshold <1 1.5 2.5 3.5 4.5 >5

TPR (sensitivity)

1.00

0.97

0.97

0.77

0.40

0.00

FPR (1-specificity)

1.00

0.70

0.63

0.27

0.00

0.00

19

The area under the ROC curve

• The area under the ROC gives the intrinsic accuracy of a diagnostic test and can be interpreted in several ways (see Hanley et al.):

• The average sensitivity for all values of specificity• The average specificity for all values of sensitivity• The probability that the diagnostic score of a diseased patient is

more of an indication of disease than the score of a patient without the disease.