Evidence Based Medicine and Evidence Based Medicine and Medical Decision MakingMedical Decision Making

Iztok Hozo, Professor of MathematicsIndiana University Northwest

European School of OncologyHow to Practice Evidence Based Oncology22-24 July, 2004. Antwerp, Belgium

(bug chunks taken from Ben. Djulbegovic – with permission)

Theories of decision-making• normative theories

• what people “ought to do”• axiomatic theories based on mathematical andstatistical proofs, usually on expected utility theory• the rationale or best course of action is the onethat maximizes expected utility

• descriptive theories• how people actually make decisions (“is” vs. “ought to”)• people rarely make decisions in accord with normative theories (e.g. to avoid regret associated with wrong decisions)

• prescriptive theories• since humans can be poor decision makers, prescriptive theory is concerned with the development of decision aids• based on modification of normative theories, but also integrate other processes (such as attitudes, biases, values, etc)

Decision-MakingDecision-Making

• how to deal with the uncertainty

Evidence and (normative) decision-making

Evidence Based Medicine

The main focus of EBM has been understanding treatmenteffects (benefits and harms) usually expressed as one of the EBM therapeutic summary measures.

These summary measures relate to the effects of the treatmenton morbidity and mortality of a disease.

EBM therapeutic summary measures: Benefits

2

12

Mrx

MrxMrx

M

MrxME

MEMrx 1

relative risk reduction (efficacy, E, RRR) = the proportional reduction in rates of bad events (deaths) between experimental (Mrx) and control (no treatment) (M) group.= the proportional reduction in rates of bad events (deaths) between experimental (Mrx1) and control (Mrx2) treatment group.

211 MrxEMrx

EBM therapeutic summary measures: Benefits

absolute risk reduction (risk difference, RD, ARD)= the actual difference in rates of bad events between experimental (Mrx, Mrx1) and control (no treatment, treatment2) (M, Mrx2) group.

12 MrxMrxMrxMRD number needed to treat (NNT)

= the reciprocal of the actual difference in rates of bad events between experimental (Mrx, Mrx1) and control (M, Mrx2) group.= the number of patients who need to be treated with the experimental treatment in order to prevent one bad outcome or attain one good outcome

2

1

12

1

111

MrxEMrxMrx

MEMrxMRDNNT

EBM therapeutic summary measures: Treatment Harms

Rates of adverse events due to treatment (R)

number needed to harm (NNH)= the reciprocal of the actual difference in rates of bad adverse events between experimental (R, R1) and control (R2) group.= the number of patients who must be treated with the experimental treatment in order for one to experience a harmful event.

RNNH

1

21

1

RRNNH

Decision Analysis

is an explicit, quantitative method of clinical decision makingthat involves the separation of the probabilities of events fromtheir relative values, or utilities.

Utilities associated with a particular clinical outcome can beexpressed in different units such as length of life, adjusted quality of life, morbidity or mortality rates, absence of pain, dollar value, or the strengthof individual patient preference for an outcome.

Decision Analysis

In choosing among several competing clinical scenarios,the optimal decision rests on selection of the strategy with thehighest expected value, which is calculated by computingthe average utilities of all possible results, weighted by theircorresponding probabilities.

A Simple Decision Tree

The first decision (blue square) is made by the physician.

The second decision (green circle) is determined by the probability of the disease.

EBM + MDM

If utilities can be expressed as the probability of freedom from the consequences of disease or the toxicity of treatment and if EBM therapeutic measures relate to the effects of the treatment on disease morbidity or mortality, then it is possible to integrate EBM indices within the framework of decision analysis.

A Simple Model (EBM utilities)

Defining outcome utilities:

U1 = U[D+,Rx] = (1-Mrx)*(1-R) = 1-Mrx-R+Mrx*R, orU1 =1-Mrx-R (Mrx*R 0 since the probability of Mrx & R occurring simultaneously in practice is usually nil; e.g. patient on chemoRx cannot die of breast cancer and toxic effects of chemoRx at the same time)

U2 = U[D-,Rx1] = 1-R

U3 = U[D+,NoRx] = 1-M

U4 = U[D-,NoRx] = 1

Treatment

No Treatment

Disease

Disease

No Disease

No Disease

1 - MRX -R

1 - R

1 - M

1

p

1 - p

p

1 - p

Treatment vs. No Treatment

Integration of EBM therapeutic measures within decision analysis

Find the threshold probability, pt, at which we are indifferent between Rx vs NoRx:

The two expected values are equal whenp*U1+(1-p)*U2 = p*U5+(1-p)*U6

Or in case of our utilities, p*(1-Mrx)*(1-R) +(1-p)*(1-R) = p*(1-M)+(1-p)*1

The solution of this equation is:

Expected value of Treatment is E[Rx] = p*U1+(1-p)*U2

Expected value of not giving Treatment is E[NoRx] = p*U5+(1-p)*U6

Expected Values

The Threshold

If the probability of a disease, pD, is greater than pt, then treatment should be given.

If pD < pt , the treatment is not indicated.

NNH

NNTNNTR

ME

R

MrxM

Rpt

A Clinical ExampleKearon C, Gent M, Hirsh J, et al.: A comparison of three months of anticoagulation with extended anticoagulation for a first episode of idiopathic venous thromboembolism. N Engl J Med 1999; 340: 901-7 :

a study in which they randomized patients who already completed a 3 month course of warfarin to determine if longer anticoagulation would be beneficial in the prevention of deep venous thrombosis (DVT) recurrence.

the NNT for the prophylaxis of DVT recurrence is 4, i.e., 4 patients need to be treated with warfarin for 1 year in order to prevent one episode of DVT.

however, the optimal duration of treatment needs to be interpreted in light of not only the benefit but also the harm of warfarin treatment.

While an NNT of 4 seems to represents a very effective therapy, this measure alone does not provide an answer to the question if this treatment is better than the alternative management strategy of observation without active treatment. To begin to address the clinical question whether to give warfarin or not, we note in the study by that the annual risk of major bleeding was 3.8% (compared to zero in placebo arm) representing an NNH=26. Based on the threshold analysis presented here, warfarin should be administered if the probability of DVT recurrence is greater than the threshold p = 15% (4/26). In this study, the recurrence rate for DVT was 27.4% per year suggesting that warfarin treatment should be continued beyond the initial 3 months of treatment in typical patients meeting eligibility criteria described in the Kearon study.

Treatment 1

Treatment 2

Disease

Disease

No Disease

No Disease

1 - MRX1 -R1

1 - R1

1 - MRX2 - R2

1 - R2

p

1 - p

p

1 - p

Treatment 1 vs. Treatment 2

Find the threshold probability, pt, at which we are indifferent between administering treatment Rx1 or treatment Rx2.

If the probability of a disease, pD, is greater than pt , then treatment should be given. If pD < pt , the treatment is not indicated.

Threshold in case of two treatments

1221

21

21

MrxMrx

RR

EEM

RRpt

B) Minimal necessary efficacy at which therapy is worth considering (Rx1 vs Rx2):

The following inequalities must be satisfied to even consider treatment Rx1as opposed to alternative treatment Rx2:

221

1 EM

RRE

Or 21

211

RR

EE

M

Or

21

21

11

NNTRR

NNT

Or 221

1 RBIM

RRRB

Or )21(21 RRSrxSrx

For example, as intuitively expected we should only give the treatment thatprovides better survival adjusted for risk difference between two treatmentoptions.

When testing is an option:

If the question is whether to administer treatment, perform test or continue observation, the solution of the model that includes testing as an option is provided by (the solution for riskless test only is provided):

where LR+ is the positive likelihood ratio and is used in the case of the testing threshold (p=ptt) and LR- is is the negative likelihood ratio of the test and is used in the case we want to determine test-treatment threshold (p=prx). Note that we should not even consider ordering the diagnostic test if treatment risk (R) is greater than its efficacy (E)(since pt<0).

11

1

RME

LRpt

Test threshold formulas

)()1(

)1(

rxerxp

terxp

tt HMESHS

HHSp

)()1( rxerxp

terxp

rx HMESHS

HHSp

If the probability of the disease (event) is less than ptt , continue with observation. If the probability is between the values of ptt and prx , order the test. Finally, if the probability is larger than prx , administer the treatment.

Conclusions

• EBM therapeutic summary measures are utilities andalone cannot be used in medical decision making

• Effective integration of EBM therapeutic summary measures of the treatment benefits and harms requirestheir linking to decision analysis

• When EBM therapeutic summary measures are linkedto decision analysis, some new principles of clinicaldecision making emerge (such as never administer treatment or order a diagnostic test if treatment risk isgreater than its efficacy)

Java Script Threshold Calculator

HTTP://www.hsc.usf.edu/~bdjulbeg/

http://www.iun.edu/~mathiho/medmath/medmath.htm