Evidence-based Practice

Dr Christine MarshallDr Richard de FerrarsDr Andrew Cochrane

Frimley VTS September 2014

What on Earth Should We Try to Cover?

The right treatment for every disease that you will ever come across?

Everything you ever needed to know about statistics?

Every important paper ever published?

What on Earth Should We Try to Cover?

Why all the fuss granddad?- EBM is so yesterday….

The big issues: - quality, risk, wrong answers

Can you make it work?- CSA and real life

What is Evidence-Based Medicine?

The art of stating

the bleeding

obvious?

There are lies..

Damned lies....

And statistics......Mark Twain

Mysterious magic

& beyond

comprehension?

What is Evidence-Based Medicine?

“The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”

Centre for Evidence-based Medicine

“Evidence based medicine is the integration of best research evidence with clinical expertise and patient values.”

David Sackett

What is Evidence-Based Medicine?

Barriers & Limitations in Primary Care

Let’s set off positively.....

What problems do you see with using EBM in your everyday work?

Time to debate…

EBM has had its day in Primary Care.

We should be glad to show it the door

1.Vote

2.Proponents 5 mins Then Q&A

3.Opponents 5 mins Then Q&A

4.Proponents 2 mins to close

5.Opponents 2 mins to close

6.Vote

Time to Debate

Spot the Flaws…

Patients rarely have one-dimensional problems Evidence is limited with older people, chronic

conditions, complex multiple co-morbidities GPs may use diagnosis by prognosis (watchful waiting)

or therapeutic response rather than pure hypothetical-deductive model

Objective measurable science vs. mysterious art of medicine

“Doctors shaping the square peg of the evidence to fit the round hole of the patient's life.” A Freeman BMJ 2001 323:1

Can it fit into a normal consultation?

What on Earth Should We Try to Cover?

Why all the fuss granddad?- EBM is so yesterday….

The big issues: - quality, risk, wrong answers

Can you make it work?- CSA and real life

Quality of data

Quality of sources of data

Let’s talk high quality…

Quality Data?

1. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

2. The NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

3. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

4. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

5. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

Rate these 5 fonts of wisdom in order of “good & sound”

1. The NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

2. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

3. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

4. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

5. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

Rate these 5 fonts of wisdom in order of “good & sound”

Quality Data?

1. The NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

2. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

3. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

4. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

5. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

Rate these 5 fonts of wisdom in order of “good & sound”

Quality Data?

1. The NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

2. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

3. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

4. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

5. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

Rate these 5 fonts of wisdom in order of “good & sound”

Quality Data?

1. The NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

2. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

3. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

4. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

5. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

Rate these 5 fonts of wisdom in order of “good & sound”

Quality Data?

1. The NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

2. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

3. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

4. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

5. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

Rate these 5 fonts of wisdom in order of “good & sound”

Quality Data?

Meta-analysis & systematic review – aggregation of several similar studies

Double-blind randomised placebo controlled studies Larger, generic products, paid for by neutral body

Double-blind randomised placebo controlled studiesSmaller, branded products, paid by the manufacturer

Observational studies

Case reports

Anecdotal experience

Big is Beautiful!

Good Studies & Bad studies – Study Hierarchy

Hierarchy of Evidence

Big is Beautiful!

Hierarchy of Evidence

Quality of Evidence - A,B C or D Grade A

Strong research-based evidence, at least one RCT, drawn from high quality scientific studies coming to same conclusions

Evidence levels:

Ia = randomised controlled trials

Ib = at least one randomised controlled trial

Grade BModerate research-based evidence drawn from

well controlled studies.

Evidence levels:

IIa = at least one well-designed controlled study without randomisation

IIb = at least one other type of well-designed quasi experimental study

III = well-designed non-experimental descriptive studies.

Quality of Evidence - A,B C or D

Grade CLimited research-based evidence drawn from

expert reports

Evidence level:

IV = expert committee reports or opinions and/or clinical experience of respected authorities

Grade D No scientific evidence.

Quality of Evidence - A,B C or D

What sources do you use? Peers

Teaching session Asking your trainer

Daily Mail Journals Medical Text Books Drug Reps EBM Reference Books EBM Websites Internet - Google

How Good?Good but...

What do they really know?

No commentGood but hard to accessGood but out of dateYou must be joking...Good but out of date

Need to know...

Quality Sources of Data

What sources do you use? Peers

Teaching session Asking your trainer

Daily Mail Journals Medical Text Books Drug Reps EBM Reference Books EBM Websites Internet - Google

How Good?Good but...

What do they really know?

No commentGood but hard to accessGood but out of dateYou must be joking...Good but out of date

Need to know...

Quality Sources of Data

EBM Websites:

Centre for Reviews & Dissemination (CRD Uni York) – Database

NHS Evidence

Cochrane Library – Browse Database

TheNNT.com – Fun & more later…

BMJ Clinical Evidence – Subscription service (£150 per year)

Google Search? - Use Google Scholar

- Use Advanced Search

Quality Sources of Data

Quality of data

Quality of sources of data

Let’s talk high quality…

What on Earth Should We Try to Cover?

Why all the fuss granddad?- EBM is so yesterday….

The big issues: - quality, risk, wrong answers

Can you make it work?- CSA and real life

Taking Risks

Relative risk vs. Absolute riskOccurrence of Diabetic retinopathy among IDDM @5yrs in DCCT trial

Usual Regime Intensive Regime

38%(Control Event Rate)

13%(Experimental Event Rate)

Absolute Risk in intensive group = 13% Relative Risk in intensive group = 13/38 =

34% Relative Risk Reduction (RRR)

(CER-EER)/CER (38-13/38 = 25/38 = 66%) Absolute Risk Reduction (ARR)

EER-CER (13-38 = -25%)

Taking Risks

Relative risk vs. Absolute riskAnnual Stroke Risk in High-risk AF Patients

Aspirin Warfarin

12%(Control Event Rate)

6%(Experimental Event Rate)

Absolute Risk in Aspirin group Relative Risk in Warfarin group Relative Risk Reduction (RRR)

(CER-EER)/CER Absolute Risk Reduction (ARR)

EER-CER

= 12%= 6/12 = 50%

12-6/12 = 6/12 = 50%

12-6 = -6%

Taking Risks

Bluffer’s Guide to Risks

Relative Risk Used by drug companies to make their product

look good. Can make small numbers look bige.g. 0.002-0.001/0.002 = 50%

Absolute Risk Used to show how rare side effects are

Not simply absolute = goodrelative = bad

Numbers Needed to Treat

The number of patients that need to be treated to create one additional favourable outcome Calculated as 1/ARR

(or how many ARRs are needed to make 100)

DCCT trial, ARR was 25 NNT is therefore 4 Treat 4 IDDM diabetes patients intensively for

5 years to prevent 1 diabetic retinopathy

Occurrence of Diabetic retinopathy among IDDM @5yrs in DCCT trial

Usual Regime Intensive Regime

38%(Control Event Rate)

13%(Experimental Event Rate)

Bluffers Guide to NNT

The trendy statistic to ask for.Easy to understand & explain to patients

Lower numbers better! Still needs thinking about and evaluating

Remember the balance - NNH Rough Guide:

NNT under 10 “very, very interested” NNT 10-50-100 “probably, possibly” NNT over 100 “how much?????”

Look at TheNNT.com

DBP 100-115. Any ideas about 5-year NNT? 5-year NNT is 120 Stroke-rate untreated = 2.4% (RRR 30%) Stroke rate treated = 1.6% ARR = 0.8%

What happens to 100 treated patients? How many would have had a stoke (untreated)? How many do have a stroke (treated)? In how many have you made a difference?

Statins & IHD risk? Any idea of NNTs?

NNT – Example with BP

What on Earth Should We Try to Cover?

Why all the fuss granddad?- EBM is so yesterday….

The big issues: - quality, risk, wrong answers

Can you make it work?- CSA and real life

Sense and Specificity

NOT a Jane Austen novel If only medical statistics made us so happy...

“It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.”

Pride & PrejudiceJane Austen

Sense and Specificity

Sensitivity = a/ a+cProportion of people with the target disorder who have a positive test

Specificity= d/ b+dProportion of people without the target disorder who have a negative test

True Diagnosis

Positive Negative

True Result

Positive a b

Negative c d

Bluffers guide to Sense & Specificity

SnNout When a sign/test/symptom has a high

Sensitivity, a Negative result rules out the diagnosis.

SpPin When a test has a high Specificity, a Positive

result rules in the diagnosis.

Memorise for the AKT....

True Diagnosis

Positive Negative

True Result

Positivea

(TP True Positive)

b(FP False Positive)

Type 1 error

Positive Predictive

ValuePPV = a/a+b

Negativec

(FN False Negative)

Type 2 error

d(TN True Negative)

Negative Predictive

ValueNPV = d/c+d

Sensitivitya/a+c

Specificityd/b+d

Sensitivity = a/ a+cProportion of people with the target disorder who have positive test

Specificity= d/ b+dProportion of people without the target disorder who have negative test

Treatment? Test? Is it any use?

Your patients rely on your

knowledge and opinion as to

how useful a particular test/

treatment actually is

Treatment: NNT NNH

Tests: Spin & Snout

As useful as…

What on Earth Should We Try to Cover?

Why all the fuss granddad?- EBM is so yesterday….

The big issues: - quality, risk, wrong answers

Can you make it work?- CSA and real life

Disease- Illness Model (Stewart & Roter 1989)

Is my practice evidence-based?Have you…

1. Identified and prioritised the clinical,

psychological, social and other problems,

taking into account the patient’s

perspective?

And..

2. Performed a sufficiently competent and

complete examination to establish the

likelihood of competing diagnoses?

And…

3. Considered additional problems and risk

factors?

And….

4. Where necessary, sought relevant

evidence - from systematic reviews,

guidelines, clinical trials, and other

sources

And…..

5. Assessed and taken into account the

completeness, quality and strength of the

evidence, and its relevance to this patient?

And…...

6. Presented the pros and cons of the

different options to the patient in a way

they can understand, and incorporated the

patient’s preferences into the final

recommendations???

Not asking much, are we?

Is my practice evidence-based?Can you do it ??????

“Why, when I was your

age, sometimes I've

believed as many as six

impossible things

before breakfast.”

The Queen of Hearts

Alice in Wonderland

Patient Preferences in Treatment Decisions

Time for some role-play vignettes Split into 3 groups Take up the challenge:

Talking risks & NNT (NNH) Dodgy tests

Patient Preferences in Treatment Decisions

Feedback from what you observedHow would you do this in the CSA?

Find out what the patients knows (wants to know) Avoid jargon Avoid percentages & fractions (100 people...) Use chunks & checks Remember patient autonomy... How do you cope with an unreasonable

expectation?

The End