Facets of Teaching Evidence Based Health Care

The OU Family Medicine Experience

Dewey Scheid MD, MPHClinical Decision Making Program Department of Family and Preventive MedicineThe University of Oklahoma Health Sciences Center

Learning to Practice and Teach Evidence Based Health Care Second Annual Workshop September 21-22, 2007 The University of Oklahoma Health Sciences Center

PRACTICE-BASED LEARNING AND IMPROVEMENT - ACGME

Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices. Residents are expected to:

analyze practice experience and perform practice-based improvement activities using a systematic methodology

locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems

obtain and use information about their own population of patients and the larger population from which their patients are drawn

apply knowledge of study designs and statistical methods to the appraisal of clinical studies and other information on diagnostic and therapeutic effectiveness

use information technology to manage information, access on-line medical information; and support their own education

facilitate the learning of students and other health care professionals

Six Facets of EBM at OU DFPM

Journal Club CDM Conferences MS3 Clerkship MS4 Elective CLIN-IQ CQI

Journal ClubA Brief Evolutionary History

Neal Clemenson, beer, and EBM

Laine McCarthy, HRSA EBM grant, 1992 UVa Information

Mastery Program Users' guides to the

medical literature After dinner articles Back to Wednesday

afternoon “Just in Time” Journal

Club

The Evolution of Journal ClubSurvival of the Fittest - Principles

If they read it, they will come

I just look at the pictures

Noble “cheating” I like to watch News! EBM Nazis EBuMers Cassandra syndrome

EBM, CDM, MDM, Clinical EpiSurvival of the Memes

Archie Cochrane Effectiveness and

Efficiency: Random Reflections on Health Services (1972)

“Best evidence" methodologies were established by the McMaster University research group led by David Sackett and Gordon Guyatt.

“Evidence based" 1990 by David Eddy.

“Evidence-based medicine" 1992 by Guyatt et al.

Introduction to Medical Decision Making by Lee B. Lusted, 1968 “uncertainty about the correlation of signs, symptoms, and diseases makes medical diagnosis a matter of probability”

Society for Medical Decision Making founded in 1979

SG Pauker, and JP Kassirer. The threshold approach to clinical decision making. NEJM 1980.

John Paul Alvan Feinstein Annals

series 1968 David Sackett

CDM – Clinical Decision Making

Evidence Based Medicine: What it is and what it isn’t. Sackett, BMJ 1996 Best external evidence Individual clinical expertise Patient choice

What is CDM? Decision analysis Psychology of decision making

CDM – Decision Analysis

UTI / OV,RX,SE1

pUTINo UTI / OV,RX,SE2

1-pUTI

Side effects

pSE

UTI / OV,RX

pUTINo UTI / OV,RX

1-pUTI

No side effects

1-pSE

Trim/Sulfa

UTI,SE/OV,CX,RX,SE

pSEUTI,NoSE/OV,CX,RX

1-pSE

CXPOS/UTI

SN

CXNEG/UTI/OV,CX,MDX

1-SN

UTI

pUTI

NoUTI,SE/OV,CX,RX,SE

pSENoUTI,NoSE/OV,CX,RX

1-pSE

CXPOS/NoUTI

1-SP

CXNEG/NoUTI/OV,CX

SP

No UTI

1-pUTI

Urine Culture

UTI / MDX

pUTINo UTI / OV

1-pUTI

Observe

Dysuria

CDM – Rolling back the tree

CDM Curriculum

Test characteristics Sn, Sp, PVP, NVP, priors Likelihood ratios ROC curves

Deconstructing normal CDM calculators: bgphthut.xl4

Bird Library/Help/Evidence Based Info/Other Resources

CDM Curriculum - Risk

EBM Risk RR vs. absolute risk -> NNT Communicating risks to physicians in

the literature Communicating personal risks to

patients

CDM Curriculum - Communicating personal risks to patients

CDM Curriculum - Communicating personal risks to patients

14

Imagine a group of 1,000 men like you, who are currently in their mid 50’s. You could be any of these men.

By the time these men reach age 70, about 780 (78%) would not have prostate cancer.

If this cancer is not discovered early and treated, about 66 of the 220 men (30%) will eventually experience symptoms of advanced prostate cancer and most will die of it.

The other 154 men (70%) would not be severely affected by the cancer and would die of other causes.

By the time these men reach age 70, about 220 men (22%) would have cancer cells in their prostates.

What are the Chances that a Man Might get Prostate Cancer?

15

Consider the 1000 men in their mid-50’s.

• As we said in the first display (page 4), by the time these men reach age 70 about 220 of them would have prostate cancer cells.

• This is what happens if they get screened

every year (annually) until age 70:

Below are 179 men with Prostate Cancer Cells who do not get any Treatment (The Watchful Waiting

Option)

Without treatment about 58 of these 179 men (32%) will eventually have advanced prostate cancer.

These men can be treated at that time with hormone control or chemotherapy, but cannot be cured.

Without treatment about 121 of these men (68%) will never experience advanced prostate cancer.

Annual screening would discover prostate cancer in about 179 of these 220 men (81%).

Annual screening would miss finding prostate cancer in about 41 men (19%).

What happens if the 179 men who know they have prostate cancer get treated or do not get treated?

Will Prostate Cancer Treatment Save a Man’s Life?

Why is EBM so hard?

CDM Curriculum – Physician decision making psychology

Scripts Collection of patterns and knowledge

structures Pattern recognition quickly identifies

appropriate knowledge structure for situation Rules within structures are tuned by

experience Surgical Intuition. What It is and How to Get It.

By CHARLES M. ABERNATHY and ROBERT M. HAMM

Ways to Improve Decision Making

1. Education2. Guidelines,

reminders3. Pull

information at point of care

4. Decision tools

5. CQI 6. Outcomes

studies

44

RecognitionRecognition--primed Decision Makingprimed Decision Making

Reassess Reassess situationsituation

Seek more Seek more informationinformation

Is the situation Is the situation familiar?familiar?

NoNo

YesYes

Are expectancies Are expectancies violated?violated?

YesYes

NoNo

Recognize the situationRecognize the situation

GoalsGoals CuesCues

ExpectanciesExpectancies Actions Actions 1...n1...n

Imagine Action Imagine Action (1)(1)

Will it Will it work?work? NoNo

YesYes

ImplementImplement

ModifyModifyYes, butYes, but

Experience the Situation in a Changing ContextExperience the Situation in a Changing Context

55 66

11

44

221,31,3

CDM Conferences

Wednesday afternoon – 1 hour

24 month schedule 15 - 30 minutes

DiagnosisRisksTreatmentOutcomesEthicsRisk communicationPhysician decision making Patient decision makingDecision analysisGuidelines, algorithmsHealth care policy

CDM – Lessons learned the hard way

Cod liver oil Spoonful of sugar Less is more Paradigms: uncertainty vs.

pathophysiology

Teaching Clinical Decision Making in a Family Practice Clerkship

CDM module imbedded in a third-year medical student (MSIII) family medicine clerkship

David R. Holtgrave, Frank H. Lawler, L. Peter Schwiebert Teaching Clinical Decision Making in a

Family Practice Clerkship. Med Decis Making 1993;13:114-117.

John Zubialde and Dewey Scheid until 2000 Diagnosis: sensitivity, specificity, PPV, NPV,

impact of prevalence Decision analysis basics

CDM MS4 Elective

Read and do exercises to learn basic concepts of clinical decision making.

Learn to use computer tools for analysis of decisions CDM calculators DATA

Consult frequently with faculty (Rob Hamm, Dewey Scheid).

Plan a month long project, carry it out, and report it in a brief paper.

CDM MS4 Elective Projects

Decision analysis of the use of tPa for stroke Evaluating post exposure prophylaxis for HIV

infected needle sticks Screening diabetics for microalbuminuria Management of fever without source in children

from 3 to 36 months in the post-H. influenza era: a decision-making analysis

Cost effectiveness of a three day inpatient stay versus seven day inpatient stay for bipolar type manic episode

Valproate use in pregnancy Decision Analysis of Trial of Labor After Cesarean

Clin-IQ Project

Critical Appraisal of Clinical Questions by Family Medicine Residents

Oklahoma Physicians Resource/Research Network (OKPRN) {and medical student and their 4th year Rural Preceptors} generate the questions

A panel of OKPRN members prioritizes the list of questions

Pairs of family practice residents (PGY2 and PGY3) select from the higher priority questions.

Residents work with a faculty mentor. Mentoring occurs in one on one sessions with faculty

focused on how to answer the questions. Multiple sessions with research division faculty to

assist residents in literature search.

Clin-IQ Project

After the resident-faculty groups complete their work, the residents present their answers to each other in small group meetings.

Feedback is used to refine the answers. The project is then reviewed by a panel of veteran

researchers who provide a written critique of the work.

Further refinement is made as a result of this review, and then the project is submitted for evaluation and dissemination to the OKPRN clinicians, 4th year Rural Preceptors, and the medical students who generated the questions.

Some questions have been published in the Journal of the Oklahoma State Medical Association

CLIN-IQ

What is the Most Sensitive Non-invasive Test for Initial Diagnosis of H.Pylori Infection in Adults?

Authors: Payne, I; Mold, JW Journal- Oklahoma State Medical Association, 2006, vol. 99,

no. 6, pp. 368-369 Clinical Question: Is Insulin Glargine More Effective?

Authors: Kitowicz, A; Criswell, DF Journal- Oklahoma State Medical Association, 2007, vol. 100,

no. 1, pp. 26-27 Clinical Question: Does Treatment with Corticosteroids

Improve Pain Outcome in Patients with Acute Pharyngitis? Authors: Rezaei, A; Criswell, D Journal- Oklahoma State Medical Association, 2007, vol. 100,

no. 2, pp. 49-51

CQI

Continuous Quality Improvement Committee of the UFMC

All divisions involved with the clinical activities of the department are represented.

Since 1998, collaborated with the faculty members responsible for the PGY2 community medicine rotation to engage residents in the process of CQI.

Residents, in groups of 2-3, select a clinical service that has been recognized nationally by quality improvement organizations as a priority for monitoring and improvement.

They conduct an audit of Family Medicine Center medical records, complete a report, and present their findings at the monthly CQI committee meeting.

CQI

Approximately 4-6 audits are performed each year.

The audit results are discussed and often used as a springboard for further quality improvement action.

Examples of audits include: pneumococcal immunization, breast cancer screening, PAP screening results management, cholesterol screening management, microalbumin screening in diabetics, tobacco cessation counseling.

CQI TITLE: OUFMC QUALITY CARE REVIEW OF ASPIRIN THERAPY FOR PATIENTS WITH DIAGNOSIS OF

CAD (ICD9 414.00) DATE: 08/07/2007 PREPARED BY: Gregory Grant MD, David Speegle MD, and Darice Wiegel MD. PERFORMED BY: GREGORY GRANT AND DAVID SPEEGLE METHODS: We selected 300 charts with the ICD-9 code 414.00, Coronary Artery Disease for the

purpose of reviewing documentation that the patient was on aspirin therapy. Only patients who had visited OUFMC once within the past 2 years were selected. Of the 300 hundred patients identified, 103 were selected at random for review. Patients with Aspirin listed on their medication list were considered to be on Aspirin therapy. Patients who did not have aspirin listed were examined further in both the face sheet and clinic notes for the previous 2 years to check for a documented aspirin adverse reaction or refusal of therapy. Adverse reaction included: GI bleed, any variation of GI intolerance, rash, or angioedema. If a patient was not on aspirin therapy, but was on other anticoagulation, then that was noted.

RESULTS: See Table and Graph attached. DISCUSSION: With only 74% of patients on aspirin therapy, 79% of patients on some type of

anticoagulation and a total of 85% either on anticoagulation or with a documented AVR, there is still a large room for improvement. We should be able to achieve 100% of patients on anticoagulation or a documented AVR. In order to quality for P4P on this issue, we will have to achieve close to this. Although the only ICD-9 code researched in this review was for CAD, a further review is warranted to include all patients with diagnosis of previous MI as well. It is likely that close to 100% of our patients have been screened for ASA therapy, but documentation must be improved in order to demonstrate that it actually is being done.

ACTION: Issue memo encouraging physicians to document clearly on face sheet any adverse reaction to any medication. Encourage physicians to make sure all patients with CAD diagnosis or previous MI are on ASA therapy. I anticipate, as with many documentation deficiencies, that the centralization of data that an EMR facilitates will improve the documentation deficit. Review in 1 year.

CQI

CQI: Aspirin Therapy with CAD (ICD-9 414.00)

No Aspirin Therapy15%

On Other Anticoagulation

5%

Adverse Reaction6%

Aspirin Therapy74%

Summary

ACGME expectations OU FM experiencesapply knowledge of study designs and statistical methods to the appraisal of clinical studies and other information on diagnostic and therapeutic effectiveness

JClub, CDM, Clin-IQ

locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems

JClub, CDM, Clin-IQ

Use information technology to manage information, access on-line information, and support their own education

JClub, Clin-IQ, clinical supervision

Summary

ACGME expectations OU FM experiencesobtain and use information about their own population of patients and the larger population from which their patients are drawn

CQI, Clin-IQ

analyze practice experience and perform practice-based improvement activities using a systematic methodology

CQI

facilitate the learning of students and other health care professionals

Resident led conferences

What are we trying to do?

Pascal likened the situation to a sphere representing all available knowledge floating in a sea of ignorance, the sphere continuously increasing in size as discoveries transform ignorance into new knowledge.

This process of growth not only causes the volume of the sphere to increase continually but also its surface area, so that in fact the size of the frontier between knowledge and the unknown also increases.

Information growth is exponential, wisdom is not…

"Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?" T.S. Eliot