Transforming GuidelinesInto Electronic Tools

Richard N. Shiffman, MD, MCIS

Yale Center for Medical Informatics

New Haven, Connecticut, USA

Overview

• Cultural contrasts• Computer-based Decision Support• Ways developers can help implementers• Implementers have some tools that may help

developers

• URL and email are on my last slide!

ulturesRick Shiffman

Suave and

debonair

Cultural difference: food

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

We Read Different Journals

We Meet in Different Places(at the same time!)

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

August 20–24, 2007

Language differences

• Guideline prose– Woolly, fudgy– Both inadvertent and deliberate

• C#, JAVA, ASBRU, GLIF, … – Precise– Ambiguity causes crashes

SAME GOAL:DIMINISHING INAPPROPRIATE PRACTICE

VARIATIONPROMOTING KNOWLEDGE-BASED CARE

78.7

64.7

63.9

53.9

53.5

45.4

39

22.8

0 20 40 60 80 100

Hypertension

Colorectal CancerAsthmaDiabetesPneumoniaHip Fx

CHF

Senile Cataract

% Receiving Recommended Care

(McGlynn. NEJM 2003)

Strategies

• Authors: Sculpting recommendations to match evidence

• Implementers: Operationalizing recommendations in a system that influences behavior

High Probability of Effectiveness

• Patient-specific reminder at time of consultation– Grimshaw JM and Russell IT. Effect of

clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993

Computer-Based Decision SupportSystematic Reviews

Mary Johnston McMaster JAMA 1994

Derek Hunt McMaster JAMA 1998

Amit Garg Univ. Western Ontario

JAMA 2005

Ken Kawamoto Duke BMJ 2005

Basit Chaudhry

UCLA Ann Intern Med 2006

Findings

• Computer-based decision support regularly—but not always—improves the process of care

• Outcomes—though infrequently measured—sometimes improve

CDSS Definition

• A system that compares patient characteristics with a knowledge base and then guides a health provider by offering patient-specific and situation-specific advice

CP FriedmanJC Wyatt

Functions of CDSS

Alerting Out-of-range lab values

Reminding Schedule preventive service

Critiquing Rejecting an electronic order

Interpreting EKG

Predicting Mortality from severity score

Diagnosing Differential based on symptoms

Assisting Tailoring dosages based on renal function

Suggesting Ventilator adjustments

Randolph AG. JAMA 1999

DocumentationTemplates Interpreter

DocumentationTemplates Calculator

Interpreter

RelevantData Summary

Interpreter

Alerts / Reminders

OrderCreation

Facilitator

InfoButton

ThoughtfulAssistant

(Electronic) Implementers Need More Help from

Developers

Language

Obstacles to Implementation

Define Recommendation Strength

• Different recommendations would be given for the same patient using …encoded representations of vaccine- and breast mass workup guidelines formulated by different members of the (same) laboratory.

Translation of Guideline Knowledge for Decision Support

Patel VL.JAMIA 1998

Why is attention to implementability important?

• “We found that it was possible—if not inevitable—that two encoders would encode the same guideline in two different ways. Sources of variation included differences in the order in which data elements are collected, differences in the level of detail represented, differences in the use of atomic sentences or composite sentences in criteria, differences in the specification of data elements, and omissions due to human error…

is not contraindicated Single dose of PCV-7 for high risk children of any age

may benefit from RSV prophylaxis for infants 32 weeks’ gestation or less

may be beneficial Meningococcal polysaccharide vaccine for travelers

will benefit from RSV monoclonal antibody for children 24 months of age or less with hemodynamically significant congenital heart disease

may be helpful Testing for asymptomatic seroconversion after varicella exposure and receipt of VZIG

most experts recommend RSV prophylaxis for infants 32 to 35 weeks’ gestation with risk factors

some experts recommend Pertussis vaccine for children who have had natural pertussis

some experts suggest Duration of face-to-face contact that qualifies for significant varicella exposure

some experts prefer Serologic testing for anti-HBsAg antibody after primary vaccine series in perinatally exposed infants

some experts consider Safety of influenza vaccine during early pregnancy

experts differ in opinion HBIG for the incompletely immunized child exposed to a discarded needle in the community

the manufacturer recommends

Avoidance of salicylates after varicella vaccine

Statement of fact is NOT a recommendation

• Adjuvant hormone therapy for locally advanced breast cancer results in improved survival in the long term.

• Clinicians should prescribe adjuvant hormone therapy for locally advanced breast cancer (when/unless?)…

“Ambiguity”: Resolution by classification

• Ambiguous statements are interpretable in more than one discrete way– “I’ll meet you at the bank.”– “MS”: morphine sulfate, magnesium sulfate; multiple

sclerosis– “LAD”: left axis deviation, lymphadenopathy

• Vague- lack a crisp threshold in a single dimension– “Fever,” “tall”

• Underspecified - lack specificity in multiple dimensions– “sufficiently ill to warrant immediate antimicrobial therapy”

Deliberate vaguenessand underspecification

• Insufficient evidence• Inability to reach consensus• Legal concerns (standard of care)• Economic reasons• Ethical/religious issues (e.g., concept of

“burden,” “futility of care”)

• What if authors were transparent aboutthe reason for AVUL?

Authors Should Be Explicit About

• WHEN {under what circumstances}

• WHO should

• Do WHAT

• To WHOM

• HOW

• WHY

AUTHORS SHOULD USE STRONG VERBS

• Active voice– Passive masks the actor

• Appropriate choice of deontic operators– The clinician must, must not; the Committee strongly

recommends– The clinician should, should not; the Committee

recommends– The clinician may; the Committee suggests

• The dreaded “consider”

GLIA (GuideLine Implementability Appraisal)

• Helps to identify obstacles to implementation• Provides feedback to guideline authors to anticipate and

address these obstacles before a draft guideline is finalized• Assists implementers to select guidelines and to focus

attention on anticipated obstacles

• GLIA is available from http://gem.med.yale.edu/glia

How does GLIA complement AGREE?

• Limited to issues related to implementation• Emphasis on individual recommendation

(rather than the guideline as a whole)• Concordance

– Of 23 AGREE items (and 31 GLIA items)• 5 have equivalent questions in GLIA• 3 have similar questions in GLIA

GuideLine Implementability Appraisal(GLIA)

• Decidability - precisely under what circumstances to do something

• Executability - exactly what to do under the circumstances defined)

• Effect on process of care - the degree to which a recommendation impacts upon the usual workflow of a care setting)

• Presentation and formatting - the degree to which the recommendation is easily recognizable and succinct

• Measurable outcomes - the degree to which the guideline identifies markers or endpoints to track the effects of implementation of this recommendation

GLIA Constructs (2)

• Apparent validity - the degree to which a recommendation reflects the intent of the developer and the strength of evidence

• Novelty/innovation - the degree to which a recommendation proposes behaviors considered unconventional by clinicians or patients

• Flexibility - the degree to which a recommendation permits interpretation and allows for alternatives in its execution

• Computability - the ease with which a recommendation can be operationalized in an electronic information system

Guideline Authors are committed to appraising the quality of scientific

evidence• As they should!• But they regularly fall short in helping users

understand how to use the information• Implementers are rarely interested in

evidence quality per se• Implementers need authors’ assessment of

strength of recommendation

Tripod of Concepts

Benefits vs. HarmsAssessment

Confidence (we’ve got benefits

& harms right:Evidence Quality)

Importance of AdherenceRecommendation Strength

American Academy of PediatricsGrading Recommendation Strength

Evidence Quality

Preponderance of Benefit or

Harm

Balance of Benefit and

Harm

A. Well designed RCTs or diagnostic studies on relevant population

B. RCTs or diagnostic studies with minor limitations;overwhelmingly consistent evidence from observational studies

C. Observational studies (case-control and cohort design)

D. Expert opinion, case reports, reasoning from first principles

X. Exceptional situations where validating studies cannot be performed and there is a clear preponderance of benefit or harm

Strong

Strong

Rec

RecOption

Option No Rec

Clinicians and Strong Recommendations

• Benefits of the recommended approach clearly exceed the harms

• Quality of the evidence is excellent • Clinicians should follow such guidance unless a

clear and compelling rationale for acting in a contrary manner is present

• Optimal source for P4P

Clinicians and Recommendations

• Benefits exceed the harms

• Quality of the evidence on which this recommendation is based is not as strong

• Clinicians generally should follow such guidance but also should be alert to new information and sensitive to patient preferences.

Clinicians and Options

• Evidence quality is suspect or well-designed, well-conducted studies have demonstrated little clear advantage to one approach versus another

• Options offer flexibility in decision making about appropriate practice, although they may set boundaries on alternatives

• Patient preference should have a substantial role in influencing clinical decision making

• Hard to hold clinicians accountable (P4P)

Electronic Implementers and Recommendation Strength

• Strong recommendation– Cannot close form until issue is addressed

• Recommendation– Recommended action is default; some

effort required to override

• Option– Radio buttons, checkboxes for choices

within limited range

(Electronic) Implementers have some tools that can help

developers

GEM

EXTRACTOR

eGLIA

Action-types

Implementation Section

Logical Analysis with Highlighters

•UTI Recommendation 3

If an infant or young child 2 months to 2 years of age with unexplained fever is assessed as being sufficiently ill to warrant immediate antimicrobial therapy, a urine specimen should be obtained by SPA or bladder catheterization; the diagnosis of UTI cannot be established by a culture of urine collected in a bag. (Strength of evidence: good) Urine obtained by SPA or urethral catheterization is unlikely to be contaminated...

XML: From a small number of discrete colors to an unlimited palette

XML

• Multi-platform, Web-based, open standard

• “Tags” enclose and describe text

<inclusion.criterion>hematuria</

inclusion.criterion>

• Human-readable, yet can be processed by

machine

• Markup can be performed by non-

programmers

• “Hot”—considerable energies invested in X-

tech

• Knowledge model for guideline documents

• GEM adopted as a standard by ASTM in 2002; GEM II updated and re-standardized in 2006

• Models heterogeneous information contained in guidelines

– Multi-level hierarchy (>100 elements) indicates relationships

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

GEM

GEM II-Top Level

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Conditional

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

GEM Cutter

MORO

Reuse OftenMarkup Once,

GQAQ Report Card

Markup OnceReuse Often

• Markup with GEM Cutter Editor

• Guideline quality appraisal with GEM-Q

• Feedback to developers with Extractor

Feedback to authors

Decision Variable Values Decidable?

Age <6m, 6m-2yr,>2yr

Attend day care Y, N

Recently treated with abx Y, N

Severe illness Y, N

Followup can be assured Y, N

Action Executable?

Amoxicillin 45 mg/kg/d in 2 divided doses

Observation is appropriate

Action-Types in405 Recommendations

Monitor

Test

Gather Interpret Perform Dispose

Action

Conclude Prescribe

Educate

Document

Procedure

Consult

Advocate

Prepare

Essaihi A. Proc AMIA 2004

Action Distribution

Refer 5%

Dispose 4%Conclude 3%Monitor 3%Document PrepareAdvocate

No Rec

Example: Application of Action-Types

• Action-type: Prescribe– Drug information– Safety alerts (allergy, drug-drug, drug-

disease, drug-lab)– Formulary check– Dosage calculation– Pharmacy transmission– Patient education– Corollary orders

Key Messages• Developers and implementers (esp electronic) represent

different cultures• Electronic DSS work• Electronic implementers need help from developers

– Watch your language!– Anticipate problems in implementation: GLIA– Define recommendation strength

• Implementers have some tools that may help developers– GEM: a standardized document representation– EXTRACTOR– Action-typing to define recurring patterns

Thank You!

http://gem.med.yale.edu

richard.shiffman@yale.edu