1

Barriers and Facilitators experienced while Barriers and Facilitators experienced while developing and implementing Athena-developing and implementing Athena-

Hypertension in VISN 1Hypertension in VISN 1

Views expressed are those of the speaker and not necessarily those of the Department of Veterans Affairs or other funding agencies or affiliated

institutions

Working with health care systems to translate research into practice

Mary K. Goldstein, MD Director, GRECC, VA Palo Alto Health Care SystemProfessor of Medicine (PCOR), Stanford University

2

Organizational Project TeamOrganizational Project Team

Acknowledgements– VA HSR&D and National Library of Medicine– Sociology colleagues for organizational component

• Ruth Cronkite, PhD; Dick Scott, PhD (and Grace Yeh)– The many participants in ATHENA development and

evaluation over years including• Co-PI Brian Hoffman, MD, VA Boston• Knowledge representation Samson Tu, Stanford Biomedical

Informatics Research (BMIR)• Site PI’s • Statistical consultants Phil Lavori, Alex Sox-Harris, Tyson

Holmes• Other team members at VAPAHCS Bob Coleman, Susana

Martins, and others

3

Translating Research into PracticeTranslating Research into Practice

Basic research

ClinicalEfficacy

trials

Change inRoutinepractice

Health ofpopulation

Animal studies,Single human,

Phase I trials, etc

Single-site;Multisite;

+/- meta study

“17 year”Translation

delay

TRIP I TRIP II

4

The “Quality Chasm”The “Quality Chasm”

Institute of Medicine (IOM) report on Crossing the Quality Chasm*Health care system must improve in– Patient safety, evidence-based practice

Improvability gaps between best practices and actual practiceInformation technology (IT) can support quality improvement– But IT is underutilized in clinical setting

* Crossing the Quality Chasm: A new health system for the 21st century. National Academy Press, 2001

5

VA HSR&D QUERIVA HSR&D QUERI

Quality Enhancement Research Inititative (QUERI)– “data-driven, outcomes-based, quality

improvement program…to achieve better health outcomes for veterans”

– “Particular emphasis is given to the documentation of best practices, implementation strategies, and dissemination”

– www.hsrd.research.va.gov/queri

6

QUERI ProcessQUERI Process

QUERI Steps1. identify high-risk/high volume disease or problems2. Identify best practices3. Define existing practice patterns and outcomes and

current variation from best practices4. Identify and implement interventions to promote

best practices5. Document that best practices improve outcomes6. Document that outcomes are associated with

improved health-related quality of life

7

Hypertension and QUERI StepsHypertension and QUERI Steps

We started pre-QUERI in 1997…can fit into the QUERI Steps retrospectively

We selected hypertension as a model for the study of guideline implementation becauseHypertension highly prevalent in adult medical practice (QUERI Step 1)Excellent evidence-based guidelines for management (QUERI Step 2)Evidence that the guidelines were not well-followed (QUERI Step 3)

8

Hypertension and Step 4Hypertension and Step 4

In 1997 when we started (pre-QUERI)– Known lack of impact of guidelines

• “Do Guidelines Guide Practice”

– Awareness to adherence model

Interventions first identified– Clinical reminders

First study in series implemented reminders (Step 4) and documents improved practice (Step 5) (next slide)

9

More “Diagnosis/Needs Assessment”More “Diagnosis/Needs Assessment”

Are primary care clinicians aware of whether or not their own prescribing for hypertension adheres to guideline recommendations?

– Combined clinician questionnaire and patient data extracts to compare perceived with actual adherence to hypertension guidelines

10

0%

20%

40%

60%

80%

100%

% o

f P

atie

nts

Me

etin

g G

uid

elin

e G

oa

ls

Adherence to Medication Guidelines

Adherence toBlood Pressure

<140/90Perceived Actual

(Expanded)Actual

(JNC VI)Perceived Actual

Perceived vs. Actual Guideline Perceived vs. Actual Guideline AdherenceAdherence

**

*

* P<.01 for difference between perceived and actual guideline adherence

Steinman et al Am J Med, 2004

Change in % Guideline Change in % Guideline Concordance for CliniciansConcordance for Clinicians

Individualized Study Group

40%

50%

60%

70%

80%

90%

100%

Baseline After

General Study Group

40%

50%

60%

70%

80%

90%

100%

Baseline After

Perc

ent C

onco

rdan

t

General Individualized

GuidelineConcordanceIncreased11% in InterventionVs 3.8% in Control p<0.01)

Goldstein et al Am J Mgd Care 11:677-85;2005

Conceptualization of Impact of Conceptualization of Impact of Decision Support SystemDecision Support System

Clinician Prescribing

Treated Patients’ BP

Journals Peers

Pharmaceutical Marketing Guidelines

Clinical Decision Support System

Lifestyle choices: diet, exercise, smoking

Competing comorbidities

Medication adherence Individual

response to therapy

13

Desirable Features of Automated Decision SupportDesirable Features of Automated Decision Support

Integrated into clinical workflow

Requires access to separate system

Requires separate data entry

Uses existing electronic data

Transferable to diverse EMRs

Limited to single EMR

Presents explanation / evidence-base for recommendations

No explanation or evidence presented

Complex logic and reasoning

Limited to simple rules

Knowledge easy to browse/update

KnowledgeIn code

14

Decision Support for Common Decision Support for Common Chronic DiseasesChronic Diseases

The “Field of Dreams” approach to medical informatics implementations:

If you build it, they will come

The physician often seen as wondering about a clinical question and then seeking out decision support:

15

16

See also Goldstein and Hoffman, in AHA Hypertension Primer, J.L. Izzo, Jr and H.R. Black, Editors. 2003, Williams & Wilkins: Baltimore.

17

ACPACP

BMJ Clinical Evidence

Reprinted with permission BMJ Publishing

Group

18

Developing ATHENA-Hypertension Developing ATHENA-Hypertension Knowledge BaseKnowledge Base * *

Built with Protégé– open-source Java tool for creation of customized

knowledge-based applications• Developed at Stanford Medical Informatics (SMI)…

now Stanford Biomedical Informatics Research (BMIR)

– http://protege.stanford.edu/overview/

* Goldstein et al Proc AMIA 2000

ATHENA Protégé top levelATHENA Protégé top levelATHENA Hypertension

Drug classes

Patient riskcategories

Blood Pressure Targets

20

ATHENA Protégé ATHENA Protégé GL managementGL management

diagramdiagram

21

Electronic Medical RecordSystem Patient Data

ATHENA ArchitectureATHENA Architecture

VISTAhierarchical

Database in M

CPRS

ATHENA HTN Guideline

Knowledge Base

GuidelineInterpreter

Treatment Recommendation

SQL Server: Relational databaseData Mediator

22

Implementing in ClinicsImplementing in Clinics

VA as site for demonstration/implementation– Electronic health record in place (CPRS)– VA organizational interest in quality improvement

Established technique for generating popup windows within the CPRS-GUI windowDesigned system for extracting patient dataTime testing for popup windowAttention to organizational contextEarly deployments at remote sites rapid (eg Durham)

Goldstein et al Translating research into practice.

J AMIA 11:368-76; 2004.

23

Sites for First Clinical Trial 2002Sites for First Clinical Trial 2002

Palo Alto (in 7 cities), San Francisco, and Durham VAMC’s (total 9 separate sites)91 clinician-clusters – 47 ATHENA– 44 Active control

34,427 visits of 11,473 patients

San Francisco VA, San Francisco VA, CACA

Palo Alto VA, Palo Alto VA, CACA

Durham VAMC, Durham VAMC, North CarolinaNorth Carolina

24

Server-ClientServer-Clientnumber of client computers (n)number of client computers (n)

VA SF Clinics

VA Palo Alto Clinics

Server in Palo Alto, CA

Durham VA Clinics

Server in Durham, NC

n = 193

n = 40

n = 118

25

Organizational Context:Organizational Context:Working with StakeholdersWorking with Stakeholders

Primary CareProviders

Office of Information and Technology

Clinical Applications Coordinators

Admin/Clinical Mgrs

programming

networking

Clinic computerSupport staff

26

Next Step: VISN CollaborativeNext Step: VISN Collaborative

Three-site study did not establish Step 5– Improved processes but did not document

improved outcomes

Larger study with 5 medical centers– QUERI VISN collaborative funding mechanism

• link clinical with research• Required VISN leadership co-PI

•Track organizational activities– Part of project plan

27

Two Organizational Structures for Two Organizational Structures for the Studythe Study

As usual for clinical trials, a site-PI and structure for managing participants within sitesA second structure for the IT deployment– IT contacts

28

IT Deployment Steps IT Deployment Steps

To implement ATHENA, IT staff from each center installs:

– 1) The M program that extracts patient data. This program consists of the KIDSBUILD.

– 2) Connections to the ATHENA server on computers used in primary care.

29

ATHENA Extract

installed

FTP file transfers into

SQL database

ATHENA client starts

System Recognizes

user

Rich Client Computer Names

Obtained

Patient Data Extracted overnight

from VistA

ATHENA Decision Support SystemATHENA Decision Support SystemTask FlowchartTask Flowchart

Thin Client installed on

user profiles

Rich Client

installed on PCs

Clinics pulled

from VistA

SQLEON; generates

Advisory precomputes

Server to Client Communication

Established*

Client listens to CPRS

*The client connects to the server to get Advisory precomputes or computes in real time.

Provider Chooses Eligible Patient and Receives ATHENA

display

30

Collaboration between research, CIO and Collaboration between research, CIO and IRMS programmer: Time to installIRMS programmer: Time to install

Task: Installing the ATHENA Extract The IRMS team at each medical center has to install the ATHENA extract program. An installation takes 6-8 hours to completeThe same ATHENA client installed in the start-up folder of rich client PCs is installed in the Start menu of the user’s profile on all the thin client servers.

0

5

10

15

20

25

Du

rati

on

(M

on

ths)

site1 site2 site3 site4 site5

install extract

install client

Site 5 has installed but still problems

31

Task: Obtaining correct computer Task: Obtaining correct computer names for rich clientsnames for rich clients

To install the shortcut on each computer, we needed computer namesSince there was no computer name list available to the ATHENA, an RA went to each site to collect all computer names in the primary care clinics This process took more than 1 year and several trips to each siteExample issue– Initial computer list developed but later a PC upgrade. Initial

list includes “a lot of neoware thin clients” no longer some other computers that are no longer there

32

Site computer changesSite computer changes

Site 1 thin clients rich clientsSite 2 thin clients rich clientsSite 3 rich clientsSite 4 mixture of thin and rich clientsSite 5 thin clients

33

Months to start trial from February 2006Months to start trial from February 2006

0

5

10

15

20

25

30

35

Du

rati

on

(M

on

ths)

site1 site2 site3 site4 site5

expected startdate

actual start

Site 5 has not started to date

34

No access to CPRS/VistA; resulting No access to CPRS/VistA; resulting effects on validation effortseffects on validation efforts

Groups involved in project

Site P.I.

Consultants

LabManagers

Their tasksATHENA Team Summary of Effects

•Prepare pt documents for Site P.I. to pull CPRS data•Validate pt data and f/up•Administer several rounds of validations when needed

•Access CPRS and update pt documents•Confirm differences between data sources

•Prepare pt documents for lab managers to verify LOINC codes•F/up once new LOINC codes were entered

•Direct technical questions to experts•Coordinate communication between various sites and consultants

•Validate LOINC codes and identify source of discrepancies

•Address questions with technical knowledge•Communicate with research team and sites

•Primary role of lab managers didn’t include our tasks and no funding provided•Time required to thoroughly validate lab discrepancies was increased

•Research team unable to act until response obtained from experts•Scheduling between sites and consultants resulted in slow responses to issues

•Delays up to 2 ½ months awaiting Site P.I. responses•Additional rounds of validations administered due to lapses in communication

35

36

Hiring a programmer: Hiring a programmer: New Features under DevelopmentNew Features under Development

• Write back capacity to VISTA:• Vitals• Notes

• Updated recommendations based on patient’s data

• Wish list: integration with computerized physician order entry

37

Reorganization of Research and OI&T: Reorganization of Research and OI&T: challenges to hire a Programmerchallenges to hire a Programmer

Dec 0

6 Pro

gram

mer

Inclu

ded

in IT

spe

ndin

g

plan

Sept 0

8, E

nd

of F

Y08

Meetings with high level official required for spending approval cancelled due to priorities. Official approved funds incorporating recommendations verbally in August meeting (4/07-8/07).

Initial request submitted to VA Central Office OI&T for programmer (1/07-3/07).

Funding approval stalled and FY07 funds were lost (9/07).

Begin

FY08

Requests for allocating IT funds for FY08 were redirected to various officials unsure who has authority to approve programmer (10/07-03/08).

Insufficient time to hire programmer even with approval prior to losing funds at end of FY08.

Approval of Project Modification providing IT funds to hire programmer (12/06).

38

Knowing and DoingKnowing and Doing

“If to do were as easy as to know what were good to do, chapels had been churches and poor men's cottages princes' palaces.”

Portia, Act I, Merchant of Venice, Shakespeare

39

The Team…early on…The Team…early on…

ATHENA Project Leadership– Mary Goldstein, MD, MSc and Brian Hoffman, MD (Co-PIs), VA Palo Alto

Health Care System and Stanford University Dept of Medicine• Brian now at Harvard Med Schl and VA West Roxbury/Boston

ATHENA Decision Support System Development– Stanford Medical Informatics EON Group

• Mark Musen, MD PhD; Samson Tu, MS; Ravi Shankar,MS; Martin O’Connor, MS; Aneel Advani, MD MPH

– ATHENA Group at VA Palo Alto and Stanford• Bob Coleman, MS Pharm; Susana Martins, MD, MSc; Parisa Gholami, MPH

Randomized Clinical Trial using ATHENA DSS as intervention– Phil Lavori, PhD, Biostatistician – Eugene Oddone, MD; Hayden Bosworth, PhD; and Michael Shlipak, MD:

investigators at VAMCs Durham and San Francisco

40

More of the Team…More of the Team…

Fellows/Resident– Patient Safety; computing active meds: Albert Chan, MD– Offline testing: S. Nicki Hastings, MD (former primary care chief

resident) – Accuracy of computer diagnoses related to HTN: Herb Szeto, MD– Clinician Questionnaire Study: Melissa Fischer, MD; Michael

Steinmann, MD (UCSF)– Provider barriers to GL adherence: Nancy Lin, ScD

Physician offline and clinic testing and monitoring– Lars Osterberg, MD; Howard Strasberg, MD; and others

Web pages for evidence displays– Emory Brock, MS

Assistance with Defining Knowledge Base – See Rules document for list of subspecialists

VISN 1 team (MAVERIC); Tyson Holmes statistics; et al!!

41

Funding SourcesFunding Sources

Development of ATHENA DSS– NLM LM05708 (PI: Musen) for development of EON architecture and

collaboration on building ATHENA DSS, built with EON Technology for guideline-based decision support systems

– VA HSR&D Career Development Award for Dr. Goldstein’s time; VA Palo Alto Health Care System for other staff time

Implementation and Clinical Trials– VA HSR&D CPI 99-275 (PI: Goldstein/Hoffman) – VA HSR&D IMV 04-062 VISN Collaborative for Improving Hypertension

Management with ATHENA-HTN.

Overall– VA support for staff

42

Stanford UniversitySchool of Medicine

43

Data SecurityData Security

Passwords must be changed in FTP every 90 days. We do not have full accessPassword changes must be coordinated with VISN1 so system does not go down.

44

END OF PRESENTATIONEND OF PRESENTATION

Following slides were removed from RIP presentation due to length of QUERI presentation

•These slides were placed at the end of the slideshow in the same order as they appear in the original slideshow. The Title of the slide that came before each group of slides is listed on the first slide of each group which is color coded by group.

45

Knowing and DoingKnowing and Doing

“If to do were as easy as to know what were good to do, chapels had been churches and poor men's cottages princes' palaces.”

Portia, Act I, Merchant of Venice, Shakespeare

46

Methods:Methods:Conceptual frameworkConceptual framework

Developed by Cabana et al (1999)

Knowledge Attitudes Behavior

Lack of familiarity

Lack of awareness

Lack of agreement

Belief that action will not lead to better outcomes

Inertia of previous practice

External factors Patient Environmental Other guidelines

Examplesof barriers

47

Conceptualization of Impact of Decision Conceptualization of Impact of Decision Support SystemSupport System

Clinician Prescribing

Treated Patients’ BP

Journals Peers

Pharmaceutical Marketing Guidelines

Clinical Decision Support System

Lifestyle choices: diet, exercise, smoking

Competing comorbidities

Medication adherence Individual

response to therapy

48

Public’s View of Public’s View of Why We Fund ResearchWhy We Fund Research

Basic research

Clinicaltrials

Change inroutinepractice

Health ofpopulation

Slide prior to this group - Organizational Project Team

49

Progression of Study TypesProgression of Study Types

Basic research

ClinicalEfficacy

trials

Change inRoutinepractice

Health ofpopulation

Animal studies,Single human,

Phase I trials, etc

Single-site;Multisite;

+/- meta study

“17 year”Translation

delay

50

Translating research into practice Translating research into practice (TRIP)(TRIP)

Two areas of translation in CTSA– TRIP I

• Process of applying discoveries generation during research in the laboratory and in preclinical studies

– TRIP II• Research aimed at enhancing the adoption of best

practices in the communityCost-effectiveness of prevention and treatment strategies is part of translational science

51

Clinical and Translational Science Clinical and Translational Science Awards (CTSA)Awards (CTSA)

NIH Roadmap for Medical ResearchCTSA Launched October 2006“Definable academic home for the discipline of clinical and translational science at institutions across the country”

• development of novel methods and approaches to clinical and translational research

• Informatics and technology resources• Improved training for new investigators to “navigate

the increasingly complex research system”

52

Two HTN ProjectsTwo HTN Projects

Automated clinical decision support– ATHENA-HTN

Group Visits (shared medical appointments)

53

Will it Be Used?Will it Be Used?

Once decision support is integrated technologically, will clinicians use it? Many clinical decision support systems are used only a tiny percent of time available

• For example, physicians viewed a hyperlipidemia guideline only 20 of 2610 visit opportunities (0.8%)

– Maviglia SM, Z.R., Paterno M, Teich JM, Bates DW, Kuperman GJ, Automating Complex Guidelines for Chronic Disease: Lessons Learned. J Am Med Inform Assoc, 2003. 10: p. 154-165.

– note that even infrequent use may still be beneficial, at very low cost

54

Clinician Interactions Clinician Interactions with ATHENA Advisorywith ATHENA Advisory

87/91 Clinicians interacted with 52% (10,740/20,524) of ATHENA AdvisoriesFocusing on survey responders:– 44/77 survey responders interacted with

60% (7,417/12,349) of ATHENA Advisories

55

Time Trend in Use at Visits: Time Trend in Use at Visits: New BP EnteredNew BP Entered

Percentage of ATHENA Hypertension Advisories with a Blood Pressure Update

0%

5%

10%

15%

20%

25%

30%

35%

40%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Month

Pe

rce

nt

of

Ad

vis

ori

es

Dis

pla

ye

d t

ha

t h

ad

a

BP

Up

da

te

56

Evaluation of Clinician ReactionEvaluation of Clinician Reaction

Clinicians used the system extensively• Data logged by system

– Speaks to usability and usefulness

Clinicians reported ATHENA-HTN affected their prescribing decisions– Questionnaire data

57

Athena Survey ResultsAthena Survey Results

Response rate: 57% (44/77)

Ease of Navigation: 80% (35/44) good or excellent64% (28/44) reported they updated the blood pressure in the ATHENA advisory often or occasionally.

58

Group Medical Visits to Improve HypertensionChronic Disease Managment

59

Our Model of Group VisitsOur Model of Group Visits

Group visit conducted by the primary care provider (PCP) with patients from his/her own panel of patients

Patient-Centered– Recognizes that whatever physician prescribes it is

patient who follows the diet and exercise regimen, takes the medication

• Based in part on Bandura self-efficacy

– Aims to enhance patients’ self-efficacy for self-management of their chronic disease

60

Group Visit FindingsGroup Visit Findings

Group medical visits for patients with hypertension yielded higher rates of blood pressure control as compared with control groups– most apparent for patients whose BP was not

meeting target BP at baselinePatient satisfaction high in all groups – improved more among the Group Visits patients as

compared with the control groups

61

Knowing and DoingKnowing and Doing

When we move from knowing to doing, must study the effect (QUERI Steps 5 and 6):

does implementing the best practices improve outcomes?are the outcomes associated with improved HRQOL?are there unintended consequences?

62

Quality Improvement ApproachQuality Improvement Approach

Solution-Driven Approach– In contrast to D/NA, this approach often characterizes QI

• Identify an innovation or research finding• Work toward rapid diffusion

– Examples• Clinical practice guidelines• CQI

As evaluations of implementations are done– Greater appreciation of contribution of context to

successful implementation of QI

63

MVK ISO approval certificateMVK ISO approval certificate and re-push and re-push

After obtaining permission to push, we pushed to an old server.We had not obtained official permission to use the new server and, as a result, had to repush again to the computers in sites 1-3.