analytics-based ehrs implementation · analytics-based ehrs implementation: improved outcomes...

1

Analytics-BasedEHRS

Implementation:

Improved Outcomes

Session 234, February 14, 2019 – 11:30 AM

John A. Rekart, Ph.D.Chief Psychologist, Quality Management and Informatics

Statewide Mental Health ProgramCalifornia Department of Corrections and Rehabilitations

2

©HIMSS 2018-2019S Suman Free use

3

MeIntroduction

YouAudience

Participation

EHRS Adoption FailureProblem

Formulation

Analytic/Evidence Based Project Management

Solutions

We Crushed ItOutcomes

Agenda

©HIMSS 2018-2019

4

• Demonstrate knowledge of how data will

improve success of electronic health

record system (EHRS) implementations

• Demonstrate an understanding of how

quality management tools can improve

training results for EHRS implementations

• Describe the metrics used to support a

data-based model of implementation over

a vendor-based cookie-cutter model

Learning Objectives

©HIMSS 2018-2019

5

California Correctional Health Care

Services– Statewide Mental Health

Program

122,901 Inmates

38,000 Mental Health

Patients

35 Institutions

1,300 Mental Health

Providers CDCR

©HIMSS 2018-2019

6

Profession

Audience Poll

Creative Commons

https://live.eventbase.com/polls?event=himss19&polls=5174

7

Electronic Healthcare Record

System

1) Audience Poll – EHRS

Creative Commons


8Implementation Goal

Audience Poll

Creative Commons


9

“In God we trust;

…all others must bring

data.”

W. Edwards Deming, physicist and quality

improvement pioneer©HIMSS 2018-2019Creative Commons

Industry EHRS Failure Rate

“50% of EMR system implementations result in failure.” International Journal of

Technology Assessment in Health Care, 1997

“Industry experts estimate that failure rates of EMR implementations range from 50-80%.” A Commonsense

Approach to EMRs, 2006

“. . . 50–80% EMR failure rate documented in the Milbank Quarterly and cited by the AMA.” 2010

Creative Commons ©HIMSS 2018-2019

More Bad News

“It is estimated that between 30-50% of EHR implementation projects fail. According to a research report by health IT research firm KLAS, by 2016, almost 50% of large hospitals will replace their current EHR.”

“Most of the EMR projects, like any large-scale enterprise-wide projects, take years to deploy and stabilize and cost as high as 3-6% of a hospital’s operating budget. And yet, 30-50% of such projects fail to deliver on the promise.”

Top 10 Reasons Why EMR Implementations Fail - Udai Kumar, May 12, 2016Creative Commons

Creative Commons

©HIMSS 2018-2019

Why they fail

Workflows Training

Adoption

&

Go Live

Poorly designedand implemented

Creative Commons©HIMSS 2018-2019

•Don’t get into stranger’s cars•Don’t meet people from the internet

• Literally summon strangers from the internet to get into their car

Carol Nichols, twitter.Creative Commons ©HIMSS 2018-2019

Data Liquidity • Data liquidity in health

information systems. ... Both had as their goals, implicit or explicit, to ensure the right dataare provided to the right person at the right time, which is one definition of "data liquidity."

• Data liquidity in health information systems. - NCBI - NIH

•https://www.ncbi.nlm.nih.gov/pubmed/21799328

Data AgentChange

Accuracy - Efficacy

Creative Commons

Better Data Liquidity Increased Process Influence

https://www.ncbi.nlm.nih.gov/pubmed/21799328

https://www.ncbi.nlm.nih.gov/pubmed/21799328

15

Data Entry and Database1. InfoPath2. SharePoint3. Access

Vendor Analytics Suite1. Discern Analytics – Front end2. LightsOn - System Analytics

Data Warehouse1. On_Demand Report2. Outlook Email Integration

Methods and Tools

©HIMSS 2018-2019

Problematic Workflow Issues

Reproduce Paper Based Workflows

Misunderstanding of EHR Capabilities

Competing and Conflicting Design Goals

Poor Integration of Interdisciplinary Objectives


Components of a Good Workflow

Increases Patient Safety

Streamlines Clinical Documentation

Increases Interoperability

Increases Provider Efficiency

Continuous Quality Improvement - OptimizationCreative Commons

18

TrainingApplication of Quality Management Principles and Practices to Training


• Increased Training Hours• Divided the curriculum

• Basic EHRS training• Process Training

• Certification of Trainers• Audit and Feedback of

Training

• Data Liquidity

Training Improvements• To Meet New Build Complexity• Organize into Conceptual

Chunks

• Assure Quality of Instructors• Assist Chiefs of Mental Health

to Manage the Training Requirements

• Getting the Data to the Actors/Agents in real time

©HIMSS 2018-2019

20

Training Audits

©HIMSS 2018-2019

Training Audits

User Training

Train-The-Trainer users (TTT) trained

Super user trained (SUT)

Track training progress

eLearning

Online pre-training

Learning Management system will link to

database

Software Simulation

Self-Assessments

Web-based

Separated by classifications

Individual and Institutional summaries

Knowledge Checks

Web-based

Multiple Choice

End user and Process questions

©HIMSS 2018-2019

22

Knowledge Assessment

Ceiling Questions

Content Domains

Floor Questions

SuperUsers

Minimally

Competent

End Users

High

Opportunity

for Change

Knowledge Check - Design Strategy

©HIMSS 2018-2019

SharePointTeam Site

©HIMSS 2018-2019

Links to InfoPath Forms

Knowledge Check for Providers(Competency)

©HIMSS 2018-2019

InfoPath

25

Self Assessments

©HIMSS 2018-2019

26

EHRS Trainee Evaluation

©HIMSS 2018-2019

Super Users Train Mocks, Clinical Cutover, and RetrainingTraining Window

0%10%20%30%40%50%60%70%80%90%

100%

Staff Training Before Go-Live

Mental Health Provider Mental Health Scheduling Psychiatry

Training Completion

©HIMSS 2018-2019

TTTs

Tra

ined

Train-The-Trainer Certification Audit

• Determined by multiple data sources to a weighted algorithm, based on past successful• Self Report• Multiple Choice• Teach-backs

• TTT certification authorizes onsite trainings. Failure disqualifies them from conducting or assisting in any onsite EHRS trainings.

©HIMSS 2018-2019

The Ability to Monitor Progression of Training at the Institutional Sites by Class/Cohort

©HIMSS 2018-2019

Training Audits – Knowledge Checks

• Knowledge Checks Breakdown

• Grouped by MH Providers, MH Scheduling, and Psychiatry

• Top right corner indicates overall score, and each question has an institutional score.

• Institution used this audit for targeted booster training and ongoing training.

©HIMSS 2018-2019

Training Audits – Self Report

• Self-Report Breakdown

• Grouped by MH Providers, MH Scheduling, and Psychiatry

• Self-Report categories are color coded to easily identify trouble areas

©HIMSS 2018-2019

Executive OverviewTraining

©HIMSS 2018-2019

Training Composite ScoresThe knowledge checks were designed using test construction theory with the intention to discriminate the trainees into three groups: potential super-users, minimally competent users, and high opportunity for learning users.

©HIMSS 2018-2019

Data Integrity and Monitoring Alternative Test Taking Methodology

cheat/CHēt/

verb

gerund or present participle: cheating

1. act dishonestly or unfairly in order to gain an advantage, especially in a game or examination.

synonyms: swindle, defraud, deceive, trick, scam, dupe, hoodwink, double-cross, gull; rip off, con, fleece, shaft, hose, sting, bilk, diddle, rook, gyp, finagle, bamboozle, flimflam, put one over on, pull a fast one on, sucker, stiff, hornswoggle;


35

©HIMSS 2018-2019

36

Cutover

Another Training opportunity


37

1. Inpatient Medications

2. Levels of Care (Acuity)

3. Scheduling Appointments

4. Document Cutover

1. Initial Assessments

2. Suicide Risk and Self-harm Evaluations

3. Developmental Disability Program

Evaluations

4. Developmental Disability Program

Designation

Cutover Data

©HIMSS 2018-2019

38

1.Track required cutover requirements

2.Denominator is known

3.Develop required work projections to

meet cutover goals through frequent SQL

report updates

4.Get cutover analytics to providers and

local management

5.Escalate immediately when progress is

not meeting needed timelines

Cutover Analytics

©HIMSS 2018-2019

Clinical Cutover AuditRequires staff to input

PowerPlans for each non-GP patient.

The green line indicates a steady pace of cutover

with successful completion Friday before

Go-Live.

The White line is the institutions progress.

The Yellow line is an alert that suggests the pace of

cutover is suboptimal and may pose a risk for a

successful go live.

The Red line is an alert that suggests that the

pace of cutover is at risk of critical failure.

Requires staff to input PowerPlans for each non-

GP patient.

The green line indicates a steady pace of cutover

with successful completion Friday before

Go-Live.

The White line is the institutions progress.

The Yellow line is an alert that suggests the pace of

cutover is suboptimal and may pose a risk for a

successful go live.

The Red line is an alert that suggests that the

pace of cutover is at risk of critical failure.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Minimum 57 115 172 230 287 344 402 459 516 574 631 689 746 803 861 918 976 103 109 114 120 126 132 137 143 149

Alert 344 387 430 473 516 560 603 646 689 732 775 818 861 904 947 990 103 107 111

High Risk 230 258 287 316 344 373 402 430 459 488 516 545 574 603 631 660 689 717 746

0

200

400

600

800

1000

1200

1400

Emai

l Au

dit

Sen

t

Progress Towards 100% Scheduling Cutover

©HIMSS 2018-2019

Progress of Document Cutover100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

CUT OVER WEEK 1

CUT OVER WEEK 2

CUT OVER WEEK 3

CUT OVER WEEK 4

Psychosocial Assessments

Suicide Risk Assessments

Developmental Disability

Assessments

Increased Provider Cutover = Better Prepared Provider

• An early institution had a difficult go live with providers feeling ill-prepared at go live.

• A review of the data showed that 10 (12%) users, completed 1,630 patients clinical cutover (over 80%) at the same institution.

©HIMSS 2018-2019

42

Go-Live/Adoption

Data Integrity – Workflow Adoption


EHRS Go-Live/AdoptionData Entry Flags

• Data Entry Flags (Indicators)• Address critical faults, workflow deviations, and threats to data

integrity• Data is filtered and sent to stakeholders/project managers

©HIMSS 2018-2019

-

200

400

600

800

1,000

1,200

1,400

CCWF CMC CTF DVI SCC SVSP VSP©HIMSS 2018-2019

Region II: Data Entry Flags January 2017 To September

2017

143

14 -

50

100

150

200

250

300

350

400

January 2017 February 2017 March 2017 April 2017 May 2017 June 2017 July 2017 August 2017 September 2017

©HIMSS 2018-2019

Institution X Data Entry Flags September 2017

• MHI change during in-transit/temp departure 3• MHI not changed within 24 hours of DSH discharge 40• Inmate in CDCR over 30 days with no MHI entered since admission 3• MH referral resolved before it was received 90• Program/sub Program change during in-transit/temp departure 1• MH referral received date later than data entry date 1• MHCB Admission with wrong MHI 2

46


Evidence Based Change Management

Evidence Based CM

Case Study CM

8.3 million vendor transactions per

day

9,300 EHRS users per day

21 out of 344 of vendor clients ranked by transactions

Creative Commons

OutcomeSentinel Events

A Sentinel Event is defined by The Joint Commission (TJC) as any unanticipated event in a healthcare setting resulting in death or serious physical or psychological injury to a patient or patients, not related to the natural course of the patient's illness.25 percent

Excluding medication errors©HIMSS 2018-2019

decrease in sentinel events after go-live for Mental Health

40%

50%

60%

70%

80%

90%

100%

-6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Reference Mental Health Services

Scheduling and Access to Care

Go

-liv

e

©HIMSS 2018-2019

“The EHRS has helped (the institution) do significantly better on their performance monitoring review (suicide prevention).”- Lindsay Hayes, National Expert on Suicide Prevention

©HIMSS 2018-2019

CDC_CA It has been awarded the 2018 Best of California Award as sponsored by

the Center for Digital Government:Best Application Serving an Agency’s

Business Needs Electronic Health Record System (EHRS),

California Correctional Health Care Services

2017 Statescoop 50- State IT Innovation of the Year

CCHCS Electronic Health Record System, California Correctional Health Care Services

2018 HIMSS Davies Community Award of Excellence©HIMSS 2018-2019

Outstanding Achievements

State IT

2018

55

Audience Poll – Continuous Quality Improvement

Feedback

Creative Commons


Questions?


Contact Information

916-691-2640 office916-385-2965 Cell

[email protected] @drrekart

Please remember to complete the online session evaluation

Statewide Mental Health Program - California Correctional Health Care ServicesCalifornia Department of Corrections and Rehabilitation

Creative Commons

analytics-based ehrs implementation · analytics-based ehrs implementation: improved outcomes...

Documents