Implementation Science and Homeless Program Model Development

Christopher J. Miller, Ph.D., Clinical Psychologist, VA Boston Healthcare System

Roger J. Casey, Ph.D., Director, Education and Dissemination,

National Center on Homelessness among Veterans

November 29, 2017

Implementation Science: What Is It and Why Should We Care?

Christopher Miller, PhD

Investigator, Center for Healthcare Organization and Implementation Research (CHOIR), VA Boston Healthcare System

Instructor, Harvard Medical School Department of Psychiatry

2

Outline of Today’s Session

Historical view of implementation science Modern need for it Different domains that affect

implementation success Methods for implementing new

practices Methods for studying implementation

3

A Long Time Ago…

1601: Scurvy = single leading cause of death among sailors

James Lancaster, an English sea captain, assigned sailors on one of his ships to receive 3 teaspoons of lemon juice/day

Mosteller, 1981; Rogers, 1995 4

A Long Time Ago…

3 other ships: no lemon juice One of the first randomized trials?

Results = incontrovertible Halfway through the journey,

110/278 sailors on the “control” ships had died Sailors from the “treatment” ship staff the other ships

5

A Long Time Ago…

1747: Full-blown RCT (first ever) by James Lind Royal Navy physician Citrus fruit vs. sea water vs. nutmeg vs. cider, etc. Citrus fruit group had to care for patients in other

treatment arms

1795: British Navy adopts citrus fruit to prevent scurvy on long sea voyages 1865: Merchant Marine follows suit

6

A Long Time Ago…

1600 1650 1700 1750 1800 1850 1900 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

7

Experimental trial shows citrus cures

scurvy

Randomized controlled

trial by Navy physician confirms

citrus cures scurvy

Royal Navy implements

citrus as scurvy cure

Merchant Marine

implements citrus as

scurvy cure

Period of unnecessary scurvy death = 264 years

Fast Forward to Now

We do much better… right?

8

Fast Forward to Now

Many innovations only achieve partial saturation… ACE inhibitors among heart failure patients: ~33% Proper hand-washing among hospital staff: ~50% MH/SA patients receiving care consistent with clinical

guidelines: ~27%

… or take 15-20 years to become routine practice Proper dosing for antipsychotics: ~15 years 15th independent positive RCT of CBT for depression: early

1990’s

9 Valenstein et al., 2001; Mair et al., 1996; Pittet et al.,

2000; Bauer, 2002; IOM, 2001; Morris et al., 2011

Fast Forward to Now

Thus, implementation science is about studying strategies to bring evidence-based practices and other innovations into routine practice by: Developing and evaluating implementation

strategies Measuring implementation rates Analyzing factors that affect implementation

effectiveness and sustainability

10

Why Doesn’t It Happen for Health Innovations?

11

Domain: Characteristics of the…

Challenges arising in healthcare (example of a new evidence-based psychotherapy manual)…

Innovation itself May require systemic changes: staff training, new documentation, patient education, etc.

Person or group championing the innovation

The people championing new innovations may be unknown to frontline clinical staff, or be seen as “not understanding our patients”

Patients receiving the innovation

Patients are not passive recipients of health interventions; evidence-based manuals may be at odds with what they want out of therapy (e.g. a chance to “vent”)

Providers applying the innovation

Providers may be happy with less structured care, or worry that structured approaches limit applicability of clinical acumen on a case-by-case basis

System in which the innovation may be implemented

Difficulty carving out time to train staff; delay in developing note templates associated with the manual; culture focused on medications over psychotherapies

Interactions among the above!

Patients wary of short-term, manualized treatments + a risk-averse system incentivized to minimize patient complaints = low uptake

Characteristics of the Innovation

Relative advantage – is it better? Complexity – is it easy? Trialability – can it be done on a trial basis? Observability – is it obvious that it’s working? Compatibility – does it fit with my beliefs?

Others added later (e.g. risks involved in

implementing the innovation) 12 Rogers, 1995

Characteristics of the Person Championing the Innovation

Does the person championing the innovation inspire trust? Homophily vs. heterophily: is the person championing the innovation similar to me? Social factors generally outweigh scientific

factors for implementation decisions

13 Rogers, 1995

Characteristics of the Patients Receiving the Innovation

Not a lot in the literature on this… … but obviously important for many

healthcare innovations Evidence-based practices Telehealth care delivery Challenges specific to

homelessness?

14

Characteristics of the Providers Applying the Innovation

Provider buy-in crucial to implementation success S-shaped

adoption curve Based on bell

curve

15 Gladwell, 2000; Rogers, 1995

“Despite the demonstrated benefits of many improvement techniques, most attempts by companies to use them have ended in failure.”

Applicability to healthcare innovations

16 Keating et al., 1999 (p. 120)

evidence-based practices

The Improvement Paradox

Worse Before Better…

17

Keating et al., 1999

The Improvement Paradox (or why good things don’t get implemented)

The Improvement Paradox

Ideal State = Virtuous Cycle

18 Keating et al., 1999

Effort allocated to implementing/ improving the

innovation

Increased productivity

The Improvement Paradox

19 Keating et al., 1999

Effort allocated to implementing/ improving the

innovation

Increased productivity

Throughput Pressure

Dip in productivity

Increased effort allocated to getting the job done at expense of

implementing innovation

This right-hand arrow represents a

negative relationship, as

throughput pressure kills

commitment to the innovation

The Improvement Paradox

20 Calvin & Hobbes by Bill Watterson

Using my old strategy will be productive right now…

… while trying this new strategy might be more productive at some point in the future…

… Meh.

Some Examples of Implementation Methods

Official “rollout” trainings, with or without certification Provision of note templates or other support materials Identify and recruit local Champions to encourage

implementation Audit and Feedback: feed back performance data to

clinicians (or post publicly) Develop disincentives (e.g. formulary restrictions, negative

performance evaluations tied to the old practice) Facilitation (next slide)

21

Some Examples of Implementation Methods

Facilitation: having experts work with a team to get the innovation implemented Example within VA: implementing a new model of care

delivery for outpatient mental health teams Internal facilitator: promotes change from within

target organization External facilitator: provides outside expertise to the

internal facilitator and team

22 Bauer et al., 2016; Kilbourne et al., 2004; Kirchner et al., 2010

How to Evaluate the Success of Implementation Efforts?

Different focus than typical clinical trials

23

Typical Clinical Trial Outcome Typical Implementation Science Outcome

- Did individual patients who received the intervention show improvement in symptoms or functioning?

- How many patients received the intervention?

- How many providers now consider the intervention the new norm?

- Were those gains maintained six months down the road?

- Is the intervention still in use six months down the road?

- Can the intervention be delivered with high fidelity by experts who have been extensively trained?

- Can the intervention be delivered with reasonable fidelity by John Q. Therapist?

How to Evaluate the Success of Implementation Efforts?

24 Curran et al., 2012

Hybrid Type 1 Hybrid Type 2 Hybrid Type 3

Focuses on clinical effectiveness, with some attention to implementation

Focuses equally on clinical outcomes and implementation outcomes

Focuses on implementation success, with some attention to clinical outcomes

Hybrid designs: include both implementation and clinical outcomes Due to complexity, often mixed methods

(quantitative + qualitative)

Summary/Discussion

Lots of healthcare innovations… … don’t get used … or get used rarely … or get used only after a long delay

Implementation Science is focused on increasing the use of good innovations Requires a shift from standard clinical trials

thinking

25

Safe Havens

Application of Implementation Science: Developing an Innovative Program Model

Roger Casey, PhD National Center on Homelessness among Veterans

November 29, 2017

Acknowledgments

University of South Florida Scott Young, PhD Paul Smits, LCSW John Schinka, PhD Colleen Clark, PhD

Save Haven: Program Design Overview • Residential Setting • Low demand – Early Recovery principles • Services and Treatment Engagement • Population and Criteria • Sobriety / Full Treatment Compliance – not required • Staff and Resident Safety • Outcomes: Permanent Supportive Housing or Treatment • Based on need • In part, meeting the intent – ending Veteran Homelessness

Save Haven: Program History Overview

• Initiative to end Veteran homelessness • USICH – Opening Doors: Veteran priority and benchmark • National Center on Homelessness among Veterans – model

development • 2010 - Solicitation Request for Proposals (RFP)

– Five funded - adjusted

• 2012 – 15 sites funded • Transition to operations 2015 -2016 • FY 2016 – 3,125 Veterans entered the program

The Discussion: Application of Implementation Science and Considerations for Homeless Interventions

• Various funding sources • Staff background, education, discipline • Unique characteristics of localities

– Resources, support • Leadership support and commitment • Population base – chronic, MH, SUD

Implementation Framework

• Interactive Systems Framework of Dissemination and Implementation (ISF) (Wandersman, et al.)

– Synthesis – distill knowledge of practices interventions – Translation – convert to practical application – Delivery – appropriate sites and commitment – Support and TA – guidance, ensure adoption

Enhanced Implementation Framework

Homeless Interventions and Services • Implementation Framework Considerations:

– Identifying practice evidence – applicability for various settings and homeless populations subpopulations;

– Reviewing local, state and federal funding and funding authorities;

– Assessing community infrastructure for capacity and evidence of commitment and collaboration among stakeholders.

Enhanced Implementation Frameworkxx

Homeless Interventions and Services • Implementation Framework Considerations:

– Evaluating community provider staff knowledge, abilities and skills;

– Developing program principles that address homeless populations:

• Access – hard to reach; • Permanent housing solutions • Assessment process for further TX.

Research

Practice

Current Offerings

Needs Assessment

TA-Technical Manuals Training

Development

Funding Requirements

Providers Operational Requirements

Authorities for Provision of Services

Program Components

Site Requirements

Programmatic Adjustments

Site Suitability

Stakeholder Buy-in

Program Operation

Staff and Site Training

Provider Contracts

Outcome Data Feedback

Bi-weekly Admin Forum/Support

Provider Site Support

Provider Admin Forum

Fidelity Review

Model Construction Model Delivery Inquiry Technical Assistance

Cost Benefit Analysis

Synthesis Translation Delivery Support

Model Refinement

Operations Adjustments

Formal Program Review

Best Practice Inform Policy

The Enhanced Implementation Framework

(References)

Synthesis - Inquiry • Research:

– evidence - journals / publications • reference the program model, or components of the model, and discusses process,

outcome, utilization, effect, etc.

• Practice: – discussion, presentations, publications

• cites the model, or components of the model, practice evidence, or model design.

• Current Offerings: – list, description, or discussion of sites or agencies

• using the model, components of the model, and, if available, effect or impact.

• Needs Assessment: – formal or non-structured assessment through VA or other agency, or

combination, providing evidence of value of implementing model.

Synthesis - Inquiry

• Technical Manuals: – presentation of the model or components of the model, in publication or

web, as implemented • under other funding authorities or agencies presenting description of operational specifics

for program managers.

• Authorities for Services Provision: – public law, directives, manuals, circulars, or other documents to authorize

services, including EDMs and other internal documents

Translation – Model Construction • Program Components:

– key program operational principles and components • intent, population served, course of treatment, outcomes , expected utilization and work

load.

• Site Requirements: – necessary elements to implement at sites

• general and not specific to individual sites, includes requirement specifics but general enough to list for all sites.

• Pragmatic Adjustments: – modifications of model

• moving from ‘perfect’ model, based on research and practice, to a model that retains core components but satisfies unique aspects of VA, site, locality.

Translation – Model Construction • Training Requirements:

– based on an assessment of design newness and current knowledge of site management and core staff

• the necessary elements in educational curriculum development.

• Funding Requirements: – developed based on constructed model, needs assessment, funding

availability, and developing site priority • the estimated total for each site and estimated total for all sites, considering start up costs,

prorated for implementation date, and possible readjustment of funds once sites are operational.

• Operational Requirements for Providers: – recognizes the VA requirements and the provider requirements

• education, facility structure, contractual agreements, etc.

Delivery – Model Delivery • Site Suitability:

– the necessary infrastructure and support, • ability to obtain facility, contract (if necessary), access, feasibility of location within

community.

• Site Buy In: – stakeholder commitment,

• VA upper, mid, line level staff support as well as community support including political, community provider, continuum, coalition leaders and staff.

• Provider Contracts or Grant Agreements: – the ‘business’ instruments

• providers may have existing or in the past and developing those contracts/agreements through processes necessary.

Delivery – Model Delivery

• Staff and Site Training: – initial educational scheme

• including curriculum and method of delivery, to inform stakeholders through overview, and line level VA and community staff.

• Program Operation: – initial operational challenges and participant impact

Support – Technical Assistance • Core Group Forum:

– weekly or bi-weekly calls for management and line staff, VA.

• Individual Site Consultation: – calls with individual sites to address unique challenges.

• Core Group Admin Forum: – calls with core administration and program leads.

• Fidelity Review: – on-site or tele-com reviews of model design through methods developed to

determine model adoption and adherence.

Support – Technical Assistance • Outcome Data:

– reviews of participant outcomes – meeting, achieving model intent, and effect.

• Formal Program Review: – on-site reviews by team staff

• review for fidelity, community integration, participant process and outcome measures.

Research

Practice

Current Offerings

Needs Assessment

TA-Technical Manuals Training

Development

Funding Requirements

Providers Operational Requirements

Authorities for Provision of Services

Program Components

Site Requirements

Programmatic Adjustments

Site Suitability

Stakeholder Buy-in

Program Operation

Staff and Site Training

Provider Contracts

Outcome Data Feedback

Bi-weekly Admin Forum/Support

Provider Site Support

Provider Admin Forum

Fidelity Review

Model Construction Model Delivery Inquiry Technical Assistance

Cost Benefit Analysis

Synthesis Translation Delivery Support

Model Refinement

Operations Adjustments

Formal Program Review

Best Practice Inform Policy

The Enhanced Implementation Framework

(References)

Definition and Description of Terms: Implementation Model

• Reference Document – Definition of Terms – See Download Box

Request for Proposals (RFP) Process

• Primary requirements – “out of the gate” • Demonstrate model design - proposal process

– Need – Assessment and review of VA and community data – Evidence of local leadership commitment vestment / support – Community stakeholder – Adherence to administrative and clinical principles of low

demand recovery approach to service

Fidelity Review - Initial

• Site reviews (assess for refinement program design) – On site reviews/interviews/program materials – Primary site program adoption

• No requirements for sobriety or continued stay • Low demand harm reduction approach

– Differences among sites • Staffing • SUD and alcohol testing • Addressing SUD / ETOH use

– Second RFP - commitment to refined model design

Fidelity Instrument – Refined

• Based on results of one year fidelity reviews – Descriptive features

• Physical facility, staffing, outreach services

– Key features • Eligibility criteria, approach to services, direct services, coordination links

with community/VA services, participant requirements, program rules, entry and exit procedures

(Fidelity Instrument – download box)

Fidelity Reviews – Second Review

• Consistent population and admission – Chronic homeless / SUD and SMI

• Engagement – Utilization of relapse, rule infringement

• Community organizations, shelters (52%) • Frequent relapse (only 2% resulted in DC) • Testing (39%) / None (61%)

Fidelity Reviews • Program “design up”

– Safe room and sober lounge – Impairment – other engagement methods – Belonging checks – Medication monitoring – Amnesty boxes – Participation in program and facility functions – Outreach

Technical Assistance • Initial training

– Two day for primary sites

• Ongoing – All-site conference calls monthly

• Forums – management and services • Challenges, innovative practices

– Individual site calls as necessary / requested – Education – related webinars

• SUD, MH, Low Demand, Stages of change, MI, safety • Resources – links, web sites, paper

Population • FY 2016 (n=3,125)

– Chronic homeless 63% • Four episodes or more in past year • Year or more

– SUD / SMI • Alcohol 50% • Dug 44% • SMI 33% • PTSD 31%

– Less than 9% worked in past 30 days

Outcomes • FY 2016 (n=3,125 – 268 receiving services)

– LOS - Average • 134 days

– Exits • 12% violence - threats • 13% voluntary • 65% own residence, permeant housing, VA treatment • 10% left without housing or unknown

– Limited differences other programs - employment

Challenges - Homeless

• Homeless populations • Uniqueness of site and facility • Staff background, culture • Local resources • Six-month requirement • Substances not allowed on premises

Summary • Implications for developing program models using the

enhanced implementation framework – Start with theory and framework – Refine review / assessment tools based on small scale

implementation if possible – Learn from the site – Re-refine review / assessment tool with “design up” – Localities are different – Funding is an influencer

Summary

• Implications for developing homeless program models using the enhanced implementation framework – Considerations for homeless populations – Community providers - gap between authority and responsibility – Disconnections

• VA and provider • Provider management and provider staff

– Contracts and grants

Summary • Program enhancements

– “Learned up” the framework and disseminated “across” the framework

• Safe room – sober lounge • amnesty boxes • Use of urine screens and breathalyzers • Contract negotiations • Need for broad scope commitment leadership/police • Support from other local provider programs (LA voa)

References

• Wandersman A, Duffy J, Flaspohler P, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008; 41: 171-181

• Casey R, Clark C, Smits P, Peters R. Application of implementation science for homeless interventions. Am J of Public Health; Supplement 2; 2013; 103: S183-S185

Questions