socially significant solutions - helping families...
TRANSCRIPT
Dean Fixsen & Karen Blase, National Implementation Research Network
1
Karen A. Blase and Dean L. Fixsen
National Implementation Research Network
FPG Child Development Institute
University of North Carolina at Chapel Hill
Helping Families Change
Conference
Feb 14, 2013Los Angeles, CA
Implementation Science –Building the Bridge from Good Intentions to Great Outcomes
The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings.
The purpose is to produce greater benefits to children, families, individuals, and societies.
Socially Significant Solutions
Dean Fixsen & Karen Blase, National Implementation Research Network
2
Socially Significant Solutions
It is one thing to say with the prophet Amos, “Let justice roll down like mighty waters,” and quite another to work out the irrigation system.
~ William Sloane CoffinUS Social activist and clergyman
Dean Fixsen & Karen Blase, National Implementation Research Network
3
Science to Service Gap
What is known is not what is used to help children, families, individuals, and communities
Implementation GapWhat is adopted is not used with fidelity and
good outcomes for consumers.
What is used with fidelity is not sustained for a useful period of time.
What is used with fidelity is not used on a scale sufficient to impact social problems.
Science to Service GapImplementation Gap
Experimental Data Show These Methods, When Used Alone, Are Insufficient:
• Diffusion/ Dissemination of information
• Training
• Passing laws/ mandates/ regulations
• Providing funding/ incentives
• Organization change/ reorganization
Data: 5% to 15% Realize Intended Outcomes
Implementation Science
NECESSARY BUT NOT SUFFICIENT
Dean Fixsen & Karen Blase, National Implementation Research Network
4
Evidence-base Actual SupportsYears 1-3
OutcomesYears 4-5
Every Teacher Trained Fewer than 50% of the teachers received some training
Fewer than 10% of the schools used the CSR as intended
Every Teacher Continually Supported
Fewer than 25% of those teachers received support
Vast majority of students did not benefit
Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006
Longitudinal Studies of a Variety of Comprehensive School Reforms
Socially Significant Solutions
• $100 billion for innovative programs (USDE)
• $63 billion for maternal health programs (USAID)
• $4 billion for homevisiting programs (ACF)
• No funding set aside for implementation supports for these program initiatives
Socially Significant Solutions
Dean Fixsen & Karen Blase, National Implementation Research Network
5
National Assessment of Education Progress (NAEP)
40 Years of Variation Around a Mediocre Mean
Invent a new future
• Stop wasting time and money on things that don’t work (and never have!)
– Sugai’s Law: For every new initiative, stop two current ones.
– De-scale; Avoid layering and fragmentation
• Set aside 15% of funds for implementation
• Require quarterly reports of fidelity data
System Reinvention
Dean Fixsen & Karen Blase, National Implementation Research Network
6
Tra dire e fare
C'e di mezzo il mare
~ Italian Proverb
Between saying and doing is the sea.
� Intervention Criteria
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science
Dean Fixsen & Karen Blase, National Implementation Research Network
7
Making It HappenActive Implementation
• Letting it happen– Recipients are accountable
• Helping it happen– Recipients are accountable
• Making it happen– Purposeful and proactive use of
implementation practice and science
– Implementation Teams are accountable
Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010)
Nĺ Fheicfidh tu plow an reimse ag casadh se os achionni d’intinn
~ Gaelic Proverb
You’ll never plough the field by turning it over in
your mind….
Dean Fixsen & Karen Blase, National Implementation Research Network
8
WHYSocially
Significant Outcomes
WHATEffective
Interventions
HOW/WHOEffective
Implementation Methods
WHEREEnabling Contexts
Formula for Success
WHYSocially
Significant Outcomes
WHAT:Effective
Interventions
HOW/WHO:Effective
Implementation Methods
WHEREEnabling Contexts
Formula for Success
Dean Fixsen & Karen Blase, National Implementation Research Network
9
� Intervention Criteria
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science
• About 18% of outcome studies (N= 1,200+) assessed the independent variable (“it”)
– About 7% linked essential components (fidelity) to outcomes (we know what produces desired outcomes)
Moncher & Prinz, 1991;Gresham, et al., 1993; Dane & Schneider, 1998; Durlak & DuPre, 2008
What is “it”?
Dean Fixsen & Karen Blase, National Implementation Research Network
10
Clear description of the program
Philosophy, values, principles (guidance)
Inclusion – exclusion criteria (beneficiaries)
Clear essential functions that define the program (core components)
Operational definitions of essential functions (practice profiles; do, say)
Practical performance assessment
Highly correlated (0.70+) with desired outcomes
Usable Intervention Criteria
Most evidence-based “interventions” do not meet usability criteria
Practitioners and managers use interventions (not standards for rigor)
Scaling for population benefits requires usable interventions
Usable Intervention Criteria
Dean Fixsen & Karen Blase, National Implementation Research Network
11
Socially Significant Outcomes
WHAT:Effective
Interventions
HOW/WHO:Effective
Implementation Methods
Enabling Contexts
Formula for Success
A fundamental truth:
• People cannot benefit from programs they do not experience
Effective Innovations
• If we cannot implement them as designed (with fidelity)
• They will not produce desired outcomes
Implementation Science
Dean Fixsen & Karen Blase, National Implementation Research Network
12
(Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)
Effective NOT Effective
Effective
NOT Effective
IMPLEMENTATIONIN
TE
RV
EN
TIO
N Actual Benefits
Inconsistent; Not Sustainable; Poor outcomes
Poor outcomes Poor outcomes; Sometimes harmful
Implementation Science
DBT
Dean Fixsen & Karen Blase, National Implementation Research Network
13
DBT TAUIndividual Psychotherapy $3,885 $2,915
Group Psychotherapy $1,514 $147
Day Treatment $11 $876
Emergency Room Visits $226 $569
Psychiatric Inpatient Days $2,614 $12,008
Medical Inpatient Days $360 $1,094
TOTAL $8,610 $17,609
ONE YEAR HEALTH CARE COSTS PER PATIENT: Dialectical Behavior TherapyLinehan et al., 1991
� Intervention Criteria
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science
Dean Fixsen & Karen Blase, National Implementation Research Network
14
Stages of Implementation
Stages of Implementation
Dean Fixsen & Karen Blase, National Implementation Research Network
15
Stages of Implementation
� You don’t get to skip any!
� If you make a judgment about “effectiveness” too early you might be making a mistake.
� Activities need to match the stage.
� Different stages for multiple initiatives� “Where are we now with this initiative?”
� Stages will need to be “revisited” – important!• New providers, new communities, new government officials,
new families
� Intervention Criteria
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science
Dean Fixsen & Karen Blase, National Implementation Research Network
16
© Fixsen & Blase, 2008
Fidelity Assessment
Coaching
Training
Selection
Systems Support & Intervention
FacilitativeAdministration
Decision SupportData System
AdaptiveTechnical
Leadership DriverLeadership Driver
Consistent Use of
The Innovation
Reliable Benefits (Outcomes)
Where the “WHAT”Meets the “HOW”
Integrated and Compensatory
© Fixsen & Blase, 2008
Fidelity Assessment
Coaching
Training
Selection
Systems Support & Intervention
FacilitativeAdministration
Decision SupportData System
AdaptiveTechnical
Leadership DriverLeadership Driver
Consistent Use of
The Innovation
Reliable Benefits (Outcomes)
Where the “WHAT”Meets the “HOW”
INFRASTRUCTUREPlan
MaintainImprove
Dean Fixsen & Karen Blase, National Implementation Research Network
17
OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate
new Skills in a Training Setting, and Use new Skills in the Classroom)
TRAININGCOMPONENTS
KnowledgeSkill
DemonstrationUse in the Classroom
Theory and Discussion
10% 5% 0%
..+Demonstration in Training
30%20%
0%
…+ Practice & Feedback in Training
60% 60% 5%
…+ Coaching in Classroom
95% 95% 95%
Joyce and Showers, 2002
Staff Training & Coaching
Monitoring/Coaching and Fidelity, and Practitioner Turnover
N=153
Aarons et al., 2009
Dean Fixsen & Karen Blase, National Implementation Research Network
18
Results from Child Wellbeing Project
Case Example: Metz et al.
Component T1 T2 T3
Selection 1.44 2.00* 1.89*
Training 1.33 1.50* 1.10
Coaching 1.27 1.73* 1.83*
Perf. Assessment 0.78 1.34 2.00*
DSDS 0.18 1.36 2.00*
Fac. Administration 1.38 2.00* 2.00*
Systems Intervention 1.29 1.86* 2.00*
Average Composite
Score 1.1 1.68* 1.83*
Fidelity (% of
cases)
18% 83% 83%
Success Coach model involved intense program development of core intervention components and accompanying implementation drivers
� Intervention Criteria
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science
Dean Fixsen & Karen Blase, National Implementation Research Network
19
What Do They Have in Common?
�Policy to Practice to Policy Cycles
�Transformation Zones
�Rapid Cycle Teams�Problem-solving�Practice Improvement
�Usability Testing
PDSA Cycles
Shewhart (1924); Deming (1948); Six-Sigma (1990)
– Plan – Develop specific things to do
– Do – Do them (make sure)
– Study – See what happens
– Act – Make adjustments
– Cycle – Do over and over again until the goal is reached or the problem is clear (again)
Dean Fixsen & Karen Blase, National Implementation Research Network
20
What Do They Have in Common?
�Policy to Practice to Policy Cycles
�Transformation Zones
�Rapid Cycle Teams�Problem-solving�Practice Improvement
�Usability Testing
� Intervention Criteria
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science
Dean Fixsen & Karen Blase, National Implementation Research Network
21
Minimum of three people (four or more preferred) with expertise in:
Innovations
Implementation
Organization change
Tolerate turnover; teams are sustainableeven when the players come and go (Higgins, Weiner, & Young, 2012; Klest & Patras, 2011)
Implementation Team
• Supply side: Effective programs go where they are most welcome
– Let it happen; Help it happen
– Islands of excellence
• Demand side: Effective programs go where they are most needed
– Make it happen; whole populations
– A sea of change
Population Benefits
Dean Fixsen & Karen Blase, National Implementation Research Network
22
Organization Supports
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Practitioner/Staff Competence
State and Community Supports
Regional Supports
Imp
lem
en
tati
on
Te
am
Simultaneous, Multi-Level Interventions
Federal and National Supports
Implementation Team
© Fixsen & Blase, 2009
80%
Implementation Team
Prepare Organizations
Prepare Practitioners and Staff
Work with Researchers
Assure Implementation
Prepare Regions Assure Intended Benefits
Create Readiness
Parents and Stakeholders 20%
Implementation Team
Dean Fixsen & Karen Blase, National Implementation Research Network
23
Expert Impl. Team NO Impl. Team
Effective
Effective use of Implementation Science & Practice
IMPLEMENTATIONIN
TE
RV
EN
TIO
N
80%, 3 Yrs 14%, 17 Yrs
Balas & Boren, 2000 Green, 2008
Fixsen, Blase, Timbers, & Wolf, 2001
Letting it Happen Helping it Happen
Implementation Team
0
10
20
30
40
50
60
70
80
90
100
Practitioner Development Organization Development
Op
era
tin
g 6
+ Y
rs.
N = 84 N = 219
Fixsen, Blase, Timbers, & Wolf (2001)
Program Sustainability
Dean Fixsen & Karen Blase, National Implementation Research Network
24
Socially Significant Outcomes
WHAT:Effective
Interventions
HOW/WHO:Effective
Implementation Methods
WHERE:Enabling Contexts
Formula for Success
You should be more You should be more You should be more You should be more explicit here in Step 2!explicit here in Step 2!explicit here in Step 2!explicit here in Step 2!
Dean Fixsen & Karen Blase, National Implementation Research Network
25
Why Focus on Context?
– Because the Current System is perfectly designed to get the current results!
– And Systems Can Trump Programs!
Existing System
Effective Innovations
Are Changed to
Fit The System
Existing System Is
Changed To Support
The Effectiveness Of
The Innovation
Effective Innovation
System Reinvention
Dean Fixsen & Karen Blase, National Implementation Research Network
26
Implementation Team
StateManagement
Team
PractitionersInnovations
Children, Families
Sys
tem
C
ha
ng
e
Adaptive Challenges
• Duplication
• Fragmentation
• Hiring criteria
• Salaries
• Credentialing
• Licensing
• Time/ scheduling
• Union contracts
• RFP methods
• Federal/ State laws
“E
xte
rnal”
Syste
m C
han
ge S
up
po
rt
Pra
cti
ce-P
olicy
Co
mm
un
icati
on
Lo
op
Po
licy E
nab
led
P
ractic
e
Reinvent Systems: Manage Change
“The fault cannot lie in
the part responsible
for the repair.”Ashby (1956)
Bureaucracy
Practitioners
Policies
Provider Organizations
Functional Systems
Socially Significant Outcomes
Dean Fixsen & Karen Blase, National Implementation Research Network
27
WHY:Socially
Significant Outcomes
WHAT:Effective
Interventions
HOW/WHO:Effective
Implementation Methods
WHERE:Enabling Contexts
Formula for Success
What Will It Take to Make Socially Significant Progress?
�Science and Data Related to…. � Interventions and Programs� Implementation, Organization, and Systems Change
�Funded Infrastructure (Drivers)�Informed & engaged consumers & communities�Linked Teams
�Skillful Purveyors & Intermediaries �Competent Providers�Supportive Organizations�Engaged communities and stakeholders
�Hospitable & Aligned Systems�Leadership at All Levels
Dean Fixsen & Karen Blase, National Implementation Research Network
28
What did you expect? What did you expect? What did you expect? What did you expect? e = mce = mce = mce = mc2222
Dean Fixsen & Karen Blase, National Implementation Research Network
29
www.implementationconference.org
August 19-21, 2013
Dean L. Fixsen, Ph.D.
– 919-966-3892
Karen A. Blase, Ph.D.
– 919-966-9050
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
http://nirn.fpg.unc.edu/www.scalingup.org
www.implementationconference.org
For More Information
Dean Fixsen & Karen Blase, National Implementation Research Network
30
HTTP://NIRN.FPG.UNC.EDU
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Implementation Research: A Synthesis of the Literature
Implementation Science
©Copyright Dean Fixsen and Karen Blase
This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercial-NoDerivs. You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter or transform this work. Any of the above conditions can be waived if you get permission from the copyright holder.
http://creativecommons.org/licenses/by-nc-nd/3.0
Dean Fixsen & Karen Blase, National Implementation Research Network
31
• Annie E. Casey Foundation (EBPs and Cultural Competence)
• William T. Grant Foundation (Implementation Literature Review)
• Substance Abuse and Mental Health Services Administration (Implementation Strategies Grants; National Implementation Awards)
• Centers for Disease Control & Prevention (Implementation Research)
• National Institute of Mental Health (Research And Training Grants)
• Juvenile Justice and Delinquency Prevention (Program Development And Evaluation Grants)
• Office of Special Education Programs (Scaling up and Capacity Development Center)
• Administration for Children and Families (Child Welfare Leadership; Capacity Development Center)
• Duke Endowment (Child Welfare Reform)
Thank You for Your Support