applied implementation science - anu · melissa van dyke, allison metz national implementation...
TRANSCRIPT
![Page 1: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/1.jpg)
Dean Fixsen, Karen Blase,
Melissa Van Dyke, Allison Metz
National Implementation Research Network
FPG Child Development Institute
University of North Carolina at Chapel Hill
Applied Implementation Science
Canberra Conference on
Research Integration and Implementation
![Page 2: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/2.jpg)
Implementation Science
![Page 3: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/3.jpg)
Making It HappenImplementation Science
• Letting it happen
– Diffusion; networking; communication
• Helping it happen
– Dissemination; manuals; websites
• Making it happen
– Purposeful and proactive use of implementation practice and science
Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010)
![Page 4: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/4.jpg)
Basic Research Applied Research
Interest-Driven Mission-Driven
Internal Validity (Rigor)
External Validity (Relevance)
Eventually Useful Immediately Useful
Contribute to Knowledge Base and Theory
![Page 5: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/5.jpg)
• Fairweather, Sanders, & Tornatzky (1974)
– Purposeful approaches to use of Lodges in random samples of hospitals
– Dissemination of manuals plus telephone consultation produced little impact
– Face-to-face “action consultation” was more effective – lots of communication to resolve conceptual, structural, and policy issues en
route to using the Lodges
Implementation Science
![Page 6: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/6.jpg)
• Fairweather, G. W., Sanders, D., Maynard, H., & Cressler, D. (1969). Community life for
the mentally ill. Chicago: Aldine Publishing.
• Tornatzky, L. G., Fergus, E. O., Avellar, J. W., Fairweather, G. W., & Fleischer, M. (1980). Innovation and social process: A national experiment in implementing social
technology. New York: Pergamon Press.
• Beard, J. H., Propst, R. N., & Malamud, T. J. (1982). The Fountain House model of rehabilitation. Psychosocial Rehabilitation Journal, 5, 47-53.
• Propst, R. (1992). Standards for clubhouse programs: Why and how they were developed. Psychosocial Rehabilitation Journal, 16, 25-30.
• Onaga, E. E. (1994). The Fairweather lodge as a psychosocial program in the 1990's. In The Publications Committee of IAPSRS (Ed.), An Introduction to Psychiatric
Rehabilitation. Columbia, MD
• Onaga, E. E., & Smith, B. A. (2000). Reinvention of the Lodge Program: A case study of program changes to promote full-time employment. Psychiatric Rehabilitation Skills.
• Lucca, A. M. (2000). A Clubhouse Fidelity Index: Preliminary Reliability and Validity Results. Mental Health Services Research, 2(2), 89-94.
• Macias, C., Propst, R., Rodican, C., & Boyd, J. (2001). Strategic Planning for ICCD Clubhouse Implementation: Development of the Clubhouse Research and Evaluation Screening Survey (CRESS). Mental Health Services Research, 3(3), 155-167.
Mission-Driven process of research and development (30+ Yrs)
Implementation Science
![Page 7: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/7.jpg)
Making It HappenImplementation Science
• Accumulation of data
• Summaries of the literature on diffusion and dissemination
• Synthesis of the implementation evaluation literature
Poised for more evidence-informed implementation of EBPs and
other innovations
![Page 8: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/8.jpg)
Making It HappenImplementation Science
• Sufficient data to move from making lists (facilitators & barriers) to making sense
– Frameworks to guide implementation work
• Meyers, Durlak, and Wandersman (2012) comparison of 25 implementation frameworks
• Tabak, Khoong, Chambers, & Brownson (2012) reviewed 61 models for diffusion, dissemination, and implementation
![Page 9: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/9.jpg)
Socially Significant Outcomes
Effective Interventions
Effective Implementation
Enabling Contexts
Formula: Applied Implementation
![Page 10: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/10.jpg)
� Usable Interventions
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science: Active
Implementation Frameworks
![Page 11: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/11.jpg)
Socially Significant Outcomes
WHAT:Effective
Interventions
Effective Implementation
Enabling Contexts
Applied Implementation
![Page 12: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/12.jpg)
Clear description of the program
Philosophy, values, principles (guidance)
Inclusion – exclusion criteria (beneficiaries)
Clear essential functions that define the program (core components)
Operational definitions of essential functions (practice profiles; do, say)
Practical performance assessment
Highly correlated (0.70+) with desired outcomes
Usable Intervention Criteria
![Page 13: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/13.jpg)
DBT
![Page 14: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/14.jpg)
0 10 20 30 40 50 60
1234567
M ean89
101112
M ean131415161718
M ean19202122232425
M ean
CONTROL
% Recidivism
Highly Competent & Competent Therapists
N=12; 204 Families
13% Recidivism
Borderline & Not Competent Therapists
N=13; 223 Families
28% Recidivism
Ab
ove
Fid
elity
C
rite
ria
Be
low
Fid
elity
C
rite
ria
Functional Family Therapists (WSIPP)
Control Group 22% Recidivism
![Page 15: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/15.jpg)
Socially Significant Outcomes
WHAT:Effective
Interventions
HOW & WHO:Effective
Implementation
Enabling Contexts
Applied Implementation
![Page 16: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/16.jpg)
2 - 4
Years
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
EXPLORATION
� Usable Interventions� Implementation Drivers� Implementation Teams� Enabling Contexts
Stages of Implementation
![Page 17: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/17.jpg)
© Fixsen & Blase, 2008
Coaching
Training
Selection
Integrated &
Compensatory
Systems Intervention
FacilitativeAdministration
Decision SupportData System
AdaptiveTechnical
Leadership DriversLeadership Drivers
Reliable Benefits
Consistent uses of Innovations
Performance Assessment (fidelity)
![Page 18: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/18.jpg)
0
10
20
30
40
50
60
70
80
90
100
Practitioner Development Organization Development
Op
era
tin
g 6
+ Y
rs.
N = 84 N = 219
Fixsen, Blase, Timbers, & Wolf (2001)
Organization & Leadership Drivers
17%
84%
![Page 19: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/19.jpg)
Improvement Cycles
Rapid cycle (PDSA) problem solvingShewhart (1931); Deming (1986)
Usability testing Rubin (1994); Nielsen (2000)
Practice-policy communication loopFixsen, Blase, Metz, & Van Dyke (2013)
![Page 20: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/20.jpg)
Minimum of three people (four or more preferred) with expertise in:
Innovations
Implementation
Organization change
Tolerate turnover; teams are sustainableeven when the players come and go (Higgins, Weiner, & Young, 2012; Klest & Patras, 2011)
Implementation Team
![Page 21: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/21.jpg)
Implementation Team
Implementation
Drivers
T1 T2 T3
Selection 1.44 2.00* 1.89*
Training 1.33 1.50* 1.10
Coaching 1.27 1.73* 1.83*
Perf. Assessment 0.78 1.34 2.00*
DSDS 0.18 1.36 2.00*
Fac. Administration 1.38 2.00* 2.00*
Systems Intervention 1.29 1.86* 2.00*
Average Composite
Score 1.1 1.68* 1.83*
Fidelity (% of cases) 18% 83% 83%
Metz et al. (2013)2 = Fully in Place1 = Partially in Place0 = Not in Place
![Page 22: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/22.jpg)
Expert Impl. Team NO Impl. Team
Effective
Effective use of Implementation Science & Practice
IMPLEMENTATION
INT
ER
VE
NT
ION
80%, 3 Yrs 14%, 17 Yrs
Balas & Boren, 2000
Green, 2008
Fixsen, Blase, Timbers, & Wolf, 2001
Saldana & Chamberlain, 2012
Letting it Happen Helping it Happen
Implementation Team
![Page 23: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/23.jpg)
Socially Significant Outcomes
WHAT:Effective
Interventions
HOW/WHO:Effective
Implementation
WHERE:Enabling Contexts
Applied Implementation
![Page 24: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/24.jpg)
Implementation Team
ExecutiveManagement
Team
PractitionersInnovations
Beneficiaries
Syste
m
Ch
an
ge
Adaptive Challenges
• Duplication
• Fragmentation
• Hiring criteria
• Salaries
• Credentialing
• Licensing
• Time/ scheduling
• Union contracts
• RFP methods
• Laws & regulations
“E
xte
rna
l” S
ys
tem
Ch
an
ge
Su
pp
ort
Pra
cti
ce
-Po
lic
y
Co
mm
un
ica
tio
n L
oo
pP
olic
y E
na
ble
d
Pra
ctic
e
Creating Enabling Contexts
“The fault cannot lie in
the part responsible
for the repair.”Ashby (1956)
![Page 25: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/25.jpg)
A fundamental truth:
• People cannot benefit from innovations they do not experience
Effective Innovations
• If we cannot use them as designed
• They will not produce desired outcomes
Implementation Science
![Page 26: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/26.jpg)
� Usable Interventions
� Implementation Stages
� Implementation Drivers
� Improvement Cycles
� Implementation Teams
APPLIED Implementation Science: Active
Implementation Frameworks
![Page 27: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/27.jpg)
Allison Metz
Melissa Van Dyke
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
http://nirn.fpg.unc.edu/www.scalingup.org
www.globalimplementation.org
For More Information
![Page 28: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/28.jpg)
HTTP://NIRN.FPG.UNC.EDU
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Implementation Research: A Synthesis of
the Literature
Implementation Science
![Page 29: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/29.jpg)
Get Connected!
www.scalingup.org
SISEP @SISEPcenter
For more on Implementation Sciencehttp://nirn.fpg.unc.edu
www.globalimplementation.org
![Page 30: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/30.jpg)
• Annie E. Casey Foundation (EBPs and Cultural Competence)
• William T. Grant Foundation (Implementation Literature Review)
• Substance Abuse and Mental Health Services Administration (Implementation Strategies Grants; National Implementation Awards)
• Centers for Disease Control & Prevention (Implementation Research)
• National Institute of Mental Health (Research And Training Grants)
• Juvenile Justice and Delinquency Prevention (Program Development And Evaluation Grants)
• Office of Special Education Programs (Scaling up and Capacity Development Center)
• Administration for Children and Families (Child Welfare Leadership; Capacity Development Center)
• Duke Endowment (Child Welfare Reform)
Thank You for Your Support
![Page 31: Applied Implementation Science - ANU · Melissa Van Dyke, Allison Metz National Implementation Research Network ... Mental Health Services Research, 2(2), 89-94. ... South Florida,](https://reader033.vdocuments.us/reader033/viewer/2022051605/600b4ab1f3856b7f54632959/html5/thumbnails/31.jpg)
©Copyright Dean Fixsen and Karen Blase
This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercial-NoDerivs. You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter or transform this work. Any of the above conditions can be waived if you get permission from the copyright holder.
http://creativecommons.org/licenses/by-nc-nd/3.0