dean l. fixsen, ph.d. karen a. blase, ph.d. national implementation research network louis de la...

Dean L. Fixsen, Ph.D.Karen A. Blase, Ph.D.

National Implementation Research NetworkLouis de la Parte Florida Mental Health Institute

Implementing Innovations with Benefits to Consumers & Stakeholders

SIG/SPDG Directors and Evaluators

Copyright © Dean L. Fixsen and Karen A. Blase, 2006

A Little Known Fact

An evidence-based program is one thing

Implementation of an evidence-based program is a very different thing


– Lisbeth Schorr, 1993

Implementation

Successful programs do not contain the seeds of their own replication.


Effective ProgramsEvidence-based programs are

effective when they are implemented with fidelity

Fidelity = the practitioners use all the core intervention components skillfully

68

92 9789

363447

59

0

20

4060

80

100

2000-01 2001-02 2002-03 2003-04

Years

Mea

n %

of S

choo

lwid

e Fe

atur

es Im

plem

ente

d

SET

ODR/100

ODR = Office Discipline Referrals ; SET = Fidelity Measure (80% Score for Criterion)

Data provided by Sugai, Horner, Eber, Dunlap: OSEP TA Center on SWPBS

PBS: Fidelity and Outcome

0102030405060708090

100

4 months 8 months 12 months Tx yeartotal

Adherent

Non-adherent

Pro

po

rtio

n o

f n

egat

ive

UA

sPROPORTION OF NEGATIVE URINALYSES BY ADHERENT AND NON-ADHERENT

DBT THERAPISTS

86.0

39.0

75.4

37.1

81.1

10.5

80.8

28.9

Linehan, Dimeff et al., 2002

FFT: Therapist Competency Ratings

29%

25%

14%

17%

22%

0%

5%

10%

15%

20%

25%

30%

35%

Not Competent Marginal Competent HighlyCompetent

Control GroupRecidivism Rate

12 M

onth

Fel

ony

Rec

idiv

ism

Source: “Implementing and Evaluating Evidence-based Intervention Programs for At-RiskYouth”, Thomas L. Sexton, Indiana University, School of Education. Available at:http://www.indiana.edu/~cafs/presentations/SchoolEDRES.ppt

0.60

0.72

0.83

0.95

1.07N

um

be

r o

f C

ha

rge

s p

er

Ye

ar

Po

st-

Tx

-0.64 -0.36 -0.09 0.19

Deviation from Mean TAM Score (TAM - .64)0 .28 .55 .83 1

MST Therapist Adherence Measure (TAM)

Schoenwald, Sheidow, & Chapman, 2006

Multisystemic Therapy (MST)

38% Improvement

Lower Higher

Therapist Adherence Predicts 2-Year Post Treatment Youth Criminal Activity* (N = 1,979 Families)

Information provided by Keller Strother and Sonja Schoenwald, MST Services.

Teaching-Family Model

0%

10%

20%

30%

40%

50%

Homes

1.0

1.2

1.5

1.7

1.9

2.0

1.1

1.4

1.6

1.8

1.3

Delinquency

(Outcomes)

Teaching

(Fidelity)

rs = – .94

Pe

rce

nt

Pa

ren

tal-

Tea

ch

ing

Wit

h Y

ou

ths

Me

an

Se

lf Rep

orte

dD

elin

qu

en

cy

Ratin

gs

1 2 3 4 5 6

Bedlington, et al., 1982

Fidelity is Essential (Reduction in Recidivism from .50 Control Group Rate)

Source: Adapted from Lipsey, 1997, 2005


Summary

Across many evidence-based programs, fidelity matters

The best outcomes are associated with high fidelity practitioners

High fidelity results from careful attention to implementation of evidence-based programs


Evidence-based programs

The usability of a program has little to do with the quality or weight of the evidence regarding that program

Evidence about intervention effectiveness for specific populations helps us choose what to implement

Evidence about the effectiveness of the intervention does not help implement the program or practice successfully


Evidence-based programs

Key aspects of core components

Clearly described (what/how)

Practical measure of fidelity

Fully operationalized (do/say)

Field tested (iterative revision)

Contextualized (school/systems fit)

Good outcomes (worth replicating)


Teachers, Staff, Schools

Teachers impact students

It is the job of superintendents, principals, and funders to align policies and structures to facilitate effective teacher practices

There is no such thing as an “administrative decision” – they are all education decisions

“Climate for Implementation”* & Success

IMPLEMENTATION EFFECTIVENESS

(e.g. fidelity)

Measures • Top mgmt support

• Goal Clarity

• Dedicated resources

• Access to training & TA

• Performance monitoring

• Reinforcement/rewards

• Removal of obstacles

• Freedom to express

doubts

INNOVATION EFFECTIVENESS

(e.g., outcomes)

+++

+++

NS

* (Practice- specific e.g., Klein & Sorra, 1996, 2001; Panzano, Roth, Crane-Ross et al, 2004, Vaidyanathan, 2004)

INTEGRATED & COMPENSATORY

CONSULTATION & COACHING

CONSULTATION & COACHING

STAFF PERFORMANCE EVALUATION

STAFF PERFORMANCE EVALUATION

FACILITATIVE ADMINISTRATIVE

SUPPORTS

FACILITATIVE ADMINISTRATIVE

SUPPORTS

RECRUITMENT AND SELECTION

RECRUITMENT AND SELECTION

PRESERVICE TRAINING

PRESERVICE TRAINING

SYSTEMSINTERVENTIONS

SYSTEMSINTERVENTIONS

Implementation Drivers

DECISION SUPPORT DATA SYSTEMS

DECISION SUPPORT DATA SYSTEMS

OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate

new Skills in a Training Setting, and Use new Skills in the Classroom)

TRAININGCOMPONENTS

KnowledgeSkill

DemonstrationUse in the Classroom

Theory and Discussion

10%

5% 0%

..+Demonstration in Training

30%20%

0%

…+ Practice & Feedback in Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

Joyce and Showers, 2002


Stages of Implementation

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

0% 100%

Implementation Outcomes

Intervention Outcomes


Active Purveyor RoleChange the behavior of adult human service professionals

“Systems don’t change, people do” (J.W.)

Change organizational structures, cultures, and climates

Change the thinking of system directors and policy makers

Successful and sustainable implementation of evidence-based practices and programs always requires organizational change.


Active Purveyor Role

Organization

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Practitioner

State, County and Local Context

System of care

Pu

rvey

or

Gro

up

Simultaneous, Multi-Level Interventions


Implementation EssentialsHelp faculty/staff acquire the knowledge, skills, and abilities to effectively provide innovations to students

Help schools/districts change to more effectively support the work of the faculty/staff

Transform education systems to effectively facilitate and sustain the use of innovations statewide


For More Information

Dean L. Fixsen

813-974-4446

[email protected]

Karen A. Blase

813-974-4463

[email protected]

National Implementation Research Network

http://nirn.fmhi.usf.edu


For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Download all or part of the monograph at:

http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm

dean l. fixsen, ph.d. karen a. blase, ph.d. national implementation research network louis de la...

Documents