vEffective Interventions for

Promoting Social Development:

Implementing and Sustaining Evidence-Based

Practices

Effective Interventions for Promoting Social Development:

Implementing and Sustaining Evidence-Based

Practices

TACSEI Management TeamTACSEI Management Team

Lise Fox, University of South Florida

Glen Dunlap,University of South Florida

Barbara J. Smith, University of Colorado- Denver

Phil Strain, University of Colorado-Denver

TACSEI Center FacultyTACSEI Center Faculty

Judith CartaUniversity of Kansas

Diane PowellUniversity of South Florida

Karen BlaseNational Implementation

Research Network

Roxane KaufmannGeorgetown University

Mary Louise HemmeterVanderbilt University

Jill Giacomini University of Colorado Denver

TACSEI MissionTACSEI Mission

To identify, disseminate and promote the implementation of evidence-based practices

in order to improve the social, emotional, and

behavioral functioning of young children with or at risk

for delays or disabilities.

Part C and 619 Child Outcomes

Part C and 619 Child Outcomes

Percent of children who demonstrate improved:Positive social emotional skills (including

positive social relationships)Acquisition and use of knowledge and

skills (including early language/ communication [and early literacy])

Use of appropriate behaviors to meet their needs

TACSEI FocusTACSEI Focus

Unified message

Collaboration

Models of effective practice

Support states

Research SynthesesResearch Syntheses

Multiple Topics

Summaries of syntheses will be prepared for dissemination to multiple audiences in a variety of formats

Providing Evidence-Based Models

Providing Evidence-Based Models

Implementation and evaluation of the

use of the Pyramid Model with children

with or at risk for delays or disabilities

Analysis of implementation and

sustainability factors

Implementation guidance

Support for selected states in the

implementation, sustainability, and

scale-up of models

Intensive TA to StatesIntensive TA to States

In years 2010-2012, six states will be selected through an application process based on “readiness” criteria

Technical Assistance ModelTechnical Assistance Model

Universal Outreach

Targeted TA

Intensive TA

Web-based Technical Assistance

Web-based Technical Assistance

Resources

Communities of Practice

Interactive opportunities

State Technical Assistance pages

Consultant BankConsultant Bank

Bank of consultants organized by

expertise

Web-based process of matching

request to consultant

Accountability procedures

TACSEI Partnerships TACSEI Partnerships

Goal of Partnerships

To collaborate with national organizations to extend the reach and impact of TACSEI

Primary Partners Primary Partners

Social CompetenceSocial Competence

“Emotional well-being and social competence provide a strong foundation for emerging cognitive abilities, and together they are the bricks and mortar that comprise the foundation of human development.”

(National Scientific Council on the Developing Child, 2007)

Pyramid ModelPyramid Model

UniversalPromotion

SecondaryPrevention

TertiaryIntervention

Nurturing and Responsive Relationships

Nurturing and Responsive Relationships

Foundation of the pyramidEssential to healthy social developmentIncludes relationships with children, families

and team members

High Quality EnvironmentsHigh Quality Environments

Inclusive early care and education environments

Supportive home environments

Supportive Home Environments

Supportive Home Environments

Supporting families and other caregivers to promote development within natural routines and community settings

Providing families and other caregivers with information, support, and new skills

Targeted Social Emotional Supports

Targeted Social Emotional Supports

Explicit instruction and support

Self-regulation, expressing and understanding emotions, developing social relationships

Individualized Intensive Interventions

Individualized Intensive Interventions

Family-centered, comprehensive interventions

Assessment-based

Skill-building

Pyramid ModelPyramid Model

UniversalPromotion

SecondaryPrevention

TertiaryIntervention

Effective WorkforceEffective Workforce

Training and technical assistanceCoaching of teachers

Ongoing professional development

Fidelity of implementation

The Teaching Pyramid: Program-Wide PBS

The Teaching Pyramid: Program-Wide PBS

Program-Wide Commitment

Teacher Training and Technical

Assistance

Data-Based Decision Making

Well-Defined Procedures

Administrative Support

Partnerships with Families

High Quality Supportive Environments

Nurturing and Responsive Caregiving Nurturing and Responsive Caregiving RelationshipsRelationships

Targeted Social Emotional Supports

Intensive Intensive InterventionsInterventions

Evaluation PlanEvaluation Plan

• Implementation• Benchmarks of Quality• Teaching Pyramid Observation Tool

• Program• Program Incidents (calls to families, dismissals,

transfer, requests for assistance, family conferences)• Behavior Incidents

• Child• Social Skills Rating System or other measure (social

skills; problem behavior)

UDMO: Benchmarks of Quality

0

0.5

1

1.5

2

2.5

3

Esta

bli

sh

lead

ers

hip

team

Sta

ff B

uy-I

n

Fam

ily

Invo

lvem

en

t

Pro

gra

m-w

ide

exp

ecta

tio

ns

Str

ate

gie

s f

or

teach

ing

exp

ecta

tio

ns

Cla

ssro

om

s

dem

on

str

ate

ad

op

tio

n o

f

“T

each

ing

Pyra

mid

”

Pro

ced

ure

s f

or

resp

on

din

g

to c

hall

en

gin

g

beh

avio

r

Sta

ff S

up

po

rt

Pla

n

Mo

nit

ori

ng

imp

lem

en

tati

on

& o

utc

om

es

Av

era

ge

sc

ore

Fidelity of Program-Wide Implementation

The Teaching Pyramid Observation Tool (TPOT)The Teaching Pyramid

Observation Tool (TPOT)

The TPOT was developed to measure the extent to which Teaching Pyramid practices are being implemented in a classroom

Provides information that can be used to:Describe “quality” of implementation of TPOT

practices

Compare implementation within and across teachers/classrooms

Identify needs of teachers for training and support

0

1

2

3

4

5

6

7

Env(#1-7)

#8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 RedFlags

Mallory's Pre-K Pre/Post TPOT Graphs

Series1

Series2

Key1-7. Environment 15. Social Skills8. Schedules 16. Express Emotions9. Transitions 17. Problem Solving10. Conversations 18. Friendship Skills11. Promot Engagement 19. Persistent Problem Behavior12. Behavior Expectations 20. Communication w / Families13. Providing Direction 21. Involving Families14. Problem Behavior 22. Relationships w / Adults 23 – 38. Red Flags

PRE

POST

ENV = 7/ 5

Anchor= .733 / 1.0667

Red Flags = 4 / 6

UDMO TPOT Scores: All

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Buena Vista Spirit Lake 1 Spirit Lake 2

Avera

ge s

co

re

NovemberApril

Overall Classroom Implementation Fidelity

Example:Program Incident Report

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Calls to families

Dismissal from program

Transfer to another program

Requests for assistance(External)

Family Conference

Indicator

Inci

den

ts p

er m

on

thDec

Jan

Feb

Mar

How Often is the Behavior?How Often is the Behavior?

What Behaviors?What Behaviors?

Who is having problem behavior?Who is having problem behavior?

Where is Problem Behavior?Where is Problem Behavior?

When is problem behavior occurring?When is problem behavior occurring?

Overall SS-SS IA0713

October 2007

0

20

40

60

80

100

120

140

1HDC 3HRD 8LJK 10MSW 14JV 16BN 22MC 26EU

1HDC

2AJCC

3HRD

4MJK

8LJK

9BJW

10MSW

13NCP

14JV

15DR

16BN

19NM

22MC

23GS

26EU

Overall PB-SS IA0713

October 2007

0

20

40

60

80

100

120

140

160

1HDC 3HRD 8LJK 10MSW 14JV 16BN 22MC 26EU

1HDC

2AJCC

3HRD

4MJK

8LJK

9BJW

10MSW

13NCP

14JV

15DR

16BN

19NM

22MC

23GS

26EU

Benchmarks of Quality by Cohort

78.8

88.4

94.4

101.5

60.67

88.89

0

20

40

60

80

100

120

Fall Spring

Av

era

ge

sc

ore

Cohort 1/Yr1

Cohort 1/Yr 2

Cohort 2/Yr 1

TPOT Scores across Program Years

2.65

3.353.43

4.14

2.51

4.43

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Fall Spring

Av

era

ge

sc

ore

Cohort1/YR1

Cohort1/YR2

Cohort2/YR1

Program Incidents by Cohort

0

1

2

3

4

5

6

Calls to families

Dismissal from program

Transfer to another program

Requests for assistance (External)

Family Conference

Type of Incident

Av

era

ge

Cohort 1/Yr 1

Cohort 1/Yr 2

Cohort 2/Yr1

Cohort 1 Behavior Incidents

0

5

10

15

20

25

30

35

40

45

50

1 2 3 4 5 6 7 8

Months

Av

era

ge

Nu

mb

er

of

Inc

ide

nts

2007

2008

SSRS Scores Across Programs

94.10

100.26

100.8101.2

90

92

94

96

98

100

102

Social Skills Problem Behavior

Av

era

ge

sc

ore

Fall

Spring

Lessons Learned Thus Far

State-wide Pyramid Implementation Initiatives

State-wide Pyramid Implementation Initiatives

State ImplementationState Implementation

CSEFEL State PartnershipsCO, MD, IA, NE, NC, VT, HI, TN

TACSEI State Partnerships (to be selected in 2009 through an application process)

Implementation Purpose and Model

Implementation Purpose and Model

Purpose: PD system to support adoption and sustainability of Pyramid Model and practices

Model of state-wide implementation based on literature and experience re: how to:

Disseminate info on what works to program level staff and families

Support adoption of what worksSupport sustained use over time of what works

Implementation StrategiesImplementation Strategies

Literature and experience indicates the following strategies:

State level collaborative planning and support

State-wide trainers

Community/program level coaches

Demonstration sites

State Level Planning and Support

State Level Planning and Support

State level collaborative planning teamPlanful group decision making processes that

promote shared ownership and limits feelings of “winners and losers”

Written, shared: vision, language, agendas, meeting summaries, action plans

Shared decision makingGround rules for conducting meetings and

decision making See www.vanderbilt.edu/csefel for some examples

Trainers and CoachesTrainers and Coaches

State wide Pyramid Model TrainersTrained to train on Pyramid and practicesTrained on effective training techniques

coaches

Pyramid Model CoachesTrained on Pyramid Model and practicesTrained on effective training practicesTrained on effective coaching strategies

Demonstration SitesDemonstration Sites

Demonstration sitesHigh quality setting, committed leadership,

enthusiastic about the Pyramid Model and being a demonstration site

Supported in implementation and sustainability of Pyramid Model and practices

Supported in data collection to ensure fidelity of implementation and positive outcomes

Supported as a site that stakeholders can visit to see the Pyramid in action!

State team (back to the)State team (back to the)

It’s work focuses on how state policies and resources will:

Train and support trainers and coachesSupport demonstration sitesEnsure that programs have access to information about the effectiveness of the Pyramid Model and practices as well as trainers, coaches and demonstration sitesEnsure that families have know about the effectiveness of the Pyramid Model and have access to programs and services that use the Model and practices

Lessons Learned So FarLessons Learned So Far

Programs and families like the Pyramid Model and practices

Programs and coaches can implement the model and practices to fidelity

Programs report improved outcomes

Lessons Learned So FarLessons Learned So Far

It takes time, time, time, timeTime for trainingTime for providers to be coachedTime for coaches to coachTime for program planning Time for data collectionTime for helping families implement the

practices

Time=Resources and Leadership!

Time=Resources and Leadership!

State resources Program resourcesState, local, community, program leadership for

the policies and funding to provide the time and support (e.g. job descriptions and compensation for coaches, job descriptions that provide the time and expectations of all service personnel to implement the practices and data collection, rewards and recognition for meeting the expectations, etc)

Some Sustainability Strategies

So Far

Some Sustainability Strategies

So Far

State agencies planning/reallocating resources together; writing a sustainability plan together

State web page on Pyramid info and opportunities

State Pyramid consultant bank (trainers, coaches, demo sites, etc

State agencies identifying resources and structures/agency(s)/program(s) to sustain effort beyond grant support, including supporting local efforts to implement and sustain

Contact TACSEIContact TACSEI

Mailing Address:University of South Florida Louis de la Parte Florida Mental Health InstituteDepartment of Child & Family Studies13301 N. Bruce. B. Downs Blvd, MHC2-1134Tampa, FL 33612-3807

Phone: (813) 974-9803