making behavior interventions happen - california · making behavior interventions happen real...

MakingBehaviorInterventionsHappenRealstrategiesforsupportingimplementation

AustinH.Johnson,PhD,BCBAAssistantProfessor,GraduateSchoolofEducationUniversityofCalifornia,Riverside

Aboutme

BA:UniversityofArizona

MA:UniversityofConnecticut

PhD:UniversityofConnecticut

BCBA&LicensedPsychologist

UCRiverside AssistantProfessor SchoolPsychologyprogram

Research Observationally‐basedbehaviorassessment Researchtopracticegapinschoolsettings Effectivepracticesinbehaviorintervention

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 1

Conflictofintereststatement

Dr.JohnsoniscurrentlyrevisingaGuilfordtextonschool‐basedbehaviorassessment,uponwhichhewillreceiveroyalties.

Whatisimplementation?

Weassumethatifwetellsomeonetodosomething,they’lldoit.Whywouldn’tthey?Becausebehaviorchangeisreallyreallyhard.

Butifpeopledon’tchange,theninterventionsdon’thappen,andpatients/clients/studentscannotbenefitfrominterventionsthattheydonotreceive.

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 2

Whatisimplementation?

HowmanypeoplemadeaNewYear’sresolutionthispastyear?

Howmanypeoplekeptit?

Thisisnotan“education”problem.

(Fleming,Nguyen,Afful,Carroll,&Woods,2018)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 3

Thisisnotan“education”problem.

Spreadofmostinfectionsinhospitalscanbestoppedwithonesimplepractice

Onaverage,healthcareworkerscleantheirhandslessthan50%ofthetimestheyshouldHospital‐basedinfectionswerealeadingcauseofdeathin2011,cost$40billion/year

(Gawande,2004;Rosenberg,2011;CDC,2018)

Thisisahumanproblem.

We’veknownitfornearlytwocenturies!1847:we’respreadingpuerperalfever,sowashyourhandsbeforedeliveringababy!

ByDay10,mostimplementersarenotdeliveringtheinterventionasintendedWhetherit’sacademicorbehavioral,individualorgroup,atschoolorinhomeorinthecommunity

Interventionnotdeliveredas

intended

Studentdoesnot“respond”tointervention

Moreintensive,morerestrictiveinterventionsorplacements

(Dufrene etal.,2012;Gilbertson,Willt,Singletary,VanDerHeyden,2007;Sundman‐Wheat,Bradley‐Klug,&Ogg,2012,Fallon,Collier‐Meek,Sanetti,Feinberg,&Kratochwill,2016;Jeffrey,McCurdy,

Ewing,&Polis,2009;Mouzakitus,Codding,Tryon,2015;Sanetti,Collier‐Meek,Long,Byron,

&Kratochwill,2015)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 4

So,whatdowedo?

2001:PeterPronovostCritical‐carespecialistatJohnsHopkinsFocusedonfixingoneproblem:infectionswheninsertingcatheterlinesFive‐stepchecklist

1. Washhandswithsoap2. Cleanpatient’sskinwithchlorhexidineantiseptic3. Putsteriledrapesoverentirepatient4. Wearsterilemask,hat,gown,andgloves5. Putsteriledressingovercathetersiteafterwards

(Gawande,2007)

So,whatdowedo?

Pronovost persuadesadministratorstoauthorizenursestostopdoctorsiftheysawanystepsskippedNeededtomakethisexplicit:“manynursesaren’tsurewhetherthisistheirplace,orwhetheragivenstepisworthaconfrontation.”

Lineinfectionrate:2001:11%2002:0%

Prevented43infections,8deaths,saved$2m

(Gawande,2007;Pronovost etal.,2006)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 5

So,whatdowedo?

Howdoyoumakebehaviorchangehappen?HelppeoplerememberwhattodoOperationalizeitintodiscretestepsPutpreventionandresponsestrategiesintoplacetomakesurethatitcanandwillhappen

(Gawande,2007)

There’snoone‐size‐fits‐allapproach

Buttheseare principleswecanuseregardlessofthe IntensityofthebehaviorStakesofdecisions

Ourjobistodecidehowsimple orhowelaboratethesesystemsshouldbeforthecasethatwe’reworkingwith

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 6

Ourplanfortoday

1. Designingausablebehaviorsupportplan

2. Buildingtreatmentfidelityassessmentintoyourbehaviorsupportplan

3. Workingwithimplementersbeforeitbreaksdown

4. Supportingimplementersafteritdoesbreakdown

1.DesigningausablebehaviorsupportplanAKA:“peopleusePinterestbecauseit’sprettyandit’sthere”

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 7

Peoplewillproblem‐solveontheirown

“IfoundthiscoolthingonPinterest…”

Somakeourplaneasytoreadanduse

WecannotexpectimplementerstoreadfullparagraphsorpagethroughBIPsEspeciallywhenrespondingtobehavior!

Practice#1– WritequickguidetoBIPusingO’Neilletal.’s(2015)format

Practice#2– CreateaquickguideBasedonsameprinciplesasPronovost’s checklistsMakethestepseasytoseeandunderstand

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 8

Practice#1– MaketheBIPreadable

Antecedent

Presentedwithmulti‐stepmathproblem

Behavior

Cries(tearsonface,loudyelling)

Consequence

Escapetask

SettingEvent

Fightonbus

Prevent

Strategy1Strategy2…

Replace

Strategy1Strategy2…

Respond

Strategy1Strategy2…

Prevent

Strategy1Strategy2…

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 9

Practice#2– Makethestepsvisible

CreateaQuickGuide(“cheatsheet”)!

TwosectionsAntecedentstrategies[whatshouldIbethinkingaboutallthetime?]Consequencestrategies[howIrespondwhenIseepositiveornegativebehavior?whatiftheyescalate?]

(Mostly)Antecedentstrategies

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 10

Consequencestrategiesandcrisisplan

Whatabouttoughkids?

Integratethecrisisplanintothequickguide

Considerintegratingdatacollectionaswell!

Lesspaperwork=moretimesupportingthekid

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 11

2.BuildingtreatmentfidelityassessmentintobehaviorsupportAKA:“yeah,it’saboutcollectingevenmoredata,butIpromiseit’sworthit!”

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 12

Seriously,whydothis?

(Burns & Gibbons, 2008)

1. Problem Identification

2. Problem Analysis

3. Plan Development4. Plan Implementation

5. Plan Evaluation

ReviseModify

IntensifyWith Expanding Support

Seriously,whydothis?

(Burns & Gibbons, 2008)

1. Problem Identification

2. Problem Analysis

3. Plan Development4. Plan Implementation

5. Plan Evaluation

ReviseModify

IntensifyWith Expanding Support

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 13

Studentoutcom

es

ImprovingContinuecurrentpractice(orconsiderfading)

Worsening Changeintervention

(Collier‐Meek,Fallon,Sanetti,&Maggin,2013)

“Awesome!"

Studentoutcom

es

ImprovingContinuecurrentpractice(orconsiderfading)

Worsening Changeintervention

(Collier‐Meek,Fallon,Sanetti,&Maggin,2013)

“Well,wetried,andit’snotworking.It’stimetostarttalkingabout

[insertmorerestrictiveenvironmenthere].”

Butwereweactuallydoingwhatwesaidwewoulddo?Asoftenaswesaidwe’ddoit?Withsufficientquality?

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 14

Implementation

Atorabovecriterion Belowcriterion

Studentoutcom

es

ImprovingContinuecurrentpractice(orconsiderfading)

Determineunknownissue

Worsening ChangeinterventionProvideperformance

feedback

(Collier‐Meek,Fallon,Sanetti,&Maggin,2013)

Consultationorevaluation?

“Let’stakedataonthestudent!”=

“Let’stakedataontheteacher/para/aide!”=

Butwait!It’sjustconsultationskills

It’sthedifferencebetween “Here’stheplan,seeyouintwoweeks.” “Iwanttomakesurethisplanworksforyou.So,here’sachecklistyoucanusetotrackwhatpartsyou’reabletodo,andwhatpartsaren’tworkingforyouandyourclassroom.Let’smeetbackuptomorrowandseewhatpartsworkedandwhatpartsdidn’t.”

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 15

Whatshouldwemeasure?

TreatmentintegrityAdherence:whetherstepsweredeliveredQuality:howwell eachstepwasdeliveredExposure:howmuchoforhowlong thetreatmentwasprovided (andProgramDifferentiation…)

Complexityshouldbedrivenbycaseintensity!Adherenceisstillthemaingameintown

Quality

Exposure

Adherence

(Schulte,Easton,&Parker,2009)

• Interventionfidelity1. Adherence:whether stepwasdelivered2. Quality:howwell thestepwasdelivered3. Exposure:howlong thestepwasdeliveredfor

https://cbitsprogram.org/forms

☐ Adherence☐ Quality☐ Exposure

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 16

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 17

n/a (great day!)

2/8/18

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 18

3.WorkingwithimplementersbeforeitbreaksdownAKA:“anounceofpreventionisworthapoundofaspirin”

Weneedtoberealistic

Interventionsareadapted inpractice

Interventionsthathavecontextualfitaremorelikelytobeimplemented

Adaptedinterventionscanstillwork!

So,planfortheinevitableandimportant!

“We can thus say now with confidence that some measure ofadaptation is inevitable and that for curriculum developersto oppose it categorically, even for the best of conceptual orempirical reasons,would appear to be futile.”

(Durlak &DuPre,2008)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 19

Implementationplanning

Planhasbeendrafted(theBIPexists)

Ideally,planwasdraftedwithabundantstakeholderinputwithaneyetowardsimplementation

Butwhatifit’sforasecondarystudent?Lotsofteachersandeducatorswhowedidn’ttalkto

So,nowweneedtofigureoutWhatmodificationsneedtobemadetotheplanWhatresourceswe’llneed,andifwecangetthemWhatbarrierstoimplementationexistHowwe’lladdressthosebarriers

ActionandCopingPlanning

1. Whatmodificationsneedtobemadetotheplan2. Whatresourceswe’llneed,andifwecangetthem

1. Whatbarrierstoimplementationexist2. Howwe’lladdressthosebarriers

BasedonresultsfromPRIMEProject

MultiplestudiestosupporteffectivenessofthisframeworkWithparentsandteachers Foracademicandbehavioralinterventions

(Fallon,Collier‐Meek,Sanetti,Feinberg,&Kratochwill,2016;Sanetti &Collier‐Meek,2015;Sanetti,Collier‐Meek,Long,Byron,&Kratochwill,2015;Sanetti,Collier‐Meek,Long,Kim,&Kratochwill,2014;Sanetti,Williamson,Long,&Kratochwill,2017)

ACTIONPLAN

COPINGPLAN

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 20

Howdoyoudothis?

Withameeting!(…hooray…)SitdownwiththestakeholdersActionPlanTaskanalyzeinterventionintodiscretesteps IdentifyresourcesneededforeachstepCopingplanDiscussforeseeablebarrierstoimplementationDiscussstrategiestosupportimplementation

Monitoroutcomes,modify,check‐in,discuss,andreviseonanongoingbasis

4.Supportingimplementersafteritdoesbreakdown

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 21

Welp,it’snothappening

Implementation

Atorabovecriterion Belowcriterion

Studentoutcom

es

ImprovingContinuecurrentpractice(orconsiderfading)

Determineunknownissue

Worsening ChangeinterventionProvideperformance

feedback

(Collier‐Meek,Fallon,Sanetti,&Maggin,2013)

Welp,it’snothappening

Implementation

Atorabovecriterion Belowcriterion

Studentoutcom

es

ImprovingContinuecurrentpractice(orconsiderfading)

Determineunknownissue

Worsening ChangeinterventionProvideperformance

feedback

(Collier‐Meek,Fallon,Sanetti,&Maggin,2013)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 22

Performancefeedback

Somanystudiesexaminingitseffectontreatmentintegrityandstudentoutcomes

MeetsWhatWorksClearinghousestandardsforanevidence‐basedpractice!

Whenitcomestoevidence‐basedimplementationsupportstrategies,performancefeedbackisthemostwell‐researched

(Fallon,Collier‐Meek,Maggin,Sanetti,&Johnson,2015)

Whatisit?

“Informationthatisprovidedtoanindividualorgroupaboutthequantityorqualityoftheirbehaviorthatprovidesinformationabouthowwelltheyaredoing” “capturing,summarizing,andpresentingdatatoteachersregardingtheirplanimplementation”

(Noell &Gansle,2014)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 23

Performancefeedback

ShowingdatatoapersonandreviewingitwiththemcanbeeffectiveonitsownCanembedwithothertrainingmethods

LogisticsBriefmeetings(5to15min)Ongoing (daily,weekly,as‐needed)Needgraphs ofdata

(Noell &Gansle,2014;Fallonetal.,2016)

Doingit

1. Ask implementerabout Interventionfidelity Studentoutcomes

2. Reviewfidelity datawithgraphs

3. Reviewoutcome dataandgoalprogress

4. Reviewandpractice stepsasnecessary

5. Followup!

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 24

OfthosestudiesthatfoundPFtobeeffective…

Typeoffeedback 97%providedverbalfeedback 69%providedgraphicTIfeedback 69%reviewedstudentdata 62%includedproblem‐solvingtosupportimplementation Less‐oftenusedweregoal‐setting(17%),reinforcementproceduresforadult(17.5%)

Frequencyoffeedback Variedgreatly(28%daily,21%weekly,7%onlywhenTIfellbelowacriterion)

Whenfeedbackwasprovidedrelativetointervention Sameday(21%),oneortwodaysafter(24%)

Studentcharacteristics 73%werewithkidsreceivingspedservices,27%withgened only

Superiortoothermethods

Resultsofrandomizedtrial

Consultationfollow‐upsBriefweeklyinterviewWeeklyinterviewwithcommitmenttoimplementPerformancefeedbackMeteverydayuntilat100%integrity,theneveryotherday,thenweekly

(Noell etal.,2005)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 25

That’ssomuch

Yes,butrealistically:“Theliteraturesuggeststhatconsultingwithteachers,designing aninterventiontheyseemacceptable,providingmaterialsnecessaryfortheintervention,discussing theimportanceofinterventionimplementation,andtrainingteacherstoimplementtheinterventionarefrequentlyinsufficienttosupportlong‐term[implementation]”

(Noell &Gansle,2014)

Howtodoit?

Canoccurinthecontextof:One‐on‐oneconsultationPros:EasierforyoutoinfluencehowthisgoesCons:Dependingontherelationship,mightbedifficulttodoPFeffectively

EmbeddedintoaProblem‐SolvingTeamPros:Lessduplicationofresources,mayameliorateconcernsaboutevaluativerolesCons:Mayexacerbatetensionsindatacollection(havetoreportoutonmydatatoallthesepeople…)

(Duhon,Mesmer,Gregerson,&Witt,2009)

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 26

Wrappingup

Supportingimplementation

1. Designthematerialstobeused,nottobefiled “Writeitlikeyouwantsomeonetoreadit”

2. Takeimplementationdata,butmakesurethedataworkforyouandthecase “WhatquestionamItryingtoanswer?”

3. Berealisticandplanforadaptations

4. Ifandwhenitfails,usethosedatayou’retakingtoimplementPF:asimple,evidence‐basedpracticeforsupportingimplementation

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 27

Wrappingup

Doesn’tseemlikemuch?Prettystraight‐forwardstuff?

Iagree!That’swhyit’sgreat!

Thisstuffisnotcomplex.

Butit’sevidence‐based!

Andittakestimeandeffort.

Wrappingup

“Thesestepsareno‐brainers;theyhavebeenknownandtaughtforyears.Soitseemedsillytomakeachecklistjustforthem.Still,Pronovost askedthenursesinhisI.C.U.toobservethedoctorsforamonthastheyputlinesintopatients,andrecordhowoftentheycompletedeachstep.Inmorethanathirdofpatients,theyskippedatleastone.” (Gawande,2007)

Interventionnotdeliveredas

intended

Studentdoesnot“respond”tointervention

Moreintensive,morerestrictiveinterventionsorplacements

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 28

Nextsteps

Researchersneedtogivepractitionersthetoolstheyneed toimplementthepracticesthattheystudy

Thankyou!

Wanttopartnerondevelopingusable,evidence‐basedbehaviorinterventionmaterials?Otherideas?Contactme!

Email:austin.johnson@ucr.edu

Phone:951‐827‐5958

Twitter:@johnsonaustinh

Web:www.ahjohnson.com

PENT Forum 2019 www.pent.ca.gov

Section 2: Strategies HO Page 29