fy 17–18 medi-cal specialty mental health external … pip... · the write-up does not contain a...

1

FY17–18MEDI-CALSPECIALTYMENTALHEALTHEXTERNALQUALITYREVIEW

PERFORMANCEIMPROVEMENTPROJECTS

BehavioralHealthConcepts,Inc.

5901ChristieAvenue,Suite502

Emeryville,CA94608

[email protected]

www.caleqro.com

855-385-3776

Preparedfor:

CaliforniaDepartmentofHealthCareServices(DHCS)

ForSiteVisitsConductedDuring:

April-June2018

2

TABLE OF CONTENTS

INTRODUCTION.......................................................................................................................................................................3

Table1.MHPsReviewedDuringApril–June2018.....................................................................................................................4

PERFORMANCEIMPROVEMENTPROJECTVALIDATION...........................................................................................5

Table2.PIPsSubmissionStandard.....................................................................................................................................................5Table3.PIPStatusDefined.....................................................................................................................................................................6Figure1.PIPSubmissionRates.............................................................................................................................................................7Table4.PIPTopicsforActiveandCompletedPIPSubmissions.............................................................................................8Table5.PIPTopicsforConceptOnlyPIPSubmissions...............................................................................................................9

FINDINGS......................................................................................................................................................................................................9

AccesstoCare...............................................................................................................................................................................................9TimelinessofCare....................................................................................................................................................................................10QualityofCare...........................................................................................................................................................................................10OutcomesofCare.....................................................................................................................................................................................11

CALEQRORATINGOFSUBMITTEDPIPS........................................................................................................................................12

Table6.PIPRatingSteps......................................................................................................................................................................13Table7.PIPRatingsDefined...............................................................................................................................................................13Table8.AveragePIPRatingsbyMHPSize...................................................................................................................................14

HISTORYOFPIPSUBMISSIONSBYMHP.......................................................................................................................................14

Figure2.PIPSubmissionHistory(FY2014-15toFY2017-18)...........................................................................................15Table9.ClinicalPIPSubmissionsbySmallRuralMHPs.........................................................................................................15Table10.Non-ClinicalPIPSubmissionsbySmallRuralMHPs............................................................................................16Table11.ClinicalPIPSubmissionsbySmallMHPs...................................................................................................................16Table12.Non-ClinicalPIPSubmissionsbySmallMHPs.........................................................................................................16Table13.ClinicalPIPSubmissionsbyMediumMHPs..............................................................................................................16Table14.Non-ClinicalPIPSubmissionsbyMediumMHPs....................................................................................................17Table15.ClinicalPIPSubmissionsbyLargeMHPs...................................................................................................................17Table16.Non-ClinicalPIPSubmissionsbyLargeMHPs........................................................................................................17

CONCLUSIONS/RECOMMENDATIONS...........................................................................................................................17

PIPTOPICS....................................................................................................................................................................................................17

PIPDESIGN/IMPLEMENTATION................................................................................................................................................................17

AreasforImprovement..........................................................................................................................................................................17RecommendationstoMHPs.................................................................................................................................................................18TechnicalAssistancetoMHPs............................................................................................................................................................19

APPENDICES..........................................................................................................................................................................20

CLINICALPIPTOPICSSUBMITTED.............................................................................................................................................................21

TimelinessofCarePIPs..........................................................................................................................................................................21QualityofCarePIPs.................................................................................................................................................................................23OutcomesofCarePIPs...........................................................................................................................................................................25

NON-CLINICALPIPTOPICSSUBMITTED....................................................................................................................................................31

AccesstoCarePIPs..................................................................................................................................................................................31TimelinessofCarePIPs..........................................................................................................................................................................35QualityofCarePIPs.................................................................................................................................................................................38OutcomesofCarePIPs...........................................................................................................................................................................39

CONCEPTONLY,NOTYETACTIVEPIPTOPICSSUBMITTED..................................................................................................................40

3

TimelinessofCarePIPs..........................................................................................................................................................................40QualityofCarePIPs.................................................................................................................................................................................42OutcomesofCarePIPs...........................................................................................................................................................................43

SUBMISSIONDETERMINEDNOTTOBEAPIP.........................................................................................................................................45

PERFORMANCEIMPROVEMENTPROJECT(PIP)VALIDATIONWORKSHEET.............................................................47

CalEQRO PIP Summary Report Q4 April - June 2018

3

INTRODUCTION

TheUnitedStatesDepartmentofHealthandHumanServices(DHHS),CentersforMedicareand

MedicaidServices(CMS)requiresanannual,independentexternalevaluationofStateMedicaid

ManagedCareprogramsbyanExternalQualityReviewOrganization(EQRO).ExternalQuality

Review(EQR)istheanalysisandevaluationbyanapprovedEQROofaggregateinformationon

quality,timeliness,andaccesstohealthcareservicesfurnishedbyPrepaidInpatientHealthPlans

(PIHPs)andtheircontractorstorecipientsofmanagedcareservices.CountyMentalHealthPlans

(MHPs)areconsideredPIHPsandarethereforesubjecttorulesgoverningPIHPs.CMSrules(42

CFR§438;MedicaidProgram,ExternalQualityReviewofMedicaidManagedCareOrganizations)

specifytherequirementsforevaluationofMedicaidManagedCareprograms.Theserulesrequire

anon-siterevieworadeskreviewofeachCountyMHP.

TheCaliforniaDepartmentofHealthCareServices(DHCS)contractswith56countyMHPsto

provideMedi-CalcoveredSpecialtyMentalHealthServices(SMHS)toMedi-Calbeneficiariesunder

theprovisionsofTitleXIXofthefederalSocialSecurityAct.

APerformanceImprovementProject(PIP)isdefinedbyCMSas“aprojectdesignedtoassessand

improveprocesses,andoutcomesofcarethatisdesigned,conductedandreportedina

methodologicallysoundmanner.”EachPIPisdesignedtoproducebeneficiary-focusedoutcomes.

TheValidatingPerformanceImprovementProjectsProtocol1specifiesthattheEQROvalidatetwoPIPsateachMHPthathavebeeninitiated,areunderway,orwerecompletedduringthereporting

year,orsomecombinationofthesethreestages.DHCSelectedtoexamineprojectsthatwere

underwayatsometimeduringthetwelvemonthsprecedingtheon-sitereview.

ThisreportpresentsasummaryofthePIPfindingsoftheon-sitereviewsconductedbythe

CaliforniaExternalQualityReviewOrganization(CalEQRO),BehavioralHealthConcepts,Inc.(BHC).

Thesummarycontainedinthisreportpertainstothereviewsthatwereconductedduringthe

fourthquarterofthe2017-18DHCSfiscalyear(FY)(April-June).Thisreportprovidessummary

informationtoDHCS,MHPs,andotherstakeholdersregardingthecompletenessofthePIP

submissionsreceivedbyCalEQROduringthequarter.EachPIPsubmissionissummarizedatthe

endofthereport.AnyfurtherinformationaboutaspecificPIPmaybeobtainedbyreviewingthat

MHP’sAnnualReport.

1DepartmentofHealthandHumanServices.CentersforMedicareandMedicaidServices(2012).ValidatingPerformanceImprovement

Projects:MandatoryProtocolforExternalQualityReview(EQR),Protocol3,Version2.0,December2012.Washington,DC:Author.


4

ThissummaryreportincludesdatathatwasanalyzedandaggregatedbyCalEQROfromtheEQR

activitydescribedbelow:

VALIDATINGPERFORMANCEIMPROVEMENTPROJECTS

EachMHPisrequiredtoconducttwoPIPsduringthe12monthsprecedingthereview.ThesePIPs

mustbesubmittedtoCalEQROforreviewandscoringinaccordancewithaValidationTool

developedbyBHC(seeAppendixB).ThisValidationToolwascreatedbyCalEQROtoincludeall

requiredelementsofreviewfromtherelevantCMSProtocol.2

ThepurposeofaPIPistoassessandimprovetheprocessesandoutcomesofhealthcareprovided

byaMHP.

ThefollowingMHPssubmittedPIPsthatwerereviewedandscoredduringon-sitereviews

conductedbyCalEQROduringthemonthsofApriltoJune2018.TheresultsoftheseMHPreviews

aredescribedinthisreport.

Table 1. MHPs Reviewed During April – June 2018

Del Norte Inyo Lassen Modoc

Mono Napa Plumas Riverside

San Bernardino Santa Barbara Santa Cruz Siskiyou

Trinity Ventura

2Ibid.


5

PERFORMANCE IMPROVEMENT PROJECT VALIDATION

ThefollowingtableillustratesthenumberofPIPsthatweresubmittedforvalidationthroughthe

CalEQROreviewbyeachMHPreviewedinApril-June2018.

Table 2. PIPs Submission Standard

MHP MHPSize

NumberofClinicalPIPs

Submitted

StatusofClinicalPIPsas

determinedbyCalEQRO

NumberofNon-Clinical

PIPsSubmitted

StatusofNon-ClinicalPIPsasdeterminedby

CalEQRO

Del Norte Small Rural 1 Active and Ongoing 1 Completed

Inyo Small Rural 0 Submission

Determined Not to be a PIP

1 Active and Ongoing

Lassen Small Rural 1 Completed 1 Active and Ongoing

Modoc Small Rural 1 Concept Only, Not Yet Active 1 Concept Only, Not

Yet Active

Mono Small Rural 1 Active and Ongoing 0 Submission


Napa Small 1 Active and Ongoing 0 No PIP Submitted

Plumas Small Rural 0 No PIP Submitted 0 No PIP Submitted

Riverside Large 1 Active and Ongoing 1 Active and Ongoing

San Bernardino Large 1 Completed 1 Completed

Santa Barbara Medium 1 Completed 1 Completed

Santa Cruz Medium 1 Completed 1 Active and Ongoing

Siskiyou Small Rural 1 Concept Only, Not Yet Active 1 Concept Only, Not

Yet Active

Trinity Small Rural 1 Concept Only, Not Yet Active 1 Active and Ongoing

Ventura Large 1 Completed 1 Completed


6

Table 3. PIP Status Defined

ActiveandOngoing

Baselineestablishedonatleastsomeoftheindicators,andatleast

someinterventionshavestarted.Anycombinationoftheseis

acceptable.

CompletedInthepast12monthsorsincethepriorEQRtheworkonthePIPhas

beencompleted.

ConceptOnly,NotYet

Active

Baselinemayhavebeenestablished,butinterventionshavenot

started.ThisisNOTanactivePIP.

Inactive,Developedina

PriorYear

Ratedlastyearandnotratedthisyear.MHPhasdoneworkonit,but

ithasnotyetstarted,orithasbeensuspendedforsomereason.This

isNOTanactivePIP.

SubmissionDetermined

NottobeaPIP

Thewrite-updoesnotcontainaplan,data,and/orindicationwhere

datawillcomefrom.ThisisNOTanactivePIP.

Ofthe14MHPswhoseon-sitereviewswereconductedduringthemonthsofApril-June2018,8

areclassifiedasSmallRural,1isclassifiedasSmall,2areclassifiedasMedium,and3areclassified

asLarge.

Thirteenofthe14MHPsincludedinthisquarter’sreviewsubmittedsomeinformationtobe

consideredforvalidationofPIPs,however,only7MHPs(50percent)metthesubmissionstandard

thatrequiressubmissionoftwoactiveorcompletedPIPs.Oftheremaining7MHPs:

• MonoandNapaMHPsmettherequirementforsubmissionofanactiveorcompletedclinical

PIP,butdidnotsubmitanactiveorcompletednon-clinicalPIP;

• InyoandTrinityMHPsmettherequirementforsubmissionofanactiveorcompletednon-

clinicalPIP,butdidnotsubmitanactiveorcompletedclinicalPIP;

• Modoc,PlumasandSiskiyouMHPsdidnotsubmitanyactiveorcompletedPIPs.

Modoc,Siskiyou,andTrinityMHPssubmitteddocumentationforPIPsforwhichinterventionshad

notbeeninitiatedatthetimeoftheon-sitereview;thesePIPsareclassifiedasConceptOnly,Not

YetActive.InyoandMonosubmitteddocumentationthatwasdeterminednottoconstituteaPIP.

Additionally,Napadidnotsubmitanyinformationforanon-clinicalPIPandPlumasdidnotsubmit

anyinformationforeitheraclinicalornon-clinicalPIP.


7

Figure 1. PIP Submission Rates

SmallRuralMHPs• 7of16requiredPIPssubmittedmetsubmissionstandards

o 5PIPswereratedasActiveandOngoingo 5PIPswereratedasConceptOnly,NotYetActiveo 2PIPswereratedasCompleteo 2SubmissionswereratedasSubmissionDeterminedNottobeaPIPo NothingwassubmittedforoneClinicalPIPo NothingwassubmittedforoneNon-clinicalPIP

SmallMHPs

• 1of2requiredPIPssubmittedmetsubmissionstandards

o 1PIPwasratedasActiveandOngoing

o NothingwassubmittedforoneNon-clinicalPIPMediumMHPs

• 4of4requiredPIPssubmittedmetsubmissionstandards

o 1PIPwasratedasActiveandOngoing

o 3PIPswereratedasComplete

LargeMHPs• 6of6requiredPIPssubmittedmetsubmissionstandards

o 2PIPswereratedasActiveandOngoing

o 4PIPswereratedasComplete

7

1

4

6

16

2

4

6

0

2

4

6

8

10

12

14

16

SmallRural(43.75%) Small(50%) Medium(100%) Large(100%)

PIPsmeetingsubmissionstandard

PIPsMeetingSubmissionStandard

PIPsRequired


8

FivePIPsreceivedaratingof0percent.OneMHPdidnotsubmitanyinformationtobeconsidered

foreitheraclinicaloranon-clinicalPIP,andoneMHPdidnotsubmitanyinformationforanon-

clinicalPIP.TwoMHPs’submissionsweredeterminednottomeetthestandardsforaPIP.

Additionally,fiveConceptOnly,NotYetActivePIPswereratedfortechnicalassistance(TA)

purposesonly,andthoseratingswerenotfactoredintotheoverallratingsdescribedinthisreport

(seeTable8).

MHPsaddressedvarioustopicsandissuesinthePIPsthatweresubmittedforreview.Eighteen

PIPsratedasActiveandOngoingorCompleted,coveredtopicsthataddressthefollowingareas:

TimelinessofCare,AccesstoCare,QualityofCare,andOutcomesofCare.Asummaryofthe

informationprovidedtoCalEQROforallPIPsisprovidedattheendofthisreport.

Table 4. PIP Topics for Active and Completed PIP Submissions

PIPTopics

PIPTitles Clinical Non-Clinical

Access to Care

Psychiatry No-Show Study* Del Norte

Open-Access Scheduling and Kept Appointments Lassen Law Enforcement Co-located Triage, Engagement, and Support (TEST) Teams* San Bernardino

Beneficiary Acuity Index* Ventura

Timeliness of Care

Rapid Connect* Santa Cruz

Timeliness to Psychiatric Services* Santa Barbara

Timeliness of Access to Services Santa Cruz

Improving Timely Access to Services Trinity

Quality of Care

Improving Treatment: Training, Beneficiary Engagement and Team Based Care* Santa Barbara

Smoking Cessation* Ventura

Improving Engagement and Retention in Services Riverside

Outcomes of Care

Rehospitalization Rates Del Norte

Early Therapeutic Alliance & Retention* Lassen

Strengths Model Intervention for Employment Related-Goals Mono

Adult Social Engagement Napa

Follow-Up After Hospitalization Riverside

Complex Care Coordination* San Bernardino

Strengths Based Interventions Inyo

*CompletedPIPs


9

Table 5. PIP Topics for Concept Only PIP Submissions

PIPTopics

PIPTitles Clinical Non-Clinical

Timeliness of Care

Timeliness Plan Modoc

Timely Access for Children and Youth Siskiyou

Quality of Care

Improving Beneficiary Outcomes through integrated treatment of Co-Occurring Disorders Modoc

Outcomes of Care

Initial Engagement and Retention in Children's Services Siskiyou

Improving Anxiety Levels of Beneficiaries Diagnosed with an Anxiety Disorder Trinity

FINDINGS

ManyPIPsaddresssimilartopicsasMHPsarefacingthesameissues.Thefindingsfurtherillustrate

thispoint.ThefindingsalsopertaintoMHPs’operationofaneffectiveManagedCareOrganization,

suchasMHPs’processesforensuringaccesstoandtimelinessofservices,andprocessesfor

improvingthequalityofcare.ThedetailsbelowreflectonlythosePIPsratedasActiveandOngoing

orCompleted.FormoreinformationregardingthePIPsdetailedbelow,pleaseseeAppendixAof

thisreport.

Access to Care

Fournon-clinicalPIPsfocusedonimprovingaccesstocareforbeneficiaries.

• DelNorte’sandLassen’snon-clinicalPIPsfocusedonimprovingno-showrates.

• SanBernardino’snon-clinicalPIPfocusedonco-locatinginlawenforcementsites.

• Ventura’snon-clinicalPIPfocusedonensuringthatbeneficiariesweregettingthe

levelofcarenecessaryfortheirindividualneeds.

DelNorteincreasedtelepsychiatrytoimproveno-showrates;however,thereductioninno-

showrateswasminimal.Lassenimplementedopenaccesstoimproveno-showrates;however,

theinterventioncenteredonbeneficiarieswhokepttheirappointmentsratherthan

amelioratingthebarriersforbeneficiarieswhowerenotabletokeepappointments.

SanBernardinoco-locatedateamofMHPclinicalstaffwithlawenforcementstafftoprovidea

moreappropriateand(clinically)informedresponsetolawenforcementcallsthatinvolve

residentswhopresentwithmentalhealthconcerns.Theresultsdemonstratethatco-locating

teamsledtoareductionininvoluntarypsychiatricholdsatallfourpolicedepartmentsusedfor

thisproject.

Ventura’sPIPaimedtoensurethatbeneficiariesidentifiedasfittingintothecategoriesofhigh,

moderateorlowneedswerereceivingalevelofcarelikelytomeettheirserviceneeds.The


10

MHPtestedthisapproachwith12staffmembersandultimatelydecidedtoimplementit

throughouttheMHP.

Timeliness of Care

Oneclinicalandthreenon-clinicalPIPsfocusedonimprovingtimelinessofservicesfor

beneficiaries.

• SantaCruz’sclinicalPIPfocusedonensuringtimelyfollow-uptoservicesfor

individualsafterdischargefromtheCrisisStabilizationProgram(CSP).

• SantaBarbara’snon-clinicalPIPaimedatreducingthetimeittakesfornew

beneficiaries(adultandyouth)tohavetheirfirstappointmentwithapsychiatrist.

• SantaCruz’snon-clinicalPIPfocusedonimprovingthetimelinessfromfirstcontact

tofirstsession.

• Trinity’snon-clinicalPIPwasdesignedtoreducethenumberofdaysfromclaimed

assessmenttoclaimedappointment.

AnalysisoftheclinicalPIPfromSantaCruzfoundthatmanybeneficiariesfromtheCSPare

dischargedtoinpatienthospitalizationservices.TheEQROobservedthatCSPstendtohavethe

purposeofpreventingbeneficiariesfrominpatienthospitalization,andthatdoesnotseemtobe

workingwellinSantaCruz.

SantaCruz’snon-clinicalPIPhaddataanalysisissues,assomeclinicianswerenotrecordingthe

timeoffirstofferedappointment.Additionally,SantaBarbara’snon-clinicalPIPhadsix

interventionsandtheMHPfounditdifficulttomeasuretheeffectivenessofthemall.

AlthoughTrinity’sinterimanalysisrevealedatrendtowarddecreasedtimetoaccessservices,

inconsistenciesincollectingthedataoccurred.Staffwerenotconsistentindocumentingthe

timelines,whichimpededstandardizeddatacollection.

Quality of Care

TwoclinicalPIPsandonenon-clinicalPIPweredesignedtoimpactqualityofcare.

• SantaBarbaradevelopedandcompletedaclinicalPIPthatfocusedonimproving

beneficiaryexperienceoftreatment.

• VenturaimplementedaclinicalPIPthatfocusedonimprovingthehealthstatusof

beneficiarieswhousetobaccoproducts.TheMHPutilizedasmokingcessationprogram

titledCallItQuits(CIQ).

• Riversideimplementedanon-clinicalPIPwiththegoalofincreasingengagementand

retentionofchildrenincounty-operatedspecialtymentalhealthoutpatientclinics.

SantaBarbara’sclinicalPIPhadonlyoneclinicalintervention:Beneficiaryengagementintreatment

planning,theremaininginterventionswerenon-clinicalinnature.TheMHPdidnottrackallthe

interventionstheyimplementedaspartofthePIP,howeverthecombinedresultsindicated


11

improvementorstabilityintheChildandAdolescentNeedsandStrengths(CANS)andMilestones

ofRecoveryScores(MORS)scores.

Ventura’sclinicalPIP’sresultshaveseennochangeinoutcomesforbeneficiaries.RelianceonCIQ

sessionsalone,evenifembeddedinclinics,seemsunlikelytoproducechangeunlessspecific

elementsaretailoredtotheMHP’spopulation.

Riverside’snon-clinicalPIPliststwointerventions:CollaborativeAssessmentandevening

availabilityforservices.However,bothinterventionslackedsufficientdetailsinthePIPnarrative

anddidnotincludesteps/activitiesthatwillbetakentoimplementthem.Priortoselectingthese

interventions,theteamappearstohavemissedthestepofdeterminingbeneficiaries’reasonsfor

lackofengagementandsubsequentlydroppingoutoftreatment.

Outcomes of Care

SixclinicalPIPsandonenon-clinicalPIPweredesignedtoimpactoutcomesofcare.

• DelNorte,RiversideandSanBernardinofocusedclinicalPIPsonimproving

outcomesrelatedtohospitalizations.

• LassenfocusedaclinicalPIPonimprovingearlytherapeuticalliance.

• MonofocuseditsclinicalPIPonimprovingbeneficiaries’employmentgoals.

• NapadesignedaclinicalPIPtoimprovesocialengagementforitsbeneficiaries.

• Inyo’snon-clinicalPIPaimstoimplementaStrengths-Basedapproachthat

incorporatesaprocessofassessment,planning,clinicalcasereviewandsupervision

ofstaff,andsupportprovidedtobeneficiariestoachieveidentifiedlife-goals.

DelNorte’sclinicalPIPproposedtheuseofanassessmentbyadrugandalcoholcounselor

withinfivedaysofacutepsychiatrichospitalizationtoreducerehospitalizationrates.This

approachyieldedan11percentimprovementintheirrehospitalizationrate.ThegoalofthePIP

istoaffordbeneficiarieswithfollow-upservicesasquicklyaspossibleposthospitalizationto

preventthetraumatizingeffectsofrehospitalizations.

Riverside’sclinicalPIPhadthegoalofincreasingbeneficiaryengagementinandaccessto

timelyoutpatientserviceswithinsevendaysfollowinghospitaldischarge,withfocuson

unengagedbeneficiarieswhoarenotalreadyknownbyandopentotheoutpatientmental

healthsystem.

SanBernardino’sclinicalPIPtargetedbeneficiarieswithcomorbidsomaticconditionswhohave

higherfrequencyandlongerdurationofpsychiatrichospitalizationscomparedtotheMHP’s

generaladultpopulation.TheMHPprovidedcoordinatedcarethataddressedbothchronic

mentalandphysicalillnessestoreducetherisk,frequency,anddurationofpsychiatric

hospitalizationattwoprogramswithintheMHP.Theprojecthadmixedresultsinthetwo

programswhereitwasimplemented,bothprogramssawareductioninratesofpsychiatric

hospitalization,butoneprogramsawanincreaseinriskofhospitalization.

Lassen’sclinicalPIPiscompletedandachievedsuccess;however,theproblemwasoriginally

describedasatimelinessproblemwhenitwastrulyanengagementproblem–beneficiaries


12

droppingoutofservice.TheMHPdidnotconductabarrieranalysistodeterminecausesand

linkinterventionstodata.

AlthoughMonodescribedthevariousaspectsoftheStrengthsModel(SM)approach,includingthe

SMAssessment,PersonalRecoveryPlan,andgroupsupervisionofclinicalstaff,nospecific

interventionwasdescribedthatrelatedtothespecificactionsofstaffwithbeneficiaries,which

wouldseemtobeakeyelementofthismodel.

NapaexperiencedmanyissueswiththeimplementationandmeasurementoftheirclinicalPIP.Due

toseveralnaturaldisasters,includingafloodandwildfire,theMHPhadtopostponeseveralofthe

interventionactivities.Specifically,themeasurementofonekeyindicatorprovedtobedifficult,as

ittakesapproximatelysixmonthsfromitsadministrationtoreceivethedata.

Inyo’snon-clinicalPIPhasthepotentialtopositivelyaffectbeneficiaryoutcomesinareas

identifiedbythebeneficiary,includinghousing,employment,andeducation.However,theMHP

hasnotcompletelyimplementedthisPIPyetandoutcomemeasurementsarestillpending.

CALEQRO RATING OF SUBMITTED PIPS

ThetablebelowliststheValidationItemsthatareratedforeachPIPbyCalEQRO.AllPIPsarerated

basedontheircompletenessandcompliancewiththestandard,therefore,PIPsubmissionsthat

wereratedasConceptOnly,NotYetActive(anddidnotreceiveratingsforeachPIPstep)arenot

includedinthetabulations,figures,andtablesinthissection.AllPIPsreceivingaratingof0percent

(i.e.,PIPsratedas:SubmissionDeterminedNottobeaPIP,Inactive,andNoPIPSubmitted)arealso

notincludedinthetabulations,figures,andtablesinthissection.Assuch,thereare22PIPs

representedinthefiguresandratingstables.

ThestandardsarefoundintheCMSPIPProtocol:ValidationofPerformanceImprovement

Projects.3WithineachoftheninePIP“Steps”therearesubsectionsthatareratedaccordingtothe

PIPValidationTool(seeAppendixB).

32012DepartmentofHealthandHumanServices,CentersforMedicareandMedicaidServiceProtocol3Version2.0,December2012.

EQRProtocol3:ValidatingPerformanceImprovementProjects.


13

Table 6. PIP Rating Steps

Step PIPSection

1 Selected Study Topics

2 Study Question

3 Study Population

4 Study Indicators

5 Improvement Strategies

6 Data Collection Procedures

7 Analysis and Interpretation of Study Results

8 Review Assessment of PIP Outcomes

9 Validity of Improvement

AllPIPsub-sectionsreceivearatingofMet;PartiallyMet;NotMet;NotApplicable;orUnabletoDetermine.

Table 7. PIP Ratings Defined Met Credible, reliable, and valid methods for the item were documented.

Partially Met Credible, reliable, or valid methods were implied or able to be established for part of the item.

Not Met Errors in logic were noted or contradictory information was presented or interpreted erroneously.

Not Applicable Only to be used in Steps 7-9 when the study period was underway for the first year.

Unable to Determine

The study did not provide enough documentation to determine whether credible, reliable, and valid methods were employed.

AratingofMetorPartiallyMetweighspositivelyintotheOverallAverageRatingreceivedbythe

PIP.EachMetitemreceivestwopoints,whileeachPartiallyMetitemreceivesonepoint.The

OverallAverageRatingforeachPIPiscalculatedwiththefollowingformula:

("#$%&')&* × 2) + ("#$%&'/0'*10223)&*)"#$%&'456772180%2&9*&$: × 2

CalEQROusedtheformulareferencedabovetocalculatearatingforeachofthenineStepsinthe

PIPValidation.ThenanoverallratingwasgiventoeachPIPandthendividedbytheTotal

ApplicablePIPsteps.PIPsubmissionsthatwereratedasConceptOnly,NotYetActive,and

thereforedidnotreceiveratingsforeachPIPstep,arenotincludedinthetabulationsinthetables

inthissection.AllPIPsreceivingaratingof0percent(i.e.,PIPsratedas:SubmissionDetermined

NottobeaPIP,Inactive,andNoPIPSubmitted)arerepresentedinthedenominatorofthe

tabulations.


14

TheMHPsreviewedduringApriltoJune2018receivedthefollowingoverallratings:

Table 8. Average PIP Ratings by MHP Size

MHPSize

Clinical

Non-Clinical

Small Rural 70.45% 62.21%

Small 52.63% NA

Medium 77% 76%

Large 70.95% 67.65% TheaverageratingsreceivedbyMediumMHP’sclinicalPIPsarehigherthanthosereceivedbyall

otherMHPs.

• ThePIPratingsforMediumMHPswerehigherthantheratingsforallothersizedMHPs.

• Duetotheratingsbeinganaverage,thePIPratingsforSmallMHPsonlyreflectstheoneMHPof

thatsizethathadanon-sitereviewduringthisperiod.

HISTORY OF PIP SUBMISSIONS BY MHP

CalEQROhasbeenvalidatingPIPsubmissionsfromMedi-CalMHPssinceFY2014-15.CalEQROhas

providedsubject-basedTAon-site,viaemail,telephone,video,andwebinar.However,numerous

MHPshavesubmittedPIPsthatdidnotmeetthesubmissionstandardofhavingtwoActiveand

OngoingorCompletedPIPsforeachreviewyear.AlthoughtheseMHPsarecontractuallyrequired

tomeetthesubmissionstandards,theycontinuetocitestaffingissues,competingpriorities,and

limitedresourcesasreasonsfornotmeetingthisrequirement.

ThefigurebelowillustratesthesubmissionhistoryofeachMHPrepresentedinthisreport:


15

Figure 2. PIP Submission History (FY 2014-15 to FY 2017-18)

ThespecificsofthesubmissionsreceivedbytheMHPsrepresentedinthisreportareasfollows:

Table 9. Clinical PIP Submissions by Small Rural MHPs

MHPFY2014-15Clinical

FY2015-16Clinical

FY2016-17Clinical

FY2017-18Clinical

Del Norte Active and Ongoing Concept Only, Not Yet Active

Concept Only, Not Yet Active Active and Ongoing

Inyo No PIP Submitted No PIP Submitted Submission


Submission Determined Not to

be a PIP

Lassen No PIP Submitted No PIP Submitted Submission


Completed

Modoc Active and Ongoing Active and Ongoing Completed Concept Only, Not Yet Active

Mono Active and Ongoing Concept Only, Not Yet Active


Plumas No PIP submitted No PIP submitted Active and Ongoing No PIP Submitted

Siskiyou No PIP Submitted Active and Ongoing Concept Only, Not Yet Active

Concept Only, Not Yet Active

Trinity Active and Ongoing Concept Only, Not Yet Active



4

3

2

5

3

2

3

8 8

7

8

2

5

6

0

1

2

3

4

5

6

7

8

Del Norte Inyo

Lassen

ModocMono

Napa

Plumas

Riversi

de

San Bern

ardino

Santa

Barbara

Santa

Cruz

Siskiy

ouTri

nity

Ventura

Number of Active or Completed PIPs Submitted

PIPs Submitted

Number of PIPs required = 8


16

Table 10. Non-Clinical PIP Submissions by Small Rural MHPs

MHPFY2014-15Non-Clinical

FY2015-16Non-Clinical



Del Norte Active and Ongoing Submission Determined Not to be a PIP

Submission Determined Not to be

a PIP Completed

Inyo Active and Ongoing Submission Determined Not to be a PIP Completed Active and Ongoing

Lassen No PIP Submitted Concept Only, Not Yet Active


a PIP Active and Ongoing

Modoc Active and Ongoing Concept Only, Not Yet Active Active and Ongoing Concept Only, Not Yet

Active

Mono Active and Ongoing Submission Determined Not to be a PIP



a PIP

Plumas Active and Ongoing Completed No PIP Submitted No PIP Submitted

Siskiyou No PIP Submitted Active and Ongoing Submission



Trinity Active and Ongoing Active and Ongoing Active and Ongoing Active and Ongoing

Table 11. Clinical PIP Submissions by Small MHPs


FY2015-16Clinical

FY2016-17Clinical

FY2017-18Clinical

Napa No PIP submitted Concept Only, Not Yet Active


Table 12. Non-Clinical PIP Submissions by Small MHPs

MHP FY2014-15Non-Clinical




Napa Concept Only, Not Yet

Active Concept Only, Not Yet

Active Active and Ongoing No PIP Submitted

Table 13. Clinical PIP Submissions by Medium MHPs


FY2015-16Clinical

FY2016-17Clinical

FY2017-18Clinical

Santa Barbara Active and Ongoing Concept Only, Not Yet Active Active and Ongoing Completed

Santa Cruz Active and Ongoing Active and Ongoing Active and Ongoing Completed


17

Table 14. Non-Clinical PIP Submissions by Medium MHPs

MHPFY2014-15Non-Clinical




Santa Barbara Active and Ongoing Active and Ongoing Active and Ongoing Completed

Santa Cruz Active and Ongoing Active and Ongoing Completed Active and Ongoing

Table 15. Clinical PIP Submissions by Large MHPs


FY2015-16Clinical

FY2016-17Clinical

FY2017-18Clinical

Riverside Active and Ongoing Completed Active and Ongoing Active and Ongoing

San Bernardino Active and Ongoing Completed Active and Ongoing Completed

Ventura Active and Ongoing Completed Concept Only, Not Yet Active Completed

Table 16. Non-Clinical PIP Submissions by Large MHPs

MHP FY2014-15Non-Clinical




Riverside Active and Ongoing Completed Active and Ongoing Active and Ongoing

San Bernardino Active and Ongoing Completed Active and Ongoing Completed

Ventura Active and Ongoing Active and Ongoing Inactive, Developed in a Prior Year Completed

CONCLUSIONS/RECOMMENDATIONS

DuringtheFY2017-18annualreviews,CalEQROfoundstrengthsinMHPprogramsandpractices

thathaveasignificantimpactontheoveralldeliverysystemanditssupportingstructure.Inthose

sameareas,CalEQROalsonotedopportunitiesforqualityimprovement.

PIP TOPICS

CalEQROobservedthat7ofthe18(39percent)PIPsratedasActiveandOngoingorCompleted,

focusedonOutcomesofCareissues.

PIP DESIGN/IMPLEMENTATION

Areas for Improvement

• 47percentofthesubmissions(16of34)requestedbyCalEQROfortheApril-June2018on-site

reviewsdidnotmeettheActiveandOngoingorCompletedstandardasrequiredforPIP

submissions.


18

o FiveofthosesubmissionswereratedasConceptOnly,NotYetActiveandwerein

variousstagesofimplementation.Onceinterventionsareimplemented,thePIPswill

beconsideredActiveandOngoing.

o OneMHPdidnotsubmitdocumentationforoneoftherequiredPIPs.

o OneMHPdidnotsubmitdocumentationforbothrequiredPIPs.

o Twosubmissionsdidnotmeettheminimumrequirementsandwereratedas

SubmissionDeterminedNottobeaPIP.

• EightPIPswereratedasCompleted.BHCemphasizedtheneedforcontinuedPIPdevelopment.

MHPsshouldnotlimitthemselvestodevelopingnewPIPswhenpreviousonesarecompleted;they

shouldconsiderPIPdevelopmentfromacontinuousqualityimprovementprocessperspective.

• 41percentofthesubmissions(46of112)requestedbyCalEQROforeachofthepriorfourFYsdid

notmeettheActiveandOngoingorCompletedstandardasrequiredforPIPsubmissions.

o 21submissionswereratedasConceptOnly,NotYetActive.

o 14PIPswerenotsubmitted.

o 10submissionswereDeterminedNottobeaPIP.

o 1PIPwasratedasInactive,DevelopedinaPriorYear.

• ThreeMHPs(Lassen,Napa,andSiskiyou)submittedonlytwoPIPsthathavemettheActiveand

OngoingorCompletedstandardduringthepastfourreviewyears.

• ThreeMHPs(Inyo,Mono,andPlumas)submittedonlythreePIPsthathavemettheActiveand

OngoingorCompletedstandardduringthepastfourreviewyears.

Recommendations to MHPs

• CalEQROcontinuestorecommendthatMHPsfosteracultureofcontinuousquality

improvementthroughouttheirorganizations.

o PIPideasshouldbegeneratedfromongoingeffortstoimprovebeneficiary

outcomes,asMHPsshouldfocusonbeneficiaryoutcomesversusorganizational

improvements.

o PIPideasshouldcomefromanyareaoftheMHPthatdirectlyimpactsbeneficiaries.

o MHPsshouldconsiderareasinwhichtodevelopPIPsonacontinuousbasis.Ifan

issuethatrequiresimprovementhasapotentialimpactonbeneficiaryoutcomes,

theMHPshouldconsiderhowaPIPcouldbedevelopedtoimprovetheissue.

• MHPsshoulddevelopaplanandputitintoaction.ActiveandOngoingPIPsarethestandardby

whichtheMHPsareevaluated.

o PIPsshouldhavemechanismsforcollectingdataquarterly,ataminimum,and

shouldhavenewactivitiesoccurringonaregularbasis.

o TheCMSprotocolrequiresatleastonenewinterventioneveryyearifan

unsuccessfulPIPistocontinue.

• PIPsshouldinvolvebeneficiaryfeedbackasmuchaspossible;beneficiaries’inputcanbe

valuableindeterminingthedirectionandinterventionsofPIPs.

o MHPsshoulddevelopPIPteamsthatarespecifictotheissuestheyareaddressing,

includingsubjectmatterexpertsandbeneficiariesasappropriate.• MHPsshouldensurethattheyhaveasolidfoundationonwhichtodesignaPIP.Todoso

requiresbackgrounddataandanalysisofbarrierspriortotheimplementationofaPIP.


19

• ItisimperativethattheMHPsparticipateinTAfromCalEQROtoimprovetheirabilitytocollect,

analyze,andusedatathathelpestablishtheneedforaPIPanddevelopameasurablestudy

question.

o PIPClinicsareofferedtoallMHPsonaquarterlybasis.

o ResourcesareavailableontheCalEQROwebsite,including:

§ PIPinstructionalvideos4

§ PIPLibrary5

§ InstructionsforCompletingPIPValidationTool6

• MHPsshouldcontactCalEQROforassistanceindevelopingPIPs;TAisavailableforallMHPs

outsideoftheon-sitereview.

Technical Assistance to MHPs

CalEQROworkedindividuallywitheachMHPwhileonsitetoprovideTAinthedevelopmentand

progressionoftheirPIPs.AdditionalTAwasprovidedattherequestofMHPs.Phonesessionswere

conductedwithMHPspriortoandfollowingon-sitereviewsasrequested.Thesephonesessions

arespecificforeachMHPbutinclude:assistancewithdefiningaproblemwithlocaldata;aidin

writingaPIPstudyquestion;andhelpwithfindingappropriateinterventions,outcomesand

indicators.

CalEQROpresentedaPIPCliniconJune21,2018thataddressedtipsforsuccessfulPIPsand

reviewedcommontopicselection,indicatorsandinterventions.Questionandanswersessionswere

conductedduringthispresentation.AllMHPswereinvitedandencouragedtoparticipateinthe

presentation.Therecordingofthiswebinarandthepresentationmaterialsusedareavailableon

CalEQRO’swebsite.

CalEQROhasrecordedthreePIPinstructionalvideosandhascollectedsuccessfulPIPsinaPIP

Librarythatisavailableonourwebsiteatwww.caleqro.com.

4http://www.caleqro.com/data/california_eqro_resources/PIP%20Library/YouTube%20-%20BHC%20PIP%20101%20-%201%20-

%20California%20Drug%20MediCal%20-%20Bringing%20Ideas%20to%20Successful%20PIP%20Concept.html5http://caleqro.com/#!california_eqro_resources/PIP%20Library6http://www.caleqro.com/data/california_eqro_resources/PIP%20Library/Instructions%20for%20Completing%20PIP%20Validation

%20Tool-PUBLIC_v.3.docx


20

APPENDICES

AppendixA:SummaryofPIPssubmittedbyMHPs

AppendixB:CalEQROPIPValidationTool


Clinical PIP topics submitted 21

CLINICAL PIP TOPICS SUBMITTED

Ofthe14ClinicalPIPsrequiredforsubmission,3wereconsideredActiveandOngoingand5wereCompleted.ThreewereratedasConceptOnlyanddidnothaveinterventionsimplementedatthetimeoftheon-sitereview.OnesubmissionwasdeterminednottobeaPIP.OnePIPwasnotsubmittedbytheMHP.AllthePIPssubmittedaresummarizedhereinthisAppendix.

Timeliness of Care PIPs

• RapidConnect(Completed)

SantaCruz

StudyQuestion

(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO

“Towhatextentwillthe

implementationofRapid

Connect,whichincludes

directcontactwith

beneficiaries/families

(foryouth)seenatthe

CrisisStabilization

Program(CSP)or

immediatephone

contactafterdischarge,

reducethenumberof

readmissionstotheCSP

within30days?”

SantaCruz’sclinicalPIPfocusedonensuringtimelyfollow-uptoservicesforindividualsafterdischargefromtheCSP.

ItinvolvesRapidConnect,aservicetofacilitaterapidfollow-upforbeneficiarieswhowereadmittedtoCSP.MobileEmergencyResponseTeam(MERT)membersorstafffromCountyAccesswillcontacttheindividualwithinonebusinessdaypostCSPstaytodeterminetreatmentneedsandlinkthemtoservices.

Therewrittenstudyquestionstilllacksspecificitythatwouldbeusefulinmeasuringoutcomes.Asecondinterventionwasnotaddedinthepastreviewyear,asrecommendedbyEQROintheFY16-17review.Thegoalofindicatornumberoneis75percent,andforindicatornumbertwo10percent.

ThestudybeganNovember2016andtheMHPconsidersthisPIPtobecompleteasofNovember2017.

EQROagreedwiththeMHPstaffpresentingthePIPthattheclinicalPIPhasthenecessarycomponentsandcanbeconsideredcompleted.

ThefollowingitemswerediscussedwiththeMHP:

Rewritethestudyquestiontomakeitmeasurableandspecific.

Expanduponthedescriptionoftheinterventions,whichwerenotdescribedsufficientlyinthewrite-upwithinthesectioninthePIPDevelopmentToolmarked“describetheintervention”.

ChangethepercentageimprovementforeachquarterinRapidConnectcontactssoitreferencesthepercentofbaselineratherthanthepercentofthetargetedgoalamount.



Redothecalculationmethodforchangeinpercentofreadmissionssothelast-quartercomparisondoesnotincludethosetoimmediatelybehospitalizedfollowingdischargefromtheCSP.TheinterventionwasonlyforthosedischargedfromtheCSPtothecommunity,andthesewereaminorityofthosedischarged.

Clearlyarticulateinsignificantdetailalltheinterventionsthatwillbeimplemented.Additionalinterventionsareneededtoimproveclinicaloutcomesandanswerthestudyquestionasrelatedtobenefittobeneficiaries.

AddoutcomemeasuresatkeyeventstostrengthenthisPIP.DeterminewhatthethresholdisforfrequencyofadministeringtheCANSandmonitorforimprovementinthedataovertime.

Consideradditionalinterventionsthatmightincludelinkageswithposthospitalizationfollow-upactivitiesandstaff.



Quality of Care PIPs

• ImprovingTreatment:Training,BeneficiaryEngagementandTeamBasedCare(Completed)

SantaBarbara

StudyQuestion


“Arebeneficiaryoutcomes,

asmeasuredbythe

ConsumerPerception

Survey(CPS),CANSand

MORS,improvedby

implementing:1)training

forclinicalstaff,2)the

team-basedcaremodel

andtoolsand3)improved

MIStreatmentrelated

reports(formanagersand

supervisors)?”

ThisPIPfocusedonimprovingbeneficiaryexperienceoftreatmentintermsof:a)ensuringthatallbeneficiarieshavehighqualitycurrent/activetreatmentplans;b)implementingteambasedcare;andc)improvingbeneficiaryengagement.Thehypothesiswasthatimprovedexperienceoftreatmentwillresultin:improvementsinspecificoutcomesasmeasuredbytheCPS,CANS(Youth)andMORS(Adult)scores.

TheonlyclinicalinterventionspresentedbytheMHPwerebeneficiaryengagementintreatmentplanningandteam-basedcaretraining(whichinandofitselfisnotanintervention).NotalltheinterventionsweretrackedaspartofthePIP,butthecombinedresultsindicatedimprovementorstabilityintheCANSandMORSscores.ElementsoftheCPSwerealsoreviewed,butthisdatawasnotwelllinkedtotheinterventionsinthisPIP.

TheTAprovidedtotheMHPbyCalEQRO,meanttoaddressareasforopportunityandfuturePIPs,consistedofsuggestionstovalidateassumptionsaboutthecauseofproblemsthroughareviewoftherelevantliteratureandmorethoroughbarrieranalysis;toassurethatallpersonsimpacted,especiallylinestaffandbeneficiary/familymembersareapartofthePIPactivestakeholdergroup;andtoidentifyvariablesnotpartofthePIPstudythatcouldimpactthechangesinbeneficiaryoutcomesbeingtracked.



• IntegratingSmokingCessationProgramsintoaBehavioralHealthSystem(Completed)

Ventura

StudyQuestion


“Willintegrationof

smokingcessationservices

withinVCBHdecreasethe

proportionofbeneficiaries

whodescribethemselves

asactivetobaccousers.

Addendum(5/7/2017):

Willintegrationofsmoking

cessationserviceswithin

VCBHresultinthe

percentageoftobacco

usersremainingquitfor3

mos.,6mos.and12mos.

aftercompletingaCallIt

Quits(CIQ)programbe

least25%,15%and10%

respectively?”

BeginninginJune2016,theMHPfocusedonimprovingthehealthstatusofbeneficiarieswhousetobaccoproducts.Thenationaldataonsmokingprevalencewassupportedbyasamplesurveyofadultbeneficiariesacrossallsites.Thesurveyvalidatedhightobaccouse(46percent),three-quartersofwhomwantedtostopsmoking.

Thebroad,positiveimpactsofthisactivitytodateincludetheMHP’sdevelopmentofaprocesstoroutinelyaskbeneficiariesabouttobaccouse,encourageandsupportsmokingcessation,andcollectionofthatdataaboutsmokingwithintheelectronichealthrecord(EHR).Thesuccessfulscreeningofsmokingstatusfor71percentoftheadultpopulationisanaccomplishment.Thedevelopmentofareferralprocess,coupledwitheffectiveinterventionsforthispopulation,followedbytrackingofresults,isimportant.

Itmustbeacknowledgedthatcurrentresultsshownochangeforbeneficiaries.RelianceonCIQsessionsalone,evenifembeddedinclinics,seemsunlikelytoproducechangeunlessspecificelementsaretailoredtotheMHP’spopulation.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionsaboutthelimitednumberofbeneficiariesimpactedbythisimprovementactivityandtheabsenceofpositiveresults.Thatsaid,thescreeningaspecthasbeenverysuccessful.StrongconsiderationwasgiventoterminationofthisPIPandexploringanotherissuethatcouldhavegreaterimpact.However,thisisanimportanttopic,involvingalargepercentageofMHPadultbeneficiaries.Thereremainsastrongrationaleforcontinuation,iftheMHPadjustsitsinterventionstrategytoimproveefficacyforitsuniquepopulation.TheMHPmayconsiderfurtherliteraturereviewforprovenstrategiesthatcouldinvolvespecificsupportiveapproachestosmokingcessationnotcurrentlyutilized.



Outcomes of Care PIPs

• RehospitalizationRates(ActiveandOngoing)

DelNorte

StudyQuestion


“Willanassessmentbya

drugandalcohol

counselorwithinfive

daysofacutepsychiatric

hospitalizationreduce

therehospitalizationrate

withinayearforDel

NorteCountyfrom12

percentto7percent?”

ThePIPaddressestheissueoftherehospitalizationrateforMedi-CalbeneficiariesinDelNorteCounty.ThisrateishigherthanthestateaverageperdatafromCalEQRO.ThegoalofthisPIPistoreducethenumberofrehospitalizationadmittancestothestateaverageorlower.TheMHPacknowledgeshowtraumatizingandstigmatizingpsychiatrichospitalizationscanbe,evenjustonetime,andthegoalofthePIPistoimproveoutcomesfromthefirstdischargeofhospitalizationandreducetheneedforasecondhospitalization.

TheMHPpresentedastatisticallysignificantdecreaseusingthecommonp-valueoffivepercent.However,thiscalculationwasusingthegiven18.8percentimprovement,whichwasdeterminedtobeinaccurate.Theactualimprovementis11percent.

Moreinformationisexpectedandwillbeavailableasthestudycontinues.Thereisnocompleteanalysisoffindingscurrently.

ThePIPwillbecontinued,andanadditionalinterventionwillbeadded.Discussiononsitelookedatwhatthenewinterventionmightbe,althoughnoconclusionwasreached.

TheTAprovidedtotheMHPbyCalEQROconsistedofonsitediscussionoffindingsofthePIP.CalEQROpointedoutthatthereisaneedformoredatatobeenteredintothestudyandmoreanalyzationofthefindings.Noexternalthreatstovaliditywerediscussedinthewrittenstudy.ThePIPfindingsneedtobeanalyzedonaquarterlybasisataminimum.



• EarlyTherapeuticAlliance&Retention(Completed)

Lassen

StudyQuestion


“Willearlytherapeutic

allianceimprove

beneficiaryretentionby

25percentasmeasured

bythepercentageof

individualsretained

fromassessmentto

initialtherapy

appointment?”

Toaddresstheidentifieddelayoftherapeuticallianceanditsimpactonretention,theMHPlookedatbarrierstoearlytherapeuticalliance.Inareviewofbeneficiaryretentionin2016,theMHPfounda54percentdropoutratefromclinicalassessmenttoinitialtherapyappointment.Onebarrier,identifiedbybeneficiariesandclinicians,wastherapistimpermanence.

Improvingbeneficiaryretentioninservicesthroughbettertherapeuticalliancewillimprovequalityofcare.However,continuedretentioninongoingserviceswillbedependentonotherfactors.Theseinclude,butarenotlimitedto,thefrequencyofongoingservicesandifwaittimesbetweenappointmentsarenotlengthy.

TheTAprovidedtotheMHPbyCalEQROconsistedofadiscussionongeneralPIPdevelopment.WhilethisPIPiscompletedandachievedsuccess,itwasaresubmissionbasedupononsitequestionsduringthereview.Forexample,howdidtheMHPmakethedecisiontoselecttheintervention?Wasthereresearchintothepossiblecausesoftheproblem?HowdidtheMHPdeterminesamplesizewhenevaluatingtheproblem?Onsitediscussionclarifiedthattheproblem,originallydescribedasatimelinessproblem,wastrulyanengagementproblem–beneficiariesdroppingoutofservice.CalEQROdiscussedtheusefulnessofbarrieranalysistodeterminecausesandlinkinterventionstodata.Datarelatedtotheproblemshouldbequantifiedintermsofscopeandsizeandshouldbegatheredbeforeimplementinginterventions.TheMHPwasadvisedtousethemostcurrentPIPsubmissionform(theyhadsubmittedcurrentPIPsonlastyear’sform).



• StrengthsModelInterventionforEmploymentRelated-Goals(ActiveandOngoing)

Mono

StudyQuestion


“WillusingtheStrengthsModelhelpbeneficiariesmakeprogresstowardtheiremployment-relatedgoals,asmeasuredbytheachievementoftheiremployment-relatedgoalsasrecordedintheStrengthsAssessmentoverthetwo-yearstudyperiod?”

TheMHPdevelopedthisPIPfromcommunitysurveydatainwhichmembersofthecommunity,includingsomecurrentandpastbeneficiaries,identifiedlifedomaingoalsinwhichsupportwasdesired.TheMHPutilizedtheSMassessmentwith14highneedbeneficiariesanddiscoveredthat11hademploymentoreconomicgoalsforlifeimprovement.

TheMHPdescribedthevariousaspectsoftheSMapproach,includingtheSMAssessment,PersonalRecoveryPlan,andgroupsupervisionofclinicalstaff.Nospecificinterventionwasdescribedthatrelatedtothespecificactionsofstaffwithbeneficiaries,whichwouldseemtobeakeyelementofthismodel.

Thestudyquestiondidnotprovidetheanticipatedquantifiableimprovementgoal,asrequiredforaPIP.Thestudyindicatorincludesachievementofemployment-relatedgoals.However,thedatatablebreaksoutseparatenumeratorsforthosewhoachieveemploymentgoalsanddeclinetosetanothergoalfromthosewhoachievethelistedgoalandthensetanewgoal.Sinceachievementofemploymentseemstobethegoalofthisactivity,themeetingofthisgoalwouldseemsufficient.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionofneededelementstoaddtothePIPandresubmissionoftheupdate.CalEQROsharedhowthelackofbaselinedata,lackofspecificityofwhatdefinesinclusioninthestudygroupbywayofbeing“stuck”or“highneed”isproblematic.OverthecourseofthisnextreviewperiodtheMHPneedstoadddataelementssuchastheserviceutilizationlevelofthese“high-need”individuals.Theinclusionofbeneficiariesseemstolackspecific,definedquantifiableparametersthatwouldsupportreplication.



• AdultSocialEngagement(ActiveandOngoing)

Napa

• Follow-UpAfterHospitalization(ActiveandOngoing)

StudyQuestion


“IfNapaCountyMental

Healthintroducesaseries

ofsocialengagement

activities,particularly

targetingthemostisolated

beneficiaries,willit

increasethenumberof

activelyengaged

individuals?”

TheMHPdeterminedthroughitsanalysisofitsCPSdatathatonechallengingareareportedbytherespondentsissocialisolationorlackofsocialengagement.Basedonthisfinding,theMHPlaunchedthisPIPtoreducesocialisolationandimprovesocialengagement.Theprimaryinterventionshaveconsistedofincreasedsocialactivitiesspecificallytargetedforthechronicallysociallyisolatedindividuals.

Duetoseveralnaturaldisastersinthepast16months,floodandwildfire,theMHPhadtopostponeseveraloftheinterventionactivities.Inaddition,onekeyindicatorobtainedfromCPSisdelayedasittakesapproximatelysixmonthsfromitsadministrationinNovemberfortheMHPtoreceivethedata.Consequently,theMHPwasnotabletoprovideanyoutcomesfromtheactivitiesthatindeedtookplacein2017.CalEQROhasthereforefoundmanyofthePIPvalidationitemstobenotapplicableatthistime.

OnsiteTAandCalEQROfeedbackconsistedofthefollowing:

PIPQuestion:TheMHPwasinformedthatthePIPquestionshouldoutlineandlinktheinterventionandtheintendedoutcomes.Followingtheonsitereview,theMHPsubmittedarevisedPIPquestionthatmetthestandards.

Indicatorsandvalidityoffuturefindings:CalEQROadvisedtheMHPthatthestudymethodologyshouldhavethepowertodetectthechangesamongtheintendedbeneficiaries.Currently,theCPSmethodologyismoregeneric,andthesamplereflectstheoveralladultbeneficiaries.CalEQROrecommendsthattheMHPconsidermorefrequentadministrationofCPSamongthetargetbeneficiaries.



Riverside

• ComplexCareCoordination(Completed)

StudyQuestion


“Willtheimplementation

ofnavigationstrategies

withinpatienttreatment

facility(ITF)discharge

teamsincreasethe

percentageofunengaged

beneficiarieswhoreceive

anoutpatientfollow-up

servicewithinsevendays

ofdischarge?”

ThegoalofthisclinicalPIPistoincreasebeneficiaryengagementinandaccesstotimelyoutpatientserviceswithinsevendaysfollowinghospitaldischarge,withfocusonunengagedbeneficiarieswhoarenotalreadyknownbyandopentotheoutpatientmentalhealthsystem.Theinterventionusespeerspecialiststhroughthepeer-runnavigationcentertoengagethesebeneficiariesandlinkthemtoservicesincludingtherapy,casemanagement,medicationmanagement,housing,detox,andothersupports.

Thestudyquestionwouldbestrengthenedbyaddingameasurabletarget.Theindicatorgoalsaretoolowtodemonstratestatisticallysignificantchange,andthereforeanychangemeasuredcannotbeattributedtothisinterventionalone.Thecurrentinterventionrequiresamoredetailedexplanationoftheclinicalactivitiesthatwilltakeplacetoengageandprovideservicestothetargetpopulationofunengagedbeneficiariesnewlydischargedfromaninpatientfacility.

TheTAprovidedtotheMHPbyCalEQROconsistedofadiscussiononthedifferencesbetweenclinicalandnon-clinicalPIPs,andsuggestionsforstrengtheningthisclinicalPIP.Asdiscussedonsite,itwouldbehelpfultoaddadescriptionofhowtheengagementactivitiesprovideadirectlinktoclinicalservicesandincludethecompositionoftheITFdischargeteam(clinicalandnon-clinicalstaff:peerspecialists,therapists,casemanagers,psychiatristsandnurses,etc.).DuringtheonsitediscussiontheMHPdidprovideadditionalclinicalinformationandjustificationwhichCalEQROagreeswith.TheMHPhasagreedtorewritethePIPDevelopmentOutlineandwillbeengaginginfurtherconsultationwithCalEQROinthecomingmonths.



SanBernardino

StudyQuestion


“Willcomplexcare

coordinationreducethe

risk,frequency,and

durationofpsychiatric

hospitalizationsby20

percentforpsychiatrically

andmedicallycomplex

beneficiarieswhorequire

themostintensivecare

coordinationservices?”

TheMHPhasidentifiedapopulationofbeneficiarieswithcomorbidsomaticconditionswhohavehigherfrequencyandlongerdurationofpsychiatrichospitalizationscomparedtotheMHP’sgeneraladultpopulation.ThegoalofthisPIPistoprovidecoordinatedcarethataddressesboththechronicmentalandphysicalillnessestoreducetherisk,frequency,anddurationofpsychiatrichospitalization.

ThePIPteamprovidedindicators(whichwerethesameastheiroutcomemeasures)thatwereobjectiveandmeasurable.However,therewerenoindicatorstoaddresstheperformanceoftheteaminapplyingthem.Indicatorsareneededthatcompare:

• actualhome/fieldvisitstoneededhome/fieldvisits

• actualaccompanimenttomedical/pharmacyappointmentsneededaccompaniment

• frequencyofinquiryintomedicationcompliance

• andothers

TheTAprovidedtotheMHPbyCalEQROconsistedoftherecommendationtooperationalizethecomponentsofcomplexcarecoordinationandtoarticulatethedifferencesintheprojectfromoneyeartothenext,whichtheMHPdid.CalEQROandtheMHPdiscussedtheMHP’splansforafutureclinicalPIP—reducingpolypharmacythroughexaminationofantipsychoticprescriptionsandimprovingcontinuityofcarebyimprovingcommunicationbetweeninpatientandoutpatientpractitioners.


Submission determined not to be a PIP 31

NON-CLINICAL PIP TOPICS SUBMITTED

Ofthe14non-clinicalPIPsrequiredforsubmission,5wereratedasActiveandOngoing;4wereratedasCompleted;2wereratedasConceptOnly,NotYetActiveanddidnothaveinterventionsimplementedatthetimeoftheon-sitereview.Additionally,onesubmissionwasDeterminedNottobeaPIPandtwoPIPswerenotsubmitted.

Access to Care PIPs • PsychiatryNo-ShowStudy(Completed)DelNorte

StudyQuestion

(aspresentedby

MHP)

FocusofPIPAreasfor

ImprovementTAProvidedbyCalEQRO

“Willno-showrates

decreasefor

beneficiariesfrom

FY2015-16to

FY2016-17witha

changefromlocum

psychiatry(e.g.,a

newdoctorevery

threemonths)to

telepsychiatry(i.e.,

thesamedoctorfor

patientforatleast

oneyear)?”

Afterincreasingpsychiatryservicesin2016throughlocumpsychiatrists,itwasfoundthatno-showratesandformalgrievancesagainstpsychiatryservicesincreased.Theprevioustelepsychiatristswerestillinplaceandneitherno-showsnorformalgrievancesshowedanincreaseforthoseproviders.TheMHPdecidedtoendthecontractwithlocumpsychiatryandincreasetelepsychiatry.ThegoalpresentedbytheMHPwastoachieveconsistencyinserviceprovision,as“consistentpsychiatryleadstobetterpatientoutcomesandlessno-showsorbeneficiarycancellation.”TheresultsofthatchangearethefocusofthisPIP.

TheMHPhypothesisincludesthatwhenbeneficiariesdonotgetthecaretheyneed,especiallywithpsychiatry,negativeoutcomesarepredictedtooccur,includingsymptomsworseningandpossibleneedforhigherlevelsofcare(e.g.psychiatrichospitalizations).TheinterventionwouldbetoincreasethehoursofpsychiatrywithKingsViewtelepsychiatryusingdoctorswhowillbearoundforayearormore.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionoftheneedtodoathoroughbarrieranalysisbeforedesigningaPIP.CalEQROalsonotedthatmeasuringresultsonceaquarterataminimumwouldbeusefultoseeifthePIPneedsanyadjustmentsmovingforward.TheMHPplanstomonitorno-showratesforallservicesmonthlyandwilltrackthisinterventionandadjustaccordingly.Issueswithcalculatingpercentversuspercentagepointswerepointedout.TheMHPwasofferedongoingTAforcreatinganewPIP;thisPIPisconsideredcomplete.TheMHPwasencouragedtoconsultwithEQROearlyandoftenduringPIPformulationforitsnextnon-clinicalPIP.



• Open-AccessSchedulingandKeptAppointments(ActiveandOngoing)

Lassen

StudyQuestion


“Doesimplementingopenaccessschedulingforoutpatientservicesimprovekeptappointmentratesbytenpercentasmeasuredbyattendance?”

TheMHPfocuseditsnon-clinicalPIPonaddressingtheno-showrateforscheduledinitialandfollow-upappointmentsthroughwalk-inaccessandopenaccessscheduling.TheMHPidentifiedopenschedulingasaviableinterventionthroughaliteraturereview.Openschedulingallowsfortheschedulingofnextvisitappointmentsonly,ratherthanbookingseveralappointments,whichresultsinleavingnospaceforotherbeneficiaries.Thepercentageofmissedappointmentsaffectsotherimportantsystemandbeneficiary-centeredfactors.

TheMHPdidnotprovideanexplanationforthedisconnectionbetweentheinterventionandtheproblem.ThePIPattemptstosolvetheproblemofno-showswithopenaccessschedulingbutdoesnotinquireindepththereasonsforno-shows.Theinterventioncenteredonbeneficiarieswhokepttheirappointmentsratherthanamelioratingthebarriersforbeneficiarieswhowerenotabletokeepappointments.Abarrieranalysiswouldhaveledtoamoreimpactfulintervention.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionsonhowtobetterdevelopPIPtopicsandimprovethesuccessofinterventions.Inthiscase,abarrieranalysisonreasonsforno-showswouldbetterinforminterventions.Toensurebothvalidityandreliability,thedatacollectionplanshouldspecifythedatatobecollected,sources,collectionmethodsincludingpersonnel,andtheinstrumentsused.TherewerealsoelementstothePIPwhichwereunder-developedduetotheMHPusingtheincorrectPIPsubmissiontool.



• LawEnforcementCo-locatedTriage,Engagement,andSupport(TEST)Teams(Completed)

SanBernardino

StudyQuestion


“Willco-locatingTriage,

Engagement,andSupport

Teams(TEST)atlaw

enforcementagencies

decreaseinvoluntaryholds

writtenbylawenforcement

andreceivedbyARMCby15

percent,whilealso

decreasingpsychiatric

hospitalizationsfor

beneficiariesservedbythe

TESTstaffby20percentand

increasingroutineoutpatient

psychiatriccareby25

percent?”

ThisisthesecondyearoftheMHP’sprojectonTEST,ateamofMHPclinicalstaffwhoareco-locatedwithlawenforcement.ThegoalofthePIPwastouseTESTtoprovideamoreappropriateand(clinically)informedresponsetolawenforcementcallsthatinvolveresidentswhopresentwithmentalhealthconcerns.

TheresultsofthestudyshowthattheTESTwassuccessfulinreducing5150sbytheFontana(by54.55percent)andRialtopolicedepartments(by28.00percent),thetwodepartmentsinurbancommunities.TheTESTdecreased5150sbytheMountainRegionpolicedepartmentby12.14percent.TheTESTdecreased5150sinVictorvilleonlymarginally(by0.84percent),towhichtheMHPpartiallyattributedtonewdeputiesjoiningthelawenforcementagencies.FollowingtheTESTencounter,beneficiaryhospitalizationsdecreased,andoutpatientservicesincreasedoverall.

TheTAprovidedtotheMHPbyCalEQROconsistedofrecommendationtoadjustthepre-andpost-measurementperiod(fromsixmonthstofourmonths)toenabletheMHPtohavecompletedatatocompareforthisreview,whichtheMHPdid.CalEQROandtheMHPdiscussedtheMHP’splansforfuturenon-clinicalPIPs.TheMHPdiscussedtheirintentionstoaddressthelowrateof7-dayfollow-upsposthospitalization.



• BeneficiaryAcuityIndex(Completed)

Ventura

StudyQuestion


“Willanacuityindexderived

fromthenature/extentof

pastpsychiatric

hospitalization(s)provide

staffwithapracticaland

effectiveguidetoservice

delivery?Willprovidingstaff

withreferencetoanobjective

measureofacuity,alongwith

associatedservicesdelivery

expectations,increase

productivityandcaseload

coveragetherebyreducing

psychiatrichospital

admissionsinthelong-term?”

TheaimofthisPIPwastoensurethatbeneficiariesidentifiedasfittingintooneofthecategoriesofhigh,moderate,low,oruncategorized,werereceivingalevelofcarelikelytomeettheirserviceneeds.TheMHPthendevelopedaminimumservicelevelwhichwaspairedtotheneedcategories.TheMHPalsoestablishedmechanismsforthebeneficiaryandstafftoprovidefeedbacktotheservicelevelsandscoring.

TheMHPdeterminedtodiscontinuethisPIPinJanuaryof2018.TheMHPbelievesitispreparedtomakesystemdecisionsbasedonthisbrieftestprocess(11months).TheadditionofalevelofcareinstrumenttosupportacomprehensiveprocessguidingservicedeliveryacrossalllevelsofneedandhelpingdetermineaconsumerflowintorecoverywouldimprovethisPIP.Furthermore,approximately50percentoftheMHP’sadultconsumersareuncategorizedduetolackofrecenthospitalizationhistory,whichmaylimittheapplicationofthisapproach.

TheTAprovidedtotheMHPbyCalEQROconsistedofonsiteencouragement,suggestingthattheuseoftestedandvalidatedlevelofcaretoolswouldprovideadditionaldatatoaugmentthecurrentapproach.TheuseofinstrumentssuchasMORS,LOCUS,andANSAprovidebroadapplicability.Theuncategorizedbeneficiaries,forwhomtheMHPlacksaspecificserviceanalyticstrategy.FormanyMHPs,thelargerchallengeisdevelopingstandardsforsatisfactorylevelofimprovementforsteppingdownservicelevels.ThisPIPwasmorebasicinitsapproach,seekingtoassurehighlevelneedbeneficiariesaresoidentifiedandreceiveaminimumlevel.




• TimelinesstoPsychiatricServices(Completed)

SantaBarbara

StudyQuestion


“Willimplementingthesix(6)

interventionsofthePIP:1)

appointmentremindercalls;2)

team-basedappointment

management;3)increased

clinical/peercontactspriorto

psychiatricassessment;4)

implementationofasingular,

standardizedappointment

schedulingsystemthroughout

thecounty;5)enhanced

recruitmentofpsychiatrists

andphysicianassistants;and

6)incentivizingpsychiatrists’

productivity,resultin:a)a

reductioninbeneficiaryno-

showratesandb)areduction

inwaittimebetween

admissionandfirstpsychiatric

appointmentintheadultand

children’ssystemsofcare?”

ThisPIPistryingtoreducethetimeittakesfornewbeneficiaries(adultandyouth)tohavetheirfirstappointmentwithapsychiatrist.ThePIPtestsseveralstrategiestodeterminewhichisthemosteffective.Theinterventionsinclude:appointmentremindercalls,team-basedappointmentmanagement,increasedclinical/peercontactpriortothefirstpsychiatricappointment,implementationofasingular,standardizedappointmentschedulingsystemthroughoutthecounty,enhancedrecruitmenteffortsforpsychiatristsandphysicianassistants,andincentivizingpsychiatrists’productivity.

Thisnon-clinicalPIPhadoriginallyplannedaninterventiontoimplementanelectronicscheduler.ThiswasnotputinplaceduetovacanciesinITstaffandsomestaffresistance.Theelectronicschedulerimplementationwouldimprovefuturetrackingofno-showsandincreaseprovidercapacity.

TheTAprovidedtotheMHPbyCalEQRO,meanttoaddresscurrentopportunitiesandfuturePIPs,consistedofsuggestionstostaggerinterventionstobetterunderstandtheimpactofanyspecificintervention;limitthefocusandnumberofinterventionsinPIPs,andassurethatallthepersonsimpacted,especiallylinestaffandbeneficiary/familymembers,areapartofthePIPactivestakeholdergroup.



• TimelinessofAccesstoServices(ActiveandOngoing)

SantaCruz

StudyQuestion


“Willdisseminationof

accuratetimelinessdataand

systemwideavailabilityof

resourcesassistMHPto

insurebeneficiariesreceive

timelyservicesuponinitial

request?”

ThePIPisfocusedonimprovingthetimelinessfromfirstcontacttofirstsession.Therationaleisthattimelinesswillreduceadverseeventstobeneficiarieswhoaresufferingandinneedoftreatmentandhastentheirroadtorecovery.

Asstatedthestudyquestiondoesnotspecificallyaddressabeneficiarybenefitofmoretimelyresponsetorequestforservicesoraccuratetimelinessdatadissemination.

Itwasfoundthatsomeclinicianswerenotrecordingtimeoffirstofferedappointment.ThePIPstudyteamdecidedthatasinglemeasurecouldencompassboth.Theysurmisedthatthetimeoffirstofferedappointmentwouldsometimesbesoonerthanfirstactualappointment,andatthelatestwouldcoincidewiththedateoffirstactualappointment.Therefore,bytakingtheearlieroffirstofferedappointment(whenthedataisavailable)orfirstactualappointment,timelinesswouldbesufficientlymeasured.

TheTAprovidedtotheMHPbyCalEQROconsistedofthefollowingfeedback:

Thechangeinmethodstoreducewaitlistsislikelyhelpfulbutisnotinitselfameasure.ThePIPmustmeasuretheimpactoftimelinessonthebeneficiariesstudied.Thismeansitisnecessarytomeasurewhatwasthechangeintimefromfirstcontacttofirstofferedappointment,andfromfirstcontacttofirstactualappointment?

CalEQROdidnotrecommendblendingthemeasuresoffirstofferedappointmentandactualappointmentdates.Itisrecommendedtheyarekeptseparate.

CalEQROalsorecommendedanalyzingforpatternsthefrequencyofmissingdatafordateoffirstofferedappointment.Inthatway,theMHPcanascertainwhattreatmentprogramshavethemostmissingdataandcanconducttargetedtrainingsothenumbersdecrease.



• ImprovingTimelyAccesstoServices(Completed)

Trinity

StudyQuestion


“Willchangingtheinternal

“accessmeeting”processfor

approval,andmodifyingthe

settingandtrackingof

clinicianappointmentsresult

inanimprovementof

timelinessofservicesfor

actualclaimedassessmentto

actualclinicalappointment

(assessmenttoclinical

appointment)froman

averageof21daystoan

averageof12days,a43

percentreduction?”

Thegoalofthisnon-clinicalPIPistoreducethenumberofdaysfromactualclaimedassessmenttoactualclaimedappointment fromanaverageof21daystoanaverageof12days,a43percentreduction.ThePIPwillevaluateallthevariousstepsintheworkflow,identifyareasofbottlenecksordelaysandimplementinterventionstoimprovethetimeliness.

Interimanalysisrevealedthatalthoughtherewasatrendtowarddecreasedtimetoaccessservicesinconsistenciesincollectingthedataoccurred.Staffwerenotconsistentindocumentingthetimelinesandstandardizationwasnotachievedthatimpacteddata.TheMHPindicatedaneedtocontinueitseffortstocollectadditionaldataandamorerobustreview.ThereisconcernthattheMHPcollectedsmallnumbersinitson-goingdatacollection.Thisshouldhavebeenaddressedtoeradicatetheproblemandtodeterminetheeffectsoftheimprovementintended.ThereliabilityindocumentingthestepsamongstproviderstaffisalsoaquestionthattheMHPwillneedtoaddress.

TheTAprovidedtotheMHPbyCalEQROconsistedofencouragingtheMHPtocompetetheactionsidentifiedfordatacollection,andcontinuetomakeimprovements,andwasencouragedtoseekthisearlyintoitsconceptandtocontinuetoseeksupportinitswrite-ups.

However,sincethisisthesecondyearofsubmittingthisPIPwithlimitedactivitiesanddataprovided,theMHPisadvisedtoinitiateanewPIPforthenextreviewcycleforrating.CalEQROalsodiscussedthetimelinesofPIPactivities,encouragingtheMHPtoidentifyitsPIPearlyintheprocess,implementinterventions,andcollectandanalyzetherelevantdatetoidentifynecessaryadjustmentssothattheratingforthePIPisacceptedasactive.TheMHPwasofferedongoingTAandtheMHPfollowedupwithanemailupdateandindicatedconsultationwillbescheduledintheupcomingmonths.




• ImprovingEngagementandRetentionServices(ActiveandOngoing)

Riverside

StudyQuestion


“Doestheuseofa

collaborativeassessment

processdecreasenoshow

ratesby25percentand

increaseby20percent

continuedengagementin

servicesfollowingtheinitial

assessmentwithcontinued

engagementdefinedasat

leasttwoserviceswithin30

daysoftheinitial

assessment?”

Thegoalofthisnon-clinicalPIPistoincreaseengagementandretentionofchildrenincounty-operatedspecialtymentalhealthoutpatientclinicsbydemonstratingthattheinterventions(collaborativeassessment,andavailabilityofeveninghours)workinoneclinic,andthenscalingthemuptootherclinics.ThedatausedforthisPIPincludedthepercentageofbeneficiarieswithfewerthanfiveservices,andtheaverageapprovedclaimsperbeneficiarywhichshowedapatternoffewerservicesperbeneficiarythanotherlargecountiesandthatstatewide.

ThePIPliststwointerventions,collaborativeassessmentandeveningavailabilityforservices.Bothlackadetaileddiscussionandsteps/activitiesthatwillbetakentoimplementthem.Priortoselectingtheseinterventions,theteamappearstohavemissedthestepofdeterminingbeneficiaries’reasonsforlackofengagementandsubsequentlydroppingoutoftreatment.Interventionsshouldbeselectedbasedonthisinformationtofullyaddressthebarriersbeneficiariesareexperiencing.

TheTAprovidedtotheMHPbyCalEQROconsistedofadiscussiononthedifferencesbetweenclinicalandnon-clinicalPIPs,andsuggestionsforstrengtheningthisnon-clinicalPIP.Asdiscussedonsite,theMHPhasagreedtoreviewthefeedbackinthePIPValidationToolandupdatetheirPIPDevelopmentOutlineinthecomingmonthstoreflectthefeedbackandtheirquarterlydataanalysesandchangesmadetotheinterventions.TheMHPwillseekfurtherconsultationwithCalEQROafterthisreporthasbeenfinalized.




• StrengthsBasedInterventions(ActiveandOngoing)

Inyo

StudyQuestion


“Willimprovingthecontent

&structureofgroup

supervisionsessions

utilizingtheUniversityof

KansasStrengthsModel

groupsupervisiontoolsand

methodologyresultinmore

beneficiariesachieving

theirself-identifiedgoals

relatedtoliving

arrangements,vocational

status,educationalstatus,

hospitalizations,or

successfulcompletionand

exitfromservices?”

TheMHPhasengagedwithathreecounty,EasternSierra,collaborativeprojectimplementingtheStrengths-BasedModel,whichisoutoftheUniversityofKansasandsupportedbytheCaliforniaInstituteofBehavioralHealth(CIBH).TheMHPdetermineditlackedaclearprocessthatidentifiedandtrackedbeneficiaryprogresstowardsidentifiedlifegoals,includingthoseaspectsthatwereoutsideofpureclinicalindicatorprogress.TheMHPfurtherexploredtheaspectofpersonallifegoalsthroughareviewofbeneficiaryrecordsanddiscoveredthatveryfewhadidentifiedormadeprogresswithimprovedhousing,employment,education,emergencyroomvisits,psychiatrichospitalizations,andgraduationfromservices.

Thestudyquestionaswrittendoesnotclearlyandsuccinctlyidentifywhatisbeingdonedifferentlywithbeneficiaries,asrequiredinaPIP,thedetailsofthatinteraction,anddoesnotproposehowmuchofachangeisexpected.

Thelistofinterventionsrelatestouseofthestrength’smodelassessment,supervision,anduseofreporttotrackpotentialbeneficiarygains.Aswritten,thiswouldbedifficulttoreplicate,forthePIPdoesnotspecifythe‘whatandhow’interventionsarebeingdonewithbeneficiariesthroughthelensofthismodel,andhowthestaff-beneficiaryinteractionisbeingchangedtoimprovelikelihoodofbeneficiaryattainmentofgoal.Theseelementsarecriticaltocorrectgoingforward.

TheTAprovidedtotheMHPbyCalEQROconsistedofonsiteandpre/postinteractionsspeakingtohowthisPIPandtheclinicalPIPutilizedthesametopic–Strength-BasedModel–andwereduplicative.Thenon-clinicalPIPappearstohavebroaderinterventionelementsandfocusthatwouldsupportutilizationforseveralreviewcyclessolongasthefocusandinterventionscontinuedtogrowandchangeovertime.Thestudyquestionrequirestheadditionofaquantifiableelement,andtheinterventionsrequirespecificinformationregardingthespecificinterventionsusedintheclinician/beneficiaryinteractiontoaccomplishchange.



CONCEPT ONLY, NOT YET ACTIVE PIP TOPICS SUBMITTED


• TimelinessPlan(ConceptOnly,NotYetActive)

Modoc–Non-clinical

StudyQuestion


“Willdesignating“access”

cliniciansdecrease

beneficiarywaittimefor

behavioralhealthnon-

urgentinitialcontactand

firstassessmentas

measuredbytimeliness

measures?”

ThePIPfocusesontimelinessfornon-urgentappointments.Reasonsidentifiedfordelayedtimelyaccesstoservicesincludedlackofsufficientcliniciansandlargecaseloaddistributions,bothofwhichareaddressedthroughtheproposedinterventionofappointing“access”clinicians.Accessclinicianswillbepoisedtoacceptnon-urgentcall-insorwalk-insastheypresentatthemainclinic.Thecurrentbaselineforaccessis11businessdays.TheMHP’sstandardistenbusinessdays.

Aswritten,theMHPisplanningtobegintheirinterventionin2019.However,thatwouldmeanagapoftenmonthswithoutanactivePIP.Indiscussion,theMHPsaidtheywouldlikelybestartingearlieroncetheyimplementanewscreeningandtriagesystem.Alsodiscussedwasthatwhileworkingtowardsthegoalofcomplyingwithamandatedmeasure,PIPsshouldbeapproachedintermsofbeneficiarybenefit.Athoroughbarrieranalysisshouldbecompletedsopatternscanbeidentified(i.e.,specificstafforclinic)todeterminewhattypesoftargetedinterventionsareneeded.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionregardingtherequirementtohavetwoactivePIPsandreviewingthecriteriaforactivePIPs(havingactive/newinterventionseachyearforunsuccessfulPIPs).Giventhatthebaselinefortimelyaccessis11businessdays,andclosetothestandardof10businessdays,continuationofthisPIPisdependentonthebarrieranalysisandwhetherituncoversabarrierthatpotentiallyaffectsmanybeneficiaries.Datashouldbeanalyzedatleastquarterly,thoughmoreoftenisrecommendedtoimplementchangesbeforetheendofayear.



• TimelyAccessforChildrenandYouth(ConceptOnly,NotYetActive)

Siskiyou–Non-Clinical

StudyQuestion


“Canschedulingwith

contractedproviderRemi

Vistaanytimethereisnotan

availableMHPappointment

slotwithin10daysresultin:

• Anincreaseintheproportionof

beneficiariesbeingoffered

anassessmentwithin10

businessdaysfrom42

percenttoagoalof90

percent;

• Areductionintheaveragenumberofdaysbetween

referralandactual

assessmentfrom19.4to

12.0;andultimately;and

• Anincreaseinthepercentageofchild/youth

beneficiarieswhoattend

anassessmentfroma

baselineof80percenttoa

goalof95percent?”

ThePIPintendstoofferassessmentappointmentswithintendaysofcontactandtotracktheaveragenumberofbusinessdaysbetweenreferralandassessment.Tocomplywiththeten-daystandard,theMHPwillreferyouthtoitsorganizationalprovider,RemiVista,fortheassessmenttobecompleted.ThisPIPisdesignedtoimprovetimelyassessmentratesforchildrenandyouthwhoareneworreturningtoservices.

AlthoughtheMHPhassubmittedthisasanon-clinicalPIP,theprojectisattheconceptonlystage.Theinterventionshavenotbeenappliedanddatahavenotbeencollected.TheMHPwillneedtoimplementinterventions,providethefollowupdata,andanalyzethedatatosupportitspremisethatimprovedtimelinesstoassessmentsoccurred.SincetheMHPdiscussesboththetimelinessandengagementstrategiesforbothPIPsforthesameagegroup,itiscriticaltocollectdatafordifferentinterventions.TheMHPwasadvisedtocontinuetodefineseparateelementsforeachPIPtodistinguishthetwo.TheMHPwillalsoneedtodefinewhatitexpectstoachieveforbeneficiarybenefitwhichappearstobeidentifiedasreducedwaittimeseveniftheoptiontorefertoRemiVistaisnotutilized.

TheTAprovidedtotheMHPbyCalEQROconsistedofpre-sitetelephonediscussionandtheon-sitediscussionoftheelementsthatwouldhelpinacceleratingtheactivitiesofthePIPprocess.PostreviewphonecallsweremadeaswellforTA.ElementsofthePIPprocessthatwereemphasizedconsistedoftheneedfortheinterventions,thedatacollectionplan,andthebenefittobeneficiaries,alltobemeasurable.TheMHPwasencouragedtoconsultwithCalEQROearlyandoftenduringPIPformulationsandtheMHPrecognizedthisoption.FurtherTAhasbeenscheduled.




• ImprovingBeneficiaryOutcomesthroughintegratedtreatmentofCo-OccurringDisorders(ConceptOnly,NotYetActive)

Modoc-Clinical

StudyQuestion


“Willimplementationof

theDrugUseScreening

QuickInventory(revised)

(DUSI-R)improvequality

ofcareandaccuracyof

diagnosesasmeasuredby

co-occurringdiagnosis

ratesandbeneficiary

retentionrates?”

TheMHPreportedaco-occurringdiagnosisrateof6percentduringtheFY16-17review.Retrospectivelytheywereunabletoduplicatethelowrate,butinsteadcalculatedtherateat21.03percent.TheMHP’sgoalremainstoincreasetheco-occurringreportingratesothatitismoreinlinewithnationalstandards.

TheMHPwasadvisedtoconductabarrieranalysistodiscoverwhatledtothelowrateofco-occurringdisordersandinaccuratediagnoses.

TheMHPplanstouseanassessmenttoolforSUDandmentalhealthtocaptureco-occurringdiagnoses.However,theoverarchinggoalshouldbetoaccuratelydiagnosebeneficiaries.Otherwise,agoalofincreasingtheidentificationanddocumentationofco-occurringdisorderscouldleadtooverdiagnosingco-occurringdisorders.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussiononwaystoinsurevalidityandreliabilityoftheirefforts.Thedatacollectionplanshouldspecifythedatatobecollected;thedatasources;howandwhenthedataaretobecollected;whowillcollectthedata;andinstrumentsusedtocollectthedata.TheMHPstatedthattheyhadrecentlyprovidedadditionaltrainingforstaffindatacollection.Asaresult,theMHPwasadvisedtoseeiftherearecurrentissueswithreportingofdiagnosesandconsiderthepossibilitythatcurrentdatamaynotindicateaproblemwarrantingaPIP.




• InitialEngagementandRetentioninChildren’sServices(ConceptOnly,NotYetActive)

Siskiyou-Clinical

StudyQuestion


Canthefollowingresultinanincreasein

thepercentageofchildrenandyouth

whoreceiveatleastoneclinical

treatmentafterassessment,froma

baselineof71percenttoagoalof90

percent(i.e.,reducethepercentageof

childandyouthbeneficiariesdischarged

afteronlyoneservicefrom29percentto

10percent)?

• Traininginandimplementationof

FIT,whichinvolvescollecting

feedbackfrombeneficiariesand

usingthatfeedbacktoimprove

listening,engagement,rapport

andtrustwithbeneficiaries;

• Immediatefollow-upappointment

schedulingbytheclinician;and

• Providingtelephoneand/orhome

visitsbybehavioralhealth

specialistsbetweenthe

assessmentandfollow-up

treatmentappointments?”

ThegoalofthisPIPistoincreasetheproportionofchildrenandyouthwhoareretainedbeyondtheirfirstserviceencounteridentifiedastheassessment.

AlthoughtheMHPsubmitteddetailedandthoughtfulindicatorsandexplainedtheinterventions,ithasnotproceededbeyondtheconceptonlystage.Theinterventionshavenotbeenapplied,anddatahavenotbeencollected.

TheTAprovidedtotheMHPbyCalEQROconsistedofphonecallspriortotheon-sitereviewandfollowingthereview.Sincethen,theMHPhasdevelopedaconceptfortheirclinicalPIP,althoughasmentioned,theMHPhasnotimplementedtheinterventions.Theon-sitediscussionsduringthereviewincludedencouragingtheMHPtocollectandreportondatamonthlyandcomparedataquarterly.ThePIPmustidentifyandmeasurethebenefittothebeneficiaryaswell.Atanypointduringthedatareview,theMHPisencouragedtocontactCalEQROregardingcontingenciesortrainingneededtoreachitsgoals.



• ImprovingAnxietyLevelsofBeneficiariesDiagnosedwithanAnxietyDisorder(ConceptOnly,NotYetActive)

Trinity-Clinical

StudyQuestion


“Willcreationofan

interventiongroupfor

treatinganxietyreducetheaverageaggregate

levelofanxietybythe

beneficiarypopulation

from2.2to2.0(onascale

of1-3)asmeasuredby

theANSAtool?”

TheoverallgoalofthisclinicalPIPistoimprovetheoutcomesofbeneficiariesdiagnosedwithananxietydisorderasmeasuredbytheANSAtool.Addressingtheissueisexpectedtoimpactanybeneficiarywithananxietydiagnosis,asmuchas27percentofthebeneficiaries,aswellasthosewithanxietysymptoms.ThegoalofthisPIPistoimproveandtoreducetheseverityofanxietyexperiencedbythebeneficiaries,potentiallyleadingtoearlierengagementwiththeadditionofagroupfocusedoncopingstrategies.

TheMHPwillofferandconductagroupfocusedonstrategiestoimprovecopingwithaMindfulness-basedStressReductionsyllabus.TheintroductionoftheMindfulness-basedStressReductiontechniqueishopedtoleadtoanimprovementinthelevelofanxietyexperiencedbyadultbeneficiariesbyprovidingcopingskillstolessensymptoms.

Thisisintheplanningphasenow,aninitialsyllabushasbeendrafted,andisintendedtosupportthebeneficiaryindealingwithandreducinglevelsofanxiety.TheMHPalsobelievesthatbeneficiarieswhouseimprovedcopingskillsmaybeabletofocusonotherbehavioralhealthissuesthatarecurrentlysupersededbytheirfocusonanxiety.Animprovementmayalsobeinassuringamoreaccuratediagnosisandmorefrequentupdatingofdiagnoses,withcorrespondingtreatmentplansforbeneficiariesinthiscategory.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussingthetimelinesofPIPactivities,encouragingtheMHPtoidentifyitsPIPearly,implementinterventions,andcollectandanalyzetherelevantdatatoidentifynecessaryadjustmentssothattheratingforthePIPisacceptedasactive.Thestudyquestion,althoughmeasurablecouldbereviewedsinceitsuggestsamarginalimprovementgoal(from2.2percentto2.0percent)andtoreviewthelanguageinthestudyquestionwiththesuggestiontochangethewordingto“Willprovidingthe…”TheMHPwasofferedon-goingTA.Nofurtherconsultshavebeenscheduled.



SUBMISSION DETERMINED NOT TO BE A PIP

• Strengths-BasedLearningCollaborative:StrengthsModelGroupSupervisionforEmployment-RelatedGoals(SubmissionDeterminedNottobeaPIP)

Mono–Non-Clinical

StudyQuestion

(aspresentedby

MHP)

FocusofPIPAreasfor

ImprovementTAProvidedbyCalEQRO

“WillusingSMGS

(StrengthsModel

GroupSupervision)

helpbeneficiaries

makeprogresstoward

theiremployment-

relatedgoals,as

measuredbythe

achievementofand/or

changeinemployment-

relatedgoalsoverthe

two-yearstudyperiod

asreportedonthe

Strengths

Assessment?”

TheMHPhasidentifiedfulfillmentoflifegoalsofbeneficiariesasnotwellsupportedbytheusualclinicalfocusofstaff,whichtendstoalignwithsymptomsandimpairmentsofillness.Theexistenceoflifedomainareasthatareunfulfilled,suchashousing,educationandemployment,hasbroughttheMHPtofocusonanapproachgearedtosupportothersuccesses.TheStrengthsModelisassociatedwithaspecificassessmentapproach,thedevelopmentofapersonalrecoveryplanandsupportedbyaspecificfocusingroupsupervision.Likelythisisassociatedwithchangesinapproachbyclinicalstaff,buttheseinterventionsarenotdescribedwithinthisPIP.

ThisPIPisverysimilartotheclinicalPIPbutwithaslightlydifferentfocus.TheoverlapissufficienttoconcludethatbothcannotbeacceptedasactivePIPsforthisMHP.

TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionoftheduplicativeaspectsofthisPIP,andidentificationofpotentialalternatePIPtopics.Onetopicthatwasdiscussedwasthatoftelepsychiatryappointmentno-shows,whichtheMHPhasbeentracking.

CalEQRO PIP Summary Report Q4 April – June 2018 Appendix A

Submission Determined Not to be a PIP 46

• StrengthsAssessment(SubmissionDeterminedNottobeaPIP)

Inyo–Clinical

StudyQuestion

(aspresentedby

MHP)

FocusofPIP AreasforImprovement TAProvidedbyCalEQRO

“Willthe

implementationofthe

StrengthsAssessment

toolfromtheStrengths

Modelhelpmoveeight

ICBHbeneficiaries

formerlymiredin

repetitiveservice

utilizationtowards

theirhighestlevelof

recoveryintheself-

identifiedgoalareasof

housing?”

InalignmentwiththeMHP’snon-clinicalPIP,thisactivityfocusedoncorrectingforthedeficitthatemanatesfromclinically-focusedtreatmentplanning,anarrowfocusonsymptomsandimpairmentsofmentalillness.PositiveachievementofbeneficiarylifegoalscanbemissediftheMHPisnotorientedtotherehabilitativeservicesmodel.

ThisPIPnarrowlyfocusedontheeightbeneficiarieswhothroughtheStrengthsAssessmenthadidentifiedimprovedhousingasakeyareaforpersonalimprovement.

TheStrengthsAssessmentPIP,submittedtomeettheclinicalrequirement,essentiallyduplicatesanarrowaspectofthenon-clinicalStrengthsModelPIP,withthecaveatthatitwasfocusedonbeneficiarieswhohaveidentifiedhighestprioritylifegoalsinthehousingdomain.However,thatnarrowactivityappropriatelybelongsintegratedwiththenon-clinicalPIP.

TheTAprovidedtotheMHPbyCalEQROconsistedofonsitediscussionandpost-reviewfollow-up,providingtheMHPwiththeopportunitytoamendthenon-clinicalPIP.ThisclinicalPIPcouldbeafirstphaseofthestrengthsmodelimplementationofthenon-clinicalPIP.Furtherguidancetoincludethedirectinterventionsofstaffwithbeneficiarieswasalsoprovided.Additionalguidanceprovidedbyemailafterthereview,includingencouragementtodevelopanewclinicalPIPtopic.

CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B

47

PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET

DEMOGRAPHIC INFORMATION

MHP: � Clinical PIP � Non-Clinical PIP

PIPTitle:

Start Date (MM/DD/YYYY) ___________

Projected Study Period (#of Months) ________

Completed: _____ Yes ______ No

Dates of On-Site Review: ___________(MM/DD/YYY)

Name of Reviewer: ___________________________

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

� Active and ongoing (baseline established and interventions started)

� Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.

� Concept only, not yet active (interventions not started)

� Inactive, developed in a prior year

� Submission determined not to be a PIP

BriefDescriptionofPIP:


48

ACTIVITY1:ASSESSTHESTUDYMETHODOLOGY

STEP1:ReviewtheSelectedStudyTopic(s)

Component/Standard Score Comments

1.1 WasthePIPtopicselectedusingstakeholderinput?DidtheMHPdevelopamulti-functionalteamcompiledofstakeholdersinvestedinthisissue?

�Met

�PartiallyMet

�NotMet

�UnabletoDetermine

1.2 Wasthetopicselectedthroughdatacollectionandanalysisofcomprehensiveaspectsofenrolleeneeds,care,andservices?

�Met

�PartiallyMet

�NotMet


SelectthecategoryforeachPIP:Clinical:�Preventionofanacuteorchroniccondition �Highvolumeservices�Careforanacuteorchroniccondition �Highriskconditions

Non-Clinical:

�Processofaccessingordeliveringcare

1.3 DidthePlan’sPIPs,overtime,addressabroadspectrumofkeyaspectsofenrolleecareandservices?

Projectmustbeclearlyfocusedonidentifyingandcorrectingdeficienciesincareorservices,ratherthanonutilizationorcostalone.

�Met

�PartiallyMet

�NotMet


1.4 DidthePlan’sPIPs,overtime,includeallenrolledpopulations(i.e.,didnotexcludecertainenrolleessuchasthosewithspecialhealthcareneeds)?

Demographics:

�AgeRange�Race/Ethnicity�Gender�Language�Other

�Met

�PartiallyMet

�NotMet


Totals <#>Met <#> PartiallyMet <#> NotMet <#> UTD


49

STEP2:ReviewtheStudyQuestion(s)

2.1 Wasthestudyquestion(s)statedclearlyinwriting?

Includestudyquestionasstatedinnarrative:

<Text>

�Met

�PartiallyMet

�NotMet



STEP3:ReviewtheIdentifiedStudyPopulation

3.1 DidthePlanclearlydefineallMedi-Calenrolleestowhomthestudyquestionandindicatorsarerelevant?

Demographics:

�AgeRange�Race/Ethnicity�Gender�Language�Other

�Met

�PartiallyMet

�NotMet


3.2 Ifthestudyincludedtheentirepopulation,diditsdatacollectionapproachcaptureallenrolleestowhomthestudyquestionapplied?

Methodsofidentifyingparticipants:

�Utilizationdata �Referral �Self-identification

�Other: <Textifchecked>

�Met

�PartiallyMet

�NotMet



STEP4:ReviewSelectedStudyIndicators

4.1 Didthestudyuseobjective,clearlydefined,measurableindicators?

Listindicators:

<Text>

�Met

�PartiallyMet

��otMet



50

4.2 Didtheindicatorsmeasurechangesinhealthstatus,functionalstatus,orenrolleesatisfaction,orprocessesofcarewithstrongassociationswithimprovedoutcomes?

Arelong-termoutcomesimplied?�Yes �No

Orarelong-termoutcomesclearlystated? �Yes �No

�HealthStatus �FunctionalStatus

�MemberSatisfaction �ProviderSatisfaction

�Met

�PartiallyMet

�NotMet



STEP5:ReviewSamplingMethods

5.1 Didthesamplingtechniqueconsiderandspecifythetrue(orestimated)frequencyofoccurrenceoftheevent,theconfidenceintervaltobeused,andthemarginoferrorthatwillbeacceptable?

�Met

�PartiallyMet

�NotMet


5.2 Werevalidsamplingtechniquesthatprotectedagainstbiasemployed?

Specifythetypeofsamplingorcensusused:

<Text>

�Met

�PartiallyMet

�NotMet


5.3Didthesamplecontainasufficientnumberofenrollees?

______Nofenrolleesinsamplingframe

______Nofsample

______Nofparticipants(i.e.–returnrate)

�Met

�PartiallyMet

�NotMet




51

STEP6:ReviewDataCollectionProcedures

6.1 Didthestudydesignclearlyspecifythedatatobecollected?

�Met

�PartiallyMet

�NotMet


6.2 Didthestudydesignclearlyspecifythesourcesofdata?

Sourcesofdata:

�Member �Claims �Provider

�Other: <Textifchecked>

�Met

�PartiallyMet

�NotMet


6.3 Didthestudydesignspecifyasystematicmethodofcollectingvalidandreliabledatathatrepresentstheentirepopulationtowhichthestudy’sindicatorsapply?

�Met

�PartiallyMet

�NotMet


6.4 Didtheinstrumentsusedfordatacollectionprovideforconsistent,accuratedatacollectionoverthetimeperiodsstudied?

Instrumentsused:

☐Survey ☐Medicalrecordabstractiontool ☐Outcomestool☐LevelofCaretools☐Other: <Textifchecked>

☐Met☐PartiallyMet☐NotMet☐UnabletoDetermine

6.5 Didthestudydesignprospectivelyspecifyadataanalysisplan?



52

6.6 Werequalifiedstaffandpersonnelusedtocollectthedata?

Projectleader:

Name: <Text>

Title: <Text>

Role:<Text>

Otherteammembers:

Names: <Text>


Totals <#> Met <#> Partially Met <#> Not Met <#> UTD

STEP7:AssessImprovementStrategies

7.1Werereasonableinterventionsundertakentoaddresscauses/barriersidentifiedthroughdataanalysisandQIprocessesundertaken?

DescribeInterventions:

<Text>

�Met

�PartiallyMet

�NotMet


Totals <#>Met<#>PartiallyMet <#>NotMet<#>NA<#>UTD

STEP8:ReviewDataAnalysisandInterpretationofStudyResults

8.1 Wasananalysisofthefindingsperformedaccordingtothedataanalysisplan?

Thiselementis“NotMet”ifthereisnoindicationofadataanalysisplan(seeStep6.5)

�Met

�PartiallyMet

�NotMet

�NotApplicable


8.2 WerethePIPresultsandfindingspresentedaccuratelyandclearly?

Aretablesandfigureslabeled?�Yes �No

Aretheylabeledclearlyandaccurately? �Yes �No

�Met

�PartiallyMet

�NotMet

�NotApplicable



53

8.3 Didtheanalysisidentify:initialandrepeatmeasurements,statisticalsignificance,factorsthatinfluencecomparabilityofinitialandrepeatmeasurements,andfactorsthatthreateninternalandexternalvalidity?

Indicatethetimeperiodsofmeasurements:___________________

Indicatethestatisticalanalysisused:_________________________

Indicatethestatisticalsignificancelevelorconfidencelevelifavailable/known:_______%______Unabletodetermine

�Met

�PartiallyMet

�NotMet

�NotApplicable


8.4 DidtheanalysisofthestudydataincludeaninterpretationoftheextenttowhichthisPIPwassuccessfulandrecommendanyfollow-upactivities?

Limitationsdescribed:

<Text>

Conclusionsregardingthesuccessoftheinterpretation:

<Text>

Recommendationsforfollow-up:

<Text>

�Met

�PartiallyMet

�NotMet

�NotApplicable



STEP9:AssessWhetherImprovementis“Real”Improvement

9.1 Wasthesamemethodologyasthebaselinemeasurementused,whenmeasurementwasrepeated?

Ask: Werethesamesourcesofdataused?

Didtheyusethesamemethodofdatacollection?

Werethesameparticipantsexamined?

Didtheyutilizethesamemeasurementtools?

�Met

�PartiallyMet

�NotMet

�NotApplicable



54

9.2 Wasthereanydocumented,quantitativeimprovementinprocessesoroutcomesofcare?

Wasthere: �Improvement �Deterioration

Statisticalsignificance: �Yes �No

Clinicalsignificance: �Yes �No

�Met

�PartiallyMet

�NotMet

�NotApplicable


9.3 Doesthereportedimprovementinperformancehaveinternalvalidity;i.e.,doestheimprovementinperformanceappeartobetheresultoftheplannedqualityimprovementintervention?

Degreetowhichtheinterventionwasthereasonforchange:

�Norelevance �Small �Fair �High

�Met

�PartiallyMet

�NotMet

�NotApplicable


9.4 Isthereanystatisticalevidencethatanyobservedperformanceimprovementistrueimprovement?

�Weak �Moderate �Strong

�Met

�PartiallyMet

�NotMet

�NotApplicable


9.5 Wassustainedimprovementdemonstratedthroughrepeatedmeasurementsovercomparabletimeperiods?

�Met

�PartiallyMet

�NotMet

�NotApplicable




55

ACTIVITY2:VERIFYINGSTUDYFINDINGS(OPTIONAL)

Component/Standard Score Comments

Weretheinitialstudyfindingsverifieduponrepeatmeasurement?

�Yes

�No

o

ACTIVITY3:OVERALLVALIDITYANDRELIABILITYOFSTUDYRESULTS:SUMMARYOFAGGREGATEVALIDATIONFINDINGS

Conclusions: <Text>

Recommendations: <Text>

Check one: � High confidence in reported Plan PIP results � Low confidence in reported Plan PIP results � Confidence in reported Plan PIP results � Reported Plan PIP results not credible

fy 17–18 medi-cal specialty mental health external … pip... · the write-up does not contain a...

Documents