1
FY17–18MEDI-CALSPECIALTYMENTALHEALTHEXTERNALQUALITYREVIEW
PERFORMANCEIMPROVEMENTPROJECTS
BehavioralHealthConcepts,Inc.
5901ChristieAvenue,Suite502
Emeryville,CA94608
www.caleqro.com
855-385-3776
Preparedfor:
CaliforniaDepartmentofHealthCareServices(DHCS)
ForSiteVisitsConductedDuring:
April-June2018
2
TABLE OF CONTENTS
INTRODUCTION.......................................................................................................................................................................3
Table1.MHPsReviewedDuringApril–June2018.....................................................................................................................4
PERFORMANCEIMPROVEMENTPROJECTVALIDATION...........................................................................................5
Table2.PIPsSubmissionStandard.....................................................................................................................................................5Table3.PIPStatusDefined.....................................................................................................................................................................6Figure1.PIPSubmissionRates.............................................................................................................................................................7Table4.PIPTopicsforActiveandCompletedPIPSubmissions.............................................................................................8Table5.PIPTopicsforConceptOnlyPIPSubmissions...............................................................................................................9
FINDINGS......................................................................................................................................................................................................9
AccesstoCare...............................................................................................................................................................................................9TimelinessofCare....................................................................................................................................................................................10QualityofCare...........................................................................................................................................................................................10OutcomesofCare.....................................................................................................................................................................................11
CALEQRORATINGOFSUBMITTEDPIPS........................................................................................................................................12
Table6.PIPRatingSteps......................................................................................................................................................................13Table7.PIPRatingsDefined...............................................................................................................................................................13Table8.AveragePIPRatingsbyMHPSize...................................................................................................................................14
HISTORYOFPIPSUBMISSIONSBYMHP.......................................................................................................................................14
Figure2.PIPSubmissionHistory(FY2014-15toFY2017-18)...........................................................................................15Table9.ClinicalPIPSubmissionsbySmallRuralMHPs.........................................................................................................15Table10.Non-ClinicalPIPSubmissionsbySmallRuralMHPs............................................................................................16Table11.ClinicalPIPSubmissionsbySmallMHPs...................................................................................................................16Table12.Non-ClinicalPIPSubmissionsbySmallMHPs.........................................................................................................16Table13.ClinicalPIPSubmissionsbyMediumMHPs..............................................................................................................16Table14.Non-ClinicalPIPSubmissionsbyMediumMHPs....................................................................................................17Table15.ClinicalPIPSubmissionsbyLargeMHPs...................................................................................................................17Table16.Non-ClinicalPIPSubmissionsbyLargeMHPs........................................................................................................17
CONCLUSIONS/RECOMMENDATIONS...........................................................................................................................17
PIPTOPICS....................................................................................................................................................................................................17
PIPDESIGN/IMPLEMENTATION................................................................................................................................................................17
AreasforImprovement..........................................................................................................................................................................17RecommendationstoMHPs.................................................................................................................................................................18TechnicalAssistancetoMHPs............................................................................................................................................................19
APPENDICES..........................................................................................................................................................................20
CLINICALPIPTOPICSSUBMITTED.............................................................................................................................................................21
TimelinessofCarePIPs..........................................................................................................................................................................21QualityofCarePIPs.................................................................................................................................................................................23OutcomesofCarePIPs...........................................................................................................................................................................25
NON-CLINICALPIPTOPICSSUBMITTED....................................................................................................................................................31
AccesstoCarePIPs..................................................................................................................................................................................31TimelinessofCarePIPs..........................................................................................................................................................................35QualityofCarePIPs.................................................................................................................................................................................38OutcomesofCarePIPs...........................................................................................................................................................................39
CONCEPTONLY,NOTYETACTIVEPIPTOPICSSUBMITTED..................................................................................................................40
3
TimelinessofCarePIPs..........................................................................................................................................................................40QualityofCarePIPs.................................................................................................................................................................................42OutcomesofCarePIPs...........................................................................................................................................................................43
SUBMISSIONDETERMINEDNOTTOBEAPIP.........................................................................................................................................45
PERFORMANCEIMPROVEMENTPROJECT(PIP)VALIDATIONWORKSHEET.............................................................47
CalEQRO PIP Summary Report Q4 April - June 2018
3
INTRODUCTION
TheUnitedStatesDepartmentofHealthandHumanServices(DHHS),CentersforMedicareand
MedicaidServices(CMS)requiresanannual,independentexternalevaluationofStateMedicaid
ManagedCareprogramsbyanExternalQualityReviewOrganization(EQRO).ExternalQuality
Review(EQR)istheanalysisandevaluationbyanapprovedEQROofaggregateinformationon
quality,timeliness,andaccesstohealthcareservicesfurnishedbyPrepaidInpatientHealthPlans
(PIHPs)andtheircontractorstorecipientsofmanagedcareservices.CountyMentalHealthPlans
(MHPs)areconsideredPIHPsandarethereforesubjecttorulesgoverningPIHPs.CMSrules(42
CFR§438;MedicaidProgram,ExternalQualityReviewofMedicaidManagedCareOrganizations)
specifytherequirementsforevaluationofMedicaidManagedCareprograms.Theserulesrequire
anon-siterevieworadeskreviewofeachCountyMHP.
TheCaliforniaDepartmentofHealthCareServices(DHCS)contractswith56countyMHPsto
provideMedi-CalcoveredSpecialtyMentalHealthServices(SMHS)toMedi-Calbeneficiariesunder
theprovisionsofTitleXIXofthefederalSocialSecurityAct.
APerformanceImprovementProject(PIP)isdefinedbyCMSas“aprojectdesignedtoassessand
improveprocesses,andoutcomesofcarethatisdesigned,conductedandreportedina
methodologicallysoundmanner.”EachPIPisdesignedtoproducebeneficiary-focusedoutcomes.
TheValidatingPerformanceImprovementProjectsProtocol1specifiesthattheEQROvalidatetwoPIPsateachMHPthathavebeeninitiated,areunderway,orwerecompletedduringthereporting
year,orsomecombinationofthesethreestages.DHCSelectedtoexamineprojectsthatwere
underwayatsometimeduringthetwelvemonthsprecedingtheon-sitereview.
ThisreportpresentsasummaryofthePIPfindingsoftheon-sitereviewsconductedbythe
CaliforniaExternalQualityReviewOrganization(CalEQRO),BehavioralHealthConcepts,Inc.(BHC).
Thesummarycontainedinthisreportpertainstothereviewsthatwereconductedduringthe
fourthquarterofthe2017-18DHCSfiscalyear(FY)(April-June).Thisreportprovidessummary
informationtoDHCS,MHPs,andotherstakeholdersregardingthecompletenessofthePIP
submissionsreceivedbyCalEQROduringthequarter.EachPIPsubmissionissummarizedatthe
endofthereport.AnyfurtherinformationaboutaspecificPIPmaybeobtainedbyreviewingthat
MHP’sAnnualReport.
1DepartmentofHealthandHumanServices.CentersforMedicareandMedicaidServices(2012).ValidatingPerformanceImprovement
Projects:MandatoryProtocolforExternalQualityReview(EQR),Protocol3,Version2.0,December2012.Washington,DC:Author.
CalEQRO PIP Summary Report Q4 April - June 2018
4
ThissummaryreportincludesdatathatwasanalyzedandaggregatedbyCalEQROfromtheEQR
activitydescribedbelow:
VALIDATINGPERFORMANCEIMPROVEMENTPROJECTS
EachMHPisrequiredtoconducttwoPIPsduringthe12monthsprecedingthereview.ThesePIPs
mustbesubmittedtoCalEQROforreviewandscoringinaccordancewithaValidationTool
developedbyBHC(seeAppendixB).ThisValidationToolwascreatedbyCalEQROtoincludeall
requiredelementsofreviewfromtherelevantCMSProtocol.2
ThepurposeofaPIPistoassessandimprovetheprocessesandoutcomesofhealthcareprovided
byaMHP.
ThefollowingMHPssubmittedPIPsthatwerereviewedandscoredduringon-sitereviews
conductedbyCalEQROduringthemonthsofApriltoJune2018.TheresultsoftheseMHPreviews
aredescribedinthisreport.
Table 1. MHPs Reviewed During April – June 2018
Del Norte Inyo Lassen Modoc
Mono Napa Plumas Riverside
San Bernardino Santa Barbara Santa Cruz Siskiyou
Trinity Ventura
2Ibid.
CalEQRO PIP Summary Report Q4 April - June 2018
5
PERFORMANCE IMPROVEMENT PROJECT VALIDATION
ThefollowingtableillustratesthenumberofPIPsthatweresubmittedforvalidationthroughthe
CalEQROreviewbyeachMHPreviewedinApril-June2018.
Table 2. PIPs Submission Standard
MHP MHPSize
NumberofClinicalPIPs
Submitted
StatusofClinicalPIPsas
determinedbyCalEQRO
NumberofNon-Clinical
PIPsSubmitted
StatusofNon-ClinicalPIPsasdeterminedby
CalEQRO
Del Norte Small Rural 1 Active and Ongoing 1 Completed
Inyo Small Rural 0 Submission
Determined Not to be a PIP
1 Active and Ongoing
Lassen Small Rural 1 Completed 1 Active and Ongoing
Modoc Small Rural 1 Concept Only, Not Yet Active 1 Concept Only, Not
Yet Active
Mono Small Rural 1 Active and Ongoing 0 Submission
Determined Not to be a PIP
Napa Small 1 Active and Ongoing 0 No PIP Submitted
Plumas Small Rural 0 No PIP Submitted 0 No PIP Submitted
Riverside Large 1 Active and Ongoing 1 Active and Ongoing
San Bernardino Large 1 Completed 1 Completed
Santa Barbara Medium 1 Completed 1 Completed
Santa Cruz Medium 1 Completed 1 Active and Ongoing
Siskiyou Small Rural 1 Concept Only, Not Yet Active 1 Concept Only, Not
Yet Active
Trinity Small Rural 1 Concept Only, Not Yet Active 1 Active and Ongoing
Ventura Large 1 Completed 1 Completed
CalEQRO PIP Summary Report Q4 April - June 2018
6
Table 3. PIP Status Defined
ActiveandOngoing
Baselineestablishedonatleastsomeoftheindicators,andatleast
someinterventionshavestarted.Anycombinationoftheseis
acceptable.
CompletedInthepast12monthsorsincethepriorEQRtheworkonthePIPhas
beencompleted.
ConceptOnly,NotYet
Active
Baselinemayhavebeenestablished,butinterventionshavenot
started.ThisisNOTanactivePIP.
Inactive,Developedina
PriorYear
Ratedlastyearandnotratedthisyear.MHPhasdoneworkonit,but
ithasnotyetstarted,orithasbeensuspendedforsomereason.This
isNOTanactivePIP.
SubmissionDetermined
NottobeaPIP
Thewrite-updoesnotcontainaplan,data,and/orindicationwhere
datawillcomefrom.ThisisNOTanactivePIP.
Ofthe14MHPswhoseon-sitereviewswereconductedduringthemonthsofApril-June2018,8
areclassifiedasSmallRural,1isclassifiedasSmall,2areclassifiedasMedium,and3areclassified
asLarge.
Thirteenofthe14MHPsincludedinthisquarter’sreviewsubmittedsomeinformationtobe
consideredforvalidationofPIPs,however,only7MHPs(50percent)metthesubmissionstandard
thatrequiressubmissionoftwoactiveorcompletedPIPs.Oftheremaining7MHPs:
• MonoandNapaMHPsmettherequirementforsubmissionofanactiveorcompletedclinical
PIP,butdidnotsubmitanactiveorcompletednon-clinicalPIP;
• InyoandTrinityMHPsmettherequirementforsubmissionofanactiveorcompletednon-
clinicalPIP,butdidnotsubmitanactiveorcompletedclinicalPIP;
• Modoc,PlumasandSiskiyouMHPsdidnotsubmitanyactiveorcompletedPIPs.
Modoc,Siskiyou,andTrinityMHPssubmitteddocumentationforPIPsforwhichinterventionshad
notbeeninitiatedatthetimeoftheon-sitereview;thesePIPsareclassifiedasConceptOnly,Not
YetActive.InyoandMonosubmitteddocumentationthatwasdeterminednottoconstituteaPIP.
Additionally,Napadidnotsubmitanyinformationforanon-clinicalPIPandPlumasdidnotsubmit
anyinformationforeitheraclinicalornon-clinicalPIP.
CalEQRO PIP Summary Report Q4 April - June 2018
7
Figure 1. PIP Submission Rates
SmallRuralMHPs• 7of16requiredPIPssubmittedmetsubmissionstandards
o 5PIPswereratedasActiveandOngoingo 5PIPswereratedasConceptOnly,NotYetActiveo 2PIPswereratedasCompleteo 2SubmissionswereratedasSubmissionDeterminedNottobeaPIPo NothingwassubmittedforoneClinicalPIPo NothingwassubmittedforoneNon-clinicalPIP
SmallMHPs
• 1of2requiredPIPssubmittedmetsubmissionstandards
o 1PIPwasratedasActiveandOngoing
o NothingwassubmittedforoneNon-clinicalPIPMediumMHPs
• 4of4requiredPIPssubmittedmetsubmissionstandards
o 1PIPwasratedasActiveandOngoing
o 3PIPswereratedasComplete
LargeMHPs• 6of6requiredPIPssubmittedmetsubmissionstandards
o 2PIPswereratedasActiveandOngoing
o 4PIPswereratedasComplete
7
1
4
6
16
2
4
6
0
2
4
6
8
10
12
14
16
SmallRural(43.75%) Small(50%) Medium(100%) Large(100%)
PIPsmeetingsubmissionstandard
PIPsMeetingSubmissionStandard
PIPsRequired
CalEQRO PIP Summary Report Q4 April - June 2018
8
FivePIPsreceivedaratingof0percent.OneMHPdidnotsubmitanyinformationtobeconsidered
foreitheraclinicaloranon-clinicalPIP,andoneMHPdidnotsubmitanyinformationforanon-
clinicalPIP.TwoMHPs’submissionsweredeterminednottomeetthestandardsforaPIP.
Additionally,fiveConceptOnly,NotYetActivePIPswereratedfortechnicalassistance(TA)
purposesonly,andthoseratingswerenotfactoredintotheoverallratingsdescribedinthisreport
(seeTable8).
MHPsaddressedvarioustopicsandissuesinthePIPsthatweresubmittedforreview.Eighteen
PIPsratedasActiveandOngoingorCompleted,coveredtopicsthataddressthefollowingareas:
TimelinessofCare,AccesstoCare,QualityofCare,andOutcomesofCare.Asummaryofthe
informationprovidedtoCalEQROforallPIPsisprovidedattheendofthisreport.
Table 4. PIP Topics for Active and Completed PIP Submissions
PIPTopics
PIPTitles Clinical Non-Clinical
Access to Care
Psychiatry No-Show Study* Del Norte
Open-Access Scheduling and Kept Appointments Lassen Law Enforcement Co-located Triage, Engagement, and Support (TEST) Teams* San Bernardino
Beneficiary Acuity Index* Ventura
Timeliness of Care
Rapid Connect* Santa Cruz
Timeliness to Psychiatric Services* Santa Barbara
Timeliness of Access to Services Santa Cruz
Improving Timely Access to Services Trinity
Quality of Care
Improving Treatment: Training, Beneficiary Engagement and Team Based Care* Santa Barbara
Smoking Cessation* Ventura
Improving Engagement and Retention in Services Riverside
Outcomes of Care
Rehospitalization Rates Del Norte
Early Therapeutic Alliance & Retention* Lassen
Strengths Model Intervention for Employment Related-Goals Mono
Adult Social Engagement Napa
Follow-Up After Hospitalization Riverside
Complex Care Coordination* San Bernardino
Strengths Based Interventions Inyo
*CompletedPIPs
CalEQRO PIP Summary Report Q4 April - June 2018
9
Table 5. PIP Topics for Concept Only PIP Submissions
PIPTopics
PIPTitles Clinical Non-Clinical
Timeliness of Care
Timeliness Plan Modoc
Timely Access for Children and Youth Siskiyou
Quality of Care
Improving Beneficiary Outcomes through integrated treatment of Co-Occurring Disorders Modoc
Outcomes of Care
Initial Engagement and Retention in Children's Services Siskiyou
Improving Anxiety Levels of Beneficiaries Diagnosed with an Anxiety Disorder Trinity
FINDINGS
ManyPIPsaddresssimilartopicsasMHPsarefacingthesameissues.Thefindingsfurtherillustrate
thispoint.ThefindingsalsopertaintoMHPs’operationofaneffectiveManagedCareOrganization,
suchasMHPs’processesforensuringaccesstoandtimelinessofservices,andprocessesfor
improvingthequalityofcare.ThedetailsbelowreflectonlythosePIPsratedasActiveandOngoing
orCompleted.FormoreinformationregardingthePIPsdetailedbelow,pleaseseeAppendixAof
thisreport.
Access to Care
Fournon-clinicalPIPsfocusedonimprovingaccesstocareforbeneficiaries.
• DelNorte’sandLassen’snon-clinicalPIPsfocusedonimprovingno-showrates.
• SanBernardino’snon-clinicalPIPfocusedonco-locatinginlawenforcementsites.
• Ventura’snon-clinicalPIPfocusedonensuringthatbeneficiariesweregettingthe
levelofcarenecessaryfortheirindividualneeds.
DelNorteincreasedtelepsychiatrytoimproveno-showrates;however,thereductioninno-
showrateswasminimal.Lassenimplementedopenaccesstoimproveno-showrates;however,
theinterventioncenteredonbeneficiarieswhokepttheirappointmentsratherthan
amelioratingthebarriersforbeneficiarieswhowerenotabletokeepappointments.
SanBernardinoco-locatedateamofMHPclinicalstaffwithlawenforcementstafftoprovidea
moreappropriateand(clinically)informedresponsetolawenforcementcallsthatinvolve
residentswhopresentwithmentalhealthconcerns.Theresultsdemonstratethatco-locating
teamsledtoareductionininvoluntarypsychiatricholdsatallfourpolicedepartmentsusedfor
thisproject.
Ventura’sPIPaimedtoensurethatbeneficiariesidentifiedasfittingintothecategoriesofhigh,
moderateorlowneedswerereceivingalevelofcarelikelytomeettheirserviceneeds.The
CalEQRO PIP Summary Report Q4 April - June 2018
10
MHPtestedthisapproachwith12staffmembersandultimatelydecidedtoimplementit
throughouttheMHP.
Timeliness of Care
Oneclinicalandthreenon-clinicalPIPsfocusedonimprovingtimelinessofservicesfor
beneficiaries.
• SantaCruz’sclinicalPIPfocusedonensuringtimelyfollow-uptoservicesfor
individualsafterdischargefromtheCrisisStabilizationProgram(CSP).
• SantaBarbara’snon-clinicalPIPaimedatreducingthetimeittakesfornew
beneficiaries(adultandyouth)tohavetheirfirstappointmentwithapsychiatrist.
• SantaCruz’snon-clinicalPIPfocusedonimprovingthetimelinessfromfirstcontact
tofirstsession.
• Trinity’snon-clinicalPIPwasdesignedtoreducethenumberofdaysfromclaimed
assessmenttoclaimedappointment.
AnalysisoftheclinicalPIPfromSantaCruzfoundthatmanybeneficiariesfromtheCSPare
dischargedtoinpatienthospitalizationservices.TheEQROobservedthatCSPstendtohavethe
purposeofpreventingbeneficiariesfrominpatienthospitalization,andthatdoesnotseemtobe
workingwellinSantaCruz.
SantaCruz’snon-clinicalPIPhaddataanalysisissues,assomeclinicianswerenotrecordingthe
timeoffirstofferedappointment.Additionally,SantaBarbara’snon-clinicalPIPhadsix
interventionsandtheMHPfounditdifficulttomeasuretheeffectivenessofthemall.
AlthoughTrinity’sinterimanalysisrevealedatrendtowarddecreasedtimetoaccessservices,
inconsistenciesincollectingthedataoccurred.Staffwerenotconsistentindocumentingthe
timelines,whichimpededstandardizeddatacollection.
Quality of Care
TwoclinicalPIPsandonenon-clinicalPIPweredesignedtoimpactqualityofcare.
• SantaBarbaradevelopedandcompletedaclinicalPIPthatfocusedonimproving
beneficiaryexperienceoftreatment.
• VenturaimplementedaclinicalPIPthatfocusedonimprovingthehealthstatusof
beneficiarieswhousetobaccoproducts.TheMHPutilizedasmokingcessationprogram
titledCallItQuits(CIQ).
• Riversideimplementedanon-clinicalPIPwiththegoalofincreasingengagementand
retentionofchildrenincounty-operatedspecialtymentalhealthoutpatientclinics.
SantaBarbara’sclinicalPIPhadonlyoneclinicalintervention:Beneficiaryengagementintreatment
planning,theremaininginterventionswerenon-clinicalinnature.TheMHPdidnottrackallthe
interventionstheyimplementedaspartofthePIP,howeverthecombinedresultsindicated
CalEQRO PIP Summary Report Q4 April - June 2018
11
improvementorstabilityintheChildandAdolescentNeedsandStrengths(CANS)andMilestones
ofRecoveryScores(MORS)scores.
Ventura’sclinicalPIP’sresultshaveseennochangeinoutcomesforbeneficiaries.RelianceonCIQ
sessionsalone,evenifembeddedinclinics,seemsunlikelytoproducechangeunlessspecific
elementsaretailoredtotheMHP’spopulation.
Riverside’snon-clinicalPIPliststwointerventions:CollaborativeAssessmentandevening
availabilityforservices.However,bothinterventionslackedsufficientdetailsinthePIPnarrative
anddidnotincludesteps/activitiesthatwillbetakentoimplementthem.Priortoselectingthese
interventions,theteamappearstohavemissedthestepofdeterminingbeneficiaries’reasonsfor
lackofengagementandsubsequentlydroppingoutoftreatment.
Outcomes of Care
SixclinicalPIPsandonenon-clinicalPIPweredesignedtoimpactoutcomesofcare.
• DelNorte,RiversideandSanBernardinofocusedclinicalPIPsonimproving
outcomesrelatedtohospitalizations.
• LassenfocusedaclinicalPIPonimprovingearlytherapeuticalliance.
• MonofocuseditsclinicalPIPonimprovingbeneficiaries’employmentgoals.
• NapadesignedaclinicalPIPtoimprovesocialengagementforitsbeneficiaries.
• Inyo’snon-clinicalPIPaimstoimplementaStrengths-Basedapproachthat
incorporatesaprocessofassessment,planning,clinicalcasereviewandsupervision
ofstaff,andsupportprovidedtobeneficiariestoachieveidentifiedlife-goals.
DelNorte’sclinicalPIPproposedtheuseofanassessmentbyadrugandalcoholcounselor
withinfivedaysofacutepsychiatrichospitalizationtoreducerehospitalizationrates.This
approachyieldedan11percentimprovementintheirrehospitalizationrate.ThegoalofthePIP
istoaffordbeneficiarieswithfollow-upservicesasquicklyaspossibleposthospitalizationto
preventthetraumatizingeffectsofrehospitalizations.
Riverside’sclinicalPIPhadthegoalofincreasingbeneficiaryengagementinandaccessto
timelyoutpatientserviceswithinsevendaysfollowinghospitaldischarge,withfocuson
unengagedbeneficiarieswhoarenotalreadyknownbyandopentotheoutpatientmental
healthsystem.
SanBernardino’sclinicalPIPtargetedbeneficiarieswithcomorbidsomaticconditionswhohave
higherfrequencyandlongerdurationofpsychiatrichospitalizationscomparedtotheMHP’s
generaladultpopulation.TheMHPprovidedcoordinatedcarethataddressedbothchronic
mentalandphysicalillnessestoreducetherisk,frequency,anddurationofpsychiatric
hospitalizationattwoprogramswithintheMHP.Theprojecthadmixedresultsinthetwo
programswhereitwasimplemented,bothprogramssawareductioninratesofpsychiatric
hospitalization,butoneprogramsawanincreaseinriskofhospitalization.
Lassen’sclinicalPIPiscompletedandachievedsuccess;however,theproblemwasoriginally
describedasatimelinessproblemwhenitwastrulyanengagementproblem–beneficiaries
CalEQRO PIP Summary Report Q4 April - June 2018
12
droppingoutofservice.TheMHPdidnotconductabarrieranalysistodeterminecausesand
linkinterventionstodata.
AlthoughMonodescribedthevariousaspectsoftheStrengthsModel(SM)approach,includingthe
SMAssessment,PersonalRecoveryPlan,andgroupsupervisionofclinicalstaff,nospecific
interventionwasdescribedthatrelatedtothespecificactionsofstaffwithbeneficiaries,which
wouldseemtobeakeyelementofthismodel.
NapaexperiencedmanyissueswiththeimplementationandmeasurementoftheirclinicalPIP.Due
toseveralnaturaldisasters,includingafloodandwildfire,theMHPhadtopostponeseveralofthe
interventionactivities.Specifically,themeasurementofonekeyindicatorprovedtobedifficult,as
ittakesapproximatelysixmonthsfromitsadministrationtoreceivethedata.
Inyo’snon-clinicalPIPhasthepotentialtopositivelyaffectbeneficiaryoutcomesinareas
identifiedbythebeneficiary,includinghousing,employment,andeducation.However,theMHP
hasnotcompletelyimplementedthisPIPyetandoutcomemeasurementsarestillpending.
CALEQRO RATING OF SUBMITTED PIPS
ThetablebelowliststheValidationItemsthatareratedforeachPIPbyCalEQRO.AllPIPsarerated
basedontheircompletenessandcompliancewiththestandard,therefore,PIPsubmissionsthat
wereratedasConceptOnly,NotYetActive(anddidnotreceiveratingsforeachPIPstep)arenot
includedinthetabulations,figures,andtablesinthissection.AllPIPsreceivingaratingof0percent
(i.e.,PIPsratedas:SubmissionDeterminedNottobeaPIP,Inactive,andNoPIPSubmitted)arealso
notincludedinthetabulations,figures,andtablesinthissection.Assuch,thereare22PIPs
representedinthefiguresandratingstables.
ThestandardsarefoundintheCMSPIPProtocol:ValidationofPerformanceImprovement
Projects.3WithineachoftheninePIP“Steps”therearesubsectionsthatareratedaccordingtothe
PIPValidationTool(seeAppendixB).
32012DepartmentofHealthandHumanServices,CentersforMedicareandMedicaidServiceProtocol3Version2.0,December2012.
EQRProtocol3:ValidatingPerformanceImprovementProjects.
CalEQRO PIP Summary Report Q4 April - June 2018
13
Table 6. PIP Rating Steps
Step PIPSection
1 Selected Study Topics
2 Study Question
3 Study Population
4 Study Indicators
5 Improvement Strategies
6 Data Collection Procedures
7 Analysis and Interpretation of Study Results
8 Review Assessment of PIP Outcomes
9 Validity of Improvement
AllPIPsub-sectionsreceivearatingofMet;PartiallyMet;NotMet;NotApplicable;orUnabletoDetermine.
Table 7. PIP Ratings Defined Met Credible, reliable, and valid methods for the item were documented.
Partially Met Credible, reliable, or valid methods were implied or able to be established for part of the item.
Not Met Errors in logic were noted or contradictory information was presented or interpreted erroneously.
Not Applicable Only to be used in Steps 7-9 when the study period was underway for the first year.
Unable to Determine
The study did not provide enough documentation to determine whether credible, reliable, and valid methods were employed.
AratingofMetorPartiallyMetweighspositivelyintotheOverallAverageRatingreceivedbythe
PIP.EachMetitemreceivestwopoints,whileeachPartiallyMetitemreceivesonepoint.The
OverallAverageRatingforeachPIPiscalculatedwiththefollowingformula:
("#$%&')&* × 2) + ("#$%&'/0'*10223)&*)"#$%&'456772180%2&9*&$: × 2
CalEQROusedtheformulareferencedabovetocalculatearatingforeachofthenineStepsinthe
PIPValidation.ThenanoverallratingwasgiventoeachPIPandthendividedbytheTotal
ApplicablePIPsteps.PIPsubmissionsthatwereratedasConceptOnly,NotYetActive,and
thereforedidnotreceiveratingsforeachPIPstep,arenotincludedinthetabulationsinthetables
inthissection.AllPIPsreceivingaratingof0percent(i.e.,PIPsratedas:SubmissionDetermined
NottobeaPIP,Inactive,andNoPIPSubmitted)arerepresentedinthedenominatorofthe
tabulations.
CalEQRO PIP Summary Report Q4 April - June 2018
14
TheMHPsreviewedduringApriltoJune2018receivedthefollowingoverallratings:
Table 8. Average PIP Ratings by MHP Size
MHPSize
Clinical
Non-Clinical
Small Rural 70.45% 62.21%
Small 52.63% NA
Medium 77% 76%
Large 70.95% 67.65% TheaverageratingsreceivedbyMediumMHP’sclinicalPIPsarehigherthanthosereceivedbyall
otherMHPs.
• ThePIPratingsforMediumMHPswerehigherthantheratingsforallothersizedMHPs.
• Duetotheratingsbeinganaverage,thePIPratingsforSmallMHPsonlyreflectstheoneMHPof
thatsizethathadanon-sitereviewduringthisperiod.
HISTORY OF PIP SUBMISSIONS BY MHP
CalEQROhasbeenvalidatingPIPsubmissionsfromMedi-CalMHPssinceFY2014-15.CalEQROhas
providedsubject-basedTAon-site,viaemail,telephone,video,andwebinar.However,numerous
MHPshavesubmittedPIPsthatdidnotmeetthesubmissionstandardofhavingtwoActiveand
OngoingorCompletedPIPsforeachreviewyear.AlthoughtheseMHPsarecontractuallyrequired
tomeetthesubmissionstandards,theycontinuetocitestaffingissues,competingpriorities,and
limitedresourcesasreasonsfornotmeetingthisrequirement.
ThefigurebelowillustratesthesubmissionhistoryofeachMHPrepresentedinthisreport:
CalEQRO PIP Summary Report Q4 April - June 2018
15
Figure 2. PIP Submission History (FY 2014-15 to FY 2017-18)
ThespecificsofthesubmissionsreceivedbytheMHPsrepresentedinthisreportareasfollows:
Table 9. Clinical PIP Submissions by Small Rural MHPs
MHPFY2014-15Clinical
FY2015-16Clinical
FY2016-17Clinical
FY2017-18Clinical
Del Norte Active and Ongoing Concept Only, Not Yet Active
Concept Only, Not Yet Active Active and Ongoing
Inyo No PIP Submitted No PIP Submitted Submission
Determined Not to be a PIP
Submission Determined Not to
be a PIP
Lassen No PIP Submitted No PIP Submitted Submission
Determined Not to be a PIP
Completed
Modoc Active and Ongoing Active and Ongoing Completed Concept Only, Not Yet Active
Mono Active and Ongoing Concept Only, Not Yet Active
Concept Only, Not Yet Active Active and Ongoing
Plumas No PIP submitted No PIP submitted Active and Ongoing No PIP Submitted
Siskiyou No PIP Submitted Active and Ongoing Concept Only, Not Yet Active
Concept Only, Not Yet Active
Trinity Active and Ongoing Concept Only, Not Yet Active
Concept Only, Not Yet Active
Concept Only, Not Yet Active
4
3
2
5
3
2
3
8 8
7
8
2
5
6
0
1
2
3
4
5
6
7
8
Del Norte Inyo
Lassen
ModocMono
Napa
Plumas
Riversi
de
San Bern
ardino
Santa
Barbara
Santa
Cruz
Siskiy
ouTri
nity
Ventura
Number of Active or Completed PIPs Submitted
PIPs Submitted
Number of PIPs required = 8
CalEQRO PIP Summary Report Q4 April - June 2018
16
Table 10. Non-Clinical PIP Submissions by Small Rural MHPs
MHPFY2014-15Non-Clinical
FY2015-16Non-Clinical
FY2016-17Non-Clinical
FY2017-18Non-Clinical
Del Norte Active and Ongoing Submission Determined Not to be a PIP
Submission Determined Not to be
a PIP Completed
Inyo Active and Ongoing Submission Determined Not to be a PIP Completed Active and Ongoing
Lassen No PIP Submitted Concept Only, Not Yet Active
Submission Determined Not to be
a PIP Active and Ongoing
Modoc Active and Ongoing Concept Only, Not Yet Active Active and Ongoing Concept Only, Not Yet
Active
Mono Active and Ongoing Submission Determined Not to be a PIP
Concept Only, Not Yet Active
Submission Determined Not to be
a PIP
Plumas Active and Ongoing Completed No PIP Submitted No PIP Submitted
Siskiyou No PIP Submitted Active and Ongoing Submission
Determined Not to be a PIP
Concept Only, Not Yet Active
Trinity Active and Ongoing Active and Ongoing Active and Ongoing Active and Ongoing
Table 11. Clinical PIP Submissions by Small MHPs
MHPFY2014-15Clinical
FY2015-16Clinical
FY2016-17Clinical
FY2017-18Clinical
Napa No PIP submitted Concept Only, Not Yet Active
Concept Only, Not Yet Active Active and Ongoing
Table 12. Non-Clinical PIP Submissions by Small MHPs
MHP FY2014-15Non-Clinical
FY2015-16Non-Clinical
FY2016-17Non-Clinical
FY2017-18Non-Clinical
Napa Concept Only, Not Yet
Active Concept Only, Not Yet
Active Active and Ongoing No PIP Submitted
Table 13. Clinical PIP Submissions by Medium MHPs
MHPFY2014-15Clinical
FY2015-16Clinical
FY2016-17Clinical
FY2017-18Clinical
Santa Barbara Active and Ongoing Concept Only, Not Yet Active Active and Ongoing Completed
Santa Cruz Active and Ongoing Active and Ongoing Active and Ongoing Completed
CalEQRO PIP Summary Report Q4 April - June 2018
17
Table 14. Non-Clinical PIP Submissions by Medium MHPs
MHPFY2014-15Non-Clinical
FY2015-16Non-Clinical
FY2016-17Non-Clinical
FY2017-18Non-Clinical
Santa Barbara Active and Ongoing Active and Ongoing Active and Ongoing Completed
Santa Cruz Active and Ongoing Active and Ongoing Completed Active and Ongoing
Table 15. Clinical PIP Submissions by Large MHPs
MHPFY2014-15Clinical
FY2015-16Clinical
FY2016-17Clinical
FY2017-18Clinical
Riverside Active and Ongoing Completed Active and Ongoing Active and Ongoing
San Bernardino Active and Ongoing Completed Active and Ongoing Completed
Ventura Active and Ongoing Completed Concept Only, Not Yet Active Completed
Table 16. Non-Clinical PIP Submissions by Large MHPs
MHP FY2014-15Non-Clinical
FY2015-16Non-Clinical
FY2016-17Non-Clinical
FY2017-18Non-Clinical
Riverside Active and Ongoing Completed Active and Ongoing Active and Ongoing
San Bernardino Active and Ongoing Completed Active and Ongoing Completed
Ventura Active and Ongoing Active and Ongoing Inactive, Developed in a Prior Year Completed
CONCLUSIONS/RECOMMENDATIONS
DuringtheFY2017-18annualreviews,CalEQROfoundstrengthsinMHPprogramsandpractices
thathaveasignificantimpactontheoveralldeliverysystemanditssupportingstructure.Inthose
sameareas,CalEQROalsonotedopportunitiesforqualityimprovement.
PIP TOPICS
CalEQROobservedthat7ofthe18(39percent)PIPsratedasActiveandOngoingorCompleted,
focusedonOutcomesofCareissues.
PIP DESIGN/IMPLEMENTATION
Areas for Improvement
• 47percentofthesubmissions(16of34)requestedbyCalEQROfortheApril-June2018on-site
reviewsdidnotmeettheActiveandOngoingorCompletedstandardasrequiredforPIP
submissions.
CalEQRO PIP Summary Report Q4 April - June 2018
18
o FiveofthosesubmissionswereratedasConceptOnly,NotYetActiveandwerein
variousstagesofimplementation.Onceinterventionsareimplemented,thePIPswill
beconsideredActiveandOngoing.
o OneMHPdidnotsubmitdocumentationforoneoftherequiredPIPs.
o OneMHPdidnotsubmitdocumentationforbothrequiredPIPs.
o Twosubmissionsdidnotmeettheminimumrequirementsandwereratedas
SubmissionDeterminedNottobeaPIP.
• EightPIPswereratedasCompleted.BHCemphasizedtheneedforcontinuedPIPdevelopment.
MHPsshouldnotlimitthemselvestodevelopingnewPIPswhenpreviousonesarecompleted;they
shouldconsiderPIPdevelopmentfromacontinuousqualityimprovementprocessperspective.
• 41percentofthesubmissions(46of112)requestedbyCalEQROforeachofthepriorfourFYsdid
notmeettheActiveandOngoingorCompletedstandardasrequiredforPIPsubmissions.
o 21submissionswereratedasConceptOnly,NotYetActive.
o 14PIPswerenotsubmitted.
o 10submissionswereDeterminedNottobeaPIP.
o 1PIPwasratedasInactive,DevelopedinaPriorYear.
• ThreeMHPs(Lassen,Napa,andSiskiyou)submittedonlytwoPIPsthathavemettheActiveand
OngoingorCompletedstandardduringthepastfourreviewyears.
• ThreeMHPs(Inyo,Mono,andPlumas)submittedonlythreePIPsthathavemettheActiveand
OngoingorCompletedstandardduringthepastfourreviewyears.
Recommendations to MHPs
• CalEQROcontinuestorecommendthatMHPsfosteracultureofcontinuousquality
improvementthroughouttheirorganizations.
o PIPideasshouldbegeneratedfromongoingeffortstoimprovebeneficiary
outcomes,asMHPsshouldfocusonbeneficiaryoutcomesversusorganizational
improvements.
o PIPideasshouldcomefromanyareaoftheMHPthatdirectlyimpactsbeneficiaries.
o MHPsshouldconsiderareasinwhichtodevelopPIPsonacontinuousbasis.Ifan
issuethatrequiresimprovementhasapotentialimpactonbeneficiaryoutcomes,
theMHPshouldconsiderhowaPIPcouldbedevelopedtoimprovetheissue.
• MHPsshoulddevelopaplanandputitintoaction.ActiveandOngoingPIPsarethestandardby
whichtheMHPsareevaluated.
o PIPsshouldhavemechanismsforcollectingdataquarterly,ataminimum,and
shouldhavenewactivitiesoccurringonaregularbasis.
o TheCMSprotocolrequiresatleastonenewinterventioneveryyearifan
unsuccessfulPIPistocontinue.
• PIPsshouldinvolvebeneficiaryfeedbackasmuchaspossible;beneficiaries’inputcanbe
valuableindeterminingthedirectionandinterventionsofPIPs.
o MHPsshoulddevelopPIPteamsthatarespecifictotheissuestheyareaddressing,
includingsubjectmatterexpertsandbeneficiariesasappropriate.• MHPsshouldensurethattheyhaveasolidfoundationonwhichtodesignaPIP.Todoso
requiresbackgrounddataandanalysisofbarrierspriortotheimplementationofaPIP.
CalEQRO PIP Summary Report Q4 April - June 2018
19
• ItisimperativethattheMHPsparticipateinTAfromCalEQROtoimprovetheirabilitytocollect,
analyze,andusedatathathelpestablishtheneedforaPIPanddevelopameasurablestudy
question.
o PIPClinicsareofferedtoallMHPsonaquarterlybasis.
o ResourcesareavailableontheCalEQROwebsite,including:
§ PIPinstructionalvideos4
§ PIPLibrary5
§ InstructionsforCompletingPIPValidationTool6
• MHPsshouldcontactCalEQROforassistanceindevelopingPIPs;TAisavailableforallMHPs
outsideoftheon-sitereview.
Technical Assistance to MHPs
CalEQROworkedindividuallywitheachMHPwhileonsitetoprovideTAinthedevelopmentand
progressionoftheirPIPs.AdditionalTAwasprovidedattherequestofMHPs.Phonesessionswere
conductedwithMHPspriortoandfollowingon-sitereviewsasrequested.Thesephonesessions
arespecificforeachMHPbutinclude:assistancewithdefiningaproblemwithlocaldata;aidin
writingaPIPstudyquestion;andhelpwithfindingappropriateinterventions,outcomesand
indicators.
CalEQROpresentedaPIPCliniconJune21,2018thataddressedtipsforsuccessfulPIPsand
reviewedcommontopicselection,indicatorsandinterventions.Questionandanswersessionswere
conductedduringthispresentation.AllMHPswereinvitedandencouragedtoparticipateinthe
presentation.Therecordingofthiswebinarandthepresentationmaterialsusedareavailableon
CalEQRO’swebsite.
CalEQROhasrecordedthreePIPinstructionalvideosandhascollectedsuccessfulPIPsinaPIP
Librarythatisavailableonourwebsiteatwww.caleqro.com.
4http://www.caleqro.com/data/california_eqro_resources/PIP%20Library/YouTube%20-%20BHC%20PIP%20101%20-%201%20-
%20California%20Drug%20MediCal%20-%20Bringing%20Ideas%20to%20Successful%20PIP%20Concept.html5http://caleqro.com/#!california_eqro_resources/PIP%20Library6http://www.caleqro.com/data/california_eqro_resources/PIP%20Library/Instructions%20for%20Completing%20PIP%20Validation
%20Tool-PUBLIC_v.3.docx
CalEQRO PIP Summary Report Q4 April - June 2018
20
APPENDICES
AppendixA:SummaryofPIPssubmittedbyMHPs
AppendixB:CalEQROPIPValidationTool
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 21
CLINICAL PIP TOPICS SUBMITTED
Ofthe14ClinicalPIPsrequiredforsubmission,3wereconsideredActiveandOngoingand5wereCompleted.ThreewereratedasConceptOnlyanddidnothaveinterventionsimplementedatthetimeoftheon-sitereview.OnesubmissionwasdeterminednottobeaPIP.OnePIPwasnotsubmittedbytheMHP.AllthePIPssubmittedaresummarizedhereinthisAppendix.
Timeliness of Care PIPs
• RapidConnect(Completed)
SantaCruz
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Towhatextentwillthe
implementationofRapid
Connect,whichincludes
directcontactwith
beneficiaries/families
(foryouth)seenatthe
CrisisStabilization
Program(CSP)or
immediatephone
contactafterdischarge,
reducethenumberof
readmissionstotheCSP
within30days?”
SantaCruz’sclinicalPIPfocusedonensuringtimelyfollow-uptoservicesforindividualsafterdischargefromtheCSP.
ItinvolvesRapidConnect,aservicetofacilitaterapidfollow-upforbeneficiarieswhowereadmittedtoCSP.MobileEmergencyResponseTeam(MERT)membersorstafffromCountyAccesswillcontacttheindividualwithinonebusinessdaypostCSPstaytodeterminetreatmentneedsandlinkthemtoservices.
Therewrittenstudyquestionstilllacksspecificitythatwouldbeusefulinmeasuringoutcomes.Asecondinterventionwasnotaddedinthepastreviewyear,asrecommendedbyEQROintheFY16-17review.Thegoalofindicatornumberoneis75percent,andforindicatornumbertwo10percent.
ThestudybeganNovember2016andtheMHPconsidersthisPIPtobecompleteasofNovember2017.
EQROagreedwiththeMHPstaffpresentingthePIPthattheclinicalPIPhasthenecessarycomponentsandcanbeconsideredcompleted.
ThefollowingitemswerediscussedwiththeMHP:
Rewritethestudyquestiontomakeitmeasurableandspecific.
Expanduponthedescriptionoftheinterventions,whichwerenotdescribedsufficientlyinthewrite-upwithinthesectioninthePIPDevelopmentToolmarked“describetheintervention”.
ChangethepercentageimprovementforeachquarterinRapidConnectcontactssoitreferencesthepercentofbaselineratherthanthepercentofthetargetedgoalamount.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 22
Redothecalculationmethodforchangeinpercentofreadmissionssothelast-quartercomparisondoesnotincludethosetoimmediatelybehospitalizedfollowingdischargefromtheCSP.TheinterventionwasonlyforthosedischargedfromtheCSPtothecommunity,andthesewereaminorityofthosedischarged.
Clearlyarticulateinsignificantdetailalltheinterventionsthatwillbeimplemented.Additionalinterventionsareneededtoimproveclinicaloutcomesandanswerthestudyquestionasrelatedtobenefittobeneficiaries.
AddoutcomemeasuresatkeyeventstostrengthenthisPIP.DeterminewhatthethresholdisforfrequencyofadministeringtheCANSandmonitorforimprovementinthedataovertime.
Consideradditionalinterventionsthatmightincludelinkageswithposthospitalizationfollow-upactivitiesandstaff.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 23
Quality of Care PIPs
• ImprovingTreatment:Training,BeneficiaryEngagementandTeamBasedCare(Completed)
SantaBarbara
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Arebeneficiaryoutcomes,
asmeasuredbythe
ConsumerPerception
Survey(CPS),CANSand
MORS,improvedby
implementing:1)training
forclinicalstaff,2)the
team-basedcaremodel
andtoolsand3)improved
MIStreatmentrelated
reports(formanagersand
supervisors)?”
ThisPIPfocusedonimprovingbeneficiaryexperienceoftreatmentintermsof:a)ensuringthatallbeneficiarieshavehighqualitycurrent/activetreatmentplans;b)implementingteambasedcare;andc)improvingbeneficiaryengagement.Thehypothesiswasthatimprovedexperienceoftreatmentwillresultin:improvementsinspecificoutcomesasmeasuredbytheCPS,CANS(Youth)andMORS(Adult)scores.
TheonlyclinicalinterventionspresentedbytheMHPwerebeneficiaryengagementintreatmentplanningandteam-basedcaretraining(whichinandofitselfisnotanintervention).NotalltheinterventionsweretrackedaspartofthePIP,butthecombinedresultsindicatedimprovementorstabilityintheCANSandMORSscores.ElementsoftheCPSwerealsoreviewed,butthisdatawasnotwelllinkedtotheinterventionsinthisPIP.
TheTAprovidedtotheMHPbyCalEQRO,meanttoaddressareasforopportunityandfuturePIPs,consistedofsuggestionstovalidateassumptionsaboutthecauseofproblemsthroughareviewoftherelevantliteratureandmorethoroughbarrieranalysis;toassurethatallpersonsimpacted,especiallylinestaffandbeneficiary/familymembersareapartofthePIPactivestakeholdergroup;andtoidentifyvariablesnotpartofthePIPstudythatcouldimpactthechangesinbeneficiaryoutcomesbeingtracked.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 24
• IntegratingSmokingCessationProgramsintoaBehavioralHealthSystem(Completed)
Ventura
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willintegrationof
smokingcessationservices
withinVCBHdecreasethe
proportionofbeneficiaries
whodescribethemselves
asactivetobaccousers.
Addendum(5/7/2017):
Willintegrationofsmoking
cessationserviceswithin
VCBHresultinthe
percentageoftobacco
usersremainingquitfor3
mos.,6mos.and12mos.
aftercompletingaCallIt
Quits(CIQ)programbe
least25%,15%and10%
respectively?”
BeginninginJune2016,theMHPfocusedonimprovingthehealthstatusofbeneficiarieswhousetobaccoproducts.Thenationaldataonsmokingprevalencewassupportedbyasamplesurveyofadultbeneficiariesacrossallsites.Thesurveyvalidatedhightobaccouse(46percent),three-quartersofwhomwantedtostopsmoking.
Thebroad,positiveimpactsofthisactivitytodateincludetheMHP’sdevelopmentofaprocesstoroutinelyaskbeneficiariesabouttobaccouse,encourageandsupportsmokingcessation,andcollectionofthatdataaboutsmokingwithintheelectronichealthrecord(EHR).Thesuccessfulscreeningofsmokingstatusfor71percentoftheadultpopulationisanaccomplishment.Thedevelopmentofareferralprocess,coupledwitheffectiveinterventionsforthispopulation,followedbytrackingofresults,isimportant.
Itmustbeacknowledgedthatcurrentresultsshownochangeforbeneficiaries.RelianceonCIQsessionsalone,evenifembeddedinclinics,seemsunlikelytoproducechangeunlessspecificelementsaretailoredtotheMHP’spopulation.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionsaboutthelimitednumberofbeneficiariesimpactedbythisimprovementactivityandtheabsenceofpositiveresults.Thatsaid,thescreeningaspecthasbeenverysuccessful.StrongconsiderationwasgiventoterminationofthisPIPandexploringanotherissuethatcouldhavegreaterimpact.However,thisisanimportanttopic,involvingalargepercentageofMHPadultbeneficiaries.Thereremainsastrongrationaleforcontinuation,iftheMHPadjustsitsinterventionstrategytoimproveefficacyforitsuniquepopulation.TheMHPmayconsiderfurtherliteraturereviewforprovenstrategiesthatcouldinvolvespecificsupportiveapproachestosmokingcessationnotcurrentlyutilized.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 25
Outcomes of Care PIPs
• RehospitalizationRates(ActiveandOngoing)
DelNorte
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willanassessmentbya
drugandalcohol
counselorwithinfive
daysofacutepsychiatric
hospitalizationreduce
therehospitalizationrate
withinayearforDel
NorteCountyfrom12
percentto7percent?”
ThePIPaddressestheissueoftherehospitalizationrateforMedi-CalbeneficiariesinDelNorteCounty.ThisrateishigherthanthestateaverageperdatafromCalEQRO.ThegoalofthisPIPistoreducethenumberofrehospitalizationadmittancestothestateaverageorlower.TheMHPacknowledgeshowtraumatizingandstigmatizingpsychiatrichospitalizationscanbe,evenjustonetime,andthegoalofthePIPistoimproveoutcomesfromthefirstdischargeofhospitalizationandreducetheneedforasecondhospitalization.
TheMHPpresentedastatisticallysignificantdecreaseusingthecommonp-valueoffivepercent.However,thiscalculationwasusingthegiven18.8percentimprovement,whichwasdeterminedtobeinaccurate.Theactualimprovementis11percent.
Moreinformationisexpectedandwillbeavailableasthestudycontinues.Thereisnocompleteanalysisoffindingscurrently.
ThePIPwillbecontinued,andanadditionalinterventionwillbeadded.Discussiononsitelookedatwhatthenewinterventionmightbe,althoughnoconclusionwasreached.
TheTAprovidedtotheMHPbyCalEQROconsistedofonsitediscussionoffindingsofthePIP.CalEQROpointedoutthatthereisaneedformoredatatobeenteredintothestudyandmoreanalyzationofthefindings.Noexternalthreatstovaliditywerediscussedinthewrittenstudy.ThePIPfindingsneedtobeanalyzedonaquarterlybasisataminimum.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 26
• EarlyTherapeuticAlliance&Retention(Completed)
Lassen
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willearlytherapeutic
allianceimprove
beneficiaryretentionby
25percentasmeasured
bythepercentageof
individualsretained
fromassessmentto
initialtherapy
appointment?”
Toaddresstheidentifieddelayoftherapeuticallianceanditsimpactonretention,theMHPlookedatbarrierstoearlytherapeuticalliance.Inareviewofbeneficiaryretentionin2016,theMHPfounda54percentdropoutratefromclinicalassessmenttoinitialtherapyappointment.Onebarrier,identifiedbybeneficiariesandclinicians,wastherapistimpermanence.
Improvingbeneficiaryretentioninservicesthroughbettertherapeuticalliancewillimprovequalityofcare.However,continuedretentioninongoingserviceswillbedependentonotherfactors.Theseinclude,butarenotlimitedto,thefrequencyofongoingservicesandifwaittimesbetweenappointmentsarenotlengthy.
TheTAprovidedtotheMHPbyCalEQROconsistedofadiscussionongeneralPIPdevelopment.WhilethisPIPiscompletedandachievedsuccess,itwasaresubmissionbasedupononsitequestionsduringthereview.Forexample,howdidtheMHPmakethedecisiontoselecttheintervention?Wasthereresearchintothepossiblecausesoftheproblem?HowdidtheMHPdeterminesamplesizewhenevaluatingtheproblem?Onsitediscussionclarifiedthattheproblem,originallydescribedasatimelinessproblem,wastrulyanengagementproblem–beneficiariesdroppingoutofservice.CalEQROdiscussedtheusefulnessofbarrieranalysistodeterminecausesandlinkinterventionstodata.Datarelatedtotheproblemshouldbequantifiedintermsofscopeandsizeandshouldbegatheredbeforeimplementinginterventions.TheMHPwasadvisedtousethemostcurrentPIPsubmissionform(theyhadsubmittedcurrentPIPsonlastyear’sform).
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 27
• StrengthsModelInterventionforEmploymentRelated-Goals(ActiveandOngoing)
Mono
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“WillusingtheStrengthsModelhelpbeneficiariesmakeprogresstowardtheiremployment-relatedgoals,asmeasuredbytheachievementoftheiremployment-relatedgoalsasrecordedintheStrengthsAssessmentoverthetwo-yearstudyperiod?”
TheMHPdevelopedthisPIPfromcommunitysurveydatainwhichmembersofthecommunity,includingsomecurrentandpastbeneficiaries,identifiedlifedomaingoalsinwhichsupportwasdesired.TheMHPutilizedtheSMassessmentwith14highneedbeneficiariesanddiscoveredthat11hademploymentoreconomicgoalsforlifeimprovement.
TheMHPdescribedthevariousaspectsoftheSMapproach,includingtheSMAssessment,PersonalRecoveryPlan,andgroupsupervisionofclinicalstaff.Nospecificinterventionwasdescribedthatrelatedtothespecificactionsofstaffwithbeneficiaries,whichwouldseemtobeakeyelementofthismodel.
Thestudyquestiondidnotprovidetheanticipatedquantifiableimprovementgoal,asrequiredforaPIP.Thestudyindicatorincludesachievementofemployment-relatedgoals.However,thedatatablebreaksoutseparatenumeratorsforthosewhoachieveemploymentgoalsanddeclinetosetanothergoalfromthosewhoachievethelistedgoalandthensetanewgoal.Sinceachievementofemploymentseemstobethegoalofthisactivity,themeetingofthisgoalwouldseemsufficient.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionofneededelementstoaddtothePIPandresubmissionoftheupdate.CalEQROsharedhowthelackofbaselinedata,lackofspecificityofwhatdefinesinclusioninthestudygroupbywayofbeing“stuck”or“highneed”isproblematic.OverthecourseofthisnextreviewperiodtheMHPneedstoadddataelementssuchastheserviceutilizationlevelofthese“high-need”individuals.Theinclusionofbeneficiariesseemstolackspecific,definedquantifiableparametersthatwouldsupportreplication.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 28
• AdultSocialEngagement(ActiveandOngoing)
Napa
• Follow-UpAfterHospitalization(ActiveandOngoing)
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“IfNapaCountyMental
Healthintroducesaseries
ofsocialengagement
activities,particularly
targetingthemostisolated
beneficiaries,willit
increasethenumberof
activelyengaged
individuals?”
TheMHPdeterminedthroughitsanalysisofitsCPSdatathatonechallengingareareportedbytherespondentsissocialisolationorlackofsocialengagement.Basedonthisfinding,theMHPlaunchedthisPIPtoreducesocialisolationandimprovesocialengagement.Theprimaryinterventionshaveconsistedofincreasedsocialactivitiesspecificallytargetedforthechronicallysociallyisolatedindividuals.
Duetoseveralnaturaldisastersinthepast16months,floodandwildfire,theMHPhadtopostponeseveraloftheinterventionactivities.Inaddition,onekeyindicatorobtainedfromCPSisdelayedasittakesapproximatelysixmonthsfromitsadministrationinNovemberfortheMHPtoreceivethedata.Consequently,theMHPwasnotabletoprovideanyoutcomesfromtheactivitiesthatindeedtookplacein2017.CalEQROhasthereforefoundmanyofthePIPvalidationitemstobenotapplicableatthistime.
OnsiteTAandCalEQROfeedbackconsistedofthefollowing:
PIPQuestion:TheMHPwasinformedthatthePIPquestionshouldoutlineandlinktheinterventionandtheintendedoutcomes.Followingtheonsitereview,theMHPsubmittedarevisedPIPquestionthatmetthestandards.
Indicatorsandvalidityoffuturefindings:CalEQROadvisedtheMHPthatthestudymethodologyshouldhavethepowertodetectthechangesamongtheintendedbeneficiaries.Currently,theCPSmethodologyismoregeneric,andthesamplereflectstheoveralladultbeneficiaries.CalEQROrecommendsthattheMHPconsidermorefrequentadministrationofCPSamongthetargetbeneficiaries.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 29
Riverside
• ComplexCareCoordination(Completed)
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willtheimplementation
ofnavigationstrategies
withinpatienttreatment
facility(ITF)discharge
teamsincreasethe
percentageofunengaged
beneficiarieswhoreceive
anoutpatientfollow-up
servicewithinsevendays
ofdischarge?”
ThegoalofthisclinicalPIPistoincreasebeneficiaryengagementinandaccesstotimelyoutpatientserviceswithinsevendaysfollowinghospitaldischarge,withfocusonunengagedbeneficiarieswhoarenotalreadyknownbyandopentotheoutpatientmentalhealthsystem.Theinterventionusespeerspecialiststhroughthepeer-runnavigationcentertoengagethesebeneficiariesandlinkthemtoservicesincludingtherapy,casemanagement,medicationmanagement,housing,detox,andothersupports.
Thestudyquestionwouldbestrengthenedbyaddingameasurabletarget.Theindicatorgoalsaretoolowtodemonstratestatisticallysignificantchange,andthereforeanychangemeasuredcannotbeattributedtothisinterventionalone.Thecurrentinterventionrequiresamoredetailedexplanationoftheclinicalactivitiesthatwilltakeplacetoengageandprovideservicestothetargetpopulationofunengagedbeneficiariesnewlydischargedfromaninpatientfacility.
TheTAprovidedtotheMHPbyCalEQROconsistedofadiscussiononthedifferencesbetweenclinicalandnon-clinicalPIPs,andsuggestionsforstrengtheningthisclinicalPIP.Asdiscussedonsite,itwouldbehelpfultoaddadescriptionofhowtheengagementactivitiesprovideadirectlinktoclinicalservicesandincludethecompositionoftheITFdischargeteam(clinicalandnon-clinicalstaff:peerspecialists,therapists,casemanagers,psychiatristsandnurses,etc.).DuringtheonsitediscussiontheMHPdidprovideadditionalclinicalinformationandjustificationwhichCalEQROagreeswith.TheMHPhasagreedtorewritethePIPDevelopmentOutlineandwillbeengaginginfurtherconsultationwithCalEQROinthecomingmonths.
CalEQRO PIP Summary Report Q4 April - June 2018
Clinical PIP topics submitted 30
SanBernardino
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willcomplexcare
coordinationreducethe
risk,frequency,and
durationofpsychiatric
hospitalizationsby20
percentforpsychiatrically
andmedicallycomplex
beneficiarieswhorequire
themostintensivecare
coordinationservices?”
TheMHPhasidentifiedapopulationofbeneficiarieswithcomorbidsomaticconditionswhohavehigherfrequencyandlongerdurationofpsychiatrichospitalizationscomparedtotheMHP’sgeneraladultpopulation.ThegoalofthisPIPistoprovidecoordinatedcarethataddressesboththechronicmentalandphysicalillnessestoreducetherisk,frequency,anddurationofpsychiatrichospitalization.
ThePIPteamprovidedindicators(whichwerethesameastheiroutcomemeasures)thatwereobjectiveandmeasurable.However,therewerenoindicatorstoaddresstheperformanceoftheteaminapplyingthem.Indicatorsareneededthatcompare:
• actualhome/fieldvisitstoneededhome/fieldvisits
• actualaccompanimenttomedical/pharmacyappointmentsneededaccompaniment
• frequencyofinquiryintomedicationcompliance
• andothers
TheTAprovidedtotheMHPbyCalEQROconsistedoftherecommendationtooperationalizethecomponentsofcomplexcarecoordinationandtoarticulatethedifferencesintheprojectfromoneyeartothenext,whichtheMHPdid.CalEQROandtheMHPdiscussedtheMHP’splansforafutureclinicalPIP—reducingpolypharmacythroughexaminationofantipsychoticprescriptionsandimprovingcontinuityofcarebyimprovingcommunicationbetweeninpatientandoutpatientpractitioners.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 31
NON-CLINICAL PIP TOPICS SUBMITTED
Ofthe14non-clinicalPIPsrequiredforsubmission,5wereratedasActiveandOngoing;4wereratedasCompleted;2wereratedasConceptOnly,NotYetActiveanddidnothaveinterventionsimplementedatthetimeoftheon-sitereview.Additionally,onesubmissionwasDeterminedNottobeaPIPandtwoPIPswerenotsubmitted.
Access to Care PIPs • PsychiatryNo-ShowStudy(Completed)DelNorte
StudyQuestion
(aspresentedby
MHP)
FocusofPIPAreasfor
ImprovementTAProvidedbyCalEQRO
“Willno-showrates
decreasefor
beneficiariesfrom
FY2015-16to
FY2016-17witha
changefromlocum
psychiatry(e.g.,a
newdoctorevery
threemonths)to
telepsychiatry(i.e.,
thesamedoctorfor
patientforatleast
oneyear)?”
Afterincreasingpsychiatryservicesin2016throughlocumpsychiatrists,itwasfoundthatno-showratesandformalgrievancesagainstpsychiatryservicesincreased.Theprevioustelepsychiatristswerestillinplaceandneitherno-showsnorformalgrievancesshowedanincreaseforthoseproviders.TheMHPdecidedtoendthecontractwithlocumpsychiatryandincreasetelepsychiatry.ThegoalpresentedbytheMHPwastoachieveconsistencyinserviceprovision,as“consistentpsychiatryleadstobetterpatientoutcomesandlessno-showsorbeneficiarycancellation.”TheresultsofthatchangearethefocusofthisPIP.
TheMHPhypothesisincludesthatwhenbeneficiariesdonotgetthecaretheyneed,especiallywithpsychiatry,negativeoutcomesarepredictedtooccur,includingsymptomsworseningandpossibleneedforhigherlevelsofcare(e.g.psychiatrichospitalizations).TheinterventionwouldbetoincreasethehoursofpsychiatrywithKingsViewtelepsychiatryusingdoctorswhowillbearoundforayearormore.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionoftheneedtodoathoroughbarrieranalysisbeforedesigningaPIP.CalEQROalsonotedthatmeasuringresultsonceaquarterataminimumwouldbeusefultoseeifthePIPneedsanyadjustmentsmovingforward.TheMHPplanstomonitorno-showratesforallservicesmonthlyandwilltrackthisinterventionandadjustaccordingly.Issueswithcalculatingpercentversuspercentagepointswerepointedout.TheMHPwasofferedongoingTAforcreatinganewPIP;thisPIPisconsideredcomplete.TheMHPwasencouragedtoconsultwithEQROearlyandoftenduringPIPformulationforitsnextnon-clinicalPIP.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 32
• Open-AccessSchedulingandKeptAppointments(ActiveandOngoing)
Lassen
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Doesimplementingopenaccessschedulingforoutpatientservicesimprovekeptappointmentratesbytenpercentasmeasuredbyattendance?”
TheMHPfocuseditsnon-clinicalPIPonaddressingtheno-showrateforscheduledinitialandfollow-upappointmentsthroughwalk-inaccessandopenaccessscheduling.TheMHPidentifiedopenschedulingasaviableinterventionthroughaliteraturereview.Openschedulingallowsfortheschedulingofnextvisitappointmentsonly,ratherthanbookingseveralappointments,whichresultsinleavingnospaceforotherbeneficiaries.Thepercentageofmissedappointmentsaffectsotherimportantsystemandbeneficiary-centeredfactors.
TheMHPdidnotprovideanexplanationforthedisconnectionbetweentheinterventionandtheproblem.ThePIPattemptstosolvetheproblemofno-showswithopenaccessschedulingbutdoesnotinquireindepththereasonsforno-shows.Theinterventioncenteredonbeneficiarieswhokepttheirappointmentsratherthanamelioratingthebarriersforbeneficiarieswhowerenotabletokeepappointments.Abarrieranalysiswouldhaveledtoamoreimpactfulintervention.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionsonhowtobetterdevelopPIPtopicsandimprovethesuccessofinterventions.Inthiscase,abarrieranalysisonreasonsforno-showswouldbetterinforminterventions.Toensurebothvalidityandreliability,thedatacollectionplanshouldspecifythedatatobecollected,sources,collectionmethodsincludingpersonnel,andtheinstrumentsused.TherewerealsoelementstothePIPwhichwereunder-developedduetotheMHPusingtheincorrectPIPsubmissiontool.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 33
• LawEnforcementCo-locatedTriage,Engagement,andSupport(TEST)Teams(Completed)
SanBernardino
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willco-locatingTriage,
Engagement,andSupport
Teams(TEST)atlaw
enforcementagencies
decreaseinvoluntaryholds
writtenbylawenforcement
andreceivedbyARMCby15
percent,whilealso
decreasingpsychiatric
hospitalizationsfor
beneficiariesservedbythe
TESTstaffby20percentand
increasingroutineoutpatient
psychiatriccareby25
percent?”
ThisisthesecondyearoftheMHP’sprojectonTEST,ateamofMHPclinicalstaffwhoareco-locatedwithlawenforcement.ThegoalofthePIPwastouseTESTtoprovideamoreappropriateand(clinically)informedresponsetolawenforcementcallsthatinvolveresidentswhopresentwithmentalhealthconcerns.
TheresultsofthestudyshowthattheTESTwassuccessfulinreducing5150sbytheFontana(by54.55percent)andRialtopolicedepartments(by28.00percent),thetwodepartmentsinurbancommunities.TheTESTdecreased5150sbytheMountainRegionpolicedepartmentby12.14percent.TheTESTdecreased5150sinVictorvilleonlymarginally(by0.84percent),towhichtheMHPpartiallyattributedtonewdeputiesjoiningthelawenforcementagencies.FollowingtheTESTencounter,beneficiaryhospitalizationsdecreased,andoutpatientservicesincreasedoverall.
TheTAprovidedtotheMHPbyCalEQROconsistedofrecommendationtoadjustthepre-andpost-measurementperiod(fromsixmonthstofourmonths)toenabletheMHPtohavecompletedatatocompareforthisreview,whichtheMHPdid.CalEQROandtheMHPdiscussedtheMHP’splansforfuturenon-clinicalPIPs.TheMHPdiscussedtheirintentionstoaddressthelowrateof7-dayfollow-upsposthospitalization.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 34
• BeneficiaryAcuityIndex(Completed)
Ventura
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willanacuityindexderived
fromthenature/extentof
pastpsychiatric
hospitalization(s)provide
staffwithapracticaland
effectiveguidetoservice
delivery?Willprovidingstaff
withreferencetoanobjective
measureofacuity,alongwith
associatedservicesdelivery
expectations,increase
productivityandcaseload
coveragetherebyreducing
psychiatrichospital
admissionsinthelong-term?”
TheaimofthisPIPwastoensurethatbeneficiariesidentifiedasfittingintooneofthecategoriesofhigh,moderate,low,oruncategorized,werereceivingalevelofcarelikelytomeettheirserviceneeds.TheMHPthendevelopedaminimumservicelevelwhichwaspairedtotheneedcategories.TheMHPalsoestablishedmechanismsforthebeneficiaryandstafftoprovidefeedbacktotheservicelevelsandscoring.
TheMHPdeterminedtodiscontinuethisPIPinJanuaryof2018.TheMHPbelievesitispreparedtomakesystemdecisionsbasedonthisbrieftestprocess(11months).TheadditionofalevelofcareinstrumenttosupportacomprehensiveprocessguidingservicedeliveryacrossalllevelsofneedandhelpingdetermineaconsumerflowintorecoverywouldimprovethisPIP.Furthermore,approximately50percentoftheMHP’sadultconsumersareuncategorizedduetolackofrecenthospitalizationhistory,whichmaylimittheapplicationofthisapproach.
TheTAprovidedtotheMHPbyCalEQROconsistedofonsiteencouragement,suggestingthattheuseoftestedandvalidatedlevelofcaretoolswouldprovideadditionaldatatoaugmentthecurrentapproach.TheuseofinstrumentssuchasMORS,LOCUS,andANSAprovidebroadapplicability.Theuncategorizedbeneficiaries,forwhomtheMHPlacksaspecificserviceanalyticstrategy.FormanyMHPs,thelargerchallengeisdevelopingstandardsforsatisfactorylevelofimprovementforsteppingdownservicelevels.ThisPIPwasmorebasicinitsapproach,seekingtoassurehighlevelneedbeneficiariesaresoidentifiedandreceiveaminimumlevel.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 35
Timeliness of Care PIPs
• TimelinesstoPsychiatricServices(Completed)
SantaBarbara
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willimplementingthesix(6)
interventionsofthePIP:1)
appointmentremindercalls;2)
team-basedappointment
management;3)increased
clinical/peercontactspriorto
psychiatricassessment;4)
implementationofasingular,
standardizedappointment
schedulingsystemthroughout
thecounty;5)enhanced
recruitmentofpsychiatrists
andphysicianassistants;and
6)incentivizingpsychiatrists’
productivity,resultin:a)a
reductioninbeneficiaryno-
showratesandb)areduction
inwaittimebetween
admissionandfirstpsychiatric
appointmentintheadultand
children’ssystemsofcare?”
ThisPIPistryingtoreducethetimeittakesfornewbeneficiaries(adultandyouth)tohavetheirfirstappointmentwithapsychiatrist.ThePIPtestsseveralstrategiestodeterminewhichisthemosteffective.Theinterventionsinclude:appointmentremindercalls,team-basedappointmentmanagement,increasedclinical/peercontactpriortothefirstpsychiatricappointment,implementationofasingular,standardizedappointmentschedulingsystemthroughoutthecounty,enhancedrecruitmenteffortsforpsychiatristsandphysicianassistants,andincentivizingpsychiatrists’productivity.
Thisnon-clinicalPIPhadoriginallyplannedaninterventiontoimplementanelectronicscheduler.ThiswasnotputinplaceduetovacanciesinITstaffandsomestaffresistance.Theelectronicschedulerimplementationwouldimprovefuturetrackingofno-showsandincreaseprovidercapacity.
TheTAprovidedtotheMHPbyCalEQRO,meanttoaddresscurrentopportunitiesandfuturePIPs,consistedofsuggestionstostaggerinterventionstobetterunderstandtheimpactofanyspecificintervention;limitthefocusandnumberofinterventionsinPIPs,andassurethatallthepersonsimpacted,especiallylinestaffandbeneficiary/familymembers,areapartofthePIPactivestakeholdergroup.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 36
• TimelinessofAccesstoServices(ActiveandOngoing)
SantaCruz
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willdisseminationof
accuratetimelinessdataand
systemwideavailabilityof
resourcesassistMHPto
insurebeneficiariesreceive
timelyservicesuponinitial
request?”
ThePIPisfocusedonimprovingthetimelinessfromfirstcontacttofirstsession.Therationaleisthattimelinesswillreduceadverseeventstobeneficiarieswhoaresufferingandinneedoftreatmentandhastentheirroadtorecovery.
Asstatedthestudyquestiondoesnotspecificallyaddressabeneficiarybenefitofmoretimelyresponsetorequestforservicesoraccuratetimelinessdatadissemination.
Itwasfoundthatsomeclinicianswerenotrecordingtimeoffirstofferedappointment.ThePIPstudyteamdecidedthatasinglemeasurecouldencompassboth.Theysurmisedthatthetimeoffirstofferedappointmentwouldsometimesbesoonerthanfirstactualappointment,andatthelatestwouldcoincidewiththedateoffirstactualappointment.Therefore,bytakingtheearlieroffirstofferedappointment(whenthedataisavailable)orfirstactualappointment,timelinesswouldbesufficientlymeasured.
TheTAprovidedtotheMHPbyCalEQROconsistedofthefollowingfeedback:
Thechangeinmethodstoreducewaitlistsislikelyhelpfulbutisnotinitselfameasure.ThePIPmustmeasuretheimpactoftimelinessonthebeneficiariesstudied.Thismeansitisnecessarytomeasurewhatwasthechangeintimefromfirstcontacttofirstofferedappointment,andfromfirstcontacttofirstactualappointment?
CalEQROdidnotrecommendblendingthemeasuresoffirstofferedappointmentandactualappointmentdates.Itisrecommendedtheyarekeptseparate.
CalEQROalsorecommendedanalyzingforpatternsthefrequencyofmissingdatafordateoffirstofferedappointment.Inthatway,theMHPcanascertainwhattreatmentprogramshavethemostmissingdataandcanconducttargetedtrainingsothenumbersdecrease.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 37
• ImprovingTimelyAccesstoServices(Completed)
Trinity
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willchangingtheinternal
“accessmeeting”processfor
approval,andmodifyingthe
settingandtrackingof
clinicianappointmentsresult
inanimprovementof
timelinessofservicesfor
actualclaimedassessmentto
actualclinicalappointment
(assessmenttoclinical
appointment)froman
averageof21daystoan
averageof12days,a43
percentreduction?”
Thegoalofthisnon-clinicalPIPistoreducethenumberofdaysfromactualclaimedassessmenttoactualclaimedappointment fromanaverageof21daystoanaverageof12days,a43percentreduction.ThePIPwillevaluateallthevariousstepsintheworkflow,identifyareasofbottlenecksordelaysandimplementinterventionstoimprovethetimeliness.
Interimanalysisrevealedthatalthoughtherewasatrendtowarddecreasedtimetoaccessservicesinconsistenciesincollectingthedataoccurred.Staffwerenotconsistentindocumentingthetimelinesandstandardizationwasnotachievedthatimpacteddata.TheMHPindicatedaneedtocontinueitseffortstocollectadditionaldataandamorerobustreview.ThereisconcernthattheMHPcollectedsmallnumbersinitson-goingdatacollection.Thisshouldhavebeenaddressedtoeradicatetheproblemandtodeterminetheeffectsoftheimprovementintended.ThereliabilityindocumentingthestepsamongstproviderstaffisalsoaquestionthattheMHPwillneedtoaddress.
TheTAprovidedtotheMHPbyCalEQROconsistedofencouragingtheMHPtocompetetheactionsidentifiedfordatacollection,andcontinuetomakeimprovements,andwasencouragedtoseekthisearlyintoitsconceptandtocontinuetoseeksupportinitswrite-ups.
However,sincethisisthesecondyearofsubmittingthisPIPwithlimitedactivitiesanddataprovided,theMHPisadvisedtoinitiateanewPIPforthenextreviewcycleforrating.CalEQROalsodiscussedthetimelinesofPIPactivities,encouragingtheMHPtoidentifyitsPIPearlyintheprocess,implementinterventions,andcollectandanalyzetherelevantdatetoidentifynecessaryadjustmentssothattheratingforthePIPisacceptedasactive.TheMHPwasofferedongoingTAandtheMHPfollowedupwithanemailupdateandindicatedconsultationwillbescheduledintheupcomingmonths.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 38
Quality of Care PIPs
• ImprovingEngagementandRetentionServices(ActiveandOngoing)
Riverside
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Doestheuseofa
collaborativeassessment
processdecreasenoshow
ratesby25percentand
increaseby20percent
continuedengagementin
servicesfollowingtheinitial
assessmentwithcontinued
engagementdefinedasat
leasttwoserviceswithin30
daysoftheinitial
assessment?”
Thegoalofthisnon-clinicalPIPistoincreaseengagementandretentionofchildrenincounty-operatedspecialtymentalhealthoutpatientclinicsbydemonstratingthattheinterventions(collaborativeassessment,andavailabilityofeveninghours)workinoneclinic,andthenscalingthemuptootherclinics.ThedatausedforthisPIPincludedthepercentageofbeneficiarieswithfewerthanfiveservices,andtheaverageapprovedclaimsperbeneficiarywhichshowedapatternoffewerservicesperbeneficiarythanotherlargecountiesandthatstatewide.
ThePIPliststwointerventions,collaborativeassessmentandeveningavailabilityforservices.Bothlackadetaileddiscussionandsteps/activitiesthatwillbetakentoimplementthem.Priortoselectingtheseinterventions,theteamappearstohavemissedthestepofdeterminingbeneficiaries’reasonsforlackofengagementandsubsequentlydroppingoutoftreatment.Interventionsshouldbeselectedbasedonthisinformationtofullyaddressthebarriersbeneficiariesareexperiencing.
TheTAprovidedtotheMHPbyCalEQROconsistedofadiscussiononthedifferencesbetweenclinicalandnon-clinicalPIPs,andsuggestionsforstrengtheningthisnon-clinicalPIP.Asdiscussedonsite,theMHPhasagreedtoreviewthefeedbackinthePIPValidationToolandupdatetheirPIPDevelopmentOutlineinthecomingmonthstoreflectthefeedbackandtheirquarterlydataanalysesandchangesmadetotheinterventions.TheMHPwillseekfurtherconsultationwithCalEQROafterthisreporthasbeenfinalized.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 39
Outcomes of Care PIPs
• StrengthsBasedInterventions(ActiveandOngoing)
Inyo
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willimprovingthecontent
&structureofgroup
supervisionsessions
utilizingtheUniversityof
KansasStrengthsModel
groupsupervisiontoolsand
methodologyresultinmore
beneficiariesachieving
theirself-identifiedgoals
relatedtoliving
arrangements,vocational
status,educationalstatus,
hospitalizations,or
successfulcompletionand
exitfromservices?”
TheMHPhasengagedwithathreecounty,EasternSierra,collaborativeprojectimplementingtheStrengths-BasedModel,whichisoutoftheUniversityofKansasandsupportedbytheCaliforniaInstituteofBehavioralHealth(CIBH).TheMHPdetermineditlackedaclearprocessthatidentifiedandtrackedbeneficiaryprogresstowardsidentifiedlifegoals,includingthoseaspectsthatwereoutsideofpureclinicalindicatorprogress.TheMHPfurtherexploredtheaspectofpersonallifegoalsthroughareviewofbeneficiaryrecordsanddiscoveredthatveryfewhadidentifiedormadeprogresswithimprovedhousing,employment,education,emergencyroomvisits,psychiatrichospitalizations,andgraduationfromservices.
Thestudyquestionaswrittendoesnotclearlyandsuccinctlyidentifywhatisbeingdonedifferentlywithbeneficiaries,asrequiredinaPIP,thedetailsofthatinteraction,anddoesnotproposehowmuchofachangeisexpected.
Thelistofinterventionsrelatestouseofthestrength’smodelassessment,supervision,anduseofreporttotrackpotentialbeneficiarygains.Aswritten,thiswouldbedifficulttoreplicate,forthePIPdoesnotspecifythe‘whatandhow’interventionsarebeingdonewithbeneficiariesthroughthelensofthismodel,andhowthestaff-beneficiaryinteractionisbeingchangedtoimprovelikelihoodofbeneficiaryattainmentofgoal.Theseelementsarecriticaltocorrectgoingforward.
TheTAprovidedtotheMHPbyCalEQROconsistedofonsiteandpre/postinteractionsspeakingtohowthisPIPandtheclinicalPIPutilizedthesametopic–Strength-BasedModel–andwereduplicative.Thenon-clinicalPIPappearstohavebroaderinterventionelementsandfocusthatwouldsupportutilizationforseveralreviewcyclessolongasthefocusandinterventionscontinuedtogrowandchangeovertime.Thestudyquestionrequirestheadditionofaquantifiableelement,andtheinterventionsrequirespecificinformationregardingthespecificinterventionsusedintheclinician/beneficiaryinteractiontoaccomplishchange.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 40
CONCEPT ONLY, NOT YET ACTIVE PIP TOPICS SUBMITTED
Timeliness of Care PIPs
• TimelinessPlan(ConceptOnly,NotYetActive)
Modoc–Non-clinical
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willdesignating“access”
cliniciansdecrease
beneficiarywaittimefor
behavioralhealthnon-
urgentinitialcontactand
firstassessmentas
measuredbytimeliness
measures?”
ThePIPfocusesontimelinessfornon-urgentappointments.Reasonsidentifiedfordelayedtimelyaccesstoservicesincludedlackofsufficientcliniciansandlargecaseloaddistributions,bothofwhichareaddressedthroughtheproposedinterventionofappointing“access”clinicians.Accessclinicianswillbepoisedtoacceptnon-urgentcall-insorwalk-insastheypresentatthemainclinic.Thecurrentbaselineforaccessis11businessdays.TheMHP’sstandardistenbusinessdays.
Aswritten,theMHPisplanningtobegintheirinterventionin2019.However,thatwouldmeanagapoftenmonthswithoutanactivePIP.Indiscussion,theMHPsaidtheywouldlikelybestartingearlieroncetheyimplementanewscreeningandtriagesystem.Alsodiscussedwasthatwhileworkingtowardsthegoalofcomplyingwithamandatedmeasure,PIPsshouldbeapproachedintermsofbeneficiarybenefit.Athoroughbarrieranalysisshouldbecompletedsopatternscanbeidentified(i.e.,specificstafforclinic)todeterminewhattypesoftargetedinterventionsareneeded.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionregardingtherequirementtohavetwoactivePIPsandreviewingthecriteriaforactivePIPs(havingactive/newinterventionseachyearforunsuccessfulPIPs).Giventhatthebaselinefortimelyaccessis11businessdays,andclosetothestandardof10businessdays,continuationofthisPIPisdependentonthebarrieranalysisandwhetherituncoversabarrierthatpotentiallyaffectsmanybeneficiaries.Datashouldbeanalyzedatleastquarterly,thoughmoreoftenisrecommendedtoimplementchangesbeforetheendofayear.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 41
• TimelyAccessforChildrenandYouth(ConceptOnly,NotYetActive)
Siskiyou–Non-Clinical
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Canschedulingwith
contractedproviderRemi
Vistaanytimethereisnotan
availableMHPappointment
slotwithin10daysresultin:
• Anincreaseintheproportionof
beneficiariesbeingoffered
anassessmentwithin10
businessdaysfrom42
percenttoagoalof90
percent;
• Areductionintheaveragenumberofdaysbetween
referralandactual
assessmentfrom19.4to
12.0;andultimately;and
• Anincreaseinthepercentageofchild/youth
beneficiarieswhoattend
anassessmentfroma
baselineof80percenttoa
goalof95percent?”
ThePIPintendstoofferassessmentappointmentswithintendaysofcontactandtotracktheaveragenumberofbusinessdaysbetweenreferralandassessment.Tocomplywiththeten-daystandard,theMHPwillreferyouthtoitsorganizationalprovider,RemiVista,fortheassessmenttobecompleted.ThisPIPisdesignedtoimprovetimelyassessmentratesforchildrenandyouthwhoareneworreturningtoservices.
AlthoughtheMHPhassubmittedthisasanon-clinicalPIP,theprojectisattheconceptonlystage.Theinterventionshavenotbeenappliedanddatahavenotbeencollected.TheMHPwillneedtoimplementinterventions,providethefollowupdata,andanalyzethedatatosupportitspremisethatimprovedtimelinesstoassessmentsoccurred.SincetheMHPdiscussesboththetimelinessandengagementstrategiesforbothPIPsforthesameagegroup,itiscriticaltocollectdatafordifferentinterventions.TheMHPwasadvisedtocontinuetodefineseparateelementsforeachPIPtodistinguishthetwo.TheMHPwillalsoneedtodefinewhatitexpectstoachieveforbeneficiarybenefitwhichappearstobeidentifiedasreducedwaittimeseveniftheoptiontorefertoRemiVistaisnotutilized.
TheTAprovidedtotheMHPbyCalEQROconsistedofpre-sitetelephonediscussionandtheon-sitediscussionoftheelementsthatwouldhelpinacceleratingtheactivitiesofthePIPprocess.PostreviewphonecallsweremadeaswellforTA.ElementsofthePIPprocessthatwereemphasizedconsistedoftheneedfortheinterventions,thedatacollectionplan,andthebenefittobeneficiaries,alltobemeasurable.TheMHPwasencouragedtoconsultwithCalEQROearlyandoftenduringPIPformulationsandtheMHPrecognizedthisoption.FurtherTAhasbeenscheduled.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 42
Quality of Care PIPs
• ImprovingBeneficiaryOutcomesthroughintegratedtreatmentofCo-OccurringDisorders(ConceptOnly,NotYetActive)
Modoc-Clinical
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willimplementationof
theDrugUseScreening
QuickInventory(revised)
(DUSI-R)improvequality
ofcareandaccuracyof
diagnosesasmeasuredby
co-occurringdiagnosis
ratesandbeneficiary
retentionrates?”
TheMHPreportedaco-occurringdiagnosisrateof6percentduringtheFY16-17review.Retrospectivelytheywereunabletoduplicatethelowrate,butinsteadcalculatedtherateat21.03percent.TheMHP’sgoalremainstoincreasetheco-occurringreportingratesothatitismoreinlinewithnationalstandards.
TheMHPwasadvisedtoconductabarrieranalysistodiscoverwhatledtothelowrateofco-occurringdisordersandinaccuratediagnoses.
TheMHPplanstouseanassessmenttoolforSUDandmentalhealthtocaptureco-occurringdiagnoses.However,theoverarchinggoalshouldbetoaccuratelydiagnosebeneficiaries.Otherwise,agoalofincreasingtheidentificationanddocumentationofco-occurringdisorderscouldleadtooverdiagnosingco-occurringdisorders.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussiononwaystoinsurevalidityandreliabilityoftheirefforts.Thedatacollectionplanshouldspecifythedatatobecollected;thedatasources;howandwhenthedataaretobecollected;whowillcollectthedata;andinstrumentsusedtocollectthedata.TheMHPstatedthattheyhadrecentlyprovidedadditionaltrainingforstaffindatacollection.Asaresult,theMHPwasadvisedtoseeiftherearecurrentissueswithreportingofdiagnosesandconsiderthepossibilitythatcurrentdatamaynotindicateaproblemwarrantingaPIP.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 43
Outcomes of Care PIPs
• InitialEngagementandRetentioninChildren’sServices(ConceptOnly,NotYetActive)
Siskiyou-Clinical
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
Canthefollowingresultinanincreasein
thepercentageofchildrenandyouth
whoreceiveatleastoneclinical
treatmentafterassessment,froma
baselineof71percenttoagoalof90
percent(i.e.,reducethepercentageof
childandyouthbeneficiariesdischarged
afteronlyoneservicefrom29percentto
10percent)?
• Traininginandimplementationof
FIT,whichinvolvescollecting
feedbackfrombeneficiariesand
usingthatfeedbacktoimprove
listening,engagement,rapport
andtrustwithbeneficiaries;
• Immediatefollow-upappointment
schedulingbytheclinician;and
• Providingtelephoneand/orhome
visitsbybehavioralhealth
specialistsbetweenthe
assessmentandfollow-up
treatmentappointments?”
ThegoalofthisPIPistoincreasetheproportionofchildrenandyouthwhoareretainedbeyondtheirfirstserviceencounteridentifiedastheassessment.
AlthoughtheMHPsubmitteddetailedandthoughtfulindicatorsandexplainedtheinterventions,ithasnotproceededbeyondtheconceptonlystage.Theinterventionshavenotbeenapplied,anddatahavenotbeencollected.
TheTAprovidedtotheMHPbyCalEQROconsistedofphonecallspriortotheon-sitereviewandfollowingthereview.Sincethen,theMHPhasdevelopedaconceptfortheirclinicalPIP,althoughasmentioned,theMHPhasnotimplementedtheinterventions.Theon-sitediscussionsduringthereviewincludedencouragingtheMHPtocollectandreportondatamonthlyandcomparedataquarterly.ThePIPmustidentifyandmeasurethebenefittothebeneficiaryaswell.Atanypointduringthedatareview,theMHPisencouragedtocontactCalEQROregardingcontingenciesortrainingneededtoreachitsgoals.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 44
• ImprovingAnxietyLevelsofBeneficiariesDiagnosedwithanAnxietyDisorder(ConceptOnly,NotYetActive)
Trinity-Clinical
StudyQuestion
(aspresentedbyMHP)FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willcreationofan
interventiongroupfor
treatinganxietyreducetheaverageaggregate
levelofanxietybythe
beneficiarypopulation
from2.2to2.0(onascale
of1-3)asmeasuredby
theANSAtool?”
TheoverallgoalofthisclinicalPIPistoimprovetheoutcomesofbeneficiariesdiagnosedwithananxietydisorderasmeasuredbytheANSAtool.Addressingtheissueisexpectedtoimpactanybeneficiarywithananxietydiagnosis,asmuchas27percentofthebeneficiaries,aswellasthosewithanxietysymptoms.ThegoalofthisPIPistoimproveandtoreducetheseverityofanxietyexperiencedbythebeneficiaries,potentiallyleadingtoearlierengagementwiththeadditionofagroupfocusedoncopingstrategies.
TheMHPwillofferandconductagroupfocusedonstrategiestoimprovecopingwithaMindfulness-basedStressReductionsyllabus.TheintroductionoftheMindfulness-basedStressReductiontechniqueishopedtoleadtoanimprovementinthelevelofanxietyexperiencedbyadultbeneficiariesbyprovidingcopingskillstolessensymptoms.
Thisisintheplanningphasenow,aninitialsyllabushasbeendrafted,andisintendedtosupportthebeneficiaryindealingwithandreducinglevelsofanxiety.TheMHPalsobelievesthatbeneficiarieswhouseimprovedcopingskillsmaybeabletofocusonotherbehavioralhealthissuesthatarecurrentlysupersededbytheirfocusonanxiety.Animprovementmayalsobeinassuringamoreaccuratediagnosisandmorefrequentupdatingofdiagnoses,withcorrespondingtreatmentplansforbeneficiariesinthiscategory.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussingthetimelinesofPIPactivities,encouragingtheMHPtoidentifyitsPIPearly,implementinterventions,andcollectandanalyzetherelevantdatatoidentifynecessaryadjustmentssothattheratingforthePIPisacceptedasactive.Thestudyquestion,althoughmeasurablecouldbereviewedsinceitsuggestsamarginalimprovementgoal(from2.2percentto2.0percent)andtoreviewthelanguageinthestudyquestionwiththesuggestiontochangethewordingto“Willprovidingthe…”TheMHPwasofferedon-goingTA.Nofurtherconsultshavebeenscheduled.
CalEQRO PIP Summary Report Q4 April - June 2018
Submission determined not to be a PIP 45
SUBMISSION DETERMINED NOT TO BE A PIP
• Strengths-BasedLearningCollaborative:StrengthsModelGroupSupervisionforEmployment-RelatedGoals(SubmissionDeterminedNottobeaPIP)
Mono–Non-Clinical
StudyQuestion
(aspresentedby
MHP)
FocusofPIPAreasfor
ImprovementTAProvidedbyCalEQRO
“WillusingSMGS
(StrengthsModel
GroupSupervision)
helpbeneficiaries
makeprogresstoward
theiremployment-
relatedgoals,as
measuredbythe
achievementofand/or
changeinemployment-
relatedgoalsoverthe
two-yearstudyperiod
asreportedonthe
Strengths
Assessment?”
TheMHPhasidentifiedfulfillmentoflifegoalsofbeneficiariesasnotwellsupportedbytheusualclinicalfocusofstaff,whichtendstoalignwithsymptomsandimpairmentsofillness.Theexistenceoflifedomainareasthatareunfulfilled,suchashousing,educationandemployment,hasbroughttheMHPtofocusonanapproachgearedtosupportothersuccesses.TheStrengthsModelisassociatedwithaspecificassessmentapproach,thedevelopmentofapersonalrecoveryplanandsupportedbyaspecificfocusingroupsupervision.Likelythisisassociatedwithchangesinapproachbyclinicalstaff,buttheseinterventionsarenotdescribedwithinthisPIP.
ThisPIPisverysimilartotheclinicalPIPbutwithaslightlydifferentfocus.TheoverlapissufficienttoconcludethatbothcannotbeacceptedasactivePIPsforthisMHP.
TheTAprovidedtotheMHPbyCalEQROconsistedofdiscussionoftheduplicativeaspectsofthisPIP,andidentificationofpotentialalternatePIPtopics.Onetopicthatwasdiscussedwasthatoftelepsychiatryappointmentno-shows,whichtheMHPhasbeentracking.
CalEQRO PIP Summary Report Q4 April – June 2018 Appendix A
Submission Determined Not to be a PIP 46
• StrengthsAssessment(SubmissionDeterminedNottobeaPIP)
Inyo–Clinical
StudyQuestion
(aspresentedby
MHP)
FocusofPIP AreasforImprovement TAProvidedbyCalEQRO
“Willthe
implementationofthe
StrengthsAssessment
toolfromtheStrengths
Modelhelpmoveeight
ICBHbeneficiaries
formerlymiredin
repetitiveservice
utilizationtowards
theirhighestlevelof
recoveryintheself-
identifiedgoalareasof
housing?”
InalignmentwiththeMHP’snon-clinicalPIP,thisactivityfocusedoncorrectingforthedeficitthatemanatesfromclinically-focusedtreatmentplanning,anarrowfocusonsymptomsandimpairmentsofmentalillness.PositiveachievementofbeneficiarylifegoalscanbemissediftheMHPisnotorientedtotherehabilitativeservicesmodel.
ThisPIPnarrowlyfocusedontheeightbeneficiarieswhothroughtheStrengthsAssessmenthadidentifiedimprovedhousingasakeyareaforpersonalimprovement.
TheStrengthsAssessmentPIP,submittedtomeettheclinicalrequirement,essentiallyduplicatesanarrowaspectofthenon-clinicalStrengthsModelPIP,withthecaveatthatitwasfocusedonbeneficiarieswhohaveidentifiedhighestprioritylifegoalsinthehousingdomain.However,thatnarrowactivityappropriatelybelongsintegratedwiththenon-clinicalPIP.
TheTAprovidedtotheMHPbyCalEQROconsistedofonsitediscussionandpost-reviewfollow-up,providingtheMHPwiththeopportunitytoamendthenon-clinicalPIP.ThisclinicalPIPcouldbeafirstphaseofthestrengthsmodelimplementationofthenon-clinicalPIP.Furtherguidancetoincludethedirectinterventionsofstaffwithbeneficiarieswasalsoprovided.Additionalguidanceprovidedbyemailafterthereview,includingencouragementtodevelopanewclinicalPIPtopic.
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
47
PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET
DEMOGRAPHIC INFORMATION
MHP: � Clinical PIP � Non-Clinical PIP
PIPTitle:
Start Date (MM/DD/YYYY) ___________
Projected Study Period (#of Months) ________
Completed: _____ Yes ______ No
Dates of On-Site Review: ___________(MM/DD/YYY)
Name of Reviewer: ___________________________
Status of PIP (Only Active and ongoing, and completed PIPs are rated):
Rated
� Active and ongoing (baseline established and interventions started)
� Completed since the prior External Quality Review (EQR)
Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.
� Concept only, not yet active (interventions not started)
� Inactive, developed in a prior year
� Submission determined not to be a PIP
BriefDescriptionofPIP:
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
48
ACTIVITY1:ASSESSTHESTUDYMETHODOLOGY
STEP1:ReviewtheSelectedStudyTopic(s)
Component/Standard Score Comments
1.1 WasthePIPtopicselectedusingstakeholderinput?DidtheMHPdevelopamulti-functionalteamcompiledofstakeholdersinvestedinthisissue?
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
1.2 Wasthetopicselectedthroughdatacollectionandanalysisofcomprehensiveaspectsofenrolleeneeds,care,andservices?
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
SelectthecategoryforeachPIP:Clinical:�Preventionofanacuteorchroniccondition �Highvolumeservices�Careforanacuteorchroniccondition �Highriskconditions
Non-Clinical:
�Processofaccessingordeliveringcare
1.3 DidthePlan’sPIPs,overtime,addressabroadspectrumofkeyaspectsofenrolleecareandservices?
Projectmustbeclearlyfocusedonidentifyingandcorrectingdeficienciesincareorservices,ratherthanonutilizationorcostalone.
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
1.4 DidthePlan’sPIPs,overtime,includeallenrolledpopulations(i.e.,didnotexcludecertainenrolleessuchasthosewithspecialhealthcareneeds)?
Demographics:
�AgeRange�Race/Ethnicity�Gender�Language�Other
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
Totals <#>Met <#> PartiallyMet <#> NotMet <#> UTD
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
49
STEP2:ReviewtheStudyQuestion(s)
2.1 Wasthestudyquestion(s)statedclearlyinwriting?
Includestudyquestionasstatedinnarrative:
<Text>
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
Totals <#>Met <#> PartiallyMet <#> NotMet <#> UTD
STEP3:ReviewtheIdentifiedStudyPopulation
3.1 DidthePlanclearlydefineallMedi-Calenrolleestowhomthestudyquestionandindicatorsarerelevant?
Demographics:
�AgeRange�Race/Ethnicity�Gender�Language�Other
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
3.2 Ifthestudyincludedtheentirepopulation,diditsdatacollectionapproachcaptureallenrolleestowhomthestudyquestionapplied?
Methodsofidentifyingparticipants:
�Utilizationdata �Referral �Self-identification
�Other: <Textifchecked>
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
Totals <#>Met <#> PartiallyMet <#> NotMet <#> UTD
STEP4:ReviewSelectedStudyIndicators
4.1 Didthestudyuseobjective,clearlydefined,measurableindicators?
Listindicators:
<Text>
�Met
�PartiallyMet
��otMet
�UnabletoDetermine
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
50
4.2 Didtheindicatorsmeasurechangesinhealthstatus,functionalstatus,orenrolleesatisfaction,orprocessesofcarewithstrongassociationswithimprovedoutcomes?
Arelong-termoutcomesimplied?�Yes �No
Orarelong-termoutcomesclearlystated? �Yes �No
�HealthStatus �FunctionalStatus
�MemberSatisfaction �ProviderSatisfaction
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
Totals <#>Met <#> PartiallyMet <#> NotMet <#> UTD
STEP5:ReviewSamplingMethods
5.1 Didthesamplingtechniqueconsiderandspecifythetrue(orestimated)frequencyofoccurrenceoftheevent,theconfidenceintervaltobeused,andthemarginoferrorthatwillbeacceptable?
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
5.2 Werevalidsamplingtechniquesthatprotectedagainstbiasemployed?
Specifythetypeofsamplingorcensusused:
<Text>
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
5.3Didthesamplecontainasufficientnumberofenrollees?
______Nofenrolleesinsamplingframe
______Nofsample
______Nofparticipants(i.e.–returnrate)
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
Totals <#>Met <#> PartiallyMet <#> NotMet <#> UTD
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
51
STEP6:ReviewDataCollectionProcedures
6.1 Didthestudydesignclearlyspecifythedatatobecollected?
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
6.2 Didthestudydesignclearlyspecifythesourcesofdata?
Sourcesofdata:
�Member �Claims �Provider
�Other: <Textifchecked>
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
6.3 Didthestudydesignspecifyasystematicmethodofcollectingvalidandreliabledatathatrepresentstheentirepopulationtowhichthestudy’sindicatorsapply?
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
6.4 Didtheinstrumentsusedfordatacollectionprovideforconsistent,accuratedatacollectionoverthetimeperiodsstudied?
Instrumentsused:
☐Survey ☐Medicalrecordabstractiontool ☐Outcomestool☐LevelofCaretools☐Other: <Textifchecked>
☐Met☐PartiallyMet☐NotMet☐UnabletoDetermine
6.5 Didthestudydesignprospectivelyspecifyadataanalysisplan?
☐Met☐PartiallyMet☐NotMet☐UnabletoDetermine
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
52
6.6 Werequalifiedstaffandpersonnelusedtocollectthedata?
Projectleader:
Name: <Text>
Title: <Text>
Role:<Text>
Otherteammembers:
Names: <Text>
☐Met☐PartiallyMet☐NotMet☐UnabletoDetermine
Totals <#> Met <#> Partially Met <#> Not Met <#> UTD
STEP7:AssessImprovementStrategies
7.1Werereasonableinterventionsundertakentoaddresscauses/barriersidentifiedthroughdataanalysisandQIprocessesundertaken?
DescribeInterventions:
<Text>
�Met
�PartiallyMet
�NotMet
�UnabletoDetermine
Totals <#>Met<#>PartiallyMet <#>NotMet<#>NA<#>UTD
STEP8:ReviewDataAnalysisandInterpretationofStudyResults
8.1 Wasananalysisofthefindingsperformedaccordingtothedataanalysisplan?
Thiselementis“NotMet”ifthereisnoindicationofadataanalysisplan(seeStep6.5)
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
8.2 WerethePIPresultsandfindingspresentedaccuratelyandclearly?
Aretablesandfigureslabeled?�Yes �No
Aretheylabeledclearlyandaccurately? �Yes �No
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
53
8.3 Didtheanalysisidentify:initialandrepeatmeasurements,statisticalsignificance,factorsthatinfluencecomparabilityofinitialandrepeatmeasurements,andfactorsthatthreateninternalandexternalvalidity?
Indicatethetimeperiodsofmeasurements:___________________
Indicatethestatisticalanalysisused:_________________________
Indicatethestatisticalsignificancelevelorconfidencelevelifavailable/known:_______%______Unabletodetermine
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
8.4 DidtheanalysisofthestudydataincludeaninterpretationoftheextenttowhichthisPIPwassuccessfulandrecommendanyfollow-upactivities?
Limitationsdescribed:
<Text>
Conclusionsregardingthesuccessoftheinterpretation:
<Text>
Recommendationsforfollow-up:
<Text>
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
Totals <#>Met<#>PartiallyMet <#>NotMet<#>NA<#>UTD
STEP9:AssessWhetherImprovementis“Real”Improvement
9.1 Wasthesamemethodologyasthebaselinemeasurementused,whenmeasurementwasrepeated?
Ask: Werethesamesourcesofdataused?
Didtheyusethesamemethodofdatacollection?
Werethesameparticipantsexamined?
Didtheyutilizethesamemeasurementtools?
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
54
9.2 Wasthereanydocumented,quantitativeimprovementinprocessesoroutcomesofcare?
Wasthere: �Improvement �Deterioration
Statisticalsignificance: �Yes �No
Clinicalsignificance: �Yes �No
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
9.3 Doesthereportedimprovementinperformancehaveinternalvalidity;i.e.,doestheimprovementinperformanceappeartobetheresultoftheplannedqualityimprovementintervention?
Degreetowhichtheinterventionwasthereasonforchange:
�Norelevance �Small �Fair �High
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
9.4 Isthereanystatisticalevidencethatanyobservedperformanceimprovementistrueimprovement?
�Weak �Moderate �Strong
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
9.5 Wassustainedimprovementdemonstratedthroughrepeatedmeasurementsovercomparabletimeperiods?
�Met
�PartiallyMet
�NotMet
�NotApplicable
�UnabletoDetermine
Totals <#>Met<#>PartiallyMet <#>NotMet<#>NA<#>UTD
CalEQRO PIP Summary Report Q4 April - June – March 2018 Appendix B
55
ACTIVITY2:VERIFYINGSTUDYFINDINGS(OPTIONAL)
Component/Standard Score Comments
Weretheinitialstudyfindingsverifieduponrepeatmeasurement?
�Yes
�No
o
ACTIVITY3:OVERALLVALIDITYANDRELIABILITYOFSTUDYRESULTS:SUMMARYOFAGGREGATEVALIDATIONFINDINGS
Conclusions: <Text>
Recommendations: <Text>
Check one: � High confidence in reported Plan PIP results � Low confidence in reported Plan PIP results � Confidence in reported Plan PIP results � Reported Plan PIP results not credible