performance assessments can: help identify potential problems in the program help identify areas...
TRANSCRIPT
Why should your drug court care about performance improvement processes?
Presented by: David Wright, Ph.D. & Lorrie Byrum
You’re Good &
You Know It!
Performance assessments can: help identify potential problems in the program help identify areas where streamlining the process could be useful & help make the program run more smoothly
help ensure fidelity to the model help ensure that the program is following the 10 key components/10 essential elements
The Importance of Performance Assessment Within Your Drug Court
Why should we care if we adhere to the 10 key components:
Courts that bring people into the program within 50 days of arrest have greater impact on reducing recidivism
Courts that drug test at least two times per week (with a true random pattern) are twice as cost effective
Courts that use jail sanctions no longer than 6 days had lower recidivism than courts with jail sanctions of 6+ days
DRUG COURTS THAT EVALUATE THE PROGRAM & MODIFY THE OPERATION BASED ON RESULTS HAD TWICE THE COST SAVINGS OF OTHER COURTS
The Importance of Performance Assessment Within Your Drug Court
How Does My Court Compare To The 10 Key Components
Taking The Temperature Of The Court
How do you evaluate your program like component
number 8 indicates?
Key Component #8:“Monitoring and evaluation measure the achievement of program goals and
gauge effectiveness.”
WEBS Is Your Data Collection Process Online
Reports
Active Counts
Outcomes
Performance Measures
Admission Data
Referral Data (FDC & MHC Only)
Phase Data
Discharge Data (Termination,
Graduation, etc.)
http://www.themagazine.ca/wp-content/uploads/2009/01/GarbageTruck.jpg
Importance of Accurate Data
Garbage In
Garbage Out
Source: http://www.trashitman.com/blog/wp-content/uploads/2011/03/Packer-truck.jpg
Inaccurate Data Affects…
•Your Court’s Active Counts
•Your Court’s Outcomes & Performance Measures
•& Ultimately, Statewide Data that Affects All Specialty Courts
Performance Improvement
Ensure Current DataData Accuracy Goal
Setting
Case Review
Identify Systemic Issues
Ensure
Contract
Compliance
Monitor
Performance
Improvement
Ensure Needs Are Being Met
Performance Improvement
Ensuring data quality:Contractual requirementTied to fundingData matchingData Review
Contracts require data elements for all active, graduated, & terminated participants.
Participants are only counted active for purposes of funding if data records have been updated in at least the last 60 days.
Additionally, contract payments can be withheld for failure to accurately report data.
The ODMHSAS holds data sharing agreements with multiple agencies including the Oklahoma Department of Corrections & the Oklahoma Employment Securities Commission.
Run program data & review for outliers or data inconsistencies, correct errors.
Performance ImprovementTied to Funding
When a review of the program’s 60 Day Active Count shows a participant or a group of participants that have not been updated in the last 60 days program staff should take steps to ensure the data are updated immediately or by the 2nd of each month. This ensures programs have current data.
Performance ImprovementData Matching
Performance ImprovementData Matching
Performance ImprovementData Matching
Performance ImprovementData Matching
Performance ImprovementData Review
When a review of outcome data shows a dramatic change, courts should review data to determine if outliers. Once identified, then correct inaccuracies. This ensures programs have accurate data.
Performance ImprovementData Review
1) Run participant-level reports to identify the outliers (Example: Which participants have decreasing income between admission & graduation.)
2) Verify data through file review (Example 1: Pull participant file & review employment verification/pay stub to verify income at graduation.)
3) Correct record to reflect the accurate income.
4) Re-run report to verify correction & view new average for this outcome at the court level (correcting individual cases impacts the overall court results).
Performance ImprovementCase Studies
When a review of outcome data shows an outlier, courts can verify & correct data (as seen previously). IF data are correct, then courts can conduct case studies to determine the explanation for why the case was an outlier.
Performance ImprovementCase Studies
1) Identify the outlier of focus
2) Case ‘Wwwww’ has no prior felony convictions, no prior arrests, no additional charges; & the current charge is Possession for which they received a 10 year prison sentence. Why would this case be in your court? The reasons are obvious for ‘meth user’, ‘Tt Tt’, & ‘nancy warren’
3) Begin case study gathering information on why this person came into the program. Review:
• Risk Screen Information (participant has strong association with known criminals, is unemployed, is homeless, & has no family support)
• ASI (participant has used drugs daily for the past15 years & began at an early age)
• Mental Health Screen (participant has a co-occurring mental health concern).
Performance ImprovementGoal Setting
With the ability to run real-time outcome reports, courts can set goals for improvement of one or more outcome measures.
Performance Improvement
Goal Setting1) Identify which outcome(s) to target
2) Set a measureable, realistic goal for improvement
3) Set a timeline to re-evaluate
Performance Improvement
Goal Setting1) Identify which outcome(s) to target
2) Set a measureable, realistic goal for improvement
3) Set a timeline to re-evaluate
4) Identify a barrier/barriers which has lead to the current issue
• Many times this is simply, it wasn’t a priority. • (Case Studies vs. Systemic Issues)
5) Choose one barrier & identify steps to remove/reduce it.
6) Re-evaluate
Performance ImprovementSystemic Issues
Data can reflect a systemic issue that needs to be addressed through policy change(s).
The use of jail as a sanction in MHC is:• Ineffective (costly and does not result in positive
behaviors)• Inappropriate• AGAINST best practices• Harmful
Performance ImprovementSystemic Issues
Data can reflect a systemic issue that needs to be addressed through policy change(s).
NOT getting people into treatment quickly is:• AGAINST best practices• Harmful to recovery (results in poor outcomes)• Delays needed treatment
Performance ImprovementSystemic Issues
1) Identify which outcome(s) to target
2) Run participant-level reports which show multiple participants impacting an outcome measure.
3) Meet with the court team to develop plan of action. Multiple participants are impacted. (Example 1: Extended jail days being used as a routine sanction. May need to re-evaluate sanction matrix. Example 2: Excessive time between referral & admission. May need to review the admissions process.)
4) Identify a timeline to re-evaluate the plan from step 3.
5) Re-evaluate
Performance Improvement Goal Setting or Systemic Issues
Performance ImprovementGoal Setting or Systemic Issues
1) Run demographic reports which show the make-up of the participants within the court.
2) Compare the court participant demographics to that of arrests within the county and the county demographics as a whole. Does the program mirror these demographics?
3) Meet with the court team to determine if the issue is due to goal setting or if the issue is systemic then develop plan of action to address the issue.
4) Identify a timeline to re-evaluate the plan from step 3.
5) Re-evaluate
Court Gender Comparisons
Male Female0%
25%
50%
75%
100%
49.5% 50.5%
75.5%
24.5%
86.1%
13.9%
64.5%
35.5%
71.4%
28.6%
Oklahoma Population All Arrests Prison Receptions
Mental Health Court Admissions Drug Court
Court Race/Ethnicity Comparisons
0%
20%
40%
60%
80%75%
8% 9% 10%
73%
17%
9%6%
56%
25%
11%
7%
69%
17%12%
1%
69%
9%
19%
3%
Oklahoma Population All Arrests Prison Receptions
Mental Health Court Admissions Drug Court
Performance Improvement
State representative role (Field Representative)
Building a relationship with the programsKeeping data as a priorityCommunicating with the programs on outcome changes
Identifying training needs/trendsAssisting in the assurance of quality dataProviding technical assistance (case studies & systemic concerns)
Reports Available on Demand
Other Uses of the Reports
Criminal Info Report Check to see if reaching target population Ensure accuracy of data used to calculate
your court’s priors data
Outcome Page Report Use to check other performance & outcomes
measures
Demographics ReportsEnsure your court is reaching its target
population
Sharing Program Successes with Community Stakeholders
Completion Rates by Race
Caucasian African American
Native American
Hispanic Total52%
54%
56%
58%
60%
62%
64%
66%
68%
70%
72%
69.0%
59.0%
62.0%
70.0%
66.0%
Unemployment by Race
Caucasian African American
Native American
Hispanic Total0%
5%
10%
15%
20%
25%
30%
35%
40%
34.0%
31.0%
38.0%
21.0%
34.0%
3.0% 4.0% 3.0%2.0%
3.0%
Entry Graduation
Income from Employment by Race
Caucasian African American
Native American
Hispanic Total$0
$200
$400
$600
$800
$1,000
$1,200
$1,400
$1,600
$1,800
$870
$577 $613
$1,046
$799
$1,700
$1,052
$1,406
$1,559 $1,574
Entry Graduation
Percent without H.S. Diploma/GED
by Race
Caucasian African American
Native American
Hispanic Total0%
10%
20%
30%
40%
50%
60%
70%
25.0% 25.0%27.0%
58.0%
26.0%
16.0%18.0%
16.0%
40.0%
17.0%
Entry Graduation
Percent of Children Living with Participants by Race
Caucasian African American
Native American
Hispanic Total0%
10%
20%
30%
40%
50%
60%
70%
39.0% 37.0% 41.0%44.0%
40.0%
60.0% 59.0%63.0% 61.0% 60.0%
Entry Graduation
Conclusion
Run all reports regularly Ensure data are current Ensure data accuracy Confirm contract compliance Monitor improvement Review case abnormalities Set goals Address systemic issues Serve county needs effectively• Risk, MH, SA, Race/Ethnicity, Gender
Contact Info
David Wright (405) 522-6169 [email protected]
Lorrie Byrum (405) 522-6172 [email protected]
Nancy Warren (405) 522-6170 [email protected]