performance metrics in integrated justice : measuring success and the need for improvement
DESCRIPTION
PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement. presented by: Bob Roper, CIO Colorado Judicial Branch Teri B. Sullivan, SEARCH Group. DO NOT PANIC—THIS IS NOT A STATISTICS COURSE. PRESENTATION OBJECTIVES. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/1.jpg)
PERFORMANCE METRICS IN INTEGRATED JUSTICE:
Measuring Success and theNeed for Improvement
presented by:Bob Roper, CIO Colorado Judicial Branch
Teri B. Sullivan, SEARCH Group
![Page 2: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/2.jpg)
DO NOT PANIC—THIS IS NOT A STATISTICS COURSE
![Page 3: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/3.jpg)
PRESENTATION OBJECTIVES
• DEVELOP A RATIONALE FOR MEASURING CJIS OBJECTIVES: Why Measure Anything?
• IDENTIFY METHODS FOR GATHERING DATA• OUTLINE SOME CJIS GOALS AND OBJECTIVES• SPECIFY SOME CJIS MEASUREMENTS LINKED
TO GOALS AND OBJECTIVES• CONVERT DATA INTO INFORMATION:
DISPLAYING THE METRICS• PROVIDE SOME CAUTIONS IN MEASURING CJIS
OBJECTIVES• IDENTIFY SOME ADDITIONAL RESOURCES
![Page 4: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/4.jpg)
WHY MEASURE CJIS OBJECTIVES?
![Page 5: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/5.jpg)
WHY EVALUATE A CJIS PROJECT?Are We Beyond the “trust me” Stage?
• INFORMATION IS CONTROL• PROVIDES FEEDBACK TO IMPROVE PRGM.• PROVIDES INFO FOR RESOURCE
ALLOCATION• ENABLES EFFECTIVE PLANNING• TESTS GENERALIZATIONS vs INDIVIDUAL
ASSUMPTIONS• MARKET TO, AND DEVELOP SUPPORT
AMONG FUNDING BODIES, CONSTITUENTS, AND STAFF
![Page 6: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/6.jpg)
LEGISLATION THAT REQUIRE QUANTIFIABLE OBJECTIVES
• Chief Financial Officers Act (CFO) of 1990• Government Performance and Results Act
(Results Act) of 1993• Federal Acquisition Streamlining Act
(FASA) of 1994• Paperwork Reduction Act of 1995• Clinger-Cohen Act of 1996• Performance Based Budgeting
![Page 7: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/7.jpg)
METHODS FORGATHERING DATA
![Page 8: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/8.jpg)
MEASURING THE SUCCESS OF THEPROGRAM: “Compared to What…?”
• …OTHER PROGRAMS• …THE CURRENT SYSTEM -
BASELINING• …OTHER JURISDICTIONS -
BENCHMARKING• …OVER TIME
![Page 9: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/9.jpg)
USE MORE THAN ONE MEASURE:The Balanced Scoreboard
• CUSTOMER PERSPECTIVE
• INTERNAL BUSINESS PERSPECTIVE
• INNOVATION AND LEARNING PERSPECTIVE
• FINANCIAL PERSPECTIVE
![Page 10: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/10.jpg)
INTEGRATED CRIMINAL JUSTICE OBJECTIVES THAT
CAN BE MEASURED
![Page 11: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/11.jpg)
DEFINING THE EVALUATION PROCESSHow Do I Know It Worked?
GOALS“Broad Statements of Interest”
PROJECT OBJECTIVES“Done on Time, Within Budget and Within Specs”
FUNCTIONAL OBJECTIVES“Did It Perform As Designed?’
MEASURABLE BUSINESS OBJECTIVES“Why Did We Pursue the Program in the First Place?”
![Page 12: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/12.jpg)
SAMPLE GOALS OF INTEGRATED JUSTICE
• ENHANCE PUBLIC SAFETY• IMPROVE ACCOUNTABILITY OF THE
JUSTICE SYSTEM• IMPROVE PUBLIC TRUST AND CONFIDENCE
IN THE JUSTICE SYSTEM• IMPROVE CASEFLOW MANAGEMENT• IMPROVE STAFF EFFICIENCIES• ENHANCE THE QUALITY OF DECISION
MAKING
![Page 13: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/13.jpg)
CHARACTERISTICS OF MEASURABLE BUSINESS OBJECTIVES
BE INCREASINGLY SPECIFIC ABOUT THE MEASURE—THE MORE SPECIFIC, THE MORE
RELIABLE THE MEASURE:• WHAT IS THE MEASURE?• WHAT’S THE OBJECT OF THE MEASURE?• WHAT’S THE VALUE OF THE MEASURE FOR THAT
OBJECT?• WHERE IS IT GOING TO BE MEASURED?• WHEN IS IT GOING TO REACH THAT VALUE?
BE SURE YOUR MEASURABLE BUSINESS OBJECTIVE IS RELATED TO A GOAL.
![Page 14: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/14.jpg)
EVOLUTION OF A MEASURABLE BUSINESS OBJECTIVEGOAL: IMPROVE CASEFLOW MGMT
• Reduce the time to disposition• Reduce the time to disposition of
misdemeanor cases• Reduce the time to disposition of
misdemeanor cases to an average of 6 months
• Reduce the time to disposition of misdemeanor cases to an average of 6 months Statewide
• Reduce the time to disposition of misdemeanor cases to an average of 6 months Statewide by 6/30/03
![Page 15: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/15.jpg)
EVOLUTION OF A MEASURABLE BUSINESS OBJECTIVE
GOAL: IMPROVE QUALITY OF DECISION-MKG
• Reduce the number of false arrests• Reduce the number of false arrests of felony
cases• Reduce the number of false arrests of felony
cases to less than 4 a month• Reduce the number of false arrests of felony
cases to an average of 6 months in Denver• Reduce the number of false arrests of felony
cases to an average of 6 months Statewide by 9/30/02
![Page 16: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/16.jpg)
EVOLUTION OF A MEASURABLE BUSINESS OBJECTIVE
GOAL: IMPROVE PUBLIC SAFETY
• Increase the percent of court dispositions that match to an arrest
• Increase the percent of court dispositions that match to an arrest in felony cases
• Increase the percent of court dispositions that match to an arrest in felony cases to 80%
• Increase the percent of court dispositions that match to an arrest in felony cases to 80% Statewide
• Increase the percent of court dispositions that match to an arrest in felony cases to 80% Statewide by 6/30/02
![Page 17: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/17.jpg)
SOME MEASURABLE OBJECTIVES FOR
INTEGRATED JUSTICE GOALS
![Page 18: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/18.jpg)
THESE ARE ONLY THE BASIC MEASURES THAT NEED TO BE EXPANDED AS DESCRIBED IN
THE PRECEDING 3 EXAMPLES IN ORDER TO BECOME FULL
MEASURABLE OBJECTIVES.
![Page 19: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/19.jpg)
GOAL: ENHANCE PUBLIC SAFETYSome Measurable Objectives Are:
• Increase the % of court dispositions that match to an arrest incident
• Decrease the # of times court cases are not reported in criminal histories
• Decrease the average response time it takes to receive a positive identification
• Decrease the # of incidents where the wrong person is released from custody
• Decrease the # of incidents where criminal records are associated with the wrong person
• Decrease the recidivism rate• Reduce the # of escapes during prisoner transport• Decrease the % discrepancy between active warrants on the court
system and those on the State Registry• Decrease the amount of time it takes to issue victim notices
![Page 20: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/20.jpg)
GOAL: IMPROVE ACCOUNTABILITYSome Measurable Objectives Are:
• Increase the # of Mgmt Reports available to the public on the WWW
• Increase the # of hits on the CJIS Web page
• Increase the # hrs the general public views data electronically
![Page 21: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/21.jpg)
GOAL: IMPROVE PUBLIC TRUST/CONFIDENCESome Measurable Objectives Are:
• Reduce the # hrs it takes to respond to a request from the general public
• Reduce the # minutes it takes to complete a criminal history background check
• Increase the % of the general public that trusts the courts
• Increase the % of the general public that is satisfied with local law enforcement
• Reduce the # of civil complaints against local law enforcement
![Page 22: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/22.jpg)
GOAL:IMPROVE CASEFLOW MGMTSome Measurable Objectives Are:
• Reduce the # of continuances per case originating from scheduling conflicts
• Reduce the # of cases without a next scheduled event
• Increase the % of hearings held as scheduled
• Reduce the average days from arrest to arraignment
• Reduce the average days held waiting for bond decisions
• Reduce the # days to process cases from arrest to disposition
• Reduce the processing time required to process inmates at the Department of Corrections
![Page 23: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/23.jpg)
GOAL: ENHANCE STAFF EFFICIENCIESSome Measurable Objectives Are:
• Reduce the # hrs staff spends entering data electronically• Reduce number of hrs to collect data manually• Reduce # of phone-calls among governmental entities• Reduce the costs of copying paper for other governmental entities• Reduce the costs of storing paper• Reduce the costs of buying paper supplies• Reduce the number of hrs spent filing paper manually• Reduce the mailing costs among governmental entities• Reduce the # of hrs spent searching other automated systems• Reduce the # of hrs spent by staff looking for hard copy files• Increase the # of law enforcement personnel that can be
reassigned from a desk to the street• Increase the # of data transfers among the agencies• Reduce the costs associated with producing forms
![Page 24: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/24.jpg)
GOAL: IMPROVE DECISION MAKING QUALITYSome Measurable Objectives Are:
• Reduce the # of false arrests• Reduce the # of hrs it takes to enter a court
disposition into the State Criminal History Repository• Reduce the amount of missing data among the
governmental entities• Reduce the # of data errors reported for correction• Decrease the amount of time it takes to enter a
warrant into the State Warrant Registry • Decrease the amount of time it takes to enter a
protective order into the Statewide Protective Order Registry
• Increase the # of query hits among the agencies• Reduce the # of inaccurate background checks
![Page 25: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/25.jpg)
DISPLAYING AND PUBLISHING THE METRICS
![Page 26: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/26.jpg)
…In order to reach the goals and objectives, one needs to
measure and publish because, “People behave
based on how they are measured.”
![Page 27: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/27.jpg)
DISPLAYING AND PUBLISHING THE RESULTS:CONVERTING DATA INTO INFORMATION
• CONVERT DATA INTO PICTURES AND GRAPHICS
• COLOR HELPS TO HIGHLIGHT YOUR POINTS
• K.I.S.S. YOUR DISPLAY
• PUBLISH REGULARLY
• DON’T OVERWHELM WITH DATA
![Page 28: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/28.jpg)
GRAPHIC DISPLAY OF DATA:DISPOSITION MATCHING IN COLORADO
![Page 29: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/29.jpg)
GRAPHIC DISPLAY OF DATA:WARRANT TRANSFER IN COLORADO—A PICTURE IS
WORTH A THOUSAND WORDS
![Page 30: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/30.jpg)
GRAPHIC DISPLAY OF DATA:RANKING LEADS TO IMPROVEMENT
![Page 31: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/31.jpg)
GRAPHIC DISPLAY OF DATA:BENCHMARKING AGE OF CRIMINAL CASES
Criminal Cases Statewide FY 2002
75%
80%
85%
90%
95%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Districts
% o
f o
pen
cas
es 0
-12
mo
nth
s
1st quarter FY 2002
2nd quarter FY 2002
2002 goals
![Page 32: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/32.jpg)
CAUTIONS IN MEASURING CJIS OBJECTIVES
![Page 33: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/33.jpg)
THINGS TO AVOID
• TOO COMPLEX• GENERALIZING FROM LIMITED DATA• CANNOT COLLECT ACCURATE OR
COMPLETE DATA • TOO MANY MEASURES• SPURIOUS RELATIONSHIPS• ASSUMPTIONS• BE CAREFUL: THE STATS YOU PUBLISH
WILL BE PUBLIC
![Page 34: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/34.jpg)
THINGS TO DO
• BE SURE TO ID THINGS/CONCEPTS THAT ARE IMPORTANT AND CAN BE MEASURED
• PILOT THE MEASURES• RECEIVE FEEDBACK• MODIFY MEASURES• REASSESS SUCCESS• BROADCAST SUCCESS: GET AN AGENT• DON’T FORGET TO TALK ABOUT SOME THINGS
THAT CAN’T BE MEASURED• MEASURE THE INTEGRITY/QUALITY OF DATA
![Page 35: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/35.jpg)
COPING WITH DATA QUALITY
• DATA RELIABILITY– COMPLETE– ACCURATE– TIMELY
• DATA VALIDITY: Are we measuring what we intend to measure?
• DECIDE ON DAY FORWARD OR CONVERSIONS—THEN ANNOUNCE
• INSERT DATA EDITS• PUBLISH THE RESULTS• GET UPPER LEVEL MANAGEMENT SUPPORT• PROVIDE MECHANISMS FOR CORRECTING DATA
![Page 36: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/36.jpg)
MEASURES SHOULD BE…
• SPECIFIC
• MEASURABLE
• ACTIONABLE
• RELEVANT
• TIMELY
![Page 37: PERFORMANCE METRICS IN INTEGRATED JUSTICE : Measuring Success and the Need for Improvement](https://reader035.vdocuments.us/reader035/viewer/2022070418/56815922550346895dc64b54/html5/thumbnails/37.jpg)
RESOURCES
• Executive Guide – Measuring Performance and Demonstrating Results of Information Technology Investments – GAO/AIMD-98-89
• Performance Measurement for Government, “Introduction to Performance Measurement”, http://accounting.rutgers.edu/raw/seagov/pmg/perfmeasures
• Manny DeVera, “Putting Performance Measures and Capital Planning Into Practice”, pg. 1 – 25, March 2001 http://wwwoirm.nih.gov/itmra/perform.html
• Robert S. Kalpan and David P. Norton, “The Balanced Scorecard – Measures that Drive Performance”, Harvard Business Review, Jan-Feb 1993
• Dave Trimble, “How to Measure Success: Uncovering the Secrets of Effective Metrics”, pg. 1- 7, http://www.prosci.com/metrics.htm
• Dave Trimble, “Benchmarking – Uncovering Best Practices and Learning from Others”, pg. 1 – 8, http://www.prosci.com/benchmarking.htm
• Mark Glover, “A Practical Guide for Measuring Program Efficiency and Effectiveness in Local Government”, Tampa, FL: The Innovation Groups, 1994
• Office of Management and Budget, “OMB Circular No. A-11 (2000)• “Performance Measures for the Criminal Justice System”, Discussion Papers
from the BJS-Princeton Project, October 1993, NCJ-143505