Tracking and Measuring Outcomes in King County, WA

Genevieve RoweMental Health, Chemical Abuse and Dependency

Services Divisiongenevieve.rowe@kingcounty.gov

King County Mental Health, Chemical Abuse and Dependency Services Division

Manages publically funded mental health & substance abuse services and services for co-occurring disorders.Situated within the Department of Community and Human Services.Coordinates with regional human services partners via King County Human Services Framework Policies.

King County's priorities for human service investments will be programs and services that help to stabilize and improve people's lives, and prevent or reduce emergency medical and criminal justice system involvement and costs.

Human Services Priority Areas 1. Effective intervention and prevention strategies 2. Job readiness and employment to increase self-sufficiency 3. Prevention and elimination of homelessness 4. Services that reduce the growth of emergency medical and criminal justice

system involvement and costs

Building a routine data collection process for evaluating supported housing and treatment programs -

• Has taken 10 years• Isn’t finished• Still includes some manual entry

and cleanup• Has considerably improved

evaluation efficiency

In the beginning we had data for MHCADSD funded services -

• MH Service Data • Community Psychiatric Hospital

Data• Crisis Data (included civil

commitments to state hospitals admit date only)

In the late 1990’s the county jails were reaching capacity. Many frequent users were enrolled in public MH/SA treatment services but were not really engaged in those services.

MH Service Data Community Psychiatric Hospital DataCrisis Data (included civil commitments to state hospital admit date only)

• King County Jail Booking Data was provided daily to help locate clients for service engagement.

The 2003 Criminal Justice Initiative funded new services and added evaluation resources. A new Data Sharing Agreement allowed use of the daily jail booking data for evaluations and added -

Daily King County Jail Booking DataMH Service Data Community Psychiatric Hospital DataCrisis Data (included civil commitments to state hospital admit date only)

• Detailed KC Jail Booking Data – a quarterly file of all bookings since 2001. Includes charges, flag if booking involved psychiatric housing, detention vs. EHD. Note: this added PHI.

Many data sources are available retroactively. There is time to work things out.

Additional data sources were added for a few small programs -

Quarterly Detailed KC Jail Booking DataDaily King County Jail Booking DataMH Service Data Community Psychiatric Hospital DataCrisis Data (included civil commitments to state hospital admit date only)

• Municipal jail booking data (manual collection and entry process)

• Emergency Department data from one major hospital (clients signed Release of Information forms)

• Monthly logs with data for clients not in MH system

• Substance Use assessment forms (faxed hard copy)

A 2007 grant funded the FACT program and a .6 FTE evaluator. IRB approval and Confidentiality Agreements allowed collection of additional data -

Municipal Jail Booking DataPartial Emergency Department DataQuarterly Detailed KC Jail Booking DataDaily King County Jail Booking DataMH Service Data Community Psychiatric Hospital DataCrisis Data (included civil commitments to state hospital admit date only)

• Detailed State Psychiatric Hospital Data (including forensic admissions) for FACT population only

• Detailed Prison Data for FACT population only

FACT demonstrated the importance of complete state hospital data for this population. An agreement was reached to provide -

Detailed State Psychiatric Hospital DataPrison DataMunicipal Jail Booking DataPartial Emergency Department DataQuarterly Detailed KC Jail Booking DataDaily King County Jail Booking DataMH Service Data Community Psychiatric Hospital DataCrisis Data (included civil commitments to state hospital admit date only)

• Detailed state psychiatric hospital data for civil admissions and flips (early 2011).

• Detailed state psychiatric hospital data for forensic admissions (mid 2011).

In 2008 the Mental Illness and Drug Dependency Action Plan, funded by a one-tenth of one cent sales tax increase, was enacted. The MIDD brought additional evaluation resources and began the long process of negotiating for complete jail data -

Complete Detailed State Psychiatric Hospital DataPrison DataMunicipal Jail Booking DataPartial Emergency Department DataQuarterly Detailed KC Jail Booking DataDaily King County Jail Booking DataMH Service Data Community Psychiatric Hospital DataCrisis Data (included civil commitments to state hospital admit date only)

• Complete Municipal Jail Data from all five jails available in December 2011.

After 10 years -

Complete Municipal Jail Data Complete Detailed State Psychiatric Hospital Data Prison Data Partial Emergency Department Data Quarterly Detailed KC Jail Booking Data Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data

- and monthly logs, other data forms

Where we are now -

• Complete KC Jail and Municipal Jail data

• Complete Community and State Psychiatric Hospital data

• Mental Health Service and Crisis data

• No Prison data (discontinued)

• Partial ED data for some programs (expanding to larger programs)

How Data Is Transferred

In the Beginning…• Secure FTP• Password protected

disks• Faxed forms and

spreadsheets

Now…• Secure FTP• Encrypted thumb

drives• Encrypted email• Faxed forms

You will be asked …..

• How much did we save?• Can you do a cost benefit analysis?• What are the cost offsets?

A cost benefit analysis is not an extra step in a program evaluation.

Jail Days Are Not All Equal

Booking Fee

Daily Maintenance

Charge

Infirmary Care

Surcharge

Acute Psychiatric

Housing Surcharge (including

Psych Care)

Psychiatric Care

SurchargeWER

Charge

1:1 Guarding Surcharge (per hour)

$95.00 $125.00 $184.00 $227.00 $55.00 $93.00 $58.00

What about a comparison group?

• Whenever the number eligibles exceeds program capacity there is an opportunity for a comparison group.

• In a non-research setting it is very difficult to execute a randomized experimental evaluation design; but it can happen.

• Randomization or any limits to access to public services probably requires IRB approval.

• In reality you have no control over a control group.

How to begin…

• Start small and simple: who, what, when, why• Look for opportunities to build capacity• Be aware of what others have the capacity for• Be patient – electronic records usually don’t

disappear• Address data security, confidentiality, and

HIPAA• Have participants sign an ROI if there is any

question about permission to access data

Sample Data Collection Sheet – Coding Instructions

Demographics_Program Services_OutcomesClient ID Client IDSurname (Last Name) Month YearGiven Names (First Name) Support Services HrsSite {see list on DemoValues tab}Gender F/M/UDOB MM/DD/YYYY (If unknown enter 01/01/1900)SSN (If unknown enter 999999999)Race1 {see list on DemoValues tab}Race2 {see list on DemoValues tab}Race3 {see list on DemoValues tab}Race4 {see list on DemoValues tab}Hispanic Origin {see list on DemoValues tab}Primary Language {see list on DemoValues tab}"Other Language" See Data DictionaryInterpreter Required N/Y/UImmigrant/Refugee N/Y/UMilitary Status N/Y/UFamily Military Status {see list on DemoValues tab}Zip CodeDisability1 {see list on DemoValues tab}Disability2 {see list on DemoValues tab}Disability3 {see list on DemoValues tab}Start DateEnd DateExit Reason {see list on DemoValues tab}Exit Disposition {see list on DemoValues tab}

This workbook is used to collect and report individually identified demographic, program and service data to MHCADSD. This version is designed for manual entry of data into the "Demographics_Program" and "Services_Outcomes" worksheets.

The "DemoValues" tab has codes and descriptions that are used for some of the data elements (columns) in the worksheets. These are used to control values entered for variables noted below.

See the "Data_Dictionary_Generic_09012010" PDF for more information about data formats, values, and definitions.

Sample Data Collection Sheet – Code Values

Code SiteXYZ {Name of housing program}

Code Race Rollup Group MH Code CD CodeAE African - Ethnic African American 871AA Black, African American or Negro African American 040 2AAR African American (general - use more specific if known)African AmericanAI American Indian or Alaska Native American Indian 021 13ASI Asian Indian Asian/Pacific Islander 031 8CAM Cambodian Asian/Pacific Islander 604 12CHI Chinese Asian/Pacific Islander 605 3FIL Filipino Asian/Pacific Islander 608 5GUA Guamanian or Chamorro Asian/Pacific Islander 660 6JAP Japanese Asian/Pacific Islander 611 9KOR Korean Asian/Pacific Islander 612 10LAO Laotian Asian/Pacific Islander 613 11NHI Native Hawaiian Asian/Pacific Islander 032 7OAS Other Asian Asian/Pacific Islander 034 23SAM Samoan Asian/Pacific Islander 695 16THA Thai Asian/Pacific Islander 618 17OPI Other Pacific Islanders Asian/Pacific Islander 033 22VIE Vietnamese Asian/Pacific Islander 619 19API Asian/Pacific Islander (general: use more specific if known)Asian/Pacific Islander 600CAU White / Caucasian Caucasian 010 20ME Middle Eastern Other Race/Unknown 24UK Not Reported / Unknown/Refused to Answer Other Race/Unknown 999 15, 21OTH Other Race (general: use more specific if known) Other Race/Unknown 050 14

Code Hispanic Origin Rollup Group MH Code CD CodeCUB Cuban Hispanic 709 1MEX Mexican/Mexican-American/Chicano Hispanic 722 2PR Puerto Rican Hispanic 727 5OSH Other Spanish/Hispanic Hispanic 799 4OTH Hispanic (general: use more specific if known) HispanicNOT Not Spanish/Hispanic NotHispanic 998 3UK Unknown/Refused to answer NotHispanic 999 6,7

Sample Data Collection Sheet – More Code Values

Code Family Military Status Drop Down Choices

01Dependent child of a person who served in the U.S. military Dependent child

02Spouse or domestic partner of a person who served in the U.S. military Spouse

03

Neither the dependent child, or the spouse or domestic partner of a person who served in the U.S. military Neither

08Not asked - the question was not asked or client refused to answer. Not asked

09Unknown - this information is not available at present. Unknown

Code Exit ReasonsCX Completed programRO Needed more/different servicesPV Violation of policy (ex: nonpayment, broke rules)MA Moved out of areaDI DiedOTH Other

Code Exit DispositionLS Move to less supportive housingMS Move to more supportive housingSS Return to streets/Sobering/shelterFF Move in with friends/familyMHT Move for mental health treatmentCDT Move for chemical dependency treatmentINC Incarcerated or jailedHOS HospitalizedNH Moved to nursing homeNA Not applicable

Sample Data Collection Sheet – More Code Values

Code DisabilityNOD None - No disabilityDD Developmental disabilitySCD Sensory or communication disabilityMPD Medical/physical disabilityOTH Other disabilities not listed Code if learning disabilityMUL Multiple disabiliites {only allowed in Disability1}UK Unknown

Code Primary Language Rollup Group MH Code CD CodeASL American Sign Language 15 3AMH Amharic 23 1ARA Arabic 51 2BOS Bosnian 52CAM Cambodian 6 7CAN Cantonese 16 5CZE Czech 26 8DUT Dutch 9ENG English 13FAR Farsi 25 10FIN Finnish 24 11FRE French 12 12GER German 14 13GRE Greek 21 14GUJ Gujarati 15HIN Hindi 53 17HMO Hmong 8 16HUN Hungarian 17 18ILO Ilocano 10 19IND Indian(General) 20ITA Italian 60 21JAP Japanese 1 22KOR Korean 2 23LAO Laotian 5 24MAL Malay 27MAN Mandarin 7 29MAR Marathi 30MIE Mien 27 28NOR Norwegian 34 31ORO Oromo 54OAF Other African 81OAS Other Asian 85OCM Other communication methods (e.g., lip-reading, finger-spelling, letterboard, cued speech) 87

Sample Data Collection Sheet – Master List and Demographics

Client ID SurnameGiven

Names Site Gender DOB SSN Race1 Race2 Race3 Race4Hispanic

OriginPrimary

Language"Other

Language"Interpreter

required

123 Test T

Immigrant /Refugee

Military Status

Family Military Status

Zip Code Disability1 Disability2 Disability3 Start Date End Date Exit Reason

Exit Disposition

Sample Data Collection Sheet – Monthly Service Hours

Client ID Month Year Support Services Hrs IDValidationFormulas Surname Given Names

123 FALSE Test TTRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/ATRUE #N/A #N/A

If the Client ID entered here is not found on the "Demographics_Program" sheet, an alert to check the Client ID will be displayed in Column D-E.

If the Client ID is correct, the name will be displayed under "Surname" and "Given Names"

One page does it all…

Monthly Reporting Spreadsheet Date

Since Enrollment: Incarcerations: ER Visits:

Name DOB

Prgram Start Date

Housing Changes (where TO)

Date of Move

Days homeless in last month

Jail Name

Booking Date

Release Date

ER Name

Visit Start Date

Visit End Date

Hospital Name

Admit Date

D/C Date

Medical or Psych Hospitalization: