using the power of predictive analytics for case outcomes · pdf fileericsa 52nd annual...

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30

Hershey Lodge ▪ Hershey, Pennsylvania

Using the Power of Predictive

Analytics for Case Outcomes

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Participants

Presenter – Edward V. Lehman, Jr.

Director, Case Processing & Data Management

Philadelphia Family Court – Domestic Relations

Division

Presenter – Steven J. Golightly, Ph.D.

Los Angeles County Child Support

Moderator – Joyce Match

Business Analyst Manager

Pennsylvania Bureau of Child Support

Enforcement

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 Hershey Lodge ▪ Hershey, Pennsylvania

Predictive Analytics in Action Edward V. Lehmann, Jr.


Access to the right information is extremely important

What is the requirement?

Increase Visible Results

Reduce Information

Overload

Work the Right Cases

Effective Case Worker Management

Determine Next Appropriate

Action Proactively


Understanding Past, Measuring Present, Predicting the Future


Predictive Analytics in Child Support

• Better estimates of delinquencies, child support collections • Fewer child support cases in arrears • Model results can be used to quantify historical process inefficiencies • Significant marketing value – “We Know Our Citizens” • More reliable payments for children

• Shift from reactive enforcement to early intervention • Learning component for new case management approaches • Assign high risk cases to case workers sooner • Different approaches for different regions and different case types • Automate certain research processes

• Reduce user guesswork and “cherry picking” • Prioritize workload based on high impact • Incorporate historical experience to drive future activities • Eliminate a one size fits all approach • Efficiency of customer service interactions • Over time reduce future workload

Improved Workforce

Effectiveness

Increased Resource Allocation Efficiency

Improved Outcomes


Predictive Modeling for Child Support

o Child support enforcement has traditionally been a reactive process.

o What if we had a tool that could help us predict which NCPs are most likely to become in-arrears in the near future?

o We could use such a tool to: o Prevent arrears o Decrease custodial parent complaints o Take the right action on a case at the right time o Assign the right case workers to the right cases o Gather information for potential policy changes

Use analytics to make smarter decisions and do more with less.


How do Models Work?

Statistical models just refine what we do naturally all the time.

For example: which of these NCP’s is most/least likely to pay next month?

Gene: 35 years old $2000 in arrears No other children on case No Salary attachment History of family violence Case is a IV-A assistance

Tom: 28 years old $100 in arrears 1 other child on case Salary attachment No history of violence Case is not an assistance case

Rick: 39 years old $8000 in arrears No other children on case No salary attachment No history of violence NCP has other child support cases


How do Models Work?


Pennsylvania’s Predictive Analytics solution included updates to the existing Performance Improvement Module (PIM) solution and a new application to calculate a predictive analytics “score.”

• Allows a worker to calculate a score for a case based on 20

variables • Score is the likelihood of the defendant to pay 80% towards

current support obligation in the next three months • Information required to generate the score is easily accessed

from the system at the time the support order is created or modified

• Provides a list of upcoming child support establishment conferences with the cases assigned to the worker to allow the score creation prior to the conference.

Pennsylvania’s Solution


PIM/Predictive Analytics

o Targeted case lists o Suggested case actions o Predictive score and

reasons o Performance metrics o Projects for targeted,

custom outreach including text messages


Payment Score Calculator Objectives

• Establish a good

payment pattern: Increased quantity and frequency of collections

• More effective meetings with defendants

• Make available new methods of reaching out

• Identifying opportunities for proactive enforcement activities

• Maximize

performance metrics

• Minimize costs related to enforcement

• Improve cost/effectiveness ratios

• Effective case

assignment based on scores

• Establishing consistent payment patterns based on successful business process actions for various scores

• Provide the right services at the right time to encourage compliance with the order

• Let the defendant know the case is being monitored


Variable Selection Final model contains 20 variables

36 variables considered for modeling after Exploratory Data Analysis [EDA] Phase

64 candidate variables created during data scrubbing / brainstorming phase

Over 400 data elements collected and considered in variable creation phase

The case for which a score is being calculated is compared against the historical behavior of thousands of cases with similar characteristics


Example Predictive Variables Number Predictive Variables

1 Collection Indicator 2 High Number of Enforcement Activities Indicator 3 Number of Cases 4 Number of Enforcement Activities 5 Balance of Arrears 6 Defendant Net Income 7 Active Income Attachment Indicator 8 Number of Defendant Member Addresses on MADD 9 Number of Defendant Employers

10 Distance between the Defendant and Plaintiff

These are just a few examples of predictive variables used in

Pennsylvania. Variables used in each State are different.


Predictive Modeling Scores Probability of NCP paying 80 percent on current support in the three months after support order issued/modified.

0 – 30% 31 – 50%

51 – 79% 80% +


Confidence in the Score

Payment Score

Average Percent of Current Support

1 51% 2 66% 3 77% 4 87%

To validate the accuracy of the payment score, a study of 5,000 PACSES cases was completed

For each payment score, the average FYTD percent of current support collected was calculated.

A direct relationship between payment score and percent of current paid was found (i.e., the

higher the payment score the higher the percent of current paid.


Planned enhancements to the existing functionality of the Payment Score Calculator • Development of a mechanism for calculating a Payment

Score for all open cases • Quarterly refresh the Payment Score Calculations on all

open cases • Allow worker to complete a guideline calculation if the

PSC has been updated within the last three months • Modification of PIM to add several pre-defined filters

which use the Payment Score as one of the factors for consideration

• Implementation planned for June, 2015

PSC Enhancements


After PIM was implemented, Pennsylvania became the first State to achieve above 80% in both percent of current and percent of arrears

PIM/Predictive Analytics Effect in Pennsylvania

PA has been ranked #1 in the United States for both percent of current and percent of arrears, both of which have increased significantly since

the implementation of the Performance Improvement Module


Revenue Impact: Exceeding 80% of collections targets

Accuracy: “Likely payers” 4x more likely to be above 50% paid than “very unlikely payers”

Timeliness: Allows taking action at time of support order, before payments are missed

Actionability: Caseworker gets clear guidance on recommended steps

Policy Insights: Ability to better segment NCPs to better understand policy impacts

Workforce effectiveness:

• Shift from reactive enforcement to pro-active education

• Ability to prioritize workload based on high impact

Efficiency: Increased effectiveness of retention outreach campaign, audits

Time to production: Scored 17,000 cases within first three months

Transparency: Unified view across all regions and business processes

Marketing value: Helps build positive relationships with our clients. Everyone wins when

payments are made

Predictive Analytics Impact in Child Support


Edward V. Lehmann, Jr. Director, Case Processing and Data Management Philadelphia Family Court Domestic Relations Section [email protected]

Feel free to contact us.

Questions? Need More Information?

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30

Hershey Lodge ▪ Hershey, Pennsylvania


WHERE DO WE START? Starting with the Why

Simon Sinek’s Golden Circle

2


WHY

3


TAILORING SERVICES USING

PREDICTIVE ANALYTICS

4

Apply a Predictive Analytics lens to Case Management


PREDICTIVE ANALYTICS – CLASSIFICATION MODEL

Forecast (predict) an outcome using multiple predictors

TARGET: Predict which cases we can expect a payment of at

least 70% of Current Support Amount in subsequent month

Built a Classification Model

• started with 100+ programmatic, demographic, and economic

variables

• narrowed down to 20 statistically significant variables

5

Model accurately

predicts target

variable for

90% of cases.


CASE SCORING

6

0%

10%

20%

30%

40%

50%

0 1 2 3 4 5 6 7 8 9

Payment Status

(Prior Month)

Prediction

(Next Month)

Confidence Score Score

(Propensity

Score)

1 Non-Payer Non-Payer .91 1

2 Non-Payer Non-Payer .73 3

3 Payer Payer .91 9

4 Payer Payer .73 7

5 Payer Non-Payer .55 5

6 Non-Payer Payer .55 5


PREDICTIVE ANALYTICS - SEGMENTATION

Built a Segmentation Model

• Started with same variables as in classification model

• Clustering Analysis identified natural groupings based on payment history

Group cases using one or more input fields

7

TARGET: Create cluster of cases based on case variables to

identify natural grouping of cases


EARLY SUCCESSES

Arrears Only Project - January 2014

9,486 cases with an Arrears balance, no minor children, no current

support, and no payment within last federal fiscal year

Strategy – Focused Caseload, Locate Experts

Results – $2 million in 12 months!

8

47.96%

55.99%

Department Arrears Only

FPM 4


PREDICTIVE ANALYTICS FINDINGS

Arrears Only Project Success

• Personalized the client’s experience

• Case Worker a subject matter expert for

case type 9

• Predictive Analytic Model identifies cases that

perform similarly

• PA Model predicts, with a high level of confidence,

how a case will perform

• PA Model identifies characteristics and motivations

of different customer types

One-Size Fits One


HOW

CREATE NEW BUSINESS MODEL Case Ownership Case Segmentation

EST

Zero Order

Non-Paying

Occasional

Paying

Arrears Only

Other

Estab.

Zero Order

Paying

Occasional Non-

Paying

Arrears Only

IGR

10


SEGMENTATION MODEL

Intake

Establishment

Arrears Only IGR

Customer Response

11

ENFORCEMENT

Div. 1 Div. 3 Div. 4 Div. 5 Div. 6


ENFORCEMENT CASELOAD

Definition –

Case has an order for $0

OR Case has a reserved or

pending supplemental

order OR Case has a Medical Only

order OR Case does not have an

active order, but case

has an arrears balance

and child is a minor

Occasional Paying

Zero Order Non-Paying

12

ZERO ORDER

QUADRANT



Definition –

Case has an order AND

Current Support Amount

is a dollar amount greater

than $0 AND

NCP pays regularly AND

NCP pays over 70% of

CS amount

Occasional Paying


13

PAYING QUADRANT



Definition –

Case has an order

AND

Current Support

Amount is a dollar

amount greater than

$0 AND

NCP pays

irregularly OR

NCP pays less than

70% of CS amount

Occasional Paying


14

OCCASIONAL PAY

QUADRANT



Definition –

Case has an order

AND

Current Support

Amount is a dollar

amount greater than

$0 AND

NCP has not made a

payment during the

current Federal Fiscal

Year

Occasional Paying


15

NON-PAY QUADRANT


Occasional

Send letters of Introduction to new case manager

UIB/DIB/ Mod review (CMT Sort)

Review for SLMS suspension – Manual submission

for NCP’s with valid license

Incarcerated Special Mod/closing Project

CMT High TMSO paying less than 90% sort

Review EFO case function cases for order for

subsequent child

Work LC005 task for IWOs

Review CMT for SSI benefits

Paying


Low Order with reported Earnings Mod Project

Admin IWO Project

Incarcerated Special Mod Project

CMT High TMSO paying less than 90% sort

UIB/DIB Mod review (CMT Sort)


subsequent child

New Order clerical contact of NCP (EI)


Zero Order


Zero Mod Project


subsequent child

Review Paid in Full cases to confirm emancipated

DP’s updated for auto closures.

Non-Paying

Send letters of Introduction to new case

manager

Review for SLMS suspension – Manual

submission for NCP’s with valid license

Incarcerated Special Mod Project

Review EFO case function cases for order

for subsequent child


Review CMT for SSI benefits

STRATEGIES BY QUADRANT

16


NEW CSTATS

17

Old CSTATS:

• Performance across the

division

• Compares branch-to-branch

New CSTATS:

• Performance across

the division

• And performance by

quadrants

• Compares branches

by quad

• Month, YTD


IMPLEMENTATION

18

Phased Roll-Out

Roadshows

Feedback/Workgroups

New Performance Measures

Reset every FFY


QUESTIONS

19

using the power of predictive analytics for case outcomes · pdf fileericsa 52nd annual...

Documents