Area 4 SHARP Face-to-Face Conference

Phenotyping Team – Centerphase Project

Assessing the Value of Phenotyping Algorithms

June 30, 2011

Topics

Centerphase Background Project Overview Hypothesis Research Design Results to-date Next Steps

Centerphase: Background

CENTERPHASE SOLUTIONS, INC. is a technology-driven services company formed through a collaboration with Mayo Clinic in 2010

The goal is to leverage electronic medical records (EMRs) and clinical expertise from academic medical centers and other research sites to address a broad array of healthcare opportunities

The initial focus is to support enhanced design, planning and execution of clinical trials

Future areas include comparative effectiveness, pharmacoeconomics, compliance and epidemiological studies

Centerphase’s role on the Phenotyping Team is to evaluate the effectiveness (cost and time) of using phenotyping algorithms for identifying patient cohorts

It’s About Speed AND Accuracy…

Hypothesis

The development of phenotyping algorithms and tools can reduce time and cost while maintaining or enhancing quality, associated with identifying patient cohorts for multiple secondary uses including clinical trials and care management.

1. Choose a use case that can provide valuable insights into a real world application

2. Develop a phenotyping methodology (“flowchart”) to identify the patient cohort

3. Generate a random sample of patients from Mayo EMR system based on ICD9-code

4. Conduct algorithm-driven and manual processes in parallel on the sample

5. Compare the time, cost and accuracy of results from the algorithm-driven to manual process

Approach

Type II Diabetes Mellitus (T2DM): 90-95% of all adult cases of diabetes

Multi-stage phenotype representing a combined adaptation of:

– The eMERGE Northwestern T2DM algorithm for clinical trial selection and

– The group practice reporting options (GPRO) as defined under NCQA for population management under the Southeast Minnesota Beacon project

Diagnosed and Undiagnosed Diabetes Source: 2005–2008 National Health and Nutrition Examination Survey

Diabetes is a growing epidemic in this country: 25.8 million (8.3% of population) have diabetes. Last year, 1.9 million new cases alone in population of 20 years or older (CDC).

Initial Use Cases

Use Cases

Case 1: Care Management Identify all high risk patients in a pool of 500 cases

Case 2: Clinical TrialIdentify patients that are good candidates for a study

Phenotype MethodologyT2DM

ICD9 Code

Screen 1: Age

Screen 2: Medications

Screen 3:Labs & Vitals

Patient Cohort

eMERGEAlgorithm for T2DM

Beacon Criteria for categorizing patient risk Identified as high risk or

“RED” patient

Note: All screens based on two-year measurement period 1/1/09 – 12/31/10

Blood Glucose: HbA1c > or = 9

Cholesterol: LDL > 130

Blood pressure: Systolic > 160 & Diastolic > 100

If ANY of the most recent values exceed allowable levels OR ANY of these elements has not been captured in the measurement period, patient is classified as high risk or RED

Identified as T2DM patient

Randomly generate ONE sample set of patient records from database:Based on T2DM ICD9 codes from at least 2 visits during measurement period

Sample Patient Records

Screens 1 -3 Screens 1 -3

Patient Result Set

Patient Result Set

Manual Process

Algorithm-DrivenProcess

Compare time, cost and accuracy of results

Study coordinator (SC) conducts manual review of patient charts, and monitors activity time

Programmer develops and runs algorithm to query records, and monitors development and run time

Research Design

Validation and Evaluation Process

“Dry Run”20 Charts

500 Charts

Step 1

• Review each chart for manual and algorithm processes to identify any screening errors

• Confirm approaches are consistent

• Refine procedures as appropriate

Step 2

• Start with 50; review results and adjust if necessary

• Complete manual reviews

• Collect time, cost and patient result sets

• Conduct data queries

• Analyze results and evaluate / compare performance of methods

Step 1 Completed Step 2 Underway

And How Did We Do…

Initial Results• 50 Charts reviewed

• Manual process*• Identified 10 “Red” (high risk) patients• Required 11.5 total hours**

• Algorithm-driven process*• Identified 8 “Red” patients

All 8 were identified in manual process Missed 2 patients (false negatives)

• Required 7.4 hours**

• For the purposes of this presentation, the following analysis extrapolated these results to evaluate the impact on 500 patients…. Actual findings will be reported upon completion of 500 charts

* Currently evaluating accuracy of both manual and algorithm-driven processes** Includes time for manual validation of all Red charts

Preliminary Analysis: Case 1 - Care Management Extrapolated to 500 Charts based upon Initial 50 Charts

Algorithm

80%Less Costly

Algorithm

90%Faster

Note: Costs and hours reflect time for secondary manual validation of all Red charts identified through both processes

Preliminary Analysis: Case 2 – Clinical TrialsExtrapolated to 500 Charts based upon Initial 50 Charts

Manual Method

Algorithm Method

80% fewer charts to review Over 30 hours saved

Preliminary Comparison: Algorithm-driven to Manual processAlmost 50% cost savings

Note: Costs and hours reflect time for secondary manual validation of all Red charts identified through both processes

Preliminary Conclusions and Next Steps

Initial takeaways:If extrapolated results are validated….

•Applying algorithms to identify subsets of patients can save time and be cost effective

•Algorithms can be most effective when search for larger numbers of patients

•More work needs to evaluate relative accuracy

Sample Patient Records

Screens 1 -3 Screens 1 -3

Patient Result Set

Patient Result Set

Manual Process

Algorithm-DrivenProcess

Next steps:•Complete review of 500 charts•Document results in white paper or manuscript