pandurang kulkarni, eli lilly presentation at cdao winter 2017

Making analytics actionable and driving value for your business Pandu Kulkarni, PhD Chief Analytics Officer VP, Biometrics & Advanced Analytics Eli Lilly and Company

Key Points

Why Smart Analytics? How can they help with business outcomes? How can we make this routine in all parts of business?

Company Confidential © 2017 Eli Lilly and Company Slide 2

Consideration #1

The Value of Controlled Experiments

Or

Why you cannot always trust ‘observational’ data.


Polio and Ice Cream

Eliminating ice cream was recommended

as part of an anti-polio diet!

Dumb Analytics

Company Confidential © 2017 Eli Lilly and Company

Slide 4

Divorce Rate & Margarine

Source: CDC and USDA Wrong Big Data Company Confidential © 2017 Eli Lilly and Company Slide 5

Current Paradigm

Big Data

Analytics Information Knowledge

Wisdom Decisions


The Better New Paradigm

SMART Analytics

Information Knowledge

Wisdom Decisions

Right Data


Direct Mail Test Design

Past Present

Randomized, Controlled Study

4,000 HCP’s Receive Specific Message

500 HCP’s Receive

No Message

Control

4,500 HCP’s Receive Specific Message

Observational Study


Question: If we email a lower volume HCP with no sales force coverage, will this generate revenue?

→ ROI of $3.3:1

→ ROI of $0.8:1

Different Conclusions

4500 HCP’s Receive Specific Message

Observational Study

+


Consideration #2

“Torture the data long enough and they will confess to anything.”

Dr. Ronald Coase Nobel Laureate, Economics (1991)


Personalized Medicine

Tailored Therapeutics Genomic Medicine Targeted Medicine Precision Medicine

Problem Statement Are there specific genes, genetic variants, proteins or other biomarkers that can be used to predict which patients will have an exceptional response* to our drug and which ones will not? *Response could be efficacy or safety.


The Basics

Gene = string of base pairs

SNP (single nucleotide polymorphism) = one difference in one base pair in one gene

Proteins


Personalized Medicine in Action • Disease State Y has no known biomarkers that predict individual

patient outcomes

• Modest size clinical trials with Drug X for Disease Y (100’s of patients)

• Doses (mg) – 0, 1, 3, 10, 30, 60, 120 • Two highest doses seemed most effective – used in analysis

• Visits – every 1-2 weeks • Samples at weeks 0, 4, 8, 16 (end of treatment)


Personalized Medicine in Action • Samples from patients before and after treatment • Perform extensive genetic assays and biomarker analysis

• Responder = combination of clinical response z1 & z2

• Eliminate biomarkers with low variability • Eliminate patients with low variability in response

• Path analysis to map biological pathways


Unsupervised Clustering Analysis


Statistical Comparisons

3,000,000 P-values and changes

100,000s++ “gene to gene” interactions

2000++ gene pathway maps

54,675 possible biomarker relationships

Probset ID Gene Symbol p-valueV2R p-value RV2vX2 X pathway Y genetic genes DS related199983 7.33E-05 0.530706 No No No200043 8.03E-05 0.240931 No No No199971 9.68E-05 0.00276855 No No No200410 9.78E-05 0.127399 No No No199725 0.000157751 0.124253 No No No199944 0.000262992 0.00463293 No No No200084 0.000277312 0.354878 No No No200001 0.000316512 0.0278983 No No No200308 0.000343172 0.00219565 No No No200045 0.000444202 0.389677 No No No199857 0.000451176 0.218905 No No No199899 0.00046328 0.00223889 No No Yes200181 0.000509819 0.00110896 No No No199908 0.000565259 0.267577 No No Yes199729 0.00065247 0.406751 No No Yes200084 0.00071322 0.604382 No No No199716 0.000717178 0.0485064 No No No199923 0.00072256 0.117219 No No No200750 0.000730452 0.265865 No No No200380 0.000789635 0.00515711 No No No199693 0.000989427 0.384253 No No No199926 0.00102509 0.0220815 No No No199744 0.00115624 0.000184645 No No Yes200157 0.00118649 0.0962912 No No No


Messina Analysis: Narrowed down to 2 biomarkers in the same class

Non-Responders

* Messina: A Novel Analysis Tool to Identify Biologically Relevant Molecules in Disease, PLoS ONE 4(4):e5337, 2009

Responders


http://www.plosone.org/article/info:doi/10.1371/journal.pone.0005337



Eureka !

• We found predictive biomarkers for Drug X in Disease State Y !

A Scientific First !!!


Confirmatory Clinical Trials

• Large scale confirmatory trials • Select doses with best efficacy and safety • Samples for biomarkers collected

• Focus on the two we found

Results • Absolutely no relationship between the biomarkers and

clinical outcomes


This is not a unique case….


The Lilly Experiment

• RANDOMLY generate very large data sets with many, many variables • Like clinical trial and genomics data

• Send to analytics companies/vendors

• They have always found patterns even when none were there.

FALSE POSITIVE FINDINGS


Observations • “Lack of Reproducibility” in research has become a major issue

• Genomics • Health Care • Clinical Trials

• Emerging view: partly due to poor/wrong analyses • Lack of statistical rigor • Under-appreciating bias • Over fitting results • Naïve belief in big data

Need for Right Data & Smart Analytics Company Confidential © 2017 Eli Lilly and Company Slide 23

Consideration #3

Making what’s advanced today routine tomorrow.


An Example of Federated (‘hub and spoke’) Model for Analytics

Role of Enterprise Analytics • Create talent/ people strategy • Maintain a small Analytics Hub for innovation and

‘platform’ capabilities • Shape analytics externalization and partnerships

(with Data Council) • “Raise the bar” on organizational capabilities

Role of Analytics “spokes” (in the business):

• Work with Bus partners on questions/projects • Execute analytics, leverage Enterprise Hub • Drive integration and adoption of analytics into

business processes and decision-making

Commercial Global

Manuf.

Enterprise Analytics

HR & Other

R&D


An Example of Creating Data Strategy

Data Leader 1

Data Leader 2

Data Leader 3

Data Leader 4

• Shape data strategy • Define data standards and

governance • Coordinate external data

partnerships • Work with IT to identify data

architecture needs • Support data stewards in the

business (“spokes”) in identifying Right Data

Role of Council: Unleash the full potential of data at the company


• Chief Analytics Officer (Chair)

• CIO

• Legal & Privacy

• Elanco (TBD)

• Manufacturing (TBD)

Existing organization

Emerging organization

New (proposed) organization

What are Right Data and Smart Analytics?

Sample Language to Use

Analytics

Data

IT

• Technical expertise to optimize business investment and model/predict future outcomes

• Knowledge of best analytical methods to address problems

The “science”

• Providing platforms to facilitate data acquisition, storage, transformation, and analysis

The “art”

• Formulating the right business questions

• Going the “last mile” to put analytics at the heart of business decision making

• Integration of disparate data sets to create (e.g., one view of the customer)

• Identifying creative uses of data to generate disruptive insights

• Making intentional, “use case” driven technology choices to get to insights faster

• Effective policies and procedures for the creation, consumption, and retention of data (internal and external)


Small but Effective Central Capability Team: Creating Advances. For example:

Working with Teams / Projects

50%

• Direct / immediate impact on the business (i.e. IMMEDIATE VALUE)

• Source of problems that need solving - ideas for innovation (i.e. FUTURE VALUE)

• Metrics • Reduce cost or time • Increase probability of success • Inform/change strategy

Dev

elop

N

ovel

Met

hods

15% 15% 15% 5%

Dev

elop

So

ftwar

e/To

ols

Trai

n on

M

etho

ds/T

ools

Exte

rnal

Influ

ence


Making Advances Routine

Teams / Projects

Novel Methods

Tools

Training External Influence

Problems

Solutions Scalability

Implementation

Acceptance


Conclusion • Big Data is here to stay

• We need to understand how to use it properly • Statistically (good methods and inference) • Socially (privacy, etc.)

• Rich Data can be equally valuable • Small amounts of well collected data

• Data is nothing without SMART ANALYTICS


The Need for Smart Analytics

“Science is built up of facts [data], as a house is built of stones; but an accumulation of facts [data] is no more a science than a heap of stones is a house.”

Henri Poincare Science and Hypothesis, 1905


pandurang kulkarni, eli lilly presentation at cdao winter 2017

Data & Analytics