addd (automated data driven decisions) – how to make it work

Automated Decision Making with Big Data – How to make it workLars Trieloff | @trieloff

ADDDAutomated Data Driven Decisions

— Holger Kisker, Forrester Research

“Even after more than 20 years of using BI, they still base nearly 45% of business decisions on qualitative decision factors instead of quantitative, fact-based evidence. “

If data is not used for decision making, what is used then?

4%Worldwide average profit margin in retail: 4%

4‰German average profit margin in retail: 4‰

Your Customer gives you this

All you got to keep is that

— –Libby Rittenberg

“Economic profits in a system of perfectly competitive markets will, in the long run, be driven to zero in all industries.”

Physiological

Safety

Love/Belonging

Esteem

Self-Actualization

— Abraham Maslov – probably never said this. It’s true anyway.“Data has Human Needs, too”

Collection

Collection

Storage

Collection

Storage

Analysis

Collection

Storage

Analysis

Prediction

Collection

Storage

Analysis

Prediction

Decision

Collection

Storage

Analysis

Prediction

Decision

Physiological

Safety

Love/Belonging

Esteem

Self-Actualization

— W. Edward Deming

“In God we trust, all others bring data”

How Data-Driven Decisions should work

Computer Collects

Computer Stores

Human Analyzes

Human Predicts

Human Decides

— Daniel Kahneman

“Prejudice against algorithms is magnified when the decisions are consequential.”

How Data-Driven Decisions REALLY work

Computer Collects

Computer Stores

Human Analyzes

C O M M U N I C AT I O N

B R E A K D O W NHuman Decides

— Led Zeppelin

Communication Breakdown, It's always the same, I'm having a nervous breakdown, Drive me insane!

• Drill-down analysis … misunderstood or distorted

• Metrics dashboards … contradictory and confusing

• Monthly reports … ignored after two iterations

• In-house analyst teams … overworked and powerless


C O M M U N I C AT I O N

B R E A K D O W N


http://dilbert.com/strips/comic/2007-05-16/

How Decisions REALLY should work

Computer Collects

Computer Stores

Computer Analyzes

Computer Predicts

C O M P U T E R D E C I D E S

— Everyone at Blue Yonder, all the time

99.9% of all business decisions can be automated

How Decisions are Being Made

90% No Decision is made

— Robin Sharma

“Making no decision is a decision. To do nothing. And nothing always brings you nowhere..”

Business Rules for Beginners

Not doing anything is the simplest business rule in the world – and also the most popular

90% No Decision is made

9% Decision Follows Rule

Advanced Business Rules

Computers are machines following rules. This means business rules are programs.

• Business rules are like programs – written by non-programmers

• Business rules can be contradictory, incomplete, and complex beyond comprehension

• Business rules have no built-in feedback mechanism: “It is the rule, because it is the rule”

Business rules are Programs, just not very good ones.

— Mark Twain

“It ain’t what we don’t know that causes trouble, it’s what we know for sure that just ain’t so”

1% Human Decision making

Human Decision Making has two systems – and only one is rational.

Not quite Almost there That’s it.

Quick: What do you see here?

— Steven Pinker, describing Moravec’s Paradox

“The hard problems are easy and the easy problems are hard.”

Quick: Add all even numbers

65 7 1 0

60 63 18 80

547039100

69 20 26 73

94 39 37 31

92 70 100 67

4956080

69 20 26 73

51 60 23 22

5 48 43 14

9525669

23 67 1 43

Correct Result:

Correct Result: 1.024

— Daniel Kahneman

“All of us would be better investors if we just made fewer decisions.”

How we are making decisions (Like the big apes we are)

Anchoring effectIKEA effect

Confirmation bias

Bandwagon effect

Substitution

Availability heuristic Texas Sharpshooter Fallacy

Rhyme as reason effect

Over-justification effect

Zero-risk bias

Framing effect

Illusory correlationSunk cost fallacy

Overconfidence

Outcome bias

Inattentional Blindness

Benjamin Franklin effect

Hindsight bias

Gambler’s fallacy

Anecdotal evidenceNegativity bias

Loss aversion

Backfire effect

• Abraham Lincoln and John F. Kennedy were both presidents of the United States, elected 100 years apart.

• Both were shot and killed by assassins who were known by three names with 15 letters, John Wilkes Booth and Lee Harvey Oswald, and neither killer would make it to trial.

• Lincoln had a secretary named Kennedy, and Kennedy had a secretary named Lincoln.

• They were both killed on a Friday while sitting next to their wives, Lincoln in the Ford Theater, Kennedy in a Lincoln made by Ford.

K-Means Clustering

Naive BayesSupport Vector Machines

Affinity Propagation

Least Angle Regression

Nearest Neighbors

Decision Trees

Markov Chain Monte Carlo

Spectral clustering

Restricted Bolzmann Machines

Logistic Regression

Computers making decisions (cold, fast, cheap, rational)

• A machine learning algorithm is a system that derives a set of rules based on a set of data

• It is based on systematic observation, double-checking and cross-validation

• There is no magic, just data – and without data there is no magic either

Machine Learning means Programs that write Programs

Better Decisions through Predictive Applications

How Predictive Applications Work

Collect & Store Analyze Correlations

Build Decision Model

Decide & Test Optimize

— Warren Buffett

“I checked the actuarial tables, and the lowest death rate is among six-year-olds, so I decided to eat like a six-year-old.”

More than half of the apps on a typical iPhone home screen are predictive applications.

Building Predictive Applications

Machine Learning ModelPredictive Application

Enterprise Integration

Story Time(Not safe for vegetarians)

The Ground Beef Dilemma

How much ground beef are we going to

sell on Friday?

How much ground beef are we going to sell on Friday?

And how much on Saturday?

Challenge #1 Accurately predict demand

Great. But how much do we need to order

each day?


each day?

Let’s reduce the risk of running out of

stock to 20%

Sales Forecasts for FridaySa

les P

roba

bilit

y

0

0,01

0,02

0,03

0,04

0 4 8 12 16

Friday Sales Amount

Sales Forecasts for SaturdaySa

les P

roba

bilit

y

0

0,01

0,02

0,03

0,04

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Saturday Sales Amount


each day?

Let’s reduce the risk of running out

of stock to 20%

So it’s 3 on Friday and 5,5 on Saturday.

Sales Forecasts for Both DaysSa

les P

roba

bilit

y

0

0,01

0,02

0,03

0,04

0 4 8 12 16

Friday Sales Amount Saturday Sales Amount

Bad news…

Bad news…

We need to skip the Saturday delivery.

Bad news…

We need to skip the Saturday delivery.

How big should we make the Friday delivery

instead?

If you need 3 on Friday and 5,5 on Saturday to fulfill 80% of the demand, how much do you need to fulfill 80% of the combined demand?

3 + 5,5 = 8,5 Common Sense isn’t it?

— Albert Einstein

Common sense is what tells us the world is flat.

Combined Sales ForecastsSa

les P

roba

bilit

y

0

0,01

0,02

0,03

0,04

0 4 8 12 16

Combined Sales Amount

If you ordered 8,5 cases, you would waste a lot of meat, the ideal order amount is 8 cases.

Predictive Apps in a NutshellBatch and streaming data ingestion, batch

and streaming delivery (with real-time option)

Reduce risk and cost » increase revenue and profit

Trend Estimation Classification Event Prediction

Optimize Returns

Collect Data Predict Results Drive Decisions

— John Maynard Keynes

“When my information changes, I alter my conclusions. What do you do, sir?”

One Common Platform for Predictive Applications

Multi-Tenant Runtime Environment

Link Store Build Run View

Link your own and third-party data, easily

integrated via API

Store your data in high-performance

database as a service

Build machine learning and

application code

Automatically runand scale ML models

and applications

Monitor and inspectresource usage and

model quality

Secure Micro Cloud Infrastructure

Domain Model Predictive Model Application Code

— Kevin Kelly

“The business plans of the next 10,000 startups are easy to forecast: Take X and add AI”

How Enterprises adopt Predictive Applications

Learn about ADDD

Define Target Process

Build Predictive App Go Live Make Lots of

Money

— Daniel Kahneman

“Prejudice against algorithms is magnified when the decisions are consequential.”

How Enterprises REALLY adopt Predictive Applications

Learn about ADDD

Define Target Process

Build Predictive App

Make Lots of Money

D O U B T S C O N C E R N S

O B J E C T I O N S

Decision Quality

Status Quo Predictive Prescriptive Automation Automation+

Ready, set, go for ADDD?

Not so fast

Data Availability

Data Availability

Predictability

Data Availability

Predictability

Understandability

Data Availability

Predictability

Understandability

Executability

Lars Trieloff @trieloff (this guy is hiring)