data analytics for marketing decision support: introduction and a wallet estimation case study

IBM Research

2006 IBM Corporation

Data Analytics for MarketingDecision Support: Introductionand a Wallet Estimation CaseStudy

Saharon RossetIBM T.J. Watson Research Center

IBM Research

2006 IBM Corporation2

Two parts: Introduction to use of Data Mining in marketing applications

(Collaborator: Naoki Abe) What are the problems we address? Comparison of Data Mining and Marketing Science

approaches Some of the challenges for Data Mining approaches Customer Wallet and Opportunity Estimation: Analytical

Approaches and Applications(Collaborators: Claudia Perlich, Rick Lawrence, SrujanaMerugu and others) Define the problem Describe analytic solutions Demonstrate performance in real application

IBM Research


The grand challenges of marketing Maximize profits (duh) Initiate, maintain and improve relationships with

customers: Acquire customers Create loyalty, prevent churn Improve profitability (lifetime value) Optimize use of resources:

Sales channels Advertising Customer targeting

IBM Research


Some of the concrete modeling problems Channel optimization Cross/up-sell (customer targeting) New customer acquisition Churn analysis Product life-cycle analysis Customer lifetime value modeling

Effect of marketing actions on LTV? Advertising allocation RFM (Recency, Frequency, Monetary) analysis ...

IBM Research


Data analytics for decision support: grand challengeBeyond modeling the current situation, we need

to offer insight about the effect or potential of possible actions and decisions: How would different channels / incentives affect LTV of

our customers? How much more money could this customer be spending

with us (customer wallet) Can we predict the effects of new actions that have never

been tried in historical data? What if they have been tried on non-representative set?

Can we be confident our results are actionable? Can we differentiate causality from correlation in our models?

IBM Research


CRM analytics: Relies on primary research (=surveys) to understand

needs and wants Relies on (more or less) detailed models of customer

behaviorUsually parametric statistical models

Often estimates customer-level parameters Data mining:

Typically relies on data in Data Warehouse /Mart Uses minimum of parametric assumptions Often attempts to fit problem into standard modeling

framework: classification, regression, clustering...

Typical marketing analytics vs. data mining

IBM Research


Comparison of approaches

-+Integrate expert input from managers and customers (wants and needs)

+-Use data to learn new, surprising patterns about customer behavior

+-Robust against incorrect assumptions about domain and problems

-+Actively collect the data to estimate model quantities (active learning)

+-Rely on existing, abundant data in Corporate Data Warehouses

-+Parametric models formalize knowledge of domain and problems

DMMarketingCriterion

IBM Research


Rust, Lemon and Zeithaml (2004), Return on Marketing: Using Customer Equity to Focus Marketing Strategy, J. of Marketing

Modeling customer equity / lifetime value Combine several previous approaches Model the brand switching matrix as a function of customer

preference, history and product properties Want to identify drivers of satisfaction (levers) Calculate effect (ROI) of marketing actions pulling levers Mostly relies on primary research collected specifically for

this study Interviews with managers Survey of consumer preferences

Example 1: modeling and improving LTV

IBM Research


Simplified version of papers business model

Marketing investment

Costs

Pullinglevers

Increasedequity

Return on marketing investment

Main goals: Identify relevant levers Quantify their effect

IBM Research


Analytic setup (main components only) logit(pijk) = 0k LASTijk + xik k

pijk is probability that customer i buys item k given they bought item j previously

LAST is a dummy variable for inertia Xik is a feature vector for customer i, product k

This is used to compute the brand switching matrix {pijk} and customer lifetime value is calculated as:CLVij = t PROFij Bijt PROF is a profit measure considering discounting, price & cost

(assumed known) Bijt is probability customer i buys product j in time t, calculated

from the stochastic matrix {pijk}

IBM Research


Data definitions Potential drivers (marketing activities) are reflected in

the components of xi Price Quality of service etc.

The data to estimate the logit model is based on: Expert (manager) input Questionnaires of customers Corporate data warehouse (not implemented in their case

study...)

IBM Research


Results: important drivers for airline industry?

Etc. (all factors deemed important)

6.56.093.609Convenience

9.86.020.199Price

10.87.041.441Quality

11.34.075.849Inertia

Z score (coeff/std)

Std errorCoefficientDriver

.

.

.

.

.

.

.

.

.

.

.

.

IBM Research


What would a data miner do? Count more (or only) on historical data in data

warehouse Variables would have different meaning Identify correlations, not necessarily drivers

Could use same analytic formulation, but also try alternative approaches Relate LTV directly to variables observed? Model transaction sizes in addition to switching? Use non-parametric modeling tools?Etc.

IBM Research


Common practice in marketing: Define static, fixed customer segments

Supposed to capture true essence of customersbehaviors, needs and wants

Often given catchy names: Upwardly mobile businessmen representing the average profile

Make marketing decisions at segment level, based on understanding of needs and wants

Example 2: the segmentation approach

IBM Research


A market segmentation methodologyBased on Kotler (2000). Marketing Management. Prentice-Hall1. Survey stage: primary research to capture motivations,

attitudes, behaviors2. Analysis stage: factor analysis, then clustering of survey

data Identify segments

3. Profiling stage: analyze segments and give them namesAdditional stage often taken is to assign all customers to the

defined segments:4. Assignment stage: build classification model to assign all

customers to learned segments

IBM Research


What would a data-miner do?Option 1: clustering

Replace primary research by warehouse data Cluster all customers Lose the needs and wants aspect

Option 2: supervised learning Treat each decision problem as separate modeling task

E.g., find positive and negative examples for each binary decision, learn model

Advantage: customized Disadvantages:

May not have right data to model decisions we want to make Past correlations may not be indicative of future outcomes

IBM Research


Comparison of approaches

-+Integrate expert input from managers and customers (wants and needs)

+-Use data to learn new, surprising patterns about customer behavior

+-Robust against incorrect assumptions about domain and problems

-+Actively collect the data to estimate model quantities (active learning)

+-Rely on existing, abundant data in Corporate Data Warehouses

-+Parametric models formalize knowledge of domain and problems

DMMarketingCriterion

IBM Research


Count on historical data as much as possible Avoid complex parametric models

Let the data guide us Still want to integrate domain knowledge Analyze and understand the special aspects of marketing

modeling problems Importance of long-term relationship (lifetime value, loyalty) Effects of competition (customer wallet vs. customer

spending) Modify existing, or develop new, data analytics

approaches to address problems properly

An integrated approach

IBM Research


Moving beyond revenue modelingTo really understand the profitability and potential of our

customers, we need to move beyond modeling their short-term revenue contribution Revenue over time: Lifetime Value modeling

How much can we expect to gain from customer over time? Incorporates loyalty/churn, prediction of future customer

revenue

LTV = t S(t) v(t) D(t) dt(S(t) is customer survival function, v(t) customer value over time, D(t) discounting factor)

Potential revenue: Customer Wallet Estimation How much revenue could we be generating from this

customer? Incorporates competition, brand switching etc.

IBM Research


LTV and Wallet: beyond standard modelingTime

RevenueNow

Future

Next year

Sales / revenue modeling

Sales forecasting

L

T

V

m

o

d

e

l

i

n

g

Potential salesActual sales

Wallet estimation

IBM Research


Types of decision support Passive decision support

Understand more about problems and causes Identify areas of need, under-performance etc. Help in making better decisions

Active decision support Model the effect of actions Actively help in deciding between alternative actions

Active decision support is typically more challenging in terms of data needed to learn models

IBM Research


Depth and actionability of insightsDepth

Actionability

Basic concepts

Real insight

Revenue modeling

ActivePassive

Correlation Causality

Revenue forecast

Lever identification

LTV modelingWallet

estimation

Understand effect of potential actions on LTV and Wallet

attainment

IBM Research


The causality challenge Predictive models discover correlation

Example: linear regressionSignificant t-statistic for coefficients imply they have a significant effect, not that they are actually causing the response

For active decision support we need to identify levers to pull to affect outcome

Only works with causality Causality is difficult to find or prove from observation data

If we have knowledge about causality, we can formalize it as (say) Bayesian network and use in our models

We can get closer to causality by case-control experiments

IBM Research


Assume we observe for some companies:X = companys marketing budget,Y = companys salesand want to understand how to affect Y by controlling X

Assume we find that X is very predictive about Y Possible scenarios:

Illustration: predictive power is not causality

Z

Y X

x y

x y

Causality successfully identified lever

Fixed percent of revenue to marketing?

Z=Company size independently determining both quantities?

IBM Research


Some other challenges Modeling effects of new/unobserved actions

Critical for active support, often difficult or impossible Even for established actions, they may have been

applied in different context than our planned campaign

Integrating expert knowledge into process Can be done formally via graphical models

Handling data issues: matching, leaks, cleaning Always critical

Delivering solutions and results

IBM Research


Example: Telecom Churn ManagementCell phone company has set of customers, some leave (churn)

every monthThe goals of a Churn Management system: Analyze the process of churn

Causes Dynamics Effects on company Design policies and actions to improve the situation

Marketing campaigns Incentive allocation (offer new features or presents) Change in plans to contend with competition

IBM Research


First step: understand current situation Who is likely to churn (predictive patterns)?

Phones features / plans Usage patterns DemographicsTools: segmentation, classification, etc. Which of these patterns are causal?

Tools: expert knowledge, Bayesian networks, etc. Which causal effects not in data?

Competition, economy etc. Which of these customers are profitable?

Short term: customer value Long term: lifetime value Growth potential: customer wallet

IBM Research


Second step: design actions Can we affect causal churn patterns?

For example, by improving customer service

Given possible incentives and marketing actions, what effect will they have?

Loyalty and relationship Current customer value and wallet attainment Customer lifetime value Cost to company

How can we optimize use of our marketing resources? Identify segments we want to retain Identify effective marketing actions

IBM Research


Survey of Useful Methodologies Utility-based classification*: Cost-sensitive and Active Learning

Motivation: need to handle utility of decision and cost of data acquisition in marketing decision problems

Example domains: Targeted marketing, Brand switch modeling

Markov Decision Processes (MDP) and Reinforcement Learning Motivation: need to consider long term profit maximization Example domain: Customer lifetime value modeling

Bayesian Networks Motivation: need to address causality vs. correlation issue; need to

formalize domain knowledge about relationships in data

Example domain: Customer wallet estimation

*c.f. Utility-Based Data Mining Workshop at KDD05 and KDD06

IBM Research


Cost-sensitive Learning for Marketing Decision Support

Use of Basic Machine Learning (e.g. Classification and Regression) in Marketing Decision Support is well accepted

Example applications include: targeted marketing, credit rating, and others But are they the best we have to offer ?

Regression is an inherently harder problem than is required One does not necessarily need to predict business outcome, customer behavior,

etc, but is merely required to make business decisions Regression may fail to detect significant patterns, especially when data is noisy

Classification is an over simplification By mapping to classification, one loses information on the degree of

goodness/badness of a business decision in the past data Cost-sensitive classification provides the desired middle ground

It simplifies the problem almost to classification and thus allows discovery of significant patterns;

Yet retains and exploits the information on the degree of goodness of business decisions, in a way that is motivated by Utility theory

IBM Research


Cost-sensitive Learning a.k.a. Utility-based Classification

In regression: given (x,r) X x R, generated from a sampling distribution, find F: F(x) r E.g. r = profit obtained by targeting customer x

In classification: given (x,y) X x {0,1} , generated from a sampling distribution, find F: F(x) y E.g. y = 1 if customer x is good, 0 otherwise

In utility-based classification: given (stochastic) utility function U and (x,y) X x {0,1} generated from a sampling distribution, find F: E[U(x,y,F(x))] is maximized (or equivalently E[C(x,y,F(x))] is minimized) E.g. U(x,1,1) = Profit(x) = Profit obtained by targeting customer x, when x is

indeed a good customer.

IBM Research


Example Cost and Utility Functions Simple formulation (cost/benefit matrix)

More realistic formulation (utility/cost dependent on individuals)

101

010

10PredictedTrue

Classification utility matrix

Credit rating utility

Interest0good

- Default Amt0bad

goodbadPredictedTrue

Targeted marketing utility

Profit C 0good

- C 0bad

goodbadPredictedTrue

011

100

10PredictedTrue

Misclassification cost matrix

IBM Research


Bayesian Approach with Regression

For each example x, choose the class that minimizes the expected cost:

Problem: Requires conditional density estimation and regression to solve a classification problem. Price is high computational and sample complexity

Merit: more flexibility and general applicability Business constraints Variability in fixed costs But, is it necessary ?

=ji

jixCxjPxi ),,()|(minarg)(*

need be estimated!

IBM Research


A Classification approach: Cost-sensitive boosting algorithm [AZL 2004]

=

T

tii yxh

1),(

),()],([ 0,y x, yxCyxCEw HyS =

||/1),(0 YyxH =

}'),(|))0(),,{((' yx, SyxwIyxT = >

S' y)(x,

S' y)(x,

),()],([ 1,y x, yxCyxCEw tHyS =

GBSE (Learner A, Expanded data S, count T)(1) For all initialize

(2) For all initialize weight

(3) For t=1 to T do(a) For all (x,y) in S update weight

(b) Let

(c) Let ht = A(T,|w|)(d) ft = Stochastic(hi )(e) Ft = (1- )Ft-1+ft

(4) Output h(x) = arg max( )

Weight updated in each iteration

the difference between average cost by the current ensemble and cost of y

Y}y' S,y)(x,y |)y'{(x,S' y)(x, =Define the expanded sample S as:

IBM Research


Gradient Boosting with Stochastic Ensembles: Illustration

C(x,y)

At learning iteration t At learning iteration t+1+

Cost C(x,y)

PredictedLabel, y

+ - - + -

Training Labels

The difference between the current average cost and the cost associated with a particular label is the boosting weight

The sign of the weight, E[C(x,y)] C(x,y), is the training label

Ave CostE[C(x,y)]

y

IBM Research


Cost-sensitive boosting outperforms existing methods of cost-sensitive learning as well as classification and regression

Existing methods

9361046108619010Satellite

584503614645Splice

85213029211513Letter

2149942831942KDD-99

48105317390237385403397Solar

34420742127121059174Annealing

GBSEMetaCostAvgCostBaggingData Set

Ave Test Set Cost (SE)

IBM Research


Active Learning a.k.a. Query Learning

Training Sample Add data to training sample

Learner

*The domain size is generally exponential

The goal is to achieve data and computational efficient learning by obtaining labeled data for points of the algorithms choosing

Existing approaches can be classified into two main categories Algorithmic approach (c.f. [Angluin]) Information-theoretic approach (c.f. [SOS 1992])

DomainSelect points necessary for learning

Active/Query Leaning

IBM Research


The Query by Committee Algorithm [STS 1991]

Maximize Uncertainty: Query a point with maximum spread

Query by Committee

Input Sample

Let Agents Predict onrandomly selected points

Agent Agent Agent Agent Agents: IdealizedRandomized Algorithms

This is a representative information theoretic active learning method Main idea is to query points at which the agent algorithms disagree the most (to

maximize information gain) Merit: data efficient learning is theoretically guaranteed, subject to assumptions

of representability of the target Weakness: the theory requires ideal (Gibbs) agent learner, and is generally not

computationally feasible

IBM Research


An Efficient Variant: Query by Bagging and Query by Boosting [AM 1998]

Queries point x* at which component algorithms disagree most

AgentLearner A

Agent Learner A

Agent Learner A

h hh

1 2T

Input Sample

Bagging: re-sampling with uniform distributionBoosting: weighted sampling with boosting weights

These methods combines a computational approach of ensemble method with information theoretic query by committee method

The method allows arbitrary deterministic agent algorithms Query by Bagging/Boosting

IBM Research


Active learning can accelerate learning

WDBC (UCI ML repository) and C4.5 Breast Cancer Wisconsin (UCI) and C4.5

It has been observed that active learning can drastically accelerate the rate of learning (e.g. 10 to 100 folds) over passive learning

Application to primary research (survey) in marketing analytics is promising but has not been exploited extensively

IBM Research


Sequential Cost-sensitive Decision Making by Reinforcement Learning Cost-sensitive classification provides an adequate framework for

single marketing decision making Real world marketing decision making is rarely made in isolation, but is

made sequentially Need to address the sequential dependency in decision making

Cost-sensitive classification Maximizes E[U(x,h(x)]

We now wish to Maximize t E[U(xt,h(xt)], where x may depend on earlier decisions

This is nothing but Reinforcement Learning, if we view x as the state Maximize t E[U(st,(st))], where st is determined stochastically

according to a transition probability determined by st-1 and (st-1).

IBM Research


Review: Markov Decision Process (MDP)

At any given time t, the agent is in some state s. It takes an action a, and makes a transition to the next state s,

dictated by transition probability T(s,a) It then receives a reward, or utility U(s,a), which also depends on

state s and action a. The goal of a reinforcement learner in MDP is to learn a policy,

namely : S A, mapping states to actions, so as to maximize the cumulative discounted reward:

),( R0t

ttt asU=

=

IBM Research


Modeling CRM process using Markov Decision Process (MDP) Customer is in some "state" (his/her attributes) at any point in time Retailer's action will move customer into another state Retailer's goal is to take sequence of actions to guide customer's path to maximize customer's

lifetime value

Reinforcement Learning produces optimized targeting rules of the form If customer is in state "s", then take marketing action "a" Customer state s represented by current customer attribute vector estimates LTV(s,a) -- best policy is to choose a to maximize LTV(s,a)

Typical CRM Process

BargainHunter

Repeater

LoyalCustomer

ValuableCustomer

One Timer

Repeater

Defector Defector

Repeater

LoyalCustomer

PotentiallyValuable

Campaign A

Campaign B

Campaign C

Campaign E

Campaign D

MDP and Reinforcement Learning provide an advanced framework for modeling customer lifetime value

p 64

IBM Research


Observed lifetime value reflects only customers lifetime value attained by current marketing policy, and therefore fails to capture their potential lifetime value

MDP based lifetime value modeling allows modeling of lifetime value based on optimized marketing policy (= the output of system !)

BargainHunter

Repeater

LoyalCustomer

ValuableCustomer

One Timer

Repeater

Defector Defector

Repeater

LoyalCustomer

PotentiallyValuable

Campaign A

Campaign B

Campaign C

Campaign E

Campaign D

Current marketing policyOptimized marketing policy

Estimated (potential) lifetime value will be based on the optimal path

Output policy will lead the customer through the same path

MDP enables genuine lifetime value modeling, in contrast to existing approaches that use observed lifetime value

Customer As path under

IBM Research


And here is how this is possible

The MDP enables the use of data for many customers in various stages (states) to determine potential lifetime value of a particular customer in a particular state

Reinforcement Learning can estimate the lifetime value (function) without explicitly estimating the MDP itself

The key lies in the value iteration procedure based on Bellmans equation

Repeater

LoyalCustomer

ValuableCustomer

Repeater Repeater

LoyalCustomer

PotentiallyValuable

Each rule is, in effect, trained with data corresponding to all subsequent states

LTV of a state = reward now + LTV of best next state

Rule a Rule b

Rule c

Rule d

)a',Q(s'maxa)]E[U(s, a)Q(s,'a+=

IBM Research


Reinforcement Learning Methods with Function Approximation Value Iteration (based on Bellman Equation)

Provides the basis for classic reinforcement learning methods like Q-learning

Batch Q-Learning (with Function Approximation) Solves value iteration as iterative regression problems

a)(s,Qmax arg (s))a',(s'Qmaxa)]E[U(s, a)(s,Q

]a)E[U(s, a)(s,Q

a

k'1k

0

+

=

+=

=

pi

a

))a',(s'Qmax),((a)(s,)Q-(1 a)(s,Qa)U(s,a)(s,Q

k'k1k

0

aasU ++

+ Estimate using function approximation (regression)

IBM Research


The graph below plots profits per campaign obtained in monthly campaigns over 2 years (in an empirical evaluation using benchmark data, i.e. KDD cup 98 data)

0

10000

20000

30000

40000

50000

60000

70000

80000

C ampaign number

Single

C C OM

Lifetime value modeling based on reinforcement learning can achieve greater long term profits than the traditional approach

to yield greater long term profits

Output policy of MDP approach (CCOM) invests in initial campaigns

Output policy of MDP approach (CCOM) invests in initial campaigns

IBM Research


Bayesian Network a.k.a Graphical Model

0.70.3

P(E)P(E)

0.30.7T

0.60.4F

P(C)P(C)E

0.40.6T T

0.80.2F T

0.10.9T F

0.70.3F F

P(R)P(R)M C

Bayesian Network is a directed acyclic graphical model and defines a probability model Here is a simple example

Economy

Marketing Competition

Revenue

P(M,E,C,R) = P(E) P(M|E) P(C|E) P(R|M,C)

0.10.9T

0.70.3F

P(M)P(M)E

IBM Research


Bayesian Network as a General Unifying Framework Bayesian Network provides a general framework that subsumes

numerous known classes of probabilistic models, e.g. Nave Bayes Classification Clustering (Mixture models) Auto regressive models Hidden Markov models, etc, etc Bayesian Network provides a framework for discussing modeling,

inference, causality, hidden variables, etc

Nave Bayes classification

Class

Variable 1 Variable N. Variable 1 Variable N.

Clustering/Mixture

Unobserved

Class

Hidden Markov Model

Symbol Symbol

State State

Unobserved

IBM Research


Bayesian Network and Causality Causality is not necessarily implied by the edge direction

Economy


Revenue

P(M,E,C,R) = P(E) P(M|E) P(C|E) P(R|M,C)P(M,E,C) = P(E) P(M|E) P(C|E)P(M,E,C) = P(M) P(E|M) P(C|E)

Economy

Marketing CompetitionAn Example Bayesian NetworkEconomy

Marketing CompetitionEconomy


P(M,E,C) = P(C) P(E|C) P(M|E)

This is actually ambiguous between

IBM Research


Causal Network and Causal Pattern Causal Network

Is a directed graph, in which the direction of edge means causality Causal Pattern

Is an equivalence class of causal networks

Economy


Revenue

Economy


Revenue

Causal Network Causal Pattern

This pattern shows that the causal relationship between E, M, and C are ambiguous

IBM Research


Edge Orientation in Bayesian/Causal Networks

[P. Spirtes, C. Glymour, and R. Scheines (2000)]

IBM Research


Inferring Structure of Bayesian/Causal Network from Data

Economy

Marketing Revenue


Revenue

P(M,E,R) = P(E) P(M|E) P(R|E)

P(M,E,C) = P(M) P(C) P(R|M,C)

M

M R | E

Economy

Marketing Revenue

Economy

Marketing Revenue

The causal structure cannot be determined from data !

P(M,E,R) = P(M) P(E|M) P(R|E) P(M,E,R) = P(R) P(E|R) P(M|E)

The causal structure can be determined from data !

It can be inferred that Marketing can bea lever for controlling Revenue !

IBM Research


Estimation and Inference with Bayesian Networks Inferring causal structure from data

Sometimes possible but in general not Bayesian network structure learning from data

It is known to be intractable for general classes It is even NP-complete to estimate polytrees robustly

Parameter estimation from data, given structure It is efficiently solvable for many model classes

Inference given model Exact inference is known to be NP-complete for sub-class including undirected

cycles It is efficiently solvable for tree structures and many models used in practice

Latent variable estimation, given structure Local optimum estimation is often possible via EM-algorithms

Given these facts, determining network structure using domain knowledge and using it to do parameter estimation and inference is common practice example

IBM Research


Lifetime Value Modeling and Cross-Channel Optimized Marketing (CCOM)

Direct Mail

Kiosk

Web

Store

Call Center

$

$ $ $ $

Optimizes targeted marketing across multiple channels for lifetime value maximization.

Combines scalable data mining and reinforcement learning methodsto realize unique capability.

IBM Research


CCOM Pilot Project with Saks Fifth Avenue

Business Problem addressed: Optimizing direct mailing to maximize lifetime revenue at the store (and other channels)

Provided solution for the Cross-Channel Challenge: No explicit linking between marketing actions in one channel and revenue in another

CCOM mailing policy shown to achieve 7-8% increase in expected revenue in the store (in laboratory experiments) !

Direct Mail

Store

$ $ $ $

$

CCOM-pilot business problem

reminder

IBM Research


Some Example FeaturesDemographic Features action rewardFULL_LINE_STORE_OF_RES.: If a full-line store exists in the area 0.018 0.004 NON_FL_STORE_OF_RES.: If a non full-line store exists in area 0.012 -0.004

Transaction Features (concerning divisions relevant to current campaign)CUR_DIV_PURCHASE_AMT_1M: Pur amt in last month in curr div 0.065 0.090CUR_DIV_PURCHASE_AMT_2_3M: Pur amt in 2-3 month in curr div 0.099 0.080CUR_DIV_PURCHASE_AMT_4_6M: Pur amt in 4-6 month in curr div 0.133 0.091CUR_DIV_PURCHASE_AMT_1Y: Pur amt in last year in curr div 0.162 0.128

CUR_DIV_PURCHASE_AMT_TOT: Total Pur amt in current division 0.153 0.147

Promotion History Features (on divisions relevant to current campaign)CUR_DIV_N_CATS_1M: Num cat sent last month in curr div 0.294 0.028CUR_DIV_N_CATS_2_3M: Num cat sent 2-3 months ago in curr div 0.260 0.025CUR_DIV_N_CATS_4_6M: Num cat sent 4-6 months ago in curr div 0.158 0.062CUR_DIV_N_CATS_TOT: Total num cat sent in curr div to date 0.254 0.062

Control VariableACTION: To mail or not to mail 1.000 0.008Target (Response) VariableREWARD: Expected cumulative profits 0.008 1.000

IBM Research


The Cross-Channel Challenge and Solution

The Challenge: No explicit linking between actions in one channel (mailing) andrewards in another (revenue)

Very low correlation observed between actions and responses Other factors determining life time value may dominate over the control variable

(marketing action) in estimation of expected value Obtained models can be independent of the action and give rise to useless rules !

The Cross-Channel Solution: Learn the relative advantage of competing actions! Standard Method

Proposed Method

Actions

Value in state s1 Value in state s2

Actionsa1 a2 a1 a2


Actionsa1 a2 a1 a2


Actions

a1 a2 a1 a2

Approximation

IBM Research


The Learning Method

Definition of Advantage A(s,a):= 1/t(Q(s,a) maxa Q(s,a))

Advantage Updating Procedure [Baird 94]

Modifications: 1. Initialization with empirical life time value 2. Batch Learning with optional function approximation

Repeat1. Learn

1.1. A(s,a):=(1-)A(s,a)+ (Amax(s)+(R(s,a)+tV(s)-V(s))/t)

1.2. Use Regression to estimate A(s,a) 1.3. V(s):=(1-)V(s)

+(V(s)+(Amax-new(s)-Amax-old(s))/)2. Normalize

A(s,a):=(1- )A(s,a)+(A(s,a)-Amax(s))

IBM Research


Evaluation Results

Significant policy advantage observed with small number of iterations

Obtained policy with 7- 8% policy advantage, i.e. 7- 8% increase in expected revenue (for 1.6 million customers considered)

Mailing policy was constrained to mail same number of catalogues in each campaign as last year

CCOM to evaluate sequence of models and output best model

Policy Advantage

-4

-202

4

68

10

1 2 3 4 5

Learning iterations

A

d

v

a

n

t

a

g

e

(

p

e

r

c

e

n

t

a

g

e

)

Policy Advantage

-4

-2

0

2

4

6

8

1 2 3 4 5

Learning iterations

A

d

v

a

n

t

a

g

e

(

p

e

r

c

e

n

t

)

Typical run (version 1)

Typical run (version 2)

IBM Research


Evaluation Method

Challenge in Evaluation: Need to evaluate new policy using data collected by existing (sampling) policy

Solution: Use bias-corrected estimation of policy advantage using data collected by sampling policy

Definition of policy advantage: (Discrete Time) Advantage

Policy Advantage

Estimating policy advantage with bias corrected sampling

A(s,a):= Q (s,a) maxa Q (s,a)

As~():= E [Ea~ [A(s,a)]]

As~():= E [((a|s)/ (a|s)) [A(s,a)]]

IBM Research


Combination of reinforcement learning (MDP) with predictive datamining enables automatic generation of trigger-based marketing targeting rules

Optimized with respect to the customers potential lifetime value

Stated in simple if then style, which supports flexibility and compatibility

Refined to make reference to detailed customer attributes and hence, well-suited to event and trigger-based marketing

This is made possible by Representing the states in MDP by customers

attribute vectors

Combining reinforcement learning with predictive data mining to estimate lifetime value as function of customer attributes and marketing actions

An example marketing targeting ruleoutput by CCOM system

IBM Research


Some examples of rules output by CCOM

Interpretation: If a customer has spent in the current division but enough catalogues have been sent, then dont mail

Avoid saturation effects

Differentiate between customers who may be near saturation and those who are not Interpretation: If a customer has spent in the current division and has received moderately many relevant catalogues, then mail

Invest in a customer until it knows it is not worth itInterpretation: If a customer has spent significantly in the past and yet has not spent much in the current division (product group) then dont mail

IBM Research


Marketing Event

Event IdentifierChannel IdentifierEvent DateEvent Category DescriptionFixed Cost

Customer

Customer IdentifierFirst NameLast NameAgeGender

Transaction

Customer IdentifierTransaction DateProduct Category IdentifierEvent IdentifierChannel IdentifierTransaction RevenueTransaction Profit

Customer Marketing Action

Event IdentifierCustomer IdentifierMarketing Action DateMarketing Action

Period

Period IdentifierPeriod Duration

Customer Profile History

Customer IdentifierProfile History DatePeriod IdentifierProduct Category IdentifierChannel IdentifierAggregated Count of EventAggregated RevenueAggegated Profit

Channel

Channel IdentifierChannel Description

Product Category

Product Category IdentifierProduct Category Description

Customer Loyalty Level History

Customer IdentifierLoyalty Level Start DateLoyalty Level End DateLoyalty Level

EventProduct Category

Event IdentifierProduct Category IdentifierWeight

CCOM Output Models

Marketing Policy Model

Model IdentifierModel TypeModel

Lifetime Value Model

Model IdentifierModel TypeModel

CCOM - Logical Data Model

Optional Entity

CCOM is generically applicable by mapping physical data to this model

*Developed with CBO

IBM Research

2006 IBM Corporation

Customer Wallet and OpportunityEstimation: Analytical Approachesand Applications

Saharon Rosset, Claudia Perlich, Rick LawrenceIBM T. J. Watson Research Center

IBM Research


Outline Wallet estimation: problems and solutions

The different wallet definitions How can we evaluate wallet models? Modeling approaches Empirical evaluation

MAP (Market Alignment Program) Description of application and goals The interview process and the feedback loop Evaluation of Wallet models performance in MAP

IBM Research


What is Wallet (AKA Opportunity)? Total amount of money a company can spend on a

certain category of products.

Company Revenue

IT Wallet

IBM Sales

IBM sales IT wallet Company revenue

IBM Research


Why Are We Interested in Wallet? Better evaluation of growth potential by

combining wallet estimates and past sales history Enables focus on high wallet, low share-of-wallet

customers

Intelligent marketing using wallet estimates for sub-categories e.g., software, hardware Evaluating success of sales personnel and

sales channel by share-of-wallet they attain Making resource assignment decisions

OnT

arg

et

MAP

IBM Research


Wallet Modeling Problem Given:

customer firmographics x (from D&B): industry, emloyeenumber, company type etc.

customer revenue r IBM relationship variables z: historical sales by product IBM sales s

Goal: model customer wallet w, then use it to predict present/future wallets

No direct training data on w or information about its distribution!

IBM Research


Historical Approaches Top down: this is the approach used by IBM

Market Intelligence in North America (called ITEM) Use econometric models to assign total opportunity to

segment (e.g., industry geography) Assign to companies in segment proportional to their size

(e.g., D&B employee counts) Bottom up: learn a model for individual companies

Get true wallet values through surveys or appropriate data repositories (exist e.g. for credit cards)

Many issues with both approaches (wont go into detail) We would like a predictive approach from raw data

IBM Research


Agenda Introduction and analytical issues

Different wallet definitions

How can we evaluate wallet models? The quantile regression loss function

Modeling approaches and results: Nearest neighbor approach Quantile regression Model decomposition approach

IBM Research


Multiple Wallet Definitions TOTAL: Total customer available budget in the

relevant area (e.g., total IT) Can we really hope to attain all of it? SERVED: Total customer spending on IT products

covered by IBM Better definition for our marketing purposes REALISTIC: IBM spending of the best similar

customers This can be concretely defined a high percentile of:

P(IBM revenue | customer attributes)

REALISTIC SERVED TOTAL

Total WalletServed Wallet

Realistic

IBM Research


Distribution of IBM sales to the customer given customer attributes: s|r,x,z ~ f,r,x,zE.g., the standard linear regression assumption:

What we are looking for is the (say) 90th percentile of this distribution

REALISTIC Wallet: Percentile of Conditional

),0(~, 2 Nzrxs +++=

E(s|r,x,z) Realistic

IBM Research


Agenda Introduction and analytical issues

Different wallet definitions

How can we evaluate wallet models? The quantile regression loss function

Modeling approaches and results: Nearest neighbor approach Quantile regression approach Model decomposition approach

IBM Research


Traditional Approaches to Model Evaluation Evaluate models based on surveys

Cost and reliability issues

Evaluate models based on high-level performance indicators: Do the wallet numbers sum up to numbers that make

sense at segment level (e.g., compared to macro-economic models)?

Does the distribution of differences between predicted Wallet and actual IBM Sales and/or Company Revenuemake sense? In particular, are the same % we expect bigger/smaller?

Problem: no observation-level evaluation

IBM Research


The Quantile Loss Function Our REALISTIC wallet definition calls for estimating the

pth quantile of P(s|data). Can we devise a loss function which is optimized in

expectation when we succeed?Answer: yes, the quantile loss function for quantile p.

This loss function is optimized in expectation when we correctly predict REALISTIC:

>

>=

yyyypyyyyp

yyLp

if )()1(

if )(),(

)|( of quantile p)|),((minarg th

xyPxyyLE py =

IBM Research


-3 -2 -1 0 1 2 3

0

1

2

3

4

Some Quantile Loss Functionsp=0.8p=0.5 (absolute loss)

Residual (observed-predicted)

IBM Research


Which Wallet Definitions to Model? We are generally interested in modeling

REALISTIC and SERVED wallets TOTAL wallets are not of real marketing interest For REALISTIC (or opportunity) we have multiple

modeling approaches Quantile k-nearest neighbors Quantile regression approaches:

Linear quantile regression Tree-based regression Kernel quantile regression, quanting,

For SERVED we have developed a graphical modeling approach will not discuss here

IBM Research


Modeling REALISTIC Wallets REALISTIC defines wallet as 90th percentile of

conditional of spending given customer attributes Implies some 10% of the customers are spending full

wallet with IBM

Two obvious ways to get at the 90th percentile: Estimate the conditional by integrating over a

neighborhood of specific customers Take 90th percentile of spending in neighborhood

Create a global model for 90th percentile Build regression models using quantile loss function

IBM Research


K-Nearest Neighbors Distance metric:

Industry match Euclidean distance on firmographics

and past IBM sales Normalization

Neighborhood sizes (k): Neighborhood size has significant

effect on prediction quality Prediction:

Quantile of firms in the neighborhoodI

n

d

u

s

t

r

y

Employees Rev

enue

Universe of IBM customers with D&B information

Neighborhood of target company

Target company i

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

IBM Research


Quantile Regression Traditional Regression:

Estimation of conditional expected value by minimizing sum of squares

Quantile Regression: Minimize Quantile loss:

Implementation: assume linear function , solution using linear

programming Linear quantile regression package in R (Koenker, 2001)

=

n

iiip xfyL

1)),((min

quantile regression

loss function

=

n

iii xfy

1

2)),((min

>

>=

yyyypyyyyp

yyLp

if )()1(

if )(),(

+= xy

IBM Research


Quantile Regression Tree Motivation:

Identify a locally optimal definition of neighborhood Inherently nonlinear Adjustments of M5/CART for Quantile prediction:

Predict the percentile rather than the mean of the leaf Splitting/pruning criteria does not require adjustment

Industry = Banking

Sales10K

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

no

yes

no

no

yes

yes

Industry = Banking

Sales10K

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

F

r

e

q

u

e

n

c

y

IBM Sales

Wallet Estimate

no

yes

no

no

yes

yes

IBM Research


Empirical Evaluation: Quantile Loss

Setup 4 Domains with monetary dependent variable including

direct mailing, housing prices, income data, IBM sales Performance on test set in terms of quantile loss Approaches: kNN, Linear quantile regression, quantile tree,

Quanting Baselines

Constant model Traditional regression models for expected values (for

skewed distributions, the expected value is actually a high quantile)

IBM Research


Performance on Quantile Loss

Conclusions If there is a time-lagged variable, linear quantile model is

best Quanting (using decision trees) and quantile tree perform

comparably Generalized kNN is not competitive

IBM Research


Residuals for Quantile Regression

Total positive holdout residuals: 90.05% (18009/20000)

IBM Research


Market Alignment Project (MAP): Background MAP - Objective:

Optimize the allocation of sales force Focus on customers with growth potential Set evaluation baselines for sales personal

MAP Components: Web-interface with customer information Analytical component: wallet estimates Workshops with Sales personal to review and correct the

wallet predictions Shift of resources towards customers with lower wallet

share

IBM Research


The MAP tool captures expert feedback from the Client Facing teams

Transaction Data

D&BData

Wallet models: Predicted

Opportunity

ResourceAssignments

Expert validated

Opportunity

Analytics and Validation

Data Integration

Insight Delivery and Capture

Post-processing

MAP Interview Team Client Facing Unit (CFU) Team

Web Interface

MAP interview process all Integrated and Aligned Coverages

The objective here is to use expert feedback (i.e. validated revenue opportunity) from from last years workshops to evaluate our latest opportunity models

IBM Research


MAP workshops overview Calculated 2005 opportunity using naive k-NN

approach 2005 MAP workshops

Displayed opportunity by brand Expert can accept or alter the opportunity Select 3 brands for evaluation: DB2, Rational, Tivoli Build ~100 models for each brand using different

approaches Compare expert opportunity to model prediction

Error measures: absolute, squared Scale: original, log, root

IBM Research


Displayed Model Predictions of kNN Distance metric

Identical Industry Euclidean distance on size (Revenue

or employees) Neighborhood sizes 20 Prediction

Median of the non-zero neighbors (Alternatives Max, Percentile) Post-Processing

Floor prediction by max of last 3 years revenue

I

n

d

u

s

t

r

y

Employees Rev

enue

Universe of IBM customers with D&B information

Neighborhood of target company

Target company i

IBM Research


0

2

4

6

8

10

12

14

16

18

20

0 2 4 6 8 10 12 14 16 18 20

E

x

p

e

r

t

F

e

e

d

b

a

c

k

MODEL_OPPTY

Expert Feedback (Log Scale) to Original Model (DB2)

Experts reduced opportunity to 0(15%)

Experts acceptopportunity (45%)

Experts changeopportunity (40%)

Increase (17%)

Decrease (23%)

IBM Research


Observations Many accounts are set for external reasons to zero

Exclude from evaluation since no model can predict this

Exponential distribution of opportunities Residual-based evaluation on the original scale suffers

from huge outliers

Experts seem to make percentage adjustments Consider log scale evaluation in addition to original scale

and root as intermediate Suspect strong anchoring bias, 45% of opportunities

were not touched

IBM Research


Evaluation Measures Different scales to avoid outlier artifacts

Original: e = model - expert Root: e = root(model) - root(expert) Log: e = log(model) - log(expert) Statistics on the distribution of the errors

Mean of e2

Mean of |e| Total of 6 criteria

IBM Research


Model comparison results: Count how often a model scores within the top 10 and 20 for each of the 6 measures

TivoliDB2RationalModel

2362204

1033616

4564435

1041316

40Quantile Tree 0.840Decomposition Center62kNN 50 + flooring21Regression Tree55Linear Quantile 0.841Max 03-05 Revenue66Displayed Model (kNN) (Anchoring)

(Best)

IBM Research


Conclusions kNN performs very well after flooring but is typically

low prior to flooring Empirically linear 80th quantile performs consistently

well (flooring has a minor effect) Experts are strongly influenced by displayed

opportunity (and displayed revenue of previous years) Models without last years revenue dont perform

well

Use Linear Quantile Regression with q=0.8 in MAP 06

IBM Research


Ongoing and Future Work Extend MAP to other geographies Quantile estimation performance of different

methods as a function of the quantile Performance as a function of the shape of the

conditional distribution of the dependent variables Theoretical generalization of the decomposition

approach

IBM Research


A graphical model approach

Wallet is unobserved, all other variables are Two families of variables --- firmographics and IBM relationship are

conditionally independent given wallet We develop inference procedures and demonstrate them

In some cases leads to simple linear regression as ML inference on wallet

See poster in this conference: Merugu, Rosset, Perlich: A new multi-view learning approach with an application to customer wallet estimation.

Company firmographics

IT spendwith IBM

Historical relationship

with IBM

CompanyIT

Wallet

back

IBM Research


References Marketing Science

R. Rust, K. Lemon and V. Zeithaml, Return on Marketing: Using Customer Equity to Focus Marketing Strategy, J. of Marketing, 2004.

P. Kotler, Marketing Management. Millennium Ed., Prentice-Hall, 2000. Cost-sensitive Learning

P. Domingo, Meta-Cost: A general method for making classifiers cost-sensitive, The 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1999.

N. Abe, B. Zadrozny and J. Langford, An Iterative Method for Multi-class Cost-sensitive Learning, The Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2004.

Active Learning H.S. Seung, M. Opper and H. Sompolinsky. Query by committee. Proceedings of

the Fifth Workshop on Computaional Learning Theory, 1992. D. Angluin. Queries and concept learning. Machine Learning, 1988.

MDP and Reinforcement Learning R. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press,

Cambridge, MA, 1998. L. P. Kaelbling, M. L. Littman, A. W. Moore, Reinforcement Learning: A Survey,

Journal of Artificial Intelligence Research, 1996.

IBM Research


References Bayesian Networks and Causal Networks

K. Murphy, A brief introduction to Bayesian Networks and Graphical Models, http://www.cs.berkeley.edu/~murphyk/Bayes/bayes.html

D. Heckerman, A tutorial on learning with Bayesian Networks, Microsoft Research MSR-TR-95-06, March 1995.

J. Pearl, Causality: Models, Reasoning, and Inference, Cambridge University Press, 2000.

P. Spirtes, C. Glymour and R. Scheines, Causation, Prediction, and Search, 2nd Edition (MIT Press), 2000.

Case Study: Customer Wallet Estimation S. Rosset, C. Perlich, B. Zadrozny, S. Merugu, S. Weiss and R.

Lawrence, Customer Wallet Estimation. 1st NYU workshop on CRM and Data Mining, 2005.

S. Merugu, S. Rosset and C. Perlich, A New Multi-View Regression Method with an Application to Customer Wallet Estimation. The Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2006.

IBM Research


Thank [email protected]

data analytics for marketing decision support: introduction and a wallet estimation case study

Documents