MOTOMARKETING…flexing DSS muscles to beat the competition
Design of Experiments Case StudyWCBF ConferenceChicago9.22.2004
CHIP LEONCorporate Director,Motorola Corporate Initiatives Group
2
2002 Business NPI SAY / DO PERFORMANCE-actual margin $ vs. business case margin $ commitment
=> 300%200%0%<= -100%Business G
Business F
Business E
Business D
Business C
Business B
NPI Sample=121
Business A
Margin Shortfall
-$B
Products <90%
%
Say/Do Target 90-120%
Net Margin Delta
-$B
Products 90-120%
%
Margin Overage
$M
Products>120%
%
Lift the ‘DO’!
3
WHAT IS OUR TYPICAL RESPONSE?
Discounts /rebates
Learn & Earn
Spiffs / incentives
advertising
Pricemoves
Rep Visits
Product /Featurespecific
PromotionsEvents
Low ‘DO’!Low ‘DO’! Affinities
PR
..and I’ll even
throw in the
kitchen sink!Which ones do we do?!
Which generate the highest return?!
4
Define:What is important?
Measure:How are we doing?
Analyze:What is wrong?
Improve:What needs to be done?
Control:How do we guarantee
performance?
• Maximize margin / lift through effective sales & marketing tactics
• Established key effectiveness measure: ROI• Standardized effectiveness reporting process• Set targets: 125-250% ROI
• Running series of Design of Experiments (DOE’s) to determine what really causes lift
• Do more of what works, kill what doesn’t
• Build a Digital Scenario Optimization Tool
WHAT TO DO TO LIFT THE ‘DO’
5
WHAT ARE DESIGN OF EXPERIMENTS?Not just correlation, but CAUSE
=
y = f (x1, x2, x3, x12, x23, x123,)
How does each tactic affect the output?
What are the effects of interactions?
Which aresignificant?
Vol / Margin lift Discounts /
rebates
Learn & Earn
Spiffs / incentives
advertising Pricemoves
Rep VisitsProduct /Featurespecific Promotions
Events
6
Location 1
WHAT ARE DESIGN OF EXPERIMENTS?Not just correlation, but Cause
advertising
Discounts /rebates
Location 2
advertising
Discounts /rebates
Location 3
advertising
Discounts /rebates
Location 4
advertising
Discounts /rebates
7
Location Interactive POS
Seeding(giving sales associate a
phone)
Sell & Earn
(Incentive variation)
1 – Joliet Yes Yes No
2 – North Ave No No Yes
3 – St. Charles No Yes No
4 – Harwood Yes Yes Yes
5 - Algonquin Yes No No
6 – N. Riverside No No No
7 - Schaumburg Yes No Yes
8 - Tinley No Yes Yes
Real Marketing DOE example: Customer A, NA
Which Tactics cause, and by what contribution, the greatest effect?
Point of Sale
Off On
Sale
s Li
ft
Sell & Earn
Off On
Sale
s Li
ft
Seeding
Off On
Sale
s Li
ftSlope of lin
e =
Contributio
n to Effe
ct
Great
er C
ontri
butio
n
Little Contribution
9
How do the interactions of differenttactics cause effects on sales?
Off On
Sale
s Li
ft
Addition of Point of Sale
Is the additional cost of point of sale worth the incremental lift
& margin return?
= more lift, however
less cause of the lift
Sell to
Earn
10
1.00.50.0-0.5-1.0
1.0
0.5
0.0
-0.5
-1.0
Choose level of Point of Sale
Cho
ose
leve
l of S
ell &
Ear
n
Precursor to Scenario Planning tool
Optimize
d Combination fo
r Greatest R
eturn!Sales = 19.8 + 6.00 POS - 1.50 Seed + 4.00 S to E
11
When making a price move at retail, we definitely must advertise the price drop• Price drop alone significant; product advertising moderately so• Interaction of price drop and advertising generates significant cause (more
than each on their own)• can we offer $20 perceived value other than actual $20 drop?
Point of Sale and Sell to Earn independently are significant• Yet less significant when combined, and less efficient (ROI)• Diminishing return when throwing more tactics to create lift?
Seeding is not a significant cause to sales lift (Yet!)• And what causation does exist, is sometimes negative• Must retest: contradictory to dealer councils, yet reflective of focus groups
and field failures• Re-test with exciting new phones; do not seed with old phones
EARLY LEARNINGS FROM FIRST PILOT
12
Test ran in 16 Customer P stores in the U.K.
Purpose is to test the causation to sales lift at retail between:
1. New experimental interactive POS V600
2. Product seeding3. Bespoke backing card
Exit interviews and mystery shoppers are used to gauge brand metrics at the store level during the test.
The test will be replicated twice to get more precision on the effects.
Real Marketing DOE example:
Customer P, EMEA
13
SEEDING Live
product provided by Motorola with dummy SIM to store controller
LIVE DEMO UNIT FSU and Live product provided
by Motorola with dummy SIM
BESPOKE BACKING CARD Motorola themed product card,
designed to stand out from the remainder of the range
14
Main Effect Analysis Indicates Seeding Best At Causing Sales Uplift …
• Main effects only analysis indicates that seeding is causing highest V600 delta sales uplift;
• Interactive POS is scoring 2nd best effect;
• Backing card has a negative impact on sales performance.
3.5
8.25 7.375
4.375
1.75
10
OFF ON OFF ON
OFF ON
15
• Largest effect (main and interaction) remains with seeding (C) bar extending the farthest. It means that, out of the 3 tactics and interaction of these tactics, seeding is the tactic causing the higher delta sales lift.
• In addition, the effect for the seeding with interactive POS (AC) is somewhat noticeable. Same for interactive POS with backing card (AB) and for interactive POS (A).
• Backing card (B) alone, the 3 tactics together (A,B,C) or backing card with seeding BC) are not generating sales lift.
Term
Standardized Effect
BC
ABC
B
A
AB
AC
C
1,81,61,41,21,00,80,60,40,20,0
1,415Factor NameA POSB Backing CardC Seeding
Pareto Chart of the Standardized Effects(response is V600 Delta Sales, Alpha = ,20)
All Effect Analysis Confirm Seeding Significance And Some Key Interaction Effects, Particularly Seeding With Interaction POS…
16
Seeding With Interactive POS Drive A Delta Of 12-14 V600 Units By Store …
• You can see that the largest effect is seeding (C) because it extends the farthest. It means that, out the 3 tactics used in the DOE, seeding is the tactic causing the higher sales lift.
• In addition, the effect for the seeding with interactive POS (AC) is somewhat noticeable. Same for interactive POS with backing card (AB) and for interactive POS (A).
17
• With no controlled conditions implemented, delta sales of V600 versus previous month are 5 units per store.
• while implementing controlled the 3 conditions, delta sales of V600 versus previous month are 17.5 units, a three-fold increase over doing nothing!
Implementing The 3 Controlled Conditions Versus Doing Nothing Drives A 3.5 Time V600 Delta Sales Increase …
* 3.5
18
Digital Scenario Optimization Tool• Runs utilizing transfer functions from DOE’s, monte carlos, and
econometrics• multiple scenario outputs: levels of levers, volume lift, margin return• With accuracy
“The way we work”• Optimization: of campaigns, planning / budgeting, RETURN • Marketing & Finance: internal effectiveness• customer tool
HOW WE WILL DRIVE DECISIONS & OPTIMIZE FUTURE CAMPAIGNS…
19Source: Maximizing Marketing ROI; 2001 American Productivity & Quality Center; in conjunction with The Advertising Research Foundation
“ROI – BASED MARKETING PAYS OFF”Best Practice of Marketing Mix Modeling (DOE’s) &
Scenario Optimization tools utilized by…
20
Ultimate method that can be applied to any function to understand and optimize ROI…• Manufacturing
-What process changes optimize yield? Time vs. temperature vs. pressure vs…
• Finance- What strategies / factors influence positive market capitalization? Cash vs. EPS vs. Market share vs…
• HR- What attracts and retains leaders and high performers? Incentive plans vs. stock options vs. health benefits vs…
• Product design & feature selection- Which feature combinations create highest consumer demand? Samsung model
•Any time you want to know TRUE causation
LET DOE’s WORK FOR YOU -