use data analytics for best return on marketing spend€¦ · the best return on your marketing...
TRANSCRIPT
C O N F I D E N T I A L
Use data analytics
for best return
on marketing spend
[email protected] @Jon_Isernhagen
EyeForTravel Smart Travel Analytics
2/12/2014
C O N F I D E N T I A L
Use Data Analytics To Get The Best Return On Your Marketing Spend
1) Implement a data-driven SEO strategy for higher ad position,
better conversion rates, higher ROIs, and lower cost per click
2) Use attribution wisely for accurate media spending in 2014
3) Only buy the media that work: track visitor behavior and
every dollar spent
4) Measure and track behavior to improve your mobile website
or application and use it to engage with customers
5) Time your marketing mix for maximum return using attribution
and reservation data.
C O N F I D E N T I A L
Agenda
1) Goal of channel spend
2) Consequences of inaccurate attribution
3) Approaches/levels of attribution a) Algorithmic attribution
b) Arbitrary attribution
i. Bracketing
ii. Tactical spend decisions by channel
C O N F I D E N T I A L
Positive Return on Incremental Channel Investment
- 4 -
ROI = (VCM – Spend)
Spend
Abbreviation Term Definition
ROI Return on Investment Indicator of investment profitability.
Positive = good.
VCM Variable Contribution Margin The amount of profit driven by a
given transaction.
Spend Channel Spend The amount spent driving traffic to
the site during the period in question
Calculated over a specified time period of investment and return.
C O N F I D E N T I A L
Which marketing treatment(s) triggered the purchase?
"Half the money I spend
on advertising is wasted;
the trouble is,
I don't know which half.“
-John Wanamaker
Father of Modern Advertising
C O N F I D E N T I A L
“Hard to Know” ≠ “Random”
We may not find the correct answer, but the answer does exist.
C O N F I D E N T I A L
Agenda
1) Goal of channel spend
2) Consequences of inaccurate attribution
3) Approaches/levels of attribution a) Algorithmic attribution
b) Arbitrary attribution
i. Bracketing
ii. Tactical spend decisions by channel
C O N F I D E N T I A L
Google Analytics attribution options
Google offers several attribution
mechanisms out of the box
Choosing the wrong model
can have consequences.
C O N F I D E N T I A L
How badly do you want to know?
Raylan: You'll pay to find that out. Boyd: What are you packing?
C O N F I D E N T I A L
Choosing your attribution strategy
Do you want/need
true attribution*?
Do you have
data guys?
Do they have
bandwidth for this?
Yes
No
No
Use your site
metric solution’s
attribution
Yes
Yes Hire a vendor
Can you access
your data?
Yes
No
*Do you have:
1) Large enough budget?
2) Multiple channels?
3) Belief in ROI “knowability?”
DIY
No
C O N F I D E N T I A L
Agenda
1) Goal of channel spend
2) Consequences of inaccurate attribution
3) Approaches/levels of attribution a) Algorithmic attribution
b) Arbitrary attribution
i. Bracketing
ii. Tactical spend decisions by channel
(skip)
C O N F I D E N T I A L
• Sends
• Opens
Collecting the necessary data
Analysis
space
• Clicks
• Visitors
• Bookings
• VCM
• Media impressions
Site metrics tool
Back office system
• Channel-specific
phone #s
Email distribution tool
Call Center IVR
• Impressions
• Spend
Ad server
e.g. SAS, Revolution Analytics,
SPSS, Teradata Warehouse Miner
SEM Bid Management tool
• Impressions
• Clicks
• Spend
• GRPs
• Spend
Television plan
• Impressions
• Clicks
• Spend
Meta search feeds
• Spend
Manual spend entry table
C O N F I D E N T I A L
Connecting the necessary data
Transactions Profiles Customers
Sessions
∞ 1 ∞ 1 Clicks:Transactions Clicks
∞ 1 1 ∞
∞ 1 ∞ 1
Calls
Sends
Opens ∞
1
∞
1
IVR
GRPs
TV plan
Impressions
Ad server
Impressions
∞ 1
∞ 1
∞ 1
Back office Site monitoring tool CRM system
1 ∞
C O N F I D E N T I A L
Forrester 2012Q2 survey of attribution vendors
Our Stats PhD audited the players, settled on pilot with new upstart Datasong (formerly “Upstream.”)
C O N F I D E N T I A L
“Given all the history we know, how likely is this shopper to convert soon?”
Datasong answers with a 2-stage model:
-Are Brand shoppers (BSEM, BSEO, DTI) likely to book? Yes
-Did your last TV spot cause their brand loyalty? Maybe
Algorithmic attribution: Survival Modeling
Model 2 Accuracy 81%
Model 1 Accuracy 68%
0
0.5
1
0 0.5 1
NonConverters Converters
0
0.5
1
0 0.5 1
NonConverters Converters
C O N F I D E N T I A L
Algorithmic attribution: Survival Modeling
1) The Axes show the time since last channel exposure
2) Dots represent converters vs. nonConverters
3) The Orange line represents the relationship between
2 variables, e.g.: a) time since last email and
b) time since last affiliate visit
4) Responders: everyone above the line. We count the
folks above the line and see what our accuracy is
5) Attribution: once satisfied with a model(s), we’d: a) take a given order,
b) see the time since last email and affiliate and based on
the timing, and;
c) where we are on the orange line, so;
d) we have a means to allocate which was more causal.
0
0.5
1
0 0.2 0.4 0.6 0.8 1
NonConverters Converters
C O N F I D E N T I A L
Survival Modeling: DIY
Recommended by one of our
PhD statisticians.
“SAS usage is not necessary.”
C O N F I D E N T I A L - 18 -
Algorithmic attribution: Interaction Method
Brand
SEM E-mail Affiliate
5.6%
Conversion
Brand
SEM Affiliate
5.8%
Conversion
Some channel interactions may actually take conversion backwards.
Only by examining all interactions can you know for sure.
C O N F I D E N T I A L
Running the process
1) For each site visitor a) Assemble visit history
b) Create variables to represent: i. Channel impressions
ii. Channel clicks
iii.Past purchases
2) Regress or use machine learning algorithm a) Ascertain which channel touches predict booking
b) Give VCM credit to causal channels
3) Calculate ROI a) Use each channel’s VCM and spend
b) Where ROI is positive, spend up
c) Where ROI is negative, cut spend or change tactics
C O N F I D E N T I A L
Agenda
1) Goal of channel spend
2) Consequences of inaccurate attribution
3) Approaches/levels of attribution a) Algorithmic attribution
b) Arbitrary attribution
i. Bracketing
ii. Tactical spend decisions by channel
(skip to end)
C O N F I D E N T I A L
“Round up all the clicks!”
• Offline bookings
• Multiple profile activity
• Impressions (Display or Television/Radio/etc)
• Cleared-cookie clicks
- 21 -
Leaves out:
C O N F I D E N T I A L
55% of shoppers clear cookies every 3 months
Cookie clearing is a serious problem causing the true impact of online
channels to be massively understated.
“The Consumer Side of Analytics” - Forrester/Jupiter Research 2009
C O N F I D E N T I A L
Bracketing
1) Assume for the sake of argument that one of the site tool’s attribution
schemes reflects reality, but you don’t know which one.
2) Find (or create) the scheme that is most and least generous to digital
channels. a) Last click: usually the least generous to digital channels
b) 3-tier: a scheme we created to favor NBSEM and Meta/Travelsearch
3) Assume that the true attribution (& ROI) lies between these extremes
4) Adjust spend and see which scheme better predicts reality.
C O N F I D E N T I A L
Tactical decision-making
Ignore the big picture and focus on the borderline projects in each channel
• Brand advertising
• SEM (Brand and Nonbrand)
C O N F I D E N T I A L
Tactical decisions: Brand
Easy decision: advertising heavily in geographies of known high
customer concentration during periods before high demand.
Hard decisions:
• national vs. regional
• whether uniform or in intermittent bursts
• invest more in air time or fresh creative
Tactical solutions:
1) A/B test across “identical” municipalities/regions, observe rise in
Brand-attributable (BSEM, BSEO, DTI) bookings.
2) Hire Comscore to measure brand awareness/intent after broadcast.
C O N F I D E N T I A L
Tactical decisions: Search Engine Marketing
Easy decision: spending to capacity on Brand SEM, which looks
profitable under almost all attribution schemes
Hard decision: whether to spend up on NonBrand SEM keywords which
don’t appear profitable no matter how you attribute.
Tactical solution:
• A/B test whether reduced BSEM spend is offset by increased BSEO
booking volume
C O N F I D E N T I A L
Tactical decisions: Email distribution
Easy decision:
• emailing up to 1x/week, hitting each site visitor with the message he or
she is most likely to convert from.
• sending messages advertising exceptional deals.
Hard decisions:
• whether to upgrade to rules-based campaign (e.g. Responsys, ET)
• whether to saturation-bomb your e-mail list, raising overall yield but
boosting unsubscription.
• whether to push loss-leading promo code, boosting overall conversion
but training site shoppers to expect discounts
C O N F I D E N T I A L
Take-away lessons
- 28 -
1) Be intentional about your attribution decisions, and creative about how to test your assumptions.
2) For top competitors in each vertical, the question is no longer “should we do algorithmic attribution?” but rather “How can we operationalize our attribution model?”
3) If your data aren’t yet in shape to support attribution & other ad hoc analyses, get that project underway.
[email protected] @Jon_Isernhagen
C O N F I D E N T I A L
Attribution: arbitrary methods
# Channel Rationale
1 Last click ABC: Always Be Closing.
2 Last paid click Direct is a point of sale, not a marketing channel.
3 Average/hyperbolic Every channel is special.
4 First click First attracted the consumer
5 First relevant click Started the conversation that led to the purchase.
Which channel(s) gets the credit in the ROI calculation for any given booking?
- 30 -
T C C C C C
NYC hotel
1/1/13
DTI
NYC hotel
1/1/13
BSEM
NYC hotel
1/1/13
NYC hotel
1/1/13
Meta search
ATL-NYC flight
1/1/13
SEO
W. Carib. cruise
6/1/13
1 2
3
4 5
45-day
lookback
All of these make assumptions about causality but give no evidence of causality.
C O N F I D E N T I A L
DataSong’s Modeling Architecture
• ETL
• N marketing channels
• Behavioral variables
• Promotional data
• Overlay data
• Functions to read Hadoop output; xdf creation
• Exploratory data analysis
• GAM survival models
• Scoring for inference
• Scoring for prediction
• 5 billion scores per day
per customer
UPSTREAM DATA
FORMAT (UDF)
CUSTOM VARIABLES
(PMML)
C O N F I D E N T I A L
What analysis method was used?
Survival Response Model on Accounts and Cookies
Analyze effects of marketing on response over time
In terms of both what worked and what didn’t
Time-dependent:
Inputs: estimate the speed of marketing effects’ decay
Outcome: look at times where a bkg has or hasn’t yet been made
Create models specific to:
LOB: the impact of SEM varies from auto to packages
Customer: repeat purchaser motivations are more knowable
Controls for non-marketing effects:
Anniversary effects: many sales driven by season demand
Customer-driven activity: many customers driven by loyalty
C O N F I D E N T I A L
VCM
Transaction revenue minus variable non-marketing expenses:
• Revenue: –Supplier Commissions,
–GDS incentives,
–Overrides (lumpy: average/booking must be assumed)
–Media (not transaction-driven, but has to be modeled in somewhere)
–Attached bookings / Lifetime value: try to gauge full value without double-
counting
• Expenses: –Website hosting/capacity costs
–Data processing expenses
The all-inclusive profitability of the transactions driven
- 33 -
C O N F I D E N T I A L
Marketing ≈ Farming
1) Fertilizing/irrigating = brand advertising
2) Planting = presence on the digital shelf
- 34 -
Which factor or factors caused you to have a good
(or bad) crop yield?
C O N F I D E N T I A L
Attribution options - broad view
Provider Method Cost
Timing Comments 1x monthly
Adobe Attribution model Varies - 4-10 wks Consulting arm willing to address attribution questions
as one-off modeling projects.
Adometry Econometric
model Low High 12 wks
Forrester-acclaimed industry co-leader.
Highly display-focused.
Datasong
Attribution model
w/
Customer
scoring
Med
Med
6 wks
Comprehensive model which pulls in both online and
offline spend impacts.
Med Per-customer demand curve to predict the expected
return on every campaign
Analytics Interactions
model - - now
Has delivered digital-only channel interactions model
to beta clients.
Gauge LLC Simple
regression High - 10 wks
General consulting analysts, not on the efficient
frontier for this deliverable.
Kenshoo Unknown
algorithm - Med 4 wks
Primarily targeted towards SEM spend, but can be
expanded to comprise other digital channels
Visual IQ Econometric and
Attribution
models
High - 4-8 wks Econometric model. Offers cross-channel allocation
guidance, including offline spend.
Med-
High - 3-6 wks
Attribution model. Offers within-channel campaign-
level spend guidance. No offline.