share. learn. connect. test, measure & optimize ...€¦ · test, measure & optimize:...
TRANSCRIPT
share. learn. connect.
March 11-13, 2015
Test, Measure & Optimize: Evaluating the impact of your omnichannel strategy Rick Child Vice President Media Horizons
by nemoa
In One Word ! Attribution
by nemoa
Attribution Defined ! The amount of orders/revenue caused by your
marketing campaign.
Caused = or ≠ Measured
by nemoa
Marketing Attribution Origins ! Not a new concept ! Developed in 1950s by
Fritz Heider, a German Psychologist
! Basic testing for attribution is decades old
Fritz Heider
by nemoa
Ghost (aka Holdout/Placebo) Panel Test
! Measure the variance from the exposed group versus the quarantined group across all channels.
Quarantined
Exposed
by nemoa
Appropriate Media ! Any media where you can identify
who receives your advertising › Direct mail › Limited Inserts & Print › Email › SMS Text › Website › Display
! Any media where you can identify the geography (weaker than above test) › Limited Inserts & Print › Paid Search › Broadcast
by nemoa
Ghost Panel Design – Direct Mail ! Identification by name and address ! Split into two groups
› Exposed Group – exposed to regular creative stream › Quarantined Group – do not mail
! Matchback mailing list to all orders all media during the response curve
! Variance between exposed and quarantined groups is the number of people campaign caused to order
by nemoa
DM Test Calculation
! Measuring only direct mail orders would have undercounted the campaign by 406 orders.
Match mailing list to the orders across all media.
Results
by nemoa
Ghost Panel Results – Direct Mail
63% 37%
Direct Mail Attributable
Incremental: On mail file but signed-up via other sources.
! 37% of Direct Mail responses were found in channels other than Direct Mail.
50%
50% OFFLINE
ONLINE
! Of the incremental sign-ups, offline/online split was 50/50.
6% 7% 2%
11%
74%
5 different offline channels
23%
11% 66%
3 different online channels
! Incremental sign-ups were distributed across 8 different sources.
by nemoa
Direct Mail Trends ! Less than 50% directly measured in last test
by nemoa
Ghost Panel Design – Display ! Identification by cookies ! Can only measure orders via online ! Split into two groups
› Exposed Group – exposed to regular creative stream › Quarantined Group – PSA Ad only
! Variance between exposed and quarantined groups is the number of people campaign drove to order via online
by nemoa
Display Results ! Compare response rate results of exposed group
to the quarantined group.
by nemoa
Display: View-thru vs Click-thru ! Click-thru orders: Consumer was exposed
to the ad and clicked on it.
! View-thru orders: Consumer was exposed to the ad but did not click on it.
! Generally for every 1 click-thru order there are 3 to 10 view-thru orders. Don’t believe it? Test it.
by nemoa
Compound Ghost Panel Testing ! Can create tests across two media at one time.
! Impact of Campaign A on Campaign B.
! Most popular: Direct Mail with Display
by nemoa
Direct Mail with Display Support ! Response typically boosted 5% to 25%
by nemoa
Omnichannel - Funnel Dilemma ! Different media have different places in the funnel
Awareness
Affinity
Conviction
Intent
Purchase
Social
Display
Inserts
Affiliate
SEM
Broadcast
Direct Mail
SEO
Direct
by nemoa
Fractional Attribution
SALE
Samantha DIRECT MAIL
DRTV
25% 45% 10%
VISIT STORE
RADIO AD
20%
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
BANNER AD
SOCIAL AD
31% 23% 14% 18% 14%
SALE
Rachel BANNER AD
25%
DRTV
16%
PRINT AD
10%
SOCIAL AD
10%
BANNER AD
11%
PAID SEARCH AD
8%
DIRECT MAIL
20%
by nemoa
Digital Only – How to Measure?
! Web Analytics Platform
! Ad Serving Platform
! Attribution Platform
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Digital Only – Web Analytics
! Web Analytics Platform › Attribution: Likely will give 100% of credit to the last
interaction. › 100% Credit to Paid Search › Consequence: Over invest in bottom of funnel media. › Symptom: Revenue is declining, losing market share
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Digital Only – Ad Serving ! Ad Serving Platform –
› Many attribution model choices › Custom models available › All digital channels must be linked or run through the ad
server
Social Orders 1,000 + Search Orders
1,000 Display Orders
1,000 + ≠ Digital Orders
1,800
by nemoa
Ad Serving Models
! “Last click, Last Impression” model › Default Model, (aka “Floodlight model”) › Attribution: Will give 100% of credit to one interaction, either
the last click or the list impression. › 100% credit to Paid Search ad › Consequence: Over invest in bottom of funnel media. › Symptom: Revenue is declining, losing market share
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Ad Serving Models
! Last Interaction Model › Attribution: Will give 100% of credit to the last
interaction. › 100% credit to Paid Search ad › Consequence: Over invest in bottom of funnel media. › Symptom: Revenue is declining, losing market share
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Ad Serving Models
! First Interaction Model › Attribution: Will give 100% of credit to the first
interaction. › 100% credit to Banner ad › Consequence: Over invest in top of funnel media. › Uses: Frequently used for new product/brand launches.
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Ad Serving Models
! Linear Model › Attribution: Will give equal credit to the each interaction. › 40% credit Banner, 20% to Social and 40% Paid Search › Consequence: Gives too much weight to the middle. › Uses: Good first step after default.
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Ad Serving Models
! Time Decay Model › Attribution: Will weight credit towards the interactions
closest to the time of purchase. › Consequence: May give too much weight to the bottom
of the funnel. › Uses: Good for impulsive buys.
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Ad Serving Models
! Position Based Model › Attribution: Will weight 40% credit to the first interaction,
40% to the last and 20% to the middle. › 80% credit to banner, 10% to social and 10% to paid
search. › Consequence: Often the best of the attribution models.
SALE
Lilly PAID SEARCH AD
PAID SEARCH AD
BANNER AD
31% 23% 14%
BANNER AD
18%
SOCIAL AD
14%
by nemoa
Ad Serving Models ! The model picked affects not only the measured success of
the campaign, but also the success of different creatives.
by nemoa
Validate the Model ! Time Lag Analysis
! Path to Conversion Analysis
! Ghost Panel Testing
by nemoa
Attribution Platforms ! “Use advanced statistical approaches to allocate
proportional credit to marketing communications and media activity across all channels, which ultimately leads to the desired customer action.” The Forrester Wave: Cross-Channel Attribution Providers, Q4 2014
SALE
Rachel BANNER AD
25%
DRTV
16%
PRINT AD
10%
SOCIAL AD
10%
BANNER AD
11%
PAID SEARCH AD
8%
DIRECT MAIL
20%
by nemoa
Getting Noticed ! Google purchased Adometry in May 2014
! AOL purchased Convertro in May 2014
! At least 8 attribution platform products had greater than 25% growth
(The Forrester Wave: Cross-Channel Attribution Providers, Q4 2014)
› Google Adometry › Convertro › Visual IQ › MarketShare
› Marketing Evolution › eBay Enterprise › Rakuten DC Storm › Abakus
by nemoa
How does it work?
! Able to connect offline, digital, including mobile ! Use the latest modeling techniques to arrive at
models that determine the fractional attribution of each media for each consumer.
SALE
Rachel BANNER AD
25%
DRTV
16%
PRINT AD
10%
SOCIAL AD
10%
BANNER AD
11%
PAID SEARCH AD
8%
DIRECT MAIL
20%
SALE
Rachel BANNER AD
18%
DRTV
10%
PRINT AD
8%
SOCIAL AD
12%
BANNER AD
16%
PAID SEARCH AD
16%
DIRECT MAIL
18%
ACTUAL
ATTRIBUTION PLATFORM
by nemoa
How do you validate the outcome? ! Ghost Panel Testing
by nemoa
Conclusion ! Consider the consequences that your attribution
technique is having on your media decisions
! Execute ghost panel testing
! The future is in Attribution Platforms
by nemoa
Questions
Rick Child Vice President, Data & Analytics Media Horizons, Inc. Contact: (203)604-1769 [email protected]