eyeforpharma nov 2014 pres

How Do You Know…

Designing Multi-Channel

Campaign Tests That

Optimize Your Budget

Jim DeLash

November 18, 2014

One version had a Conversion Rate 24.6% higher

than the other one

Page layout

Moving registration to the right side form

increased the site’s Conversion Rate by 24.6%

Page layout

How do you know?

The right frequency

The best subject line

The optimal channel mix

Designing successful multi-

channel tests requires….

Establishing hypotheses

Developing test strategies by

segment

Creating a measurement plan

Establishing hypotheses

Statements to be proven

All tests must support a hypothesis

Extrapolating the results to other brands

Low unaided recall scores for

advertising creative will require

additional frequency….

Establishing hypotheses

Low unaided recall scores for

advertising creative will require

additional frequency….

Establishing hypotheses

Low unaided recall scores for

advertising creative will require

additional frequency….

BUT HOW MUCH MORE?

Establishing hypotheses

TEST…

More frequency in one channel

More frequency in multiple channels

Different levels of frequency

Establishing hypotheses

TEST…

More direct mail or e-mail

More frequency in multiple channels

Different levels of frequency

Establishing hypotheses

TEST…

More direct mail or e-mail

More direct mail and e-mail

Different levels of frequency

Establishing hypotheses

TEST…

More direct mail or e-mail

More direct mail and e-mail

10 15 20

Establishing hypotheses

TEST…

More frequency in one channel

More frequency in multiple channels

Different levels of frequency

Establishing hypotheses

The best design is not always the best approach

Do not let design conflate the objective of an email or message

Every unnecessary piece of content is waste and reduces your

chances of getting a click

Designing successful multi-

channel tests requires….

Establishing hypotheses

Developing test strategies by

segment

Creating a measurement plan

Developing Test Strategies by

Segment

Specialist or Generalist

Current User or Prospect

Decile

Awareness & Trial Offer

Developing Test Strategies by

Segment

Multiple

Brands Cadence

What will we learn from this test….

1. Best frequency by Specialty and Quintile

2. One channel vs. Two channels (Segments 5,7,9)

3. Total frequency: 10 vs. 12 vs. 14

Version B, the one with the ‘Save Now’ button, increased opt-ins

for logged-in users by 333% at a 99.9% confidence level.

Designing successful multi-

channel tests requires….

Establishing hypotheses

Developing test strategies by

segment

Creating a measurement plan

Control Groups

Measurement Milestones

Test Analysis

Creating a Measurement Plan

There are only two ways to get

fired from Harrah’s:

stealing from the company,

or

failing to include a proper

control group in your business

experiment

Control Groups…

Not a ‘holdout’ group

Must include all relevant variables

Large enough to be statistically valid

Creating a Measurement Plan

Measurement Milestones…

Track progress against goal

Forecast test impact early

Connect KPIs with Rx impact

Creating a Measurement Plan

Test Analysis…

Evaluate overall campaign impact

Both TRx and NRx

Creating a Measurement Plan

How can an e-mail test be

successful with a low open rate?

MICRO TESTS

MICRO TESTS

Version B, the subject line broken into three consumable

sections, increased opens by 73.7%.

Version B, the ‘Get My Free Paper’ Call to Action,

increased downloads by 25% at a 95% confidence level.

Version B, the ambiguous subject line, increased

opens by 112% at a 95% confidence rate.

Designing successful multi-

channel tests requires….

Establishing hypotheses

Developing test strategies by

segment

Creating a measurement plan

What is the optimal number of

samples to offer in a multi-channel

campaign?

So, how will you know?

This is the last slide

The Influence of Color

21.4% Conversion

Rate improvement

with the red button

The Influence of Color

Headline Copy

Headline Copy

32.3%

Conversion Rate

improvement

A Picture is Worth 1000 Words… But Which Picture? (Images)

A Picture is Worth 1000 Words… But Which Picture? (Images)

The female model in the header convinced 108%

more visitors to fill out and submit the lead gen

form