test for success: a guide to a/b testing on emails

52
Test for Success A Guide to A/B Testing on Emails & Landing Pages

Upload: marketo

Post on 20-Aug-2015

4.998 views

Category:

Technology


2 download

TRANSCRIPT

Test for Success A Guide to A/B Testing on Emails

& Landing Pages

#abtesting

Our presenters

Jessica LangensandMarketing Programs ManagerMarketo

Allison SparrowMarketing Automation ManagerOptimizely

#abtesting

A few things first...

● Slides and recording emailed to you immediately after the webinar.

● Ask questions in the chat box & on Twitter! o Use #abtesting.

#abtesting

What you’ll learn

● Why should you A/B test?● Setting up your first (effective) A/B test● Testing Ideas and Success Stories

#abtesting

Why Should You Care About A/B

tests?

#abtesting

#abtesting

#abtesting

There are 4.1 Billion email accounts in existence.

Source: The Radicati Group

#abtesting

In 2013, SMBs voted that email marketing was #2 most important marketing channel.

#1, having a website.Source: eMarketer

#abtesting

2.3%{

#abtesting

A/B Testing is the process of showing different versions of a site to visitors, in order to see which one better achieves your goals.

#abtesting

An A/B test is...

#abtesting

#abtesting

One Page/Email

#abtesting

One Component

One Page/Email

#abtesting

One Page/Email

One Component

One Split Audience (50/50)

#abtesting

A B

#abtesting

A B

43.4% more pre-orders

#abtesting

Why should you care about A/B tests?

● Let the data do the work● Build internal alignment● Your website and email channels are invaluable

(make the most of it)● It’s easy, and cheap

#abtesting

What is an (effective) A/B test?

#abtesting

What is an (effective) A/B test?

● Identify your company’s goals● Choose where to test● Choose what to test● State your hypothesis● Track and report results

#abtesting

What are my company’s goals?

#abtesting

What are my company’s goals?

● Purchases/conversions● Revenue per visitor● Lead generation● Pageviews

#abtesting

Where to test?Welcome Page Product Page

#abtesting

Where to test?

● Welcome email● Home page● Product page● Contact Form page● 30 day trial expiration email● Download confirmation email● Cancellation confirmation email

#abtesting

What to test?

#abtesting

What to test?

● Headline● Subject Line● Value Proposition● CTA

#abtesting

Create a Hypothesis

“Visitors are getting distracted and feeling overwhelmed with too many competing elements on the page. Removing elements will reduce the clutter, focus user attention on article content and result in an increase in article pageviews.”

#abtesting

State a Hypothesis

● Identify a theory about the specific user experience

● Define the metrics for success● Define the quantifiable change in the key metric

#abtesting

Keep a Record!

#abtesting

How to Determine a Winner

#abtesting

Email

#abtesting

Statistical Significance

Source: ZettaSphere.com

#abtesting

Statistical Significance

Source: ZettaSphere.com

#abtesting

Landing Pages

#abtesting

Significant?

Source: SplitTestCalculator.com

#abtesting

Testing Ideas

#abtesting

Format

• Font• Text Size

• Color• Branding• Images :)

#abtesting

Subject LinesA

B

#abtesting

A

B+ 33% click through rate

Subject Lines

#abtesting

Subject Lines

● Vary a Replicable Element[New Report] 2014 Marketing Automation Vendor Comparisonvs.[Download] 2014 Marketing Automation Vendor Comparison

● Complete Subject Line Testo Test on a small group, wait 24 hours and send winner to

remainder of your list

#abtesting

“From” Line

“Scott Magri”

vs.

“Marketo Premium Content”

+ 4% opens +3% click-through

#abtesting

Button Placement

+6% CTR

#abtesting

Headers

#abtesting

Banners

#abtesting

Banners

10% CTR 5% CTR

#abtesting

Button Color

#abtesting

Button Color

+21% CTR

#abtesting

A/B testing isn’t about what you like. It’s about what your audience responds to.

#abtesting

#abtesting

Questions?Use Chat

#abtesting

Thank you!Recording is on the way!