the top 5 marketing discoveries of 2015: 12 months of research in just 60 minutes

The Top 5 Marketing Discoveries of 2015:12 months of research in just 60 minutes

#WebClinic

Today’s Session

• 5 illustrative experiments of each discovery• Cross-referenced recent examples• Summary key points

We’re sharing on Twitter!#WebClinic

#WebClinic

Today’s speakers

Jon PowellSenior Manager of Executive Research and Development,MECLABS Institute

Ben HuppertzSenior Manager, Research and StrategyMECLABS Institute

Discovery #1

Experiment: Background

Background: A company offering a variety of dieting programs and memberships with the goal of helping their audience lead a healthier lifestyle.

Goal: To increase landing page membership conversions on mobile and tablet devices.

Research Question: Which use of video will generate the highest conversion rate?

Test Design: Multi-treatment variable cluster split

Experiment ID: TP 11157Record Location: MECLABS Research LibraryResearch Partner: (Protected)

Mobile side-by-side

*Anonymized

Control Treatment A Treatment B Treatment C

*Video is the same for the Control and Treatments

Design Conversion Rate Level of Confidence % Relative Change

Control 0.14% ---

Treatment A 0.19% 99% 34.4%

Treatment B 0.17% 97% 22%

Treatment C 0.16% 89% 14.5%

Experiment: Mobile results

Relative increase in conversion rate34%Treatment A resulted in a 34.4% relative increase in conversion rate.

Two questions:1. What common thing did the three treatments do (in different

ways) to change the customer experience?

Two questions:

2. What did treatment A do that was different than B and C?

1. The impact of immediate authority in content marketingContent is not free. There may not be a cost financially, but there is always a cost psychologically.

Top 5 Marketing Discoveries in 2015

Example Source: https://blog.bufferapp.com/power-of-story

From This To This

296%More Full-page reads

520%Average time on page

https://blog.bufferapp.com/power-of-story

Discovery #2

#WebClinic

What would you test/change to increase engagement?

#WebClinic


Background: A tourism commission seeking to enhance visitor interaction with their website content in order to boost appeal for choosing their city over other destinations.

Goal: Increase visitor engagement with site content.

Primary Research Question: Which treatment will increase site engagement?

Test Design: A/B/C split test

Experiment ID: TP 11211Record Location: MECLABS Research LibraryResearch Partner: (Protected)

#WebClinic

#WebClinic

Experiment: Treatments

*Anonymized

Treatment A

Treatment B

• Primary navigation tabs were streamlined based upon visitor interaction data with site content over a calendar year.

• The most engaged-with category content and secondary content were prioritized and emphasized, while less popular content was removed in order to reduce friction.

#WebClinic

Design Clickthrough Rate Level of Confidence % Relative Change

Control 19.39% ---

Treatment A 26.26% 99% 35.39%

Treatment B 19.58% 55% 0.99%

Experiment: Results

Relative increase in navigation CTR35%Treatment A resulted in a 35.39% relative increase in navigation clickthrough rate.

#WebClinic

F

Design Lead Gen Rate Level of Confidence % Relative Change

Control 1.01% ---

Treatment A 1.36% 99% 34.93%

Treatment B 1.19% 91% 18.67%

Experiment: Results

Relative increase in total leads34%Treatment A resulted in a 34.93% relative increase in total leads.

What you need to understand:Navigation content was curated based upon visitor behavior with the control nav.


2. The impact of intentional navigation in initial engagementHaving relevant content is not enough. We must learn to not only optimize the essence (focus) of our content, but also it’s expression (form).


ExampleFrom This To This

548%Conversions

Discovery #3

Experiment #1: Background

Background: A regional marketing commission.

Goal: To raise awareness of local activities and events, increasing number of travelers and tourist spending in the area.

Research Question: Which subject line will generate the most opens and clickthrough?

Test Design: A/B split test

*Note: Test has been anonymized to protect partner confidentiality.

Experiment ID: TP 20211Record Location: MECLABS Research LibraryResearch Partner: (Protected)*

Experiment #1: Version A

SUBJECT: Are you a fall foodie? See what’s going on in Boston this November.

Experiment #1: Version B

SUBJECT: Clam Chowder, Lobster Rolls and More: Calling all foodies for Boston’s best fall fun

Experiment #1: Which subject line will win?

Version A

SUBJECT: Are you a fall foodie? See what’s going on in Boston this November

SUBJECT: Clam Chowder, Lobster Rolls & More: Calling All Foodies for Boston’s Best Fall Fun

Version B

Experiment #1: Results

Relative increase in opens10%

VersionOpen Rate

Relative Difference

Level ofConfidence

Version A 19.8% -- 99%

Version B 21.7% 10% 99%


Relative increase in clickthrough rate15%Open Rate

Relative Difference

Level ofConfidence

Version A 2.7% -- 99%

Version B 3.1% 15% 99%

Experiment #2: Background


Goal: To raise awareness of local activities and events, increasing number of travelers and tourist spending in the area.

Research Question: Which subject line will generate the most opens and clickthrough?



Experiment ID: TP 20219Record Location: MECLABS Research LibraryResearch Partner: (Protected)*

Experiment #2: Version A

SUBJECT: Holiday shopping, historic tours, Pops Orchestra concerts and more Boston traditions

Experiment #2: Version B

SUBJECT: Book now to save up to 30% on hotel stays this December in Boston

Experiment #2: Which subject line will win?

Version A


SUBJECT: Holiday Shopping, Historic Tours, Pops Orchestra Concerts and more Boston Traditions

Version B


Relative increase in opens9%Version

Open Rate

Relative Difference

Level ofConfidence

Version A 19.3% 9% 99%

Version B 17.6% -- 99%


Relative increase in clickthrough rate26%Version

ClickRate

Relative Difference

Level ofConfidence

Treatment 3.1% 26% 99%

Control 2.5% -- 99%

What do winning subject lines have in common?





Losing Versions Winning Versions

Why?



3. The impact of proper specificity in the email subject lineMany marketers worry about their ability to write persuasive copy, but the marketer’s art is not persuasion; it is clarity.


Experiment #1:

Loser



Winner

• Confusing• Vague

Experiment #1:

Loser

SUBJECT: Are you a fall foodie? See what’s going on inBoston this November


Winner 10% opens

SpecificSubject Line

• Tangible• Clear

• Confusing• Vague

Experiment #2:

Loser


• Salesy• Ambiguous

Experiment #2:

Loser



Winner 9% opens

SpecificSubject Line

• Specific• Valuable

• Salesy• Ambiguous

Discovery #4


Background: Think tank soliciting year-end donations to meet annual fundraising goal.

Goal: To increase donations.

Research Question: Which email will generate the most revenue?


Experiment ID: #616Record Location: NextAfter Digital Research LibraryResearch Partner: The Heritage Foundation

Experiment: Version A

• This email was sent by Jim DeMint, the president and most well-known leader of The Heritage Foundation, asking recipients for their support with a year-end gift

• The tone of the email is formal and professional

Experiment: Version B

• The email was sent by Christie Fogarty, the foundation’s lesser known Director of Membership Programs, requesting donations

• The email uses a much friendlier, informal tone

Version A

Experiment: Side-by-side comparison

Version B

?Audience Question:Which email will result in more donations?

DesignRevenue per

VisitorRelative

DifferenceStatistical

Significance

Version A – Jim DeMint $0.03 - -

Version B – New Sender $0.14 380.7%

% Relative Change: 380.7%

99%

Experiment: Results

Increase in Revenue381%The optimized version’s revenue increased by 380.7%.




4. The growing effectiveness of an “everyday tone” in email People still don’t buy from (or give to) companies, stores or websites; people buy from people.


An Everyday Tone

• The winning email hypothesized (and confirmed) that a lesser-known sender with a shorter, briefer message would imply a more believable, personal email send

An Everyday Tone

• The winning email uses language that more closely aligns to the personal nature of email

• In fact, it was designed to read like a conversation and is true to the style of the actual sender

An Everyday Tone

Version B

• The winning email is plain about the purpose of the ask and how it will be used

• In addition, the call-to-action is very clear and direct

Example

60%In Click-Rate

159%In Completion Rate

From This To This

Discovery #5


Experiment ID: TP11081Record Location: MECLABS Research LibraryResearch Partner: Protected*


Goal: To raise awareness of local activities and events, increasing number of travelers and tourist spending in [City].

Research Question: Which sign-up page will generate the most response?



The control

Control

The control

The treatment

Control Treatment

Side-by-side

Experiment: Results

Treatments Conversion RatePercent Relative

ChangeLevel of

Confidence

Control 56.4% -- --

Treatment 37.3% -33.94 99%

34% Decrease in conversions

Control Treatment

How did brand standards affect the user experience?

5. The impact of brand standards on end-behaviorWe must ensure that our defined elements of our brand help, rather than hinder, people when it comes to making the right decision.


Example

From This To This

40%Subscriptions

Summary




4. The growing effectiveness of an “everyday tone” in email People still don’t buy from (or give to) companies, stores or websites; people buy from people.

Review: Top 5 Marketing Discoveries in 2015

5. The impact of brand standards on end-behaviorWe must ensure that our defined elements of our brand help, rather than hinder, people when it comes to making the right decision.

Review: Top 5 Marketing Discoveries in 2015

Live Optimization

URL: https://www.hesonline.com

Primary Audience: B2B

Page:Health Enhancement Systems

https://www.hesonline.com/

URL: http://www.north-american-waterway.com/

Primary Audience: B2C

Page: North American Waterway

http://www.north-american-waterway.com/

URL: http://www.alpineclubofcanada.ca/

Primary Audience: B2C

Page: Alpine Club of Canada

http://www.alpineclubofcanada.ca/