Download - Special Live Test Clinic: The MECLABS team reveals the results of LifeWay’s email copy test
Case Study: Background
Background: A Web tool that lets you quickly create and customize Bible studies.
Goal: To increase clickthrough rate.
Research Question: Which email will produce the highest clickthrough rate?
Approach: A/B split test
Brand: SmallGroup.comLocation: MarketingSherpa Research LibraryProtocol ID: Pending
Case Study: Background
• Julie brought an email to Email Summit 2015 to gain live feedback from her peers.
Case Study: Original email
Subject Line:
The fast, free way to create custom
discipleship content.
Campaign Objective:
Increase free trial starts
About the List:
Pastors, small-group leaders, paid church staff in general
Case Study: Original email
Subject Line:
The fast, free way to create custom
discipleship content.
Campaign Objective:
Increase free trial starts
About the List:
Pastors, small-group leaders, paid church staff in general
How would you improve this email?
Audience suggestions
1. “The headline is meaningless … it needs to be in sentence form.”
2. “That image doesn’t give me any context for the content.”
3. “The sub-header is not related to the content.”4. “Strip out that whole last paragraph … it should
just be boiled down to ‘Try the Free Preview.’”5. “Make [the button look] more like a button. It
looks more like a banner.” 6. Change button text to “Learn more”7. Change button text to “Learn about our free
two-week preview”
Case Study: Treatment
• A treatment was created using most of the feedback (but not all of it).• To see the full recording go
to: http://goo.gl/vJSTlt
Case Study: Results
Email Design Total Click Rate Button Click Rate
Original 0.8% 0.2%
Treatment 1.1% 1.0%
% Relative Change: 42% 358%
Increase in Button Clicks358%The email design that utilized suggestions from the live audience achieved significantly more total clicks and button clicks.
Case Study: Results
Email Design Total Click Rate Button Click Rate
Original 0.8% 0.2%
Treatment 1.1% 1.0%
% Relative Change: 42% 358%
Increase in Button Clicks358%The email design utilizing suggestions from the live audience achieved significantly more total clicks as well as button clicks.
How did we develop a treatment from the live feedback that achieved this result?
Case Study: Results
• Often, we generate unnecessary costs by conflating the objective of an email with the objective of a landing page. The goal of most emails is simply to get a “click.”
• We must challenge our emails with this question: “Is there a single word or piece of content on the page that does not help to achieve a click?” Every unnecessary piece of content is waste and reduces your chance of achieving a click.
Key Principles
Which of these suggestions can help us get the click and which won’t?
1. “The headline is meaningless … it needs to be in sentence form.”2. “That image doesn’t give me any context for the content.”3. “The sub-header is not related to the content.”4. “Strip out that whole last paragraph … it should just be boiled
down to ‘Try the Free Preview.’”5. “Make [the button look] more like a button. It looks more like a
banner.” 6. Change button text to “Learn more”7. Change button text to “Learn about our free two-week preview”
What does this look like when translated into the treatment?
1. The headline is made into a complete, understandable sentence.
2. The image size is reduced and adjusted.3. The sub-header is now more specific to the
content .4. The whole last paragraph is replaced with
relevant bullet points.5. The button is trimmed down and given a drop
shadow so it resembles a button.6. The button copy is changed to a low-
commitment action.
Key lessons learned
• Collaboration is good, but not all feedback is necessarily helpful.
• To ensure that peer review helps achieve your marketing collateral objectives, you must discipline the feedback with a framework/heuristic.• For example: eme = rv (of + i) – (f + a)
• After you’ve applied the framework, you must discipline the analysis with a hypothesis/experimentation cycle, guided by a robust design of experiments.• To learn about “Humanizing Email,” go to: http://goo.gl/RLh8e8