01.16.08

Multivariate Testing &Conversion Tweaking

Jason CooperDirector, Value Added Services

jason_cooper

November 11, 2009

Performance-Based Design

2

We can no longer assume that what we think as designers is best for users.

Nov. 1996Oct. 1997Nov. 1999Oct. 2000

Nov. 2002Nov. 2003Nov. 2004

Nov. 2006

Today

• Technology on the web is maturing

• Broadband penetration

• Down economy has made getting “more for less” a top goal

• Availability of free technology

How Did We Get Here?

3

• Growing awareness of and interest in testing

• Greater ROI through post-click optimization

• Hunger for more data

What Does It Mean?

4

+

• To find the granular synergies that persuade visitors you need complex statistical analysis

• Available tools remove the complexity so you can focus on what to test not how to test

• Variety of options and costs

The Technology

5

• Simple Answer = EVERYTHING

• Challenge is knowing where to start

What Should You Test?

6

• First impressions• User-experience • General layouts • Radical redesigns• Better for A/B

When we think like designers

• Specific messaging• User-experience • Individual elements• Variable alternatives• Better for multivariate

When we think like marketers

What’s sticky? What resonates?

Specific Tweaks

7

Call to ActionCall to Action

HeadlineHeadline

Brand PresentationBrand Presentation

Hero Shot / Main ImageHero Shot / Main Image

Product ImageProduct Image

ButtonsButtons

IncentivesIncentives

“The Process”“The Process”

Test What Matters

Worst

+15%

+30%

The Data Inputs

9

“Plans based on average assumptions are wrong on average.”

-from The Flaw of Averages by Sam Savage

Interpreting the Data

10

• It depends on your business and the tools you have available

How Should You Segment Your Data?

11

Age

Gender

Language

Country

Connection Speed

Paid Search

Natural Search

Social Media

Affiliates

Keywords

Research Mode

Consideration

Motivated

Repeat Visitor

Loyal Customer

• Qualitative data can provide flavor, texture, and nuance to quantitative data

• Typically quantitative data can’t tell you “why”

Combining Data Types

12

+ = MeaningfulSegments

+ = UnderstandThe “Why”

• Slice, dice, and compare data any way you can to find relevant relationships

• Look for cause and effect

• Data on your competitors can provide the missing piece of the puzzle

• Utilize 3rd party tools to add more “color” to your data

Find the Relationships

Twitter Spectrum

Facebook Lexicon

• Google’s “41 shades of blue”

• Involve all of your business units

• Set aside a portion of your budget for testing

• Data can breed creativity—allows you to take risks

Build a Culture of Testing

16

• Doubling your conversion rate is far easier (and cheaper) than doubling your traffic

• Get to know your data (all of it) intimately so you can uncover relationships

• Segment and combine your data for more meaningful interpretation

• Test everything until you know what matters

• Develop a culture of testing

• Evolve through continuous testing and improvement cycles—become data-driven

Take Aways

17

01.16.08

Thank You.jason.cooper@efrontier.com

Jason CooperDirector, Value Added Services

jason_cooper

November 11, 2009