Testing & Optimisation

What is it and how can it help your business?

Irene Kalkanis, February 2014

We’ll find out:• What ‘testing and optimisation’ means

• How to figure out whether you need it

• How to choose the right tool

• How to make sure you get value

What is Optimisation Testing?  Increase conversions from landing pages through to sale or goal

completion

  Decrease bounce rate from your landing page

  Make your content / services / products easier to find and easier to buy

  NOT Search Engine Optimisation (SEO)

  NOT a substitute for web analytics

What is an A/B test?

Buy Buy

50% conversion 20% conversion

What is an A/B/n test?

50% conversion 20% conversion 80% conversion 60% conversion

Copy ---------------------------------------- Buy

Copy --------------------------------------------------

Add to

Basket

… you can add as many variations as you like, each one needs to be designed separately

Read more

Buy

What is an MVT test?

Buy Buy

50% conversion 20% conversion 20% conversion 70% conversion

Add to Basket Save Buy now

Buynow

… add more versions of each element and the tool will generate many different combinations of these elements

A/B/n Testing MVT TestingNo real constrictions on which creative or page design can be tested

Test many small page elements and measure their impact on conversion rate

Test dramatic lay out and design differences Measure effect of interaction between elements

Less traffic required to reach clear statistical significance

Make small, incremental changes to your site to reduce the ‘yuck’ effect

Ability to analyse effects of different elements and combinations in detail

A/B/n Testing MVT TestingTests need to be planned carefully to ensure that we understand why one version wins over another

Requires much more traffic to reach statistical significance

All test pages must be designed manually Usually give approximate results

Can lead to sudden, major changes which can result in the ‘yuck’ effect

The methodology of focusing on small elements can constrain creativity

Can’t test major layout changes easily

Pros

Cons

Do I need to test my site?

The first step is admitting you have a problem

Greenpeace website 2004 Can you spot the call to action?

Greenpeace website 2014 DONATE!! LIKE!! PROTECT TIGERS!!!!

Every website can be optimisedYou’re already optimising by trial and error, a testing tool will let you:

• make decisions informed by what your audience actually does rather than what they tell you, or your own ‘gut feeling’

• learn what works much faster

• test changes on small portions of your audience before going live

• test your changes and hypotheses in a scientific way

These tools offer both MVT and A/B/n testing

Some tools are more focused on UX

Some tools are more focused on marketing

Many will design variants, set up the tests and analyse the results for you

Different tools & set ups work better for different sites, vendors will be able to tell you what they can do for your site

Remember that you don’t have to use the same vendor for different tools

Make sure you get value from your tool• Invest in a good support structure

- It takes a village: optimisation testing can be work intensive and needs buy in from stakeholders up front

- You may need creative and analysts to build and understand your tests- Heat mapping tools help you dig deeper into A/B/n tests

• Don’t test everything- Agree on specific hypotheses you want to test- Agree exactly what a ‘conversion’ means to your business - Focus your tests on landing pages and key conversion journeys

• Don’t test forever- If you haven’t reached a statistically significant result within a pre-agreed period of time there’s no reason to

make a change

Crazy egg heatmap – confetti view

Expedia drops optional form, earns $12m profit

Two optimisation case studiesfrom e-consultancy.com

ASOS halves cart abandonment by losing the word ‘register’

These tools also offer targeting and ‘personalisation’

Summary1. A/B and MVT testing: a way to optimise you site by test different versions at the same

time2. Picking the right tool: get information and quotes from vendors, seek non-partial advice

from an expert3. Get value: use tools wisely on well known problems, build a good team of creative and

analysts around the tool