ux webinar: always be testing

60
A.B.T. (Always Be Testing) How UX testing can help you meet and exceed program goals

Upload: charity-dynamics

Post on 22-Jan-2017

59 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: UX Webinar: Always Be Testing

A.B.T. (Always Be Testing)How UX testing can help you meet and exceed program goals

Page 2: UX Webinar: Always Be Testing

Why Test UX?1

Page 3: UX Webinar: Always Be Testing

You might be thinking…

• It takes time and expertise to collect & analyze information…

• Testing might expose a need for more extensive work...• This all impacts the cost and schedule of my project…

…So why should I do it?

Page 4: UX Webinar: Always Be Testing

4

You’re losing money if you aren’t testing.

Testing helps identify problems before time and money are spent creating something that doesn’t

work well.

Page 5: UX Webinar: Always Be Testing
Page 6: UX Webinar: Always Be Testing

6

Page 7: UX Webinar: Always Be Testing

7

Page 8: UX Webinar: Always Be Testing

8

Testing will help you meet and

exceed your goals.

Page 9: UX Webinar: Always Be Testing

User

Experience

User

Engagement

User

Satisfaction

Conversions / Meeting

Goals

Page 10: UX Webinar: Always Be Testing

10

Testing will validate or

invalidate your assumptions.

Page 11: UX Webinar: Always Be Testing

Challenging Your Expectations

• Did you see what you expected to see?• How are users defying your expectations?• What is working well? • What isn’t working well?

Page 12: UX Webinar: Always Be Testing

12

Testing will help you think like

your users.

Page 13: UX Webinar: Always Be Testing

Thinking Like Your Users

• See your content from the viewpoint of someone who isn’t you (your organization, your BOD, etc.)

• Learn things you didn’t even know you should be thinking about

Page 14: UX Webinar: Always Be Testing

The Testing Blueprint2

Page 15: UX Webinar: Always Be Testing

When?The time is always…

Page 16: UX Webinar: Always Be Testing

Testing Blueprint: When to Test – Now

Start testing right now, focusing on these areas:

1. High traffic pages2. Conversion pages3. Email campaigns

Page 17: UX Webinar: Always Be Testing

Testing Blueprint: When to Test – Optimal Times

Optimal testing times:

1. Before a redesign2. Before EOY campaigns3. Before large fundraising pushes (awareness or action months)4. Don’t test during your largest and most critical campaigns, test

before.

Page 18: UX Webinar: Always Be Testing

Testing Toolkit

Page 19: UX Webinar: Always Be Testing

Testing Toolkit: Tools for the Job

• Native Platform Analytics• Google Analytics• Crazy Egg• Google Experiments• Optimizely• Moveable Ink (Email Campaigns)

Page 20: UX Webinar: Always Be Testing

Testing Toolkit: Native Platform Analytics

• Purpose: Data Collection• Cost: Free/Included with

Platform• Learning Curve: Easy• Examples:

– Jetpack for Wordpress– Luminate Online Reporting

Jetpack for Wordpress

Page 21: UX Webinar: Always Be Testing

Testing Toolkit: Google Analytics

• Purpose: Data Collection• Cost: Free• Learning Curve: Medium• Setup: Include JavaScript

snippet on every page of your website (or any page you need tracking)

Google Analytics Dashboard

Page 22: UX Webinar: Always Be Testing

Testing Toolkit: Crazy Egg

• Purpose: Data Collection• Cost: Low• Learning Curve: Easy• Use: View user clicks, hover

and scroll tendencies to determine effect UX design with heat mapping.

Crazy Egg Heat Map

Page 23: UX Webinar: Always Be Testing

Testing Toolkit: Google Experiments

• Purpose: Variation Testing (A/B/C)

• Cost: Free• Learning Curve: Hard• Setup: Use JavaScript to

change elements, Google Analytics/Experiments handles testing parameters and reporting

Google Analytics Dashboard

Page 24: UX Webinar: Always Be Testing

Testing Toolkit: Optimizely

• Purpose:– Variation Testing (A/B/C)– Data Collection

• Cost: Enterprise• Learning Curve: Easy• Setup: Insert JavaScript

snippet and run experiments using Optimizely’s platform

Optimizely Experiment

Page 25: UX Webinar: Always Be Testing

Testing Toolkit: Movable Ink

• Purpose: Email Engagement• Cost: Low (based on email

sends)• Learning Curve: Easy• Use: Engage users with

dynamic images based on location, constituent information or date/time.

Movable Ink Dashboard

Page 26: UX Webinar: Always Be Testing

Realistic GoalsTo-Do List:

1. Wait for Tonight2. Try and Take Over the World

Page 27: UX Webinar: Always Be Testing

Brainstorm Goals: The “How Can We?” Question

What are your goals? Start broad with a brainstorming session and generalized language:

“How can we…”– increase donations?– get more email sign-ups?– increase engagement with email campaigns?

Page 28: UX Webinar: Always Be Testing

Refine Goals: The “What Are We Measuring?” Question

Next, include a clear measurable you can monitor with data:

“How can we increase donation…”– click throughs?– conversions?– average amounts?

Page 29: UX Webinar: Always Be Testing

Define Goals: The “Would This Work?” Question

Now refined, it’s time to define your goals with an action:

“Would this work?”– A donation lightbox on high-traffic pages to increase donation click-

throughs.– A geo-targeted banner on the homepage to increase event

registrations.– Stronger language in our email campaigns to increase our CTR.

Page 30: UX Webinar: Always Be Testing

Measure Twice…Test Once

Page 31: UX Webinar: Always Be Testing

Measuring Success: The First Measure

Before you test, answer these questions:

1. What is your current performance on these pages/campaigns?2. What is the industry and internal organization average

performance?3. How long will we run the test to ensure statistically significant

data?4. What is a realistic expectation for success?

Page 32: UX Webinar: Always Be Testing

Measuring Success: The Test

While you test, here are your new mantras:

1. Be patient! Stick to the timeline and see it to the end.

2. Monitor consistently

1. Set up reports daily/weekly2. Perform real-life tests periodically3. Tweak if results show dramatic winner half-way through

Page 33: UX Webinar: Always Be Testing

Measuring Success: The Second Measure

After your test, debrief and decide:

1. Were the results statistically significant?2. Did the test run long enough to ensure a large enough sample

size for scaling?3. Were our expectations met?4. What did we learn?

Page 34: UX Webinar: Always Be Testing

Testing for a Website3

Page 35: UX Webinar: Always Be Testing

35

Google Analytics Content Experiment: Donation Intercept Test

GOAL:

Determine the best CTA option to increase traffic to the donation form

TOOL:Google Analytics (GA)• No additional investment

required• GA already set-up to track

the donation funnel

Page 36: UX Webinar: Always Be Testing

36

Control Lightbox Test

Page 37: UX Webinar: Always Be Testing

37

Donation Intercept Test: Measures/Outcome

• Control: – Donation button in header– Donation array in footer

• Lightbox Version A:– Lightbox appears after 15

seconds• Lightbox Version B:

– Lightbox appears after 75% scroll

Measures

• Version A resulted in highest traffic generation

• The control and Version A delivered similar numbers of completed donations

• Version B proved not to be a valid option to drive either traffic or donations

Outcome

If you have 1,000 donations per year, imagine what a 10% increase would do to your bottom line!

Page 38: UX Webinar: Always Be Testing

38

Optimizely: Donation Array Test

GOAL: • Determine the most

valuable offering for donation array

• Determine if there is any variance based on entry point to the donation form

TOOL:

Optimizely• GA didn’t track donation funnel• Easy to replicate and host

multiple forms • Minimal disruption for their

internal web & IT teams• Ability to segment audiences

and select traffic sources to determine differences

Page 39: UX Webinar: Always Be Testing

39

Donation Array Test

Three Versions:CONTROL VERSION A

(Lower)VERSION B (Higher)

$50 $35 $55

$100 $50 $100

$250 $100 $250

$500 $250 $500

$1,000 $500 $1,500

Page 40: UX Webinar: Always Be Testing

40

Donation Array Test: Measures/Outcome

Test GroupHome Page

Click-through Rate

High Traffic Internal Page Click-through

RateDonation Form

Completion RateAvg.

Donation

Variation 1 - Control 2.1% 0.014% 10.4% $99

Variation 2 - Lower 1.9% 0.020% 12.8% $106

Variation 3 - Higher 2.2% 0.022% 13.4% $144

2015 Industry Avg. is 7%

Imagine if all of their donations from the first half of the year had come in at $144 vs. $99 avg. gift ?

Page 41: UX Webinar: Always Be Testing

41

Optimizely : Geo-Targeted Event Recruitment Banners

GOAL: • Determine if we could

convert Home Page visitors to Walk participants

TOOL:

Optimizely• Easy to modify pages without

diving deep into code• Minimal disruption for their

internal web & IT teams• Easy to serve specific creative to

different audiences (geo-targeted)

• Ability to set up timed experiments based on event date

Page 42: UX Webinar: Always Be Testing

42

Geo-Targeted Event Registration Banners

Desktop Mobile

Page 43: UX Webinar: Always Be Testing

43

Geo-Targeted Recruitment Banners: Measures/Outcome

• Not a traditional test• No versions to test against• 6 pilot locations• Each banner was coded to display to

a unique visitor within a specified geographic catchment area

• Banners shut-off automatically the day before the event

Measures• Banners resulted in 2.5 – 6%

engagement rate, depending on locations (vs. 0.08% for a traditional paid display ad)

• No change in bounce rates or drop off rates

• Registrations ranged from .5% – 1%

Outcome

What would a 1% increase in event registrations mean for your event?

Page 44: UX Webinar: Always Be Testing

44

Crazy Egg: Heat Mapping to Inform UX Design

GOAL:

• Re-design or Re-fresh

• Prove that the investment in the current website design still sound

TOOL: Crazy Egg Heat-Mapping• Site already used best practices

for successful event sites• No obvious challenge areas• Observe user behavior to

identify opportunities and gaps• Connects to GA account via

simple code inserted into page • Low monthly fee – very

affordable testing option

Page 45: UX Webinar: Always Be Testing

Heat Mapping to Inform UX Design

The Site Heatmap Overlay

Are site visitors doing what you want them to do?

Page 46: UX Webinar: Always Be Testing

Long Scroll PagesWhere are they spending their time?

Are they getting to the most valuable information?

What information are they missing ?

Page 47: UX Webinar: Always Be Testing

47

Heat-mapping: Measures/Outcome

• Understanding how your site visitors are interacting with your content to inform possible updates or future testing

• Provides navigation cues to streamline content to ensure focus is on your main CTAs: register, donate, log in

• Identifies content placement priorities for long-scrolling pages

Measures

• Proved that the overall structure of the site is good

• Prioritize content on long-scrolling pages

• Optimal placement for social sharing tools

Outcome

Age old question: Website re-design or website re-fresh? Let your site visitors tell you what they need.

Page 48: UX Webinar: Always Be Testing

Testing for an Email Campaign4

Page 49: UX Webinar: Always Be Testing

49

Why email testing?

The channel where you are consistently reaching out to communicate directly with your most dedicated supporters.

Really learn about your constituents and include those lessons in your ongoing campaign strategies…

• What inspires them?• What drives them?• What might fall flat?

Page 50: UX Webinar: Always Be Testing

50

A/B Testing

What is it?

It’s split testing! Comparing two versions of a web page or email to see which one performs better. You do this by splitting your audience and serving up a different variant to find which creative results in a better conversion rate.

Most email marketing tools have built-in, user-friendly A/B testing features.

Page 51: UX Webinar: Always Be Testing

51

A/B Testing: General vs Hard-Hitting Language

GOAL:

• Prove theory to internal review teams: hard-hitting facts and statements would result in better performance

• Statistically significant results

• Lift KPIs, including amount raised

TOOL: Built in A/B Testing Functionality• Ability to easily split segments

and serve up A/B creative

• Manageable resource lift to create test build from control email build

• Easy reporting to identify winners early in campaign for later sends and, ultimately, determine statistically significant results

Page 52: UX Webinar: Always Be Testing

52

General Language Version Hard-Hitting Language Version

Page 53: UX Webinar: Always Be Testing

53

A/B Testing: General vs Hard-Hitting Language

• Developed Marquee Testing strategy over 3 largest campaigns through first half of year for 6 months of data on ONE test

• Followed Engagement KPIs: Open Rates, CTR, Conversion Rate, # of Gifts, Amount Raised, Avg Gift

Measures• Hard-hitting copy had a direct impact not

only on the decision to give but the amount of which the donor decided to give– Amount raised and average gift KPIs

for launch up 100% or more compared to control

• Confidence in a global brand refresh that heads in bold direction

• All teams on the same page to focus on new learnings

Outcome

If hard-hitting language was right for your organization and raised a $30K campaign by 30-40%, you could make an

additional $9,000 to $12,000!

Page 54: UX Webinar: Always Be Testing

54

Movable Ink

What is it?

Contextual email marketing software that allows you to dynamically change the content of an email upon the moment of open to tailor messaging in real-time.

Page 55: UX Webinar: Always Be Testing

55

Movable InkWhat you can do with Movable Ink:

• Image Personalization• Geo-Targeted Personalization • Weather Personalization• Device-Targeted Personalization• Data Automation, Image Swaps• Live Polling• Live Social Feeds• Real-Time Optimization

Page 56: UX Webinar: Always Be Testing

56

Example: Humane Society CTA Button Testing

Page 57: UX Webinar: Always Be Testing

57

Example: Humane Society Average Gift Suggestion Test

Page 58: UX Webinar: Always Be Testing

58

ABT: Always Be Testing

• There is no wrong time to start testing

• Testing is way worth the investment

• Start by testing your assumptions

• Test in increments, one element at a time

Page 59: UX Webinar: Always Be Testing

59

WELCOME TO THE FUTURE.

Coming soon…

Page 60: UX Webinar: Always Be Testing

Questions?