empowering data-driven marketers: how ux research & usability testing can positively impact your...

Empowering Data-Driven Marketers: How UX Research & Usability Testing Can Positively

Impact Your Overall Brand ExperienceFeaturing Jeff Sauro

Quick Housekeeping

• Control panel on the side of your screen if you have any comments during the presentation

• Time at the end for Q&A

• Today’s webinar will be recorded for future viewing

• All attendees will receive a copy of the slides/recording

• Continue the discussion using #uzwebinar

2016 goals

1. Provide more UX resources to UX professionals.

2. Improve brand consistency.3. Improve “International” UXPA.

User Experience Professionals Association

(UXPA)

Annual Conference in Seattle, WAMay 31 – June 3uxpa2016.org

Monthly webinarsUser Experience Magazine

The Journal of Usability Studies

Visit www.uxpa.org for more info about the conference, webinars,

and other UX resources!

Measuring U 2016 4

UserZoom at a Glance

• All-in-one solution to rapidly test Usability & Measure UX

• Remote Unmoderated Usability Test (Qualitative or Think-out-loud)

• Usability Benchmarking • Card Sorting• Tree Testing• Screenshot Click Testing• Screenshot Timeout Testing• Online Surveys

• Test any digital UI at anytime during the product life cycle

The All-in-One UX Research Platform for the Data-Driven Enterprise

Measuring U 2016 5

Meet the Speaker

Jeff Sauro PhDFounding Principal | MeasuringU Measuringu.com @MeasuringU

BS Syracuse University

MA Stanford University

PhD University of Denver

IT & TV/Radio Film

Learning Design & Technology

Research Methods & Statistics

Measuring U 2016 6

That’s Not What UX does…That’s Not Marketing’s Job!

MarketingUX David RicardoLaw of Comparative

Advantage

Measuring U 2016 7

Division of Labor

Marketing UX

Quantitative Qualitative

Measures branding and satisfaction Measures behavior (if they measure anything)

Segmentation Personas

Considers what customers say Considers what customers do

Quantitative Qualitative

Measuring U 2016 8

Mixing Methods & Mindsets

• It’s OK to mix quantitative and qualitative

•Don’t be afraid to use surveys and observation

•Understand how the user experience affects brand attitudes and vice versa.

•Measure what people think and what they do, often in the same study.

Measuring U 2016 9

Budget Enterprise1

2

3

4

5

6

7

4.7 5.35.3 4.5

PrePost

Website Experiences Affect Brand Attitudes

Enterprise Brand favorability ratings dropped by 15% after users attempted to rent a car.

Very

Fav

orab

leVe

ry U

nfav

orab

le

12% 15%

Measuring U 2016 10

Mix Methods and Mindset

Steps1. Identify and collect Key Performance Indicator (KPIs)2. Uncover your customer segments and build personas

3. Collect a measure of UX Quality

4. Understand customers’ top tasks

5. Benchmark the Experience and link to KPIs

6. Mix quant and qual. methods to improve

7. Measure again

8. Link to changes in KPIs

Traditional “Owner”

Mar

ket

Rese

arch

UX

Rese

arch

Measuring U 2016 11

1. Identify & Collect the Key Performance Indicators (KPIs)

Branding Questions• Hotels.com is a Brand I can trust.• When thinking about renting a car, which company is your first choice?

Satisfaction (By Touchpoint)• How satisfied were you with installing the product?• How satisfied were you with recent call to customer support ?

Loyalty• How likely are you to recommend to a colleague or friend?

Future Intent• How likely are you to return and book a hotel on this website?

Look to leverage existing company KPIs

These are what you want to

IMPROVE

Measuring U 2016 12

2. Uncover customer segments and build personas

Key DifferentiatorsAge

Online Comfort

Income

Zillow Trust

Realtor Trust

Low

High

Low

High

Low

High

Low

High

Low

High “I don’t believe in it. I’d

rather make large monetary exchanges face-to-face.”

DemographicsAge: 57 years oldIncome: $50K-$100K Ethnicity: Caucasian Children: No children living in the homeMarital Status: DivorcedNext home purchase: 2+ yearsFirst home purchase: 20 years ago

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

23%

60%

18%

N = 1200Personas should come from large scale surveys and in depth interviews.

Measuring U 2016 13

3. Collect a measure of Website UX Quality : SUPR-Q

USABILITY CREDIBILITY (TRUST, VALUE & COMFORT)

LOYALTY APPEARANCE

• It is easy to navigate with this website

• This website is easy to use

• I will likely visit the website in the future

• How likely are you to recommend this website to a friend or colleague? [NPS]

• The information on this website is trustworthy.

• The information on this website is credible.

• The website has a clean and simple presentation

• I found the website to be attractive

Standardized User Experience Percentile Rank Questionnaire

More at suprq.com

Measuring U 2016 14

4. Find what customers want to do: Top Task Analysis

Separate the trivial many from the vi-tal few

TOP TASK ANALYSISVital few tasks

Trivial Tasks

Measuring U 2016 15

Measuring U 2016 16

Automotive Websites Top Tasks

Pick Only Five

Measuring U 2016 17

Automotive Website: 94 Tasks

-5%

0%

5%

10%

15%

20%

25%

30%

Trivial “Long Tail” Tasks

Long Neck

How many miles per gallon will a car get?

Measuring U 2016 18

Top Tasks: Health Insurance Website

0%

1%

2%

3%

4%

5%

6%

7%“Find someone close to my home/office”“See if my healthcare professional will accept my insurance”

“See if a particular doctor is in my plan”

Measuring U 2016 19

5. Benchmark The Experience: SUPRQ

Error bars represent 90% confidence intervals | N: 394

SUPR-Q Usability Trust, Value & Comfort Loyalty Appearance NPS (Raw %)0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

81% 76% 79% 75% 75% 35%

Measuring U 2016 20

Task Data

Error bars represent 90% confidence intervals | N: 394

Search Browse Checkout Fulfillment0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

70% 61% 62% 63%

These are your top-tasks per user segment.

Measuring U 2016 21

Link activities to KPIsHow likely are you to book a hotel on your next visit?

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Percent of Maximum Score

Impo

rtanc

e (B

eta

Coeffi

cient

)

Brand Awareness

Trust in Price

Website Usability

Measuring U 2016 22

Link activities to KPIsHow likely are you to recommend this website to a friend? [Categorized Verbatim Responses]

NPS %Simplify/Ease/Nav

Specific Suggestions

Design Ads Conditions & Opinions

Search Speed Readability Comparisons Zip Codes Survey Comments

-19% 0% 45% 100% -25% -29% 80% 50% 25% 0% 0%

0%

5%

10%

15%

20%

25%

30%

35%

40%

34% 18% 10% 7% 7% 6% 5% 4% 4% 3% 3%

Measuring U 2016 23

6. Mix Quant & Qual. Methods to Improve

CARD SORTING

HEURISTICEVALUATIONS

FIRST CLICKTESTING

KEYSTROKELEVEL MODELING

TREE TESTING

AnalyticEmpirical

USABILITY TESTING

Measuring U 2016 24

6. Compare After Changes: SUPR-Q Year-Over-Year

Error bars represent 90% confidence intervals | N: 394, 148

Statistical improvement in UX Quality Scores

SUPR-Q Usability Trust, Value & Comfort Loyalty Appearance NPS (Raw %)0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

81% 76% 79% 75% 75% 35%95% 91% 98% 87% 94% 62%

Desktop 12/13Desktop 15/16

DATA COLLECTION

Measuring U 2016 26

Data Collection Methods

WEBSITE INTERCEPTS

TARGETED SURVEYS

Leverage existing survey efforts where possible.

Measuring U 2016 27

Typical Combined Study Flow in UserZoom

Welcome Page Segmentation & Branding Questions

Task Description

Tasks Post Task Questions Post Study Questions

Measuring U 2016 28

Integrate Survey Data with Website Behaviors

UserZoom Google Analytics

Measuring U 2016 29

Mix Methods and Mindset

Steps1. Identify and collect Key Performance Indicator (KPIs)2. Uncover your customer segments and build personas

3. Collect a measure of UX Quality

4. Understand customers’ top tasks

5. Benchmark the Experience and link to KPIs

6. Mix quant and qual. methods to improve

7. Measure again

8. Link to changes in KPIs

Traditional “Owner”

Mar

ket

Rese

arch

UX

Rese

arch

Learn MoreLondon June 16th

Creating a Plan to Measure the User Experience Hosted By UserZoom @ Altitude 360 London

Rome June 20th-22ndUX Boot CampMeasuringu.com/italyux.php

Chicago July 15th Quantifying the User Experience (Half Day Workshop)Measuringu.com/chicago.php

Denver Aug 17th-19th UX Boot Campdenverux.com

Q&A

Need more info?Get answers about UserZoom. We are here to help. Talk with one of our experts.

Contact us

UserZoom10 Almaden Blvd. #250San Jose, CA 95113

Toll free number: (866) 599 1550

www.linkedin.com/company/userzoom

@UserZoomFollow us