ux research at scale

UX RESEARCH AT SCALENICK CAWTHON : TRUE UNIVERSITY 2015

HUMAN FACTORS

UX RESEARCH AT SCALE

NBACKGROUND CHALLENGES METHODOLOGY RESULTS WRAP

YOU

6

• UNSURE OF WHERE DESIGN FITS IN PROCESS / ORGANIZATION

• HAVE 100 OTHER THINGS TO FOCUS ON

• KNOW GOOD DESIGN WHEN YOU SEE IT

• THE ONLY DESIGNER / NOT A DESIGNER

, OR POSSIBLY YOU

THREE GOALS

7

• ILLUSTRATE CHALLENGES OF SCALE

• PROVIDE MAINTAINABLE METHODS

• SHOWCASE VALUE OF RESEARCH EFFORTS

DIRECTOR OF USER EXPERIENCELOGGLY

NICK CAWTHON

IXDA ORGANIZERSAN FRANCISCO CHAPTER

ACADEMIC BACKGROUNDB.S. ARCH & APPLIED ARTS | B.F.A. NEW MEDIA | MPHIL. HCI / DATAVIZ

8

@NCAWTHON

9

PERC

ENTA

GE O

F PO

STIN

GS

JAN 06 JAN 07 JAN 08 JAN 09 JAN 10 JAN 11 JAN 12

INFORMATION ARCHITECT

INTERACTION DESIGNER

UX DESIGNER

UI DESIGNER

PRODUCT DESIGNER

USABILITY ENGINEER0

.06

.04

.02

10

UX11

UR

UI IXD

COMPARATIVE TESTING

USER SESSIONS

ETHNOGRAPHIC RESEARCH

PROTOTYPING

INFORMATION ARCH.FEATURE DEVELOPMENT

FRONT-END CODING

VISUAL STYLEGUIDES

BRAND EXTENSIONS

11

UX12

ENGINEERING

PRODUCT

MARKETING

DESIGN

12

UR

UI IXD

13

CHALLENGES

BACKGROUND CHALLENGES METHODOLOGY RESULTS WRAP

• SOFTWARE AS A SERVICE

• SINGLE PAGE APPLICATION

• DEVOPS / SYSADMIN FOR SMBS

• HERMIT DEMOGRAPHIC

• 6,000+ ACTIVE USERS

15

CONTEXT OF LOGGLY

• EXPECTATION OF CHANGE

• EVERYTHING IS TESTABLE

• TRADITIONAL METRICS IRRELEVANT

• CONSTANT THREAT OF CHURN

• PRESSURE TO DIFFERENTIATE

16

CHALLENGES OF SAAS

17

“HOW M’ DOIN?”

17

“HOW M’ DOIN?”

18

IN THE BEGINNING

19

19

RECRUITING

SCHEDULING

RECORDING

DOCUMENTING

METHODOLOGY

BACKGROUND CHALLENGES METHODOLOGY RESULTS WRAP

21

QUANTITIVE METRICS

INTERNAL USAGE ANALYTICS

• SIGNUP CONVERSION

• ACTIVATION RATES

• UPGRADES V. DOWNGRADES

• CUSTOMER CHURN

• REPEAT LOGINS

• INTERNAL EVANGELISM

• FEATURE ADOPTION

• REVENUE

NET PROMOTER SCORES

X X

NET PROMOTER SCORES

• WIDELY-ADOPTED

• OPEN-ENDED COMPONENT

1 2 3 4 5 6 7 8 9 10

DETRACTORS PASSIVE PROMOTERS

• THIRD-PARTY INTEGRATIONS

• SELF-ADMINISTERED METHODS

24

HIGH TRUST

MED TRUST

LOW TRUST

GREENRED YELLOW

-30 -20 -10 +10 +30 +40 +60+50+20

INDUSTRY TRUST /TRANSPARENCY

NPSSCORE

TELCOM

FINANCIALSERVICES

BUSINESS CONSULTING

COMPUTER HARDWARE

WEB SERVICES

B2B AVGHIGH

GROWTHFEEDTHEBEAST.BIZ

CLERGY

25

BE CONSISTENT

26

QUALITATIVE

OLARK INTERCOM

• BASIC IMPLEMENTATION

• LIVE CHAT ONLY

• CUSTOMIZABLE

• MORE ADVANCED

• OFFLINE INPUTS

• CASE MANAGEMENT

• SUPPORT TICKET REPLACEMENT

28

CONFERENCES

SESSION PATHS

29

UNINITIATED INITIATED

• SETUP / CONFIG FLOW

• BASIC FEATURES

• INTUITION / IMPRESSIONS

• CURRENT SOLUTIONS

• SPECIFIC USAGE FLOWS

• BLOCKERS

• ANY EDGE CASES

• WORKAROUNDS

30

AFTER THE PARTY

REMOTE SESSIONS : FEEDBACK DRIP

31

TIER = FREE / PAID• HAS LOGGED IN THIS WEEK• OPPORTUNITY - STAGELOGIN COUNT < 10LAST LOGIN < 30 DAYS

GREATER THAN 12 MONTHSSTAGE DOES NOT CONTAIN:

• NOT ENGAGED• WORKING• 2-WAY CONVERSION• TRIAL CLOSE• CLOSED / WON

RESULTS

BACKGROUND CHALLENGES METHODOLOGY RESULTS WRAP

33

34

“THIS IS THE MOST FRUSTRATING THING ABOUT YOUR PRODUCT IS…”

35

36

“…I WANT YOU TO CREATE A SAVED SEARCH”

37

“…GENERATE A PIE CHART FROM THE FIELD JSON.CITY”

39

“NOBODY ELSE DOES THIS…”

41

SOME RANDOM DUDE

45

WRAP

BACKGROUND CHALLENGES METHODOLOGY RESULTS WRAP

TAKEAWAYS

49

• CONSTANTLY REMIND YOURSELF OF USER FRUSTRATIONS

• CUSTOMER-FOCUSED PRIORITIZATION

• MAINTAIN A SCALABLE APPROACH TO RESEARCH

• EXTERNALIZE AFTER EVERY LAUNCH / CONFERENCE

50

Q&A

CALEND.LY

52

• SHARABLE LINKS• MAX SESSIONS PER DAY• CALENDAR SYNC• ENFORCED GAPS• CUSTOM FIELDS• REMINDER TRIGGERS

SILVERBACK

53

• SCREEN RECORDING• CUSTOM HOTKEYS• HIGHLIGHT MARKERS• TAGGING, NOTES• TASK COMPLETION MARKERS

SATECHI MEDIAREMOTE

54

• BLUETOOTH REMOTE• POCKET HOTKEY TRIGGERING• MINI-KEYBOARD

BLUE SNOWBALL• OMNI DIRECTIONAL MIC• USB INTERFACE• CANCEL BACKGROUND NOISE