user-testing, testing, 1,2,3
Post on 15-Jul-2015
1.948 Views
Preview:
TRANSCRIPT
Agenda
• The W’s
• Types of User-Testing
• How User-Testing Fits in the Web Development Process
– Persona Development
– Information Architecture Testing (IAT)
– Visual Affordance Testing (VAT)
– User Acceptance Testing (UAT)
– Multivariate Testing (MVT)
What?
• What is User-Testing?
User-Testing is a focal part
of the User Centered Design
philosophy that aims to gain
direct feedback from users and
collect actionable data
Why?
• Why Test Users?
• We often think we know what is best for our users
• You want to add innovative functionality to your site
• You have low conversion rates
• You have high customer support costs
When?
Benchmark
Test User Expectations
Test Visual Affordance
Test User Performance
Test Live Website
Test Expectations / Mental Model
Who?
• Users, not Testers
• Real people who use your
website
• Rinse, Lather, REPEAT!
• Who Do We Test?
Who?
• Who Do We Listen
To? We need to place four ad
banners on the homepage
I want to read content
that is fresh and
interesting
We need to drive users
to download our latest
whitepaper
I don’t want my
decisions manipulated
We want to display
company news and our
stock ticker on the
homepageI know what I want and
want to find it
without being
distracted
Where?
• Where Do We Find Users to Test?
•On your website
•Through a recruiter
•In their natural
environment
How?
• How can we avoid having to
figure all this out on our own?
Let the users do
all the hard work
for you!
Types of User-Testing:
Remote
• Users sit in
their natural
environment
• Users are asked
to carry out
specific tasks
using prototypes
• Their mouse
movements and the
phone
conversation are
recorded for
analysis
Types of User-Testing: In
Lab
• Video camera
records users’
facial expressions
• A screen recorder
records their
screen movements
• A facilitator is in
the room with them
• Observers are in
another room,
usually looking
through a 2 way
mirror
Types of User-Testing:
Automated• Users are
intercepted when they appear on a site
• Users are asked to fill out a short survey before navigating the site
• All the user’s movements are recorded while navigating the site
• Users complete an exit survey when they leave the site
Types of User-Testing
Type Pros Cons What to Use it On
Remote Nationally diverse geographic reach
Quick
Less Costly
Lose the ability to see users’ expressions and gestures
Unnatural environment
E-commerce web sites
Large, informational web sites
Web Applications
Intranets
In Lab Obtain data based on user’s body movement and facial expressions
More costly
Lack of geographic reach
Highly secure client/server applications
Handheld Apps or other products with a significant hardware component
Automated Captures users that come to the site and records users’ activities
Natural environment
Actual users
Steps in between start and goal is not elicited
E-commerce web sites
Large, informational web sites
Web Applications
Intranets
Types of Data Collected
• Quantitative
– Statistics,
Trends
– Large Sample Size
– Structured Data
Collection
Methods
• Qualitative
– Opinions,
Details
– Small Sample
Size
– Unstructured
Data Collection
MethodsExample:
―I tried clicking on the button
because the white text on the
blue background caught my eye
and the text was large‖
Example:
•30% of users clicked on the red
button
•70% of users clicked on the blue
button
Informal vs. Formal
Testing
• Informal
– Casual
– 3-4 Days
– In Your Office
– Friends & Family
– Paper Prototypes
– Less Common
– Bias
– Incorrect
Analysis
• Formal
– Structured
– 3-4 Weeks
– In a Lab
– Broad Range of
Users
– High Fidelity
– More Common
– Trained Moderators
& Analysts
The Web Development Process
Benchmark
Test User Expectations
Test Visual Affordance
Test User Performance
Test Live Website
Test Expectations / Mental Model
The Testing Methods
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing
Card Sorting
PD:
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing &
Optimization
Card Sorting
PD:
How do we develop a website that
is tailored to our users needs,
behaviors, experience, and
technology?
Ask
them!
Q:
A
:
PD: What is a Persona?
• A persona tells the story about the users of your website and attributes lifelike features to make them more relatable
• Based on real data, the persona serves as a tool to help everyone involved in development understand who and what the website is for
• The persona creates a shared vision
PD: Market Segments vs.
Personas
Market Segment
• Gender: 75% Female, 25% Male
• Region: 99% in United States
• Size: 70% metro, 20% suburban, 10% rural
• Generation: 10% baby-boomers, 20% Gen-X, 70% Gen-Y
• Income: 25k – 50k
• Occupation: Student, Administrative Assistant,
Intern
• Education: High School Diploma
• Family Lifecycle: Single
PD: Surveys & Interviews
Surveys
• Quantitative Method
• Need about 400
responses to account
for margin of error
Interviews (contextual +
individual)
• Qualitative Method
• Need about 8 participants
per segment
PD: Benefits
• Focus on the users’ goals and
needs throughout the development
process
• Prioritize features, design,
efforts based on what is most
needed by the personas
• Any internal battles will can be
settled by referring to the
personas
• Develop a site that users will
PD: Tips for Success
• Always include:
– A Name
– A Photo
– Demographics
– Description based on data collected
– Goals
– Common tasks carried out on the website
– Habits
• Always refer back to the personas!
IAT: Card Sorting
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing &
Optimization
Card Sorting
IAT: Card Sorting
How do we ensure that the
taxonomy and site linking
structure are intuitive for our
users to find our products and
services?
Have the user label
and organize pages
for you!
A
:
Q:
IAT: What is Card Sorting?
• Users are given ―cards‖ with
labels on them and asked to
categorize the information on
your site into meaningful
groupings.
• Method: In-Lab or Automated
• Sample Size: At least 15 per
segment
IAT: Types of Card Sorting
Open Card Sorting
• Categories and category labels can be created or changed by users
• Used early during the development cycle
• More difficult to analyze data
Closed Card Sorting
• Fixed categories are provided to the user and they may not re-label or created new categories.
• Used later in development cycle for further testing, or to add new content to the website
• Easier to analyze data statistically
IAT: Card Sorting Tools
(Online)
• Online tools allow for easier
data collection / analysis and
larger sample sizes
IAT: Card Sorting Benefits• Easy to administer
• Sets the foundation for the website’s navigation
and structure
• Intuitive taxonomy in the users’ ―natural
language,‖ and matches what users are searching for
(SEO)
• Intuitive taxonomy in the users’ ―natural
language,‖ and matches what users are searching for
(SEO)
IAT: Card Sorting Tips for
Success
• If in-lab, ask users to talk out
loud
• Limit the study to a reasonable
amount of cards (~50)
• Note the relationships between
cards and situations that indicate
cross-linking between categories
will be helpful for users
IAT: Wireframe Testing
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing &
Optimization
Card Sorting
IAT: Wireframe Testing
How do we find out if the layout
of information on the site is
intuitive, and if the placement of
images, text, links, messaging, and
calls-to-action help users find
what they need?
Watch users as they
navigate through
the wireframes!
Q:
A
:
IAT: What is Wireframe
Testing?
• Wireframe testing is a usability test of
the prototype pages of your site
excluding colors and other details. This
involves creating realistic tasks and
having users navigate the prototype to
fulfill those tasks.
IAT: Wireframe Testing
• Method: In-Lab or Remote
• Sample Size: At least 5 per
segment
– Why 5 Users?
IAT: Wireframe Testing
Example
• You are out in the
field and you need to
find information on
how to troubleshoot
the network. Where
would you start?
• Your business has just
switched over to VoIP.
Where would you go to
look for information
on how to manage your
new VoIP network?
IAT: Wireframe Testing:
Benefits
• Find out if the layout of the page
achieves your users’ goals and your
goals
• Low cost and quick to make changes
to a wireframe versus a real
website
IAT: Wireframe Testing: Tips
for Success
• Create tasks that are real
situations for the user (refer to
the personas)
• Do not bias the participant by
leading or priming
• Be patient!
VAT:
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing &
Optimization
Card Sorting
VAT:
How do we make sure the design is
not only aesthetically pleasing,
but also that messaging and calls-
to-action are clear for the user?
Ask users, and
watch as they
navigate through
the designs!
Q:
A
:
VAT: What is Affordance?
• Affordance is a quality of an
object, or an environment, that
allows an individual to perform
an action.
• A visual clue to the function
of an object
VAT: What is Visual
Affordance Testing?
• Once images, colors, and other
design elements have been
implemented based on the
wireframes, VAT requires users to
indicate what they perceive as
clickable on the design
• Method: In-Lab or Remote
• Sample Size: At least 5 per segment
VAT: Benefits
• Validates if key calls-to-action
are even seen by users
• Reveals elements of a design which
may mislead users to believe are
clickable
• Reduce user frustration by
addressing problems found in the
VAT
VAT: Tips for Success
• Pay close attention to any
hesitation or vocal expressions
• Visual affordance does not only
apply to calls-to-action or links
UAT:
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing &
Optimization
Card Sorting
UAT:
How do we find out if all the
content, interactions, and media
on the prototype we’ve built will
resonate with our visitors, and if
they negatively affected our
site’s usability?
Ask users, and watch as
they navigate through the
prototype!
Q:
A
:
UAT: What is User Acceptance
Testing?
• User Acceptance Testing is a
usability test conducted on a
fully functional prototype
prior to launching the website
• Method: In-Lab or Remote
• Sample Size: At least 5 per
segment
UAT: Benefits
• Find out how users will really
use the website
• Figure out if all functionality
and interactivity is useful to
the user (Flash, Videos, AJAX)
• Observe where people may have
difficulty or fail and how they
recover from those errors
UAT: Tips for Success
• User Acceptance Testing is not
Quality Assurance!
• Allow for free exploration with
the user thinking aloud
• Test iteratively
POLL
• Have you added any new content
to your website in the past 6
months (whitepapers, downloads,
videos)?
MVT:
Persona Development
Wireframe Testing
Visual Affordance
Testing
User Acceptance
Testing
Multivariate Testing &
Optimization
Card Sorting
MVT:
How do we make sure that our
site is always usable and that the
user-experience of our visitors is
always optimal?
Continue to test and
optimize your site!
Q:
A
:
MVT: What is Multivariate
Testing?
• Once the website is launched,
Multivariate Testing optimizes
performance of the website by
segmenting visitors and displaying
different images/ buttons/ links/
content to them
• Method: Automated, In-Lab, Remote
• Sample Size: At least 500 visitors
MVT: A/B Testing vs.
Multivariate Testing
A/B MVTLanding Page Version 1 Landing Page Version 2
Views 185,854 187,900
Form Submissions 4,154 5,232
Conversion Rate 2.24% 2.78%
Landing Page Element Views Form Submissions Conversion Rate
1C - Banner [static image] 1,300 517 39.77%
2A - Button ["Buy Now"] 1,287 270 20.98%
3B - Text [black] 1,340 245 18.28%
2C - Button ["Submit"] 1,310 230 17.56%
1B - Banner [AJAX] 1,305 125 9.58%
3A - Text [gray] 1,290 105 8.14%
3C - Text [blue] 1,297 105 8.09%
2B - Button ["Next"] 1,304 99 7.60%
1A - Banner [Flash] 1,289 88 6.83%
MVT: Benefits
• Minimize risk with any website
changes
• Data drives decisions
• Data is collected 24/7
• Tangible Results
• Test New Applications
• Test the ―Third Screen‖
―If ease of use was the only valid
criterion, people would stick to
tricycles and never try bicycles‖Doug Engelbart
Human-Computer Interaction Pioneer
and inventor of the computer mouse
Thank You!Jegan Chen, Usability Engineer
BusinessOnLine
Jegan.Chen@BusinessOL.com
http://www.businessol.com
Additional BusinessOnLine
Resources:
Re:Cognition: BusinessOnLine’s Usability Blog
http://www.businessol.com/usability_blog
Upcoming Webinar:
SEO Webinar Series:
Session 3: Not Just Video—VideoSEO
June 17, 2009 11am PST (2pm EST)
http://www.businessol.com/news/main-webinars/
top related