rumm_handout_or__final__paper.ppt
Post on 01-Nov-2014
280 Views
Preview:
DESCRIPTION
TRANSCRIPT
Information Architecture Summit 2004 Uzanto Consulting 1
Rapid User Mental Modeling (RUMM)at eBay.com: A Case Study
Jonathan BoutelleRashmi Sinha
& Kirsten Swearingen
Uzanto Consulting
Larry Cornett&
Ido Dan
eBay, Inc.
Information Architecture Summit 2004 Uzanto Consulting 2
Background to Information Architecture Redesign of eBay.com eBay website had grown in organic fashion, leading to a
somewhat inconsistent user experience Although much design and usability work is done on site itself,
this project was first effort to delineate user mental models for the entire website.
Basic eBay facts World’s largest online marketplace 16 million listings on any given day (as of Q2, 2003) Almost 34 million active users worldwide
Buyers and sellers New and experienced Occasional and full-time Individuals and corporations Different countries and cultures
Information Architecture Summit 2004 Uzanto Consulting 3
Challenges
New IA would need to: Support site’s vision of global economic democracy Serve a large and diverse user population with varying goals and
mental models Be future-oriented, allowing for planned change and growth Occur through evolution not revolution -- must accommodate
current users who are very used to the site as it exists today Senior management at eBay needed to see proof of the
need for a drastic restructuring. First phase of project (background research) to be
completed in only 6 weeks
Information Architecture Summit 2004 Uzanto Consulting 4
Overview of Design Research Methodology
Stage 1: Explore
Interviews and free-listing with users and stakeholders
Stage 3: Verify / Refine
Large-sample closed card-sorting study
Rapid User Mental Modeling
Stakeholders
User mental models
Preliminary IA
Stage 2: Understand
Open card-sorting study
site.com
TopCategory 3
TopCategory 1
TopCategory 2
Sub cat1
SubCat2
SubCat3
xx
SubCat1
Subcat4
Subcat3
Subcat2
xx
xx
xxx
SubCat3
SubCat5
SubCat4
SubCat6
SubCat4
xx
xxx
xx
xxx
xxx
xxx
SubCat4
SubCat5
SubCat1
SubCat2
xxx
xx
xx
Verified categorization
Top-level
categories
Scope of content &
functionality
Information Architecture Summit 2004 Uzanto Consulting 5
Our Approach to Design Research: Rapid User Mental Modeling (RUMM) Rigorous, three-stage design research methodology to lay
foundation for IA redesign Bring together established techniques in a novel way
Three stages: Exploration Understanding Verification Individual components complement and build upon each other
Process moves from open-ended exploration to focused information-gathering and hypothesis testing
Addresses individual and group differences in a systematic way Produces reliable results (can be reproduced) Uses mix of qualitative and quantitative methods
Qualitative: rich, open-ended understanding of users and domain Quantitative: generates recommendations firmly grounded in data (can
help make case to management)
Information Architecture Summit 2004 Uzanto Consulting 6
Towards Persuasive, Defensible Design Research Important to create not just good design, but
successful design: Serves user Satisfies the business goals Is accepted by management
Results of design research phase must provide rich picture of user, but also be persuasive and defensible. Defensible: firmly grounded in data, can stand up to
scrutiny Persuasive: persuades stakeholders that your solution
makes sense and will provide benefit to the business
Information Architecture Summit 2004 Uzanto Consulting 7
Stage 1: Explore Domain Through Stakeholder Interviews Goals:
Explore domain in open-ended fashion Identify both current and planned content and
functionality Method:
Interviews with eBay users about specific incidents of site usage
Interviews with stakeholders: top management, members of design team etc.
Identify problems, constraints, and specific plans for future
Generate list of current tasks and “Horizon Tasks” (tasks planned for future)
Stakeholder analysis Task list development
Information Architecture Summit 2004 Uzanto Consulting 8
Stakeholder Interviews
Spoke to 16 eBay managers and designers Asked 3 questions:
What do you imagine a user doing on the eBay site two to five years from now that they can’t do today?
What one problem, if it were solved today, would cause a large increase in the number of transactions executed on the eBay platform?
What user population is least well served by the current site?
Information Architecture Summit 2004 Uzanto Consulting 9
Purpose of Stakeholder Analysis
Gain valuable perspectives for design research: Insure that design aligns with business strategy. Tap into a company’s existing knowledge in a
systematic fashion. Ensure success of project by getting input at the
research phase from those who have ability to say yes or no.
Helps stakeholders feel involved in and understand the design research and process.
Generate task list: helps map the scope and boundaries of the information domain
Information Architecture Summit 2004 Uzanto Consulting 10
Generating the Task List Using Free-Listing Decided not to conduct inventory of site
Incredibly time-consuming Might tie new IA too closely to current site’s design
Focused on tasks, since all eBay content is in context of completing tasks:
Pay for an item List an item
Generated initial list of 300 tasks by using free-listing exercises with:
Users Designers Stakeholders (especially important for identifying future tasks)
Information Architecture Summit 2004 Uzanto Consulting 11
The Free-Listing Exercise This simple, semi-structured method helps:
Determine the scope of a domain Gain some insight into how the domain is structured
Example: “List all the animals you can think of” Results:
Indices of salience: Frequency Rank
Indirect indices of similarity: Co-occurrence. How often are cat and dog both mentioned? Average rank distance. Are cat and dog mentioned together
more closely than cat and horse? In this case, we asked “List all the tasks at eBay.com a
person can do / might be able to do in the future.”
Information Architecture Summit 2004 Uzanto Consulting 12
Refining the Task List
Consolidated task list, removing redundant items Divided tasks into groups:
Current Tasks (can be done on existing eBay website): Core Tasks form core functionality of website Boundary Tasks mark boundary of functionality offered
Horizon Tasks Might be done on future versions of website Important for creating a future-oriented IA
Selected representative items from each group Result: comprehensive list of 100 tasks marking scope
and boundary of what users can accomplish on the eBay site.
We were ready to begin studying user mental models!
Information Architecture Summit 2004 Uzanto Consulting 13
Stage 2: Understand User Mental Models
Goals: Gain further insight into user mental models and identify top-level categories for site content and functionality
Method: open card sorting Used list of 100 tasks (including 20 Horizon
Tasks) generated in previous stage. 28 participants, representing the main types of
eBay users (sellers and buyers; new and advanced users).
22 completed card-sorts individually 6 worked as a group to create a structure (mix of
buyers, sellers, new, and advanced)
Information Architecture Summit 2004 Uzanto Consulting 14
Identifying the Top Levels of the Hierarchy Cluster analysis used to generate
aggregate mental maps. These were directly used to create
the hierarchical structure. Looked at mental maps for four
major segments. Identified the inconsistencies between
the mental maps of different groups. Understood how to reconcile those to
create a new map. Result: preliminary IA, including 5
top-level categories that accounted for all site content and functionality.
0.000
0.200
0.400
0.600
0.800
1.000
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact sellerContact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task XTask B
Task ZTask N
Dendrogram- User group !
0.000
0.200
0.400
0.600
0.800
1.000
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact sellerContact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task XTask B
Task ZTask N
0.000
0.200
0.400
0.600
0.800
1.000
0.000
0.200
0.400
0.600
0.800
1.000
0.000
0.200
0.400
0.600
0.800
1.000
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact sellerContact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task XTask B
Task ZTask N
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact sellerContact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task XTask B
Task ZTask N
Dendrogram- User group !
Aggregate maps from cluster analysis
Information Architecture Summit 2004 Uzanto Consulting 15
Stage 3: Verify and Refine 5-Category Scheme Goals:
Verify that the five top-level groupings cover all 100 tasks/concepts.
Identify tasks/concepts that do not fit in, as well as where users do expect to find such information.
Examine differences in groupings among various user groups.
Further develop IA for eBay site. Establish method for future verification and refinement
of eBay IA.
Information Architecture Summit 2004 Uzanto Consulting 16
Stage 3: Verify and Refine 5-Category Scheme Method: closed card sorting
Large sample study with more than 1,000 participants representing all segments of eBay users
Study conducted online. Participants asked to categorize tasks to one of the five
top-level categories (or to select Other if it didn’t belong)
Result: site-map showing the structure for the main parts of the eBay site.
Information Architecture Summit 2004 Uzanto Consulting 17
Analysis of Closed Card Sort Results
Category-belonging scores: Percent of users who think a task belongs to a particular category Lack of consensus or high scores in “Other” category may signal a
design issue
Category Memberships across all participants
Task NCategory 1
Category 2
Category 3
Category 4
Category 5
Category 6
Task 1 870 48% 7% 21% 14% 3% 6%
Task 2 991 8% 1% 85% 1% 2% 3%
Task 3 959 7% 0% 86% 2% 1% 4%
Task 4 918 0% 0% 87% 3% 6% 4%
Task 5 929 0% 2% 83% 6% 6% 2%
Task 6 896 46% 5% 7% 10% 27% 5%
Task 7 908 55% 4% 15% 9% 11% 6%
Task 8 893 35% 2% 49% 1% 1% 12%
Task 9 870 1% 1% 67% 1% 3% 27%
Task 10 993 76% 1% 19% 0% 1% 4%
A color-coded visualization of closed card sorting data
Information Architecture Summit 2004 Uzanto Consulting 18
Reconciling Different Mental Models
Results suggest structural possibilities--they do not prescribe one solution
In both open and closed card sorting, found differences in mental models Reconcile through design, if possible (revisit
dendograms) The most important, contentious issues may ultimately
require a decision driven by company brand and business goals (revisit stakeholder analysis)
Information Architecture Summit 2004 Uzanto Consulting 19
Project Summary
Three-stage RUMM methodology used to: Understand business context Delineate hierarchical structure of
user mental models Final deliverable: site-map
showing structure for main parts of entire eBay website. This map is currently being used to
guide the stages of the IA redesign.
site.com
TopCategory 3
TopCategory1
TopCategory 2
TopCategory 4
SubCat1Sub cat1
SubCat2
xx
SubCat3
xx
xx
SubCat2
SubCat3
SubCat5
xx
xx
xx
xx
SubCat1
Subcat4
Subcat3
Subcat2
xx
xx
xxx
SubCat3
SubCat5
SubCat4
SubCat6
SubCat4
xxx
xx
xx
xxx
xx
xxx
xxx
xxx
SubCat7
SubCat4
SubCat5
SubCat6
SubCat1
SubCat2
xx
xx
xxx
xx
xxx
xx
xx
SubCat4
xx
Preliminary IA
Information Architecture Summit 2004 Uzanto Consulting 20
Impact of Project Helped convince management that current site did not
match user mental models and that IA redesign was needed.
Provided basis for a long-term redesign plan to implement a structured site-wide model.
Provided immediate guidance to near-term projects in order to begin moving the site in the long-term direction.
Broadened eBay designers’ perspective on overall design direction and goals by providing:
Access to user mental models Site-map for all content and functionality of eBay.com
Cleared the road towards a more consistent and unified user experience.
top related