rich guntherjeff janisscott butler the ucd decision tool

Rich Gunther Jeff Janis Scott Butler

The UCD Decision Tool

www.ovostudios.com/upa2001

Abstract

• Present background and results from a survey of 100 usability practitioners

• Identify trends in the state-of-the-practice

• Introduce you to an online summary of data you can use (www.ovostudios.com/upa2001).

Gunther, Janis, & Butler – UPA 2001

Audience Expectations

What methods should I learn first?

How can I be more effective during development?

How can I better demonstrate return on investment?

How can I structure and position my team?

How can I influence …you sell UCD and what push-back can you expect?


Inspiration for This Presentation

• Recognized the wealth of knowledge that lies latent in our community of practitioners

• Wanted to learn how practitioners address day-to-day challenges: – where we position UCD Tools and Processes

within the Product Development Lifecycle– how we communicate with our internal and

external customers, build teams, and structure processes

– how this varies based on demographic information


This Presentation Is…

• An opportunity to learn how 100 of your colleagues structure their teams and services

• A discussion of the interaction between product development lifecycles and user-centered design

• A forum for usability practitioners to talk about their successful and not-so-successful experiences engaging with development teams.


This Presentation Is Not…

• An in-depth tutorial on UCD methodologies

• A criticism of:– Consultants – Corporate practitioners – Marketing departments– Developers– Management


Usability’s Challenges


The Lessons of the Past

• “The way in which we deliver the message must be pointed and unambiguous… we must deliver punch and clarity”. (Kreitzberg and Shneiderman)

• “Competing claims about usability, …ambiguous terminology,…competing paradigms,… and limited audience interest” (Paul McInerney)

• We need to engage with development teams from the beginning through the end of design. (Darnell and Halgren)


Examples from Practitioners

• “I’m looking for a consultant who is not just going to usability test the product to death.”

• “I don’t know why Jakob says to do this… ask him.”

• “Yeah, our company wants to do usability.”


Examples from Developers

• “You guys make our products easier to use… what else do you do?”

• “That usability team! All they care about is our customers!”

• “Did you make this up, because if a user said that, then they don’t know what they are talking about!”


Example from Dilbert, 1

(Insert bridge here.)


Example from Dilbert, 2

(Insert bridge here.)


Audience Experience

• What challenges can audience members add?


So, What Are We Doing about All This?

Lessons of the Past: are we following them?

Practitioner Quotes: what are we doing to adjust the state-of-the-practice (if it needs adjusting)?

Developer Quotes: what are we doing to correct the misperceptions?

Dilbert: if comics reflect our dynamics, there must be a problem.


Methodology


Goal of the Survey

By surveying 100 usability practitioners about their team composition and mission, company demographic, and the successes and failures they had implementing UCD processes, we hoped to get an idea of which variables influenced their ultimate level of success or failure.


Question mapping

• Company demographic – size, dev type

Survey Collection Method

• Constructed 10-question internet survey with single-selection, multi-selection, and free form text input

• Requested participation from UTEST, HFES, SIGCHI, and the STC mailing lists

• Posted the survey for roughly two weeks• Took it offline when responses reached the

100-response mark.


Analysis Method

• Collated responses into an Excel spreadsheet• Calculated overall results for each question• Personalized results by organization size,

practitioner type, team size, and team tenure • Investigated trends in mission statements,

UCD method positioning, and follow-up activities.


Results


Respondent Profile

The following profile information is presented for respondents:

1. Organization type

2. Company size

3. Usability group tenure

4. Team size


Organization Type

Most of us work with developers who are internal to our own company.


Company Size

The majority works for companies with less than 3500 employees.


Usability Group Tenure

The majority works in a a team less than 5 years old.


Team Size

Mean = 6.4Median = 4Mode = 1Min = 1Max = 74


Team Size

18

16

910 10

8

34

2

16

2

0

5

10

15

20

1 2 3 4 5 6 7 8 9 10+VariesNumber of Team Members

N

Tracks


Three Tracks

• Beginner/Intermediate: focus on skill and career development or change

• Manager: focus on building a team

• Influencer: focus on usability strategy, influencing, deployment


Beginner/Intermediate Track

1. What UCD skills should I learn first?

2. What UCD skills should I learn next?

3. What if I want to move beyond usability?


What UCD Skills Should I Learn First?

• Respondents indicated the following activities were their most successful:

– Usability testing (39%)– Paper or other prototyping (19%)– Heuristic evaluation (10%).

• Best selling activities across the development lifecycle were:

– Customer interviews (Ph.1, 46%)– Paper or other prototyping (Ph.2, 65%)– Usability test (Ph.3, 41% and Ph.4, 36%).


What UCD Skills ShouldI Learn Next?

More than 60% of respondents:

– Usability test (71%)

– Paper or other prototypes (65%)

– Heuristic review (65%)

More than 40% of respondents:

– Cust Intvws (56%)– GUI Spec’s (47%)– Use cases (46%)– UCD Sessions

(45%)– Vision/Scope

documents (41%)


Moving Beyond Usability

• Let’s say you want to re-tool for another role on your team. What roles are most common?– Usability specialist (86%)– UI designer (61%)– Developer (33%)– Information developer (32%)– Graphic artist (21%)


Questions?


Manager Track

1. What is a typical mission for a team like mine?

2. Who should be on my team?

3. What UCD activities should we offer?

4. What processes and procedures should we have in place?


What is a Typical Mission for a Team Like Mine?

• An item analysis of mission statements indicated that:– 61% conduct miscellaneous UCD activities– 37% impact product design– 22% are tasked with making things better.

• 52% of teams have well-defined missions or team responsibilities.


Who Should be onMy Team?

• UCD teams have team members in the following roles according to a sample of 100 survey respondents:– Usability specialist (86%)– UI designer (61%)– Developer (33%)– Information developer (32%)– Graphic artist (21%).


What UCD Activities Should We Offer?

• Your peers’ most successful activities were:– Usability testing (39%)– Paper or other prototyping (19%)– Heuristic reviews (10%).

• Over 60% of respondents said their teams performed the following activities:– Usability tests (71%)– Paper prototypes (68%)– Heuristic reviews (65%).


What Processes & Procedures Do I Need?

• A majority of respondents had the following documentation in place:– Usability activity deliverables (71%) – Roles and responsibilities (55%) – Usability activity timeframes (53%).

• A large proportion also had:– Design Standards (45%)– Follow-up expectations (40%).


Audience Brainstorm

• Any other manager issues?


Influencers Track

1. What level of developer resistance is typical?

2. What kind of developer resistance can I expect in different phases of engagement?

3. What development lifecycle phases can I sell into?

4. Does life get easier if we engage early in the lifecycle?

5. How can I demonstrate to management that my group is impacting product?


What Level of Developer Resistance is Typical?


• 83 respondents reported resistance of some type during interactions with developers.

• 4 respondents explicitly said that they had no problems.

What Kind of Resistance Can I Expect?


•When engaging– General resistance– Usability unawareness.

•When working with– General resistance– Usability unawareness.

•When following-up– General resistance– Implementation challenges.

What Development Lifecycle Phases Can I Sell Into?

All of them.

Best Seller Runner-UpReq’ts Cust.Interview Use CasesDesign Paper Prototype Usability TestDev’t Usability Test Heur.ReviewTest Usability Test Heur.Review


Does Life Get Easier If We Engage Earlier?

Problems reported by more than 20% of respondents were:• General resistance (engage, work, follow-up)• Usability unawareness (engage, work)• Implementation challenges (follow-up).


How Can I DemonstrateImpact on Product?

• 82 (n) hold follow-up meetings• 62 (n) document resolutions made in follow-

up meetings• 26 (n) track implementation rates (IR).

– Report IR to R&D managers on a quarterly basis– Report IR to Quality Management– Input usability defects in our DCR then query for

IR.


Audience Brainstorm

• Any other influencer issues?


Web Site Tour



Wrap-Up


Suggestions

• Use the principle of “punch and clarity” to guide activities from engagement, through working with, and following-up.

• Counteract “competing paradigms” with clear mission statements and procedures.

• Learn new UCD methods so you can engage throughout the development lifecycle.


Future Directions

• - Teams with a clearly defined mission statement, as well as documented roles and responsibilities, will have significantly less trouble engaging with developers.

• - Teams who position their UCD methods appropriately in the product development lifecycle will have significantly less trouble working with developers.

- Teams that track the implementation of design recommendations and document the resolution of these recommendations will have significantly less trouble following up with developers.


Rich Gunther Jeff Janis Scott Butler


rich guntherjeff janisscott butler the ucd decision tool

Documents

butler upa

halgren gunther

usability practitioners

usabilitys challenges

development teams

usability team

practitioners im

community of practitioners