making usable software - at agile indy / uxpa joint meeting

Making Usable Software

September 10, 2012Indiana UXPA and AgileIndy

@Carologic

"The biggest waste of all is building something no one wants"

- @ericries #LeanStartupMI in 2011 via @MelBugai

Create a great, usable, accessible, and relevant experience

Workshop

Best Practices

Integrating with AgilePhase

/

Spri

nt

0User ResearchDesign for S1

Spri

nt

1 Design for S2UR for S3

Spri

nt

2 U TestDesign for S3UR for S4

Spri

nt

3 U TestDesign for S4UR for S5

Phase

/

Spri

nt

0Pre-Dev

Spri

nt

1 DevSpri

nt

2 Dev

Spri

nt

3 Dev

Increased understanding of Users

Dev

UX

Agile Integration Use each study to pick up information Additional user research done in

parallel

Spri

nt 0User

ResearchDesign for S1

Spri

nt 1Design for

S2UR for S3 S

pri

nt 2U Test

Design for S3UR for S4

Spri

nt 3U Test


Spri

nt 4U Test


Spri

nt 5U Test


Spri

nt 6U Test


Spri

nt 0Pre-

Dev Spri

nt 1Dev

Spri

nt 2Dev

Spri

nt 3Dev

Spri

nt 4Dev

Spri

nt 5Dev

Spri

nt 6Dev

Increased understanding of Users

Personas

User Observations SurveyInterviews

Dev

UX

Experience Driven Agile:Developing Up to an Experience,

Not Down to a Feature

Rehash of a Webinar by: Kaleb Walton & Brian Anderson

The “Pitch”

Quickly conveys background of problem, proposed solution and statement of value

Shirt-size estimates make for easy prioritization (story points are fine too)

Sprinkle in risk and value to make prioritization even easier

Prioritize dozens of experiences, not hundreds

General format:The problem is <problem>. Imagine if <solution>. This solution would result in <value statement>.

Lightweight precursor to...

Effective Prioritization and Assignment of Work Items

The problem is that systems managers spend too much time prioritizing and assigning their team's daily work efforts. Imagine if Systems Manager Plus offered better prioritization capabilities and automated assignment based on definable business rules. This solution would result in reduced cost for systems managers by enabling more efficient work assignment, leading to better response times.

Copyright © 2012 Kaleb Walton, Brian Anderson, Michael Hughes and Terri Whitt

The “Scenario”

Borrowed from UX discipline Paints a clear picture of an entire experience Extremely versatile and ready for use outside

development Our definition:

“A real-world example of a person's experience with a product, describing context with a problem and a proposed solution.”


Scenarios Are Agile

Minimum Viable Product: What is the minimum experience someone would pay for?

Lightweight: Low cost to develop, flexible and quick to communicate.

Better Contract: More reliable as it's written in terms of Experience rather than Features.

Just Barely Good Enough and Just in Time: Fidelity naturally matches immediate need.

Ya Ain’t Gonna Need It: Does it enable the scenario?


Telling a Story


Example Scenario

EFFECTIVE PRIORITIZATION AND ASSIGNMENT OF WORK ITEMS

PROBLEMMary, a systems manager at ABC Health, is responsible for a team of 12 system administrators who handle steady state support of their health care systems and network. One of her biggest time sinks is prioritizing and assigning her teams daily work efforts. The tool she uses, Systems Manager Plus, doesn't give her any prioritization features except for the ability to sort on a 'priority' field when reviewing work items.As she spends half of her time prioritizing she ends up working over time to tend to her other duties.

SOLUTIONAfter a major update Mary signs into Systems Manager Plus, heads to the work items area and is pleasantly surprised to see a number of new prioritization capabilities. There are more fields available to sort and filter, as well as a “smart assignment” system that enables her to specify rules that will result in automatic assignment to specific members of her team. Mary creates a few rules, applies them to existing work items, and is excited to see that over a quarter of the items were automatically assigned. She proceeds to sort and filter the remaining work items to prioritize and assign to her team. As more work items trickle in she notices that many of them are being auto-assigned.These improvements have enabled Mary to focus less on prioritizing and more on doing.Copyright © 2012 Kaleb Walton, Brian Anderson, Michael Hughes and Terri

Whitt

Easily Pull Out Stories and Epics

Additional sorting capabilitiesAs a systems manager I want to sort work items by additional fields such as created date, severity and platform so that I can more effectively prioritize them.

Additional filtering capabilitiesAs a systems manager I want to filter work items by additional fields such as created date, severity and platform so that I can more effectively prioritize them.Smart assignment system (epic)

As a systems manager I want to specify assignment rules for the system to use to automatically assign work items so that I don't have to assign every work item manually.

Apply new smart assignment rules to existing work itemsAs a systems manager I want to apply new smart assignment rules to existing work items so that I can use smart assignment on work items created after the smart assignment process has executed.Copyright © 2012 Kaleb Walton, Brian Anderson, Michael Hughes and Terri

Whitt

Scenarios Are Agile

Minimum Viable Product: What is the minimum experience someone would pay for?

Lightweight: Low cost to develop, flexible and quick to communicate.

Better Contract: More reliable as it's written in terms of Experience rather than Features.

Just Barely Good Enough and Just in Time: Fidelity naturally matches immediate need.

Ya Ain’t Gonna Need It: Does it enable the scenario?


Basic Experience Driven Agile

Product Backlog Iteration Backlog

Scenarios StoriesPitches

Estimate, ValuateAssess, Prioritize

Estimate

Product Owners,UX Analysts, Architects and Stakeholders

Scrum Masters, Developersand Testers

Involvement Over Time by Role

Pro

duct

Mgt

Act

ivit

ies


Experience Driven Agile At Scale

Portfolio Backlog

Pitches

Estimate, ValuateAssess, Prioritize

Estimate

Product Owners,UX Analysts, Architects and Stakeholders

Scrum Masters, Developersand Testers

Involvement Over Time by Role

Pro

duct

Mgt

Act

ivit

ies Product

BacklogsIterationBacklogs

StoriesScenarios

Scenarios

Prioritize


Contact Us

Kaleb [email protected]

http://experiencedrivenagile.com

Brian [email protected]

Thanks to Other Experience Driven Agile Contributors

Michael Hughes, [email protected]

Terri [email protected]

Usability Testing in Agile Environments

Any Method Can be Adapted Quick Bare minimum of effort Get needed feedback Provide recommendations Repeatable

Scope Effort Consider budget, resources Time

Recruiting Facilitating Analyzing

Adding participants increases budget & time

Paper, Clickable or Real Code? Always start with paper

Guerilla / hallway test Users may misunderstand

Clickable prototypes Easier to understand Can easily change

Real Code Great if it’s the right solution

Paper or Clickable Prototype Rapid Iterative Testing & Evaluation

(RITE) Traditional Testing

In-Person Remote more challenging

Rapid Iterative Testing & Evaluation Qualitative user feedback

actions + comments Series of small usability tests 3 participants each day Minimum of 3 days of testing

Iteration between testing days Total of 5 days

RITE Process

Test Update Test

1

2

3

High

Medium

Low

Priority& Level of

Effort

25

Recap Sessions End of each day - after the last session Room with a whiteboard. About 30 minutes. Discuss:

trends seen concerns recommendations prioritize changes for the next round list lower priority changes for future iterations

26

RITE Results Final prototype

Vetted with users Base for recommendations

Light Report: “Caterpillar to Butterfly” Screenshots show progressions What changes were made and why

What Works for RITE Best used early in project lifecycle

Early concepts Need to be vetted with users Can assist in quickly shaping designs

28

General Testing

Traditional Testing In-Person Remote

Moderated or Unmoderated Less users, shorter sessions: analyze

at lunch Recommend 3 or more users Half hour to 1 hour each

Regular Testing

(Yes, this is an old idea; a great one!)

Bring it On! Small focused tests Reduce waiting for recruitment Once per week or per Sprint Same day mid-week (not Monday or

Friday)

User Testing Day! Make team aware Invite everyone

Watch remotely Recurring meeting invites for

stakeholders

What could I test? Identify what to test at start of Sprint

Work in Progress Multiple projects Prototypes Concepts, rough ideas, brainstorming Competing designs, (A/B testing) Comparative studies across market Conduct interviews to inform research More…

“Teams should stretch to get work into that day’s test

and use the cadence to drive productivity.”

- Jeff Gothelf - http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-agile/

Why Regular? Team becomes:

accustomed to steady stream of qualitative insight

insight ensures quick decisions…line up with business and user goals

Adapted from Jeff Gothelf - http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-agile/

Include PWD People with disabilities

“We are all only temporarily able-bodied. Accessibility is good for us all.”

Get to spirit of the law (Section 508, WCAG 2.0)

-@mollydotcom at #stirtrek 2011 via @carologic

Make it Repeatable

Pre-Book Your Rooms Test & Observation Rooms Any location will do

Conference rooms Offices Quiet corner of cafeteria Remote

Create Reusable Templates Screener Technology use/experience Knowledge of topic

Scripts/Guides Consent Forms Data Collection

Debriefing After Testing

Find Patterns Quickly

Issue P1 P2 P3

Search Used Yes No No

Widget 1 Used N/A Used – unsure about

Task 1 Notes 3 – easy 2 – needed help

3 – easy

Task 2 Notes 2 – needed help

2 – easy 2 – needed help


3 – easy Ran out of time


3 – easy Ran out of time

True Statements All interfaces have usability problems Limited resources to fix them More problems than resources Less serious problems distract Intense focus on fixing most serious

problems first

Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. By Steve Krug

Debrief with Team Assumes stakeholders watched tests

If not, wait for UX analysis Quick analysis to quick decisions All decision makers MUST be present

Goal Identify top 5 or 10 most serious

issues Top 3 from each list Prioritize from lists Commit resources for next sprint Stop

Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. By Steve Krug

Guidelines Stay on Topic Be Constructive Don’t get distracted by small

problems Intense focus on fixing most serious

problems first

Make Useful & Usable Recommendations - Quickly

Transform Data Look for patterns Read “between the lines” Know what you’ve got

Sort, reorganize, review, repeat What refutes your expectations? Surprises? Outliers?

Short and Direct Communication Email or One Pager

Think about audience How will it be used?

Include Goal of study What will be fixed and who assigned to Tasks attempted Who observed Future research/enhancements

Tweak, Don’t Redesign Small iterative changes

Make it better now Don’t break something else

Take something away Reduce distractions Don’t add – question it

Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. By Steve Krug

Do UX Early & Often Make users visible Information radiators

Test findings Artifacts Personas Word Clouds - IA

Recommended Readings

51

Gothelf, Jeff. Lean UX: Getting Out of the Deliverables Business. (Anticipated in Feb. 2013)

Contact Carol @Carologic

Email: [email protected]

SlideShare.net/Carologic

SpeakerRate.com/speakers/15585-CarolJSmith

Tool Considerations• In-person or remote? • Lab or on-site?• Prototype limitations (can it be online?, is it a

document or a clickable site?)• Number of observers, number of participants?• Number of facilitators?• Logging and video editing needs (time on task,

highlight video creation)?• Surveys before or after?• Eye tracking?

Usability Testing Software• Morae • Ovo• SilverBack (Mac only)• UserWorks• Noldus• Tobii (Eye-tracker)• SMI (Eye-tracker)• SurveyMonkey

Screen Sharing Software GoToMeeting –

http://www.gotomeeting.com Lotus Sametime Unyte –

http://www.unyte.com YuuGuu -- http://www.yuuguu.com WebEx – http://www.webex.com Yugma -- https://www.yugma.com/

Trouble Shooting: CoPilot - https://www.copilot.com/

Recommended Sites Usability.gov W3C Web Accessibility Initiative

http://www.w3.org/WAI/ Accessibility Standards in US (Section 508)

http://www.access-board.gov/sec508/508standards.htm Jakob Nielsen

http://www.useit.com UPA – professional UX association

http://www.upainternational.org/

References Albert, Bill, Tom Tullis, and Donna Tedesco. Beyond the Usability

Lab. Beyer, Hugh. User-Centered Agile Methods (Synthesis Lectures on

Human-Centered Informatics) Gothelf , Jeff. http://blog.usabilla.com/5-effective-ways-for-usability-

testing-to-play-nice-with-agile/ Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability

Testing in Diverse Situations. Tutorial, 2003 UPA Conference. Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to

Finding and Fixing Usability Problems. Ratcliffe, Lindsay and Marc McNeill. Agile Experience Design: A

Digital Designer's Guide to Agile, Lean, and Continuous. Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing:

How to Plan, Design, and Conduct Effective Tests. John Wiley & Sons, Inc.