incorporating pragmatic usability testing into a software test plan carla merrill, ph.d. focused...
Post on 29-Dec-2015
220 Views
Preview:
TRANSCRIPT
Incorporating Pragmatic Usability Testing Into a Software Test Plan
Carla Merrill, Ph.D. Focused Designfocuseddesign.com
merrill@focuseddesign.com
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 2
Presentation Topics
1. How important is usability for a software product?
2. Why is it hard to incorporate usability testing into test plans?
3. A solution through design
4. Integrating the solution into an agile process
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 3
The Importance of Usability
An interactive thought experiment for MS Word users
FIRST:Think of something you liked about the Word. Why do you like it? (because it does what you need)
What if I told you that the code implementing that feature was done in an old
programming language and was inelegant, even sloppy. Knowing that, think
about that feature again. Do you picture the code or the UI? (UI)Do you care about the structure of the code? (no)
SECOND: Think about something you don’t like about the product. Why don’t you like it? (because it doesn’t do what you need)
What if I told you that the code for that part of the product was extremely
elegantly written in a programming language du jour. Would that fact make
you like the feature? (no)
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 4
The Importance of Usability
Paradigm Shift:
1. Users define and evaluate a software product in terms of what they experience: the user interface.
2. Long-term success of a software product depends on how well the user interface meets the needs of target users.
3. Good-quality code doesn’t matter to users if the features it implements don’t meet their needs. Users don’t experience the code.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 5
Perceived Difficulties of IncorporatingUsability Testing into a Software Test Plan
1. Specialized Human Factors skills required.
2. Specialized environments (test labs) required .
3. Appropriate test subjects required.
4. What is to be tested?
5. More project time and money required after standard test phase.
6. Test results and recommended changes are difficult to track and difficult to implement before product release.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 6
Solution: Take Advantage of the Software Design Phase
Require the user interface design process to deliver user scenarios that can be added to test plans.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 7
Required Format for User Scenarios
1. Pragmatic: only scenarios that capture the primary tasks of your target users (contrast with use cases).
2. Created from the user’s perspective with user-based vocabulary that is implementation free.
3. Step-by-step format, limited to 10 steps per scenario.
4. Long tasks are chunked into a series of scenarios that are prioritized by importance to target users.
5. Scenarios are validated with users before testing begins.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 8
Adding Scenarios To Test Plans
1. Tester can define a test procedure for each scenario by adding implementation language.
2. Tester can evaluate success/failure of a scenario by standard test methods: (time, system failure points, UI failure points, number of implementation steps required to complete the scenario).
3. Tester can log bugs for failure points in the test procedure.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 9
Advantages of Incorporating User Scenarios: Usability Can Be Tested!
1. Testers have the skills.
2. Usability tests are clearly defined.
3. Standard test procedures are used.
4. Testing cycle allows usability bugs to be fixed before release.
5. Usability bugs can be triaged appropriately based on priority ratings of scenarios.
6. Usability of the product can be measured using standard testing metrics.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 10
The Goal-Oriented Design Solution (GOD-S)
Note: Slides 11-15 will not be discussed in this presentation. They are included for reference only.
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 11
User Profiling: What do we need to know about users?
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 12
User Profiling: How do we gather the information we need?
Gain access to users
Meet users in the right environment
Prepare for the meetings
Practice Guided Listening™
Create user reports
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 13
Persona and Scenario Development: How do we make effective use of the information we’ve gathered?
The major challenges of UI design “Many users — one user interface” Feature-creep and elastic users Who’s to say what users want?
What is a persona?*
What are scenarios?
A persona with scenarios can help solve the problems of UI design
* Based on the usage of the term coined by Alan Cooper
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 14
Step 1. Construct a Matrix from User Profiles
Step 2. Analyze the Matrix
Step 3. Build Goal-Oriented Scenarios for a Persona
Persona and Scenario Development: How do we create a persona and scenarios?
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 15
User Interface Design: Design the User Interface Based on Goal-Oriented Scenarios
A sound strategy for UI design: Top-down design – follow the user
1 step at a time – focus on enabling scenarios
There is no “right answer” – verify the designwith users
It’s creative
It requires the UI design skillset
It’s iterative and needs scheduled time in the project
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 16
How does GOD work with different development models?
How to merge GOD with a Development Model
Serial Development Model
Agile Development Model
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 17
Serial Development ModelC
om
ple
ten
ess,
Sta
bilit
y
Time
Requirements
Testing
Design
Development
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 18
Agile Development Model ( after Alistair Cockburn, http://alistair.cockburn.us )
Com
ple
ten
ess,
Sta
bilit
y
Time
Requirements
Testing
Design
Development
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 19
Potential Problems of the Merger
Agile Development
• Bottom–up coding – don’t need the UI framework
• Use cases focus on system needs
• Code chunks are pieces isolated from the whole application
• UI chunks designed in context of a UI framework
• UI chunks based on user taskflows
• UI design is always ahead of the code
GOD• Top-down design – a UI framework first
• Scenarios focus on primary needs of a person
• Code chunks are based on programming methodology
• Coding of UI is simpler, less critical, and done last – the “lipstick”
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 20
Potential Synergies of the Merger
Agile Development• Bottom–up coding allows rapid progress, while absorbing thin topcoat
• Use cases can focus on user needs, but care for system needs
• Code chunks allow for iteration, absorbing usability feedback
• Top-down design cares for the Big Picture
• Scenarios keep the focus on user needs
• The Big Picture design can provide a logical scheme for chunking
GOD
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 21
Strategies and Tactics for a Successful Merger
• Build UI design prework into the process
• Create persona/scenarios
• Create UI framework/primary navigation (inch-deep, top-down coding)
• Get buy-in on UI framework from stakeholders
• Create chunks based on UI framework
• Continue inch-deep top-down design and coding in each iteration
• Create user-focused use cases for chunks
• Include usability testing in each iteration and absorb the feedback
• Enable teaming between UI designers and UI coders
• Require UI design realism: designers use most efficient GUI controls
• Bleed UI deficits into the next iteration until UI is complete
Agile Development + GOD
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 22
Timeline for a Successful Agile + GOD Project
Project start Product delivery
High-level business requirements
GOD prework
Schedule/chunking
Team definition
System framework design
Chunking refinement as needed
Agile iterations 1…n
with multiple teams Integration
Usability testingBeta
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 23
Timeline for a Successful Agile + GOD Iteration
V starts End of V’s final QA
GOD usability testing
Reflections mtg
Business reqs
Chunk definition
GOD V prework GOD wireframing
DEV post V cleanup/integration DEV coding
GOD UI deficit list
QA post V bug tracking
Use case development
QA new V test scripting
= code drop
QA testing
Start the next V
= code test
October 2005 Incorporating Pragmatic Usability Testing into a Software Test Plan © 2005 USA by Carla Merrill, Focused Design. 24
Summary
1. Usability testing is important for the success of a software product.
2. Pragmatic usability testing can be incorporated into the standard test plan through user scenarios developed in the design phase.
3. Testing can be done using existing QA resources and bug tracking systems.
4. Goal-Oriented Design supports pragmatic usability testing in an agile software development environment.
top related