paper prototype evaluation
Post on 01-Nov-2014
2.241 Views
Preview:
DESCRIPTION
TRANSCRIPT
Paper Prototype Evaluation
Evaluation
• Well, before addressing the evaluation of paper prototypes, perhaps we should consider reasoning upon what we should evaluate, right?– Let’s then take a look into concepts such as
usefulness, usability and user experience
Usefulness
• Usefulness is a crucial quality of any product or service– Concerns the degree to which a product enables a
user to achieve his or her goals, and is an assessment of the user’s willingness to use the product or service at all• Without usefulness, other measures make little or no
sense, because it will just sit on the shelf
• We name this the fundamental quality of a product or service
Usability
• According to the ISO standard– Usability is the extent to which a product or
service can be used by specific users to achieve predefined goals in a specified context of use
– The standard defines three main usability dimensions:• Effectiveness,• Efficiency, and• Satisfaction
Usability
• Anyway, when looking into usability, we normally account for:– Effectiveness– Efficiency– Satisfaction – Learnability – Memorability
• We name these the pragmatic qualities of a product or service
Usability
• Common reason for the delivery of less usable products are– System focused process– Poorly integrated teams– Design and development mismatches– And of course, designing usable products is
difficult• That’s what we are pushing paper prototyping in this
moment of the process
User Experience
• According to its ISO standard, user experience is a person’s perception and responses that result from the use or anticipated use of a product or service
User Experience
• User experience subsumes usability and includes includes– emotions, beliefs, preferences, perceptions,
physical and psychological responses, behaviors and accomplishments that occur before, during and after use
• We name these the effected of the hedonic, or pleasure related, qualities of a product or service
Evaluation
• As stating in the beginning, before addressing the evaluation of paper prototypes, we should consider reasoning upon what we should evaluate– And now that we briefly cover usefulness, usability
and user experience, I believe it’s fair to say that we should target going as far as usability
Evaluation
• In general, product and service evaluation can be formative or summative
• Paper prototype evaluation is mostly formative
Evaluation
• Paper prototype evaluation focus on…– The most significant issues preventing users from
accomplishing their goals– What works and what do users find frustrating– What are the most common errors or mistakes
users are making– Assessing the improvements being made from one
design interaction to the next– Identifying issues that are expected to remain even
when the product is launched
Evaluation
• Today we look into– Inspection methods• These are methods where an expert evaluator inspects
a product or service
– Testing methods• These are methods where products and services are
evaluated by testing them on real users
Inspection methods
• There are also several but we will address:– Cognitive walk-through– Heuristic evaluation
Cognitive walk-through
• The purpose is to verify if the paper prototype actually allows the fulfillment of the selected user stories– This is a within team activity that ensures that
your prototype complies with the identified user stories
Cognitive walk-through
• Designers and developers of the product or service then walk through the steps as a group, asking themselves a set of questions at each step– Data is gathered during the walk-through, and
afterwards a report of potential issues is compiled– Finally the evaluated proposition is redesigned to
address the issues identified
Heuristic evaluation
• This is a type of evaluation ideally carried out by an expert.– It specifically involves evaluators examining the
design and judging its compliance with recognized principles, such as Jakob Nielsen’s 10 usability heuristics• These evaluation methods are now widely taught and
practiced in the new media sector, where products and services are often designed in a short space of time on a budget that may restrict the amount of money available to provide for other types of interface testing
Testing methods
• There are several but we will address:– Co-discovery– Wizard of Oz
Co-discovery
• Two users attempt to perform tasks together while being observed– They are to help each other in the same manner
as they would if they were working together to accomplish a common goal using the product
– They are encouraged to explain what they are thinking about while working on the tasks
Co-discovery
• The designers and developers should refrain from explaining the design decisions and rather focus on getting the most of the pair of users tacking with your prototype– Note taking is fundamental and you should run
this kind of test until no significant additional information is feed back into the design process
Wizard of OZ
• This is a testing approach built upon a paper device instead of using a working technological artifact– This king of testing involves systematic observation
under controlled conditions to determine how well people can use a product or service
– Rather than showing users a rough draft and asking, Do you understand this?, this kind of testing involves watching people trying to use something for its intended purpose
Wizard of OZ
• Setting up such a test involves asking the test subjects to recreate a set of user stories after being introduced to the underlying scenario– For example, to test the attachment function of an
e-mail program, a scenario would describe a situation where a person needs to send an e-mail attachment, and ask him or her to undertake this task
– The aim is to observe how people function in a realistic manner, so that developers can see problem areas, and what people like.
Testing methods
• This kind of evaluation should be repeated until no significant added value comes from bring in an additional subject– The usual number is 5, but other claim otherwise
Pre and post-tests
• When applying testing methods, pre-test and post-test questionnaires are also used to gather feedback on the product being tested– A common questionnaire is the 25 years old
System Usability Scale
System Usability Scale
Evaluation protocol
• But these methods are useless without an adequate evaluation protocol
• While designing your evaluation protocol, you should take into account…– A mixed selection of inspection and testing
methods;– The user stories supported by your prototype; and– The affordances of your paper prototype.
Evaluation protocol
• Examples are provided in the companion blog but please note that…– In all cases, evaluation protocols should be piloted
to ensure that once they are being applied, you are actually focusing in assessing the prototype and not on solving evaluation protocol issues
Now it’s up to you to make it happen
paperprototypingateia.wordpress.com
top related