usability testing teppo räisänen teraisan/ [email protected]
TRANSCRIPT
General Information
A person belonging to product’s targeting group commits testing
Either finished product of prototype can be used
Observations made during testing session are used in improving usability and UI
General Information
Often usability tests are used during iterative stages of product development
If prototypes are to be used, the appropriate moment must be chosen: Testing with too crude prototypes is not
sensible Too late testing will often lead to
increased development costs
General Information
There are times, when an UI of earlier product can be used as a point of comparison
The aim is often set at surpassing some qualities of the earlier product
In testing it is important to use formal methods instead of ad hoc reasoning
Testing Vs. Heuristic Evaluation
Heuristics and other methods are not exclusionary but complementary
Problems that are revealed by testing or by heuristics are often of different kind
To get the most out of product, several methods are often used
Challenges of Testing
Conditions in usability laboratories are not completely realistic
Even field tests have some differences between real usage situations
When a person knows he/she is observed, it often haves an effect in person’s behaviour
Challenges of Testing
A Hawthorne effect The productivity of a factory had been
decreasing An observer was reqruited to find out
reasons behind decrease Productivity rose back to former level
without any changes to factory’s procedures
Challenges of Testing
Finding suitable test person can be difficult
Often knowledge about presumed end users of a product can be faulty
Marketing researches are used to find out correct user groups, but results of researches may be misleading
Challenges of Testing
Heuristics based on a few experts’ opinions are more cost effective than usability sessions
In a long run it is still worthwhile to use both methods for gathering information
Preparation for Testing
Preparation is a demanding task consisting of many subparts: Test persons are chosen Area of focus is decided Testing environment is prepared
Often a pilot test is conducted to try out the arrangements
Preparation for Testing Testing should follow a precise plan
(like all software testing) Test plan provides a common
guideline for persons involved in testing
The quality of testing process is evaluated at some later point using Test plan Test reports
Preparation for Testing
Test persons are chosen based on some predefined criteria such as Age group Sex ratio Educational background (Computer user experience) Number of users
Example of User Data
AGE DISTRIBUTION15 – 25 5 %25 – 35 45 %35 – 45 30 %45 – 55 15 %55 - 5 %
Preparation for Testing If resources available for testing
are limited, test persons should be representatives of central target groups
With more resources the profile can be expanded
Correct definition of end users is especially important when product is of a new kind
Preparation for Testing
Often test persons are instructed to perform a set of tasks
Those tasks must be defined for tests to be consistent Tasks must be defined in test plan
Often tasks used are the most common ones during product’s intented usage
Preparation for Testing
In connection with tasks numerical values can be used for defining wanted qualities, e.g. ”User must be able to remove a line from document and save edits in less than 30 seconds”
Often earlier products or products of competitors can be helpful in defining limiting values
Preparation for Testing
Test lab must be prepared Installation of software and hardware Other elements of infrastructure Field tests are especially demanding
(electricity, batteries, lightning etc.) A pilot test should be conducted to
avoid problems during actual test sessions
Completion of Testing
Test situation should be as natural as possible
Normally field tests are easier in this respect
Usability labs should resemble the actual using environments as much as possible
Completion of Testing
Some products are impossible to test out in lab conditions leaving field tests for only possibility
Secretiveness should be avoided in test situation Tell the test person about test lab’s
facilities Tell about usability persons’ roles
Completion of Testing
After initial procedures the actual test is completed User performs a given set of tasks User’s actions are recorded User should be able to complete tasks
without any outside help If, however, user gets stuck he/she
must be given some advice
Completion of Testing Actions of observators should be
as unnoticeable as possible When measuring discrete values it
is important that test are conducted exactly in the same way for each of the test persons
A thorough test plan is therefore a necessity
Completion of Testing
To provide equal test conditions Test sessions should be completed
within short period of time Same personnel should be used for all
of the sessions Instructions should be given in written
form Test plan should be carefully followed
and reports written in an uniform way
Completion of Testing
If something goes wrong during testing the case should be carefully documented
Afterwards the value of test data collected during faulty test session must be evaluated
After completion of tasks test
persons can be interviewed to gather further information
Often test persons are rewarded in some way
Note: Legistlation &/ bureaucracy may prohibit using money as a form of reward