usability evaluation in exclusive domains_presentation
DESCRIPTION
TRANSCRIPT
Technology for a better society 1
Asbjørn Følstad
1st European workshop on HCI design and evaluation
Limassol, Cyprus, April 8., 2011
Usability evaluation in exclusive domains: How to access domain knowledge
Thoughts in progress!
Conclusions up for discussion
Technology for a better society 2
The two loves of the usability evaluator
Usability testing
Usability inspection
Technology for a better society 3
The two loves of the usability evaluator
Usability testing
Usability inspection considering users
observing users
What do we miss by not accessing the user’s knowledge during our usability evaluations?
Technology for a better society 4
We do not miss much, unless the user …
What do we miss by not accessing the user’s knowledge during our usability evaluations?
… holds domain knowledge we cannot access
Technology for a better society 5
What we may miss if the user knows more of the domain than the usability evaluator:
Review of two studies of usability evaluation methods that allow us to access the users’ knowledge.
Technology for a better society
• Mobile applications for • Medical personnel at hospital wards• Parking wardens• Political advisors
• Evaluated with groups of either• Domain experts• Usability experts
6
Consider: Three cases of Group-based expert walkthrough**) an inspection method allowing domain experts to serve as usability inspectors
12 % overlap in findings between evaluator groups
Domain experts: 59 % domain specific findings
Usability experts: 15 % domain specific findings
Følstad, A., 2007. Work-domain experts as evaluators: usability inspection of domain-specific work-support systems. International Journal of Human– Computer Interaction 22 (3), 217–245.
Technology for a better society
• Mobile applications for • Medical personnel at hospital wards• Parking wardens• Political advisors
• Evaluated with groups of either• Domain experts• Usability experts
7
Consider: Three cases of Group-based expert walkthrough**) an inspection method allowing domain experts to serve as usability inspectors
54 % of domain experts’ findings given high priority by client
27 % of usability experts’ findings given the same priority
Følstad, A., 2007. Work-domain experts as evaluators: usability inspection of domain-specific work-support systems. International Journal of Human– Computer Interaction 22 (3), 217–245.
Technology for a better society
• Mobile applications for • Mobile sales personnel• Medical emergency personnel
• Usability issues identified through • Observation (in the interaction phases)• User dialogue (in the interpretation phases)
8
Consider also: Two cases of cooperative usability testing**) usability testing with phases of interpretative dialogue with the user participant
Findings through user dialogue ...
covered a broader range of themes:-Static design- Interaction design- Needed information- Needed functionality- Req’s for use- Req’s for content- Technical issues
Følstad, A. & Hornbæk, K. (2010) Work-domain knowledge in usability evaluation: Experiences withCooperative Usability Testing. Journal of systems and software, 83, 2019-2030.
Technology for a better society
• Mobile applications for • Mobile sales personnel• Medical emergency personnel
• Usability issues identified through • Observation (in the interaction phases)• User dialogue (in the interpretation phases)
9
Consider also: Two cases of cooperative usability testing**) usability testing with phases of interpretative dialogue with the user participant
Findings through user dialogue ...
had similar impact on subsequent developmentcompared to findings through observation
Følstad, A. & Hornbæk, K. (2010) Work-domain knowledge in usability evaluation: Experiences withCooperative Usability Testing. Journal of systems and software, 83, 2019-2030.
Technology for a better society 10
Why did it pay off to access the users’ knowledge in these cases?
Technology for a better society 11
Domain exclusiveness*
*) The unavailability of domain knowledge to the outsiders of a specific domain
Level of training
Level of specialization
High
High
Low
Low
Specialist(Medical care at hospital wards; medical emergency response)
Generalist(General office work; project leadership)
Popular(eCommerce and eGovernment customership)
Limited training(Professional sales; parking enforcemenet)
Usability testing
Usability inspection
Usabilitytesting
Usabilityinspecti
on
Usability testing
Usability inspection
Technology for a better society 12
In conclusion:
When doing usability evaluation in exclusive domains …
… consider using methods that allow you to access the domain knowledge of users