The relevance of UX models and measures

I-UxSED 2010

Asbjørn FølstadSINTEF

Can simple rating scales and commenting mechanisms improve the relevance of UX measurement?

Some usability components of the early 90’s

Efficiency

Helpfulness

Affect

Control

Learnability

Satisfaction

Memorability

Error rate

Effectiveness

Usability component examples from J. Nielsen (Usbility engineering) and SUMI

Complex subjective usability measures

Example: SUMI• Software usability measurement inventory

• Questionnaire to be filled out by users

• Covers five usability components• Efficiency• Affect• Helpfulness• Control• Learnability

• Respondent to answer 50 (!) items

SUMI questionnaire: http://sumi.ucc.ie/en/

A consequence of complex models and measures?

[T]he measurement of satisfaction seems in a state of disarray […] The diversity of words used in five-point or seven point semantic differentials or Likert-type rating scales is simply astonishing […] Of the 112 studies that measure satisfaction […] only twelve studies employ standardized questionnaires.

Could the same happen to UX models and measures?

Flow

Engagement

Motivation

Trust

Beauty

Goodness

Hedonic quality

Pragmatic quality

Affect

(The growing number of proposed UX components may suggest so)

”UX-measures” in the real world

Data on users’ experience

collected through a rating

scale and a free text field

”UX-measures” in the real world

Data on users’ experience

collected through a rating

scale and a free text field

”UX-measures” in the real world

Data on users’ experience

collected through a rating

scale and a free text field

Suggestion: Maybe HCI researchers could study and use ...

Simple UX measures and ad-hoc models

Rate(single scale)

Users:

Usability professional:

Explain(free text)

AnalyzeAd-hoc model

with key aspects

Case example:

User feedback on concepts through a simple rating scale and a free text field

Design brief: New concepts for textbased communication on mobile phones

Concepts by J.O. Eikenes, L.M. Vedeler andT. Tveterås, the Oslo School of Architecture and Design

Case example:

User feedback on concepts through a simple rating scale and a free text field

Design brief: New concepts for textbased communication on mobile phones

Concepts by J.O. Eikenes, L.M. Vedeler andT. Tveterås, the Oslo School of Architecture and Design

MESSAGES AT LOCATIONS

Case example:

User feedback on concepts through a simple rating scale and a free text field

User rating and free text feedback indicated potential UX issues

Ad-hoc model components in this particular case could include:- Privacy- Reliability- Utility

Theoretical implicationsFrom the field of Marketing• Bergkvist & Rossiter (2007)

• Multi-item measures de-facto standard in research

• Following Churchill (1979)

• Practitioners use single-item measures

• Empirical trial of multi-item vs. single-item measures inducate no difference in predictive validity

• … when measuring concrete and singular objects

• Attitude toward ad• Attitide toward brand

Could UX be seen as concrete and singular?(The spread of single-item measures in the real world may indicate so)

And could ad-hoc UX model components be seen as the user’s rationalization of their UX?(Thereby indicating relevant issues for redesign)

So – what is your UX of this presentation?

What do you like / dislike?

Submit