Carmen Gerea Valeria Herskovic

November 2015

Measuring User Experience in Latin America: An Exploratory Survey.

• Introduction • Relative work • Our previous related work and motivation • Study • Conclusions

Index

• UX - phenomenon, field of study and field of practice [7] • Challenges of developing HCI and UX communities [8] • Many UX methods [10] • UX - still not clearly defined nor understood [4]

Introduction Some concerns about UX

• Finland, USA, UK, Netherlands, 2008 / 2009: Country of residence and other sociocultural factors could explain some of the variability of responses [5].

• 2014, 97% participants from Europe, North America, Asia. Survey was in English, French and German [3].

• 2014, UX Measurement Attitudes Survey (UXMAS) [4].

Related work at an international level Small participation from LATAM

Our previous related work and motivation [Master´s Degree Project. Nov 2013 - Jan 2014]

Research questions about UX practitioners

RQ1: “what are the most important difficulties they encounter in their UX projects”.

RQ2: “what measurement and validation methods do they use when designing or launching a new system”.

Previous related work and motivation [Master´s Degree Project. Nov 2013 - Jan 2014]

Interview [6] Survey [34]

Research questions about UX practitioners

RQ1: “what are the most important difficulties they encounter in their UX projects”.

RQ2: “what measurement and validation methods do they use when designing or launching a new system”.

Previous related work and motivation [Master´s Degree Project. Nov 203 - Jan 2014]

Findings - Insights

• Lack of a common language between UX professionals and clients or colleagues from other areas of the company.

• Lack of time on a regularly basis: to correctly define requirements, to test at different stages and to communicate with the team.

• Issues of recruiting users for testing: It is difficult to recruit users on a frequent basis in order to assure user testing is part of the process.

• Uncertainty regarding design and development decisions during a system development and after launching.

Study June - July 2015

Structure of the questionnaire Based on adapted UXMAS + new background questions Introduction explaining the interest of the research topic, followed by:

Questions • Background • Measurability of UX • Deep discussion

Online study June - July 2015

112 participated 40 fully responded

Structure of the questionnaire Based on adapted UXMAS + new background questions (32 q) Introduction explaining the interest of the research topic, followed by:

Questions • Background • Measurability of UX • Deep discussion Since then:

146 participated 42 fully responded

Participants Country of origin

6%4%4%

7%

11%67%

Chile Argentina

Peru

Mexico Spain Other*

*Brasil (1), Colombia (2), Costa Rica (1), Nicaragua (1), Venezuela (2)

Our study UXMAS

40% less than 30 years old 6% more than 40 years old

16% 41%

1) UX as a field in LATAM is even younger than in US / Europe 2) More junior and middle managers vs UXMAS (high

executives ?)

Participants Background

Some hypothesis

Our study UXMAS

87% practitioners 13% researchers

51% 59%

1) UXMAS was largely distributed at academic events 2) Lot of LATAM practitioners do not consider themselves as

a researchers

Participants Background

Some hypothesis

Participants Activities associated with UX most commonly done

Activity MentionsPrototyping 74

Interaction design 69Visual design 62

Usability Testing 60Personas / Archetypes 58

Web analytics 56Card sorting 45

Heuristic analysis 44Programming 42

Field research (ethnography) 30Eye-tracking 13No response 14

Future research should address what they say they do versus what they really do.

Measuring UX “What a measure is”…

ActionCompare, evaluate, analyze, validate, monitor, calculate, quantify

Input / OutputParameters, data, results, performance, knowledge

Tools

Processes

36 mentions

11 mentions

3 mentions

3 mentions

Measuring UX

When it should be measured Is important for…

77% during an interaction 72% after an interaction 31% before an interaction

Our study allowed several options. UXMAS allowed just one.

91% design and evaluation

Measuring UX Experiential qualities they mention

Meaningful, personally

encountered events [2]

that are memorized and

used to construct individual mental

references.

“”

Confidence (3) Satisfaction (3) Motivation (2) Efficiency (2)

Entertainment (2)

79% of those who mentioned one think it is measurable

Only 25% could mention a theoretical and

methodological argument. 28% mentioned a practical

one.

Measuring UX Some thoughts

Subjectivism, most of the time measures can have different interpretations.

“Experiential quality

can not be validated scientifically.

“

Designed experiences have to be validated in order to know if

users feel comfortable with the system or they hate it.

“

“

Measuring UX is difficult and

complex.

Measuring UX Practical issues

Our study UXMAS50% Cost of the UX measurement process

26% Time

Lack of a representative number of users for studies

Lack of knowledge and experience in UX measurement

“Lack of knowledge in exploiting feedback of UX for future system development”

Struggling with basic

issues vs more specific

ones?

Conclusions and discussion

Research issues • Length of the questionnaire. • The complexity of the vocabulary had an impact on

(1) Rate of response (2) Content of the response (some aspects considered methodological in UXMAS were mentioned as practical ones in our study)

As an insight, comparability is not always trivial.

Conclusions and discussion

Future work • More participants from several countries • Understand how cultural background and industry or

research maturity may play a role in the state of the art of the practice of UX.

• Search for methods and tools to solve specific issues related to the Latin American context.

• Develop specific education and research programs aimed to take UX field of study and research to a more mature level.

References

Icons from thenounproject.com.

[1] Hassenzahl, M., Tractinsky, N. 2006. User experience - a research agenda, Behaviour & Information Technology, 25:2, 91-97.

[2] Hassenzahl, M. 2014. User Experience and Experience Design. In: Soegaard, Mads and Dam, Rikke Friis (eds.). The Encyclopedia of Human-Computer Interaction, 2nd Ed.. Aarhus, Denmark: The Interaction Design Foundation: https://www.interactiondesign.org/encyclopedia/user_experience_and_experience_d esign.html

[3] Lallemand, C., Gronier, G. and Koenig, V. 2015. User experience: A concept without consensus? Exploring practitioners´s perspectives through an international survey. Computers in Human Behavior 43 (2015), 35-48.

[4] Law, E., van Schaik, P. and Roto, V. 2014. Attitudes towards user experience (UX) measurement. Int. J. Human-Computer Studies 72 (2014), 526-541.

[5] Law, E., Roto, V. and Hassenzahl, M. 2009. Understanding, Scoping and Defining User eXperience: A Survey Approach. In Proceedings of CHI 2009 (Boston, Massachusets, USA, April 7th, 2009).

[6] McGrath, J. Groups: Interaction and Performance. PrenticeHall, 1984.

[7] Roto, V., Law, E., Vermeeren A. and Hoonhout, J. 2011. User experience white paper. Bringing clarity to the concept of user experience: http://www.allaboutux.org/files/UXWhitePaper.pdf (last access, July 2015)

[8] Sánchez, A., Furtado, E. and Vivas, 2014. N. Challenges for Establishing a Latin American Community in HCI/UX, In CSCW Workshop (2014) http://researcher.watson.ibm.com/researcher/files/brropaula/challenges-CSCW-VF-1.pdf (last access, July 2015)

[9] Tokkonen, E., and Saariluoma, P. 2013. How User Experience is Understood? In Proceedings of Science and Information Conference 2013 (London, UK, October 07-09, 2013).

[10] Vermeeren, A., Law, E., Roto, V., Obrist, M., Hoonhout, J., and Väänänen-Vainio-Mattila, K.: User experience evaluation methods: current state and development needs. Proceedings of the 6th Nordic Conference on Human Computer Interaction: Extending Boundaries (Reykjavik, Iceland, 2010), 521-530.

Thank you! Gracias. Merci. Obrigado. Mulțumesc.

Download the paper: dl.acm.org/citation.cfm?id=2824914 Stay in contact: @carmenFR The survey: https://es.surveymonkey.com/r/MLF79CV