user research - different approaches and methods

58
user research: dierent approaches and methods aga szóstek(at)gmail.com

Upload: aga-szostek

Post on 11-Aug-2014

2.807 views

Category:

Design


39 download

DESCRIPTION

This is the second part of my fourth lecture at HITLab, Canterbury University, Christchurch, New Zealand about user research. I am presenting the three levels of understanding user needs and the methods that correspond with investigating these needs. The idea is to show how different methods enable a designer to dig for different insights and how to conduct exemplary studies for each type of the method.

TRANSCRIPT

Page 1: User research  - different approaches and methods

user research: different approaches and methods aga szóstek(at)gmail.com

Page 2: User research  - different approaches and methods

what people level of knowledge

research methods

Fraukje Sleesvijk Visser

Page 3: User research  - different approaches and methods

what people level of knowledge

tell think explicit interviews

research methods

Fraukje Sleesvijk Visser

Page 4: User research  - different approaches and methods

what people level of knowledge

tell think

do act

explicit

observational

interviews

observations

research methods

Fraukje Sleesvijk Visser

Page 5: User research  - different approaches and methods

what people level of knowledge

tell think

do act

know feel

dream

explicit

observational

subconscious latent deep

interviews

observations

generative techniques

research methods

Fraukje Sleesvijk Visser

Page 6: User research  - different approaches and methods

before discussing the different methods it is crucial to know about the concurrent and retrospective experiences

Page 7: User research  - different approaches and methods

-  when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports

-  affective experiences are fleeting and become not available once the feeling is gone

-  once the feeling disappears the affective experiences need to be reconstructed

Page 8: User research  - different approaches and methods

-  when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports

-  affective experiences are fleeting and become not available once the feeling is gone

-  once the feeling disappears the affective experiences need to be reconstructed

-  reports of past feelings are based on semantic knowledge -  when asked how they “usually” feel during a particular

activity, people draw on their general beliefs about that activity

-  the actual experience does not figure prominently in these reports because the experience itself is no longer accessible

-  the same knowledge is a basis for predicting future feelings, for which episodic information is not available to begin with

Page 9: User research  - different approaches and methods

so, let’s discuss the methods

Page 10: User research  - different approaches and methods

what people level of knowledge

tell think explicit interviews

research methods

Page 11: User research  - different approaches and methods

introspective versus retrospective methods

Page 12: User research  - different approaches and methods

introspective methods Analysis of present behaviour and gathering of preferences by combining observations and interviews in context: -  cognitive walkthorugh: heuristic identification of problems

connected to the current ways of behaviour and interaction -  think aloud: heuristic identification of problems arising from

the current state of interaction through verbal articulation of user observations while interacting with the solution

-  co-discovery: a method where two users collaborate in order to resolve the problems found in the process of interaction with the current solution

-  Contextual Inquiry: gathering of detailed information about behaviour and motivations of users while the user is actually performing a given task

Page 13: User research  - different approaches and methods

retrospective methods

Support for retrospective assessment of user preferences towards a given solution:   -   interviews: a discussion with the user aiming to collect rich

material about his / her interactions and attitudes -  focus groups: group discussion about a given problem or

solution -  retrospective analysis of behaviour: and interview supported by

a qualitative recoding of user behaviour (e.g., video or audio recordings)

Page 14: User research  - different approaches and methods

when to apply?

-  exploration phase -  to get a better understanding -  to find inspiration for further research

-  requirements validation -  to check whether study assumptions are correct

-  evaluation -  to discuss and verify design decisions

Page 15: User research  - different approaches and methods

types of studies

Page 16: User research  - different approaches and methods

-  fully structured: the entire study takes place according to the earlier prepared scenario -  good for comparison across individuals -  more extensive answers comparing to written surveys -  inability ask unplanned questions

Page 17: User research  - different approaches and methods

-  fully structured: the entire study takes place according to the earlier prepared scenario -  good for comparison across individuals -  more extensive answers comparing to written surveys -  inability ask unplanned questions

-  semi-structured: there are certain guidelines to the elements of the study but the order is unimportant -  pre-specified questions -  questions as a starting point for discussion -  digression from the script is acceptable

Page 18: User research  - different approaches and methods

-  fully structured: the entire study takes place according to the earlier prepared scenario -  good for comparison across individuals -  more extensive answers comparing to written surveys -  inability ask unplanned questions

-  semi-structured: there are certain guidelines to the elements of the study but the order is unimportant -  pre-specified questions -  questions as a starting point for discussion -  digression from the script is acceptable

-  unstructured: going with the flow, only the initial question and a possible list of topics is defined -  good for open ended exploration -  difficult to conduct and analyse

Page 19: User research  - different approaches and methods

study questions -  close-ended

-  simple questions get simple answers -  “On a scale of 1-10, 10 being best, how did you like the prototype?” -  “Did you like the prototype?”

-  easy to analyse, but may not be that informative

-  open-ended -  invite elaboration and discussion -  build conversation and stories

-  “What did you think about the prototype?”

-  avoid -  long questions, jargon, leading questions, unconscious biases, negative

questions

Page 20: User research  - different approaches and methods

-  ask why? -  even when you think you know the answer, ask people why they do or

say things -  encourage stories

-  whether or not the stories people tell are true, they reveal how they think about the world

-  look for inconsistencies -  sometimes what people say and what they do are different. These

inconsistencies often hide interesting insights

-  observe nonverbal cues -  be aware of body language and emotions.

-  do not be afraid of silence -  if you allow for silence, a person can reflect on what they’ve just said

and may reveal something deeper

Page 21: User research  - different approaches and methods

preparing the study

-  brainstorm questions -  identify and order themes -  pilot the study -  write an interview guide -  prepare for capturing the interview -  prepare logistic backup

Page 22: User research  - different approaches and methods

conducting the study

-  you are the host -  be friendly, respectful and non-judgmental -  do not bias through body language -  be flexible (unless fully structured) but keep things on track -  ask for clarifications if something is unclear -  listen (don’t talk too much) and read between the lines -  be consistent

-  outline -  briefly introduce testing goals -  complete paperwork (informed consent) -  simple questions first, hard questions later

Page 23: User research  - different approaches and methods

capturing the study

-  have someone to help taking notes or do the audio or video recording

-  capture insights, non-verbal responses, etc. -  balance between detail and conciseness -  summarize immediately after the interview -  do transcriptions -  respect privacy and anonymity

Page 24: User research  - different approaches and methods

debriefing

-  ask for any final comments -  provide more detail about research goals -  offer a brief summary of findings -  turn off recording devices -  say “thanks”! -  reflect and summarize notes immediately

Page 25: User research  - different approaches and methods

what people level of knowledge

tell think

do act

explicit

observational

interviews

observations

research methods

Page 26: User research  - different approaches and methods

-  roots in ethnographic research -  observing people in action -  recording what is being observed -  helping the researcher learn perspectives held by participants -  often used in conjunction with other methods (e.g. interviews,

focus groups, content analysis)

Page 27: User research  - different approaches and methods

-  observations: observing and recording of user behaviour without a direct contact between the user and the researcher

-  shadowing: direct observation of user behaviour by the researcher without intervening with his / her actions

-  interaction logging: logging previously defined behaviours and system states that pertain to the specified user behaviour

-  diaries: systematic reporting by the user his / her interactions in a form of a diary or a blog

-  experience sampling: a systematic way of having participants provide samples of their ongoing behavior when participants' reports are dependent on either a signal, pre-established intervals or the occurrence of some event

-  day reconstruction method: capturing daily experiences of the users at the end of each day where participants are asked to listed all activities of the day that somehow related to the product or a situation that is under investigation

Page 28: User research  - different approaches and methods

data collection

-  field notes -  video -  audio -  transcripts -  behaviour matrix -  situational drawings

Page 29: User research  - different approaches and methods

pros and cons -  flexible and open-ended -  directly measures behavior -  gives the researcher

insider view -  allows for morphing of the

study

-  hard to be an insider -  invasive and intrusive -  difficult to record -  subjective -  Hawthorne effect -  time consuming -  not generalizable -  does not measure

cognitive or affective aspects

Page 30: User research  - different approaches and methods

phases of observational research

-  phase 1: develop positive relationships with participants, gatekeeper, etc.

-  phase 2: improve design, refocus, redefine questions after initial observations (may be ongoing)

-  phase 3: select additional participants as necessary -  phase 4: follow up and probe deeper

Page 31: User research  - different approaches and methods

ethical considerations

-  how much do you disclose? -  make your intentions clear -  get an informed consent -  develop a code of practice -  maintain confidentiality

Page 32: User research  - different approaches and methods

(Mack, et al., 2005)

Page 33: User research  - different approaches and methods

being an effective observer

-  know what you are researching -  check in with your lens, biases, experiences and expectations --

have a blank mind -  go where people are engaged in their daily lives and find the

right informants -  understand the culture -  rehearse how you will explain the purpose of your study -  decide how you will do the observation and collect the notes -  document what you observe without expectations -  summarize and expand on field notes as soon as possible -  be open to the research problem reformulation

Page 34: User research  - different approaches and methods

what people level of knowledge

tell think

do act

know feel

dream

explicit

observational

subconscious latent deep

interviews

observations

generative techniques

research methods

Page 35: User research  - different approaches and methods

using design as a research tool

Page 36: User research  - different approaches and methods

cultural probes

Page 37: User research  - different approaches and methods

-  tools for better understanding of people’s latent needs and dreams

-  small craft objects used in empathic engagements with individuals around issues centered on self-identity and personal significance

-  their form should reflect the research question and its context

-  asking questions by applying a delicate, provoking and creative means

-  and offering intriguing ways to consider the asked questions and to look for surprising answers

Page 38: User research  - different approaches and methods

understanding what people do with their invoices

Aga

Szós

tek,

Mar

cin

Piot

row

ski,

   

Page 39: User research  - different approaches and methods

what is agood and a bad invoice?

Page 40: User research  - different approaches and methods

what happens to an invoice once it arrives?

Page 41: User research  - different approaches and methods

what is the best way to pay an invoice?

Page 42: User research  - different approaches and methods

how should the email

accompanying an invoice look like?

Page 43: User research  - different approaches and methods

which elements should actually stay on an invoice?

Page 44: User research  - different approaches and methods

what should happen if you forgot to pay?

Page 45: User research  - different approaches and methods

technology probes

Page 46: User research  - different approaches and methods

-  simple, flexible, adaptable interactive installations -  combining collecting data about the use of the technology in a

real-world setting, testing the technology and inspiring users (and designers) to think of new kinds of technology

-  installed in user environment with the goal to observe them over a period of time

-  helping to identify design partners determining which kinds of technologies would be interesting to pursue

-  open-ended, inspiring new activities and encouraging experimentation

Page 47: User research  - different approaches and methods

managing interruptions | project supervised by Aga Szóstek

Page 48: User research  - different approaches and methods

dealing with stress at work

Page 49: User research  - different approaches and methods

design probes

Page 50: User research  - different approaches and methods

-  collaboration with a singular person based on a very deep engagement in the topic that is personally significant

-  designing objects, which reflect individual experiences and their meaning for the user

-  objects that support reflection and auto-reflection -  such objects often become a part of an intimate relationship

between the designer and the user

Page 51: User research  - different approaches and methods

Top Trumps probe cards to mark objects which are perceived to have magical power

Page 52: User research  - different approaches and methods

helping to keep a straight posture | Magda Rydygier project supervised by Aga Szóstek

Page 53: User research  - different approaches and methods
Page 54: User research  - different approaches and methods

preparing the study -  brainstorm the context to the given problem (e.g. through

contextmapping exercises) -  brainstorm on user expectations and the ideal experience -  prepare a list of questions regarding the current experience -  decide on the format of the cultural probe (digital elements,

physical element and intangible elements) -  select the criteria for choosing participants and prepare awards -  develop the probes and the instruction -  organize a session with participants to explain to them the

elements of the probe and the study procedure -  provide informed consent forms -  personally give the probes to the participants

Page 55: User research  - different approaches and methods

running the study -  the study typically lasts no longer that one week (but this is

very individual to the research problem) -  usually the study happens without an active involvement of

the researcher -  if the study involves an online diary – check the entries every

day and comment on them -  give participants a phone number and an email address where

they can contact you in the case of any problems -  at the end of the study invite all participates either to a joint

session (a Focus Group or a creative workshop) or for individual interviews

-  record the sessions for later reference

Page 56: User research  - different approaches and methods

summary

-  the research method should reflect the information you are trying to obtain

-  don’t be afraid to combine the methods and experiment with them

-  think of creating visual materials for your studies – they engage the users bringing you more insight

-  user research is a moment when you approach users with open mind – try not to bias them and put your ideas into their mouth

-  the quality of the data you collect will likely be directly reflected in the ideas you will create later on

Page 57: User research  - different approaches and methods
Page 58: User research  - different approaches and methods

references

Gaver, William W., et al. "Cultural probes and the value of uncertainty." interactions 11.5 (2004): 53-56. Gaver, Bill, Tony Dunne, and Elena Pacenti. "Design: cultural probes." interactions 6.1 (1999): 21-29. Mattelmäki, Tuuli. Design probes. Aalto University, 2006. Hutchinson, Hilary, et al. "Technology probes: inspiring design for and with families." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2003. Buijs, Jan. The Delft innovation method: A design thinker's guide to innovation. Eleven International Publishing, 2012.