cscd 487/587 human computer interface winter 2013 lecture 14 observation and ethnography
TRANSCRIPT
CSCD 487/587Human Computer Interface
Winter 2013
Lecture 14Observation and Ethnography
Outline There are benefits and challenges of
different types of observations How to observe as an onlooker, a
participant, and as an ethnographer
Observation can be Direct
– You observe what users do or
Indirect– You use diary studies and
logging
What and When to Observe
Observation is usable at all stages during product development
Goals and questions determine the paradigms and techniques used
Observers can be– Onlookers– Participants– Ethnographers
Degree of immersion that evaluators adopt varies across a broad outsider-insider spectrum
Types of Observation
ObservationObservationControlled Controlled
Environment Environment (e.g. Lab)(e.g. Lab)
Field Field Environment Environment (e.g. Natural)(e.g. Natural)
Outsider looking on
““Quick and Dirty” Quick and Dirty” oror
In usability In usability testingtesting
““Quick and Dirty” Quick and Dirty” or or
In field testingIn field testing
InsiderInsider N.AN.A..
Participant Participant observation (e.g. observation (e.g. in ethnography)in ethnography)
Approaches to Observation
“Quick and Dirty” Observation– It can occur anywhere, anytime– Least formality involved
Observation in Usability Testing– Video and interaction logs capture all user
operations– One-way mirrors or remote TV screen– Data is used to analyze what users are doing and
provide insight into users’ reactions– Can augment other techniques– Users can't always explain what they do
• Thus, limits of surveys and interviews !!!– Direct observation helps fill in details
Approaches to Observation
Observation in Field Studies– Observer may be
• Passive observer,• Participant observer or• Ethnographer
– Goal is to cause as little disruption as possible
Approaches to Observation
Passive or Outside Observer Example: An observer in class of boys and
girls whose primary job to keep track of how much time a particular technology is used by each gender
Participant Observer Participates in social conventions of a group,
combines participation with maintenance of professional distance for unbiased observation
Ethnography Becomes one of the group, takes weeks,
months or longer Inside information of community activity is
obtained
How to ObserveIn Controlled Environment
Practical Issues Need to Plan Advance– Decide where users are located so that equipment
can be set up in advance– Decide about modes of data capture e.g. video,
interaction logs– Equipment testing, audio and video– Get legal user consent– Prepare a script to guide users through set of
questions – It is important to make users feel comfortable– Problem with this approach
• Observers do not know what users are thinking
Controlled Environment Think-Aloud Technique
User is trying to perform certain task Observer wonders, what is going on, what is he thinking, why did he do that, Externalize thought process
Controlled Environment Think Aloud Technique
Checklist of things to Plan Before Going into Field
Select a framework to guide activities in field Decide how to record events Think about how to gain acceptance and trust of users under observation Be prepared to refine and refocus study for same or new user groups Think about how to handle sensitive issues Consider checking notes with a participant and plan to look at situation from a different perspective
Frameworks for In the field
There are Frameworks to guide observation, provides structureSimple– The person. Who? – The place. Where?– The thing. What?
The Goetz and LeCompte (1984) framework:– Who is present? – What is happening? – When does the activity occur?– Where is it happening? – Why is it happening? – How is the activity organized?
Frameworks forIn the field
The Robinson (1993) framework– Space. What is the physical space like?
– Actors. Who is involved?
– Activities. What are they doing?
– Objects. What objects are present?
– Acts. What are individuals doing?
– Events. What kind of event is it?
– Goals. What do they to accomplish?
– Feelings. What is the mood of the group and of individuals?
Data Collection Techniques
Notes plus still camera– The least technical way of collecting data– Transcription the first step in data analysis– Photographs, sketches, etc. can be easily collected
Audio recording plus still camera +– Less intrusive than video – More flexible and mobile – One drawback is transcribing the data which can
be onerous if many hours have to be transcribed
Data Collection Techniques
Video– Advantage of both audio and
video data, but more intrusive– Attention becomes focused on
what is seen through lens – Analysis of video can be long
and time consuming– Sound may get muffled when
recording in noisy conditions
Comparison
Table source: Preece, Rogers, Sharp: Interaction Design: Beyond Human-Computer Interaction. Wiley
Comparison (contd.)
Table source: Preece, Rogers, Sharp: Interaction Design: Beyond Human-Computer Interaction. Wiley
Indirect Observation
Diaries Diaries provide record of What users did, When they did it and What they thought about their interactions with technology Useful when users are scattered and unreachable Inexpensive and requires no special equipment or
expertise They rely on participants being reliable and
remembering to complete them
Indirect Observation
Interaction Logging - Includes analysing data from key presses, mouse movements etc.
- Usually synchronized with video and audio logs - Time stamped to calculate how much time a user
spends on a particular task
E.g. Explicit counters that record visits to a website
If you want to find out if adding a bulletin board increases number of visits, comparison of traffic before and after is useful
- Unobtrusive and large volumes of data can be collected
Ethnography
Ethnography’s is notion of participant observation
– Idea that you learn about other people's cultural practices by
– Going there, being there, and by doing it with them
– Most traditional anthropologists who would consider themselves to be ethnographers have spent years living in other cultures with people, and not just watching what they do, but actually doing it too
21
Ethnography
Ethnography
• Ethnography is an analytic perspective that is associated with doing interactive fieldwork- Term was coined or invented by anthropologist
Bronislaw Malinowski circa 1922 to describe his approach towards understanding other people’s cultures through fieldwork
- Term “ethnography” is often used to describe fieldwork in systems design
Design Scenario
Task for our Ethnographic study:– Design an email system for the 21st Century
networked home– We need to first look at how mail is currently
delivered in the home
An exampleHandling mail in the home
- Letterbox, pigeon hole, porch, front door
- In other words a known in common delivery / collection point … … …
- Where does mail arrive?
Known in common collection point
Contingent • But known in common by
household members
What happens next?
Mail arrives then what?
- Someone collects it. Who?
- But can anybody open it? No
- So it gets put some place where members can see that new mail has arrived
- A known in common place where members can sort through it to see if any mail has arrived for them
Known in common sorting point
• Just about anyone (even the dog :-)
26www.id-book.com
Who is it for?Specifically, how do we determine who is entitled to open particular mail items?- Name and ….
- Logos, handwriting, postmarks, senders’ address, etc.
• All articulate entitlement to open (not necessarily name)
Sorting the mail: members can see at-a-glance who it is for
- Visible features of envelopes enable members to determine at-a-glance who an item is for and what to do with it• junk it, leave it, open it
What if it is for someone else?What happens then? Do we leave the mail at the sorting point?
- Members usually put mail for others in places that are relevant to them
Relevant recipient point
• Places the recipient usually sits
– At the kitchen table, on their favourite chair, on the arm of the sofa, or nest of tables next to it, etc.
• Or places where the recipient cannot miss it (e.g., outside the bedroom door of your teenage son)• Mail items that are not for but may be of interest to other members are also placed in this way (postcards, letters, leaflets, etc.)
What do we do with it?
Lots of mail requires that we do something with it (e.g., pay a bill). Members place mail to reflect its action status.
- Mail requiring immediate action is placed such that members can see that at-a-glance
• By the mobile phone charger, or by the car keys, or in porch next to the front door, etc., if it’s has to be taken out of the home and dealt with
• Or by a computer if its to be dealt with online
Immediate action point
What if a response isn’t needed?
Not all mail requiring action requires that we respond to it
– e.g., bank statements, insurance certificates, mortgage payment records, etc.
- We may still need to take action on it though (e.g., checking and archiving).
- Mail where action may need to be taken is placed to show that action is pending.
Pending action point
30www.id-book.com
What if a response will wait?
Other mail items require action but not until a later date
– e.g., appointments, invitations, promotional offers, etc.
- They are placed to display their relevance to up and coming events
- e.g., on notice boards next to calendars, shopping lists, takeaway menus, etc.
Pending event points
31www.id-book.com
What if it’s Special?
Not all of the mail we receive is utilitarian bills, bank statements, appointments,We also receive “special” mail items: birthday cards, thank you cards, congratulations, etc.
- Special items are placed to display their essentially social character
• On window sills, mantelpieces, cupboards, dressers, etc.
Social display points
Naturally Accountable Organization
The organization of mail handling in the home, observable and reportable by members1. Collecting mail from a known in common delivery / collection point
2. Placing mail at a known in common sorting point
3. Sorting mail based on household member’s entitlement to do so, which is read off the visible
features of envelopes and not just the name
4. Placing sorted mail items in places relevant to their recipients
5. Placing mail to indicate its action status
Immediate action
Pending action
Pending event
6. Placing mail to display its special social status
33www.id-book.com
Design scenario revisited
What does the naturally accountable organization of mail handling suggest about the design of email for the 21st Century networked home?
– How might the field study “inform” design?• What are the main organizational features or “findings” of the
study?
• Thought experiment part two: who factored the distribution of email across, and its display at, different locations into their design sketches?- Can you see the value of looking at relevant activities in the real
world?
- Mail is distributed around the home- Mail is distributed through its physical placement and display- The display of mail performs various functions:
• At-a-glance it announces that new mail has arrived, that mail has arrived for you, that immediate action needs taking, and so on
Summary
Many techniques for collecting user data
Observation, survey, questionnaires, think aloud and ethnographic studies
Need to capture user's interactions with the object of study
Time, expense will certainly affect choice of method
And, ultimately what you need to learn from the users
35
End
Usability testing with Earth based interface
See Assignments page for short, fun assignment