1 user-centered design and development instructor: franz j. kurfess computer science dept. cal poly...
Post on 19-Dec-2015
216 Views
Preview:
TRANSCRIPT
1
User-Centered Design and Development
Instructor: Franz J. KurfessComputer Science Dept.
Cal Poly San Luis Obispo
FJK 2005
5
Chapter 12 Observing Users
FJK 2005
6
Chapter Overview
• Goals, Questions, and Paradigms• Observation• Data Collection• Tracking Activities• Analysis, Interpretation, and Presentation of
Data
FJK 2005
7FJK 2005
Motivation
• the actual observation of users as they perform tasks with the system may reveal information that is difficult to obtain otherwise
• technologies like video cameras and audio recorders make it fairly easy to capture many aspects of the observation
• the activities of users can also be tracked indirectly
• the analysis, interpretation and presentation is essential after the actual observation
8FJK 2005
Objectives
• be familiar with the main observation methods• be aware of the different perspectives of an
on-looker, a participant, and an ethnographer• appreciate the importance of analyzing,
interpreting, and presenting data obtained from observations
• gain experience in performing and evaluating observations
12
What and when to observe
• Goals & questions determine the paradigms and techniques used.
• Observation is valuable any time during design.• Quick & dirty observations early in design• Observation can be done in the field (i.e., field
studies) and in controlled environments (i.e., usability studies)
13
Roles of Observers
• outsiders looking on– sometimes through a window or one-way mirror– should the participant be aware of it?
• participating observers– the observer works with other participants– emphasis of the observer should be on observation,
not on activities to perform the task
• ethnographers– intentional immersion into the “world” of the
participants
FJK 2005
14
Observation Frameworks
• The person. Who? • The place. Where?• The thing. What?
Framework Examples– Goetz and LeCompte (1984)– Robinson (1993)
15
The Goetz and LeCompte (1984) Framework
• Who is present? • What is their role? • What is happening? • When does the activity occur?• Where is it happening? • Why is it happening? • How is the activity organized?
15
16
The Robinson (1993) Framework
• Space – What is the physical space like?
• Actors– Who is involved?
• Activities – What are they doing?
• Objects– What objects are present?
• Acts– What are individuals doing?
• Events – What kind of event is it?
• Goals– What do they to accomplish?
• Feelings– What is the mood of the group and of individuals?
16
17
Aspects to Consider
• goals and questions• selection of framework, techniques• data collection• equipment to use• acceptance and trust by participants• handling of sensitive issues• involvement of informants• data analysis• need for triangulation
– different perspectives
18
Observing as an Outsider
• same technique as in usability testing• more objective than participant observation• often done in usability labs
– layout of the facilities– equipment is in place
• recording is continuous• analysis & observation almost simultaneous• danger of drowning in data• analysis
– can be coarse or fine grained
• presentation– Video clips can be powerful for telling astory
19
Participant Observation and Ethnography
• debate about differences• participant observation is key component of
ethnography• co-operation of people observed• informants can be useful
– but also sensitive
• data analysis is continuous• interpretivist technique• questions get refined as understanding grows• reports usually contain examples
20
Activity: Observation in RPGs
• is it ethical to join a community for the main purpose of observation– with or without the knowledge of other participants– human observer or computer agent
• do you know of any incidents in games you’re participating in
FJK 2005
21
Data Collection Techniques
• notes – often selective, and include some interpretation
• visual data– still camera– video
• audio data– tape recorder– usually included in video– computer
• tracking users:– diaries– interaction logging
22
Observation Activity: Review
• data collection techniques used– appropriate– worth the effort
• constraints imposed by the selected techniques– missed data– aspects not captured– observer bias– differences between observers
• analysis and interpretation– bias, differences, jumping to conclusions
FJK 2005
23
Data Analysis
• qualitative data – interpreted and used to tell the ‘story’ about what
was observed – categorized using techniques such as content
analysis
• quantitative data – collected from interaction and video logs– presented as values, tables, charts, graphs and
treated statistically.
24
Interpretive Data Analysis
• key events – drive the group’s activity
• patterns of behavior • triangulation
– testing of data sources against each other
• presentation– report findings in a convincing and honest way
• ‘rich’ or ‘thick descriptions’ – include quotes, pictures, and anecdotes
• software tools can be useful – NUDIST, nVivo (see http://www.qsr.com.au/)– Ethnograph (see http://www.qualisresearch.com/)
augmented FJK 2005
25
Looking for Patterns
• critical incident analysis– coarse analysis of the overall data set to identify
important events– only those events are analyzed in more detail
• content analysis– systematic categorization of events– requires orthogonal categories (no overlap)
• discourse analysis– interpretation of dialog– also considers meaning, intention, context
• quantitative analysis (statistics)– means, standard deviation, etc.
augmented FJK 2005
26
Key Points
• observation from outside or as a participant• analyzing video and data logs can be time-
consuming• participant observation includes collections of
comments, incidents, and artifacts • ethnography is a philosophy with a set of
techniques that include participant observation and interviews
• ethnographers immerse themselves in the culture that they study
top related