ch. 12 observing users reece, rogers, sharp. beyond human computer interaction. team 1:andy, nikhil,...
TRANSCRIPT
![Page 1: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/1.jpg)
Ch. 12 Observing Users
Reece, Rogers, Sharp. Beyond human computer interaction.
Team 1:Andy, Nikhil, Vladimir, Sajay
![Page 2: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/2.jpg)
12.1 Introduction
Learning Goals Benefits and challenges
of different methods How to observe How to deal with data Key issues in think-aloud,
diary studies, and interaction logging
Selecting and doing observations
Additional Content Critical Incident
Reporting: A type of diary observation
CIF: common industry format
Summative vs Formative Interaction Logging for the
Web Uzilla: A tool for web
usability testing by Andy
![Page 3: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/3.jpg)
12.2 Observing in a controlled environment
i.e., usability testingPlan user and equipment location.Test the setup.Provide participants with a written
consent form and script of the study.Use think-aloud technique (p. 365) to find
out what participants are thinking.
![Page 4: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/4.jpg)
Observing as an outsider
More objective than participant observation Lab equipment is in place Recording is continuous Analysis & observation almost simultaneous Care needed to avoid drowning in data Analysis can be coarse or fine grained Video clips can be powerful for telling story
![Page 5: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/5.jpg)
Frameworks to guide observation
The Goetz and LeCompte (1984) framework:- Who is present? - What is their role? - What is happening? - When does the activity occur?- Where is it happening? - Why is it happening? - How is the activity organized?
![Page 6: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/6.jpg)
The Robinson (1993) framework Space. What is the physical space like? Actors. Who is involved? Activities. What are they doing? Objects. What objects are present? Acts. What are individuals doing? Events. What kind of event is it? Goals. What do they to accomplish? Feelings. What is the mood of the group and of
individuals?
Frameworks to guide observation
![Page 7: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/7.jpg)
Participant observation & ethnography
Must get co-operation of people observedInformants are usefulData analysis is continuousMove backwards and forwards from broad
to specific questionsCollect variety of dataHolistic approach – broad analysis
![Page 8: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/8.jpg)
You need to consider
Goals & questionsWhich framework & techniquesHow to collect dataWhich equipment to useHow to gain acceptanceHow to handle sensitive issuesWhether and how to involve informantsHow to analyze the dataWhether to triangulate
![Page 9: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/9.jpg)
12.4 Data collection techniques
Notes & still cameraAudio & still cameraVideoTracking users:
- diaries- interaction logging
![Page 10: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/10.jpg)
Notes plus still camera Taking notes
Least technical way Most feasible way Could prove a challenge as one has to observe and
listen while taking notes. Writing skills such as writing speed, use of short
hand could prove useful One needs to translate and transcribe the written
content into useful information Other alternative are include another person
in the data collection process; use a laptop Use a camera to record images
![Page 11: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/11.jpg)
Audio recording plus still camera
Audio recording Light and easy to carry Less intrusive than a video camera Transcribing long interview hours could be a
challenge Lack of a visual record Cannot be used in a noisy environment
![Page 12: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/12.jpg)
Video Captures both visual and audio data Extremely intrusive Instruments could be expensive but
digicams/handheld camera are a cheaper option One may tend to focus only on what is seen
through the lens and ignore what has been missed out.
Sound may get muffled when recording under noisy conditions
Analysis of video data could be very time consuming.
Lets compare the Data-collection techniques – Table 12.2 page 376
![Page 13: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/13.jpg)
12.5 Indirect Observation
Done when direct observation is not possible (obtrusive or evaluators cannot be present)
Tracking users Diaries Critical Incidents Interaction logging
![Page 14: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/14.jpg)
Diaries
Classical paper based or using online forms linked to database etc.
Participant records what she did and experience she had while doing the task
Diaries collected from all the participants are then analyzed
Barry Brown’s research using diarys’ for HP design framework for devices used to capture or record information.
http://www.dcs.gla.ac.uk/~barry/papers/Diary%20study-chi%20paper.pdf
![Page 15: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/15.jpg)
Advantages
InexpensiveSubjects could be tested location
independentExperimenter not required to be presentLarge amount of data could be gathered
![Page 16: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/16.jpg)
Disadvantages
Relies on the reliability of the participantProblem of participant not remembering
actual experience of event (better or worse)
Subject is the experimenter and hence variability in data – not much control other than structures templates to collect information.
![Page 17: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/17.jpg)
Interaction logging
Record all the interactions of the subject with the system (key presses, mouse movements, eye tracking, etc.) combined with audio and video logs.
Log is time stamped.Data can be analyzed to understand
cognitive aspects, user behaviors and reactions to variations in task etc.
![Page 18: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/18.jpg)
Interaction logging - Uses
Useful for web studies.Testing Web layouts, displays, GUI
interfaces etc.Understanding the users’ reactions to
variations in tasks and layout combinations.
Testing the efficiency of the interface and its ease of use.
![Page 19: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/19.jpg)
Interaction Logging on the Web
Server log file miss cached pages
To circumvent Web bugs – useful for
visitors at large Proxies – ala WebQuilt
http://guir.berkeley.edu/projects/webquilt/
Instrumented Browsers Andy’s Uzilla.net PARC’s WebLogger
Challenges Inferring user goals Analyzing paths
Triangulation Establishing the
magnitude of problems observed in testing from log files
![Page 20: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/20.jpg)
12.6 Analyzing, Interpreting, and Presenting Data
12.6.1 Qualitative analysis to tell a story Hilight tapes, effective communication
12.6.2 Qualitative analysis for categorization Advanced linguistic analysis tools
12.6.3 Quantitative Data Analysis Measure learning, efficiency
12.6.4 Feeding the findings back into design Picture in a picture highlight tapes
![Page 21: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/21.jpg)
Common Industry Format (CIF)
For summative testing (later stages in software development)
Subjective and Quantitative Measures On key tasks
Intended for vendors to communicate usability Boeing has adopted for all products and
suppliers
![Page 22: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/22.jpg)
Uzilla: A product suite for web usability testing
Commercial venture by Andy Edmonds Overview talk SCIP 2002 Edmonds, K. A. (in press). Uzilla: A new tool for web usability testing. Behavior Research
Methods, Instrumentation and Computers. May, Special Issue of Soc. For Computers in
Psychology.
Automates test designInstruments data collectionAggregates resultsFree for student projects
![Page 23: Ch. 12 Observing Users Reece, Rogers, Sharp. Beyond human computer interaction. Team 1:Andy, Nikhil, Vladimir, Sajay](https://reader034.vdocuments.us/reader034/viewer/2022051516/56649ea35503460f94ba70f7/html5/thumbnails/23.jpg)
Sarah Bly Interview
Unfinished book on non-speech audio in UI design
At Chi ’03 Visual Support for Conversations: People's Choice
Tutorial, Steve Harrison (Dyxsis D/R Consulting), Sara Bly (Sara Bly Consulting)To familiarize the participant with the underlying ideas and subtle design considerations that make shared drawing systems essential in many conversational settings.