constructing images eyes-free: a grid-based dynamic drawing tool for the blind hesham m. kamel james...
TRANSCRIPT
Constructing Images Eyes-free: A Grid-based Dynamic Drawing
Tool for the Blind
Hesham M. KamelJames A. Landay
Group for User Interface ResearchEECS Department
University of California, Berkeley
2
Why do blind people need to draw?
Drawing by a totally blind user of the Integrated Communication 2 Draw (IC2D) tool
3
Presentation Overview
MotivationOther Graphical Systems for the BlindThe Integrated Communication 2 Draw (IC2D)IC2D Usability StudyResults and DiscussionLive DemoConclusions
4
Motivations Behind Creating a Drawing Tool for the Blind
Misconception: you can’t carry out visual tasks in a non-visual environmentTruth: there can be a bridge between mediumsMaking graphical information accessible for the blind is a challenge, not a problem the challenge is to match the physical
environment with a proper user interface
Our goal: allow the manipulation of on-screen objects in a non-visual format
5
Related Work: Creation & Access
Creating graphics TDraw (Kurze ‘96)
Communicating graphics Mercator (Mynatt ‘95)
6
Related Work:Image Sonification Systems
Auditory shape perception (Hollander ‘94) use of a virtual speaker array to represent
shapes
Scene sonification (Meijer ‘92) pixel-based image-to-sound conversion
7
Major Problems with Drawing UIs
Graphical user interfaces total reliance on visual feedback imagine drawing with the monitor off
where is the cursor? what’s on the screen? how do I get back to where I was?
let’s do a little experiment…
Haptic user interfaces hard to carry expensive
8
Grid-based Drawing Model
Based on telephone keypad known by most blind individuals nine fixed cells navigated by
keyboard directional keys or numbers 1-9
each cell is a unique point of reference & can be selected
equivalent to point & click
The grid supports feedback finding relative & absolute locations measuring distances determining common angles
(Kamel & Landay 2000)
1 2 3
4 5 6
7 8 9
9
Grid Recursion
Allows more precise point selections3 levels of recursion resolution of 27 x 27 cells permits drawing objects at different
scales
Objects drawn at full screen resolution
10
IC2D Provides a Medium for Accommodation & Collaboration
Blind accustomed to screen readers IC2D outputs voice & non-speech sounds
Users can create & share drawings reflecting self-expressionFacilitates sighted users communicating graphically w/ blind users & vice versa (Kamel & Landay 2001)
Allows users to enhance semanticsof drawings
11
Labeling in IC2D
Allows objects & groups of objects to be annotatedUses a hierarchical labeling structure
This car, created with helpof a sighted user, self-describes its “parts”
(Kamel & Landay 2001)
13
Circle
Label tire & spokes
Example of Labeling (Part 2)
Line Line
Spoke1
Line
Spoke2
Line
Front Tire
Circle
14
Circle
Group individual spokes togetherLabel that group
Example of Labeling (Part 3)
Line Line
Spoke1
Line
Spoke2
Line
Front Tire
Circle
Front Spokes
Group
15
Circle
Group front tire & front spokes togetherLabel that group
Example of Labeling (Part 4)
Line Line
Spoke1
Line
Spoke2
Line
Front Tire
Circle
Front Spokes
Group
Front Wheel
Group
16
Evaluation GoalsTest if users could carry out instructed drawing with precisionTest users’ recognition of interrelational representations of objects recognize objects in relation to grid positions &
to each other
Test labeling as method for understanding drawing via semantic enhancementTest if users could draw representations of real-world objects
17
MethodologyUsed keyboard input & voice synthesizer output16 participants:
8 sighted 8 visually impaired (5 partially sighted, 3 congenitally blind) 19-49 years old sighted & partially sighted participants wore blindfolds
Equal amount of familiarization timeUsed a verbal protocol & recorded along with outputResults evaluated on
task completion time participants’ self-assessed confidence level quality of the drawing, as rated by sighted third-party judge
18
Task 1 - Part ADraw 3 circles, describe spatial arrangementTests interralational drawing 38-year-old congenitally blind participant: “I can
understand the grid concept abstractly, but not in a visual sense, because I have never been able to see.”
21-year-old sighted participant: “I can almost visualize everything on the screen without having to look at it.”
Three circles drawn & described by this participant
19
Task 1 - Part BDraw triangle at smallest grid level in bottom right, draw rectangle in top left, then connect with lineTests instructed drawing & interralational reps
20
Task 2Explore & describe 3 labeled drawings 3 different arrangements of
a man, tree, and house one visually impaired
participant said: “I liked when it told you what the picture was, and where it was. It was very easy to picture that in my mind.”
Tests whether labels match mental model of the image
As the participants navigated the cursor around this figure, they heard “top of tree”, “bottom of tree”, “my man”, etc.
21
Draw a representation of a physical cubeTests if users can draw real world objects
a) blindfolded partially sighted participant successfully represented a physical cube by counting grid positions (t=10 minutes)
b) blindfolded sighted participant drew this representation of the cube by visualizing it (t=9 minutes)
Task 3
a) b)
22
Time for Task Completion (in minutes)
Time for Task Completion
0
2
4
6
8
10
12
14
1 2 3
Task Number
Tim
e in
Min
ute
s
Sighted Participants
Visually ImpairedParticipants
Tasks 1a, 1b, & 3: no significant differences in time, user confidence, or the judge’s ratingTask 2: visually impaired group significantly faster, more confident, & rated higher
23
Self-Assessed Confidence RatingAverage Confidence
0
2
4
6
8
10
12
1 2 3
Task Number
Rati
ng
Sighted Participants
Visually ImpairedParticipants
Blind participants more acclimated to voice outputSighted participants found the experience “intense”One blind participant: “…the way this grid is laid out is not complicated, you will be able to know exactly where you are all the time.”
24
Performance Rating (by judge)
Participants' Performance Rating
0
1
2
3
4
5
6
7
8
9
10
1 2 3
Task Number
Rati
ng
Sighted Participants
Visually ImpairedParticipants
Visually impaired participants on average performed at least as well as sighted participants
25
DiscussionGrid-based model can be comprehended by sighted & blind participantsInput/output keyboard effective navigational
device for interacting w/ graphics voice synthesizer doesn’t hinder
communicating graphical info
Labeling technique effective method for blind users to build mental models of images
Drawing by partially sighted participant using
IC2D
27
ConclusionsDesigned IC2D interface to match users’ abilitiesGrid-based model is a useful schema for creating graphics in a non-visual environmentLabeling technique effective for users building mental models of visual informationIC2D can be used to increase communication between sighted & blindCreating & accessing graphics in a non-visual environment is not a barrier, it is a challenge
Summary of Statistics(n=16) Visually
ImpairedSighted(blindfolded)
Task1a time 6 (average) 7.3 (average)Task1a confidence 8.1 6.8Task1a performance
8.8 8.8
Task1b time 8.3 8.8Task1b confidence 8.1 6.9Task1b performance
8.3 7.6
Task2 time 5 8.1Task2 confidence 8.8 6.9Task2 performance 8.4 6.8Task3 time 8.9 9.8Task3 confidence 6.3 5.8Task3 performance 3.5 3.5