Objective comparisons between

Exploration and Gesture based

keyboards for the blind

Adit Gupta, IIT Gandhinagar

Dr. Nikhil Balram, President and CEO,

Ricoh Innovations Corporation

11/26/2013

Motivation

About 8.075 million of the total 35 million

blind population is in India (about 21%)

But only 5% of them use or receive support from

assistive technology of any form! – WHO report

Pascolini D, Mariotti SPM. Global estimates of visual impairment: 2010. British Journal Ophthalmology Online First published December 1, 2011 as 10.1136/bjophthalmol-2011-

300539.

11/26/2013

Introduction

Current Assistive Technology for

text typing

Range from Braille Sense Plus (6000 USD) with a

Braille Display to GalaTee (400 USD) in the USA

Braille Sense Plus GalaTee

Images from Nanopac.com and amazon.com respectively

11/26/2013

Technology

What about making information

available to everyone on the go?

11/26/2013

But how does a blind individual

type and reach information without

tactile feedback?This question became the backbone of our

research

11/26/2013

Research

Goal

Before going further,

A bit of background in Braille

3 by 2 binary matrix, encodes upto 63 characters

Mario Romero, Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In Proceedings of the 13th International

Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 707-709.

11/26/2013

Braille

The current cutting edge research

on Eyes-Free text typing.

- Highly researched area in the West.

- We focused on the two which are in vogue

keeping Indian Blind participants in mind

11/26/2013

BrailleTouch

Georgia Tech Research

Drawbacks

Numeric keypad is not simple to use

Both hands are required

Highly Memory dependent (Braille)

Mario Romero, Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In

Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New

York, NY, USA, 707-709.

11/26/2013

BrailleTouch

Google’s Android TalkBack

Google Inc. Enables Explore-by-Touch

Drawbacks

Numeric keypad is not simple to use

11/26/2013

Talkbac

k

So we put them Head-to-Head

and compared their results

objectively

BrailleTouch vs Android Talkback Keyboard

11/26/2013

BPA, Ahmedabad

Established in 1947, has 15 campuses across

Gujarat, with 300 odd employees and many more

students

Taught Braille in 3 languages- English, Hindi &

Gujarati

Photograph from bpa.org11/26/2013

BPA

We recruited 4 completely Blind

participants for our study…

Participant Age Braille education Smartphone

education

P1 52 K-12 0

P2 27 K-10 0

P3 29 K-12 0

P4 30 K-12 4 months

Each participants used mobile phones with a TTS on a daily basis.

Nokia C5 series was a phone commonly used by them.

Quite often keep in touch with me via email!

11/26/2013

Demographics

Usability testing strategy

11/26/2013

Research

strategy

The Experiment App

Simple Key logger; No need for a very aesthetically

appealing design

BrailleTouch

Android Talkback

11/26/2013

App

0

2000

4000

6000

8000

10000

12000

14000

BrailleTouch Android Stock Keyboard

Aggregated results – Keystroke

Timings

Tim

e r

equ

ire

d (

mill

ise

co

nd

s)

11/26/2013

results

Can we now conclude that

BrailleTouch is a better technique?

Let’s take a closer look at the typing errors by our subjects

11/26/2013

Aggregated results – Keystroke

Timings

Nu

mb

er

of

err

ors

to

tal

0

0.5

1

1.5

2

2.5

3

3.5

BrailleTouch Android Stock Keyboard

11/26/2013

Results

Keystroke time inclusive of errors

Hence,

Actual time required to type ‘h’

= 3067 + 3313 + 3530

= 9910 millisecs! (inclusive of errors) Whereas it’s 3530 millisecs in the previous method

Typed

character

Target

characterMatch Timestamp

Timestamp

(Milliseconds)Textbox Input

Keystroke Time

(Milliseconds)

t t 1 16:11:32 1925859247 t 2539

j h 0 16:11:35 1925862314 tj 3067

backspace h 0 16:11:38 1925865627 t 3313

h h 1 16:11:41 1925869157 th 3530

e e 1 16:11:45 1925872317 the 3160

space space 1 16:11:48 1925875522 the 3205

11/26/2013

Data manipulation

Aggregated results –

Error Inclusive Keystroke Timings

11/26/2013

Insights

Take home #1

Is BrailleTouch or Android

Talkback keyboard ideal?

Maybe a hybrid approach between exploration and

gesture might be ideal since both have huge

drawbacks.

A longer term study on the participants would make

it clear11/26/2013

Learnings

Take home #2

Indian Blind users prefer the British

accented voice feedback more than

the American accent!

A simple reason for this is that the colloquial terms

are crisp and clear in the British accent

11/26/2013

Learnings

Take home #3

Have lesser motor control in their

ring finger to type

Clear from the errors increasing with the dots in the

Braille cell

11/26/2013

Learnings

Take home #4

Editing previously typed words

functionality is essential

Through my direct observations and the user’s

feedback

11/26/2013

Learnings

Future: A Hybrid approach?

A longer term study would be required for this

research.

11/26/2013

Learnings

Thanks!

11/26/2013