porta ce cursor a contextual eye cursor for general pointing in windows environments

8
Copyright © 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail [email protected] . ETRA 2010, Austin, TX, March 22 – 24, 2010. © 2010 ACM 978-1-60558-994-7/10/0003 $10.00 ceCursor, a Contextual Eye Cursor for General Pointing in Windows Environments Marco Porta, Alice Ravarelli, Giovanni Spagnoli Dipartimento di Informatica e Sistemistica – Università di Pavia Via Ferrata, 1 – 27100 – Pavia – Italy [email protected], [email protected], [email protected] Abstract Eye gaze interaction for disabled people is often dealt with by designing ad-hoc interfaces, in which the big size of their ele- ments compensates for both the inaccuracy of eye trackers and the instability of the human eye. Unless solutions for reliable eye cursor control are employed, gaze pointing in ordinary graphical operating environments is a very difficult task. In this paper we present an eye-driven cursor for MS Windows which behaves differently according to the “context”. When the user’s gaze is perceived within the desktop or a folder, the cursor can be dis- cretely shifted from one icon to another. Within an application window or where there are no icons, on the contrary, the cursor can be continuously and precisely moved. Shifts in the four di- rections (up, down, left, right) occur through dedicated buttons. To increase user awareness of the currently pointed spot on the screen while continuously moving the cursor, a replica of the spot is provided within the active direction button, resulting in improved pointing performance. CR Categories: H.1.2 [Models and Principles]: User/Machine Systems—Human Factors; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Input Devices and Strategies, Interaction Styles Keywords: gaze interaction, eye tracking, eye cursor, eye point- ing, assistive technology, alternative communication 1 Introduction People affected by severe motor impairments need effective me- thods for providing input to the computer. Exploiting eye gaze as a substitute for the mouse is potentially the most intuitive way to interact with a PC without using the hands: the “point-and- click” paradigm at the basis of current operative environments is universally adopted, and probably also the one most suitable for two-dimensional interfaces. However, while pointing tasks are inherently connected with eye fixations [Smith et al. 2000] — using the mouse, we look at a target and then move the cursor to it by means of a precise ocu- lar-hand coordination — there are both physiological and tech- nological obstacles which limit pure eye-based pointing. On the one hand, even during fixations the eyes are not perfectly still, but are characterized by jitters of different kinds [Yarbus 1967], such as microsaccades [Engbert and Kliegl 2004]; unless a me- chanism for stabilizing the detected gaze position is employed, the eye-controlled pointer will tremble to some extent. On the other hand, even very recent eye trackers have a limited preci- sion (typically, 0.5 degrees), and consecutive gaze samples ac- quired by the device cannot be exactly centered on the same point. For these reasons, the basic approach which simply dis- plays the cursor where the user’s gaze is detected on the screen is hardly practicable — a shaking cursor is generally annoying, and precise pointing on small targets is practically impossible. Indeed, most existing eye-controlled interfaces are specifically designed to make up for such limitations. For instance, they are characterized by big graphical elements (e.g. buttons), which can be easily selected even if the user’s gaze is detected in slightly different positions. For object selection, the dwell time principle is typically exploited: the mouse click (or double click) is simu- lated by looking at a target for a certain time. Usually, conti- nuous gaze feedback is avoided, thus eliminating the bad effect of a trembling cursor constantly displayed on the screen. De- pending on the application, other kinds of feedback may be used, associated with elements of the interface (for example, a button may change its color progressively as it is fixed and the dwell time approaches). Program suites developed for commer- cially-available eye trackers (e.g. MyTobii [Tobii 2009]) are col- lections of applications sharing graphical look and interaction mechanisms, designed on-purpose for eye gaze interaction. While a dedicated environment for the execution of eye- controlled programs has undoubtedly a number of advantages, it has some limitations as well. First of all, it constrains the user to employ only the software available in the suite: any other appli- cation installed on the computer cannot be controlled by means of the eyes (or, if so, the task is very difficult, because elements of ordinary graphical user interfaces are usually small and not designed for easy eye pointing). Moreover, program suites are often associated with specific eye trackers: if, for any reason, the user wants to change the device, the old applications may not work properly on the new system. When the market of eye trackers will expand (in a hopefully not too far future), the de- crease of prices is likely to accentuate such problem. Whatever the reason why a software purposely designed for eye pointing is not available, the possibility to efficiently use the eyes like a mouse is desirable in many situations. However, until eye trackers will become extremely precise machines, proper interaction mechanisms are necessary to compensate for their lack of accuracy, as well as to make up for the intrinsic instable behavior of the human eye. Several approaches have been pro- posed to date for reliable eye pointing, trying to find good tra- 331

Upload: kalle

Post on 13-Jan-2015

1.440 views

Category:

Documents


1 download

DESCRIPTION

Eye gaze interaction for disabled people is often dealt with by designing ad-hoc interfaces, in which the big size of their elements compensates for both the inaccuracy of eye trackers and the instability of the human eye. Unless solutions for reliable eye cursor control are employed, gaze pointing in ordinary graphical operating environments is a very difficult task. In this paper we present an eye-driven cursor for MS Windows which behaves differently according to the “context”. When the user’s gaze is perceived within the desktop or a folder, the cursor can be discretely shifted from one icon to another. Within an application window or where there are no icons, on the contrary, the cursor can be continuously and precisely moved. Shifts in the four directions (up, down, left, right) occur through dedicated buttons. To increase user awareness of the currently pointed spot on the screen while continuously moving the cursor, a replica of the spot is provided within the active direction button, resulting in improved pointing performance.

TRANSCRIPT

Page 1: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

Copyright © 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail [email protected]. ETRA 2010, Austin, TX, March 22 – 24, 2010. © 2010 ACM 978-1-60558-994-7/10/0003 $10.00

ceCursor, a Contextual Eye Cursor for General Pointing in Windows Environments

Marco Porta, Alice Ravarelli, Giovanni Spagnoli Dipartimento di Informatica e Sistemistica – Università di Pavia

Via Ferrata, 1 – 27100 – Pavia – Italy [email protected], [email protected], [email protected]

Abstract

Eye gaze interaction for disabled people is often dealt with by designing ad-hoc interfaces, in which the big size of their ele-ments compensates for both the inaccuracy of eye trackers and the instability of the human eye. Unless solutions for reliable eye cursor control are employed, gaze pointing in ordinary graphical operating environments is a very difficult task. In this paper we present an eye-driven cursor for MS Windows which behaves differently according to the “context”. When the user’s gaze is perceived within the desktop or a folder, the cursor can be dis-cretely shifted from one icon to another. Within an application window or where there are no icons, on the contrary, the cursor can be continuously and precisely moved. Shifts in the four di-rections (up, down, left, right) occur through dedicated buttons. To increase user awareness of the currently pointed spot on the screen while continuously moving the cursor, a replica of the spot is provided within the active direction button, resulting in improved pointing performance.

CR Categories: H.1.2 [Models and Principles]: User/Machine Systems—Human Factors; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Input Devices and Strategies, Interaction Styles

Keywords: gaze interaction, eye tracking, eye cursor, eye point-ing, assistive technology, alternative communication

1 Introduction

People affected by severe motor impairments need effective me-thods for providing input to the computer. Exploiting eye gaze as a substitute for the mouse is potentially the most intuitive way to interact with a PC without using the hands: the “point-and-click” paradigm at the basis of current operative environments is universally adopted, and probably also the one most suitable for two-dimensional interfaces.

However, while pointing tasks are inherently connected with eye fixations [Smith et al. 2000] — using the mouse, we look at a target and then move the cursor to it by means of a precise ocu-lar-hand coordination — there are both physiological and tech-

nological obstacles which limit pure eye-based pointing. On the one hand, even during fixations the eyes are not perfectly still, but are characterized by jitters of different kinds [Yarbus 1967], such as microsaccades [Engbert and Kliegl 2004]; unless a me-chanism for stabilizing the detected gaze position is employed, the eye-controlled pointer will tremble to some extent. On the other hand, even very recent eye trackers have a limited preci-sion (typically, 0.5 degrees), and consecutive gaze samples ac-quired by the device cannot be exactly centered on the same point. For these reasons, the basic approach which simply dis-plays the cursor where the user’s gaze is detected on the screen is hardly practicable — a shaking cursor is generally annoying, and precise pointing on small targets is practically impossible.

Indeed, most existing eye-controlled interfaces are specifically designed to make up for such limitations. For instance, they are characterized by big graphical elements (e.g. buttons), which can be easily selected even if the user’s gaze is detected in slightly different positions. For object selection, the dwell time principle is typically exploited: the mouse click (or double click) is simu-lated by looking at a target for a certain time. Usually, conti-nuous gaze feedback is avoided, thus eliminating the bad effect of a trembling cursor constantly displayed on the screen. De-pending on the application, other kinds of feedback may be used, associated with elements of the interface (for example, a button may change its color progressively as it is fixed and the dwell time approaches). Program suites developed for commer-cially-available eye trackers (e.g. MyTobii [Tobii 2009]) are col-lections of applications sharing graphical look and interaction mechanisms, designed on-purpose for eye gaze interaction.

While a dedicated environment for the execution of eye-controlled programs has undoubtedly a number of advantages, it has some limitations as well. First of all, it constrains the user to employ only the software available in the suite: any other appli-cation installed on the computer cannot be controlled by means of the eyes (or, if so, the task is very difficult, because elements of ordinary graphical user interfaces are usually small and not designed for easy eye pointing). Moreover, program suites are often associated with specific eye trackers: if, for any reason, the user wants to change the device, the old applications may not work properly on the new system. When the market of eye trackers will expand (in a hopefully not too far future), the de-crease of prices is likely to accentuate such problem.

Whatever the reason why a software purposely designed for eye pointing is not available, the possibility to efficiently use the eyes like a mouse is desirable in many situations. However, until eye trackers will become extremely precise machines, proper interaction mechanisms are necessary to compensate for their lack of accuracy, as well as to make up for the intrinsic instable behavior of the human eye. Several approaches have been pro-posed to date for reliable eye pointing, trying to find good tra-

331

Page 2: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

deoffs between accuracy and ease of use. In this paper we present ceCursor, a special pointer which can be controlled through the eyes in different ways, according to the specific con-text. The cursor, designed for Microsoft Windows operating sys-tems, allows both “rough” and accurate pointing within applica-tion windows, while icon selection (within folders and on the desktop) occurs in a “discrete” way.

The paper is structured as follows. Section 2 briefly presents some research projects related to eye cursors and eye pointing in general. Section 3 describes the features of ceCursor and the way it can be employed. Section 4 provides a few technical de-tails about the system. Section 5 illustrates and discusses expe-rimental results. Section 6, at last, draws some conclusions.

2 Related Work

The implementation of reliable eye-controlled cursors has been a stimulating challenge for many years.

Among the oldest projects, it is worth citing Eye Mouse, a com-munication aid based on electrooculogram (EOG) signals allow-ing the user to control a normal mouse with a combination of eye movements and blinks [Norris and Wilson 1997]. While ra-ther primitive, Eye Mouse was one of the first attempts at relia-bly controlling an on-screen cursor for general computer interac-tion. The famous MAGIC (Manual And Gaze Input Cascaded) pointing project by IBM came shortly after [Zhai et al. 1999]. Starting from the observation that it is unnatural to overload a perceptual channel such as vision with motor control duties, gaze in MAGIC is only used to approximately position the pointing cursor, while the small movements necessary to pre-cisely move it are made by hand — a good approach for people with normal motor abilities, but a totally unsuitable strategy for severely disabled users, unfortunately. After MAGIC, several techniques for eye-hand mixed input have been developed, aimed at improving the performance of common mouse-based operations. For example, very recent are the Ninja [Räihä and Špakov 2009] and Rake [Blanch and Ortega 2009] cursors me-thods, where several cursors are displayed on the screen at the same time and eye gaze is exploited to select the currently active one.

Limiting our investigation to pure eye-based interaction, given the small size of ordinary interface components, help to precise eye pointing can come from zooming. If the fixed area on the screen is enlarged, it becomes easier to select small elements. First studies in this direction date back to ten years ago [Bates 1999], with experiments aimed at comparing eye-only and eye-with-zoom interaction in target acquisition tests. Successive re-search definitely demonstrated that zooming makes usable eye interaction possible, and that target size is the overriding factor affecting device performance [Bates and Istance 2002]. One of the first projects where zooming was practically exploited to interact with a “normal” operating environment (Microsoft Win-dows, in particular) is ERICA [Lankford 2000]. In this system, if the user looks at a specific spot on the screen for more than a dwell time, a window appears where the region around which the user was fixating is displayed magnified. Looking at a cer-tain point within such window, mouse clicks are triggered using again the dwell time principle. An analogous approach is fol-lowed by Kumar et al. [2007] in the more recent EyePoint project. In this case, if the user looks at a certain location on the screen and, at the same time, presses a specific key on the key-board, the observed screen portion is magnified, and a grid of dots appears over it. Single, double and right click actions are

then performed as soon as the user releases the key. Although this method requires the user to perform a certain physical action (e.g. press a key) to accomplish the selection process, which may not be possible for a disabled person, other solutions could be adopted as well (e.g. dwell time). An interesting variant of the zooming technique is the so-called “fish eye” lens effect [Ashmore et al. 2005]. Like when looking through a magnifying lens, the fixed area is expanded, allowing the user to maintain an overview of the screen while selectively zooming in on the re-gion of interest.

Whatever the pointing strategy adopted, the improvement of eye pointing precision is among the main desiderata of people need-ing eye-based interaction. For instance, Zhang et al. [2008] pro-pose three methods to increase eye cursor stability, namely force field, speed reduction, and warping to target center. The pur-pose of these techniques is to adjust eye cursor trajectories by offsetting eye jitters, which are the main cause of destabilizing the eye cursor. As another example of recent research of this kind, Kumar et al. [2008] propose an algorithm for real-time saccade detection, which is used to smooth eye tracking data in real-time. Such algorithm tries to identify gaze jitters within sac-cades, which could be misled for new saccades and deceive the eye tracker.

Because of the limitations in the steadiness and accuracy of cur-sor control provided by eye trackers, there are also approaches which combine gaze detection with electromyogram (EMG) sig-nals generated by the facial muscles (e.g. [Chin et al. 2008]). These solutions, although generally slower, can be more accu-rate than eye-only control, but are unfortunately more invasive, since the user has to wear electrodes on the face.

There are also several implementations of very cheap eye input systems which use normal webcams as an input source. For ex-ample, the systems by Gorodnichy and Roth [2004] and by Siri-luck et al. [2007] exploit face movements to control mouse pointing position, and eye blinking to generate mouse clicks. Performances of such solutions, however, are usually very li-mited and may not be suitable for individuals who can only move the eyes.

3 System Description

ceCursor is basically composed of a square (whose central point indicates the actual pointer position) and of four direction but-tons placed around it (Figure 1).

Figure 1 ceCursor

Direction buttons are in the shape of triangles, and are pressed by eye gaze. The cursor is displayed on the screen with a semi-

332

Page 3: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

transparent effect, and its size depends on the precision of the employed eye tracker, as well as on the eye pointing ability of the user (a cursor 300 pixels high and large is usually fine, un-less the user is totally novice). The side of the central square is one third of the cursor width.

As will be explained in the next subsections, ceCursor behaves differently according to where it is at a certain moment. In any case, looking inside the central square causes a mouse click to be generated in its center after a dwell time (for instance, one second). Time lapsing is graphically represented by concentric circles progressively appearing within the square and filling it toward the center (Figure 2). After the first click, if another click is generated in the same position, it is interpreted as a double-click.

Figure 2 Click generation process

If the user looks outside the cursor (that is, neither within the central square nor in direction buttons), after a dwell time it is shifted to a new position — the nearest icon if the cursor is on the desktop or within a folder, or the user fixation point if the cursor is within an application. A typical dwell time value is one second.

The ‘M’ placed in the lower-right area near ceCursor, when fixed for a certain time, causes the icon of a mouse to appear (Figure 3): looking at it, the user can change the currently active mouse button (right/left and vice versa, alternatively).

Figure 3 Icon for changing the active mouse button

The small circle located in the upper-right area near ceCursor is instead used to “stick” it in a certain place on the screen (it be-comes more transparent and its color changes to red). This func-tion allows the cursor not to be in the way of other user activities (e.g. reading) when not necessary.

3.1 Case 1: ceCursor on the Desktop or within a Folder

In presence of icons, ceCursor is “captured” by them. In other words, if the user looks at an area where there are icons, the cur-sor is automatically positioned on the nearest one. This behavior is in line with usual activities carried out within a folder or on the desktop, which necessarily involve icons.

When ceCursor is positioned over an icon and the user looks at a direction button, the cursor “jumps” over the next icon in that direction (if there is one). This way, if the direct pointing was

not successful, it is very easy to shift the cursor to the right icon. On the one hand, precise pointing is difficult, and it may be hard for the user to select an icon at the first attempt (especially if it is small). On the other hand, since there are no other possible ac-tions that can be performed, it would be useless — or better, slower — to move the cursor in a “continuous” manner by means of direction buttons: a discrete motion strategy has the advantage of both simplifying the pointing task and speeding up the selection process. Figure 4 shows an example with icons on the desktop.

On the desktop, there is a threshold distance from icons beyond which the “capture process” does not occur (350 pixels in our experiments), and the cursor is moved like within an application window (see Section 3.2). The reason for this is because on the desktop the cursor may be moved to select other elements be-sides icons, such as parts of application windows. Moreover, when ceCursor is too close to a screen edge where there are icons, it is automatically shifted to the nearest outermost one. “Too close” means that the cursor, if moved further, would not be totally included in the screen, because a direction button would be partially or totally concealed through the edge (which would make other shifts in that direction difficult, or even im-possible). Once “hooked” at an icon on the edge, ceCursor can be easily moved to the desired icon using the opposite direction button.

Within a folder, ceCursor can operate with any visualization mode of MS Windows (small and big icons, preview, details, etc.): the cursor is able to recognize the way icons are arranged, as well as their size, to correctly move among them (Figure 5).

. . . . . . . .

Figure 4 Discrete movement of ceCursor for icon selection

on the Desktop

Figure 5 ceCursor with big (left) and small (right) icons

333

Page 4: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

Actually, it is especially with small icons that the “jumping” mo-tion modality of ceCursor can be appreciated, since in this case the pointing task becomes extremely difficult. To simplify the three common operations performed on a folder window, i.e. “Minimize”, “Up one level” and “Close”, when a folder is opened, three big buttons are displayed over it, which work like the standard buttons of any window (Figure 6). Look-ing at them for a certain time, the corresponding actions are per-formed.

Figure 6 Control buttons displayed over a folder window

3.2 Case 2: ceCursor within an “Icon Free” Area

When ceCursor is within an application window, or on the desk-top but sufficiently far from icons, it can be precisely moved to point at the desired target.

Looking anywhere within an “icon free” area causes the cursor to be shifted to the fixed spot. However, small interface ele-ments may be difficult to achieve at the first attempt. To exactly position the cursor, the user can then use direction buttons. As long as a direction button is fixed, the cursor is continuously and smoothly moved in that direction (Figure 7). Speed, initially rel-atively low (50 pixels/sec), raises progressively (with an in-crease of 50 pixels/sec every two seconds).

The motion of the cursor stops as soon as the user looks outside the button. Once the center of ceCursor (identified by a red cir-

cle) is over the target, the user can look inside the central square and start the click generation process.

Indeed, recognizing that the cursor is over the desired (maybe small) target is not always so easy. After a first implementation of ceCursor, we soon realized that the pointing task through di-rection buttons is characterized by very frequent shifts between the button and the central square: accurate adjustments require the user to alternatively look at the pointed spot, to check whether the target has been reached, and at direction buttons, to move the cursor further. Through several informal trials, we found that such a pointing mechanism, besides not being as fast as we would expect, may become annoying in the long run. We therefore implemented a new version of ceCursor, which turned out to be more effective.

In this new version, during cursor movement the area included in the central square is replicated within the active direction but-ton (Figure 8). This way, the user can always be aware of what is being pointed by the cursor at a certain moment, even while constantly looking at a direction button to reach the target.

Such a solution makes it possible for the user not to loose the “context” of the cursor, avoiding repeated gaze shifts between the central square and the direction button. Indeed, the adopted strategy is especially effective if two “mental steps” are fol-lowed in sequence:

1. Identification of a desired target in the central square 2. Cursor movement by means of direction buttons, with the

target clearly in mind

As will be illustrated in Section 5, our experiments have shown that this last implementation of ceCursor, besides being very appreciated by users, provides better performances in terms of time to complete pointing tasks.

Analogously to what happens within an area containing icons, when ceCursor gets too close to a screen edge (that is, one of the direction buttons starts disappearing), it is shifted so that its cen-ter is exactly on the border. The cursor can then be moved pre-cisely to the desired target using the opposite direction button. Without such a mechanism, it would, for example, be impossible to click the ‘close’ button of an application opened in full screen, or to select one of its menus (Figure 9).

a b

Figure 8 Replica of the currently pointed area displayed within the direction button (the cursor is moving rightward in

a and downward in b)

Figure 7 Schematization of the continuous motion of ceCursor (1 pixel every 1/50 sec in the first two seconds)

334

Page 5: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

4 A Few Technical Details

ceCursor is implemented in C# within the Microsoft .NET framework. As an eye tracker, we used the Tobii 1750 [Tobii Technology AB 2004], which integrates all its components (camera, near-infrared lighting, etc.) into a 17’’ monitor. The sampling rate of the device is 50 Hz, i.e. gaze data are acquired 50 times a second.

The system was developed for and tested with MS Windows XP Home Edition. To access the several Windows data and features necessary for ceCursor to work (e.g. information on folder visu-alization modes, icon size and position, etc.), functions from the user32.dll and kernel32.dll libraries were imported in C#. Cursor rendering was double-buffered, to avoid flickering effects.

A control panel allows all system parameters (e.g. dwell times and level of transparency) to be set through text textboxes and sliders, as well as to perform eye tracker calibration.

5 Experiments

Besides informally testing ceCursor many times during its de-velopment, we also carried out two more structured experiments (E1 and E2) once it was fully implemented.

Nine testers (aged between 19 and 45, 25.22 on average, seven males and two females) took part in experiment E1. None of these testers had any previous experience with eye tracking de-vices and eye-controlled interfaces. Two testers (26 and 20, males) participated in experiment E2. Both of them were not totally novice, as they had been involved in some eye tracking tests before.

5.1 Procedure

Both E1 and E2 were composed of two tests, TA and TB, struc-tured as follows:

TA. Within a folder containing seven other folders in the form of icons (Figure 10a), the user had to open ‘fold-er3’ (task 1) and then, in that folder, which contained in turn seven small folders (Figure 10b), to open ‘folder5’ (task 2).

TB. Within an empty panel displayed in full screen (Figure 11), the user had to click, for five times, a small button appearing in five random positions. The size of the but-ton was the same as that of the “close window” button of folders in MS Windows XP.

For both TA and TB, the dependent variable was the time to complete the task (on a single button). Moreover, we introduced a binary sub-variable “Success”, whose value was 1 if the user finished the task correctly within a timeout of 30 seconds, 0 oth-erwise. “Correctly” means that no wrong operations were per-formed (such as, for example in TA, opening the wrong folder). For TB, we used a further variable, “Number of Attempts”, which measured the number of clicks generated until the button was correctly pressed (unless the timeout was reached).

In order to compare ceCursor with the more “traditional” way of interacting with interfaces through the eyes, we also imple-mented a simple cursor (simpleC in the following) which merely displayed an empty small square where the user’s gaze was per-ceived. For the equivalent of a mouse double-click to be gener-ated, 100 consecutive gaze samples (i.e. a dwell time of two seconds) had to be detected within a circle with a radius of 10 pixels; the click was centered on the point with coordinates giv-en by the mean values of the acquired samples.

For test TB, we employed two versions of ceCursor, one with the replica of the currently pointed area — we will simply indi-cate this version with ceCursor — and one without the replica, like in the first implementation — we will call this other version ceCursorWR. For both cases, parameter values used in the expe-riments were the following:

Cursor size: 351 pixels (a relatively big cursor, since all the testers in experiment E1 were new to eye gaze input and had a very short training period)

Number of samples to be perceived within the central square for the first click to be generated: 60 (dwell time of a little more than one second)

a b

Figure 10 Folders used for test TA

Figure 11 Panel used for test TB

Figure 9 ceCursor is automatically shifted to the upper border of the screen

335

Page 6: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

Number of samples for the second click (double-click) to be generated: 60

Number of samples outside the cursor area for the cursor to move there (both in an area with icons and not): 60

Number of samples on a direction button for the cursor to move in that direction (both in an area with icons and not): 60

Each tester tried both TA and TB. For TA, only simpleC and ceCursor were used (since in areas with icons there are no repli-cas), while for TB all the three cursors were employed. Cursor order was randomized. Screen resolution was 1280x1024.

In E1, prior to the actual test session each tester was clearly ex-plained how to use the cursors and assisted in exercising with them (five minutes for each one, thus resulting in a 15 minutes total training time). The two testers of E2 could instead exercise with the three cursors for a much longer time — 15 minutes each, with a total training period of 45 minutes.

5.2 Experiment E1 - Test TA

Task 1: opening a folder within a folder containing big icons. With both simpleC and ceCursor, all the testers succeeded in the task. A repeated-measures ANOVA (within-subjects design) did not show a clear relation between cursor type and times (F=3.37, p=.1), but the means were significantly different (4.03 sec for simpleC and 8.14 sec for ceCursor). As could be expected, with big elements that are sufficiently separated each other simpleC can provide good results in terms of time to complete the task: if the user is able to maintain the gaze adequately focused on a (large) target, there is no real need to use mechanisms for pre-cisely tuning the position of the cursor.

Task 2: opening a folder within a folder containing small icons. In this case, all the testers succeeded with ceCursor, but only two out of nine (22.22%) managed to open the small folder with simpleC: the trembling behavior of this cursor makes it ex-tremely difficult to aim at small targets. When successful, simp-leC was relatively fast (mean of 4.2 sec for the two positive tri-als, versus 11.95 sec for the nine positive outcomes of ceCur-sor), but cannot be used for reliable pointing.

5.3 Experiment E1 - Test TB

Considering a time value of 31 seconds when the timeout of 30 seconds was reached (i.e. the trial was unsuccessful), a repeated-measures ANOVA did not show any relation between cursor type and time to complete the task (F=.86, p=0.43). Nonetheless, although means were similar (15.32 sec for simpleC, 14.64 sec for ceCursorWR and 13.1 sec for ceCursor), ceCursor showed a slightly better performance.

Looking at success percentages (73.33% for simpleC, 93.33% for ceCursorWR and 97.78% for ceCursor), it is clear that ce-Cursor resulted a little more effective than its counterpart with-out the replica — and much more effective than the basic cursor. This becomes even more evident if we consider the number of clicks generated until button press (or until the available 30 seconds were over). A repeated-measures ANOVA showed a plain relation between cursor type and number of clicks (F=26.39, p<.001), with mean values of 4.33 for simpleC, 1.44 for ceCursorWR and 1.2 for ceCursor.

5.4 Experiment E2 - Test TA

Task 1: opening a folder within a folder containing big icons. With both cursors, the testers succeeded in the task. Times measured with simpleC were 5.1 sec for the first tester and 3.5 for the second (mean: 4.3). Times measured with ceCursor were 3.6 sec for the first tester and 4.5 of the second (mean: 4.05). Comparing these values with the corresponding means for the same test and task of experiment E1 (4.03 and 8.14 for the two cursors, respectively), it is evident how in the two cases the per-formances of simpleC are similar, while they are very different for ceCursor (Figure 12a): it seems that a longer training period can actually help speeding up the pointing action.

Task 2: opening a folder within a folder containing small icons. None of the two testers succeeded in the task with simp-leC (they both opened the wrong folder). Despite the extended training time, the pointing precision is so limited that opening the right folder becomes probably a matter of pure chance. Defi-nitely better results were instead provided by ceCursor: 4 and 8.5 sec, with a mean of 6.25 sec. Comparing this value with the corresponding mean for the same test and task of experiment E1 (11.95 sec), also in this case a longer training period seems to be helpful (Figure 12b).

5.5 Experiment E2 - Test TB

Considering a time value of 31 seconds when the timeout of 30 seconds was reached, the following results (average times to click the button, in seconds) were obtained.

simpleC: Tester 1 13.91, Tester 2 15.73, Tester 1 + Tester 2 14.82 (four successful trials out of five for both Tester 1 and Tester 2).

ceCursorWR: Tester 1 8.64, Tester 2 13.8, Tester 1 + Tester 2 11.22 (all successful trials).

ceCursor: Tester 1 6.7, Tester 2 10.86, Tester 1 + Tester 2 8.78 (all successful trials).

As can be seen, while the mean time for simpleC is about the same as for experiment E1, for ceCursorWR and ceCursor sig-nificant reductions can be noted (Figure 13). Moreover, in this case too, ceCursor provided a better performance compared to ceCursorWR.

As for the number of clicks generated until button press (or until the available 30 seconds were over), while only one attempt was necessary with both ceCursorWR and ceCursor, an average of

0,00

2,00

4,00

6,00

8,00

10,00

simpleC ceCursor

E1

E1

E2 E2

0,00

5,00

10,00

15,00

ceCursor

E2E1

a b Figure 12 Test TA: results of experiment E1 vs. results of

experiment E2 (mean times) – (a) task 1, (b) task 2

336

Page 7: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

5.2 attempts for Tester 1 and of 6.2 for Tester 2 were needed with simpleC. In a real usage scenario with MS Windows appli-cations, employing simpleC would mean having a very high probability to click the wrong target.

5.6 User Preference

Both in E1 and E2, at the end of the experiments the testers were asked to express a preference regarding the three cursors. In E1, eight testers out of nine said to prefer ceCursor, and one ceCur-sorWR. In E2, both the testers said to prefer ceCursor.

6 Conclusions

Reliable eye pointing in ordinary operating environments is a challenging problem. Small graphical elements need specific mechanisms for precise selection: as demonstrated by our tests, and easily guessed by anybody who has experienced eye gaze input, a trembling cursor (visible or not) can only be used when targets are sufficiently wide and spaced each other.

ceCursor has been designed with the purpose to allow potential-ly any interface element in MS Windows to be effectively se-lected. Its simple structure, made up of a central pointing area and of four direction buttons around it, implicitly suggests its use. Compared with strategies based on zooming, ceCursor has the advantage of not requiring any desktop enlargement or de-formation, which, if frequent, may be annoying for the user.

One distinctive feature of the solution adopted for ceCursor is its different behavior according to the “context”: it can be moved discretely in areas containing icons and continuously within application windows and in “icon-free” areas on the desk-top. This makes it simpler and faster for the user to accomplish tasks of different kinds, such as opening applications, navigating in the folders structure, selecting links within web pages, press-ing small control buttons, etc.

Another distinguishing characteristic of our approach is (for continuous motion) the replica of the currently pointed area within direction buttons. This strategy has proved to be very helpful for precise pointing, allowing the user not too loose the “context” of the cursor (i.e. what is being aimed at a certain moment), without the need for constant shifts between the cen-tral square and direction buttons. As our tests have shown, the performance of ceCursor is generally better than that of its counterpart without the replica. Once the pointing mechanism is clear (step 1: identification of a definite target, step 2: search for that target in the direction button), there is little chance to be

confused by content duplication within the cursor: in our expe-riments, ten out of eleven testers said to prefer this solution.

Our tests also show that times to accomplish the pointing tasks exhibit a decreasing trend with increase of the training period. Although we were not able to implement experiment E2 with the same number of testers as in experiment E1, the tendency seems to be this. Moreover, times could be further reduced by dimi-nishing cursor size (especially in test TB, ceCursor was occa-sionally “captured” by screen borders) and by lowering dwell times.

Acknowledgement

This work was supported by funds from the Italian FIRB project “Software and Communication Platforms for High-Performance Collaborative Grid” (grant RBIN043TKY).

References

ASHMORE, M., DUCHOWSKI, A. T., AND SHOEMAKER, G. 2005. Efficient Eye Pointing with a Fisheye Lens. In Proceedings of Graphics Interface, Victoria, British Columbia, 203-210.

BATES, R. 1999. Multimodal Eye-Based Interaction for Zoomed Target - Selection on a Standard Graphical User Interface. In Proceedings of Interact'99, vol. II 7-8, Edinburgh, Scotland, UK, British Computer Society, 7-8.

BATES, R., AND ISTANCE, H. 2002. Zooming interfaces!: enhanc-ing the performance of eye controlled pointing devices. In Pro-ceedings of the 5th International ACM Conference on Assistive Technologies, Edinburgh, Scotland, UK, 119-126.

BLANCH, R., AND ORTEGA, M. 2009. Rake Cursor: Improving Pointing Performance with Concurrent Input Channels. In Pro-ceedings of CHI 2009, Boston, MA, USA, 1415-1418.

CHIN, C. A., BARRETO, A., CREMADES, J. G., AND ADJOUADI, C. M. 2008. Integrated electromyogram and eye-gaze tracking cur-sor control system for computer users with motor disabilities. Journal of Rehabilitation Research & Development, Vol. 45, No. 1, 161-174.

ENGBERT, R., AND KLIEGL, R. 2004. Microsaccades Keep the Eyes’ Balance During Fixation. Psychological Science, Vol. 15, No. 6, 431-436.

GORODNICHY, D. O., AND ROTH, G. 2004. Nouse ‘use your nose as a mouse’ perceptual vision technology for hands-free games and interfaces. Image and Vision Computing, Vol. 22, No. 12, 931-942.

KUMAR, M., PAEPCKE, A., AND WINOGRAD, T. 2007. EyePoint: Practical Point and Selection Using Gaze and Keyboard. In Pro-ceedings CHI 2007, 421-430.

KUMAR, M., KLINGNER, J., PURANIK, R., WINOGRAD, T., AND PAEPCKE, A. 2008. Improving the Accuracy of Gaze Input. In Proceedings of ETRA 2008, Savannah, GA, USA, 65-68.

LANKFORD, C. 2000. Effective Eye Gaze Input into Windows. In Proceedings of ETRA 2000, Palm Beach Garden, FL, USA, ACM, 23-27.

0,00

2,00

4,00

6,00

8,00

10,00

12,00

14,00

16,00

simpleC ceCursorWR ceCursor

E1E1

E1E2

E2

E2

Figure 13 Test TB: results of experiment E1 vs. results of experiment E2 (mean times)

337

Page 8: Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Environments

NORRIS, G., AND WILSON, E. 1997. The Eye Mouse, an eye communication device. In Proceedings of the 23rd IEEE North-east Bioengineering Conference, Durham, NH, USA, 66-67.

RÄIHÄ, K., AND ŠPAKOV, O. 2009. Disambiguating Ninja Cursors with Eye Gaze. In Proceedings of CHI 2009, Boston, MA, USA, 1411-1414.

SIRILUCK, W., KAMOLPHIWONG, S., KAMOLPHIWONG, T., AND SAE-WHONG, S. 2007. Blink and Click. In Proceedings of the 1st International Convention on Rehabilitation Engineering & As-sistive Technology, Singapore, 43-46.

SMITH, B. A., HO, J., ARK, W., AND ZHAI, S. 2000. Hand eye coordination patterns in target selection. In Proceedings of ETRA 2000, Palm Beach Garden, FL, USA, 117-122.

TOBII 2009. MyTobii User Manual, Version 2.4. Available:

http://www.tobii.com/assistive_technology/support_downloads/downloads.aspx (retrieved September 18th, 2009).

TOBII Technology AB 2004. Tobii 50 Series – Product Descrip-tion. Available: http://tobii.se/downloads/Tobii_50series _PD_Aug04.pdf, retrieved: October 5, 2009.

YARBUS, A. L. 1967. Eye Movements and Vision. New York: Plenum Press.

ZHAI, S., MORIMOTO, C., AND IHDE, S. 1999. Manual And Gaze Input Cascaded (MAGIC) Pointing. In Proceedings of CHI 1999, Pittsburgh, PA, USA, 246-253.

ZHANG, X., REN, X., AND ZHA, H. 2008. Improving Eye Cursor’s Stability for Eye Pointing Tasks. In Proceedings of CHI 2008, Florence, Italy, 525-534.

338