input devices, interaction techniques, and interaction tasks

Input Devices, Interaction Techniques,

and Interaction TasksChapter 8Chapter 8

Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks

2

High-quality user interfaces are in many ways High-quality user interfaces are in many ways the last frontier in providing computing to a the last frontier in providing computing to a wide variety of users.wide variety of users.

The quality of user interface often determines The quality of user interface often determines whether users enjoy or despise a systemwhether users enjoy or despise a system

We focus in this chapter on input devices, We focus in this chapter on input devices, interaction technologies, and interaction tasks.interaction technologies, and interaction tasks. These are the basic building blocks from which user These are the basic building blocks from which user

interfaces are constructed.interfaces are constructed.


3

1. Interaction Hardware

Here we introduce some interaction devices not Here we introduce some interaction devices not covered in Section 4.5, elaborate on how they covered in Section 4.5, elaborate on how they work, and discuss the advantages and work, and discuss the advantages and disadvantages of various devices.disadvantages of various devices. advantages and disadvantages can be discussed on advantages and disadvantages can be discussed on

three levels: three levels: • device -- hardware characteristicsdevice -- hardware characteristics

• task -- we might compare interaction techniques uding task -- we might compare interaction techniques uding different devices for the same taskdifferent devices for the same task

• dialogue -- we consider not just individual interaction tasks dialogue -- we consider not just individual interaction tasks but also sequences of tasks.but also sequences of tasks.


4

1.1 Locator Devices

It is useful to classify locator devices It is useful to classify locator devices according to three independent according to three independent characteristics:characteristics: absolute or relativeabsolute or relative direct or indirectdirect or indirect discrete or continuousdiscrete or continuous


5

Absolute devicesAbsolute devices have a frame of reference or origin, and report have a frame of reference or origin, and report

positions with respect to that originpositions with respect to that origin examples include: data tablets or touch panelexamples include: data tablets or touch panel

Relative DevicesRelative Devices have no absolute origin and report only changes have no absolute origin and report only changes

from their former positionfrom their former position Examples include: mice, trackballs, and velocity-Examples include: mice, trackballs, and velocity-

control joystickscontrol joysticks


6

Direct DeviceDirect Device the user points directly at the screen with a finger or the user points directly at the screen with a finger or

surrogate finger.surrogate finger. Examples include a touch screenExamples include a touch screen

Indirect DeviceIndirect Device the user moves a cursor on the screen using a device the user moves a cursor on the screen using a device

not on the screen.not on the screen. Examples include: a tablet, mouse, or joystickExamples include: a tablet, mouse, or joystick


7

Continuous DeviceContinuous Device one in which smooth hand motion can create a one in which smooth hand motion can create a

smooth cursos motion.smooth cursos motion. Examples include: tablets, joysticks, and miceExamples include: tablets, joysticks, and mice These typically allow more natural, easier, and These typically allow more natural, easier, and

faster cursor movement than do discrete devices.faster cursor movement than do discrete devices.

Discrete DeviceDiscrete Device Examples include: cursor control keys.Examples include: cursor control keys.


8

Speed of cursor positioning with a continuous Speed of cursor positioning with a continuous device is affected by the device is affected by the control-to-display control-to-display ratioratio, commonly called the C/D ratio., commonly called the C/D ratio. It is the ratio between hand movement (the control) It is the ratio between hand movement (the control)

and cursor movement (the display)and cursor movement (the display) A large ratio is good for accuracy, but makes rapid A large ratio is good for accuracy, but makes rapid

movements tedious.movements tedious. A small ratio is good for speed, but not for accuracy.A small ratio is good for speed, but not for accuracy. Fortunately, for a relative positioning device the Fortunately, for a relative positioning device the

ratio does not have to be constant, but can be ratio does not have to be constant, but can be changed adaptively.changed adaptively.


9

1.2 Keyboard Devices

The well known QWERTY keyboard has The well known QWERTY keyboard has been with us for many years.been with us for many years.

It is ironic that this keyboard was originally It is ironic that this keyboard was originally designed to designed to slow downslow down typists, so that the typists, so that the typewriter hammers would not be so likely typewriter hammers would not be so likely to jam.to jam.


10

1.3 Valuator Devices

Some valuators are bounded (like the Some valuators are bounded (like the volume control on a radio).volume control on a radio).

A continuous-turn potentiometer can be A continuous-turn potentiometer can be turned an unbounded number of times in turned an unbounded number of times in either direction.either direction. given an initial value, it can return absolute given an initial value, it can return absolute

values, otherwise it returns a relative value.values, otherwise it returns a relative value.


11

1.4 Choice Devices

Function keys are common choice devices.Function keys are common choice devices. Their placement affects their usability:Their placement affects their usability:

keys on the CRT are harder to use than keys on keys on the CRT are harder to use than keys on the keyboard, or in a nearby separate unit.the keyboard, or in a nearby separate unit.

A foot switch can be used in applications in A foot switch can be used in applications in which the user’s hands are engaged yet a single which the user’s hands are engaged yet a single switch closure must be frequently made.switch closure must be frequently made.


12

1.5 Other Devices

Here we discuss some of the less common, Here we discuss some of the less common, and in some ways experimental, 2D and in some ways experimental, 2D interaction devices.interaction devices. voice recognizersvoice recognizers pressure sensitive data tablets.pressure sensitive data tablets.


13

1.6 3D Interaction Devices

Some 2D interaction devices are readily Some 2D interaction devices are readily extended to 3D.extended to 3D. joysticks can have a shaft that twists for a third joysticks can have a shaft that twists for a third

dimensiondimension trackballs can be made to sense rotation about trackballs can be made to sense rotation about

the vertical axis in addition to that about the the vertical axis in addition to that about the two horizontal axes.two horizontal axes.


14

A number of devices can record 3D hand A number of devices can record 3D hand movementsmovements The data glove records hand position and orientation The data glove records hand position and orientation

as well as finger movement.as well as finger movement.

FIGURE 8.2FIGURE 8.2


15

Several other technologies can be used to Several other technologies can be used to record 3D positions.record 3D positions. In one, using optical sensors, LEDs are mounted on In one, using optical sensors, LEDs are mounted on

the user the user • (either at a single point, such as the finger tip, or all over (either at a single point, such as the finger tip, or all over

the body, to measure body movement)the body, to measure body movement)

• Light sensors are mounted high in the corners of a Light sensors are mounted high in the corners of a semidarkened room in which the user works, and each semidarkened room in which the user works, and each LED is intensified in turn.LED is intensified in turn.

• The sensors can determine the plane in which the LED The sensors can determine the plane in which the LED lieslies

In another, a television camera records hand In another, a television camera records hand movementsmovements

• image processing techniques are used to find the position image processing techniques are used to find the position of the fingers and hand. of the fingers and hand.


16

2. Basic Interaction Tasks (BITs)

With a basic interaction task, the user of an With a basic interaction task, the user of an interactive system enters a unit of information interactive system enters a unit of information that is meaningful in the context of the that is meaningful in the context of the application.application.

BITs are indivisible. We discuss composite BITs are indivisible. We discuss composite interaction tasks in the next section.interaction tasks in the next section.

Examples include: positioning, selecting, Examples include: positioning, selecting, entering text, and entering numeric quantities.entering text, and entering numeric quantities.


17

2.1 The Position Interaction Task

The positioning task involves specifying a (x,y) The positioning task involves specifying a (x,y) or an (x,y,z) position to the application or an (x,y,z) position to the application program.program. The customary interaction techniques for carrying The customary interaction techniques for carrying

out this task involve either moving a screen cursor out this task involve either moving a screen cursor to the desired location and then pushing a button, or to the desired location and then pushing a button, or typing the desired position’s coordinates on either a typing the desired position’s coordinates on either a real or simulated keyboard.real or simulated keyboard.

The positioning device can either be direct or The positioning device can either be direct or indirect, continuous or discrete, absolute or relative.indirect, continuous or discrete, absolute or relative.


18

There are two types of positioning tasks, spatial There are two types of positioning tasks, spatial and linguistic.and linguistic.

In a spatial positioning task, the user knows where In a spatial positioning task, the user knows where the intended position is, in spatial relation to nearby the intended position is, in spatial relation to nearby elements (as in drawing a line between two elements (as in drawing a line between two rectangles, or centering an object between two rectangles, or centering an object between two others).others).

In a linguistic positioning task, the user knows the In a linguistic positioning task, the user knows the numeric values of the (x,y) coordinates of the numeric values of the (x,y) coordinates of the positionposition


19

2.2 The Select Interaction Task -- Variable-Sized Set of Choices

The selection task is that of choosing an element The selection task is that of choosing an element from a choice set.from a choice set. Typical sets are commands, attribute values, object Typical sets are commands, attribute values, object

classes, object instances, …classes, object instances, … Some interaction techniques can be used to select from Some interaction techniques can be used to select from

any of these four types of choice sets; others are less any of these four types of choice sets; others are less general.general.

• For example, pointing at a visual representation of a set element For example, pointing at a visual representation of a set element can serve to select it, no matter what the set typecan serve to select it, no matter what the set type

• On the other hand, it is difficult to use function keys, since the On the other hand, it is difficult to use function keys, since the size of the choice set is variable, and often is large. (larger than size of the choice set is variable, and often is large. (larger than the number of function keys.)the number of function keys.)


20

We use the terms We use the terms (relatively) fixed-size choice (relatively) fixed-size choice setset and and varying-sized choice setvarying-sized choice set The first characterizes command, attribute, and The first characterizes command, attribute, and

object-class choice setsobject-class choice sets• The term relatively is used because the size of the set can The term relatively is used because the size of the set can

change, but it will not change frequently, or very much.change, but it will not change frequently, or very much.

Varying-sized choice sets, on the other hand, can Varying-sized choice sets, on the other hand, can become quite large, and can change frequently.become quite large, and can change frequently.

In this section, we discuss techniques that are In this section, we discuss techniques that are particularly well suited to potentially large particularly well suited to potentially large varying-sized choice sets. In the next section varying-sized choice sets. In the next section we cover techniques for fixed-size choice sets.we cover techniques for fixed-size choice sets.


21

Selecting Objects by NamingSelecting Objects by Naming The user can type the choice’s name.The user can type the choice’s name.

But they may not know, or have no reason to know But they may not know, or have no reason to know the objects name.the objects name.

Nevertheless this technique is useful in several Nevertheless this technique is useful in several situations.situations.

• If you know the name, it may be easier & faster to use it.If you know the name, it may be easier & faster to use it.

If naming by typing is necessary, feedback can If naming by typing is necessary, feedback can help expedite the process.help expedite the process. A useful form of feedback is autocompletionA useful form of feedback is autocompletion


22

Selecting Objects by Pointing Selecting Objects by Pointing Any of the pointing techniques mentioned in Any of the pointing techniques mentioned in

the intro to this section can be used to select an the intro to this section can be used to select an object.object.

If an object has multiple levels of hierarchy (as If an object has multiple levels of hierarchy (as does a robot), you may want multiple select does a robot), you may want multiple select commands (select_robot, select_arm,…)commands (select_robot, select_arm,…)

If the number of hierarchies is large and If the number of hierarchies is large and unknown, at least two commands are needed:unknown, at least two commands are needed: up_hierarchyup_hierarchy down_hierarchydown_hierarchy


23


24

2.3 The Select Interaction Task -- Relatively Fixed-Size Choice Set

Menu selection is one of the richest Menu selection is one of the richest techniques for selecting from a relatively techniques for selecting from a relatively fixed-size choice set.fixed-size choice set.

In this section we will discuss several key In this section we will discuss several key factors in menu designfactors in menu design


25

Single-level vs. hierarchical designSingle-level vs. hierarchical design The set is too large to display all at once, what The set is too large to display all at once, what

do you do?do you do? Scroll bar on a menuScroll bar on a menu

hierarchical menu.hierarchical menu.


26

Menu placementMenu placement Menus can static and permanently visible, or Menus can static and permanently visible, or

can appear dynamically on request. (tear-off, can appear dynamically on request. (tear-off, appearing, pop-up, pull-down, and pull-out)appearing, pop-up, pull-down, and pull-out)


27

Current selectionCurrent selection If a system has the concept of a currently If a system has the concept of a currently

selected element of a choice set, a radio button selected element of a choice set, a radio button selection technique is one way to implement selection technique is one way to implement this.this.


28

Size and shape of menu itemsSize and shape of menu items Pointing accuracy and speed are affected by the Pointing accuracy and speed are affected by the

size of each individual item.size of each individual item. Large items are faster to selectLarge items are faster to select Smaller items take less space, and permit more Smaller items take less space, and permit more

menu items to be displayed in the fixed area, but menu items to be displayed in the fixed area, but they induce more errors during selection.they induce more errors during selection.

Thus you have a tension between the two in Thus you have a tension between the two in your designyour design


29

Pattern RecognitionPattern Recognition In selection techniques involving pattern In selection techniques involving pattern

recognition, the user makes a sequence of recognition, the user makes a sequence of movements with a continuous-positioning movements with a continuous-positioning device, such as a tablet or mouse.device, such as a tablet or mouse. Proofreader’s marks indicating delete, capitalize, Proofreader’s marks indicating delete, capitalize,

move, and so on are attractive candidates for this move, and so on are attractive candidates for this approachapproach

Advances in these type of algorithms have led Advances in these type of algorithms have led to the explosion in handheld PDAs (palm,…)to the explosion in handheld PDAs (palm,…)


30

Function keysFunction keys Elements of the choice set can be associated Elements of the choice set can be associated

with function keys with function keys We can think of every key as a function keyWe can think of every key as a function key Unfortunately there never seem to be enough keys Unfortunately there never seem to be enough keys

to go around.to go around.

The keys can be used in a hierarchical-selection The keys can be used in a hierarchical-selection fashion, or their meanings can be altered using fashion, or their meanings can be altered using chords of keys.chords of keys.


31

2.4 The Text Interaction Task

The text-string input task entails entering a The text-string input task entails entering a character string to which the application does character string to which the application does not ascribe any special meaning.not ascribe any special meaning. Typing a command name is not a text-entry task.Typing a command name is not a text-entry task. Typing legends for a graph and typing text into a Typing legends for a graph and typing text into a

word processor are.word processor are. Clearly the most common text-input technique is Clearly the most common text-input technique is

the use of the QWERTY keyboard.the use of the QWERTY keyboard.


32

2.5 The Quantify Interaction Task

The quantify interaction task involves The quantify interaction task involves specifying a numeric value between some specifying a numeric value between some minimum and maximum value.minimum and maximum value.

Typical interaction techniques are Typical interaction techniques are typing the valuetyping the value setting a dial to the valuesetting a dial to the value using an up-down counter to select the valueusing an up-down counter to select the value


33

One means of entering values is a One means of entering values is a potentiometer (pot). potentiometer (pot). Then you have to decide on a rotary or linear one.Then you have to decide on a rotary or linear one.


34

2.6 3D Interaction Tasks

Two of the four interaction tasks described Two of the four interaction tasks described previously for 2D applications become more previously for 2D applications become more complicated in 3D: position and selectcomplicated in 3D: position and select

The first part of this section deals with The first part of this section deals with positioning and selection, which are closely positioning and selection, which are closely related.related.


35

This figure shows a common way to position in This figure shows a common way to position in 3D. 3D.

By providing three projections of an object, it is By providing three projections of an object, it is sometimes easier to manipulate it.sometimes easier to manipulate it. It is also common to provide an oblique/perspective It is also common to provide an oblique/perspective

view of the scene.view of the scene.


36

As with locating and selecting, the issues in 3D As with locating and selecting, the issues in 3D rotation are rotation are understanding depth relationshipsunderstanding depth relationships mapping 2D interaction devices into 3Dmapping 2D interaction devices into 3D ensuring stimulus-response compatibility.ensuring stimulus-response compatibility.


37

An easily implemented 3D rotation technique An easily implemented 3D rotation technique provides slider dials or gauges that control provides slider dials or gauges that control rotation about the three axes.rotation about the three axes.


38

It is often necessary to combine 3D interaction It is often necessary to combine 3D interaction tasks.tasks.

Thus, rotation requires Thus, rotation requires a select task for the object to be rotateda select task for the object to be rotated a position task for the center of the rotationa position task for the center of the rotation and an orient task for the actual rotationand an orient task for the actual rotation


39

3. Composite Interaction Tasks

CITs are built on top of Basic Interaction Tasks CITs are built on top of Basic Interaction Tasks described in the previous section, and are described in the previous section, and are actually a combination of BITsactually a combination of BITs

There are basically three major forms of CITsThere are basically three major forms of CITs dialogue boxes -- used to specify multiple units of dialogue boxes -- used to specify multiple units of

informationinformation construction -- used to create objects requiring two construction -- used to create objects requiring two

or more positionsor more positions manipulation -- used to reshape existing objectsmanipulation -- used to reshape existing objects


40

3.1 Dialogue Boxes

We often need to select multiple elements of a We often need to select multiple elements of a selection setselection set For example: text attributes (italic, bold, underline,For example: text attributes (italic, bold, underline,

…)…) Menu approaches are useful for a single one, but Menu approaches are useful for a single one, but

disappear when a selection is madedisappear when a selection is made

This problem can be overcome with dialogue This problem can be overcome with dialogue boxesboxes the box is typically dismissed explicitly with a the box is typically dismissed explicitly with a

command.command.


41

3.2 Construction Techniques

One way to construct a line is to have the user One way to construct a line is to have the user indicate one endpoint and then the other.indicate one endpoint and then the other. Then the line is drawn.Then the line is drawn.

A far superior technique is rubberbanding A far superior technique is rubberbanding (discussed in Chapter 2)(discussed in Chapter 2) when the user pushes a button, the beginning point is when the user pushes a button, the beginning point is

selected.selected. As the cursor moves, so does the endpoint of the line. As the cursor moves, so does the endpoint of the line. When the button is released, the endpoint is frozen.When the button is released, the endpoint is frozen.


42


43

An entire genre of interaction techniques is An entire genre of interaction techniques is derived from rubberband line drawing.derived from rubberband line drawing. Rubber-rectangle Rubber-rectangle

• Anchor one corner and the opposite corner is dynamically Anchor one corner and the opposite corner is dynamically lined to the cursor until the button is released.lined to the cursor until the button is released.

Rubber-circle Rubber-circle • The center is at the initial positionThe center is at the initial position

• The circle passes through the current cursor position.The circle passes through the current cursor position.

All these techniques have a common user-All these techniques have a common user-interaction sequence of button-down, move locator interaction sequence of button-down, move locator and see feedback, button-up.and see feedback, button-up.


44

3.3 Dynamic Manipulation

It is not sufficient to create lines, rectangles, It is not sufficient to create lines, rectangles, and so on. In many situations, the user and so on. In many situations, the user must be able to modify previously created must be able to modify previously created geometric entities.geometric entities.

Dragging, rotating, and scaling are Dragging, rotating, and scaling are important techniques that affect an entire important techniques that affect an entire object and the scene it is in.object and the scene it is in.


45

Click and DragClick and Drag dragging moves a selected symbol from one dragging moves a selected symbol from one

position to another under control of a cursor.position to another under control of a cursor.


46

HandlesHandles The concept of handles is useful to provide The concept of handles is useful to provide

scaling of an objectscaling of an object


47

Changing components of an object.Changing components of an object. Vertices could be named, and the user could Vertices could be named, and the user could

enter the new (x,y) coordinate.enter the new (x,y) coordinate. But the same point-and-drag strategy used to But the same point-and-drag strategy used to

move an entire object is more attractive.move an entire object is more attractive.


48

4. Interaction-Technique Toolkits.

The look and feel of a user-computer interface is The look and feel of a user-computer interface is determined largely by the collection of determined largely by the collection of interaction techniques provided for it.interaction techniques provided for it. Designing and implementing a good set of Designing and implementing a good set of

interaction techniques is time consuminginteraction techniques is time consuming Using interaction-technique toolkits helps ensure a Using interaction-technique toolkits helps ensure a

consistent look and feel among application programs, consistent look and feel among application programs, and is clearly a sound software-engineering practice.and is clearly a sound software-engineering practice.

Examples include: the Macintosh toolkit, Motif, Examples include: the Macintosh toolkit, Motif, OPEN LOOK, … OPEN LOOK, …


49

Summary

We have presented some of the msot important We have presented some of the msot important concepts of user interfaces:concepts of user interfaces: input devicesinput devices interaction techniquesinteraction techniques interaction tasksinteraction tasks

There are many more aspects of user interface There are many more aspects of user interface techniques and design, however, that we have not techniques and design, however, that we have not discussed.discussed. Foley, vanDam, et.al (the big white book) has a Foley, vanDam, et.al (the big white book) has a

thorough treatment of those topics. thorough treatment of those topics.

input devices, interaction techniques, and interaction tasks

Documents