1

CMT 3210, Week 9

Interaction As a Topic in Its Own Right

2

Topics

Paradigms for interactionDirect manipulation

Visual and spatial representations, visual thinking Object action interface model (OAI)

Conversation based interaction (command line style) Speech acts Conversation for action Structure of every day conversations

3

Paradigms for Interaction

Direct manipulation You act in the computer’s

world Act on work-domain

objects Windows, Mac Affordances: pressability,

dragability, turnability, etc

4

Paradigms for Interaction

“Conversation-based” interfaces You ask the computer to

act on your behalf Actions expressed in a

command language “Command line” interfaces

(Unix, DOS) Syntax is important

bob12> ls

letter.doc letter-old.doc

paper.dvi paper.log

paper.ps paper.tex

bob12> cp paper.ps backup.ps

bob12> lpr paper.ps

5

Characteristics of Direct Manipulation

Continuous representation of objects of interestPhysical actions or labelled buttons instead on

complex syntaxRapid incremental reversible operations, whose

impact is immediately visibleWYSIWYG principle: What you see is what you

get. (1990)

6

Visual & Spatial Representations

Spreadsheets (Visicalc, Lotus notes, Excel) (Video-) Games (PONG, Gameboy,…)Computer-aided Design (CAD)Computer-aided Manufacturing (CAM)Office automation (Word, Macintosh programs,

Windows) Information from databases.

7

Advantages of Direct Manipulation

Virtuality: Representation of reality that can be manipulated

Transparency User applies intellect to the task at hand, the tool

itself seems to disappear into the background

Problem solving and learning by using virtual/physical objects

Retention and manipulation of physical, spatial or visual objects is easier

8

Problems

Visual representations can be space consuming Consequences: scrolling or deeper layers

Users have to learn the meaning of visual representations e.g. Icons

Using a mouse can be slower than typing Visual representations can be ambiguous and

misleading Visual representations can be inappropriate for certain

type of information. Clarity: “Stop” is clearer than any sign Amount and detail of Information required: text

9

Structure of Visual Objects, Particularly Icons

Lexical qualities Pixel shape, colour, brightness, blinking

Syntactics Appearance and movement: lines, patterns, modular

parts, size, shape

Semantics Type of objects represented: concrete vs. Abstract,

part vs. Whole

10

Structure of Visual Objects, Particularly Icons

Pragmatics Overall legibility, utility, identifiability, memorability,

appeal

Dynamics Receptivity to clicks: Highlightening, dragging,

combining, etc.

11

Object Action Interface Model

Universe

atoms

Intention

steps

Objects Actions

Task

12

Object Action Interface Model

Metaphor

pixels

Plan

clicks

Objects Actions

Interface

13

Stages of Development (Piaget)

Sensorimotor (birth to 2 years) Touching, grabbing, smelling, tasting,…

Pre-operational (2 to 7 years) Differentiation of shapes, colours, texture, etc

Concrete operational (7 to 11 years) Manipulation of physical objects

Formal operations (11+ years ) Symbol manipulation

14

Conversation Based Interaction

User asks the computer to do something on his/her behalf

Syntax is importantCan we learn something for these interfaces

from language and interaction theories?

15

Language and Interaction

Linguists understand language on three levels: Syntax

the surface structure of a communication

Semanticswhat it means

Pragmaticswhat to do about it

16

Language and Interaction, Example 1

In the bar: “Do you have any peanuts?” Syntax: Sentence-Question, do-auxiliary verb, have-

verb, peanuts-noun (plural), etc. Semantics: Somebody asks for peanuts. Pragmatics: Give him/her peanuts, reject request,

ignore request, divert conversation,…

17

Language and Interaction Example 2:

From the compiler: “error 87 at line 36.” Syntax: incomplete sentence, error-noun, 87-number,

at-preposition, line-noun, 36-number. Semantics: the compiler detected an error, while

compiling the source code at line 36. (That does not necessarily mean that the mistake in the source code is at line 36.)

Pragmatics: go back to source code and check from line 36 backwards.

18

Language and Action

The “language/action perspective”: Speech Acts Language is used to do things, change the state of

the world, etc. (Not just to describe) Utterances are designed to carry out "speech acts" Speech acts used as part of dialogue between people Types of speech acts categorise the different

functions of utterances

19

Speech Acts….

Assertivescommit the speaker to something being the case

Directivesattempt to get the listener to do something

Commissivescommit the speaker to some course of action

Expressivesexpress a psychological state

Declarationsbring about a state of affairs

20

Speech Acts and Dialogue

A: two pints of lager please

B: coming right up

A: thanks A: two pints of lager please

B: the lager's off

A: I'll have bitter then

B: coming right up

A: thanksA: two pints of lager please

B: would you like bitter instead

A: don't think I'll bother

B: suit yourself

21

Conversations for Action

1 2 3 4 5

8

67

9

A: Request B: Promise B: Assert A: Declare

A: Counter

B: Counter

A: Reject

A: Renege

A: Declare

22

Potential Applications

Structure of human-computer dialogues Can we capture the range of human-computer

interactions in a similar way?Categorisations of actions? Or task sequences?

Computer support for human-human dialogues Specialised communication tools?

Knowing about the structure of conversations can allow special kinds of support to be provided

Project management tools

23

Questions:

Similarities and differences between human-human interaction and human-computer interaction. Similarities: messages going back and forth Differences: context sensitivity, non-verbal

communication, humor, ‘repair actions’ in case of misunderstanings, metacommunication, etc.

Are humans interacting with a computer or are they interacting with the organisation behind the computer (e.g. cash machines)?

24

Properties of Language

Speech acts considered too restrictive Study the way language used in ordinary

conversation:Openings, closings, adjacency pairs

25

Openings and Closings: Example

A: Do you want another

pint?

B: what's the time?

A: ten to ten

B: OK, yes thanks

Opening

Opening

Closing

Closing

26

Openings and Closings

bob12> cp paper.ps temp

Disc full

bob12> rm paper-old

Removed paper-old

bob12> cp paper.ps temp

copied paper.ps

bob12>

Opening

OpeningClosing

OpeningClosing

?

27

Adjacancy Pairs

Action-response pairs First part proposes some

joint task Second part may “take

up” task of the first Different forms of take-up

A: who is speaking?

B: it’s Bob Fields

C: is he going to the US?

D: well, I heard that ….

E: who is calling?

F: can I leave a message?

28

Design Applications?

Structure of human-computer conversations match openings and closings or categorise responses in dialogue using adjacency

pairs?

29

Summary

Paradigms for interaction: direct manipulation conversation-based / command-line

Visual representation and OAI modelsLanguage/action perspectiveConversation analysis

openings, closings and adjacency pairs

30

Further reading

E. Hutchins, J. Hollan & D. Norman (1986) Direct Manipulation Interfaces. In Norman & Draper (eds, 1986) User Centred System Design. LEA.

T. Winograd & F. Flores (1987) Understanding computers and cognition: a new foundation for design. Addison-Wesley.

H. Clark (1997) Using Language. CUP.