Human Perception &

Information Presentation

Kaan Başlı

2006638500

”It's not about making computers more human, but about removing the barriers between humans and computers” (B.J. Fogg)

Overview

• Motivation for ”perception and cognition” • Design possibilites • Perception

▫ Visual, auditiv, haptic, taste, smell ▫ Multimodal systems

• Cognition • Attention • Communication • Emotions

What motivates such knowledge?

• Knowledge of perception and cognition (already has and) will affect areas like AI, Ambient Intelligence, HCI, design, architecture and more…

Intelligent products

• ”An intelligent product is an everyday artefact, where computation is used to invisibly support or enhance its intended use”

(Lars Erik Holmquist)

Ambient Intelligence

• Ubiquitous computing (system integration, ad hoc-, wireless networking)

• Context awareness (sensors, tracking, position) • Intelligence (learning algorithms, user profiling,

recommenders, autonomous intelligence) • Natural user -system interaction, ambient

technologies, multimodal interaction, interaction styles…

European Symposium on Ambient Intelligence Nov 3-4, 2003

User perspective

• ”The users goal is not to interact with an intelligent product but to

▫ create, communicate, explore, plan, draw, design, learn…”

(Keyson)

Interaction Design is about:

• Defining the behavior of artifacts, environments, and systems (i.e., products) …and therefore concerned with: ▫ Defining the form of products as they relate to their

behavior and use ▫ Anticipating how the use of products will mediate

human relationships and affect human understanding ▫ Exploring the dialogue between products, people, and

contexts (physical, cultural, historical) (R. Reimann)

Cogntion & Perception

• Relates to how humans percieve their world, reason and act in it.

• Any design of products, intelligent or not, will be percieved and interacted with, based on people’s perception, cognition in a specific situation.

Cognition

• Processes that have a ”content” (Lundh, Montgomery, Waern, 2001):

▫ Perception

▫ Memory

▫ Language

▫ Thought

Perception

• The acqusition and processing of sensory information in order to see, hear , taste, smell, or feel objects in the world; also guides an organism’s actions with respect to those objects.

Sekuler & Blake (1994)

The desktop computer

• How are our senses used in the design?

• Acting by touch, getting visual feedback, ..

▫ sound feedback…

▫ touch feedback?

▫ taste, smell?

Consider the things we've given up in

the physical world which might be nice

to have back…

but augmented with computation and connectivity. Paintbrush and pencils and musical instruments; a single personal key that lets you into home, car and work and has a distinct feel as you insert it in a lock depending on whether your spouse, a friend or a stranger has been by in your absence; a bank card that feels as heavy as your account balance when you swipe it in the ATM.

Two ”perspectives” of how to use

knowledge of human perception • … for systems/artefacts that have attentive

and reactive users (HCI)

• …for designing attentive and reactive objects/systems (AI or not)

First perspective: Using perception in

design • Information visualisation (Tufte, 1990) • General HCI design guidelines, like

▫ Group similar information… ▫ Give proper feedback.. ▫ Minimalistic design…

Using perception in design • Affordances of objects (Norman, 1990)

▫ Constraints that guides use and can be physical, logical, semantic or cultural.

Affordances

Artefacts that ”imitate” perception

• Furby, A/Barney, Aibo, Spookies • Sensing capabilites similar to humans • Constrained sensing capabilites vs. capabilites

”beyond” humans

Back to our perception…

The acqusition and processing of sensory information in order to see, hear , taste, smell, or feel objects in the world; also guides an organism’s actions with respect to those objects.

Visual perception

• Ocularcentrism

• ”We live in a visual society”…

(electron micrograph by Scott Mittman)

Visual perception

• The brain assume that we live in a three-dimensional world

• The brain tries to find depth in everything that it sees.

• The brain assume that everything is constant in shape, color and size.

Visual perception example

Visual perception

• Organisation of objects

• Movement perception

• Spatial perception

• Perception of objects

(e.g. Gestalt Principles)

Gestalt Principles

• The principle of

▫ Proximity

▫ Similarity

▫ Good continuity

▫ Closure

▫ Movement

Gestalt Principles

• The principle of Proximity

▫ features which are close together are associated

Gestalt Principles

• The principle of similarity

Gestalt Principles

• The principle of good continuity

Gestalt Principles

• The principle of closure

▫ Close unfinished shapes

▫ When the information is diffuse or indistinct, our expectations will create the impression for us..

▫ Group information

Closure

Perception of objects

• How do we recognize something e.g. A chair as a chair?

• Figure/ground organization • Learned ”Stereotypes” or ”prototypes”

▫ a A a A a A ▫ Useful when designing icons

Spatial perception

• We perceive relative sizes and distance by using ▫ Perspective (e.g. Linear)

▫ Overlap

▫ Known sizes

▫ Texture gradients

▫ Shading

• Absolute distance- Relative distance

Movement perception

• The principle of Movement

▫ React fast on movement

▫ The eye is drawn towards movement

▫ Movement towards background

Hearing •Recognize identity of

soundsource

•Give information on the

nature of the environment

via echoes, reverberation

(normal room, cathedral,

open field)

Hearing example Multimodal system presented

at the Interact Conference’03:

•Non-visual exploration of digital

pictures (Root, 2003)

•Active auditory representation

•Passive auditory representation

(Verbal summary)

•Haptic representation (contour, surface

Smell

• Olfactory • Can distinguish different 10.000 smells • A chemical sense (A substance has to be

volatile…) • Recognition • influence mood, memory, emotions • We can actually communicate by smell

Smell example

• Fire fighting training, remote surgery, entertainment..

• Joseph Kaye (MIT)

▫ The olfactory display of Abstract Information

▫ Remember dinner at five… (smelling curry)

Taste

• Also a chemical sense… • Gustatory sensation • Substance must be soluable • Sour, salty sweet, bitter by tastebuds are also

located in cheeks, in the throat… • Recognition, Influence memory…

Taste example

• Food simulator, Siggraph Conference 2003

• Haptic displays displays with biting force

• Auditory, chemical sensations of eating

Tactile perception

• Haptics ▫ Thermoreceptors (temperature) ▫ Mechanoreceptors (pressure, vibration) ▫ Painreceptors

• Kinesthetics • Vision/audition usually dominate haptic

perception • Muscle memory

Haptics example

• Tangible interfaces

• Brygg Ullmer (MIT and ZIB)

▫ Tangible bits

▫ MetaDESK

▫ MediaBlocks

▫ Tangible Query Interfaces

Why use tactile information?

• Clarify ambiquity in dominant modality

• Some parameters not availible in dominant modality

• Continous control vs Discrete control

(Karon Maclean)

Multimodal distinctions

• Crossmodal, intermodal. One modality subconsciously influences perception in another modality

• Multimodal: an event is perceived and integrated by multiple senses

• Supramodal: phenomenon that applies to all senses.

• Grounded vs ungrounded interfaces

A multimodal example

Why use multimodal interaction?

• Avoid overkill: find most efficient path to desired result

• Exploit illusions: work around hardware limitations to clever compensation

• Design rules: control net percept in user • Ecological verity: respect perceptual latency

thresholds for perceived synchrony

But how many things can we attend

to in the same time?

Attention

• Selective ability

• Our brain compensates…

Change blindness

Attention

• Guidance of attention ▫ High level control ▫ Low level salience (movement) ▫ Can send the attention of the user to an appropriate

location at an appropriate time. ▫ Change blindness transitions can become effectively

invisible if attention is not drawn to them. ▫ Soft warning: User automatically sees what they

should see!

Cognition

• The world as an external memory

▫ Distributed cogntion (Hollan, Hutchins, Kirsh, 2000)

Culture, history & context affects distr cognition

▫ Situated cognition

▫ Embodied cognition

Cognition is also about language….

and thus communication

Communication

• Verbal, Nonverbal… • Sender Reciever • Language (speaking) are means for

communication • Technology can act as an ”amplifyer” of our

ability to communicate. Makes communication possible despite the distance e.g. Phones, sms Constraints/shapes the communication….

• People have to ”communicate” with a device in order to be able to create, explore, plan, draw, design, learn…”

An example…

• Amigo (instant messaging) vs. SMS

One last thing that affect cogntion…

and that is affected by our senses…

Emotions

• “If you want a golden rule that will fit everybody, this is it: Have nothing in your houses that you do not know to be useful, or believe to be beautiful.”

William Morris “The Beauty of Life,” 1880

Emotions

• Moods vs emotions towards an object

• Affective computing

▫ ”computing that relates to, arises from, or deliberately influences emotions”

Affective design…

• Robots and artificial creatures on computer displays are being designed to show more emotion. The goal is better communication with people.

• Panasonic’s Tama • Nec’s Robot R100

So, why try to understand cognition

and perception? • It affects how we percieve and interact with

devices around us, thus also what you intend to design.

• It can be used as inspiration to think beyond current interfaces…

Thank you for listening

Kaan Başlı

2006638500