Multisensory and Multimedia

Bill Moggridge – Designing Interactions

ID 501 – Kıvılcım Çınar

Content: 5 senses: Vision, hearing, touch, smell & taste Connection with computers “sensory deprived and

physically limited” explores the opportunities for interaction design to become

multisensory and to take advantage of multimedia.

Interviews:

Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT)

Durrell Bishop (Teacher of Product Design @ Royal College of Art in London)

Joy Mountford (Member of Human Interface Group @ Apple) Bill Gaver (Professor of Interaction Research @ Royal College of

Art in London)

Vision « Visions of Japan » by Arata Isozaki Realms of

Cliché (presented various entertainments (such arts as the tea ceremony, flower arrangement and popular and classical performing arts)

Kitsch (examined the nature of competitiveness in games and sports)

Simulation (demonstrated a dramatic expansion of the potential for visual display )

The images produced by these systems are displayed on screens but they are completely separated from the real things themselves—processed, edited, and otherwise qualitatively changed, often intosomething completely different.

“blurring the line between real and simulated”

Hiroshi Ishii Physical & Digital world Interaction requires 2 key components:

Controls ( manipulate access to digital informations and computations)

Representation (people can perceive the results of computations)

Hiroshi Ishii Physical & Digital world Interaction requires 2 key components:

Controls ( manipulate access to digital informations and computations)

Representation (people can perceive the results of computations)

Intangible pixels- windows,menus,folders

To control representations- remote controls,mouse

Tangible representation (control mechanism) allow user to directly grap and manipulate.

Dynamic output or feedback(video, sound)

Tangible User Interfaces Ping Pong Plus(http://www.youtube.com/watch?v=AZO8sfmpKIQ&playnext=1&list=PL38930FA142C5C68F)

a ping pong table that could sense ball hits, which it used to control various visualizations projected on the table.

Music Bottleshttp://www.youtube.com/watch?v=U4IYyNL4ld8&feature=related

bottles as containers and controls for digital information

bottles enable each one to be wirelessly identified

simple affordance of opening and closing

Durrell Bishop

Things should be themselves. (self-

evident - to describe what they are

actually doing)

Monopoly versus Video Tape Recorder

Monopoly makes the functions of the game self-

evident.

There is a physical representation of the different

elements of the software of the game.

For VTR, you might want to know whether it is

recording or not, how much space there is left, or

where things are.

Physical object represent meaning.. We remember the things,

experience them repeatedly Unusual or surprising to our memory

Example: coin (has value – has country, ownership)

«...make virtual items, both space and objects, seem just as real as physical items by designing them to say more about themselves.»

physical object pointer to a window or folder,

the physical environment is just as real as the screen environment

Marble Answering Machine prototype telephone answering

machine. Incoming voice messages are

represented by marbles, the user can grasp and then

drop to play the message dial the caller automatically.

It shows that computing doesn’t have to take place at a desk, but it can be integrated into everyday objects

The Marble Answering Machine demonstrates the great potential of making digital information graspable.

http://ccdc11.wordpress.com/2011/02/15/durrell-bishops-marble-answering-machine/

• He suggests that physical tags

can be designed to represent any

abstract item, as long as we can

remember what they mean by

recognizing their form and behavior.

• The marbles are chosen to

physical represent incoming phone

messages which demonstrated the

great potential of making digital

information.

Joy Mountford Quick Time Mountford & Mike Mills developed designs to include images

and videos in graphical user interfaces, integrating QuickTime and moving the personal computer towards multimedia.

Personal computers started to grow. Bringing video to personal computer turn the PC into

multimedia device. Paper animation techniques was replaced by Quick Time.

People started to be familiar with film

and video editing tools and

techniques.

( edit clips of their children)

Hand controller Navigable movies ( became Quick

Time VR ) – Golden Gate Bridge had a video camera pointing at an object,

seemingly doing the same thing day after day. You could move around the camera’s view of the

room just like you could manipulate a 3-D view of an object with the virtual sphere.

http://www.panoramas.dk/US/golden-gate.html

The Bead Box sound should be used more in

interface design, integrating music, voice, and more sophisticated sound effects.

70 percent of the population think they’re amateur musicians, which is pretty high.

age – type of music listened Gives opportunities to people who

want to make music but have not mastered a conventional instrument notation or the meaning of a score.

Bill Gaver studied sound perception his work is about auditory icons meet Joy Mountford – started to work as an intern at Apple developed SonicFinder – tried to add sound to the software

of Finder when you clicked on a file, you would hear a tapping noise

History Tablecloth

The History Tablecloth uses

weight sensing, measuring how

long things have been left on the

surface.

Provide us to see history of

objects in the home

load sensors calculate product’s

position, and cells are lit to form

a halo that grows slowly over a

period of hours.

Key Table and Picture Frame

provide a place for people to leave

the things they carry with them when

out of the house—keys, mobile phone

contains a load sensor mounted

under the top surface

to measure the duration of oscillation

caused by a new load, a good

indication of impact force.

When objects are put on the Key

Table, this measurement is wirelessly

transmitted to the picture frame,

which tilts accordingly.