The History of Interaction I.

Dr Brigitta Zics brigitta.zics@ncl.ac.uk

http://www.youtube.com/watch?v=i-G54kVrhbE

Interactions

Human - Human Human – Technology – Human

Human – Technology ( Human – Computer Interaction: HCI)

Interaction as Mental Process

For Example: Art Work – Spectator Relationship

Definition of Interaction •  Interaction is understood as an action-reaction

potentiality that results not only in a dynamic change in the structure of the technological system but also in the user’s cognition. Thus the term interaction not only refers to a mental level between the user ( or participant)

and the computer (artwork) but also on a level of structural transformation in the system.

Definition of Interactive System

•  Interactive System: complexity of networked elements, which, through action-affection processes produce a

scheme of experience.

The “interactor” (Rokeby, 1995)

participant subject participant

user (active) spectator

•  Active spectator describes a participant subject who actively contributes in the „making art‟ (production)

process. •  The term extends the historical continuum of the

„artist-artwork-spectator‟ triangle whose application defines a primary understanding of art production.

•  This version of spectatorship acknowledges the importance of the spectator as the co-creator but

amplifies the idea of active spectator.

Interaction as Active Spectatorship in Art

artwork

spectator

interaction

computer

interaction

user

HCI ART

Interface

membrane or screen, the place where contact between two entities occurs

(human – machine, participant -system) 1972 – Alan Kay

Graphical User Interface (GUI)

1972 – Alan Kay (at Xerox PARC) introduced the idea of iconic, graphical representations of computing

functions – the folders, menus, and overlapping windows found on the desktop – based on his

research into the intuitive processes of learning and creativity.

•  Kay's research took root in the conviction that hypermedia, or "dynamic media" as he called it,

represented a profound departure from static media such as painting, television, photography, print publishing, and

film. •  He saw in hypermedia the radical interactivity that

would characterise. Copy this into the address bar of your browser: http://www.archive.org/details/AlanKeyD1987 http://www.archive.org/details/AlanKeyD1987_2

Alan Kay: Doing with Images Makes Symbols (1987) History of Interfaces as a Tool

Quality of Interaction •  Responsiveness:

The notion of responsiveness describes a reaction capability (such as feedback) and its various qualities in the system; this reveals the technological aspect of an interactive system. Practically, responsiveness appears on the technological surface of the system as the „interface‟, which is the medium on which the user is invited to act upon.

•  Immersion/embodiment: Immersion can be broadly defined as: „…the experience of being physically immersed within a virtual environment experience‟, as Blade and Padgett (2002, p.20) suggest. It is understood as a cognitive state of humans, where every faculty of the person is fixed on one phenomenon. More closely, immersion is a contemporary notion of interactive media, especially VR. In this manner, Oliver Grau (2003, p.13) exemplifies this in the context of cognition: „Immersion can be an intellectually stimulating process; however, in the present as in the past, in most cases immersion is mentally absorbing and a process, a change, a passage from one mental state to another.‟

Technology as Medium or

Technology as Tool

Interactive System: when both the tool and medium is interactive

Lansdown-model (1888): Technology as tool or medium

Technology as •  tool: in which the computer is used to help us to do

things that could be done more laboriously or more slowly by hand;

•  medium: in the way that printing, or oil painting, or lithography can be seen as a medium;

catalyst: in other words, as a way of inspiring new creative approaches to art and design;

smart apprentice: as assistant having enough intelligent to explore new possibilities and approaches when shown how to do so. (Lansdown, 1988, p. 147)

•  Tool: artist implements in the ‘making art’ process (exp. production process, software's, programming languages)

•  Medium: the surface what the user/spectator uses in the ‘experiencing art’ process (exp. Unique interfaces, projections, installations)

•  INTERACTIVE ART: art form which implements emerging interactive technologies as tool an medium

Applying Lansdown Model in Interactive Art

Gruivnhiturb

tool

medium

interface USER

Examples of Mixed Technologies

Tool: interactive Medium: traditional

William Latham’s The Evolution of Forms, 1990

Tool: traditional Medium: interactive

Digital Archiving

The History of Interaction as a Medium Emerging New Art Form: Interactive Art

Forerunners of Interactive Art

Participatory art form here stands for art works or movements which characterized an audience involvement

in the art work.

DADA The desire immediately interacting with the audience brought to art the

form of ‘art nugatory’ of dada movement at the beginning of twenty

century.

It haven’t existed an implemental technique, the artists generated a

new appearance form of public congregation, demonstration

FLUXUS MOVEMENT/HAPPENING •  Later from this movement (dada) and

John Cage’s influential chance performances was outgrowing the Fluxus ‘attitude’ which presented intimate relationship between artist performer and audience.

•  In the happenings (in live performances) artists carried out artistic ideas what performance’s outcome was rely on the spectator’s reactions.

John Cage MUSIC WALK Performed by Cage , David Tudor , Ono , and Mayuzumi Toshiro Sogetu Art Center , Tokyo 1962

JOHN CAGE •  A pioneer of chance music, electronic music and non-standard use of musical instruments • Participatory music

ALAN KAPROW Working from a carefully conceived and tightly scripted score, he created an interactive environment that manipulated the audience to a degree virtually unprecedented in 20th century art.

•  Medium + Tool is technological

•  Participatory characteristic: dada, John Cage, happenings

•  Emphasised the interaction between artist and spectator

 Interaction happen trough the  artist – artwork participant triangle

Interactive Art

Pioneers of Interactive Art: Myron Krueger

•  With the American Myron Krueger the development of computer-controlled Interactive Art started.

•  He studied computer science at the University of Wisconsin.

•  Krueger has been working together with artists since 1969.

•  He has been working on Videoplace since 1974. In that same year, he received his PhD in “Computer-Controlled Responsive

Environments”. •  He is the inventor of ARTIFICIAL REALITY.

Responsive Environment (Myron Krueger, 1977)

Krueger (2001, p.106) describes the responsive environment as follows: ‘… a computer perceives the actions of those who enter and respond intelligently through complex visual and auditory displays’.

Ivan Sutherland -- first virtual reality interfaces (1965) head-maintained displays (HMD) and goggles.

Krueger (1991, p.18) envisioned a new ‘reality’ without the physical presence of technology (as HMD was visible interface) focusing on the participant’s behavior in order to enable an ‘automated experience’.

Krueger wanted to involve broad audiences and neglected Sutherland’s interface

•  Closed-Circuit video installation

•  TESTBED for Interactive Installation

•  The interaction happens as a visitor enters the room. A camera captures video-images from the visitor, which after an abstraction are projected on the wall again. The visitor sees an graphically reduced mirror image on the wall.

•  Myron Krueger, who is located in another room, also gets this image on a screen. He can interact with the visitor, by drawing sketches with a graphic-tablet, which get merged with the image that is shown on the projection. Krueger can interact with the visitor via the video-projection.

Krueger: Metaplay

One game that evolved was when the artist would draw to make the participant feel that the participant was drawing. When the participant moved his hand, a line would be drawn. Thus, by moving his hand he could draw on the screen.

Krueger’s Videoplace

After several other experiments, VIDEOPLACE was created where the computer had control over the relationship between the participant's image and the objects in the graphic scene.

It could coordinate the movement of a graphic object with the actions of the participant.

Videoplace http://www.youtube.com/watch?v=dmmxVA5xhuo

The participant's image is digitized to create silhouettes which is analysed by specialized processors.

The processors analyse the image's posture, rate of movement, and its relationship to other graphic objects in the system.

They then react to the movement of the participant and create a series of responses-be they visual or auditory reactions. Two or more environments could also be linked.

Jeffrey Shaw

•  Jeffrey Shaw, artist and theorist, has pioneered the use of interactivity and virtuality

in his many art installations, which have been exhibited worldwide at major museums and

festivals.

•  As the founding director of the Institute for Visual Media he has been working at ZKM

(Karlsruhe, Germany) from 1991 till 2003. He is founding co-director of the Center for

Interactive Cinema Research (iCinema) at the UNSW in Sidney.

Jeffrey Shaw: The Legible City •  Visitors to ‘The Legible City’ are seated on a stationary bicycle and

‘move' through streets projected onto the surface in front of them.

•  In contrast to those of a normal city, the streets here are literally legible, lined not by buildings but by letters.

•  On their passage through the city, cyclist-visitors can pursue various narrative threads, accumulating their own history of the city.

http://www.youtube.com/watch?v=61l7Y4MS4aU

Shaw: A golden calf, 1984

•  A golden calf appears as a virtual 3D sculpture on a portable LCD display which the observer holds in his hands and with which he can freely move around an empty plinth standing in a room.

•  This real plinth possesses a virtual counterpart which serves as the rest for the golden calf in the digital representation. Real-space and virtual image are closely related.

http://www.youtube.com/watch?v=paaacEIF6wU

Shaw/Hegedüs/Lintermann: conFIGURING the CAVE, 1996

•  is a computer based interactive video installation that assumes a set of technical and pictorial procedures to identify various paradigmatic conjunctions of body and space.

•  The work utilizes the CAVE technology stereographic virtual reality environment with contiguous projections on three walls and the floor.

•  The user interface is a near life-size wooden puppet that is formed like the prosaic artists' mannequin; this figure can be handled by the viewers to control real time transformations of the computer generated imagery and the sound composition.

http://www.youtube.com/watch?v=epyUYSLAEAA

Pioneers: David Rokeby

David Rokeby has won acclaim in both artistic and technical fields for his new media artworks. A pioneer in interactive art and an acknowledged innovator in interactive technologies, Rokeby has achieved international recognition as an artist and seen the technologies which he develops for his work given unique applications by a broad range of arts practitioners and medical scientists.

Rokeby; Very Nervous System

Rokeby's best known work, Very Nervous System (1986-90) premiered at the Venice Biennale in 1996, won the first Petro-Canada Award for Media Arts (1988) and is permanently installed in several museums around the world.

The work uses video cameras, computers, and synthesizers to create an interactive space in which body movements are translated into music. The technology Rokeby developed for this work is widely used by composers, choreographers, musicians, and artists. It is also used in music therapy applications and is currently being tested as an activity enabler for victims of Parkinson's Disease.

http://vimeo.com/8120954

Christa Sommerer Laurent Mignonneau

Christa Sommerer is an internationally renowned media artist working in the field of interactive computer installation. She currently holds positions as Professor for Interface Culture at the University of Art and Design in Linz Austria.

Since 1992 Sommerer collaborates with French media artist Laurent Mignonneau. Their interactive artworks have been called "epoch making" (Toshiharu Itoh, NTT-ICC museum) for pioneering the use of natural interfaces to create a new language of interactivity based on artificial life and evolutionary image processes.

Their collaboration has been influenced by the combination of their different fields of interest, including art, biology, modern installation, performance, music, computer graphics and communication. Sommerer and Mignonneau have won major international media awards.

A-Volve

Behavior in space is, so to speak, an expression of form. Form is an expression of adaptation to the environment.

In the interactive real-time environment "A-Volve" visitors interact with virtual creatures in the space of a water filled glass pool. These virtual creatures are products of evolutionary rules and influenced by human creation and decision.

Designing any kind of shape and profile with their finger on a touch screen, visitors will "bear" virtual three dimensional creatures, that are automatically "alive" and swim in the real water of the pool.

http://www.youtube.com/watch?v=cZ3v1jcCXmk

Eau de jardin (2004)

•  “Eau de Jardin” is an interactive installation which transports visitors into the imaginary world of virtual water gardens.

•  When visitors approach themselves towards the amphorae, the plants capture the visitors presence and use the occurring tensions to draw virtual water plants on the large projection screens. The virtual plants on the screen resemble the real aquatic plants in the amphorae.

http://www.youtube.com/watch?v=8sI1P-cF1vM

Mobile Feelings, 2002/03

•  "Mobile Feelings" is an artistic project that explores the ambivalence of sharing personal information with an anonymous audience. Instead of communication via voice or images to people we know, "Mobile Feelings" lets people communicate with strangers through virtual touch and body sensations including smell and sweat using specially designed mobile phones.

•  These devices host miniature bio-sensors and actuators that capture the users' heartbeat, blood volume pressure and pulse, skin conductivity, sweat and smell. http://www.youtube.com/watch?v=VD6MvIfGfzc http://www.youtube.com/watch?v=PiQsAwTAxRs

The Quality of Proactive Engagement of Interface Passive Interaction>< Active Interaction

ACTIVE INTERACTION: Bodily Controlled Interface; full body immersion, active bodily

motion. tangible interfaces

PASSIVE INTERACTION: Cognitive Response Evaluation of the Spectator

Brigitta Zics - Mirror_SPACE 2004/05

Digital Luthiers: Reactable, 2003/05

Iwai Toshio, Nishibori Ty: Tenori-On 2005

Examples of Active Interaction:

Squidsoup - Driftnet, Responsive Environnent 2007

Biofeedback Applications Arts & Science

Early as 1965 Alvin Lucier applies biofeedback to generate sound .

He uses human brainwaves for his performance: Music for Solo Performer (1965)

The performance was recently recreated by Andrew Brouse & Maxime Rioux (1999)

http://www.youtube.com/watch?v=1qDKu692JZ0&

Photo by Phil Makanna Andrew Brouse Maxime Rioux “Automates Ki”

Biofeedback Application for HCI Producing Aesthetic Experiences

Bioofeedback Art Archive on MediaArt Tube: http://www.youtube.com/view_play_list?p=C51BC89CDE587C1F

Laura Colmenares Guerra - Lungs: The Breather 2008

BIOS - Bidirectional Input/Output System 2002/03

Christa Sommerer & Laurent Mignonneau: Mobile Feelings 2002/03

Andrea Polli: Intuitive Ocusonics, Eye-driven sound interface 1998

These artistic works investigated the

interconnection between cognitive processes and

aesthetic meaning production with the aim of providing an

aesthetic experience.

Bodily passive interaction focuses on bodily reactions that most explicitly express the cognitive, emotional or behavioral characteristics of the participant. The participant's body awareness is transformed by introverted actions of technological triggers while the functional bodily actions are reduced. The technological applications of passive interaction are affective computing or instant affection technologies.

•  Bodily active interaction---which is the most common modality of interaction---predominantly focuses on the physical engagement of the participant through tangible or full body interfaces. In the former, passive participants achieve immersive states by self-observation and reflection; in the latter, the active participants' physical action-reaction capacity produces the same state.

Advanced Definitions of Interaction The Type of Embodiment

Reading/Online Lecture The History of the Interface in Interactive Art

Söke Dinkla, 1994 http://www.kenfeingold.com/dinkla_history.html

Transforming Mirrors: Subjectivity and Control in Interactive Media

David Rokeby, 1995 www.arts.rpi.edu/century/MMC10/Rokeby.pdf

pp.133-158

ZICS, B., 2008. Transparency, Cognition and Interactivity: Toward a New Aesthetic for Media Art. PhD Thesis. Newport, Wales:

University of Wales http://www.trans-techresearch.net/?page_id=26

Video Lecture: Alan Kay: Doing with Images Makes Symbols (1987)

•  http://www.archive.org/details/AlanKeyD1987 •  http://www.archive.org/details/AlanKeyD1987_2