depth modulation composing motion in immersive audiovisual spaces

8
Organised Sound http://journals.cambridge.org/OSO Additional services for Organised Sound: Email alerts: Click here Subscriptions: Click here Commercial reprints: Click here Terms of use : Click here Depth Modulation: Composing motion in immersive audiovisual spaces Ewa Trębacz Organised Sound / Volume 17 / Issue 02 / August 2012, pp 156 162 DOI: 10.1017/S1355771812000088, Published online: 19 July 2012 Link to this article: http://journals.cambridge.org/abstract_S1355771812000088 How to cite this article: Ewa Trębacz (2012). Depth Modulation: Composing motion in immersive audiovisual spaces. Organised Sound, 17, pp 156162 doi:10.1017/S1355771812000088 Request Permissions : Click here Downloaded from http://journals.cambridge.org/OSO, IP address: 137.222.19.117 on 06 Dec 2012

Upload: jackhsiao

Post on 05-Sep-2015

213 views

Category:

Documents


1 download

DESCRIPTION

Depth Modulation Composing Motion in Immersive Audiovisual Spaces

TRANSCRIPT

  • OrganisedSoundhttp://journals.cambridge.org/OSO

    AdditionalservicesforOrganisedSound:

    Emailalerts:ClickhereSubscriptions:ClickhereCommercialreprints:ClickhereTermsofuse:Clickhere

    DepthModulation:Composingmotioninimmersiveaudiovisualspaces

    EwaTrbacz

    OrganisedSound/Volume17/Issue02/August2012,pp156162DOI:10.1017/S1355771812000088,Publishedonline:19July2012

    Linktothisarticle:http://journals.cambridge.org/abstract_S1355771812000088

    Howtocitethisarticle:EwaTrbacz(2012).DepthModulation:Composingmotioninimmersiveaudiovisualspaces.OrganisedSound,17,pp156162doi:10.1017/S1355771812000088

    RequestPermissions:Clickhere

    Downloadedfromhttp://journals.cambridge.org/OSO,IPaddress:137.222.19.117on06Dec2012

  • Depth Modulation: Composing motion inimmersive audiovisual spaces

    EWA TREBACZ

    Center for Digital Arts and Experimental Media (DXARTS), Box 353414, University of Washington, Seattle, WA 98195-3680, USAE-mail: [email protected], [email protected]

    The field of electroacoustic music has witnessed years of

    extensive exploration of aural spatial perception and an

    abundance of spatialisation techniques. Today the growing

    ubiquity of visual 3D technologies gives artists a similar

    opportunity in the realm of visual music. With the use of

    stereoscopic video we now have the ability to compose

    individual depth cues independently. The process of

    continuous change of the perceived depth of the audiovisual

    space over time is being referred to as depth modulation, and

    can only be fully appreciated through motion.

    What can be achieved through the separation and

    manipulation of visual and sonic spatial cues? What can we

    learn about the way we perceive space if the basic components

    building our understanding of the surrounding environment

    are artificially split and re-arranged?

    Visual music appears to be a perfect field for such

    experimentation. Strata of visual and aural depth cues can be

    used to create audiovisual counterpoints in three-dimensional

    spaces. The choice of abstract imagery and the lack of

    obvious narrative storylines allow us to focus our perception

    on the evolution of the immersive audiovisual space itself.

    A new language of an immersive audiovisual medium should

    emerge as a delicate, ever-changing balance between all

    previously separated and altered components.

    1. INTRODUCTION

    The field of electroacoustic music has witnessedmany years of extensive exploration of aural spatialperception. Research based on the separation andmanipulation of monaural and binaural cues resultedin an abundance of sound spatialisation techniquesand approaches. Today the growing ubiquity ofvisual 3D technologies gives artists a similar opportu-nity in the realm of visual music: with the useof stereoscopic video we now have the ability tocompose individual depth cues independently. Thisability to manipulate depth cues independently opensnew areas of artistic exploration.Visual music appears to be a perfect field for such

    experimentation. Strata of depth cues can be used tocreate audiovisual counterpoints in three-dimensionalspaces. Using abstract imagery enables artists to focustheir creative attention on the evolution of the audio-visual spaces themselves. This is still largely unexploredterritory, which gives artists the opportunity to pose

    some fundamental questions regarding the nature ofhuman perception of space and its boundaries. Moreimportantly, such questions would be posed directlythrough the language of art, while at the same timeenriching and transforming the traditional vocabularyof visual music.I will approach these questions through an experi-

    ment in art, introducing the term depth modulationto describe the process of continuous change of theperceived depth of audiovisual space over time.My own work Errai, which premiered at the

    52nd International Festival of Contemporary MusicWarsaw Autumn (2009), represents an early effort toexplore this rich territory. It was realised with the useof stereoscopic video and ambisonic sound, combinedwith live music performance and computer-controlledlight animation. Errai will be used to give examples ofhow depth modulation can be applied in practice, witha particular focus on the visual realm. However, thescope of this article is much broader than just theapplications of these techniques in my own work, andwill hopefully open a discussion about the potential forthe creative use of depth cues as building blocks forrelations and structures in the audiovisual space.

    2. SENSORY OVERLOAD OR THE FABRIC OFART OF THE FUTURE?

    The late nineteenth century synaesthetical concept ofarts initiated a process of blending the boundariesbetween traditional art disciplines. As a result of anemphasis on the holistic concept of perception, thestrict divisions between time-based and space-basedarts began to dissolve, with this process continuingand intensifying through the entire twentieth century.The phenomenological philosopher Maurice Merleau-Ponty claims that all visible objects are palpableobjects at the same time, while tactile and visible arethe two demonstrations of the same being (quoted inde la Motte-Haber 2002: 32). If a tactile object issimply an embodiment of a visible object, should spacebe understood as a sequence of tangible objects?Visual and tactile spaces show a mutual relationship.Also the visible and the audible appear to be stronglyinterconnected. In experimental works of musique

    Organised Sound 17(2): 156162 & Cambridge University Press, 2012. doi:10.1017/S1355771812000088

  • concre`te (in the 1940s and 1950s) sound is treated as anobject that can be characterised in a way resemblingdescriptions so far reserved for the visible, such ashaving a volume or density.The concept of the intersensory nature of human

    sensory perception inspired artists to cross bound-aries between the traditional arts, which resulted in avariety of hybrid forms. Similar tendencies have beenpresent in parallel in the visual arts and avant-gardemusic. The most obvious manifestation of this pro-cess is found in installations, in which sculpturalelements are expanded by sound. The simultaneouscharacter of a visual experience and the sequentialcharacter of aural experience create together anew concept of space, understood as a network oftemporal relations. Physical (architectural) spacesare often enriched by sound or video projection. Inthe late twentieth century and the first decade of thetwenty-first century architectural spaces were oftenmodified and disconnected from their utilitarianfunctions by video projection. Also, experiments invirtual and augmented reality attempt to blend thefrontier between the real and the illusionary.The years since the late 1990s have witnessed a real

    explosion in various 3D technologies, both visualand sonic, and the true re-birth of stereoscopy as alegitimate approach to filmmaking. Digital cinemasuccessfully brought large audiences back to theatres,reviving the interest in three-dimensional movies for thefirst time since 1956. Yet, despite the large commercialsuccess of selected films, 3D cinema is still awaiting thecreation of its own language, a language that wouldtruly express the uniqueness of this medium. Withoutsuch a language, 3D films will remain reduced to mereentertainment with their sporadic commercial successesbased on overwhelming the senses rather than produ-cing works that would equal the masterpieces of thetraditional cinematic art. Experimental art, cautiouslybut steadily approaching 3D technologies, is in dangerof following a similar fate. If interest in such techniquesis not met by the emergence of a unique language,capable of expressing the nature of immersive media,the field may continue to languish despite the readyavailability of the underlying technologies.The phenomenon of sensory overload has become

    so obvious that we seem to have overlooked themoment when it seemingly turned into our everydayexperience. Not only are we providing our senseswith amounts and combinations of signals that arereaching the physical limits of our perception, but,with the rapid development of new technologies, weare now able to extend our eyes and ears beyond theaudible and the visible. As we approach the physicalboundaries of human perception, the question of therole of sensory experience in perceiving a work of artreturns with a vengeance. If the physical limitationsof our senses are the final frontier, then further

    development in art may appear only as a possibilityrather than something that is granted.This inevitably raises a very well-known question

    about the relationship between the direct experienceof art and the work of art itself. Is direct sensoryexperience necessary to comprehend a work of artas a whole? Can a work of art function at all ifits fundamental aspects are inaccessible throughdirect experience? Is intellectual compensation forour imperfect senses sufficient for an artwork to beperceived as intended by an artist? Finally is itpossible that the further development of art might bebound to the alteration of human perceptual skills?These and many similar questions have inspired

    me to pursue projects in the realm of audiovisualtime-based art, focusing on the intersensory percep-tion of space. I have been particularly interested inthe following problems:

    > What can we learn about the way we perceivespace if the basic components building ourunderstanding of the surrounding environmentare artificially split and rearranged?

    > What can be achieved through the separation andmanipulation of visual and sonic spatial cues?

    > How do we perceive phenomena not availablethrough direct experience in the real world thatcan be artificially created, for instance a contin-uous change of image depth over time?

    > In such an artificial environment, how would weperceive the relationship between various mediaand how would we interpret the informationperceived by various senses? Would we be ableto seamlessly switch between concurrent lines ofmovement?

    Before we begin to examine these intriguing problems,it will be necessary to investigate the role of individualdepth cues and relations among them in the perceptionof audiovisual spaces. What follows is a brief overviewof visual depth cues, and a discussion of so-called directdepth modulation.

    3. DIRECT DEPTH MODULATION AND ITSPERCEIVED EFFECTS

    If depth modulation is defined as a process of con-tinuous change of the perceived depth of the audio-visual space over time, then direct depth modulationis such a change using binocular or binaural cuesonly. Direct depth modulation is a phenomenonunknown to real-world experience. It is howeveravailable to us in the visual realm through the directmodification of visual cues using stereoscopic video,and in my opinion this process shows great potentialfor artistic experimentation.Perceiving space is one of the essential skills guid-

    ing animals and humans through the environments

    Depth Modulation 157

  • they live in. Humans develop a full set of spatialperceptual skills by the first year of life through anextensive learning process. It is worth noting that inhuman infants the full ability to perceive spatial rela-tions develops much earlier than the full set of motorskills. Information about depth and distances isstrongly related to information about motion, and isgathered through combinations of numerous sources.Humans visually perceive three-dimensional space

    by interpreting the two-dimensional images in eacheye using a network of visual cues. A long list of thesecues can be found in the literature and is usuallydivided into two categories: monocular cues andbinocular cues. Monocular depth cues can be kineticor pictorial. A classic example of a kinetic cue ismotion parallax: as we move, objects that are locatedfurther away seem to be moving slower relative tocloser objects. Examples of pictorial cues include therelative or familiar size of objects, shadows, elevation,orientation, occlusion, texture gradient, colour, linearperspective and atmospheric perspective.Binocular depth cues include disparity (binocular

    parallax) and convergence. Disparity indicates thatthe retina of each eye records a slightly differentimage. Convergence is related to an ability of theeyes to turn towards each other in order to zoomand focus on objects and preserve binocular vision(Solso 1994).Computer graphics software and modern immer-

    sive technologies have finally given us the freedom tocompose depth cues independently in various typesof virtual environments. Using 3D animation soft-ware or specially rigged stereo cameras we can nowdirectly manipulate binocular depth cues. Directdepth modulation is accomplished by changing thedistance between the stereoscopic cameras over time.As the distance between the cameras changes, so dothe depth cues associated with binocular vision.At any given time the human left and right eye

    converge on a point in space, known as the point offixation. This convergence, and the angle it creates, isused to determine distances. The difference betweenangles of fixation between different objects deter-mines the relative depths of these objects within astereoscopic image.Changes in the distance between the two virtual

    cameras change the depth cues that rely on stereopsis,the sense of relative distance based on binoculardisparity (Gillam 1995: 41). As the cameras movetowards each other, the perceived depth of an imageflattens as the relative difference between anglescreated by points of fixation grow smaller. Whileother depth cues, such as those for perspective, mayremain, those associated with binocular vision arelost. When the cameras occupy the same space, theleft and right stereo images become identical, in effectcreating a normal 2D image. Further, the cameras

    can exchange positions by passing through eachother, which results in transforming the image fromstereoscopic, to flat, to pseudo-stereoscopic.Figure 1 shows how, as the distance changes, so

    does the angle. This angle determines the relativedepth of an object. The greater the angle, the closerthe object appears.Figure 2 shows how changing separation of the

    virtual cameras can change this angle without chan-ging the actual distance of the point of fixation. Thiscreates the illusion of a change in depth. Angles a andc from above can be achieved simply by changing thedistance between cameras.

    3.1. Perceived depth

    Direct depth modulation has two interesting effectson how one perceives the image. The first and mostobvious is how it changes the perceived distance ofobjects from the viewer. As the cameras move closertogether, stereoscopic depth cues fade and objectsappear to flatten towards the depth of the physicalscreen. Objects in the background are drawn forward,objects in the foreground move back, and objectswith perceivable depth get flatter.Figures 3ac illustrate a fairly simple scenario, with

    three unmoving objects. In these figures we see a top-down view of a three-dimensional space showing theperceived position of these objects in space along withthe positions of the cameras. Objects A and B areconsidered to be two-dimensional objects with nodepth cues of their own; object C has depth.Because the cameras in this example are parallel,

    rather than convergent, there is no predeterminedpoint of fixation. The viewer determines the actual

    a

    b

    c

    Figure 1. Angular disparity and perceived depth of image.

    158 Ewa Trebacz

  • point of fixation based on the object he or she istracking. Without other depth cues, the physicalscreen becomes a natural point of fixation for theviewer. Therefore as objects lose their depth cues theyappear to converge towards the screen.Figure 3a shows the cameras at their designated

    starting distance, with object A in front, object B atthe same depth as the physical screen, and object Cstarting at the depth of the physical screen and pro-jecting into the background.Figure 3b illustrates the effects of moving the cam-

    eras towards each other. As the cameras move towardseach other, it appears as if object A is approachingcloser to the screen line and as if object C is losingdepth. Because object B is already at screen depth thereis no perceived change. When the cameras are coin-cident, there are no binocular depth cues.As the cameras exchange positions by passing

    through each other, with the left camera on the rightside and the right camera on the left, the binocular

    depth cues are reversed, creating a pseudo-stereoscopicimage. In the example given in Figure 3c, as the cam-eras move away from each other, object A appears tomove to the background and object C appears toproject towards the viewer.It should be noted that this does not change per-

    spective cues nor does it change how the objects areordered. Objects should be carefully chosen to avoidboth perspective problems and problems where anobject is occluded by another object.

    3.2. Perceived size

    Along with changes in perceived depth, depth mod-ulation also influences the perceived size of objects inthe image. This effect is created by the clash betweentwo depth cues, binocular disparity and perspective.Figure 4a shows the real-world conditions: as an

    object moves towards the viewer it fills a greaterportion of the field of view, and as it moves awayfrom the viewer it fills less of the field of view. This isan effect known as looming (Gillam 1995: 26). Theangles represent the field of view commanded by theobject as it changes depth.When objects move towards and away from

    the viewer, perspective and binocular disparity areproperly synchronised, matching our real-worldexpectations. However when only depth modulationis applied with no changes to the objects themselves,only binocular disparity is affected. Perspective isunchanged. Even while the object appears to be

    ab c

    Figure 2. Angular disparity and separation between virtual

    cameras.

    Object A

    Object B

    Object C

    CameraLeft

    CameraRight

    screen line

    front

    back

    Object AObject B

    Object C

    CameraLeft

    CameraRight

    screen line

    front

    back

    Object A

    Object B

    Object C

    CameraRight

    CameraLeft

    screen linefront

    back

    (a) (b)

    (c)

    Figure 3. (a) Virtual camera movement and perceived depth: example A (static distance). (b) Virtual camera movement and

    perceived depth: example B. (c) Virtual camera movement and perceived depth: example C.

    Depth Modulation 159

  • moving backward or forward in space, there is nolooming.Figure 4b shows the field of view taken up by

    the object. Without looming the amount of space theobject takes up in our field of view, represented bythe angle subtended by the edges of the object, isunchanged.

    4. STRATA OF DEPTH CUES

    Direct depth modulation is only the most obviousform of depth modulation. In my preliminaryexperiments with stereoscopic projection, my majorfocus was on creating a particular impression ofpulsation of the depth dimension. The illusion seemedquite powerful and showed a lot of yet unexploredpotential. Later I noticed that this effect was

    even more intriguing when direct manipulation ofdepth was independent from monocular cues, usuallyassociated with depth perception. For instance, subtlechanges of focus, colour or contrast could be usedto provide conflicting cues between the video layers.Similarly, selected monocular cues can be used insynchronisation with the direct depth modula-tion to reinforce the illusion. One can also imaginesituations in which the entire audiovisual space isevolving in one direction, but the timing of changevaries between individual monocular and binoculardepth cues.These experiments with stereoscopic video projec-

    tion, as well as my ongoing interest in sound spatial-isation, led me in 2009 to a larger work, in which Iattempted to treat an immersive audiovisual spaceas a single whole. Errai (2009) combined computer-animated stereoscopic video and ambisonic soundwith live computer-controlled animated lighting andlive music performance. During my work on Errai Igradually realised that depth modulation can beapproached on several simultaneous layers, and theselayers can be organised in hierarchical way, which Ilater called strata of depth cues. I will describe each ofthe layers or strata and provide brief examples basedon my own work. However, the concept of strata ofdepth cues is much broader than its implementationin Errai.To better demonstrate the nearly unlimited potential

    for creating powerful illusions and audiovisual coun-terpoints by composing layers of independent depthcues, I propose the representation shown in figure 5.Complex relations can be created between elements ofeach layer and between the layers themselves.

    4.1. Direct depth modulation

    The first layer represents the ability to modulate bin-ocular and binaural depth cues only. This is the mostdirect example of depth modulation: we are modulatingthe differences between each eye and each ear.

    (a) (b)

    Figure 4. (a) Distance changes and perceived size. (b) Direct

    depth modulation and perceived size.

    Figure 5. Strata of depth cues.

    160 Ewa Trebacz

  • Using an example from my own work, the stereo-scopic video in Errai explores a full spectrum of con-tinuous change of perceived depth over time. This wasachieved by directly modulating distances between pairsof virtual cameras in a 3D animation environment withseveral independently evolving video layers. As a result,these layers evolve simultaneously and independentlyfrom each other: from true stereoscopic through flat2D to pseudo-stereoscopic images. At times the onlyperceivable movement is an expansion or contraction ofthe visual space itself.Partial expansion or contraction of the perceiv-

    able depth occurs on various layers throughoutthe entire work, but a full cycle with over a 2-minute-long transition from a pseudo- to a true stereoscopicimage is presented only once, becoming the extendedclimax of the work. While this transition fully revealsthe strongest formative principle of Errai, it is at thesame time the most obvious demonstration of howrelative our perception can be. The exact time whenone becomes aware of the change happening in thedepth dimension will vary greatly between individ-ual viewers. It also depends on the decision ofwhich parts of the image a particular viewer woulddecide to track at any given moment. Such a choicehas significant consequences: some elements ofthe image that are neglected while being in thebackground, suddenly and surprisingly becomerelevant after their position was shifted towardsthe foreground. Since we cannot observe similarprocesses in the real world, it is also somewhat diffi-cult to recognise the exact moment when depthstarts contracting or expanding. As observed duringseveral tests, it will vary not only between viewers butalso between different experiences of the work by thesame viewer.In case of the audio component of Errai, the situa-

    tion is a little different. Unlike the video, the initialaudio material was not generated by computer soft-ware. Instead it was recorded in ambisonics with thesimultaneous use of two Soundfield microphonesplaced at different elevations and distances from thesources. The recording sites were carefully chosen fortheir unique acoustic properties, with the assumptionthat their spatial qualities will play a key role indefining not only the character of the musical narration,but also ultimately the formal shape of the work. Thismaterial was later processed, spatially manipulated inB-format, and composed into a sequence of sounds-capes. As a result, I did not independently composebinaural depth cues, but rather attempted to createcontinuous transitions between three-dimensionalsound images, captured simultaneously by these twomicrophones. While there is some analogy to directdepth modulation in the video layers, this processmay not exactly fit its strict definition, and calls forfurther investigations.

    4.2. Indirect depth modulation

    The second layer represents monocular and monauraldepth cues. For instance, binocular cues could indi-cate that the space is expanding, while monocularcues would suggest contraction. Alternatively, acombination of monocular and binocular cues canbe used to create a stronger illusion of a continuousdepth change over time.In Errai I often used selected monocular depth

    cues to reinforce the effects of direct depth modulation.The visual environment of Errai was constructed insuch a way that the entire image could be easily flipped,choosing imagery with semi-transparent and lumines-cent objects in order to avoid problems with occlusion.Background could become foreground and vice versa,with continuous transitions possible between those twostates. More importantly during such a transitionor shift, different elements of the image or differentmovements within a selected layer are exposed, allow-ing the viewers different interpretations of the stereo-scopic video. Selected elements of the image (suchas objects or movement) gradually become revealedduring the transition, as the visual layers change theirperceived depth. These elements can be seen as morerelevant or dominant depending on the stage of thedepth modulation process. These changes of depth areat times related to subtle changes of colour, contrastand movement on each layer. Combined with directdepth modulation, this results in changing of hierarchybetween layers: elements gain and lose importance asdepth evolves.The use of monaural depth cues is very obvious

    in the audio layer. In Errai, single sound objectsare placed in space at various distances from thelistener, becoming components of larger entities, butrarely changing their location independently. Thesethree-dimensional sound images are then spatiallymanipulated as a whole. However, the perceiveddistance of sound objects from the listener is a veryimportant element of the structure of this work, andthe differences between these locations are at timesquite extreme. This is achieved by the direct appli-cation of monaural depth cues, such as amount ofreverberation, amount of high frequencies in thesound signal and its perceived loudness.

    4.3. Sound and image

    On the top of that we can design relationships betweenthree-dimensional image and three-dimensional sound.This can be achieved by creating and resolving tensionusing conflicting depth cues between the sound andimage, and by creating contrapuntal relations betweenparallel layers of movement.The audio and video layers of Errai were for the

    most part created independently, but they are notseparate entities. They share the same common pace

    Depth Modulation 161

  • and oscillate around the same formal plan. No strict,note-to-note synchronisation is required for the audioand video layers, but they do meet within each of theten sections of the piece.The video and audio layers can be considered

    as two groups of independent voices, constantlycompeting for the audiences attention. This rela-tionship is most transparent while comparing climaxpoints of audio and video layers. The effective climaxof the video layer is a long transition from thepseudo-stereoscopic to the true stereoscopic image.This is immediately followed by a section that may beconsidered a climax of the audio layer, thus shiftingthe audiences attention between the visual and auralspaces.

    4.4. Competing spaces

    The last layer represents the relationship between thepre-recorded or artificially created audiovisualspaces, reproduced during the performance, and aphysical space, in which the work is presented. Thephysical space is understood here as an architecturalspace with all its unique properties like reverbera-tion time, size and shape, lighting and those prop-erties can be referenced and enhanced by addingan element of live interaction, such as live musicperformance or lighting.Errai for example used both live lighting (designed

    by Polish artist Robert Sowa) and live music perform-ance (soprano and horn). The lighting in Erraicreates a contrapuntal relationship to the stereoscopicvideo, becoming an equally important element of thedramaturgy. It is used to construct an immersive three-dimensional space, with continuity between the virtualworld of the stereoscopic video projection, and the realworld, represented by the physical space, in which theaudience was enclosed.The most important layer of the lighting design

    scheme is a direct interaction with the stereoscopicprojection. Several short interventions of directlight were combined with the stereoscopic video bythe use of masked animated lights pointed directlyat the projection screen. This interaction constitutesone of the major formative principles organisingthe perceptual space of the piece. It was given the roleas a bridge between the virtual world perceivedthrough the 3D glasses and the semi-architecturalspace, created anew by the lighting of the physicalspace.Similarly, the live audio performance (horn,

    soprano) creates a reference to the physical spaceof the performance hall, which is now considered tobe the body of one extended musical instrument.Representing here and now, the live performancecreates a dialogue with the pre-recorded soundscapesand the acoustics of the hall itself.

    5. CONCLUSIONS

    Earlier in this article, I posed several questions thatinspired me to pursue projects in the realm of audio-visual immersive art. I introduced the term depth mod-ulation, and attempted to show how this process can beapplied in practice, giving examples from my workErrai. While Errai was not intended to be a scientificexperiment, it has led me to several interesting obser-vations regarding the visual perception of depth and therelationship between the various spatial components ofthe work. For example, following and comprehending acontinuous change of depth over time seems to be achallenging task. Since we are not originally equipped toperceive such phenomena, it seemed like the perceptionof depth modulation had a discrete character. It wasvery difficult to follow the entire process withoutany interruptions or distractions. After perceiving thechange at several time intervals, viewers were often ableto realise that the process was continuous. But evenafter understanding this, the actual perception of thisphenomenon remained stubbornly discrete.While the examples I have given here may not have

    directly answered or exhausted any of the questionsI posed, I hope they can open a broader discussion ofthis subject. With the development of new immersivetechnologies, new questions will inevitably appear andagain challenge our understanding of human spatialperception. I strongly believe that the nature of humanperception of space can and should be investigatedthrough art. The potential for artistic exploration isnearly limitless. Hopefully investigations into humanspecial perception will be continued not only byresearchers, but also by artists, through their innovativework.The realm of visual music, where individual depth

    cues can be used to compose motion in depth, seems tobe a very promising area for such exploration. Whatwe are ultimately looking for is a new language, notonly enriching the existing vocabulary of visual music,but also capable of expressing the unique natureof immersive media. Such a new language shouldemerge as a delicate, ever-changing balance between allpreviously separated and altered components.

    REFERENCES

    Gillam, B. 1995. The Perception of Spatial Layout from

    Static Optical Information. In W. Epstein and S.J.

    Rogers (eds) Perception of Space and Motion. San

    Diego, CA: Academic Press, 2367.

    Motte-Haber, H. de la. 2002. Esthetic Perception in

    New Artistic Contexts: Aspects hypotheses unfinished

    thoughts. In B. Schulz (ed.) Resonanzen/Resonances:

    Aspekte der Klangkunst/Aspects of Sound Art. Heidelberg:

    Kehrer Verlag, 2937.

    Solso, R.L. 1994. Cognition and the Visual Arts. Cambridge,

    MA: The MIT Press.

    162 Ewa Trebacz