hearing in numbers: virtual sound as next nature

1

Balance Unbalance 2015

Water, Climate & Place [Reimagining Environments]

2

Hearing in Numbers Virtual Sound as Next Nature

Phylis Johnson, PhDSouthern Illinois University, Carbondale, IL USA

Jay Jay Jegathesan, PhD CandidateUniversity of Western Australia, Australia, Perth

3

Sonification: Art & Science

Barass & Vickers (2011) point toward an open-ended process that includes perceptual & ethnographical data, and perhaps speculative theatrical representation of ecological sound and abstract data, and most relevant to virtual world designs those that incorporate first person avatar immersion as exhibited in computer games and platforms.

4

Figure 2. The architecture of our real-time interactive system follows the sensing/processing/response paradigm. (LeGroux et al. 2007)

5

Figure 9. This diagram shows the different type ofdata we are dealing with during the sonification process. (LeGroux et al. 2007)

6

The Virtual Theater

Cinema of the FutureMorton Heilig, 1955

7

8

9

10

Sound Design in Second Life

11

Prim – Unit of Measurement

12

Scripting the Prim

13

Welcome to Virlantis

14

Virlantis

15

Virlantis

16

Perceptual Placement

17

18

19

Positioning Sound

20

21

22

23

VIRTUAL WORLD SOUND DESIGNSound Prim Measurement

Sound Time Ambient LoopObject Unit

24

25

Koert van Mensvoort

26

27

28

Shopping for Virtual Sound

Acoustic Alchemy in Second life

29

30

Sound Objects

31

32

SL Sound Barriers

10 second limitations per clip

Every avatar hears sound differently

Client Settings – System & Client

ISP performanceLagLimited Playback options

33

Break Sound Barriers

Hear sounds all aroundNo Time Limit on clipsRandomize volume - 11 PresetsPlayback flexibility (day, night options)

3D placement flexibility -variable borders without subdividing land into parcels (N, S, E, W, up, down)

34

35

36

37

38

Virtu

al Soun

d Archi

ve

39

Sonic Continuum (Toward Invisibility)

World Virtual Reality POV Avatar Human Placement Data Perceptual Locative Studio Field Purpose Archive Action Source Non-Present Present Dimensional Linear Layered Emotive Non-Referential Referential Memory Fragmented Immersive Medium Headphone Ear Sound Paths Waves Nature Visible Invisible

40

41

Schaeffer to Schafer

42

Reich, Cage, & Cardiff

43

The Cage Museum

44

Imaginary Soundscape

45

Sonic Luggage

46

Across Time & Soundscapes

47

48

The Art of Noise

49

Sonic Role Play

Why should we care?How will it help, hinder?Revisit, Reimagine, and

Speculate on What’s Next for Sound?

50

Virtual Reality

Other Virtual Worlds and

Environments

Audio Avatars

51

Virtual Dublin

52

Discover Ireland

53

John Cage

54

Sources (cited in presentation)

Barrass, S., & Vickers, P. (2011). Sonification Design and Aesthetics. Sonification Handbook (pp. 145-171). Johnson, P. ( 2011), A Second Life for Sound: Crossing Sonic Paths in Virtual Worlds. Soundscape, 11 (1), 52-56. Kramer, G., Walker, B., Bonebright, T., Cook, P., Flowers, J. H., Miner, N. and John

Neuhoff, N. (1999). Sonification Report: Status of the Field and Research Agenda. Technical report, ICAD/NSF. McLuhan, M. (2005). Visual and Acoustic Space. In Christopher Cox & Daniel Warner (eds)., Audio Culture.

Readings in Modern Music, New York: Continuum, 67-72. Ong, W. J. (2012). Orality and Literacy: 30th Anniversary Edition. New York, NY: Routledge. Ribeiro, F., Florencio, D., Chou, P., and Zhang, Z. (2012). Auditory Augument Reality: Object Sonification for the

Visually Impaired. MMSP Proceedings: IEEE, September. Sample, Ian. (2015). 'Noise pollution makes us oblivious to nature.' The Guardian Weekly, February 27. Schafer, R. M (2004). The Music of the Environment. In Christopher Cox & Daniel Warner (eds.), Audio Culture.

Readings in Modern Music, New York: Continuum, 29-39 Schafer, R. Murray (1992). Music, Non-Music and the Soundscape. In John Paynter et al. (eds)., Companion to Contemporary

Musical Thought, London: Routledge, 34–45. The Interactive Sonification Workshop (ISON), 2013.  van Mensvoort, Koert. (2012). Next Nature. Barcelona, Spain: Actar.

Additional references available upon request.

55