interactive sound rendering in dynamic virtual environmentsacm mm 09 october 19 23, 2009 1 resound...
TRANSCRIPT
![Page 1: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/1.jpg)
ACM MM 09October 1923, 2009
1
RESoundInteractive Sound Rendering inDynamic Virtual Environments
Micah Taylor, Anish ChandakLakulish Antani, Dinesh Manocha
University of North Carolina
![Page 2: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/2.jpg)
ACM MM 09October 1923, 2009
2
● Sound rendering and applications
● Details of propagation
● Our system: RESound
![Page 3: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/3.jpg)
ACM MM 09October 1923, 2009
3
● Sound rendering and applications
● Details of propagation
● Our system: RESound
![Page 4: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/4.jpg)
ACM MM 09October 1923, 2009
4
Sound Rendering● Three main steps
● Signal input
● Sound propagation
● Audio output
![Page 5: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/5.jpg)
ACM MM 09October 1923, 2009
5
Sound Rendering: Signal Input
● Recorded sample● Simple and fast● Played with events● Static
● Synthesized sound● Physics simulation
generates sound● Matches virtual events
[Raghuvanshi 2006][Matt Hileo]
![Page 6: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/6.jpg)
ACM MM 09October 1923, 2009
6
Sound Rendering: Signal Input● Synthesized sound
● Uses physical models[Florens et al. 1991]
● Interactive rates with many objects[Raghuvanshi et al. 2006]
● Correlates closely with visual scene[Ren et al. 2009]
![Page 7: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/7.jpg)
ACM MM 09October 1923, 2009
7
Sound Rendering: Propagation● Goal: Model environment influences
● Echoes● Delay from distance● Attenuation from distance● Frequency shifts
● Output: Impulse response● Represents room's effect on input signal
![Page 8: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/8.jpg)
ACM MM 09October 1923, 2009
8
Sound Rendering: Propagation● Common methods
● No propagation direct path only● Geometric simulation● Numerical simulation
![Page 9: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/9.jpg)
ACM MM 09October 1923, 2009
9
Sound Rendering: Audio Output
● Goal● Combine many sounds from environment● Apply any needed effects● Output to user's audio device
● Uses the output from prior steps● Input signal● Room impulse response
![Page 10: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/10.jpg)
ACM MM 09October 1923, 2009
10
Sound Rendering: Audio Output● Common output methods
● Mono– Fast, simple– No spatialization
● Stereo– Fast, simple, left+right spatialization
● 3d sound– Head Related Transfer Functions (HRTF)– Complex, very good spatialization
![Page 11: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/11.jpg)
ACM MM 09October 1923, 2009
11
Applications● Video games
● Helps player avoid monsters
● Provides sound cues to environment size
● Used in most 3d video games
![Page 12: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/12.jpg)
ACM MM 09October 1923, 2009
12
Applications● Training simulators
● Improves realism● Decreases incorrect
training
● Current uses● Tactical training● EMT training
![Page 13: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/13.jpg)
ACM MM 09October 1923, 2009
13
Applications● Multimedia
● Auditory displays– Enhance data
visualization● Telephony and Video
conferencing
![Page 14: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/14.jpg)
ACM MM 09October 1923, 2009
14
Applications● Computer aided
design● Relay cues about
environment design● Preview room
acoustics before construction
![Page 15: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/15.jpg)
ACM MM 09October 1923, 2009
15
● Sound rendering and applications
● Details of propagation
● Our system: RESound
![Page 16: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/16.jpg)
ACM MM 09October 1923, 2009
16
Propagation● Simplest method:
● Direct path between source and listener
● Add echoes with postprocess filter
● Fast● Widely used
![Page 17: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/17.jpg)
ACM MM 09October 1923, 2009
17
Propagation● However
● Not physically based● Spatialization
incorrect● Echoes do not match
environment
![Page 18: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/18.jpg)
ACM MM 09October 1923, 2009
18
Propagation● Acoustic simulations
● Numerical– Solves acoustic wave equation– Slow, but getting faster [Raghuvanshi et al. 2009]
● Geometric– High frequency approximation– Very fast – interactive– Models sound as ray
![Page 19: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/19.jpg)
ACM MM 09October 1923, 2009
19
Propagation● Specular reflection
● Mirrorlike reflections● Reflections decrease
amplitude● Longer paths,
longer delays
![Page 20: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/20.jpg)
ACM MM 09October 1923, 2009
20
Propagation● Specular reflection
● Mirrorlike reflections● Reflections decrease
amplitude● Longer paths,
longer delays● Often many reflection
paths
![Page 21: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/21.jpg)
ACM MM 09October 1923, 2009
21
Propagation● Diffuse reflection
● Scattering reflections
![Page 22: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/22.jpg)
ACM MM 09October 1923, 2009
22
Propagation● Diffuse reflection
● Scattering reflections● Scattered sound
reaches listener
![Page 23: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/23.jpg)
ACM MM 09October 1923, 2009
23
Propagation● Diffraction
● Sound 'bends' around corners
● Can change phase
![Page 24: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/24.jpg)
ACM MM 09October 1923, 2009
24
Propagation● Diffraction
● Sound 'bends' around corners
● Can change phase● Often many diffraction
paths
![Page 25: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/25.jpg)
ACM MM 09October 1923, 2009
25
Propagation● Combine
● Direct● Specular● Diffuse● Diffraction
● Early contributions● 45 recursions
![Page 26: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/26.jpg)
ACM MM 09October 1923, 2009
26
Propagation● Reverberation
● Late contributions● Impulses decays over
time● Hundreds of
recursions● Gives 'feel' of the
room
![Page 27: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/27.jpg)
ACM MM 09October 1923, 2009
27
Propagation● Specular reflections
● Imagesource method [Allen et al. 1979]
● From source● Reflect against all scene triangles
– Creates imagesources– Is listener visible
● Reflect image sources– and so on...
![Page 28: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/28.jpg)
ACM MM 09October 1923, 2009
28
Propagation● However
● Very compute intensive● Need to accelerate
● Graphic acceleration● Remove nonvisible triangles
● Sound acceleration● Remove nonreflecting triangles
![Page 29: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/29.jpg)
ACM MM 09October 1923, 2009
29
Propagation● Accelerated by
● Ray tracing [Vorlander 1989]
● Beam tracing [Funkhouser et al. 1998]
● Frustum tracing [Lauterbach et al. 2007]
● And others...
● Often require precomputation● Nonmoving source
![Page 30: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/30.jpg)
ACM MM 09October 1923, 2009
30
Propagation● Diffuse reflections
● Often modeled by ray tracing [Dalenbaeck 1996]
● Radiosity [Siltanen et al. 2004]
● Compute intensive● Fixed source and receiver● No scene movement
![Page 31: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/31.jpg)
ACM MM 09October 1923, 2009
31
Propagation● Diffraction
● Added to– Beam tracing [Tsingos et al. 2001]
– Ray tracing [Stephenson et al. 2007]
– Frustum tracing [Taylor et al. 2009]
– Image source [Shroeder et al. 2009]
![Page 32: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/32.jpg)
ACM MM 09October 1923, 2009
32
Propagation● Reverberation
● Ray tracing– Slow, accurate [Hodgson 1990]
● Statistical– Fast, some error [Savioja et al. 1999]
![Page 33: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/33.jpg)
ACM MM 09October 1923, 2009
33
● Sound rendering and applications
● Details of propagation
● Our system: RESound
![Page 34: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/34.jpg)
ACM MM 09October 1923, 2009
34
RESound● Simulates all mentioned effects● Interactive update rates● Dynamic scenes
● Handles propagation and output● Given input sound + environment
● Propagates sound through environment● Renders signal at receiver's position
![Page 35: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/35.jpg)
ACM MM 09October 1923, 2009
35
RESoundSystem overview
![Page 36: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/36.jpg)
ACM MM 09October 1923, 2009
36
RESound● Early contributions by simulation
● Specular + diffraction● Diffuse reflection
● Late contributions by statistics● 3d audio output
![Page 37: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/37.jpg)
ACM MM 09October 1923, 2009
37
RESound● Unified engine
● Frustum tracing● Ray tracing
● Ray primitive● Single acceleration structure
● Bounding Volume Hierarchy● Allows dynamic scenes● Fast ray tracing
![Page 38: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/38.jpg)
ACM MM 09October 1923, 2009
38
RESound● Scene acceleration hierarchy
● Bounding Volume Hierarchy [Lauterbach et al. 2006]
– Fast construction times– Allows interactive visual ray tracing– Allows dynamic scene changes
● Can accelerate frustum and ray tracing
![Page 39: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/39.jpg)
ACM MM 09October 1923, 2009
39
● Specular + diffraction● Frustum tracing● Volumetric, finds most
paths● Dynamic scenes● Fast
● Diffuse● Ray tracing● Shares scene
structure● Dynamic scenes● Fast
![Page 40: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/40.jpg)
ACM MM 09October 1923, 2009
40
RESound● Frustum tracing
● Specular reflection
![Page 41: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/41.jpg)
ACM MM 09October 1923, 2009
41
RESound● Frustum tracing
● Specular reflection
● Frustum is bounded by rays
![Page 42: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/42.jpg)
ACM MM 09October 1923, 2009
42
RESound● Frustum tracing
● Specular reflection
● Check if receiver is inside bounded volume
![Page 43: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/43.jpg)
ACM MM 09October 1923, 2009
43
RESound● Frustum tracing
● Specular reflection
● Bounding rays can be reflected
![Page 44: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/44.jpg)
ACM MM 09October 1923, 2009
44
RESound● Frustum tracing
● Specular reflection
● Sound path is linear combination of rays
![Page 45: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/45.jpg)
ACM MM 09October 1923, 2009
45
RESound● Diffraction
● Covers more area● Allows smooth
transitions– Fades out
![Page 46: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/46.jpg)
ACM MM 09October 1923, 2009
46
RESound● Diffraction
● Covers more area● Allows smooth
transitions– Fades out
● First step● Find diffracting edges
![Page 47: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/47.jpg)
ACM MM 09October 1923, 2009
47
RESound● Frustum tracing
● Edge diffraction
![Page 48: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/48.jpg)
ACM MM 09October 1923, 2009
48
RESound● Frustum tracing
● Edge diffraction
● From source– Trace many frusta
![Page 49: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/49.jpg)
ACM MM 09October 1923, 2009
49
RESound● Frustum tracing
● Edge diffraction
● Receiver is hidden from source
![Page 50: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/50.jpg)
ACM MM 09October 1923, 2009
50
RESound● Frustum tracing
● Edge diffraction
● But diffracting edgeis visible
![Page 51: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/51.jpg)
ACM MM 09October 1923, 2009
51
RESound● Frustum tracing
● Edge diffraction
● Create diffractionfrustum
![Page 52: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/52.jpg)
ACM MM 09October 1923, 2009
52
RESound● Frustum tracing
● Edge diffraction
● Diffracting soundreaches thereceiver
![Page 53: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/53.jpg)
ACM MM 09October 1923, 2009
53
RESound● Diffuse reflections
● Uses ray tracing
● Collection sphere● Same size as listener's head (0.3 m)● Record rays that hit collection sphere
![Page 54: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/54.jpg)
ACM MM 09October 1923, 2009
54
RESound● Ray tracing
● Diffuse reflection
![Page 55: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/55.jpg)
ACM MM 09October 1923, 2009
55
RESound● Ray tracing
● Diffuse reflection● Shoot rays from
source
![Page 56: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/56.jpg)
ACM MM 09October 1923, 2009
56
RESound● Ray tracing
● Diffuse reflection● Rays diffusely reflect
![Page 57: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/57.jpg)
ACM MM 09October 1923, 2009
57
RESound● Ray tracing
● Diffuse reflection● Some rays hit this
collection sphere
![Page 58: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/58.jpg)
ACM MM 09October 1923, 2009
58
RESound● Update stronger paths more often:
● Three simulations● Frustum tracing (first order, 1 thread)● Frustum tracing (third order, 7 threads)● Ray tracing, 200k rays (third order, 7 threads)
![Page 59: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/59.jpg)
ACM MM 09October 1923, 2009
59
RESound● From 3 simulations● Now have impulse response of:
● Direct sound● Specular reflection● Diffuse reflection● Edge diffraction
![Page 60: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/60.jpg)
ACM MM 09October 1923, 2009
60
RESound● Audio output
● Reverberation● 3d sound rendering● Dynamic scenes
![Page 61: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/61.jpg)
ACM MM 09October 1923, 2009
61
RESound● Reverberation
● Need to fill in late contributions● Use Eyring model [Eyring 1930]
● Statistically estimate sound decay
● Combing impulse responses● Frustum + frustum + ray tracing
![Page 62: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/62.jpg)
ACM MM 09October 1923, 2009
62
RESound● Reverberation
● Fit curve to impulse response
● Estimate time for signal to decay to 0.001% (RT60)
● Create reverberation filter with sound system
![Page 63: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/63.jpg)
ACM MM 09October 1923, 2009
63
RESound● HRTF is expensive
● Three impulse responses– 1st order frustum tracing– 3rd order frustum tracing– 3rd order ray tracing
● Compute only for 1st order frustum tracing● Other impulses use simple convolution
![Page 64: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/64.jpg)
ACM MM 09October 1923, 2009
64
RESound● Dynamic scenes
● Impulse response may change drastically● Can cause artifacts (clicking)
● Restrict motion speed● Crossfade audio frames
![Page 65: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/65.jpg)
ACM MM 09October 1923, 2009
65
Results● Test scenes
Roo
mC
onfe
renc
e
Sib
enik
Spo
nza
![Page 66: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/66.jpg)
ACM MM 09October 1923, 2009
66
Results
● Open scenes● Many triangles visible● Many reflections
![Page 67: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/67.jpg)
ACM MM 09October 1923, 2009
67
Results
![Page 68: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/68.jpg)
ACM MM 09October 1923, 2009
68
Results● Reverberation
● Begin with 6m cathedral● Dynamically expand cathedral to 30m
● With reverb and without
![Page 69: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/69.jpg)
ACM MM 09October 1923, 2009
69
Results
![Page 70: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/70.jpg)
ACM MM 09October 1923, 2009
70
Results● Limitations
● Must shoot many rays for diffuse reflections● Certain diffraction paths may not be found● Frustum tracing is approximate visibility
– May miss some paths
![Page 71: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/71.jpg)
ACM MM 09October 1923, 2009
71
Results● Specular + diffuse + diffraction components
● Uses unified representation: ray● Single acceleration structure
● Interactive rates on multicore PC● Handles large scenes● Moving source and listener● Scene can be dynamic
![Page 72: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/72.jpg)
ACM MM 09October 1923, 2009
72
Related and Future Work● Conservative frustum tracing [Chandak et al. 2009]
● GPU acceleration● Robust diffraction
● Conservative diffraction region● From region visibility – advanced diffraction
![Page 73: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/73.jpg)
ACM MM 09October 1923, 2009
73
Acknowledgements● Nikunj Raghuvanshi and Paul Calamia for
helpful advice● Sponsors
– ARO– NSF– DARPA/RDECOM– Intel– Microsoft
![Page 74: Interactive Sound Rendering in Dynamic Virtual EnvironmentsACM MM 09 October 19 23, 2009 1 RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak](https://reader036.vdocuments.us/reader036/viewer/2022090610/60700215a7621442220f74eb/html5/thumbnails/74.jpg)
ACM MM 09October 1923, 2009
74
Project websitehttp://gamma.cs.unc.edu/Sound/RESound/
Thanks!