3-d spatialization and localization and simulated surround sound with headphones lucas o’neil...
Post on 20-Dec-2015
219 views
TRANSCRIPT
![Page 1: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/1.jpg)
3-D Spatialization and Localizationand Simulated Surround Sound with
Headphones
Lucas O’NeilBrendan Cassidy
![Page 2: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/2.jpg)
Overview
• 3D with headphones– HRTF Model– Convolution– 360 + elevation Panning
• Upmixing– Pro Logic– Delays– Filters– Sub– Autopanning
• Downmixing with HRTF
![Page 3: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/3.jpg)
Mathematical HRTF Model
• We tell 3D directionality through 3 cues other than just ITD and IID
• Pinna Reflections
• Shoulder and Torso Reflections
• Head Shadow and ITD
• Can model using filters/delays.
![Page 4: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/4.jpg)
• Shoulder/Torso reflection simulated by echo:
• Pinna reflections via a tapped delay line
![Page 5: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/5.jpg)
• Head shadow diffracts the sound wave.• Simulated in digital domain by 1st order IIR filter:
• ITD due to separation obtained by allpass filter with group delay:
![Page 6: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/6.jpg)
• Input azimuth and elevation angle.
• Delay by shoulder echo.
• Add delay line due to pinna reflections.
• Filter through Head Shadow and ITD filters.
• spatialization~!
![Page 7: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/7.jpg)
Convolving with HRIR
• HRIR = Head Related Impulse Response
• Measured with KEMAR dummy (MIT)
• Convolve audio with impulse response corresponding to appropriate angle
![Page 8: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/8.jpg)
![Page 9: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/9.jpg)
360 Corkscrew Panning
• Demo to shown point source spatialization.
• Pick rotation frequency for azimuth and elevation.
• Breaks up signal into blocks and performs HRTF with different angles on each block to simulate 360 rotation around head and elevation from -90 to +90
• Done with both mathematical model and convolution techniques.
![Page 10: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/10.jpg)
5.1 Surround Sound
![Page 11: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/11.jpg)
Upmixing to 5.1 Surround
• Investigated Dolby Pro Logic decoder.
• Initially used gains/phase shift matrix to split up stereo signal
• Tweaked further adding delays to center and surround channels
![Page 12: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/12.jpg)
• Pro Logic II has 3 modes of operation:– Movie (not used in this project)– ‘Pro Logic’– Music
• Surround channel uses 7kHz LPF in Pro Logic mode
• Surround channel uses Shelving Filter in music mode– Used 4kz cutoff for shelving.
• Surround channel has 20ms delay in Pro Logic mode, but not music mode
![Page 13: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/13.jpg)
Subwoofer Simulation
• 5 channel surround was losing some low frequency due to cross talk corellation and phase cancellation in surround channels.
• Solved by cloning low frequencies of signal (using 300Hz LPF), then mixing them back in the stereo channel after downmixing the 5 channels
![Page 14: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/14.jpg)
Autopanning
• Pro Logic mode uses autopanning to detect directionality and adjust 5 speaker mix.
• Preserves RMS energy in signal.
![Page 15: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/15.jpg)
Sub-band Autopanning
• Scope of project did not allow for implementation
• Surround upmixer breaks signals into bands and pans those bands to the appropriate location
• Can detect different instruments in music (like frequency keying in DAW software) and localize each instrument.
![Page 16: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/16.jpg)
Dolby Pro Logic Decoder
![Page 17: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/17.jpg)
Our Implementation of Upmixer
![Page 18: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/18.jpg)
Downmixing with HRTF
![Page 19: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/19.jpg)
• Resultant stereo sound file has convincing spatialization effects.
• Pro Logic mode autopanning implemented without sub-band separation tends to have vocals that jump back and forth across the left and right channels.
• Music mode sounded better (for music).
![Page 20: 3-D Spatialization and Localization and Simulated Surround Sound with Headphones Lucas O’Neil Brendan Cassidy](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4c5503460f94a2a9ab/html5/thumbnails/20.jpg)
Questions?