aerophones in flatland - microsoft.comย ยท gpu implementation r g b a ๐ ๐ฏ ๐ฏ state per...
TRANSCRIPT
Aerophones in FlatlandInteractive Wave Simulation of Wind Instruments
Andrew Allen
Nikunj Raghuvanshi
Spectrogram
Simulation Domain
Mouthpiece
Wind Instruments
BellMouthpiece Bore, Toneholes
reedmouth bore
Non-linear Linear
Feedback
Resonant Cavity
Excitation models: Single Reed (Clarinet)
Dalmont, J.P., Gilbert J., and Ollivier, S. 2003.
Excitation models: Lips (Trumpet)
Adachi and Sato. 1996.
Excitation models: Air Jet (Flute)
de La Cuadra. 2006.
Bell
Realtime synthesis: Digital Waveguides
Bell radiation
Mouthpiece Bore
breath pressure
+shift
shift
+
ร Bell reflection
SMITH, JULIUS O. 2010. Physical Audio Signal Processing.
http://ccrma.stanford.edu/หjos/pasp/
(online book, accessed Jan 2014).
Bell
Realtime synthesis: Digital Waveguides
Bell radiation
Mouthpiece Bore
breath
+shift
shift
+
ร
Non-linear
excitation model
Bell reflection
= frequency-dependent filter
Bell
Realtime synthesis: Digital Waveguides
Bell radiation
Mouthpiece Bore
breath
+
Tonehole
radiation
Tonehole
reflection
Tonehole
shift
shift
+
ร
Non-linear
excitation model
Bell reflection
Tonehole
radiation
Tonehole
reflection
Tonehole
= frequency-dependent filter
Bell
Our approach
Bell radiation
breath pressure
Tonehole
radiation
Tonehole
reflection
shift
shift
Bell reflection
Tonehole
radiation
Tonehole
reflection
2D wave simulation on GPU
Mouthpiece Bore
Color = pressure
Borrow and adapt
Non-linear
excitation model
+
+
ร
Non-linear
excitation model
Advantages
โข Signal processing networks require expertise to design and ensure physical plausibility.
โข Geometric manipulation is intuitive.
โข Guaranteed physical plausibility.
โข Lower expertise bar for musical experimentation.
Challenges
โข System is driven non-linearly and has perceptually salient transients (note beginnings/ends).
โข Direct time-domain finite-difference solution.
โข Standard finite difference generates artifacts on changing geometry.
โข Need millimeter-scale resolution.
โข Numerical stability requires small time-steps for wave equation.
โข ~3.8mm resolution at 128,000Hz on the GPU.
Linear Wave Equation
๐๐
๐๐ก+ ๐โฒ๐ = โ๐๐2๐ป โ ๐ฏ
๐ฝ๐๐ฏ
๐๐ก+ ๐โฒ๐ฏ = ๐ฝ2
โ๐ป๐
๐+ ๐โฒ๐ฏ๐
(๐โฒ = 1 + ๐ฝ โ ๐)
Perfectly matched layer (PML)
๐๐
๐๐ก+ ๐โฒ๐ = โ๐๐2๐ป โ ๐ฏ
๐ฝ๐๐ฏ
๐๐ก+ ๐โฒ๐ฏ = ๐ฝ2
โ๐ป๐
๐+ ๐โฒ๐ฏ๐
(๐โฒ = 1 + ๐ฝ โ ๐) ๐ โ 0
๐ = 0
Dynamic Geometry
Tone Holes, Valves, Slides, Mutes
Abrupt geometric changes: clicks
Our formulation (time-varying PML)
โข ๐ฝ(๐, ๐ก) โ [0,1] introduces smoothly-varying dynamic geometry.
โข ๐ฏ๐ enforces boundary conditions and input flow from mouthpiece.
โข Handles all phenomena we model.
๐๐
๐๐ก+ (1 โ ๐ฝ + ๐)๐ = โ๐๐2๐ป โ ๐ฏ
๐ฝ๐๐ฏ
๐๐ก+ 1 โ ๐ฝ + ๐ ๐ฏ = ๐ฝ2
โ๐ป๐
๐+ (1 โ ๐ฝ + ๐)๐ฏ๐
Our formulation (time-varying PML)
๐ฝ๐๐ฏ
๐๐ก+ 1 โ ๐ฝ + ๐ ๐ฏ = ๐ฝ2
โ๐ป๐
๐+ 1 โ ๐ฝ + ๐ ๐ฏ๐
๐ฝ๐๐ฏ
๐๐ก+ 1 โ ๐ฝ + ๐ ๐ฏ = ๐ฝ2
โ๐ป๐
๐+ 1 โ ๐ฝ + ๐ ๐ฏ๐
๐ฝ = 1: Air
๐ฝ๐๐ฏ
๐๐ก+ 1 โ ๐ฝ + ๐ ๐ฏ = ๐ฝ2
โ๐ป๐
๐+ 1 โ ๐ฝ + ๐ ๐ฏ๐
๐ฝ = 0: Boundary
Smoothly interpolates
between Boundary and
Air state in every cell(๐ = 0 inside domain)
Our formulation: natural transients
โข The transition rate of ฮฒ controls the smoothness of the transition.
โข Results in a simple conditional-free update equation for the entire domain.
Off On
Wall losses
โข 2D simulations support transverse resonances
โข Wall loss modeling is required (unlike 1D models)
Off On
High-amplitude non-linearity
โข Brass instruments have high amplitudes inside the bore.
โข Makes brass sound brighter.
Off On
๐๐ฆ
๐๐ฅ
๐๐ฆ
๐๐ฅ
GPU Implementation
โข Solving Finite Difference uses a 5-point 2D stencil.
โข Neighbor pressures and velocities are used to update center pressure.
๐๐ฟ
๐๐ท
๐๐
๐๐
๐
GPU Implementation
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
Per Fragment
๐๐ฆ
๐๐ฅ๐
๐๐ฆ
๐ ๐๐ฅ
State structure stores:
โข ๐ฝ value (boundary-ness)
โข ๐ value (PML)
โข Is Excitation?
โข Is Listener?
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
Framebuffer Texture
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
Per Fragment
1 2
3 4
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
Framebuffer Texture
Per Fragment
1 2
3 4
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
Framebuffer Texture
๐ก โ 2
๐ก๐ก + 1
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
๐ก โ 1
1 2
3 4
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
Framebuffer Texture
๐ก + 1
๐ก โ 1๐ก
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
๐ก โ 2
1 2
3 4
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
Framebuffer Texture
๐ก
๐ก โ 2๐ก โ 1
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
๐ก + 1
1 2
3 4
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
Framebuffer Texture
๐ก โ 1
๐ก + 1๐ก โ 2
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
๐ก
1 2
3 4
GPU Implementation
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Output pressure written out in block in reserved space on FBO
Framebuffer Texture
๐ก โ 2
๐ก๐ก + 1
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
๐ก โ 1
1 2
3 4
GPU Implementation
Framebuffer Texture
๐ก โ 2
๐ก๐ก + 1
R G B A
๐ ๐ฏ๐ฅ ๐ฏ๐ฆ state
๐ก โ 1
โข Values for each cell are stored in color channels.
โข Simulation grid is represented as a 2D texture.
โข Four copies of the simulation are stored in one large texture.
โข Ping-pong with R/Ws on one texture.
โข Write output pressure (sound) to reserved space on the FBO.
Clarinet
Chalumeau melody Altissimo melody (register key)
Saxophone
Simple melody Fast Squeaks
Flute
โRobotโ Performer Wind Controller Interface
Bugle & Trumpet (brasses)
Lips Overblowing Valve System
Trumpet w/o Bell and w/ Mutes
Bell On/Off Straight, Cup and Harmon Mute
โSlide Whistleโ and โMenorahโ
Dynamic Bore Geometry Interlocking Valve System
โTuba?โ and โHybridโ
Implausible-to-construct Instrument Reed, Lips, Valve, Tonehole, Bell
Comparisons to STK (Digital Waveguides)
Low note High note
Conclusions and Future Work
โข First system for real-time 2D simulation of Aerophones
โข Improving the control of excitation mechanisms
โข Automatic tuning of geometry
โข Generalized excitation model
โข Modeling of larynx/syrinx (speech synthesis/bird song)
Thank You! Questions?
Special thanks for providing performances โ
โข Kyle Rowan, clarinetist
โข Paul Hembree, trumpeter