PowerPoint Presentation

1

The NESS ProjectWork supported by the European Research CouncilMUSICA Seminar Series University of Edinburgh 13 February 2013Stefan Bilbao, Paul Graham, Alan Gray, Brian Hamilton Kostas Kavoussanakis, James Perry, Alberto Torin, and Craig Webb

Acoustics GroupEPCC

2Background

2

3Background

3

4

4The HPC Centre of the University of Edinburgh 70 staff4M turnover (almost) all from external sources5Distributed digital asset management systemAccurate supervision of the progress of a film productionSecure asset transfer between entities working on the filmComplements existing film editing software (e.g. Nucoda)Software support for current day-to-day manual workflow processes

Case Study: FilmGrid5

Photo by Brad & Sabrina6

Case Study: FilmGrid6

Photo by Jeremy Keith7

7Academic:Permanent staff with degrees in various backgrounds:Physics, Computer Science, Mathematics, Chemistry, BiologyEducation and Training provider:MSc in HPCBespoke training on HPC, accelerators, multi-core computingStrengths:Project managementInterdisciplinary projects: small pilots, distributed programmesUser supportTechnical:Software development for academia and industryCode optimisation (serial, parallel, accelerators)Simulation and modellingWide language, operating system and computer architecture expertise

http://www.epcc.ed.ac.uk/

8NESSAn ERC funded project (PE6: Informatics), running at Edinburgh, 20122017.

The goal: digital sound synthesis

of musical sound

Today:

Digital sound synthesis and physical modelingNess project: structure and activitiesGroup member presentations

9Digital Sound SynthesisA longstanding attempt to move away from the use of recorded sound (sampling)Analogous to computer graphics rendering?At the philosophical levelyes

At the technical levelnot really!

Sampling

Synthesis10Abstract Digital Sound Synthesis

Early synthesis methods (1950s1960s): based on simple heuristic building blocks---efficient and easy to program

Frequency modulation synthesis: a very successful variant!Still extremely popular!Can be difficult to control (lots of user input), and sound quality is generally very artificial!Wavetables:Sinusoidal Oscillators:

FM trumpet

FM bell11Physical Modeling Sound SynthesisPhysical models:based on physical descriptions of musical objectscan be computationally demanding potentially very realistic* sound control parameters: few in number, and perceptually meaningful

* realism needs a good definition, if there is not a real-world reference!

A fair degree of hybridization abounds:physical modeling + sampling is analogous to, say, motion capture!

12Methods: Lumped Mass Spring Networks

networks of masses/springs/dampers + simple ODE solver

basis for Cordis and Cordis Anima systems (Cadoz and collaborators, Grenoble, from 1979 to present!)

generally abstract (modular), but if put in a regular arrangement, it is possible to simulate distributed objects (strings, membranes, etc.)

Earliest large scale attempt at physical modeling synthesis:

CymbalTimpani13Methods: Modal Synthesis

basis for Modalys synthesis system (IRCAM, 1985present)

geared towards linear objects (with interesting extensions to the nonlinear caseVolterra series, e.g.)

a lot of offline precomputation (modal shapes, frequencies)A different approach: decompose dynamics of vibrating object into modes

14Methods: Digital Waveguides

Yet another approach: decompose dynamics of vibrating object into traveling waves

developed at CCRMA, Stanford University, 1985present

roots in early scattering-based speech synthesis methods (Kelly Lochbaum, 1962)

basis for many synthesis systems, including Yamaha VL1 (1994)

meant for simulating distributed linear objects in 1D (strings, acoustic tubes)extremely efficient!

Guitar15Methods: Time-stepping Methods (FDTD, FVTD, etc.)

The obvious approach: represent dynamics of vibrating object on a grid and integrate using direct solvers

distinct roots in musical acoustics investigations, and a few early synthesis attempts (Ruiz, vibrating string, 1969!)

tools exist to handle virtually any systemmuch more general than other methods

buta lot of specialization work for audio applications16NESS: Target Systems

Brass Instruments

Electromechanical Instruments

Nonlinear Plate and Shell Vibration

Trying to span the full range of musical acoustic systemsdifficult to approach using other physical modeling techniques

TrumpetGongCymbal17NESS: Target Systems

Modular Synthesis Environments

Embeddings and Spatialization

Room Acoustics Modelling

Excerpt: Orbit, G. Delap, 2009Snare Drum18Sample rates and bandwidthA basic constraint for audio: choice of sample rate Fs, and time step k = 1/FsConstraint 1 (necessary): Need to be able to fully render audio up to limits of human audio perception, so:Fs 40 000 Hz, k 1/40000 sConstraint 2 (desirable): Dont want to render audio above this range, for efficiency reasons, so:Fs 40 000 Hz, k 1/40000 s

Time step is smalllots of computational work to doSome aspects of time stepping methods need to be reconsidered in this light!

19Grids and Bandwidth Limitation

Suppose operation at a given sample rateneed to choose the grid carefully, for perceptual reasons:decrease in operation countIncrease in grid spacingdecrease in output bandwidth an additional constraint in numerical designcertain techniques (grid refinement) are dangerous in an audio contextx 2x 4Simple string model

20Audibility of Numerical Dispersion

Example: thin bar, simple explicit FD method:Mistuning!

Exact:44100 Hz:Phase velocityExact and NumericalCareful design necessarymethods with free parameters allow a means of tuning the schemeat the price of linear system solutions (hard on GPU!)

Numerical Dispersion: speed of wave propagation is incorrect, numerically!

21Numerical InstabilityA critical concern in synthesis design for non expert users

Linear membrane instabilityNonlinear shock wave instabilityNot too hard to deal withHarder to deal with without compromising audio output22Modular Instability

Consider two rudimentary systems

Problems can appear here if the connection is not handled properly:

Stable ConnectionUnstable ConnectionMass/springIdeal StringDifficulties are compounded for more complex systems

23Energy-based Stability

Numerical energy conserved to machine accuracy:

Extremely useful in debugging, and in designing complex modular systems:giving a stability guarantee24Computational Costs and HPCAudio sample rates are high: 44 100 Hz, 48 000 Hz, 92 000 HzFlop rates/memory requirement scale as power of sample rate (2,3,4)arithmetic operations/second output, at 48 000 Hz:optimal realtime performance on commercially available single coreNonlinear plates/shellsBrass instrumentsElectromechanical Instruments1061071081091010101110121013101410151016 1017Small embeddingsSmall roomsConcert HallsMusical use/experimentation: reasonable compute time (no overnight jobs!)Solutions: Parallel implementations (GPU, e.g.)New algorithmic issues: parallelizability, memory management, stability in finite precision3-D Room Acoustics(Brian Hamilton, Acoustics Group)25

25Simulating 3-D Room AcousticsA wave is a spatial field that changes over timeSound propagates as a pressure waveSimulating sound wave propagation:Pick some 3-D lattice (grid) of pointsCalculate sound pressure at each pointIterate in time...Problem to solve:How to do this as efficiently as possible?Any audible artifacts? How to minimise them?2626Spatial Lattices

27Which to choose?27Spatial Lattices

28

Many choices...28Spatial Lattices

29Waves should propagate uniformly in every directionSymmetry is key!

Stacking fruit29Numerical DispersionNumerical dispersion wave speed errorSimulated waves propagate along axes of grid

Wave speed depends on grid orientation!30Example: Without Dispersion31

31Example: With Dispersion 32

32Wave Speed Error33We want the error to be isotropic (direction independent)Delicate cancellation of error in space and time

33Wave Speed Error34

34Wave Speed Error35

35Audible Artifacts: Examples36Castanet (clean)

Castanet (dispersive)

Guitar (clean)

Guitar (dispersive)

3637Percussion InstrumentsMembranesPlates and ShellsLLow excitation!NLHigh excitation!L = Linear, NL = Non-linear

(Alberto Torin, Acoustics)3738Linear Plates

w = transverse displacement = density, H = thickness, D = stiffness parameter There is no interaction between different modes!

3839Non-linear PlatesAdd non-linear terms to previous equation

F = Airys function, E = Youngs modulus

von Krmn equations for non-linear plates3940Non-linear Plates Energy exchange between different modes is allowed!

- Crashes, Pitch glide effects

4041Air couplingAdd the pressure on the plate

Introduce the acoustic field , that obeys the wave equation

Add coupling conditions between the air and the plate4142Numerical schemesStability and Energy conservation Need for a Fast algorithm Bottleneck of the code is the solution of a sparse linear system (matrices involved have a few non-zero entries) We can use iterative solvers

works well for the simple plate needs extra work when air coupling is present4243Example: MultiPlate3D

Roll gestureSeveral strikes4344What is a GPU?Graphics processing unitOriginally designed for rendering 3D graphics fastNow also used for general purpose computations (GPGPU)Very well suited for problems like oursEspecially the 3D onesSame simple computation required for huge number of points44

45Porting ProcessPort from Matlab to CFaster than Matlab, will run anywhereEasier to debug and modify than CUDAGood basis for CUDA port45MatlabCCUDAOptimized CUDAPort from C to CUDASome code (e.g. setup code) remains in CTime critical main loop is ported to CUDA

Optimize CUDA codeGain high performance (as far as possible)

46Example3dabc codeSimulates a 3D box of air with various boundary conditionsRun times:

46VersionRun time (s)Speed upMatlab311xC0.44170xCUDA0.0786395x

Matlab version not optimisedSmall simulation size - would expect larger speed-up from C to CUDA for large size47Large-scale 3D virtual acoustics(Craig Webb, Acoustics)Computing 3D wave propagation in a virtual space.Dynamic simulations, with full wave behavior. Can inject dry audio to produce reverberation.Or embed virtual instruments.

48Computation SizeAt a sample rate of 44.1kHz : 1 cubic metre requires 422 thousand grid points. 1 second of output requires 185G floating-point operations. 2,000 cubic metres 370T operations. A 10 second simulation 3.7P operations, thats 3,700,000,000,000,000.

Use multiple GPU cards to accelerate the model. With 4 cards, speedup over serial C code is in the range x100 ~ x140. Under an hour per second of simulation, instead of 5 days in serial C. Requires 10Gb of memory at single precision.

49Hall Simulation: Dry Audio InputAudio examples of 2,000 cubic metre hall

Dry guitar input : Output :

More guitars :

Opera singer (anechoic) : Output :

Can move sound around during runtime :

50Embedded InstrumentsTimpani Drum

The timpani drum is a good test case for 3D physical modeling.

We use a non-linear circular membrane, attached toa parabolic shell with fixed boundaries.

This is then placed inside the room simulations, andwe can model any number of timpani inside the space.These can then be played together, by specifying thetiming and type of strikes on each drum.

Audio examples

One Timpani : Two Timpani : Three Timpani : Four Timpani :

51Creative Uses: Composition

A new world of sound for musicians and composers---fully multichannel, synthetic music environmentsBut---a learning curve! As for any mature instrument design

52Control and InterfacesNESS: audio only! Not really any attempt at building live, performable instruments, or developing complex interfaces

Two subsequent levels of work:Figuring out useful, parsimonious ways of representing inputUI design (simple!)53NESSThank you for your attention

Questions?53