digital audio effects processing & reverberation - aalto · 1 digital audio effects processing...

1

Digital Audio EffectsProcessing & Reverberation

Vesa Välimäki & Fabián Esqueda

ELEC-E5620 Audio Signal Processing24th Feb. 2017

Course Schedule in 2016 (Periods III, IV)

0. General issues (Vesa)1. History and future of audio DSP (Vesa) 2. Digital filters used in audio (Vesa) 3. Audio filter design (Vesa) 4. Analysis of audio signals (Vesa) 5. Audio effects processing (Fabian) 6. Synthesis of audio signals (Fabian) 7. Physics-based sound synthesis (Vesa) 8. 3-D sound and virtual acoustics (Prof. Ville Pulkki) 9. Sampling rate conversion (Vesa)10. Audio coding (Vesa)

©Välimäki, Parker

2

Outline

Part 1: Digital audio effects• Echo/delay• Flanging and Phasing• Chorus • Dynamic processing

(compression & expansion)• Other effects

Part 2: Artificial reverberation• Convolution• Schroeder and Moorer reverbs• Feedback delay network• Sparse noise reverb• Electromechanical reverb (L7!)

& Esqueda, 26.2.2016©Välimäki, Parker

Two Demos Today!!!• Virtual Analog Phasers• Modal Reverb


Source: http://ultimateclassicroc k.com/fi les/2 013/ 01/Eddie-Va n-Hale n.j pg

3

What is an audio effect?§ Any kind of audio signal processing applied to a

recorded or synthesized sound for creative purposes§ Possible purposes:

§ Impression of space (echo, reverb)§ Increasing perceived size of a sound (chorus)§ Introducing movement into a static sound (flanging, phasing)§ Altering timbre (distortion)§ Altering dynamics (compression, limiting)


Echo/Delay§ One of the simplest and earliest audio effects

§ Initially they were made using tape loops§ Digital version very simple

§ Delay line with feedback§ Filtering or distortion can be added to the feedback loop§ Extra taps can be added for more complex pattern§ Real-time implementation using “circular buffer”.*

& Esqueda, 26.2.2016

Sound example taken from: http://en.wikipedia.org/wiki/Delay_%28audio_effect%29*Good reference: The Audio Programming Book by R. Boulanger & V. Lazzarini


4

Flanging§ Invented by Les Paul (1915-2009) in 1945, but the name

came from John Lennon in 1966 (http://en.wikipedia.org/wiki/Flanging)

§ Original analog method for flanging § Copy the same sound on two open-reel tapes§ Play the 2 tapes on 2 synchronized tape machines§ Touch the flange of one tape reel to slow it down§ Get a nice �wooshing” phase-cancellation effect

§ Modern analog flanger pedals commonly use exotic Bucket-Brigade Device (BBD).


Does flanging sound familiar?

§ Many everyday cases§ C. Huygens (1693): the

sound of a fountain has a pitch when it reflects from a staircase

§ Moving and hissing sound source (or listener moving)

§ Jet airplane flying over a city§ Direct sound and its echo

§ Time-varying delay


5

Digital Flanger – Naive Version

§ A copy of the signal is fed through a variable digital delay line and added to the original

§ Produces a time-varying comb filter§ Magnitude response contains many uniformly spaced, moving

notches


Digital Flanger – Naive Version with LFO

§ Delay-line length is modulated with a Low Frequency Oscillator (LFO)§ Slow modulation frequency, approx 0.1 Hz – 1 Hz

& Esqueda, 26.2.2016

Pink noise Pink noise

Guitar examples by Timo Hiekkanen and Tuukka Lyly, TKK, 2007

E-Guitar

Drums Drums

E-Guitar


6

Digital Flanger – Thru Zero

§ Problem with naive implementation§ Dry and delayed signal never coincide exactly§ Modulation no longer centered about point of max/min effect

§ Solution: Add a static delay to the dry/static path.

z�mx(n) y(n)

z�l

Static path

Modulated path


Interpolated Variable Delay Line

§ In flanging, the delay-line length must vary smoothly to avoid discontinuities and clicks§ Otherwise “zipper noise” is produced

§ A fractional delay is needed§ Usually an FIR interpolation filter

& Esqueda, 26.2.2016

z-1x(n)

h(0) h(1)

y(n)

z-1 z-1

h(N)h(2) ...


7

Delay Line with Linear Interpolation

• For digital audio effects, linear interpolation may be sufficiently good

• Good idea for Learning Diary! (cough, cough)

& Esqueda, 26.2.2016

dd−1

)(nx

)(ˆ)( dMnxny −−=

1−zMz −


Flanging – The Movie

& Esqueda, 26.2.2016

Flanging - No Interpolation Flanging - Linear Interpolation


8


Demo Time!

Virtual Analog PhasersAleksi & Ricardo

Flanging vs. Phasing

§ Flanging§ Variable time-delay§ Short delay ( < 10ms)§ Hundreds of notches§ Notches harmonically related§ Number of notches is time-

varying

§ Phasing§ Variable phase shift§ Very short delay § Few notches (1-10)§ Notches not harmonic§ Notches can be individually

modulated§ Number of notches is fixed


9

Flanging or Phasing?

& Esqueda, 26.2.2016

Flanger Phaser Phaser

Flanger Phaser Flanger


Chorus§ The goal: make one source sound like many sources

§ Useful for vocals and electrical instrument sounds§ Very similar structure to flanger and echo effects

& Esqueda, 26.2.2016

⇒


10

Famous Chorus Examples


Chorus Unit: EHX Small Clone

Chorus Unit: Boss Chorus CE-2

Chorus Implementations (1)§ One choice: multiple feedforward paths with modulated

delay-lines (Orfanidis, 1996)§ Modulation waveforms may be sinewaves or lowpass-filtered

noise


11

Chorus Implementations (2)§ ‘Industry standard’ (Dattorro, 1997)


Chorus Implementations (3)§ The ‘industry standard’ structure is cheap to implement

§ Use one for each stereo channel, or more§ Generalized allpass-comb filter

§ Becomes an allpass filter, when delays and coefficients are equal

§ Negative feedback is used for flattening the spectrum (�white chorus�)

§ For clean effect, allpass fractional delay filter must be used for the variable delay, not linear interpolation

§ Another great idea for Learning Diary!


12

Chorus Implementations (4)§ The industry standard structure can produce many effects

§ Vibrato: blend = 0, feedback gain = 0, large modulation depth§ Flanger: small delay (< 10 ms) § Doubling (double tracking) when blend = feedforward gain, feedback

gain = 0, large delay (> 10 ms), with random modulation § Echo: feedback or feedforward gain is zero; a lowpass filter is inserted

in the non-zero path; delay is large (> 50 ms) § Stereo effects: modulating sine waves out of phase or in quadrature

for the 2 channels


Crazy cartoon-like effects!Great reference for this:

Udo Zölzer’s DAFX Book.

Chorus vs. Flanging§ Flanging

§ Small time delays (<10ms)§ Signals not separable by ear

(integration time of ear ≈ 2ms)§ Min. delay = 1 or 0 sample(s)§ Deep notches wanted for

strong effect

§ Chorus§ Larger time delays (>5ms)§ Separate signals perceived§ Min. delay approx 5 ms§ Notches usually

undesirable

& Esqueda, 26.2.2016

z�mx(n) y(n)

z�l

Static path

Modulated path


13

Dynamics Processing§ Compressor reduces the dynamic range of an audio signal

§ Pop music, radio and TV broadcasting, live PA systems§ Special case: Limiter, which saturates at a certain max. level

§ Expander increases the dynamic range§ For example to reduce background noise in silent passages§ Special case: Gate, which mutes the signal below a threshold

Compressor

x0 x (dB)

Limitery0

y (dB)

Expander

x (dB)

Gate

y0

y (dB)

x0


Time-Domain View of Compression§ When signal level gets high, the gain is reduced

automatically§ 2 parameters: attack time TA and release time TR


14

Digital Feed-forward Compressor§ Signal level detector

§ Temporal envelope: average power over a short time interval§ Gain must not be changed instantaneously (aliasing can occur)

§ Gain computer§ Gain G is adjusted based on signal level (power)

Level detector


Level Detector§ Full-wave rectification (abs) and temporal averaging

§ For example, a leaky integratory(n) = (1 – a1) |x(n)| + a1 y(n – 1)where a1 = 1 – ε (such as a1 = 0.99)


15

Compression of Musical Signals§ After compressing the signal, the overall gain is

increased by applying make-up gain§ Many potential uses

§ Maximizing loudness, like in the recent “loudness war” § Controlling transients/emphasizing decay


Gating and Limiting of Musical Signals§ Limiting is an extreme version of a compressor

§ Signal value is not allowed to exceed a certain level§ Mainly used to maximize loudness

§ Gating mutes audio when below a certain threshold amplitude§ Used to e.g. remove background noise between notes§ Can also be used creatively (e.g. gate chord sound based on

amplitude of high-hats)


16

Multiband Compression• Split audio signal into sub-bands for improved performance.• Avoid “pumping” effect.


Demo by Tae Ho Kim and Elias Raninen, ASP-2016

comp

comp

comp

comp

A Brief Note on Distortion§ Distortion very importantant for many devices, not just

for guitars (recall Moog demo).§ Produces harmonics + intermodulation components§ Simplest digital implementation uses algebraic function

to map input samples to output samples (assume memoryless behavior, not true in tube amps!)

§ More complex models (e.g. WDFs) describe I-V relationships in a circuit (e.g. diode clipper simulators)

f(x)x(n) y(n)


17

A Brief Note on Distortion (cont’d)§ Simple fuzz-type guitar distortion could use hard

clipping, or softer function such as tanh(x)§ Add asymmetry to function to sound like tube distortion§ Oversampling required to avoid aliasing

−2 −1 0 1 2−1

−0.5

0

0.5

1

Input

Output

tanh(x)

& Esqueda, 26.2.2016

tanh( chirp )


Other Effects (not exhaustive)§ Vocoder

§ Technique from speech coding misused for effect. § Uses large banks of bandpass filters to analyse the spectral envelope of a

sound and apply it to another sound§ Sounds like robot voice§ Analog versions also exist! For example, Korg VC-10 Vocoder

§ Frequency shifting (do not confuse with pitch-shifting)§ Shifts all frequencies by a fixed amount additively, hence ruining harmonic

relationships§ Can also be used to produce very rich chorus sounds

§ Autotune (1998-)§ Highly popular pitch corrector with quantization § Uses LPC and interpolation, or spectral techniques


18

Other Effects (not exhaustive)§ Wah-wah

§ Center frequency of a resonator is modulated with envelope follower, control pedal, or LFO

§ Filtering (telephone sound, resonances etc.)§ Enhancer

§ Harmonic distortion of only high frequencies to increase brightness

§ Time and pitch modification§ Can be implemented using granular or spectral methods§ More in L9!

§ Spatial audio effects§ Stereo expansion, 3-D sound etc.§ More?



Demo Time!

Modal ReverbGeorg & Lauri

19

Part 2: Artificial Reverberation§ Two main goals

1. Adding ambience for dry recordings§ Synthetic sounds and studio recordings are (almost) anechoic§ Sound post-production (e.g., films), computer game audio§ Auditory approach: sound effects

2. Simulation of room acoustics§ Virtual reality § Physical approach: make more natural-sounding

§ In both cases, sound propagation in rooms is imitated§ Propagation delay, reflections from surfaces, decay caused by

distance and air damping


Room Impulse Response (RIR)§ Direct sound

§ Speed of sound ≈ 345 m/s, delay usually about 10…200 ms

§ Early reflections§ Surfaces reflect, absorb, and scatter sound waves§ About 50…100 ms after the direct sound

§ Late reverberation


20

Historical Artificial Reverberation§ Demand for artificial reverberation predates digital

technology§ Two common early principles 1. Echo Chambers – specially designed rooms2. Electromechanical Reverb – vibrating metal objects

like springs and plate (L7!)


Convolution Reverb§ Measure and store impulse responses of real spaces

§ A long FIR filter with RIR samples as coefficients (cf. sampling)

§ A “dry” musical signal is filtered with the FIR filter§ Computationally intensive

§ Example: A stereo impulse response of 2 seconds (fs = 50 kHz) → 2 � 100,000 multiplications / sample→ 1010 multiplications / s (= 10 GFLOPS)

§ Straightforward but limited§ Applicable to imitation of real spaces only§ RIRs difficult to measure and processing needed to clean them

up (audible noise floor) § Limited flexible control


21

Algorithmic Reverberation§ Direct sound (perhaps with delay and attenuation) § Early reflections: a sparse FIR filter (Schroeder, 1970)

§ A long delay line with some taps to model the reflections

§ Late reverberation: impulse response reminiscent of noise§ One option is to imitate it with an exponentially decaying noise burst§ Another option is an IIR structure employing long delay-lines and feedback§ Devising various techniques for this purpose is a popular sport (Schroeder

1962, Moorer 1979, Jot & Chaigne 1991, … Välimäki et al., 2012)


Schroeder Reverb Algorithm

§ Schroeder (1962) developed the 1st reverb algorithm§ Four comb filters in

parallel§ Different delays§ Loop gain < 1

§ A few allpass-comb filters in cascade to spread the impulse


Great learning diary experiment! :3 Ref: https://ccrma.stanford.edu/~jos/pasp/Schroeder_Reverberators.html

22

What Happens Inside a Schroeder Reverb?§ Let’s consider this specific design developed by John Chowning and

track a single impulse!


Ref: https://ccrma.stanford.edu/~jos/pasp/Schroeder_Reverberators.html

AP1 AP2 AP3

imp

Moorer Reverb Algorithm§ Moorer (1979) inserted a one-pole lowpass filter inside

the comb filters: Lowpass-comb filter§ Long impulse response, where high frequencies decay faster

than low frequencies§ More natural behavior§ Less �metallic� sound

One-pole filter


23

Moorer Reverb Algorithm (2)§ More comb filters → Better reverberation

§ At least 6 lowpass-comb filters in parallel (Moorer, 1977)

KP = Allpass filter

Direct sound

AK = Lowpass-comb filter


Feedback Delay Network• A generalized comb filter structure that uses a

feedback matrix (Jot and Chaigne 1991)• Lossless mixing matrix provides diffusion

An order 3 FDN


24

Feedback Delay Network

• The loop filter controls the frequency-dependent delay• FDN sound examples

(FDN with 4 delay lines)• Original• Fairly dry room– T60 = 0.7 s at 0 Hz– T60 = 0.14 s at 22 kHz• Very reverberant– T60 = 3.0 s at 0 Hz– T60 = 0.6 s at 22 kHz

(Sound examples and figure produced by Riitta Väänänen, 2000)


© 2001-2011 Vesa Välimäki

Reverberator Using SFIR Filtering• A feedback loop containing a sparse FIR (SFIR) filter is a

more efficient structure than convolution (Rubak & Johansen, 1998)

• The SFIR coefficients can be random numbers• Velvet noise = sparse random noise (Karjalainen & Järveläinen, 2007)

http

://w

ww

.aco

ustic

s.hu

t.fi/d

emos

/Vel

vetR

ever

b/


Source: Reverberation Modeling Using Velvet Noise by Matti Karjalainen and Hanna Järveläinen

Real

Modeled

25

Velvet Noise• +1 or -1 at a random position, with many zero samples• Sounds smoother (less rough) than Gaussian noise (Karjalainen

and Järvinen, 2007)• Signals can be convolved with ‘velvet noise’ without multiplications,

and with only a small number of additions/subtractions

Gaussian noise Velvet noise: 3000 spikes/sec 1000 spikes/sec

http

://w

ww

.aco

ustic

s.hu

t.fi/d

emos

/Vel

vetR

ever

b/

200 spikes/sec


Literature – Reverb§ V. Välimäki, J.D. Parker, L. Savioja, J.O. Smith, J.S. Abel, “Fifty years of artificial

reverberation”, IEEE Transactions on Audio, Speech and Language Processing, July 2012, pp 1421-1448

§ J. Dattorro, �Effect design—part 1: Reverberator and other filters,� J. Audio Eng. Soc., vol. 45, no. 9, pp. 660-684, Sept. 1997. Available online at: http://www.stanford.edu/~dattorro/research.html

§ W. G. Gardner, �Efficient convolution without input-output delay,� J. Audio Eng. Soc., vol. 43, no. 3, pp. 127-136, March 1995.

§ W. G. Gardner, �Reverberation algorithms,� in M. Kahrs and K. Brandenburg (eds.), Applications of Digital Signal Processing to Audio and Acoustics. Kluwer, 1998, pp. 85-131.

§ J. M. Jot and A. Chaigne, �Digital delay networks for designing artificial reverberator,� in Proc. AES 90th Convention, Paris, France, Feb. 1991.

§ M. Karjalainen and H. Järveläinen, �Reverberation modeling using velvet noise,�in Proc. AES 30th Int. Conf. Intelligent Audio Environments (AES-30), Saariselkä, Finland, March 2007.


26

Literature – Reverb (2)§ K.-S. Lee, J. S. Abel, V. Välimäki, T. Stilson, and D. P. Berners, “The switched

convolution reverberator,” J. Audio Eng. Soc., vol. 60, no. 4, pp. 227–236, April2012.

§ J. A. Moorer, �About this reverberation business,� Computer Music J., vol. 3, no.2, pp. 13-28, 1979. Reprinted in: C. Roads and J. Strawn (eds.), Foundations ofComputer Music. MIT Press, 1985.

§ P. Rubak and L. G. Johansen, �Artificial reverberation based on a pseudo-random impulse response,� in Proc. AES 104th Convention, Preprint 4725,Amsterdam, 1998.

§ M. Schroeder, �Natural Sounding Reverberation,� J. Audio Eng. Soc., vol. 36,no. 9, 1962.

§ J. O. Smith, �A new approach to digital reverberation using closed waveguidenetworks,� in Proc. Int. Computer Music Conf., Vancouver, pp. 47-53, 1985.

§ U. Zölzer, Digital Audio Signal Processing. Wiley, 1997. Chapter 6 (�RoomSimulation�), pp. 181-205.


Literature – Effects§ J. Dattorro, �Effect design—part 2: Delay line modulation and chorus,� J. Audio

Eng. Soc., vol. 45, no. 10, pp. 764–788, Oct. 1997. Available online at:http://www.stanford.edu/~dattorro/research.html

§ R. Dobson, A Dictionary of Electronic & Computer Music Technology. OxfordUniversity Press, 1992.

§ W. M. Hartmann, �Flanging and phasing,� J. Audio Eng. Soc., vol. 26, no. 6, pp.439–443, June 1978.

§ S. J. Orfanidis, Introduction to Signal Processing. Prentice-Hall, 1996. Section8.2 (�Digital Audio Effects�), pp. 355-388.

§ J. D. Parker, “A simple digital model of the diode-based ring-modulator,” in Proc.14th Int. Conf. Digital Audio Effects (DAFx-11), Paris, France, Sept. 2011.Sound examples: http://www.acoustics.hut.fi/publications/papers/dafx11-ringmod/

§ G. D. White, The Audio Dictionary (2nd ed.). University of Washington Press,1991.


27

Literature – Effects (2) • V. Välimäki, S. González, J. Parviainen, and O. Kimmelma, ”Digital audio

antiquing – Signal processing methods for imitating the sound quality ofhistorical recordings,” Journal of the Audio Engineering Society, vol. 56, no. 3,pp. 115–139, March 2008.

• P. A. A. Esquef, L. W. P. Biscainho, and V. Välimäki, “An efficient algorithm forthe restoration of audio signals corrupted with low-frequency pulses,” Journal ofthe Audio Engineering Society, vol. 51, no. 6, pp. 502–517, June 2003.


digital audio effects processing & reverberation - aalto · 1 digital audio effects processing...

Documents