Interactive MultimediaSound

Mikael Fernström

Data sources

• Microphones and transducers– Sample acoustic reality

• Synthesis– Simulate reality (and beyond ;-)– Many different forms of synthesis, e.g.

additive, subtractive, FM, FOF, PM…• Algorithms

– e.g. auralisation/sonification

Primary processing

• Recording– Get clear recordings!– Signal-to-noise ratio– Acoustics

• Filtering– Supressing or emphasizing parts of the spectrum

• Mixing– Blending of several sounds sources, foreground,

background, speech, music, f/x

Sampling

• “So called” CD quality– 16 bit, 44.1 kHz– => 90 dB dynamic range– Does not match normal human perception!

• Current professional standards– 24 bit, 96 kHz– => 138 dB dynamic range

Sampling…

• 8 bit sounds crackling, noisy• Sample rates lower than 44.1 kHz, loss of

treble• Example:

– Lowest standard encoding for human speech approx 6 bits 5 kHz (sounds like bad CB radio)

Quantisation

Simple sine wave

Spectrum of sine

Noise spectrum

Aliasing

sound

FFT

Aliasing

Half of the FFT array gets reflected into the other half of the array

Hearing threshold

Masking

Formats

• Common formats:– WAV, AIFF, RAW…– Stores data “as is”

• Masking is used in perceptual compression, e.g. MP3, AAC, etc…

• MP3 is not MPEG!– It’s Layer 3 of MPEG-1 standard

Streaming

• Download complete file before playing• Start playing file when length of file is

sufficient to accommodate rest of download• Start playing when length of buffer matches

data rate

Streaming

MIDI

• Musical Instrument Digital Interface• Only contains control info, no sound!

– e.g. note-on, note-off, velocity/loudness

MIDI

• Sounds different depending on User synth, mapping, etc.

• General MIDI “standard”, GM• Lack of expression, only 7 bit• Sometimes problems with latency, 32.5 kb/s

– Playing 10 notes simultaneously => 9.2 ms