audible design - diagrammatic appendix
DESCRIPTION
: Audible Design -DiagrammaticAppendixTRANSCRIPT
AUDIBLE DESIGN
TREVOR WISHART
APPENDIX 2
A diagrammatic guide to sound compositional processes
Published by Orpheus the Pantomime Ltd 1994
Copyright Trevor Wishart 1994
1
UNIVERSITY OF ILLINOIS UBRARV
AT URBANA - CHAMP~IGr MUSIC
JJL 3~S-
J
WAVEFORH
sound represented in the time domain
When looking at much longer blocks of time waveform would be very compressed in display so a different format is used
SPECTRUM
sound represented in the frequency domain NB Frequency axis evenly spaced with respect to frequency amp NOT to pitch
Spectrum displaying analysis channels
LOUDNESS TRAJECTORY (loudness envelope) of a sound
at a mix
display
ALL REPRESENTATIONS ARE SCHEMATIC
to represent realistic waveforms spectra loudness trajectories but only to illustrate the principles involved in the various
Waveform spectral etc shapes have been chosen to clear as possible in diagrammatic form
JA
KEY
~tlme ~a~mp~~~~~~~~
a d
amp Wf
9
rt If tt time
time ~ IgtP
(Nsec~cent
bTII~
(Nse~
CUgtlParp
Op frq
~~
MIX PLACEMENT represents the times which sounds in begin Does NOT the whole of each sound
NB ~
We do not attempt or mix placements compositional processes described
v
make their transformations as
----
SAHPLING
afgt
1JIv~v~v~vti~O~i~~~a~O~~~~~t~dw~e~~~~ is stored as a sequence of O~p individual values or samples A ~ A derived from the wavefo rm
_ AIt Ii I AI shown
SEQUENCE ~rwri~ as
GElflRATION
It is often useful to generate sequences of timing information for musical composition A HIDI instrument does this by sending
data when a key is pressed or released We may also calculate sequences of times automatically
II1I 1IIII1IJ~gmiddot Regular times
t I I I I I I I I j I ItrM~ Times growing by addition of a time-unit
f I I I I I I I I I I I It~ Times growing exponentially
~1 I I ~ btll Times following Fibonacc i series
I I II IIF II II tmpound
Regular times slightly randomised
II I I II II I IIQ~ I IIIIIII~ III ~ 11111 ~ JIIIIIII~ II tQ Times completely random Times completely random but denser
~
uaveforrn ~ c l)
~ ~ raquo ~ Vl
~ ~
~
Spectrum
Il IIH Il
~ H ~ Z ~
~ ltV
~ H
~ J 0 It
+
II
f-
+
a a a-
l17me domain iepre senttltiort
IF(~uen~ domain representatlonl
I()
6
-CL
~ I
0 lt) ~ ~
FreqyencJ
~
0 I
c E lts
0 () f()
Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~
re~ s ~~~
Ij
~~III ~ V1
~Ili l ~ ~ pound en( 01
S ~
~ o
~I ~
tt
t ~ Q)
~
I
o o N
o
TItlE DOHAIN amp FREQUENCY DOHAIN
Q~ Igt I II - ~ tjl II
~ ~ ~ ~ c ~ pound ~ pound ~
WAVELENGTH FREQUENCY FITCH
Time do_in Frequency Domain ~p amp
I ~ f ti~ I
~e oat
WI1Fl7Jm~r ~i-~-j~ ~~~ h
JJVV) When wavelength becomes 12 as long there are twice as many wavecycles
in the same time Frequency = wavecycles-per-second
So Halve wavelength to double frequency Double wavelength to halve frequencY4
1 l
t~ - l~
20 4 a 60 V~ IDa
A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)
nultiplying all partial frequencies by 2 (or any number)
preserves this relationship
WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE
Pitch is therefore distributed differently to frequency
Equal frequency steps
rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)
Or viewed on a scale which akes equal pitch steps look the same
Equal frequency steps i i ii WdA
I 11 1 1 1 11~~~ I I l I I Equal pitch steps
4
I ~ ~ l CIl s 0 H Q
CI2 2 ~ H ttJ
~
~~ ~4 sshy~ gtlt ~
-iT It ~ ~V E ~~
() 9shyt T
~
shy ~ 1
lt ~H ttJ
2
~ ~ o ~ 2 ~ H d () ttJ
shy t ~
2 lt l~ I l~0 11I
H ~
~
CIl ttJ
s 0 (II
(1)
luf
lt11111111111 M middotqlu-o
l 1111 I IIIII 111
~ 1lltf lt10
~1If 111 111111 1ut middotcj o
11 rr Ill Ill rll i1 ltj1lJU
-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co
3
~
~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J
S4 ~ ~e -ty~ A 01
-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1
I
(
~ ~~ ~i
_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0
~~ g +-9S 11
~j - )-- ~ i-D ( v--~1 ~ ~ Q
jllb~ f q
r Ii
Jgt
S4
~ QIf ~r f
n ~
~ ~ ~ ~
5
~
0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt
t
3 ro ~ 0 3 Q)
01 ~ C
n lt ~
~1e4-- ~~(__---l bull I i =========t
~ f
W Il4 [Il n1lli1l11111
QRlp
I T9shy rq7
HIGH PASS FILTER
lT111 IIlll II I ptttp
I
II frlt fyq
LOW PASS FILTER
fittTIlllllll) r I 1111 )fnt f~~
BAND PASS FILTER
nil INIIY[I imp
I
I f~~ fr~
BAND REJECT or NOTCH FILTER
(AntP each filter h
~tCOl Iml fof a part of
~~~~ _ 111 1 1111gtft FILTER BANK
7
cent
~ ~
0 cent
~ shy
i-
A
~(
FILTERS
lt-t
~ lt ~~------~
~
I ---1
~
( Cl
lttshy
Jl ( I ~~(-__J
d
t
~ ~
[I1 Ul H
o Z
U H
Z o ~
~ ~ H
U H
Z o
~ z
~ o z o H f ~ t-1 o gt[I1 I
~ H
f
6
---IILl
TIME ) I
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
1
UNIVERSITY OF ILLINOIS UBRARV
AT URBANA - CHAMP~IGr MUSIC
JJL 3~S-
J
WAVEFORH
sound represented in the time domain
When looking at much longer blocks of time waveform would be very compressed in display so a different format is used
SPECTRUM
sound represented in the frequency domain NB Frequency axis evenly spaced with respect to frequency amp NOT to pitch
Spectrum displaying analysis channels
LOUDNESS TRAJECTORY (loudness envelope) of a sound
at a mix
display
ALL REPRESENTATIONS ARE SCHEMATIC
to represent realistic waveforms spectra loudness trajectories but only to illustrate the principles involved in the various
Waveform spectral etc shapes have been chosen to clear as possible in diagrammatic form
JA
KEY
~tlme ~a~mp~~~~~~~~
a d
amp Wf
9
rt If tt time
time ~ IgtP
(Nsec~cent
bTII~
(Nse~
CUgtlParp
Op frq
~~
MIX PLACEMENT represents the times which sounds in begin Does NOT the whole of each sound
NB ~
We do not attempt or mix placements compositional processes described
v
make their transformations as
----
SAHPLING
afgt
1JIv~v~v~vti~O~i~~~a~O~~~~~t~dw~e~~~~ is stored as a sequence of O~p individual values or samples A ~ A derived from the wavefo rm
_ AIt Ii I AI shown
SEQUENCE ~rwri~ as
GElflRATION
It is often useful to generate sequences of timing information for musical composition A HIDI instrument does this by sending
data when a key is pressed or released We may also calculate sequences of times automatically
II1I 1IIII1IJ~gmiddot Regular times
t I I I I I I I I j I ItrM~ Times growing by addition of a time-unit
f I I I I I I I I I I I It~ Times growing exponentially
~1 I I ~ btll Times following Fibonacc i series
I I II IIF II II tmpound
Regular times slightly randomised
II I I II II I IIQ~ I IIIIIII~ III ~ 11111 ~ JIIIIIII~ II tQ Times completely random Times completely random but denser
~
uaveforrn ~ c l)
~ ~ raquo ~ Vl
~ ~
~
Spectrum
Il IIH Il
~ H ~ Z ~
~ ltV
~ H
~ J 0 It
+
II
f-
+
a a a-
l17me domain iepre senttltiort
IF(~uen~ domain representatlonl
I()
6
-CL
~ I
0 lt) ~ ~
FreqyencJ
~
0 I
c E lts
0 () f()
Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~
re~ s ~~~
Ij
~~III ~ V1
~Ili l ~ ~ pound en( 01
S ~
~ o
~I ~
tt
t ~ Q)
~
I
o o N
o
TItlE DOHAIN amp FREQUENCY DOHAIN
Q~ Igt I II - ~ tjl II
~ ~ ~ ~ c ~ pound ~ pound ~
WAVELENGTH FREQUENCY FITCH
Time do_in Frequency Domain ~p amp
I ~ f ti~ I
~e oat
WI1Fl7Jm~r ~i-~-j~ ~~~ h
JJVV) When wavelength becomes 12 as long there are twice as many wavecycles
in the same time Frequency = wavecycles-per-second
So Halve wavelength to double frequency Double wavelength to halve frequencY4
1 l
t~ - l~
20 4 a 60 V~ IDa
A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)
nultiplying all partial frequencies by 2 (or any number)
preserves this relationship
WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE
Pitch is therefore distributed differently to frequency
Equal frequency steps
rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)
Or viewed on a scale which akes equal pitch steps look the same
Equal frequency steps i i ii WdA
I 11 1 1 1 11~~~ I I l I I Equal pitch steps
4
I ~ ~ l CIl s 0 H Q
CI2 2 ~ H ttJ
~
~~ ~4 sshy~ gtlt ~
-iT It ~ ~V E ~~
() 9shyt T
~
shy ~ 1
lt ~H ttJ
2
~ ~ o ~ 2 ~ H d () ttJ
shy t ~
2 lt l~ I l~0 11I
H ~
~
CIl ttJ
s 0 (II
(1)
luf
lt11111111111 M middotqlu-o
l 1111 I IIIII 111
~ 1lltf lt10
~1If 111 111111 1ut middotcj o
11 rr Ill Ill rll i1 ltj1lJU
-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co
3
~
~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J
S4 ~ ~e -ty~ A 01
-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1
I
(
~ ~~ ~i
_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0
~~ g +-9S 11
~j - )-- ~ i-D ( v--~1 ~ ~ Q
jllb~ f q
r Ii
Jgt
S4
~ QIf ~r f
n ~
~ ~ ~ ~
5
~
0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt
t
3 ro ~ 0 3 Q)
01 ~ C
n lt ~
~1e4-- ~~(__---l bull I i =========t
~ f
W Il4 [Il n1lli1l11111
QRlp
I T9shy rq7
HIGH PASS FILTER
lT111 IIlll II I ptttp
I
II frlt fyq
LOW PASS FILTER
fittTIlllllll) r I 1111 )fnt f~~
BAND PASS FILTER
nil INIIY[I imp
I
I f~~ fr~
BAND REJECT or NOTCH FILTER
(AntP each filter h
~tCOl Iml fof a part of
~~~~ _ 111 1 1111gtft FILTER BANK
7
cent
~ ~
0 cent
~ shy
i-
A
~(
FILTERS
lt-t
~ lt ~~------~
~
I ---1
~
( Cl
lttshy
Jl ( I ~~(-__J
d
t
~ ~
[I1 Ul H
o Z
U H
Z o ~
~ ~ H
U H
Z o
~ z
~ o z o H f ~ t-1 o gt[I1 I
~ H
f
6
---IILl
TIME ) I
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
----
SAHPLING
afgt
1JIv~v~v~vti~O~i~~~a~O~~~~~t~dw~e~~~~ is stored as a sequence of O~p individual values or samples A ~ A derived from the wavefo rm
_ AIt Ii I AI shown
SEQUENCE ~rwri~ as
GElflRATION
It is often useful to generate sequences of timing information for musical composition A HIDI instrument does this by sending
data when a key is pressed or released We may also calculate sequences of times automatically
II1I 1IIII1IJ~gmiddot Regular times
t I I I I I I I I j I ItrM~ Times growing by addition of a time-unit
f I I I I I I I I I I I It~ Times growing exponentially
~1 I I ~ btll Times following Fibonacc i series
I I II IIF II II tmpound
Regular times slightly randomised
II I I II II I IIQ~ I IIIIIII~ III ~ 11111 ~ JIIIIIII~ II tQ Times completely random Times completely random but denser
~
uaveforrn ~ c l)
~ ~ raquo ~ Vl
~ ~
~
Spectrum
Il IIH Il
~ H ~ Z ~
~ ltV
~ H
~ J 0 It
+
II
f-
+
a a a-
l17me domain iepre senttltiort
IF(~uen~ domain representatlonl
I()
6
-CL
~ I
0 lt) ~ ~
FreqyencJ
~
0 I
c E lts
0 () f()
Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~
re~ s ~~~
Ij
~~III ~ V1
~Ili l ~ ~ pound en( 01
S ~
~ o
~I ~
tt
t ~ Q)
~
I
o o N
o
TItlE DOHAIN amp FREQUENCY DOHAIN
Q~ Igt I II - ~ tjl II
~ ~ ~ ~ c ~ pound ~ pound ~
WAVELENGTH FREQUENCY FITCH
Time do_in Frequency Domain ~p amp
I ~ f ti~ I
~e oat
WI1Fl7Jm~r ~i-~-j~ ~~~ h
JJVV) When wavelength becomes 12 as long there are twice as many wavecycles
in the same time Frequency = wavecycles-per-second
So Halve wavelength to double frequency Double wavelength to halve frequencY4
1 l
t~ - l~
20 4 a 60 V~ IDa
A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)
nultiplying all partial frequencies by 2 (or any number)
preserves this relationship
WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE
Pitch is therefore distributed differently to frequency
Equal frequency steps
rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)
Or viewed on a scale which akes equal pitch steps look the same
Equal frequency steps i i ii WdA
I 11 1 1 1 11~~~ I I l I I Equal pitch steps
4
I ~ ~ l CIl s 0 H Q
CI2 2 ~ H ttJ
~
~~ ~4 sshy~ gtlt ~
-iT It ~ ~V E ~~
() 9shyt T
~
shy ~ 1
lt ~H ttJ
2
~ ~ o ~ 2 ~ H d () ttJ
shy t ~
2 lt l~ I l~0 11I
H ~
~
CIl ttJ
s 0 (II
(1)
luf
lt11111111111 M middotqlu-o
l 1111 I IIIII 111
~ 1lltf lt10
~1If 111 111111 1ut middotcj o
11 rr Ill Ill rll i1 ltj1lJU
-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co
3
~
~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J
S4 ~ ~e -ty~ A 01
-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1
I
(
~ ~~ ~i
_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0
~~ g +-9S 11
~j - )-- ~ i-D ( v--~1 ~ ~ Q
jllb~ f q
r Ii
Jgt
S4
~ QIf ~r f
n ~
~ ~ ~ ~
5
~
0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt
t
3 ro ~ 0 3 Q)
01 ~ C
n lt ~
~1e4-- ~~(__---l bull I i =========t
~ f
W Il4 [Il n1lli1l11111
QRlp
I T9shy rq7
HIGH PASS FILTER
lT111 IIlll II I ptttp
I
II frlt fyq
LOW PASS FILTER
fittTIlllllll) r I 1111 )fnt f~~
BAND PASS FILTER
nil INIIY[I imp
I
I f~~ fr~
BAND REJECT or NOTCH FILTER
(AntP each filter h
~tCOl Iml fof a part of
~~~~ _ 111 1 1111gtft FILTER BANK
7
cent
~ ~
0 cent
~ shy
i-
A
~(
FILTERS
lt-t
~ lt ~~------~
~
I ---1
~
( Cl
lttshy
Jl ( I ~~(-__J
d
t
~ ~
[I1 Ul H
o Z
U H
Z o ~
~ ~ H
U H
Z o
~ z
~ o z o H f ~ t-1 o gt[I1 I
~ H
f
6
---IILl
TIME ) I
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
uaveforrn ~ c l)
~ ~ raquo ~ Vl
~ ~
~
Spectrum
Il IIH Il
~ H ~ Z ~
~ ltV
~ H
~ J 0 It
+
II
f-
+
a a a-
l17me domain iepre senttltiort
IF(~uen~ domain representatlonl
I()
6
-CL
~ I
0 lt) ~ ~
FreqyencJ
~
0 I
c E lts
0 () f()
Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~
re~ s ~~~
Ij
~~III ~ V1
~Ili l ~ ~ pound en( 01
S ~
~ o
~I ~
tt
t ~ Q)
~
I
o o N
o
TItlE DOHAIN amp FREQUENCY DOHAIN
Q~ Igt I II - ~ tjl II
~ ~ ~ ~ c ~ pound ~ pound ~
WAVELENGTH FREQUENCY FITCH
Time do_in Frequency Domain ~p amp
I ~ f ti~ I
~e oat
WI1Fl7Jm~r ~i-~-j~ ~~~ h
JJVV) When wavelength becomes 12 as long there are twice as many wavecycles
in the same time Frequency = wavecycles-per-second
So Halve wavelength to double frequency Double wavelength to halve frequencY4
1 l
t~ - l~
20 4 a 60 V~ IDa
A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)
nultiplying all partial frequencies by 2 (or any number)
preserves this relationship
WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE
Pitch is therefore distributed differently to frequency
Equal frequency steps
rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)
Or viewed on a scale which akes equal pitch steps look the same
Equal frequency steps i i ii WdA
I 11 1 1 1 11~~~ I I l I I Equal pitch steps
4
I ~ ~ l CIl s 0 H Q
CI2 2 ~ H ttJ
~
~~ ~4 sshy~ gtlt ~
-iT It ~ ~V E ~~
() 9shyt T
~
shy ~ 1
lt ~H ttJ
2
~ ~ o ~ 2 ~ H d () ttJ
shy t ~
2 lt l~ I l~0 11I
H ~
~
CIl ttJ
s 0 (II
(1)
luf
lt11111111111 M middotqlu-o
l 1111 I IIIII 111
~ 1lltf lt10
~1If 111 111111 1ut middotcj o
11 rr Ill Ill rll i1 ltj1lJU
-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co
3
~
~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J
S4 ~ ~e -ty~ A 01
-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1
I
(
~ ~~ ~i
_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0
~~ g +-9S 11
~j - )-- ~ i-D ( v--~1 ~ ~ Q
jllb~ f q
r Ii
Jgt
S4
~ QIf ~r f
n ~
~ ~ ~ ~
5
~
0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt
t
3 ro ~ 0 3 Q)
01 ~ C
n lt ~
~1e4-- ~~(__---l bull I i =========t
~ f
W Il4 [Il n1lli1l11111
QRlp
I T9shy rq7
HIGH PASS FILTER
lT111 IIlll II I ptttp
I
II frlt fyq
LOW PASS FILTER
fittTIlllllll) r I 1111 )fnt f~~
BAND PASS FILTER
nil INIIY[I imp
I
I f~~ fr~
BAND REJECT or NOTCH FILTER
(AntP each filter h
~tCOl Iml fof a part of
~~~~ _ 111 1 1111gtft FILTER BANK
7
cent
~ ~
0 cent
~ shy
i-
A
~(
FILTERS
lt-t
~ lt ~~------~
~
I ---1
~
( Cl
lttshy
Jl ( I ~~(-__J
d
t
~ ~
[I1 Ul H
o Z
U H
Z o ~
~ ~ H
U H
Z o
~ z
~ o z o H f ~ t-1 o gt[I1 I
~ H
f
6
---IILl
TIME ) I
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
WAVELENGTH FREQUENCY FITCH
Time do_in Frequency Domain ~p amp
I ~ f ti~ I
~e oat
WI1Fl7Jm~r ~i-~-j~ ~~~ h
JJVV) When wavelength becomes 12 as long there are twice as many wavecycles
in the same time Frequency = wavecycles-per-second
So Halve wavelength to double frequency Double wavelength to halve frequencY4
1 l
t~ - l~
20 4 a 60 V~ IDa
A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)
nultiplying all partial frequencies by 2 (or any number)
preserves this relationship
WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE
Pitch is therefore distributed differently to frequency
Equal frequency steps
rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)
Or viewed on a scale which akes equal pitch steps look the same
Equal frequency steps i i ii WdA
I 11 1 1 1 11~~~ I I l I I Equal pitch steps
4
I ~ ~ l CIl s 0 H Q
CI2 2 ~ H ttJ
~
~~ ~4 sshy~ gtlt ~
-iT It ~ ~V E ~~
() 9shyt T
~
shy ~ 1
lt ~H ttJ
2
~ ~ o ~ 2 ~ H d () ttJ
shy t ~
2 lt l~ I l~0 11I
H ~
~
CIl ttJ
s 0 (II
(1)
luf
lt11111111111 M middotqlu-o
l 1111 I IIIII 111
~ 1lltf lt10
~1If 111 111111 1ut middotcj o
11 rr Ill Ill rll i1 ltj1lJU
-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co
3
~
~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J
S4 ~ ~e -ty~ A 01
-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1
I
(
~ ~~ ~i
_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0
~~ g +-9S 11
~j - )-- ~ i-D ( v--~1 ~ ~ Q
jllb~ f q
r Ii
Jgt
S4
~ QIf ~r f
n ~
~ ~ ~ ~
5
~
0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt
t
3 ro ~ 0 3 Q)
01 ~ C
n lt ~
~1e4-- ~~(__---l bull I i =========t
~ f
W Il4 [Il n1lli1l11111
QRlp
I T9shy rq7
HIGH PASS FILTER
lT111 IIlll II I ptttp
I
II frlt fyq
LOW PASS FILTER
fittTIlllllll) r I 1111 )fnt f~~
BAND PASS FILTER
nil INIIY[I imp
I
I f~~ fr~
BAND REJECT or NOTCH FILTER
(AntP each filter h
~tCOl Iml fof a part of
~~~~ _ 111 1 1111gtft FILTER BANK
7
cent
~ ~
0 cent
~ shy
i-
A
~(
FILTERS
lt-t
~ lt ~~------~
~
I ---1
~
( Cl
lttshy
Jl ( I ~~(-__J
d
t
~ ~
[I1 Ul H
o Z
U H
Z o ~
~ ~ H
U H
Z o
~ z
~ o z o H f ~ t-1 o gt[I1 I
~ H
f
6
---IILl
TIME ) I
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
n lt ~
~1e4-- ~~(__---l bull I i =========t
~ f
W Il4 [Il n1lli1l11111
QRlp
I T9shy rq7
HIGH PASS FILTER
lT111 IIlll II I ptttp
I
II frlt fyq
LOW PASS FILTER
fittTIlllllll) r I 1111 )fnt f~~
BAND PASS FILTER
nil INIIY[I imp
I
I f~~ fr~
BAND REJECT or NOTCH FILTER
(AntP each filter h
~tCOl Iml fof a part of
~~~~ _ 111 1 1111gtft FILTER BANK
7
cent
~ ~
0 cent
~ shy
i-
A
~(
FILTERS
lt-t
~ lt ~~------~
~
I ---1
~
( Cl
lttshy
Jl ( I ~~(-__J
d
t
~ ~
[I1 Ul H
o Z
U H
Z o ~
~ ~ H
U H
Z o
~ z
~ o z o H f ~ t-1 o gt[I1 I
~ H
f
6
---IILl
TIME ) I
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
FILTERS cont~nued
Q
The Q of a filter determines steepness of cut-off (see diagrams)
Qfgt~ T
~ fret- Band-pass f11 ter fl-ct
centred at some frequency
with low Q
0
II
I I
I I
~ 111 I I fat- ft~Band-pass filter
centred at same frequency with high Q
Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)
o~ --- 1 L
[h --
f~t f~
B
ALL PASS FILTER PHASING
An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see
diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)
-~ 2OmP
lpha~e~hft 1 af I I pha~e ~hft
)1 II ~~ieII ~ I i) 1
The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time
If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in
popular music and is known as PHASING
PHASE
Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as
the phase
~cP)~ 2
When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the
end of the waveform
Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This
fact enables the Phase Vocoder to track the frequenCies of partials in a sound
9
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
FORnANTS
Formants are peaks in the spectral contour
tlfgt Ir-- j il r t-
r I(~r 1 _c~ r 1 I
Pitch may change
~(tlt r ~
without changing the formants e g sing a downward sliding pitch on a fixed vowel a
ITrCI f [h-tJ
[[11[11[ [fhl-)
lmttm am [rnlr)
Formants may change without changing the pitch
e g sing a -gt u on a fixed pitch
ITh-rlT[-
-
~
~
-
~ (i
1
3QnjJl4WV
a VI
s ~
~luIfbi
~ Vl tIl
T lt 0 ()
00 t1 tIl ~
I 1 J
~ Vi)~Jn1d--d
10 11
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
LINEAR PREDICTIVE CODING (LPC)pressure
~ r
This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations
The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )
64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in
the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true
We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion
The coeffi cients turn out to be the ~ time numbers we need to define a filter - and
all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the
previous 8 values Can we predict the
using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-
~
0gt aI r
~ r ~ lt 11 rf
~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr
T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~
co aI aI lt 1 1 rf 3 C) ~GlHco 0 co
co (t or aI ~ Potltl 11 l()rf ~ ~
rfrf0 ~ co n rf
Po ~
1 1 rf~ trOal
J
o 1 tgtl aI 1 rftl IaI
lt IIgt I Porf1II
I C) l C)
n rf v
1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III
I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0
anrf II Po aI Po 1 Oal n aI 1
rfI1 rfItrn Trf
ao tl 1 co aI
1 tr aI aI rf 3 () aI ao l 1 PI PI
ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf
tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co
ntr aI rf trn
o Porf n T-tT
f- 1 Po () C) ~ ~
U)
I I r
II shy
I
I I~ I 1fJ
I)~ I~
0
0
10
J f
-h - _
~ ~
c
f
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
J)isfribuhor1 CJ FDrmanfs relafive to PdclL
FOrmfnt 511apc
Jat low feuro~ueVcies1 I 1 I I I I
STEPS
Formants equally Coding) ranges
have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all
LPC AND FORHANTS
14
Distribuhon
-too little data formarrt shape n~t resoveJ
~111 at low f~e~ueYdes)
~
I EQUAL PITCH STEPS I
-1 f -( rf l i_ ~-I
EQUAL FREIlUENC STEPS
Y
I
1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I
[at hiSh frequendes
1 1 1_ -f -1--1 1 I I 1 I
I I I I 1
I -shy -t- -+ I 1 I
1
Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges
PHASE VOCODER AND FORHANTS
15
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
CHANGING THE SPECTRUn
USUALLY CHANGES SPECTRAL CONTOUR
~p ()~~
Transpose (uplAlants) b~ syenuftinJ par-has
()fgt amp
1~ 7~
Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials
pitcJ1 transpositol1 amp- spectral traniformation do not normalllj
preserve the spedYat contour
16
FORnANT PRESERVING SPECTRAL nANIPULATION
for evelj lJIiVldow in the- fre~ue1cy domain ap
speuroCtr-a1 COIItourpoundgtCtroct
(ft nEnCE formants) ~ MP~
P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -
j
lshy___transform sp~drutn
0 +- il shy l 1- - --T r shy - - - - -
II I I III II ~frCJ
~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur
I I )ofpound
FORHANT PRESERVING PITCH SHIFTING
works s1m11ar1y 17
I
II Ibull
I
j I
bull II I
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
ltD
SPECTRAL SHIFTING
~tltr
ori)inal Pltcshy
amp
I I
-r
origiTlal Sinsle pitch
a
11
ap no Shift
Shifted
~fY~LJL)~+I____~~_~ ~
2 pitches
I - - i
~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt
~ - f~ - + ~ + ~ t - ~~F sect~~ ~
~hjfted - bull Sound _-shy
-~
J
I I + - shy r-shy ---- Y- ~ - - ~ -
L_ TIME gt p
~ I 1
(1) (2 ) ( 3) (4)
amp
I-rI 1+ J-I
frashy I I
Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift
I I ~frf-
18
SPECTRAL STRETCHING
1 I I
I
~r
~I-
Q)
bull stretch gt1
bull stretch lt 1
rl llhrn ) 11 51 ~hJ
I stretdt more attop than at bottoM
stretch more at bottom tha at -tor
~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -
~ 1 I ~ I
I
MF
rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching
19
11 I I I
Ij
jbull 1
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
SPECTRAL FOCUSING
~
15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i
~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111
L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C
20
i- shy --shy
~I-shy~
o J
---------t-shyt shy
o
PARTIAL TRACKING
- shy shy -~ I- shy -ishy~ t shy -- ~ --- -
Ishy - - Ishy-~ --- shy shy
~
- l- ~-
fshy
TIME
t--~ E~ E ) I ) I ~ --~iI
~
each partial regardless of which analysis channel
1 II I 4 11 [ I 2 3
1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy
l-I shy
~
-~
---
t
- A
gt g
it falls in
Fine-grained analysis extracts partials
1 1 - (
f I 11
I ~ ( I - I _ - - ---- -~~- - -
21
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
SPECTRAL SHAKING SPECTRAL FREEZING
orisinal sound
y 1 - -
--------
Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt
I o gt a I~~
1 centI
frt f~
~lIu I _ ---r
lpoundreeze onamplifude) ure~~f~~e~~~ shy
+shy
--I 1
11 I 1
r 11 [l1~f 11 ~ ~ f~
III II ID U ) v r
ft fro
23 22
I I qll
I
11shy
-1 +shy
l
f --t
lh
10--((111 ~ p
(JP
o gt t--t-- t-
f
I
~ r-- II
I
~ I ~ shy
I--
rshy Vf--
t
~ I-shy
1I p
1I--- r-J
--J
yr 1 1--1
7
i I v V ~ lshy
o gt
~
~ j--
r- V~ ~ t-- t--shy
J L I
I-shy v 1II ~ I ~ ~
rmtll ~
-+
+shy
+shyI I
II J~ ~ t-- I 7
I
1 I ~hy
+shy1 I I
~m ~~ Ii
-+14+ffn r
1
7 1 I
I ~ ~t
17
1 ~
[1I ~
r~ 7
l ~ ~
f- i r-- I
~ II
LJ I 1
I r- li l7 J~ h i
~ 1~ I
[ P 1 -r
If I II 1h i1V - frshy
+shy
fshy4
~
roc ~~rtr
1shy~I ~ r IV
-r -- --0
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
SPECTRAL ARPEGGIATION
Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed
Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt
0 9 8
ii4 3 2 I 0
10 9 Y 7
J 3 2shyI
0
As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m
bull i bull I i7
d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ
Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-
10 9 fl
5
7
2 ~ ~~ tI 0 0 tl tI d 24
-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J
SPECTRAL TRACING
LIJJIIrII -+ Spectral tracing for noise-reduction
OYlspna ( sound spect(aIiJi-tr(fced ~u(t
~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f
~I 11 I I 111-111 11111lid 11111111)11 hl lili
new ite~noqgtresentl T In prevIous Window Ill
bull frq frq
Spectral tracing creating revealed melodies as new partials enter the most-prominent set
In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
origlnal sound
I I I ~ I I I I
SPECTRAL BLURRING
selected Windows irrterpoded ffsult
cent I I - I I
Phase Vocoder windows are selected interpolated on channel-by-channel basis
Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)
cent
Q~r 1I
and + a 11-
I
I
TI ftmiddot I I I~
in II - I~
I 1I 1I r Ilt ID I I
~ i Ii~~ I-
cent - I I t I~illT]
T
I I
TRACE AND BLUR
selected ltraced windows irtferpoaled result
ltf o~r lis7 II r I 1 cent
L ~ I I Igt I I i Ir shy
I I I I ~
r I f I
I
I i I I I Y
~
I
in i I 7
I I I
i~h I ) ~~j -I cent ~l
(TJI J I -+I I 17
I
I trl(Tracing retains only 4 most prominent data items in channels in the selected windows
Interpolation takes place between traced windows
27
~
l Il
SPECTRAL
orlcnal sound
-
-
J
- I
-
of
I
q
cent q
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
SPECTRAL UNDULATION
harmonic - inharmonicshy ormarrt wobbleogt tIp
Uht[ ( r[~J I -11111111111 iA fY~
tIlleTr-dl]) t11I1 II II II I)
111 II II II II I
1T lhJUhr-I-shyI11II II I I I bull ~
t11I1I I I III I bull IT[lrY(llh ~
IT1 tl-rTlTr [-~11111 II II II I)
11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh
o p
111111111111 l1lhdlTfhshyf-if- A
28
SPECTRAL SPLITTING
Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)
2 3
bulll(h 1
CD
1
~ cent
~I f~
II J I shy
I
I
I J )
1
I I 1 ) )
I
11J1
29
)
)
~
I
- -t
I_gt~ YI~ATO A IVl8RATO 8
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING
Hethod 1
Q~
- ~ of1 r 011 ~
~ Ii1 ~ ~cp ~
~ II r shy
fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)
TItlE-WARPING
bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime
bull gttime-ste1dtinr
bullbull=_lCgt tirne-shrnkinJ
- gt tirne-vC~inj tim( - sfretc~
Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices
generated from input windows Using method 2 by varying interpolation process which
generates new windows
30
Hethod 2
Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j
ClJgt
IshyZ m ~ j
I
~ -IIl II 1 IQ
I ~
111 a p
1 - shy
rdSe ap algtJgt
~ fYQshy
oriainol wndowshy -samphng rare
~II 111
T ~
~ m
c) flf
Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows
ltm ~
~
l ~
~ ~L__~__~______~I~gt
I m
--shy
if-shy31
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
SPECTRAL INTERPOLATION
TIME
~ =j(l I
0
~
I~~ ~~ ~ -
n r -
~ f
~ d
r
---
~ 8L-- __ _ s
L ~
IIgt 0
i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy
I VIL Lshy
~
11 o 5
VI
~ g ~
til o C s
oL
8 s ~
rshy ~
~ 0 c JI ~
~I I~~~-
I---- shy
shy8( -
+-
TYPES OF SPECTRAL INTERPOLATION
----
Different types of interpolation
a
~ i move eve I1llj
II raquo- 1OWCI frOl71 1st SoonO
l towand~ 211rJ sound
+gt i
imhal1 move slowllj awayfrorn 1st sound
the jnc~51~(1 qUiCiJy toLuaYCilt
2V1d SDuVll(
) Ihitally
f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj
tDavd~ 2nd SOu d
iYliballj move V(rj ra~dll away
1 lt0
Independent interpolation
j-r
~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj
-0gtNmlS 211 d 5oun d
l ~ 1(1) 1) 0 )
o lt
S ll
III Ib n o
tl VI o c raquo-
VI l)
o l Q
lt0 o l ~
C) (1)
~
J ITl
j Z 11
-4
I f1
~ -I 11
of amplitude and frequency
~ I
~ I
I
-
W~ ~ I
~ I1
I
Clmpir-lJd e ~ frctutIKJ interpolafed at- II
cj fferent times
omp g free nte~laltd I or di ert1t
tim g in 11) drfampotf- I
WampIjS ~
op Qlnpg frltL Inter-polsted I ctt same tim~ b4t
in different
9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~
j) tutend5 after it ~ ~
~ I ~ ~ ~ ~ I ~
~
32 33
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
VOCODING
50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp
1
Frtt1(M~Jse dru~JI
~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op
~ J
soun~~
I
frt
SPECTRAL HASKING
1I
-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull
~I rr-J
SPECTRAL INTERLEAVING
~ +shyI u4 ~4 ~~=J
shy
~
gtlt (j~ ltII I
(j (
C QJ
d A
x
QJ
QJ -E ~ ~~ a~ ~ E
Jl f ltl
nJ ltII I
( ~-) I ( E c -frat ~
r
I ~(
Nt
QJ
E nJ Ilt I IV E
nJ C tJ 0
c H -fr
4
is~ lt
Clt4 t
c~
)~ _ ( ( ~
~~ ---~- - ~
~~ I
I
~ t
It
~
Il II
D-_ pound-t
J
rAo~
)
I I
I-shy1J=5l
0 ~ ~
0 E cs
35 34
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
TAPE-SPEED VARIATION a
For transposition and time-stretchingtlIhr
tAAAA middot i I V [ ytlm~
rIQ~P Lengthen waves less cycles per second lower frequency
A A A A )V V V V V~I~e but also increases duration
Original spectrum and spectral contour
crman-C atlfgt I I I fltJrmant I
[- --r- -T--]---[-- r ~ -[- -t Tfrq
Pitch moves down partials move down formants move down
~lTlr[[-mh TAPE ACCELERATION I
[ I -L_- _____ J=gt edbme end~ed~~(I Mwf
Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start
36
TAPE-SPEED VARIATION DIGITALLY
I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)
I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ
l~lll llL ~~ i - Ielt
I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI
37
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
HARl10NISER
I I
~
III r
amp--A
I
L--
Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted
(see below) they will rejoin into the exact duration of the original sound
~cent~ ~cent~~ Change segment pitch by tape-speed variation
which also modifies segment lengths
Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration
39
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
CUTTING
fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i
~ a brief fade-outV 7 ~ to zero loudness
ZERO CUTTING
SPLICING
Unconventionally we may cut a sound at a zero-crossing
A normal splice is a very brief cross-fade
Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ
2~ 3 ___--J
L--J 5J
6L____ ~
11 I etc
ant
I
I I
RANDOM CUTTING
time
Succe5Slve rondom cuts
SHREDDING
e
I I I I
-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order
[lT2r -3-shyr-4-[51
cut ere rearra~e etc
41
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
LOOPING AND ITERATION
Ilp
~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time
L-J IOOp~d elemet
r I f ( I te3lJla tilrlll1~
lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If
I M II etc
I I I I I Slishttl tandomiHd tm3
~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf
~ ~ etc sl(nt~ ~c4ttercd pirchts
PROGRESSIVE LOOPING
~
11 ~etc v- step
This is in fact a simple
CUMp
SOUND REVERSING
Compare this with granular reversal
ZIGZAGING
Zigzag on a fixed element
reOld OIj~1 sound at or~i~aspped) ) General zigzaging
-=---~ CIS ShdwYl gt~
43
case of Brassage
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
BRASSAGE
segments must be in grain time-frame
1 to w o
OPTIONS
lt
~
B BRASSAGE (continued)
PITCH-SHIFTING OPTIONS bull
nJ~ pi(cJ
Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping
Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~
Random pitch-shifts of segments
pitcrshy pi1th bullc t~ F~ -~ bull
Cyclical pitch-shifts of segments
Give small random delays to segments
and mix with original source
SPATIALISATION
ItLeft
I I I I I I I I
RI~ht
eg random spatialisation within a range
---------shy
[
eft
- ~~-~~~~-- RI5gth1
45
A For time-warping
1IIIIIfbullbull time~r-ink
~UI ~- cr ~ ~_tO
~ lt ~ Ul to
r 1- 0 (T 3 Ul to Ul
0 r 10 000 1 a IA to to
IA
a~ ~ ~ to
1 ~ 7 ~ IA to 3 to
0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to
1 I ~a e ltD~
I I
t i
w o
VARIOUS
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
each mix element assigned a start time and a loudness
as a graphic icon on a screen
nIXING
resultant mix
Lefl(-l)
~ I t i~ It I~161 j ~ I ~ Gl 61~ 10
pa -b 2
Ri~ht (+1) c 4shy
rn~ source sound
C( 01degc
ltI~~t~_~~ each mix eleent assigned
a start tiae and a loudness on a line in a aixing score
spatialisation of mix
IN-BET~ING
Ngt~~ ~ ~ k~~
nIX SHUFFLING
Original mix Original mix
b
reve so-d 5li shy ~ d c A
h 9 J a
t i ampi ~ random scatter mix times
I I randomise sound Sfltluence
TIME WARPING lodflf5~ Original mix of ~ovnds b
add fixed CIJourn to mi)( brne S
Original mix
a_ ~ c ct ef h
Ri5nt Ar 3n lYli x I~ft t o ttght
a ~r
C E3 q ~~ (We
f )
b
etc REshy SPATIALISING
rondom-spatialise mix ~gt
a ~
ciOgt Jshy~~
~3C b
~~ e ~-- h lj
h n~
47
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
OCTAVE
Octave shift without change in duration
p(e g by spectral shifting)
for every window spectrum of original sound
spectrum of original sound
shifted octave down
etc
superimposition of the two
etc
STACKING
Octave shift with duration change
(eg by tape-speed variation)
~QP ~ r ~
till
pitch
octave up (shorter)
octave down (longer)
with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation
L_________toime
~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory
t ime
If sounds do not have a sudden onset attack-synchronisation might be
are appropriate
ONSET SYNCHRONISATION
tape-speed varied versions d~fferent
1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e
t_ ~~ bull I
~~~ 1
49
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
I
50
TRANSPOSITION
pOOnhle 1~2
I~- I~V1 ~
I I
lwAVESET REVERSAL
r MP~W V~~W)o
II WAVESET SHAKING
VESET INVERSION
2 6
WAVESET OMISSION
VI - Y VI1-
WAVECyenCLES AND WAVESETS
WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing
waveCllces tJ Ip-- I I I f ~
rpssuu ~ I
f I I 161 11 1 r
6me - ~I ~I _I I I i
wavesels J 1 I fit~~middot
Wlvesets I V V v V tim~ 1 I I I I
___JI 1___~I
tuavecJdes( I I i V ~~I I I
I ~
I V I -I I I v I I V I I VI
II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J
WAVESET SHUFFLING
blgtle IiI
I I I Iwaveseb~ 1 ~W--J
I I t I I I I I I I time 1111 II t 11 111 I
Wave~t~ --JUJI~U~IIIVL--lJULJlW
I V
I
51
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
I 1 I I I A I I
WAVESET HARHONIC DISTORTION
gtr(50S ure
~ I
1 I l 1M 1
~ ~ etc 1
I Orljrntl WIesees I
~II bull
poundPa etc I I I I12IId haoiltt
I 1I I
ffv1(1 ~ etc ~ I
13rd hucinc~u J etc -lo
each weitrteL
eneh Summed
I
I etc
WAVESET AVERAGING
prusure I
euro
Qvea~e over 3 wale~ds I
WAVESET ENVELOPING
lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)
~ envelope 0 if sin~e tya yesets
rvvWANv~~ ~ envelope over 2 wavesets_
nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6
no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~
~gg4yenirj~~~wlillIiii127271222
~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~
Am~~~~~M~tltO ~~ UPtVd t-tt~ set~
--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s
~ 53
WAVESET DISTORTION
p ~ fV) I~QC~ WtM~
dsfogtflo~ by folole fa(jor
~ V amp OV -VZV0
C~bseJ power flcor
~v v q~AroA 4
WAVESET SUBSTITUTION
e~~(-dtft4tJ tv~~~~ ~
5Ub~btt+lQ ~ C-P~tmiddot~~J hetet
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
wavesel tim I I
WAVESET TRANSFER bull
bull
~ n2
00 Viv6V gt
~ tak =t f~
INTERLEAVING method 1
I
55
I
ETC
bull
TIHE-STRETCHING____________
~IV _ti~
tim x 2 repar eo Wdt lt2
(vj lln f 1 AW~lJuTv V V gt
timestrdcJ x 6 ~peat each wovesec x 6
bull I
TInE-SHRINKING
b1 c C1
~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4
I
I I
wal~ets
timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3
I
source 1 c d
inta-~ve pairS of bJove~e~~ a bT eDT
I
method 2
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
the grains
GRANULAR REORDERING
d e f bullbullbull t t~ grains (pitch sequence is disturbed)
~ Time-shrink via grain deletion
GRANULAR REVERSAL
Source SDuhci
in the sound
Reverses the grains themselves
------~~----------~--
pir-~
r= Reorder
ptdo
Alter pitch of grains (retain original order)
Alter pitch and scatter timing of grains
TIHE-STRETCHING
Time-shrink by reducing Q bed e
57
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
ENVELOPE FOLLOWING ENVELOPING
Coarse-grained envelope following Follows overall contour ~ I
PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot
~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness
of speakers voice
separates words or syllables
ENVELOPE SUBSTITUTION
eg in speak~s voice
to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime
e g separates individual grains of a ro ~ led r ~
in speakers voice
Itle
58
Fine-grained envelope following
Follows finest detail of contour
59
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION
~~-raquo
~ fi~ ---------
~ ----=-thrtSllold
~--0-~-eL )
~
l~~ ~ti~ I~-~
~N )
D~OUm
tlme )
()IP Transform enve lope e g by inversion
~)m~ SUBSTITUTE new envelope OR
on original sound
~ONI~ bt e
~o ~~e~_4J-~--4--~L~~tll~e
envelope for original sound
~iPte
-_t 60 61
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
TRIGGERING
Envelope generating triggers I a~pto be triggered a~ 1 Sound
I A bull A Ii tnreshold
e
6me I
Resulting sound
DUCKING
(backing) sound to be ducked
6 li T 0I-----fhreshold I I( K envelope of (lead) sound
to cause ducking
ducked (backing) sound
62
63
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
DELAY
Sound delayed and repeated with successive repetitions
quieter and quieter
COHB FILTERING
Very short delay time
1234618-
illfillEc --r---
number of repeats per second is very high
(eg 16+ per second) we hear a pitch
REVERBERATION
In an enclosed space sound reaching the ear directly
is mixed with many differently delayed
versions of the sound The delays are caused by the different distances the sound must travel
on its various paths to the ear The delayed versions
are (usually) quieter than the direct sound
amp have been changed in quality through the filtering effects
of reflection from different kinds of surfaces
C1gt
8~ 01
3ir (raquo
f 33 0 o s ~ ~
1lt VI lt) ~
~~ ~
lt
(t) ~ ()
(i) (T
~ ~
~D
~ cal
~i0 e 3 II
OJ
amp
~ -0shy
Q)
tLls ~ So
1
~
~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H
o Z
65
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
VIBRATO
~ir-cl pirch
time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular
cyclical fluctuations of pitch
f~Vibrato created True vibrato
merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves
TREHOLO
I _ lt lt lt )~me
Original loudness trajectory - -----
~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo
For
_ 66 ~~shy67
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
TEXTURE
pitc ( l) r-n)a J)b ~middotmiddotJe
Jrr middotfJJ additionallymIn rp IJ1~j ~ ~
rp lJJ) pJrp ~ iJlm III)
specify variation of motif properties
1]J Tip III ~ 1]1 lJJ) rn
motif type number of motifs
time
lI Cgtab ~a bc Pklif r~Ptpoundumber
etc
WEDGEING
pif-cJ
I deJ1SJrY euroVOhbullbulltoY -~~lt
-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )
we~e dtuation
pitLlt1rojectorj
WEdfje WIdehshy
time
OF HOTXFS
( b Ubi t b )OR 0 u=a ~ s(rrnk =c
69
i
jl ~I
I f ~
TEXTURE
specify variation of
pitch bull event density
bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness
bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position
bull til1l
I111fQ1UttI ~ Idens ity I loudness
low pitdt range
time
pitch
hlgtt I low
~ quiet event otAd even
l(Janthed Cgtrandoltl
181f110111~random ied -V
TEXTURE OF GROUPS
additionallypitci1 -V specify variation of ~ group properties
Imiddot ~ loudness trajectory ~ hi amp 10 limits of size
( ~
hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange
~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy
A til-heuroshy
2-4q3-6 IqyoUP SIze
etc
68
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
11 il _ 11 I __llLJ I 11LampJ11i11
PITCH-TRACKING BY AUTO-CORRELATION
Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero
tI ft)--J 11 1 ~ EL i ~ bullbull bull 1
Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value
(1) the window value] value]
are zero X zero zero must be a -ve value amp hence less than case 1
At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window
(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero
(2) Add all products Only 4 +ve products in window (see above) Less than case 1
(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero
(2) Add all products Sum 15 zero and therefore less than case 1
Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem
7
4 _ N=5
l T bull -----f--I
~
At all other sample steps
PITCH-TRACKING BY PARTIAL ANALYSIS
i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p
For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform
to an equal-pitch grid spaced in quarter-tones
a -lll iii
bl bull II~II II
I rI 1 W ~~
I Iii
IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window
we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance
To find a pitch effectively we must proceed as follows
For each quarter-tone channel on the pitch-grid
(1) Find the value in that channel (2) Find values in those channels which fallon the
harmonic-series template above that channel (abc) (3) Form a weighted sum of these values
nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)
Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum
The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in
the (suspected) spectrum
Weightings for specific instruments (or other sound sources) may be well known
Difficulties arise with sources like the voice which can change their spectral contour from window to window 71
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So
fliid bull (
I
seqrnent sttFshy
~-~
Ditto output times scattered
~GhdrtrE ampOn~
Pitch scatbred time
-shy b~
I OUtput denll1ty GREATER THlH 1
Segment size gt step
eg output times scattered
--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only
t Spatial location scattered within
H Left__e~~ng range
-shy -_-- RIht - - shy - -
73
j
+
II I I I
I
-
-
Inrvduc ed
SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN
Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage
Qmp
f~
pohltlh
hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull
SOUND PLUCKING
or~lnal ~OIAI1d
MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot
( w(A1f~$etI~synstnctl 0 n So