towards the automatic generation of sound synthesis techniques: preparatory steps

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Towards the Automatic Generation of Sound

Synthesis Techniques:Preparatory Steps

Ricardo A. GarcíaMassachusetts Institute of Technology

Machine Listening Group -- Media Lab

2000

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Goal: Having a TARGET sound and a set of Control Signals, find a “black box” capable of synthesizing the desired sound

C ontro lS igna ls

B lack B oxTargetS ound

?

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Conference plan:

• Sound Synthesis Techniques

• SST space

• Genetic Programming– Individuals, genetic operations, testing, fitness

function

• Conclusions

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Sound Synthesis Techniques (SST)

• Digital computer: AlgorithmSound Samples

• Algorithm: Topology composed by functional blocks/parameters /connections A dditive Synthesis

+

FM S ynthesis

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Basic Construction Blocks

F T L O

F T L O

F T L O

F T L O

f2

f3

f4

S P L ITA D D

A D D

A D D

S P L IT

S P L IT

S P L IT

A dditive S ynthesis

F T L O F T L Of1

k

FM S ynthesis

• Few types• Highly functional

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

SST Space

• Conformed by all the possible combinations of blocks/parameters/connections (topologies)

• Multidimensional (many dimensions!)

• Assumption: There is at least one topology capable of synthesizing the TARGET sound

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Searching the SST space

• Use of evolutionary techniques: Genetic Programming– Well suited for searching in spaces with many

dimensions– No need to specify the size (topology) of the

solution– Multiple search paths/solutions

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Genetic ProgrammingFirst Popu la tion

(random )

Test Indiv idua ls

F itness Function

Solution Found?

C reate newpopulation using

G enetic O perations

Y E S

N O

FIN ISH

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Key elements in Genetic Programming

• Individual: Sound synthesis algorithm

• Genetic Operations: Performed on the Individuals

• Test of the Individual: Produce sound

• Fitness Function: Distance measure

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Individual• “Description” of a sound synthesis

algorithm (topology)

N on-cyclic topology descrip tion

F T L O

F T L O

F T L O

F T L O

f2

f1

f3

f4

S P L ITA D D

A D D

A D D

S P L IT

S P L IT

S P L IT

C yclic topology

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Genetic Operations

• Copy

• Mutation

• Crossover

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Testing the individual

•Translate (compile) description into a working topology

•Run the topology: use the Control Signals and produce an OUTPUT sound

Should be capable of running ANY topology that could be suggested by the program

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Fitness Function

• Distance measure between the OUTPUT and TARGET sounds– Objective: Analytical distance– Subjective: Perceptual distance

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

In each generation...Ind iv idual #5

T ra n s la tio n

5 4 3 2

To p o lo g y d e sc r ip tio n

To p o lo g y im p le m e n ta tio n

C o n tro l S ig n a ls

O u tp u t S o u n d

Ta rg e t S o u n d

F i tn e s s S c o re =

Ind iv idual #9

T ra n s la tio n

3 2 11 9

To p o lo g y d e sc r ip tio n

To p o lo g y im p le m e n ta tio n

C o n tro l S ig n a ls

O u tp u t S o u n d

Ta rg e t S o u n d

F i tn e s s S c o re =

Ind iv idual #53

T ra n s la tio n

11 9 0 0 1

To p o lo g y d e sc r ip tio n

To p o lo g y im p le m e n ta tio n

C o n tro l S ig n a ls

O u tp u t S o u n d

Ta rg e t S o u n d

F i tn e s s S c o re =

Ind iv idual #71

T ra n s la tio n

7 4 4 6

To p o lo g y d e sc r ip tio n

To p o lo g y im p le m e n ta tio n

C o n tro l S ig n a ls

O u tp u t S o u n d

Ta rg e t S o u n d

F i tn e s s S c o re =

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Conclusions

• Define a standardized SST space

• Use of Genetic Programming for searching– Work with topologies descriptions– Very robust scheme to run the topologies– Perceptual vs Analytical fitness function

Automatic Generation of Sound Synthesis Techniques

MIT Media Lab -- Ricardo A. García

Contact Information

• Ricardo A. García

• email: rago@media.mit.edu

• phone: (617) 253 0112