proceedings of smc sound and music computing …smcnetwork.org/system/files/visions of sound - the...

Visions of sound: The Centro di Sonologia Computazionale, from ComputerMusic to Sound and Music Computing

Sergio CanazzaCSC-DEI, Univ. Padova

[email protected]

Giovanni De PoliCSC-DEI, Univ. Padova

[email protected]

Alvise VidolinCSC-DEI, Univ. Padova

[email protected]

ABSTRACT

Centro di Sonologia Computazionale (CSC) scientific re-search was the premise for subsequent activities of musi-cal informatics, and is still one of the main activities of theCentre. Today CSC activities rely on a composite groupof people, which include the Center board of directors andpersonnel, guest researchers and musicians, and particu-larly on master students attending the course “Sound andMusic Computing” at Dept. of Information Engineering(DEI), which is historically tightly linked to the CSC. Thedissemination of scientific results as well as the relation-ship between art and science is hard and surely not trivial.With this aim, this paper describes an exhibition that illus-trated the history of CSC, from the scientific, technolog-ical and artistic points of view. This exhibition is one ofthe first examples of “a museum” of Computer Music andSMC researches.

1. INTRODUCTION

Since the invention of musical instruments, art and tech-nology have stimulated and benefit one another. The crafts-manship required to make a violin is a classic example,but the invention of music-writing techniques was also anachievement, often based on complex mathematics, whichenabled musicians in the late Middle Ages to create intri-cate combinations of sounds.

Over the centuries, Padova institutes, musicians and schol-ars have helped to revolutionize the science and art of sound.During the late 20th century, the Centro di Sonologia Com-putazionale 1 (CSC, Center of Computational Sonology)of Padova University (Italy) and the Electronic Music classat the Conservatory “Cesare Pollini” in Padova gave birthto a unique scientific, technological, and artistic experi-ence, which stemmed from individual collaborations andmultidisciplinary exchanges.

Between the 1970s and 1990s, CSC emerged as one ofthe world leading centres for research into “Computer mu-sic”. The design and development of software programs

1 CSC was founded by Giovanni Battista Debiasi (1928-2012): thispaper is humbly and affectionately dedicated to his memory, a leadingresearcher and an outstanding teacher whose brightness and kindness wewill always remember.

Copyright: c©2013 Sergio Canazza et al. This is an open-access article distributed

under the terms of the Creative Commons Attribution 3.0 Unported License, which

permits unrestricted use, distribution, and reproduction in any medium, provided

the original author and source are credited.

and hardware devices (filters, computers) conducted by Pa-duan researchers have since then produced state-of-the artresults from both the technological and musical standpoints,and have generated collaborations with several renownedcontemporary composers. CSC engineering skills have beenused to build electronic and digital instruments, augmentedreality systems, immersive video-games, and measuringinstruments. It has also led to advances in such widelydiffering fields as sound design, musical cultural heritagepreservation and promotion, and cognitive/physical reha-bilitation.

The CSC is carrying out a project for the preservationand restoration of electrophone equipments and audio doc-uments. An important (from both scientific and dissemi-nation points of view) moment in this project was the re-alization of an exhibition by the University of Padova, incollaboration with the SaMPL Lab of the Conservatory “C.Pollini”: Visions of sound. Electronic music at the Uni-versity of Padova, open from April 3 to July 18, 2012 atthe exhibition halls of the Botanical Garden. The exhibi-tion showed the history of the computer music produced inPadova and was assisted by various events, including a se-ries of educational seminars held by CSC researchers andsome concerts. The dissemination of the Computer Musichistory is hard and complicated, because of its multi-facednature. It is necessary to emphasize the communicationof all its different aspects, in particular it is important thatgeneral public understand also the genesis of the computermusic works. This paper presents our experience.

The exhibition illustrated the history of CSC, from scien-tific, technological and artistic points of view. From thefirst experiments by Teresa Rampazzi and by the groupNuove Proposte Sonore (NPS) in the sixties, the close col-laboration among the Conservatory, the CSC and the Com-puting Centre of the University, to the present, it was possi-ble to expose historic equipments, as the original magnetictape recorder used by Teresa Rampazzi, a Synthi AKS,an ARP 2500 (now the last example in Italy), and the 4iSystem, devices which allowed the realization of the elec-tronic music of the last decades, art music as well as con-sumer music. It was also possible to listen to some majorworks realized at the CSC, e.g., Prometeo by Luigi Nono,Perseo e Andromeda by Salvatore Sciarrino, and Medea byAdriano Guarnieri: for this latter musical work the originalmulti-channel installation was recreated, for the first timeafter its premiere in 2002 at the Teatro La Fenice in Venice.The exhibition was enriched by numerous interactive in-stallations, specially designed and realized by researchers

639

Proceedings of the Sound and Music Computing Conference 2013, SMC 2013, Stockholm, Sweden

mailto:[email protected]



http://creativecommons.org/licenses/by/3.0/

Figure 1. Giovanni Battista Debiasi, Vicenza (Italy), 4thJune 1928 – Padova (Italy), 24th June 2012.

Figure 2. System panels for recording, sound synthesisand processing in 1979.

at CSC – which today is with the Department of Informa-tion Engineering (DEI) – to introduce visitors to the worldof sound, applications of technology, the result of the novelresearch in the Sound and Music Computing field, in par-ticular immersive reality systems, preservation and restora-tion of musical cultural heritage, information systems forenhanced learning and for rehabilitation of disabled peo-ple.

This exhibit aims at showing the deep connections be-tween academic research and the multifaceted world ofsound, and their influences on both art music and popularmusic, especially from the Seventies.

This exhibition was important (i) as a moment of culturalreflection, because it has led to the comparison of the dif-ferent research areas that have occurred since the sixties tothe present, and (ii) as a stimulus to overcome the prob-lems related to the preservation and restoration of culturalheritage music the CSC.

In Sec. 2 the history of the Centre is concisely summa-rized. Then all the sections of the exhibition are detailed, asan example of dissemination to a general public, reportedhere for consideration by the SMC community.

2. CENTRO DI SONOLOGIA COMPUTAZIONALE

The CSC was born in 1979 [1], but it was already activesince the late sixties as a point of reference for the birth and

development of computer music in the world (the musicalworks realized in CSC are listed at http://csc.dei.unipd.it/musical_productions.html). At thesame time, with its own set of electronic equipment (filters,digital signal processors, computers) specially designed andprogrammed by researchers at the Department of Informa-tion Engineering, it is a striking witness to the technologi-cal era and its evolution in recent decades (Sec. 4).CSC, today directed by Giovanni De Poli, was foundedby Giovanni Battista Debiasi (fig. 1). In 1957 GiovanniBattista Debiasi, at the University of Padova, proposed anoriginal work about an electronic organ based on photodi-odes. This was the first step of a multidisciplinary futurefor electric/electronic engineering and music in Padova. Inthe early seventies Debiasi carried out research on speechanalysis and synthesis, in collaboration with Gian AntonioMian and Carlo Offelli [2, 3]. In the eighties and nineties,in advance to the international scientific community, De-biasi studied issues related to the preservation and restora-tion of cultural musical heritage. He trained hundreds ofstudents: his research fields are now everywhere, in Italyand in the world, and this gives the sign of the importancethat he played in the birth and development of Sound andMusic Computing (see, at least, [1, 4, 5]).

Fig. 2 shows the system panels for recording, sound syn-thesis and processing in 1979: this system was also used inthe Summer Schools organized in CSC and that were con-sidered as world references in the field of computer music.Among the various hardware systems of CSC, particularlyimportant from the history and the musicology points ofview, was the project – granted by the Laboratory for Com-puter Music at the La Biennale (LIMB) in Venice, in col-laboration with IRCAM in Paris – that led to the realizationof the 4i System (fig. 3 and Sec. 7).

CSC has been mainly a centre of promotion and culturaldiffusion of music informatics since its foundation. Thanksto close collaboration among experts of various disciplines,it has been possible to create an interdisciplinary group,which has become an international reference in the field,and has come to be part of contemporary music history.Activities of CSC can be grouped into four main areas:scientific research, music research, production and perfor-mance of music works, teaching and dissemination.

The rapid evolution known by computers and microelec-tronic devices in the second half of the last century hasled to the development of several sound synthesis methods(Sec. 5) and to reduction the processing times, allowingto recover the performer-instrument relationship and thenreintroducing the causality between gesture and sound typ-ical of the musician with his/her instrument. This evolu-tion permitted to integrate the electronic medium with tra-ditional instruments, mixing freely the sound of mechan-ical devices with sound processing generated during theperformance: arising the live electronics performer, whichallowed to recover the absence of the performer typical ofelectroacoustic music (Sec. 7), when the public was con-fused in front of stages with only loudspeakers. The com-

640


http://csc.dei.unipd.it/musical_productions.html

http://csc.dei.unipd.it/musical_productions.html

Figure 3. 4i System developed by Giuseppe Di Giugno.

puter allows to control individual processes (synthesis andsound processing) to a more abstract level than that reachedby the electrophone equipments of the sixties (generallybased on voltage control). The use of systems with mul-tiple speakers, thanks to which the sounds came from dif-ferent directions (front, back, side, top, bottom) made ob-solete the traditional concert halls and the placement ofchairs lined up in the theater. Even at this stage the CSCplayed a pioneering role (see Sections 7, 8, and 9), becom-ing a leader in the opera Prometeo by Luigi Nono (VeniceLa Biennale, 1984; Teatro alla Scala, 1985) and the workPerseo e Andromeda by Salvatore Sciarrino (StaatstheaterStuttgart, 1991; Teatro alla Scala, 1992).Now the CSC is carrying out researches in all the areas ofSMC field (Sec. 6).

3. WELL-CALCULATED MUSIC: PREMISES

The first section of the exhibition introduces the roots ofmusic in Padova. One of the major breakthroughs in the14th century Italy was the development of written musicand musical symbols. Music had traditionally been handeddown orally, but musicians and composers had come torealize that complex musical constructs had to be writ-ten down and that symbols were needed to set the time-values between different sounds. The composer and the-orist Marchetto da Padova pioneered these developmentsand his arithmetic- and geometry-based studies paved theway for musical notation, the forerunner of modern mu-sic scores. Mathematical studies are also at the core ofGiuseppe Tartini’s theories (1692-1770). He was “first vi-olin and head of concerts” at St. Antonio’s Basilica inPadova. He discovered a “terzo suono” (literally a “thirdsound”), which he heard when two different notes wereplayed together on a violin. His work was devoted to link-ing the physics to a musical and metaphysical theory. Tar-tini is known for his art of bowing, as he used a speciallydesign bow to create virtuoso effects.In this section of the exhibition the following items wereexhibited:

• original manuscripts (unique source worldwide) ofthe first half of 15th century;

• the original Trattato di musica secondo la vera scienzadell’armonia (Treatise on music according to the true

science of harmony) by Giuseppe Tartini, 1754. Inthis treatise, published in 1754, the violinist and com-poser Giuseppe Tartini accounted for his research onthe phenomenon of the third sound. He included el-ements of physics, arithmetic, and geometry, orga-nized into a complex theory which sparkled a livelydiscussion;

• original ancient violins and bows.

4. WELL-CALCULATED MUSIC:THE 20TH CENTURY

When the first instruments able to generate “new” soundsappeared in the 1950s’, composers and musicians welcomedenthusiastically this revolution. Electronic music was born,i.e. music realized with either analogue electronic (1950s’and 1960s’) or digital (since the 1970s’) devices. In themost important international research centre, technologywas used to create new sounds, or to explore and processsounds recorded and produced with this equipment. Thenew music had no performers, and the loudspeaker – themain mean to deliver sound to listeners – became the new“star” of concert halls. Musical structures became morefree, while the need for accurate control of durations andfor adequate notation posed new problems.

At the CSC, Teresa Rampazzi 2 was an electronic-musicpioneer. She and Ennio Chiggio set up the NPS GroupNuove Proposte Sonore (New Sound Proposals) in Padovain 1965. Chiggio was part of Gruppo Enne, a group whichapplied kinetics to visual art. The NPS Group conductedresearch into the timbre and density of “sound events”,creating “sound objects” (or “sounding objects” accordingto Rampazzi’s own terminology) and more or less com-plex tracks which explored acoustic phenomena. In 1972,Rampazzi donated her equipment to the Conservatory ofPadova, which was one of the very first italian Conservato-ries where electronic music classes were started – follow-ing Firenze.In this section of the exhibition the following equipmentswere exhibited:

• the original ARP 2500 (see fig. 4). It is an early1970s analogue synthesizer: it was one of the mostversatile and powerful professional synthesisers ofits time. The synthesizer came with a wide rangeof compatible modules which could be connected togenerate and manipulate sound;

• EMS Synthi AKS portable analogue synthesizer man-ufactured by Electronic Music Studios in London(fig. 5). Its built-in pin matrix, sequencer and key-board pack the power of an electronic-music labora-tory into a portable briefcase;

• TEAC A 3340 S (see fig. 4). Four-track tape recorderintroduced in the mid-1970s. It played tracks throughfour loudspeakers and paved the way for modern“surround sound”;

2 The title of this section of the exhibition was an usual question byTeresa Rampazzi to her students: “but do you have it [your music] wellcalculated?”

641


Figure 4. The original ARP 2500, the stop-watch and theTEAC reel to reel tape recorder (4-tracks) used by TeresaRampazzi.

Figure 5. The Synthi AKS used by Teresa Rampazzi.

• Junghans stop-watch – used by Teresa Rampazzi forthe realization of her well-calculated music (see fig.4);

• Teletype, electromechanical device used to transmittext messages and employed by early computers atCSC for data input/output purposes;

• Digital-to-analogue and analogue-to-digital convert-ers at 12 and 16 bits, originally connected to the IBMSystem/7, with programmable clock filter, low-passfilters at 4.5 kHz, 7 kHz, and 14 kHz.

These items, following the musical instruments history (Sec.3), allow the general public to understand the genesis of theComputer Music.

5. NUMBER AND SOUND

In the Seventies, composers discovered the potential of in-formation technology and adopted computers and electron-ics devices: the born of Computer Music. in the interna-tional field sound synthesis had an extraordinary impacton music writing, allowing composers to better understandthe way in which sounds are formed and their aural effect,transforming sometimes even the orchestral writing [6].Sec. 3 showed how Padova in the 14th century has beena research laboratory in musical writing (in particular withMarchetto da Padova). In line with this, it is interesting tonote that in the Seventies the CSC contributed to the de-

velopment of a formal musical notation language for com-puter [7].The CSC is among the pioneers of the most innovative andinteresting methods of synthesis, based on sound source(e.g., a musical instrument) modeling, instead of signalmodeling [8]. This synthesis uses algorithms that producethe sound as a side effect of a process of simulation ofphysical phenomena, i.e., reproducing what occurs in na-ture. The bow-string interaction in the physical reality,studied by Tartini in his treatise (Sec. 3), in this way be-comes a mathematical model.

The results of the research conducted by computer musicbrings a terrific deepening of knowledge within the acous-tic and psychoacoustic. It is with these studies that thefoundations are laid for the development of auditory com-munication in multimedia and multimodal environments(virtual and augmented reality). In this section multimediainstallations were exhibited, in which the visitors could in-teract with different sound synthesis techniques; a digitaljuke-box with some of the most important musical worksrealized in CSC, restored on purpose by the authors; aprintout MARCR J578 A, the publication by Enore Zaf-firi Musica per un anno (Music for a year), DUCHAMPCenter; a folder NPS (Nuove Proposte Sonore), with var-ious enclosed documents; a copy of the magazine OggettiSonori (Sound Objects), or Oggetto Sonoro (Sound Ob-ject); two original video works by Ennio Chiggio: smalltelevision in plexiglass display cabinet, with video board,and Dischi a rotazione apparente (Discs with apparent ro-tation) – Marcel Rotour (1967, Photographic tape, plex-iglass and wooden frame, 50x50x20 cm). These differ-ent items helps general public to contextualize the musicalworks, showing the relationship among music and othersarts.

6. SOUND AND SOCIETY

At the end of the Nineties, the international computer mu-sic research domain evolved into Sound and Music Com-puting (SMC), which also includes non-musical areas re-lated to research on sound. The results are manifold.Researchers in CSC developed multimodal interactive sys-tems for teaching with special interfaces, specifically de-signed to enhance the learning of students with disabilities.The research on the preservation and restoration of audiodocuments (see [9] for a review) are combined with tech-nology’s innovations in information retrieval to meet theneeds of today society where everything has to be stored,browsable, and available “anybody, anytime and everywhere”.This implies the definition of new strategies for data stor-age and study of new techniques of content search (e.g.,by humming) in data mining, as well as listening strategiesappropriate to each situation (the living room, the concerthall, the walkman/iPod headphones). Innovative 3D au-dio techniques [10] allow to virtually recreate an environ-ment in which various sound sources are located at dif-ferent moving points in space, with important applicationsin virtual reality systems, from immersive video games

642


Figure 6. The model – as shown in the exhibition – ofthe original building designed by Renzo Piano in 1984 forPrometeo by Nono.

Figure 7. An interactive installation dedicated to explainthe results of the 3D audio research domain. The visitorcould appreciate the change of the sound spatialization de-pending on of his head movement.

to flight simulators (see an example in fig. 7). Micro-phones arrays systems with variable geometry are specifi-cally designed for both monitoring of urban environmentsfor homeland security and as musicians tracking system forlive electronics [11].

7. MUSIC AND SPACE

The initial absence of typical performers in the electronicmusic repertoire is overcome with the development of com-puters able to generate electronic sounds in live contest andto process the sound signal (voices or musical instruments)in real time. Live electronics was born [12], which now isused in a large music repertoire all over the world and withit also grows new professional figures with a double train-ing: musical and scientific.The CSC also developed new interfaces to play these in-struments, necessary to control the musical timbre and thevirtual space and polarizing the interest of many composers.The traditional keyboard organ is not suited to control mul-tiple parameters simultaneously, synthesis algorithms, andsound spatialization. In the Eighties, CSC in Padua, IR-

CAM in Paris and LIMB of the Venice La Biennale jointlydeveloped the 4i System, a digital signal processors basedsystem for live electronics. This system was used in someof the most important musical works of the second halfof the Twentieth century, including Prometeo, la tragediadell’ascolto (1984-85) by Luigi Nono, based on the move-ment of sound in space. The fig. 6 shows the arrangementof the choir and orchestra of the Prometeo in a model (dis-played in the exhibition) of the original building designedby Renzo Piano for the representation at the Venice La Bi-ennale in 1984. In this section the following items wereexhibited:

• 4i System (see fig. 3). It is realized by means of a128-kbyte memory PDP11 computer with a 4i digi-tal sound processor (designed by Giuseppe Di Giugno),a 16-bit digital-to-analogue converter and a controlinterface for performance parameters;

• an interactive system (developed on purpose) in whichthe user can control the live electronics software ofthe Prometeo and contemporaneously observe theoriginal gesture of the live electronics performer;

• original scores with notes handwritten by Luigi Nono;

• heliography of the Prologue of Prometeo (in the 1984version), with several original corrections and anno-tations, probably added during the early rehearsalsin Venice;

• a multimedia installation for the interactive listen-ing of the Perseo e Andromeda (1990) by SalvatoreSciarrino, in which the synthesized sounds replacethe traditional orchestra. The visitors can listen theentire work, some parts and/or the single sound ob-ject, observing the related score.

8. MEDEA BY ADRIANO GUARNIERI (2002)

In the exhibition two large and innovative musical workswere showed: the musical theatre opera Medea (2002) byAdriano Guarnieri and the interactive multimedia installa-tion Casetta delle immagini by Carlo De Pirro.Medea is a video-opera in three part loosely based on Eu-ripide’s tragedy, for video sequences, soloists, chorus, or-chestra and live electronics, in which the sound directionbecomes almost visual and the spatial sound seems to al-ternate close-ups and overviews. It was showed in this ex-hibition (see fig. 8) by means of the original stage sound-design, using the eight-channel audio recording made dur-ing the first performance at the PalaFenice in Venice.The mythical story of Medea, represented by three femalevoices, merges with the play of the dynamics of sound inspace. The sound produced by the singers and by the or-chestra is detected by 68 microphones, processed by liveelectronics software and finally diffused by dozens of speak-ers distributed among the public. The sound movement inthe room, besides, is controlled in various ways (e.g.,musicians’gestures) and reinterpreted in real time by live electronicssoftware. This work is one of the greatest artistic studies

643


Figure 8. The stage of Medea – as represented in the exhi-bition – using the original eight-channel audio recording aswell as three video output. Guarnieri himself, visiting theexhibition, recognized this installation as “this is my realMedea”.

in expressive gesture and sound interaction, a domain bornin the late 90s, which bring interesting results even in theanalysis of musical performance.This section of the exhibition showed also some of thebetter models developed at CSC related to most recent re-search studying the possible connections between two uni-verses that may seem antithetical, the emotions and the ma-chines, deepening the procedures that enable the comput-ers to communicate and simulate expressive components,emotions, intentions and affects [13].

9. CASETTA DELLE IMMAGINIBY CARLO DE PIRRO (2002)

This interactive multimedia installation was designed bythe composer Carlo De Pirro 3 at the CSC for Piazza Pinoc-chio, the Italian space at the Expo 2002 in Neuchatel (Switzer-land), Section Artificial Intelligence and Robotics. Thework uses the results of research in the fields of analy-sis, modeling and communication of expressive contentand emotional non-verbal interaction, by means of multi-sensorial interfaces in mixed reality environments. TheCasetta delle immagini (Little house of the appearances) isa sort of magic room for children, where every gesture be-comes sound, images and colors. The visitors’ movementswere captured by cameras and analyzed by specially devel-oped software able to process a virtual gesture model andthus generate projection images and rhythmic sequences ofmusic.A similar idea is now implemented in Stanza Logo-motoria[14], a systems for educational purposes used in many Ital-ian schools, that exploits a multimodal interactive environ-ment aimed at learning through the movement and can beused in situations of learning difficulties or for childrenwith multi-disabilities.

3 Adria, 1956 – Padova, 2008. Carlo, professor of Music Composi-tion at Rovigo Conservatoire, collaborated with the CSC for more thanfifteen years: his musical compositions were (and are) a great stimulusfor the researches carried out in CSC, thanks to his innovative and artisticapproach.

In this section the original Casetta delle immagini was ex-hibited, restored on purpose by the authors.

10. CONCLUSIONS

CSC scientific research was the premise for the other ac-tivities of musical informatics, and it is the main focusof the Centre. Today the CSC still supports productionof musical works, thanks to significant investments in re-search that begun in 1979 when the Centre was officiallyfounded. In the early days the research was mainly fo-cused on sound synthesis. Nowadays, the Centre is work-ing, in synergy with the SaMPL Lab of the Conservatoryof Padova, on preservation and restoration of audio doc-uments, new sound synthesis techniques, analytical tools,techniques of sound spatialization, complex dynamic sys-tems and analysis and morphing of expressive content inmusic performances. Today CSC activities rely on a com-posite group of people, which include the Center board ofdirectors and personnel, guest researchers and musicians,and particularly on master students attending the courses“Sound and Music Computing” at Dep. of Information En-gineering of the University of Padova.The CSC is carrying out a project for the preservation andrestoration of electrophone equipments and audio docu-ments. The principal output of this project is the realiza-tion of an interactive “museum” of Computer Music and ofresearches in SMC field. The first attempt was the exhibi-tion Visions of sound. Electronic music at the University ofPadua. In the authors’ opinion, it is time to start a debateon how the scientific SMC community wants to preserveits history and what kind of access tools we are able todevelop, in order to communicate the (scientific and ap-plicative) potential of its researches to the general publicand (no less important) to potential investors [15].

Acknowledgments

In the realization of the exhibition Visions of sound the ef-forts of the CSC researchers, of the entire University ofPadova, of the Conservatory “C. Pollini”, of Padova and ofthe Veneto Region were terrific. The authors deeply thankin particular the scientific and the organization boards ofthe exhibition, the Luigi Nono’s Archive (in particular thepresident Nuria Schoenberg-Nono), the Rectorate and theMuseums Center of the University of Padova, the direc-tor of the Conservatory Maria Nevilla Massaro and NicolaBernardini (professor of Electronic Music), and Ivo Rossi(vice-Mayor of Padova).

11. REFERENCES

[1] G. B. Debiasi, G. De Poli, G. Tisato, and A. Vidolin,“Centro di Sonologia Computazionale C. S. C. Uni-versity of Padova,” in Proc. of International ComputerMusic Conference, 1984, pp. 287–294.

[2] G. Debiasi, G. De Poli, G. A. Mian, G. Mildonian, andC. Offelli, “Italian speech sinthesis from unrestrictedtext for an automatic answerback system,” in Proc. of8th Inf. Congress of Acoustics, London, 1974, p. 296.

644


[3] G. A. Mian, F. Morgantini, and C. Offelli, “An applica-tion of the linear prediction technique to efficient cod-ing of speech segments,” in Proc. of 1976 IEEE Int.Conf. Acoustic, Speech, Signal Processing, Philadel-phia, April 12-14, 1976, p. 722.

[4] G. Francini, G. Debiasi, and R. Spinabelli, “Study ofa system of minimal speech reproducing units for ital-ian,” JASA, vol. 43, pp. 1282–1286, 1968.

[5] G. B. Debiasi and M. Rubazzer, “Architecture for adigital sound synthesys processor,” in Proc. of ICMC,1982, pp. 225–231.

[6] G. De Poli, “A tutorial on digital sound synthesis tech-niques,” Computer Music Journal, vol. 7, no. 4, pp.8–26, 1991.

[7] G. B. Debiasi and G. De Poli, “MUSICA, A Languagefor the Transcription of Musical texts for Computers,”Interface, vol. 11, no. 1, pp. 1–27, 1982.

[8] G. Borin, G. De Poli, and A. Sarti, “Algorithms andstructures for synthesis using physical models,” Com-puter Music Journal, vol. 16, no. 4, pp. 30–42, 1992.

[9] S. Canazza, “The digital curation of ethnic music au-dio archives: from preservation to restoration,” Inter-national Journal of Digital Libraries, vol. 12, no. 2-3,pp. 121–135, 2012.

[10] S. Spagnol, M. Geronazzo, and F. Avanzini, “On therelation between pinna reflection patterns and head-

related transfer function features,” IEEE Trans. Audio,Speech, and Language Process., vol. 21, no. 3, pp.508–519, March 2013.

[11] D. Salvati and S. Canazza, “Adaptive time delay es-timation using filter length constraints for source lo-calization in reverberant acoustic environments,” IEEESignal Processing Letters, vol. 20, no. 5, pp. 507–510,2013.

[12] A. Vidolin, “Musical interpretation and signal process-ing,” in Musical Signal Processing, C. Roads, S. T.Pope, A. Piccialli, and G. De Poli, Eds. Lisse: Swetsand Zeitlinger, 1997, pp. 439–459.

[13] S. Canazza, G. De Poli, A. Roda, and A. Vidolin,“Expressiveness in music performance: Analysis,models, mapping, encoding,” in Structuring Musicthrough Markup Language: Designs and Architec-tures, J. Steyn, Ed. IGI Global, 2012, pp. 156–186.

[14] S. Zanolla, A. Roda, F. Romano, F. Scattolin,S. Canazza, and G. L. Foresti, “When sound teaches,”in Proc. of Sound and Music Computing Conference,2011, pp. 64–69.

[15] N. Bernardini and G. De Poli, “The future of soundand music computing,” Journal of New Music Research(special issue), vol. 36, no. 3, pp. 139–239, 2007.

645


SMOOTHNESS UNDER PARAMETER CHANGES: DERIVATIVES ANDTOTAL VARIATION

Risto Holopainen

ABSTRACT

Apart from the sounds they make, synthesis models aredistinguished by how the sound is controlled by synthesisparameters. Smoothness under parameter changes is oftena desirable aspect of a synthesis model. The concept ofsmoothness can be made more accurate by regarding thesynthesis model as a function that maps points in parameterspace to points in a perceptual feature space. We introducenew conceptual tools for analyzing the smoothness relatedto the derivative and total variation of a function and applythem to FM synthesis and an ordinary differential equation.The proposed methods can be used to find well behavedregions in parameter space.

1. INTRODUCTION

Some synthesis parameters are like switches that can as-sume only a discrete set of values, other parameters are likeknobs that can be seamlessly adjusted within some range.Only the latter kind of parameter will be discussed here.Usually, a small change in some parameter would be ex-pected to yield a small change in the sound. As far as thisis the case, the synthesis model may be said to have wellbehaved parameters.

A set of criteria for the evaluation of synthesis modelswere suggested by Jaffe [1]. Three of the criteria seem rel-evant in this context: 1) How intuitive are the parameters?2) How perceptible are parameter changes? 3) How wellbehaved are the parameters? The vague notion of smooth-ness under parameter changes (which is not the name ofone of Jaffe’s criteria) can be made more precise by theapproach taken in this paper.

From a user’s perspective, the mapping from controllersto synthesis parameters is important [2]. In synthesis mod-els with reasonably well behaved parameters, there are goodprospects of designing mappings that turn the synthesismodel and its user interface into a versatile instrument.However, a synthesis model does not necessarily have tohave well behaved parameters to be musically useful. De-spite the counter-intuitive parameter dependencies in com-plicated nonlinear feedback systems, some musicians areusing them [3]. Likewise, acoustic instruments may have

Copyright: c©2013 Risto Holopainen et al. This is

an open-access article distributed under the terms of the

Creative Commons Attribution 3.0 Unported License, which permits unre-

stricted use, distribution, and reproduction in any medium, provided the original

author and source are credited.

far from smooth responses to changes in physical controlvariables (e.g. overblowing in wind instruments).

The smoothness of transitions has been proposed as a cri-terion for evaluating sound morphings [4]. As the mor-phing parameter is varied between its extremes, one wouldexpect the perceived sound to pass through all intermediatestages as well. However, because of categorical perceptionsome transitions may not be experienced as gradual. It maybe impossible to create a convincing morph between, say,a banjo tone and a sustained trombone tone.

Quantitative descriptions of the smoothness of a synthesisparameter should use a measure of the amount of change inthe sound, which can be regarded as a distance in a percep-tual space. Similarity ratings of pairs of tones have beenused in research on timbre perception, where multidimen-sional scaling is then used to find a small number of di-mensions that account for the perceived distances betweenstimuli [5]. In several studies, two to four timbral dimen-sions have been found and related to various acoustic cor-relates, often including the attack time, spectral centroid,spectral flux and spectral irregularity [6]. The importanceof spectrotemporal patterns was stressed in a more recentstudy [7] where five perceptual dimensions were found.

Most timbre studies have focused on pitched, harmonicsounds, in effect neglecting a large part of the possiblerange of sounds that can be synthesized. At the other ex-treme, the problem of similarity between pieces of musichas been addressed in music information retrieval [8]. Thedifficulty in comparing two pieces of music is that theymay differ in so many ways, including tempo, instrumen-tation, melodic features and so on. Most synthesis modelsof interest to musicians are also able to vary along severaldimensions of sound, e.g., pitch, loudness, modulation rateand many timbral aspects. A thorough study of the per-ceived changes of sound would include listening tests foreach synthesis model under investigation. A more tractablesolution is to use signal descriptors as a proxy for suchtests.

There are numerous signal descriptors to choose from [9],but the descriptors should respond to parameter changes ina given synthesis model. For example, in a study of thetimbre perception of a physical model of the clarinet, theattack time, spectral centroid and the ratio of odd to evenharmonics were found to be the salient parameters [10].Since a synthesis model may be well behaved with re-spect to certain perceptual dimensions but not to others,the smoothness may be assessed individually for each of aset of complementary signal descriptors.

A synthesis model will be thought of as a function that

646



http://creativecommons.org/licenses/by/3.0/

maps a set of parameter values to a one-sided sequence ofreal numbers, representing the audio samples. It will beassumed that all synthesis parameters are set at the begin-ning of a note event and remain fixed during the note. Dy-namically varying parameters can be modelled by an LFOor envelope generator, but for simplicity we will consideronly synthesis parameters that remain constant over time.

The effects of parameter changes may be studied eitherlocally near a specific point in parameter space, or glob-ally as a parameter varies throughout some range. The lo-cal perspective leads to a notion of the derivative of a syn-thesis model, which is developed in section 2. Parameterchanges over a range of values are better described by thetotal variation, which is introduced in section 3. Then, sec-tions 4 and 5 are devoted to case studies of the smoothnessof FM synthesis and the Rossler attractor. Some applica-tions and limitations of the methods are discussed in theconclusion.

2. SMOOTHNESS BY DERIVATIVE

In order to formalize the notion of smoothness, we will for-mulate a synthesis model explicitly as a function and de-scribe what it means for that function to be smooth. First,we define a suitable version of the derivative. Then, in Sec-tions 2.2 and 2.3, the practicalities of an implementationare discussed.

2.1 Definition of the derivative

Consider a synthesis model as a function G : Rp → RN

that maps parameters c ∈ Rp to a one-sided sequence ofsamples xn, n = 0, 1, 2, . . ., where the sample sequencewill be notated X(c) to indicate its dependence on the pa-rameters. Then the question of smoothness under param-eter changes is related to the degree of change in the se-quence X(c) as the point c in parameter space varies. Inpractice, the distance in the output of the synthesis modelwill be measured through a signal descriptor rather thanfrom the raw output signal. If a distance were to be cal-culated from the signals themselves, two periodic signalswith identical amplitude and frequency but different phasemight end up being widely separated according to the met-ric, despite sounding indistinguishable to the human ear.Signal descriptors that are clearly affected by the synthesisparameters and that can be interpreted in perceptual termsare preferable.

In order to treat the synthesis model as a function, it willbe assumed to be deterministic in the sense that the samepoint in parameter space always yields identical sample se-quences. The idea of relating how much a function f(x)changes as the independent variable x changes by a smallamount leads to the concept of derivative. Functions thathave derivatives of all orders are called smooth. A morerefined concept is to say that a function is k times con-tinuously differentiable; the larger k is, the smoother thefunction.

Now, we would like to apply some suitably defined deriva-tive to synthesis models considered as functions. To thisend, a distance metric is needed for points in the parameter

space, and another distance metric is needed for points inthe space of sample sequences. Let dp(c, c′) be a metricin parameter space, and let ds(X(c), X(c′)) be a metric inthe sequence space. The derivative can then be defined asthe limit

lim‖δ‖→0

ds(X(c), X(c+ δ))

dp(c, c+ δ)(1)

where δ ∈ Rp is some small displacement in parameterspace. The limit, if it exists, is the derivative evaluated atthe point c.

In general, synthesis parameters do not make up a uni-form space. Different parameters play different roles; theyaffect the sound subtly or dramatically and may interact sothat the effect of one parameter depends on the settings ofother parameters. This makes it hard to suggest a generaldistance metric that would be suitable for any synthesismodel. Our solution will be to consider the effects of vary-ing a single synthesis parameter cj at a time, so the distancedp(c, c

′) in (1) reduces to∣∣cj − c′j∣∣. Furthermore, consider

a scalar valued signal descriptor φ(i)(c) ≡ φ(i)(X(c))which itself is a signal that depends on the sample se-quence and the parameter value. Thus, we arrive at a kindof partial derivative evaluated with respect to the parametercj using a signal descriptor φ(i),

∂φ(i) ◦G(c)

∂cj= limh→0

ds(φ(i)(c), φ(i)(c+ hej))

h(2)

where ej is the jth unit vector in the parameter space.Clearly the magnitude of this derivative depends on thespecifics of the signal descriptors used and which synthesisparameters are considered. In a finite dimensional space,all partial derivatives should exist and be continuous forthe derivative to exist. Such a strict concept of derivativedoes not make sense in the present context where any num-ber of different signal descriptors can be employed, so onlythe partial derivatives (2) will be considered.

Before discussing the implementation, let us recall someintuitive conceptions of the derivative. As William Thurstonhas pointed out [11], mathematicians understand the deriva-tive in multiple ways, including the following.

• The derivative is the slope of a line tangent to thegraph, if it has a tangent.

• In terms of symbolic operations, ddxx

n = nxn−1.

• The derivative is the best linear approximation to thefunction near a point.

• It is the limit of what you get by looking at a functionunder a microscope of higher and higher power.

Synthesis models are typically very complicated if con-sidered as mathematical functions; hence the analytic ap-proach to differentiation is out of the question and one hasto rely upon numerical approximations. The various intu-itions of what the derivative is may guide a practical nu-merical implementation in different directions, as will befurther discussed in Section 2.3.

647


Numerical estimation of the derivative is highly sensitiveto measurement noise. Here one source of measurementnoise are the signal descriptors. Whereas one would liketo magnify a curve in order to find its derivative at a point,doing so will also reveal more fine details caused by thenoise, which may lead to false estimates. When properlyestimated, the derivative will exaggerate irregularities andmake them easier to detect.

2.2 Pointwise or time-average distance?

The distance metric ds in sequence space has so far beenleft unspecified. We propose two alternatives, each suit-able in different situations. The signal descriptors that willbe used are based on short-time Fourier transforms of thesignal X(c) at regular intervals, using a hop size equal tothe FFT window length, L. Hence, the signal descriptoris a sequence which we write concisely as φm(c), wherem = bn/Lc is a time index.

Using a pointwise distance metric, one may follow thetwo signals over time and take the sum over their distances|φm(c)− φm(c′)| at each moment. Since these are infi-nite sequences, the sum may not converge. Therefore, anexponentially decaying weighting function is applied in thedistance metric

ds(X(c), X(c′)) =

[ ∞∑m=0

γm (φm(c)− φm(c′))2

]1/2(3)

where γ ∈ (0, 1) controls the decay rate. Convergence isthen guaranteed if the signal descriptors φm are bounded.

The second approach involves first taking an average overthe sequence φm(c), m = 0, 1, . . . ,M and then compar-ing averages of two sequences. Thus, the distance becomes

ds(X(c), X(c′)) = |〈φ(c)〉 − 〈φ(c′)〉| (4)

where we take time averages

〈φ(c)〉 = limM→∞

1

M

M−1∑m=0

φm(c) (5)

before computing the distance. For time-varying signals,the drawback of the second approach is that two differenttemporal sequences φm may average to the same value.

As an illustration, consider two signals of equal averageamplitude, the first having constant amplitude and the sec-ond with a periodic amplitude modulation. Suppose wecompare the RMS amplitudes of the two signals using thesecond approach (4). When averaged over sufficiently longtime, both signals will appear to have the same averageamplitude. In contrast, the pointwise distance measure (3)will detect their difference.

2.3 Estimation of the derivative

A numerical computation of the derivative may return anumber even if the limit (1) or (2) does not exist. There-fore, a measure of the reliability of the estimate, or “degreeof differentiability”, should be added.

Although the synthesis model is assumed to be deter-ministic, all signal descriptors will introduce measurementnoise. If a number of windowed segments of the signal areanalyzed, then the spectrum of these segments will fluctu-ate unless some integer number of periods fit exactly intothe window. The fluctuation can be reduced by using thetime-averaged version of the distance metric (4).

Several methods for the estimation of derivatives exist[12]. Theoretically, it may be possible to arrive at ana-lytical expressions for the derivative of a synthesis modelconsidered as a function, at least in some trivial cases. Inpractice, numerical estimates have to be used. A simpleapproach would be to evaluate (2) directly at two pointsc and c′. Another approach is to fit a polynomial to thecurve φ(c), and then do a symbolic differentiation of thepolynomial.

The method of estimation of derivatives that will be usedhere is similar to one described in ref. [12, p. 231] butslightly simpler. The derivative at a point c0 is approx-imated by a sequence of symmetric differences with de-creasing distance h. A linear regression of this sequencegives the derivative as the intercept. Suppose a sequenceof slopes

yi(c0;hi) =φ(c0 + hi)− φ(c0 − hi)

2hi(6)

are given. Then the limit as h → 0 can be found as they-intercept of the fitted line

yi = d+ bhi + ηi, (7)

which gives the estimated derivative d. This method alsoprovides a hint about the badness of fit, for which the rootmean square error (RMSE) of the residuals η can be used.

3. TOTAL VARIATION

Whereas the derivative is concerned with local behaviourof a function, an even more useful perspective on the smooth-ness of a synthesis model may be to look at its propertiesover intervals of a parameter. One possible way to do so isto measure the length of the curve that a signal descriptortraces out as the parameter traverses some interval. If thiscurve is highly wrinkled, the curve becomes rather long,whereas a straight line connecting the endpoints means thatthe parameter changes are smooth. The total variation ofa function may be used for such a measure; intuitively, itmeasures the length travelled back and forth on the y-axisof a function y = f(x), x ∈ [a, b].

Let f(x) be a real function defined on an interval x0 ≤x ≤ xk, and suppose x0 < x1 < · · · < xk is a partition ofthe interval. Then the total variation of f(x), x0 ≤ x ≤ xkis defined as

Vxkx0

(f) = supk∑j=1

|f(xj)− f(xj−1)| (8)

taking the supremum over all partitions of the function. Iff is differentiable, the total variation is bounded and canbe expressed as

648


Vxkx0

(f) =

xk∫x0

|f ′(x)| dx. (9)

Also, recall that one way for a function to fail to be differ-entiable is that its total variation diverges to infinity.

The mesh of the partition, which is the greatest distance|xj − xj−1|, needs to be fine enough when estimating thetotal variation numerically. A global description of thefunction’s smoothness is obtained from considerations ofthe limit of the total variation as the mesh gets finer. Sup-pose the partition of [x0, xk] is uniform with each pointseparated from its nearest neighbours by |xj−xj−1| = ∆.Then, the question is whether a limit exists as ∆→ 0.

For the present purposes it will suffice to consider ap-proximations of the total variation using a small but fixedmesh. Certain functions may appear to have different amountsof total variation when observed at different scales. A slowincrease in total variation as the mesh is successively madefiner indicates that the estimation process goes as intended.

An alternative to measuring the total variation would beto measure the arc length, which can be thought of as thelength of a string fitted to the curve if it is continuous.Fractal curves on the plane have the property that their arclength grows as the measurement scale gets smaller.

When measuring the total variation of a signal descriptorover a range of synthesis parameter values, there are stilltwo possible approaches to how the distance is measured.As discussed above in section 2.2, either a pointwise dis-tance may be taken, or the distance may be taken over timeaverages of the signal descriptors. The latter approach willbe used here because it is better suited for the case of staticparameters. Applications of the derivative and total varia-tion to two synthesis models will be demonstrated next.

4. FM SYNTHESIS

With only three synthesis parameters, basic FM synthesisis convenient for investigations of the smoothness of itsparameter space. The formula that will be used is

xn = sin(2πfcn/fs + I sin(2πfmn/fs)) (10)

with modulation index I , carrier frequency fc, modula-tor frequency fm and sample rate fs = 48 kHz. Sincethe spectrum of the signal (10) is governed by a sum ofBessel functions [13], it may actually be possible to esti-mate some related signal descriptors directly from the for-mula, although we will not attempt to do so. The oscil-lations of the Bessel functions give FM synthesis its char-acteristic timbral flavour of partials that fade in and out asthe modulation index I increases, with the overall bright-ness increasing with the modulation index. Brightness isrelated to the spectral centroid, which will be used to studythe effects of parameter changes.

In the top of Figure 1, the centroid is shown as a functionof I at two different carrier to modulator (C:M) ratios. Thecentroid, given in units of normalized frequency, is mea-sured as the time average over 25 FFT windows using a

Modulation index

Cen

troi

d

0 3 6 9 12

0.05

0.10

0.15

fc : fm = 1

fc : fm = 1 2

Der

ivat

ive

0 3 6 9 12

−0.

025

0.0

0.02

5

Figure 1. FM synthesis. Top: centroid as a function ofmodulation index for fc = fm = 440 Hz (solid line) andfc = 311.1, fm = 440 Hz (dashed line). The outer linesindicate one standard deviation of the centroid. Bottom:the derivative of the centroid at fc = fm = 440 Hz.

1024 point Hamming window. As can be seen, the C:M ra-tio 1 gives a rather bumpy curve with a general rising trendof the centroid, but with several local peaks. The bottompart shows the derivative, estimated with the method de-scribed in the end of Section 2.3. Evidently, the derivativeis discontinuous at each of the peaks. The RMSE of thelinear regression used in the estimation of the derivative istypically very small, but has sharp peaks around the dis-continuities. It turned out to be necessary to re-initializethe oscillator’s initial phase at the beginning of each run ata new parameter value, otherwise there would be oscilla-tions in the centroid as a function of modulation index thatwould prevent the derivative from converging.

The total variation of the centroid over the range 0 <I ≤ 12.5 is about 0.127 for the inharmonic ratio fc/fm =1/√

2, and increases to about 0.188 for fc/fm = 1. Wemay now ask how the total variation changes as a functionof the C:M ratio. This is shown in Figure 2. Narrow peaksarise at the simple C:M ratios 1 : 2, 1 and 3 : 2. Inso-far as FM synthesis is reputed for its timbral variability asthe modulation index varies, this phenomenon is more pro-

649


FM / spectral centroid

C:M ratio

Tota

l Var

iatio

n

0.5 1.0 1.5

0.10

0.15

0.20

I ∈ (0, 12.5)

Figure 2. Total variation of the centroid of FM signals forI ∈ [0, 12.5] as a function of the C:M ratio.

nounced at the simple C:M ratios that result in harmonicspectra.

Since the density of the spectrum depends on the modu-lation index as well as on the C:M ratio, signal descriptorsrelated to spectral density may provide additional insights.The spectral entropy will be used for this purpose. Spec-tral entropy is measured from the amplitude spectrum, nor-malized so that all bins ak sum to 1. Then, the normalizedentropy is

H = − 1

norm

∑k

ak log ak (11)

where a perfectly flat spectrum yields the maximum spec-tral entropy H = 1, and a sinusoid results in the smallestpossible entropy of a signal that is not completely silent.

In Figure 3, the spectral entropy is shown as a functionof the C:M ratio as well as the modulation index. De-spite an even geometric progression of the modulation in-dex I ∈ [0.25, 20], the curves are slightly irregularly dis-tributed. Two dips in spectral entropy can be seen at thesimple ratios C : M = 1, 2. These dips can be understoodto result from the fact that, at harmonic C:M ratios, severalpartials overlap (negative frequencies match positive fre-quencies), whereas for inharmonic ratios, there are moredistinct partials in the spectrum.

The total variation of spectral entropy over the range ofC:M ratios shown in Figure 3 is about 1 for I = 0.25, andit increases monotonically to a maximum value of 2.5 atI = 1.25. For higher modulation indices, the total varia-tion decreases. These results can be interpreted as indicat-ing that, if the modulation index is set at a fixed value andthe C:M ratio is varied, then the sounds will change less forlow modulation indices, and the maximum change occursfor I = 1.25.

5. THE ROSSLER SYSTEM

Ordinary differential equations with bounded and oscil-lating solutions are good candidates for sound synthesis.

C:M ratio

Spe

ctra

l ent

ropy

0 1 2 3

0.4

0.5

0.6

0.7

0.8

incr

easi

ng m

odul

atio

n in

dex

(I =

0.2

5 −

20)

Figure 3. Spectral entropy of FM as a function of C:Mratio (horizontal) and modulation index (vertical).

Figure 4. Poincare section of the Rossler system showingbifurcations for c ∈ [1, 8] and a = b = 0.3.

In particular, there are many nonlinear oscillators capa-ble of both chaotic and periodic behaviour. Rossler’s sys-tem [14],

x = −y − zy = x+ ay (12)z = b+ z(x− c)

is known to have a chaotic attractor at a = b = 0.2, c =5.7. For lower values of c there are periodic solutions.A Poincare section across the ray x = −y, x ≥ 0 ata = b = 0.3 and a range of values of c reveals a perioddoubling route to chaos, after which there is a period twowindow (see Figure 4). In the following, (12) is solvedwith the fourth order Runge-Kutta method. The system isallowed time to approach an attractor by iterating at least25000 time steps of size 0.025 before any measurementsare taken.

The system rotates in the xy-plane, with occasional spikesin the z variable. Therefore, the x and y variables are suit-able for use as audio signals, after they have been suitably

650


●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●●●

●●●●●●●●

●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●

●●●●

●●●●●

●●●●●●●●●●●●

●●●●●●

●●●

●●●●

●●

●

●

●●●

●

●●●●●●

●

●

●●●●●●●

●●●●●●●●

●●●

●●●

●●●●

●●●

●●●●●●

●

●

●●●●●

●●●

●

●

●●●

●

●●

●●

●

●●●●●●●

●

●●

●

●

●●

●●●●

●●●●●●

●●●

●

●

●

●●

●●

●

●

●

●●●●

●

●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●

●●●●●●●●

●●●●●●●●●

●●●●●●

●●●●●●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●

●

●●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●

●●●

●

●

●

●●●

●●●

●●

●

●●

●

●●

●

●

●●

●

●●

●●●●

●●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●●

●

●●

●●

●

●

●

●

●●

●

c

RM

S a

mpl

itude

1 3 5 7

25

8

(x + y)/2

z

Figure 5. RMS amplitude of the Rossler system; the aver-age of x and y is greater than z for low values of c.

scaled in amplitude. The first thing to check with an or-dinary differential equation intended for use as an audiooscillator is its amplitude range and stability. As can beseen from Figure 4, the amplitude grows approximatelylinearly with c over the displayed range. By measuring theRMS amplitude of each coordinate, one gets a more de-tailed overview of the amplitude’s dependence on the pa-rameter c (see Figure 5). Because the amplitudes of x andy are typically not very different, their average has beenplotted together with the amplitude of the z coordinate.

Bifurcation plots already reveal a few things about thesmoothness under parameter changes. Each bifurcation isa point where the system’s behaviour changes in a discon-tinuous way, whereas the behaviour between bifurcationscan be expected to vary more smoothly.

Before going further, let us recall that dynamic systemsmay depend critically on the initial condition. Indeed, chaosis defined in terms of the exponential divergence of twoorbits starting from infinitesimally separated initial condi-tions, which is measured with the largest Lyapunov expo-nent [15]. Even more dramatically, different initial con-ditions may lead to different kinds of behaviour. In con-servative systems, orbits may be periodic, quasiperiodic orchaotic depending on the initial condition. Dissipative sys-tems, such as Rossler’s, have a basin of attraction of pointsthat end up on the attractor, but should an orbit be startedfrom outside the basin of attraction, it may wander off toinfinity.

It is important to distinguish the properties of the orbit it-self (chaotic versus regular) from the bifurcation scenariosas a parameter is varied. When looking at bifurcation di-agrams, there are intervals of smooth change and intervalsthat are very irregular. It is tempting to guess that the ir-regular parts correspond to chaotic orbits, and the smoothparts to periodic orbits. This is only a half-truth; in fact,there are periodic windows interspersed with all the chaos.

As already seen, the RMS amplitude changes smoothlyin some regions and irregularly in others. A quick compar-ison with the largest Lyapunov exponent λ indicates thatthe irregular parts correspond to chaotic regions (see Fig-ure 6). Although it is easy to pick out “irregular regions”by visual inspection, a localized version of total variationcan also achieve this. The local variation (LV) is defined as

the total variation over a short interval of length δ centredabout a point x:

LV (f ;x, δ) = Vx+δ/2x−δ/2 (f) (13)

A mathematical definition of the LV would probably in-volve taking the limit δ → 0, but for practical purposes asmall but finite interval must be used. Now the smooth-ness of a curve may be described in the neighbourhood ofany point x0, which is computed by partitioning the inter-val into a suitably large number of points and proceedingas described above in Section 3. In the following example,δ = 0.02 has been subdivided into 16 steps to find the localvariation.

c

λ

1 3 5 7

00.

050.

10.

15

Loca

l Var

iatio

n

1 3 5 7

02

46

L.V. of RMS amplitude

Figure 6. Greatest Lyapunov exponent (top) and localvariation of the RMS amplitude (bottom) for the Rosslersystem as a function of the parameter c.

The local variation of the average RMS amplitude of thex and y coordinates of the Rossler system are shown inFigure 6 below a plot of the largest Lyapunov exponentover the same parameter range. When λ = 0, the dynamicsis regular (either periodic or quasi-periodic), whereas λ >0 indicates chaos. It is worth noting that regions of regulardynamics correspond to low values of the local variation,i.e., the amplitude changes smoothly. At chaotic regions,the local variation obtains higher values, although there is

651


c

d|z|

1 2 3 4

00.

250.

5

●

●

●

Figure 7. Derivative of the peak amplitude of the z coor-dinate as a function of c. Points of bifurcations are markedwith circles.

no simple correlation between λ and LV. The higher valuesof LV in chaotic regions can be partly explained by theexistence of periodic windows which may be very thin, yetare known to be dense in the chaotic regions.

In the interval 1 ≤ c ≤ 4, there is a sequence of pe-riod doubling bifurcations. Most changes in amplitude aretoo subtle to notice directly (compare Figure 5), but takingthe derivative, as shown in Figure 7, reveals points wherethe slope changes. In fact, the bifurcation points would beeven easier to detect by plotting the second derivative ofthe peak amplitude.

In this study of the Rossler system, the effects of tran-sients and dynamic parameter changes have been mini-mized. On the contrary, in a performance situation whenusing the Rossler system as an audio oscillator, its param-eters would typically change over time. Then one may no-tice effects of hysteresis near bifucrations and in the chaoticregions. Approaching the same parameter value from dif-ferent directions may then result in different behaviour.

6. CONCLUSION

By conceiving of a synthesis model as a function frompoints in parameter space to one-sided real sequences ofaudio samples, we have introduced a concept of derivativeand total variation that can be used to describe the smooth-ness properties of the synthesis model. The derivative re-lates to local properties near specific points in parameterspace, whereas the total variation characterizes the amountof change over intervals of a parameter. Interesting find-ings were that the total variation of the centroid with re-spect to the modulation index in FM synthesis is greaterfor simple harmonic C:M ratios than for other ratios. Inother words, FM becomes smoother for inharmonic C:Mratios than for simple ratios. In the study of the Rosslersystem, we found that regular dynamics corresponds tosmooth variation in the RMS amplitude. Chaotic regionsare generally less smooth in parameter space, but thereis some variation and relatively smooth parameter regionsmay exist where the system is chaotic as well.

The methods of characterizing the smoothness of synthe-sis models can be applied to analog synthesis and even to

acoustic instruments using mechanical transducers to ex-cite them. Mechanical transducers may be needed also forthe automated control of acoustic instruments by MIDI orother means, but the response characteristics of the trans-ducer and the instrument considered together may not beknown in advance and need to be mapped out. Analog,voltage controlled synthesizers can be similarly studied byapplying some control voltage to one of its inputs. Then,studying the signal’s response to changes in control volt-age can further elucidate input to output relations and thesmoothness of the parameter. Although smoothness prop-erties can be roughly assessed by visual inspection, thederivative, and the total and local variations provide quan-titative measures of smoothness.

Comparisons of smoothness properties across differentsynthesis models are, however, not so straightforward. Onemight intuitively want to argue that the Rossler system isless smooth, on the whole, than FM synthesis, but the setof synthesis parameters have entirely different meaningsin the two models, so a direct comparison will be prob-lematic. The same signal descriptors and distance metricsmust of course be used for both synthesis models, and onemust decide what parameter ranges to compare.

Noise is used in many kinds of synthesis. If the noise isprominent in the output signal, it will increase the varianceof the signal descriptors and make the estimation of deriva-tives and total variation more complicated. If the noiseis mild enough not to alter the behaviour of the synthe-sis model altogether, one can take ensemble averages overmany runs of the system. Stochastic synthesis such as Xe-nakis’ Gendyn algorithm [16] may however be beyond thescope of the present methods.

Ordinary differential equations and nonlinear feedbacksystems may exhibit hysteresis. In synthesis models withhysteresis, there is no longer a unique correspondence be-tween the point in parameter space and the resulting out-put signal. This fact invalidates the assumption that thesynthesis model can be thought of as a function that mapspoints in parameter space to sequences in the sample se-quence space. Sometimes a transition from one type ofbehaviour to another may depend not only on the directionof the changing parameter, but also the speed of its change.

We began by making the assumption that signal descrip-tors could be used instead of conducting listening tests.This is obviously an exaggeration. Firstly, one needs toknow what perceptual characteristics of sound are capturedby various signal descriptors. Second, we have been look-ing at rather small variations in these descriptors and mag-nified them with the derivative or considered their totalvariation. It is very easy to gain a false impression thatminor variations or roughnesses in the curves would be au-dible. Listening tests would be necessary in order to assesshow the smoothness and irregularity of parameter changesare really perceived.

The assumption that maximally smooth parameters arealways preferable is not necessarily true. Monotonicityand smoothness may be good, because then the parame-ter can be remapped in a way that is more practical for theuser. Nevertheless, the rugged appearance of the parame-

652


ter space of a chaotic system should not detract musiciansfrom using them.

7. REFERENCES

[1] D. Jaffe, “Ten criteria for evaluating synthesis tech-niques,” Computer Music Journal, vol. 19, no. 1, pp.76–87, Spring 1995.

[2] A. Hunt, M. Wanderley, and M. Paradis, “The impor-tance of parameter mapping in electronic instrumentdesign,” in Proceedings of the 2002 Conference onNew Instruments for Musical Expression (NIME-02),Dublin, Ireland, 2002.

[3] D. Sanfilippo and A. Valle, “Towards a typology offeedback systems,” in Proc. of the ICMC 2012, Ljubl-jana, Slovenia, September 2012, pp. 30–37.

[4] M. Caetano and N. Osaka, “A formal evaluation frame-work for sound morphing,” in Proc. of the ICMC 2012,Ljubljana, Slovenia, 2012, pp. 104–107.

[5] J. Grey, “Multidimensional perceptual scaling of mu-sical timbres,” J. Acoust. Soc. Am, vol. 61, no. 5, pp.1270–1277, May 1977.

[6] A. Caclin, S. McAdams, B. Smith, and S. Winsberg,“Acoustic correlates of timbre space dimensions: Aconfirmatory study using synthetic tones,” J. Acoust.Soc. Am, vol. 118, no. 1, pp. 471–482, 2005.

[7] T. Elliott, L. Hamilton, and F. Theunissen, “Acousticstructure of the five perceptual dimensions of timbre inorchestral instrument tones,” J. Acoust. Soc. Am, vol.133, no. 1, pp. 389–404, January 2013.

[8] J.-J. Aucouturier and F. Pachet, “Music similarity mea-sures: What’s the use?” in Proceedings of the Interna-tional Symposium on Music Information Retrieval (IS-MIR), Paris, France, October 2002.

[9] G. Peeters, B. Giordano, P. Susini, N. Misdariis, andS. McAdams, “The timbre toolbox: Extracting audiodescriptors from musical signals,” J. Acoust. Soc. Am,vol. 130, no. 5, pp. 2902–2916, November 2011.

[10] M. Barthet, P. Guillemain, R. Kronland-Martinet, andS. Ystad, “From clarinet control to timbre perception,”Acta Acustica united with Acustica, vol. 96, pp. 678–689, 2010.

[11] W. Thurston, “On proof and progress in mathemat-ics,” Bulletin of the American Mathematical Society,vol. 30, no. 2, pp. 161–177, April 1994.

[12] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery,Numerical Recipes. The Art of Scientific Computing,3rd ed. Cambridge University Press, 2007.

[13] J. Chowning, “The synthesis of complex audio spectraby means of frequency modulation,” Journal of the Au-dio Engineering Society, vol. 21, no. 7, pp. 526–534,September 1973.

[14] O. Rossler, “An equation for continuous chaos,”Physics Letters, vol. 57A, no. 5, pp. 397–398, July1976.

[15] T. Tel and M. Gruiz, Chaotic Dynamics. An Introduc-tion Based on Classical Mechanics. Cambridge Uni-versity Press, 2006.

[16] I. Xenakis, Formalized Music. Thought and Mathemat-ics in Music. Stuyvesant: Pendragon Press, 1992.

653


proceedings of smc sound and music computing …smcnetwork.org/system/files/visions of sound - the...

Documents