a perceptually-motivated harmonic compatibility method for

A Perceptually-Motivated HarmonicCompatibility Method for Music Mixing

Gilberto Bernardes1 Matthew E. P. Davies1, and Carlos Guedes1,2

1 Sound and Music Computing, INESC TEC2 New York University Abu Dhabi

{gba, mdavies} @inesctec.pt, [email protected]

Abstract. We present a method for assisting users in the process ofmusic mashup creation. Our main contribution is a harmonic compat-ibility metric between musical audio samples which combines existingperceptual relatedness (i.e., chroma vectors or key a�nity) and conso-nance approaches. Our harmonic compatibility metric is derived fromTonal Interval Space indicators, which we adapt to robustly describe theharmonic content of musical audio. Additional attributes of key, rhyth-mic (density), and spectral content are computed from musical audioto enhance the compatibility representation of a sample collection in aninteractive visualization. An evaluation of our harmonic compatibilitymethod shows that it complies with the principles embodied in Westerntonal harmony to a greater extent than previous approaches.

Keywords: music mashup; digital DJ interfaces; audio content analysis;music information retrieval.

1 Introduction

Mashup creation is a music composition practice strongly linked to the varioussub-genres of Electronic Dance Music (EDM) and the role of the DJ [22]. Itentails the recombination of existing (pre-recorded) musical audio as a meansfor creative endeavor [22]. As such, it can be seen as a byproduct of the ex-isting mass preservation mechanisms and inscribed within the artistic view ofthe database as a symbol of postmodern culture [15]. Urged by the need toretrieve and manipulate musical audio from the ever-growing banks of digitalmusic, mashup creation is typically confined to technology-fluent composers, asit requires expertise which extends from the understanding of musical structureto the navigation and retrieval of musical audio from large databases. Both in-dustry and academia have been devoting e↵orts to enhance the experience ofdigital tools for mashup creation by streamlining the time-consuming search forcompatible musical audio.

Early research on computational mashup creation, focused on rhythmic-onlyfeatures, particularly those relevant to the temporal alignment of two or moremusical tracks [10]. Recent research on this topic has expanded the range of mu-sical attributes under consideration, notably including harmonic-driven features

Proc. of the 13th International Symposium on CMMR, Matosinhos, Portugal, Sept. 25-28, 2017

104

2 Bernardes, Davies and Guedes

to identify compatible musical audio, commonly referred to as harmonic mixing.We can identify three major harmonic mixing methods: key a�nity, chroma vec-tors similarity, and (minimal) psychoacoustic dissonance. The a�nity betweenmusical keys is a prominent method in commercial applications which favors rela-tive major-minor and intervals of fifth relations across musical keys [16]. Chromavector similarity inspects the (cosine) distance between chroma vectors repre-sentations of pitch shifted versions of two given audio samples as a measure oftheir compatibility [6, 7, 14]. Psychoacoustic dissonance models have been usedto search for pitch shifted versions of overlapping musical audio which minimizetheir combined level of sensory roughness [9].

While these approaches have shown to correlate well with user enjoyment,we argue that they misrepresent some aspects of harmonic compatibility. First,all possible overlaps between in-key (or diatonic) pitch configurations result inhighly contrasting sonorities with significant levels of enjoyment [2], thus moti-vating a harmonic compatibility metric below the key level. Second, while chromavector distances are e↵ective in capturing highly similar matches between anytwo given audio samples, they lack a perceptually-aware basis for comparingpitch configurations [1], and can thus fail to provide an e↵ective ranking be-tween musical audio collections. Third, while psychoacoustic models show en-hanced performance over remaining approaches, they not only prove to be oflimited use when the spectral content of the samples do not overlap (or no inter-action exists within each critical band), but also violate the harmonic principlesembodied in Western music [12].

In light of these limitations, we propose a new method for computing the har-monic compatibility between a beat-matched collection of audio samples basedon two indicators from the perceptually-motivated Tonal Interval Space: percep-tual relatedness and consonance [1]. The proposed method, not only synthesizesthe two latter approaches on a perceptually-relevant space, but also providesin-key harmonic compatibility metrics, irrespective of the spectral region eachfile occupies. Moreover, our method considers two additional dimensions thatcan help users defining compositional goals in terms of rhythmic (onset density)and spectral (region) content. A prototype created in Pure Data [19] providesan interactive visualization of the compatibility attributes, which allow users tointuitivly navigate through musical audio collections.

Fig. 1 shows the component modules of our harmonic mixing method whichwill be detailed in the remainder of this paper as follows. Sec. 2 reviews theTonal Interval Space, which we adapt towards an enhanced representation ofthe harmonic content of musical audio. Sec. 3 presents content-driven harmonic,rhythmic, and spectral analysis of musical audio. Sec. 4 introduces a novel met-ric for computing the harmonic compatibility between audio samples. Sec. 5details an interactive visualization which exposes the compatibility of an audiosample collection. Sec. 6 presents an evaluation of the harmonic compatibilityindicator, which underpins the harmonic mixing method. Finally, Sec. 7 presentsconclusions and areas for future work.


105

A Perceptually-Motivated Harmonic Compatibility Method 3

Fig. 1. Diagram of the component modules of our compatibility method for musicmixing. Arrows indicate the data flux.

2 Adapting Tonal Interval Vectors for Musical Audio

We represent the harmonic content of musical audio samples as 12-dimensionalTonal Interval Vectors (TIVs) [1]. This vector space creates an extended repre-sentation of tonal pitch in the context of the Tonnetz [8], named Tonal Inter-val Space, where the most salient pitch levels of tonal Western music—pitches,chords, and keys—exist as unique locations. TIVs, T (k), are computed from anaudio signal as the weighted Discrete Fourier Transform (DFT) of a L1 normal-ized chroma vector, c(n), such that:

T (k) = wa(k)

N�1X

n=0

c(n)e�j2⇡kn

N , k 2 Z . (1)

where N = 12 is the dimension of the chroma vector, each of which expressesthe energy of the 12 pitch classes, and wa(k) are weights derived from empiricalratings of dyads consonance used to adjust the contribution of each dimension k(or interpreted musical interval) of the space, which we detail over the next para-graphs at length. We set k to 1 k 6 for T (k) since the remaining coe�cientsare symmetric. T (k) uses c(n) which is c(n) normalized by the DC component

T (0) =PN�1

n=0 c(n) to allow the representation and comparison of music at dif-ferent hierarchical levels of tonal pitch [1]. To represent variable-length audiosamples, we accumulate chroma vectors, c(n), resulting from ⇡ 372ms analysiswindows with 50% overlap across the sample duration.

Table 1. Composite consonance ratings of dyads consonance [11].

Interval Class m2/M7 M2/m7 m3/M6 M3/m6 P4/P5 TTConsonance -1.428 -.582 .594 .386 1.240 -.453

In [1], we used two complementary sources of empirical data—empirical rat-ings of dyads consonance shown in Table 1 [11] and the ranking order of tri-


106


ads consonance: {maj, min, sus4, dim, aug} [5]—to make the Tonal IntervalSpace perceptually relevant for symbolic music input representations (i.e., bi-nary chroma vectors). Here, we revisit the task to comply with the timbralcomponents of musical audio. Our goal is to find a set of weights, wa(k), whichregulate the importance of the DFT coe�cients k in Eq. 1, so that the spaceconveys a reliable consonance indicator correlated with the aforementioned em-pirical ratings of dyad [11] and triad consonance [5]. The applied method followsthe previously used brute force approach [1], which guaranteed a near optimalresult.

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43

Average Spectrum of Orchestral Instruments

0.0

0.2

0.4

0.6

Harmonic

Rel

ativ

e A

mpl

itude

Fig. 2. Average harmonic spectrum of 1338 tones from orchestral instruments [18].

A major problem in defining a set of weights for robustly representing musi-cal audio in the Tonal Interval Space is the variability of timbre across musicalinstruments and registers. A refined model capable of tracing the idiosyncratictimbral attributes of a particular instrument raises scalability and complexityissues which would defeat the value of the Tonal Interval Space in providinge↵ective and, most importantly, e�cient perceptual indicators of tonal pitch. Tocircumvent these issues, we adopt the 43-partials harmonic spectrum templateshown in Fig. 2 to represent the harmonic content of musical audio. The tem-plate results from averaging 1338 recorded instrument tones from 23 Westernorchestral instruments and can be understood as a time-invariant spectrum ofan “average instrument” [18].

To allow a computationally tractable search for weights to represent musicalaudio in the Tonal Interval Space, we split the task into three steps. In the firststep, we find the weights, wa(k), from all possible 6-element combinations (withrepetition and order relevance), of the set I = {1, 19} 2 Z : I = 2I + 1 (atotal of one million combinations), which maintain in the Tonal Interval Spacethe empirical ranking order of common triads consonance [5]. Following [1], wecompute the consonance of musical audio triads in the Tonal Interval Space asthe norm of TIVs, kT (k)k, which we fully detail in Sec. 3.2. In the second step,from the resulting set of 111 weight vectors which preserve the ranking order ofempirical triads consonance (i.e., a Spearman rank correlation ⇢ = 1), we identifythose which have the highest linear correlation to the empirical dyads consonanceratings shown in Table 1. In the third step, we perform a refined optimization


107


on the two weight vectors ({1, 7, 15, 11, 13, 7} and {3, 7, 15, 11, 13, 7}) with thehighest linear correlation (r = .988), below our minimal interval. To this end,we repeated steps 1 and 2 on 4096 6-element combinations (with repetition andorder relevance) of the {0, .5, 1, 1.5} set, which we add to the two aforementionedweight vectors.

Fig. 3. The set of weights that maximize the linear (r > 0.995) and ranking order(⇢ = 1) correlation of Tonal Interval Space’s musical audio consonance indicator withempirical ratings of dyads and triads consonance, respectively. The bold line corre-sponds to the set of weights w

a

(k) used in Eq. 1 and the dashed line to the weights,w

s

(k), defined in [1] for a symbolic based Tonal Interval Space.

Fig. 3 shows 11 sets of weights, wa(k), which preserve empirical triads(⇢ = 1) and dyads (r > .99) consonance for musical audio. Given the inher-ent similarity in shape of the di↵erent sets of weights and their almost per-fect linear relationship, we do not believe the choice over exactly which set ofweights to be critical. However, we ultimately selected the weights with thegreatest mutual separation between the triads according to consonance, thuswa(k) = {3, 8, 11.5, 11.5, 15, 14.5, 7.5}.

3 Musical Audio Analysis

3.1 Dissonance and Perceptual Relatedness

To provide a mathematical representation of Western tonal harmony perceptionas distances in the Tonal Interval Space, we distorted a DFT space according tothe weights, wa(k), derived from empirical consonance ratings. In light of thisdesign feature and following previous metrics detailed in [1], we can computetwo indicators of consonance, C, and perceptual relatedness, R, from the spaceas distance metrics.

C = kT (k)k , (2)

Cij =kaiTi(k) + ajTj(k)kai + aj

PM=6k=1 w(k)

, (3)


108


and

Ri,j =

vuutMX

k=1

|Ti(k) � Tj(k)|2 . (4)

Eq. 2 computes the norm of a TIV, T (k), which we adopt as a measureof consonance of musical audio. Given that the location of multi-pitch TIVs,is equal to the linear combination of its component pitch classes [1], we cane�ciently compute the consonance of two combined TIVs, Ti(k) and Tj(k), rep-resenting two overlapping audio samples, using Eq. 3, where ai and aj are theamplitudes of Ti(k) and Tj(k). Eq. 4 computes the perceptual relatedness, Ri,j ,as the Euclidean distance between the TIVs, Ti(k) and Tj(k).

3.2 Musical Key

Using the method from [3], we estimate the global key of a musical audio sample,Q, in the Tonal Interval Space by finding the closest key TIV from an input audioTIV, as the minimum Euclidean distance function of an audio input TIV, T (k),from the 12 major and 12 minor key TIVs, Tp(k), such that:

Q = argminp

vuut6X

k=1

|T (k) · ↵ � Tp(k)|2 . (5)

where Tp(k) is derived from a collection of templates (understood here as chromavectors) representing pitch class distributions for each of the 12 major and 12minor keys [21]. When p 6 11, we adopt the major profile and when p > 12, theminor profile. ↵ = 0.35 is a factor which displaces input sample TIVs to balancepredictions across modes [3]. The estimated key, Q, ranges between 0 � 11 formajor keys and 12 � 23 for minor keys, where 0 corresponds to C major, 1 toC# major, and so on through to 23 being B minor.

3.3 Note Onset Density

We compute a note onset density function, D, from the musical audio input asa relevant rhythmic attribute to guide users in the search for samples from acollection which meet particular compositional goals. The task is performed bya threefold approach. First, we compute a spectral flux onset detection function,from a windowed power spectrum representation of the audio signal (window size⇡ 46 ms with 50% overlap), using the timbreID [4] library within Pure Data.Second, we extract the peaks from the function above a user-defined threshold,t, whose temporal location we assume to indicate note onset times. Prior to thepeak detection stage, we apply a bi-directional low-pass IIR filter, with a cuto↵frequency of 5Hz to avoid spurious detections. The note onset density, D, of anaudio sample is then computed as the ratio between the number of onsets andthe duration of the audio file in seconds.


109


3.4 Spectral Region

We use the perceptually motivated Bark frequency scale [23] to represent thespectral content of an input audio signal. From accumulated Bark spectrumrepresentations across an audio file, we extract the centroid as an indicator of itsspectral region, S, using Eq. 6. To compute the Bark spectrum, Bi, we use thetimbreID [4] library within Pure Data, which warps a power spectrum to the 24critical bands of the human auditory system (i.e., Bark bands). In adopting aBark spectral representation, we balance the resolution across the human hearingrange, with increased resolution in the low frequency region.

S =

P19i=1 Bi · i

P19i=1 Bi

. (6)

where Bi is the energy of the bark band i. The S indicator can range from 1 to24 Bark bands.

4 Harmonic Compatibility Measure

The level of harmonic compatibility is expressed as the combination of two har-monic audio indicators from the Tonal Interval Space detailed in Sec. 3.1: per-ceptual relatedness, R, and consonance, C. The first indicator finds sonoritieswhich have a strong perceptual a�nity and thus range from a perfect matchto sonorities with di↵erent timbres and similar pitch content, to an array ofsonorities with increased levels of perceptual distance. We envisage it as an ex-tension of the chroma vector similarity method used as a measure of harmoniccompatibility in prior studies [6, 7, 14].

By assuming that the prevalence of musical elements in tonal Western musicprimarily depends on their consonance and, to account for cases in which thesame-level of perceptual relatedness, R, is exhibited, we merged it with the con-sonance indicator, C, to ensure some preference over more consonance resultingaudio mixes, and disambiguate equally distant perceptual configurations. Theresulting harmonic compatibility, H, between two samples, i and j, is then com-puted as the product of the two indicators:

Hi,j = Ri,j · (1 � Cij) , (7)

where R and C are R and C scaled to the range {0, 1} 2 R to balance theimportance of both indicators in the compatibility metric. In Eq. 7, we take(1 � C) to match the R metric, where small distances values indicate relatedconfigurations.

5 Interactive Visualization

Inspired by [13], we created a prototype in Pure Data which provides an in-teractive visualization of the musical audio analysis detailed in Sec. 3 and 4


110


towards an enhanced search for compatible samples from a musical audio col-lection. Audio samples are represented as regular polygons (see Fig. 4), whosedistance indicate their level of harmonic compatibility, H. The computation ofcoordinates for each musical audio sample from a square matrix of all pairwisesamples distances (i.e., harmonic compatibility) is a classical problem, which canbe solved by a specific class of algorithms, notably including MultidimensionalScaling (MDS). In greater detail, from m musical audio samples in a collection,we compute a Hm · Hm harmonic compatibility square matrix, from which anMDS representation extracts two-dimensional coordinates for each sample. Theresulting representation attempts to preserve the inter-sample harmonic com-patibility with minimal distortion.

Additional musical attributes of each sample are represented by graphicalattributes of the polygons. The number of sides, ranging from three to six, exposethe note onset density, D. The mapping between the note onset density, D, andthe polygons’ sides is done by scaling and rounding the D values to the range{3, 6} 2 Z. The higher the number of onsets, the higher the number of sides ofthe polygon. The spectral region of each sample is represented by the polygons’color, ranging from continuous shades of yellow to red. The mapping betweenthe color scheme and the spectral region (i.e., the centroid of the bark spectrumrepresentation for 24 bands) is linear. Bark band 1 corresponds to yellow, andbark band 24 to red. Between these values, the colors are linearly mixed.

Fig. 4. a) Interactive visualization of the harmonic compatibility (distance) betweenmusical audio samples from a collection. The graphical attributes of the polygons,representing individual samples, show onset density (number of sides) and spectralregion (color). Polygons with thick lines indicate selected files currently playing andthe octagons key locations. b) Ranking order of low to high onset density and spectralregion.


111


The user can interact with the visualization by clicking on the polygons totrigger their playback, thus promoting an intuitive search for compatible sam-ples as well as strategies for serendipity and experimentation, rather than afully automatic method for mashup creation. Users can trigger up to 4 sam-ples simultaneously, whose durations are then time-stretched (while preservingthe original pitch content) to the minimal duration of the active files for syn-chronization. A demo of this interactive visualization can be found online at:http://bit.ly/2pGvAnO.

6 Evaluation

To evaluate our harmonic mixing method, we address three aspects of the sys-tem which underpin the search for harmonically compatible musical samplesat the chordal level—at which most mashup creation exists. First, we demon-strate how the weights, wa(k), implemented as a design feature of the TonalInterval Space to provide a consonance indicator of musical audio, compare toa i) uniformly-weighted space (e.g., wu(k) = {1, 1, 1, 1, 1, 1}); ii) the previouslyproposed weights, ws(k) = {2, 11, 17, 16, 19, 7}, adjusted for symbolic music rep-resentations [1]; and iii) other metrics adopted in computational mashup cre-ation (namely, psychoacoustic models [17, 20, 9]) in measuring the consonanceof common triads. Second, we assess how the perceptual relatedness, R, metriccompares to previous harmonic metrics used in computational mashup creation.Finally, we demonstrate the extent to which the harmonic compatibility met-ric, H, resulting from the combination of the two previous indicators, promotesprinciples embodied in Western tonal harmony. In all experiments, the pitch con-tent of musical audio is represented using the harmonic template of an “averageorchestral instrument” shown in Fig. 2.

In Table 2, we show that the Tonal Interval Space is more consistent in mea-suring the consonance of common triads than psychoacoustic models used inrelated mashup literature [20, 17, 9], as it preserves to a higher degree the empir-ical ranking of triads consonance. Moreover, we demonstrate that the weights,wa(k), computed in Sec. 2 are decisive in capturing the consonance of musicalaudio in the Tonal Interval Space, as both a uniformly-weighted space and thepreviously proposed weights for symbolic music inputs [1] fail at providing aranking of triads consonance from musical audio in line with empirical ratings.

In Table 3, we show the perceptual relatednesses, R, of dyads in the TonalInterval Space to which we compare to dyads distances in the (cosine) chromavector space. The ranking order of dyads perceptual relatedness in the TonalInterval Space is consistent with tonal harmony principles in the sense it pro-motes tertian harmony as a result of having fifths and thirds at a closer distancethan all remaining intervals. The (cosine) distance between pitch class chromavectors, adopted as a harmonic compatibility metric in previous computationalmashup works [7, 6, 14], largely agrees with the dyads perceptual ranking fromthe Tonal Interval Space, with the notable exception of the minor seconds andmajor seventh which are closer in this metric space than the major and minor


112


Table 2. Ranking order of common triads consonance from empirical data [5], psy-choacoustic models [17, 20] and the Tonal Interval Space using uniform weights, w

u

(k),the previously symbolic-adjusted weights, w

s

(k), and the newly proposed weights formusical audio, w

a

(k). 1 corresponds to the most consonant chord and 5 the most dis-sonant.

Empirical PsychoacousticTonal Interval Space

Data ModelsChord Cook et al. Parncutt Sethares

wu

(k) ws

(k) wa

(k)quality [5] [17] [20]major 1 2 2 2 1 1minor 2 3 2 2 1 1sus4 3 – 1 1 2 2dim 4 4 4 3 3 3aug 5 1 5 2 1 4

Table 3. Dyads distance in the Tonal Interval Space–using the perceptual relatedness,R, metric in Eq. 2, and in a (cosine) space between chroma vectors, c(n). The smallerthe distance the more related two dyads are assumed to be. Values are normalized tomaximum distance for enhanced readability.

Distance P1 m2/M7 M2/m7 m3/M6 M3/m6 P4/P5 TTR 0 1 .94 .84 .88 .77 .94

c(n) 0 .91 1 .99 .94 .87 .98

thirds or their complementary minor and major sixth. Consequently, the chromaspace disrupts a preference for tertian harmonies.

In Table 4, we show the six best ranking triads resulting from overlapping alldyad combinations (without repetition and order relevance) of the {1 � 11} 2 Zinteger set to the C or 0 pitch class (i.e., 55 triads) using four metrics: the cosinedistance between chroma vectors, c(n); perceptual relatedness, R; consonance, C;and harmonic compatibility, H. The six best ranking triads in the cosine chromaspace and the perceptual relatedness, R, are aligned with the findings shown inTable 3. The chroma space favors chords including P4/P5 and m2/M7 inter-vals. Conversely, the perceptual relatedness, R, favors chords including P5/P4,m3/M6, and M3/m7 intervals. As such, combining sonorities with small R val-ues, results in extended chords with stacked fifths and thirds. However, while thechords resulting from these vertical aggregates are building blocks of Westerntonal harmony, the best ranked chords result in multiple seventh chords (withomitted notes), and not, in the ideally expected, triads (e.g., major, minor, anddiminished).

When combining the perceptual relatedness, R, and consonance, C, indi-cators, we enforce the preference for common major, minor, and suspendedfourth triads—the most common building blocks of the Western tonal harmony.Nonetheless, the harmonic compatibility, H, ranking in Table 4 ignores thekey level promoting chromaticism (non-diatonic) progressions between neigh-bor sonorities. To account for this hierarchical level of analysis we provide in our


113


Table 4. Ranking order of triads resulting from overlapping all dyad combinations tothe C or 0 pitch class using four metrics: the cosine distance of chroma vectors, c(n),perceptual relatedness, R, Consonance, C, and harmonic compatibility, H. To the pitchclass set of the resulting triads, we include the chord label, whenever unambiguous andcomplete triads are formed. A musical notation along with sounding examples of thetable contents are available at: http://bit.ly/2pGvAnO.

c(n)0, 5, 7 0, 1, 7 0, 5, 11 0, 1, 5 0, 7, 11 0, 4, 7 0, 5, 8C sus4 C F-

R0, 5, 7 0, 3, 5 0, 7, 9 0, 7, 8 0, 4, 5 0, 3, 7 0, 5, 9C sus4 C- F

C0, 3, 8 0, 3, 7 0, 5, 8 0, 4, 7 0, 5, 9 0, 4, 9 0, 5, 7Ab C- F- C A F F sus4

R · C0, 5, 7 0, 3, 7 0, 3, 8 0, 5, 8 0, 5, 9 0, 4, 7 0, 4, 9C sus4 Ab C- F- F C A-

interactive visualization a layer of information which can guide users in selecting(in-key) diatonic mixes.

7 Conclusion and Future Work

In this paper we have presented a novel harmonic compatibility metric as a com-bination of two indicators of perceptual relatedness and consonance computedfrom the Tonal Interval Space [1]. To this end, we adapted the Tonal Inter-val Space to provide a robust indicator of tonal consonance for musical audiobased on a harmonic spectrum template of an “average orchestral instrument.”Our evaluation has shown that the indicators computed from the Tonal Inter-val Space are aligned with principles of the Western tonal music, as they favorthe most common building blocks of tertian harmony with enhanced consonance.An interactive visualization of the harmonic compatibility between musical audiosamples from a collection was presented. It includes an extra layer of informationwhich links audio samples to their estimated key as well as exposes the onsetdensity and spectral region as graphical attributes (i.e., color and shape).

In future work, we plan to assess the perceptual basis of the indicators fromthe Tonal Interval Space using musical audio with di↵erent timbral attributes,as well as understand the relevancy of the proposed method for EDM and thepractice of the DJ in meaningful world-case application scenarios for assistingtheir online and o✏ine creative flow.

Acknowledgments. Project TEC4Growth—Pervasive Intelligence, Enhancers andProofs of Concept with Industrial Impact/NORTE-01-0145-FEDER-000020 is financedby the North Portugal Regional Operational Programme (NORTE 2020), under thePORTUGAL 2020 Partnership Agreement, and through the European Regional Devel-opment Fund (ERDF). This research is also supported by the Portuguese FCT, underthe project IF/01566/2015 and post-doctoral grant SFRH/BPD/109457/2015.


114


References

1. Bernardes, G., Cocharro, D., Caetano, M., Guedes, C., Davies, M.: A multi-leveltonal interval space for modelling pitch relatedness and musical consonance. Jour-nal of New Music Research 45(4), 281–294 (2016)

2. Bernardes, G., Cocharro, D., Guedes, C., Davies, M.E.P.: Harmony generationdriven by a perceptually motivated tonal interval space. ACM Computers in En-tertainment 14(2) (2016)

3. Bernardes, G., Davies, M., Guedes, C.: Audio key estimation with adaptive modebias. In: Proceedings of ICASSP (2017)

4. Brent, W.: A timbre analysis and classification toolkit for pure data. In: Proceed-ings of ICMC (2010)

5. Cook, N.: Harmony, Perspective, and Triadic Cognition. Cambridge UniversityPress (2012)

6. Davies, M., Stark, A., Gouyon, F., Goto, M.: Improvasher: A real-time mashupsystem for live musical input. In: Proceedings of NIME. pp. 541–544 (2014)

7. Davies, M.E.P., Hamel, P., Yoshii, K., Goto, M.: Automashupper: Automatic cre-ation of multi-song music mashups. IEEE Trans. ASLP 22(12), 1726–1737 (2014)

8. Euler, L.: Tentamen novae theoriae musicae. Broude (1968/1739)9. Gebhardt, R., Davies, M., Seeber, B.: Psychoacoustic approaches for harmonic

music mixing. Applied Sciences 6(5) (2016)10. Gri�n, G., Kim, Y., Turnbull, D.: Beat-sync-mash-coder: A web application for

real-time creation of beat-synchronous music mashups. In: Proceedings of ICASSP.pp. 2–5 (2010)

11. Huron, D.: Interval-class content in equally tempered pitch-class sets: Commonscales exhibit optimum tonal consonance. Music Perception 11(3), 289–305 (1994)

12. Johnson-Laird, P.N., Kang, O.E., Leong, Y.C.: On musical dissonance. Music Per-ception 30(1), 19–35 (2012)

13. Jorda, S., Kaltenbrunner, M., Geiger, G., Alonso, M.: The reactable: a tangibletabletop musical instrument and collaborative workbench. In: ACM SIGGRAPH2006 Sketches. p. 91. ACM (2006)

14. Lee, C.L., Lin, Y.T., Yao, Z.R., Lee, F.Y., Wu, J.L.: Automatic mashup creation byconsidering both vertical and horizontal mashabilities. In: Proceedings of ISMIR.pp. 399–405 (2015)

15. Manovich, L.: The Language of New Media. MIT press (2001)16. Mixed in Key: Mashup 2, http://mashup.mixedinkey.com, last accessed on 28

March 201717. Parncutt, R.: Harmony: A Psychoacoustical Approach. Springer (1989)18. Plazak, J., Huron, D., Williams, B.: Fixed average spectra of orchestral instrument

tones. Empirical Musicology Review 5(1) (2010)19. Puckette, M.: Pure data. In: Proceedings of ICMC. pp. 224–227 (1996)20. Sethares, W.: Tuning, Timbre, Spectrum, Scale. Springer-Verlag (1999)21. Sha’ath, I.: Estimation of key in digital music recordings. Master’s thesis, Birkbeck

College, University of London (2011)22. Shiga, J.: Copy-and-persist: The logic of mash-up culture. Critical Studies in Media

Communication 24(2), 93–114 (2007)23. Zwicker, E., Fastl, H.: Psychoacoustics–Facts and Models. Springer Verlag (1990)


115

a perceptually-motivated harmonic compatibility method for

Documents