Automatic Piano Transcription

By Catherine Massie-Laberge MUMT 621

Music Transcription

Music transcription can be described as: “the analysis of an acoustic signal so as to write down the pitch, onset time, duration, and source of each sound that occurs in it.” (Klapuri 2006, p. 3).

!2

Applications

• MIR

• Musicology

• Music processing

• Amateur musicians

!3

Techniques• Comb filters and autocorrelation

• Blackboard system

• Signal processing

• Probabilistic models

• Neural networks

!4

Comb filtering & Autocorrelation

• Moorer (1975): first experiments with transcription of melodic lines of two voices

• Comb filtering and autocorrelation methods

• Limited to two instruments of different timbres and frequency ranges

• Limited to a two octaves range

!5

Blackboard system• Martin (1996): Blackboard system to incorporate

various forms of knowledge

• Piano performances of Bach’s chorales

• Data space: hierarchically structured with inputs (lowest level) and a solution (highest level); scheduler

• Limited to Bach’s chorales (notes have long duration)

!6

Control structure

!7 (Martin 1996)

Raw audio signal

Bridge the gap between tracks and notes

Semitones distance

Interval hypothesis

Pitch and three component intervals

Blackboard & NNs

• Bello and Sandler (2000): Blackboard and NN that interacts with it

• Output can adjust its hypothesis

• Limited with octave transcriptions

!8

Signal processing-based techniques

• Dixon (2000): STFT from which peaks in the time-frequency domain are extracted

• Kelly and Walker (2007): Masking approach

!9

Probabilistic models• Raphael (2002): Hidden Markov Models trained

according to a likelihood model that generates hypotheses

• Results derived from the 3rd movement of Mozart’s Sonata 18, K. 570

• Restricted to notes that felt in the range of the movement

!10

Neural Networks• Marolt (1999, 2004, 2005): NN to determine the

notes and their starting times

• Partial tracking model:

-Auditory model: Reproduces the characteristics of the human hearing system

-Adaptive oscillators extract partials from outputs of the auditory model

!11

Auditory system

!12

(Moore 2003)

Cross section of the cochlea

!13

(Marolt 2001)

Network of adaptive oscillators

!14(Marolt 2004)

Note recognition • 76 NNs to perform note recognition

• Each network recognizes one note, so only one output

• Trained on a database of ~120 synthesized piano pieces

• Inputs of each network from output of the partial tracking module

• Time-delay NNs

!15

SONIC

• Partial tracking model and neural networks are incorporated into SONIC

• Includes an onset detector, module for detecting repeated notes, an algorithm for length and loudness

!16

ExamplesCompare:

• Synthesis recordings

J.S. Bach, Partita no. 4 in D major, BWV828

S. Joplin and S. Hayden, Kismet Rag

• Real recordings

J.S. Bach, Goldberg variations, BWV988, variation no. 10, performer Andrei Gavrilov, Deutsche Grammophon 463 019-2

M. Dixon, R. Henderson, Bye Bye Blackbird (intro), performer Oscar Peterson, MPS 821 843-2

!17

Conclusion• Major limitations:

Octave errors

Missed onset (or added extra onset)

repeated notes

rapid succession of notes (trills, arpeggios, ornaments)

Low pitch notes (less than 200 Hz)

• Need to consider other perceptual nuances (e.g., timbre)

• Moving toward a human-like approach that learns through examples and adapts to various sources

!18

BibliographyBello, J. P., G. Monti, and M. Sandler. 2000. Techniques for automatic music transcription. In Proceedings of the First International Conference on Music Information Retrieval (ISMIR), Plymouth, MA.! !Bello, J. P., and M. Sandler. 2000. Blackboard system and top-down processing for the transcription of simple polyphonic music. In Proceedings of the COST G-6 Conference on Digital Audio Effects (DAFX-00), Verona, Italy. !!Dixon, S. 2000. On the computer recognition of solo piano music. In Proceedings of the Australasian Computer Music Association Conference, Brisbane, Australia. !!Kelly, R., and J. Walker. 2007. Automatic transcription of polyphonic piano music using a note masking technique. In Proceedings of the International Conference of Music Communication Science, Sydney, Australia. !!Klapuri, A., and M. Davy. 2006. Signal processing methods for music transcription. New York, NY: Springer.!!Klapuri, A, T. Virtanen, A. Eronen, and J. Seppanen. 2001. Automatic transcription of musical recordings. In Proceedings of the Consistent & Reliable Cues Workshop (CRAC-01), Aalborg, Denmark. !!Marolt, M. 2005. A connectionist model of finding partial groups in music recordings with application to music transcription. In Proceedings of the Seventh International Conference on Adaptive and Natural Computing Algorithms, Coimbra, Portugal. !

!19

!Marolt, M. 2004. A connectionist approach to automatic transcription of polyphonic piano music. IEEE Transactions on Multimedia 6 (3): 494-95. !!Marolt, M. 1999. A comparison of feed forward neural network architectures for piano music transcription. Proceedings of the 1999 International Computer Music Conference, Beijing, China. !!Martin, K. D. 1996. A blackboard system for automatic transcription of simple polyphonic music. MIT Media Laboratory Perceptual Computing Section Technical Report No. 385. !!Moorer, J. A. 1977. On transcription of musical sound by computer. Computer Music Journal 1(4): 32-8.!!Poliner, G. E., and D., P. W. Ellis. 2006. A discriminative model for polyphonic piano transcription. EURASIP Journal on Advances in Signal Processing, New York, NY. !!Poliner, G. E., and D., P. W. Ellis. 2005. A classification approach to melody transcription. In Proceedings of the Sixth Annual Conference on Music Information Retrieval (ISMIR), London, UK. !!Raphael, C. 2002. Automatic transcription of piano music. In Proceedings of the Third International Conference on Music Information Retrieval (ISMIR), Paris, France.

!20