2018 mueller mp-beattracking · 2018-06-30 · por una cabeza donau. beat and tempo steady pulse...
TRANSCRIPT
![Page 1: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/1.jpg)
Music Processing
Meinard Müller
Lecture
Tempo and Beat Tracking
International Audio Laboratories [email protected]
![Page 2: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/2.jpg)
Book: Fundamentals of Music Processing
Meinard MüllerFundamentals of Music ProcessingAudio, Analysis, Algorithms, Applications483 p., 249 illus., hardcoverISBN: 978-3-319-21944-8Springer, 2015
Accompanying website: www.music-processing.de
![Page 3: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/3.jpg)
Book: Fundamentals of Music Processing
Meinard MüllerFundamentals of Music ProcessingAudio, Analysis, Algorithms, Applications483 p., 249 illus., hardcoverISBN: 978-3-319-21944-8Springer, 2015
Accompanying website: www.music-processing.de
![Page 4: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/4.jpg)
Book: Fundamentals of Music Processing
Meinard MüllerFundamentals of Music ProcessingAudio, Analysis, Algorithms, Applications483 p., 249 illus., hardcoverISBN: 978-3-319-21944-8Springer, 2015
Accompanying website: www.music-processing.de
![Page 5: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/5.jpg)
Chapter 6: Tempo and Beat Tracking
Tempo and beat are further fundamental properties of music. In Chapter 6, weintroduce the basic ideas on how to extract tempo-related information fromaudio recordings. In this scenario, a first challenge is to locate note onsetinformation—a task that requires methods for detecting changes in energy andspectral content. To derive tempo and beat information, note onset candidatesare then analyzed with regard to quasiperiodic patterns. This leads us to thestudy of general methods for local periodicity analysis of time series.
6.1 Onset Detection6.2 Tempo Analysis6.3 Beat and Pulse Tracking6.4 Further Notes
![Page 6: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/6.jpg)
Introduction
Basic beat tracking task:
Given an audio recording of a piece of music, determine the periodic sequence of beat positions.
“Tapping the foot when listening to music’’
![Page 7: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/7.jpg)
Time (seconds)
Example: Queen – Another One Bites The Dust
Introduction
![Page 8: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/8.jpg)
Example: Queen – Another One Bites The Dust
Introduction
Time (seconds)
![Page 9: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/9.jpg)
Introduction
Example: Happy Birthday to you
Pulse level: Measure
![Page 10: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/10.jpg)
Introduction
Example: Happy Birthday to you
Pulse level: Tactus (beat)
![Page 11: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/11.jpg)
Introduction
Example: Happy Birthday to you
Pulse level: Tatum (temporal atom)
![Page 12: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/12.jpg)
Example: Chopin – Mazurka Op. 68-3
Pulse level: Quarter note
Tempo: ???
Introduction
![Page 13: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/13.jpg)
Example: Chopin – Mazurka Op. 68-3
Pulse level: Quarter note
Tempo: 50-200 BPM
Time (beats)
Tem
po (B
PM)
50
200
Tempo curve
Introduction
![Page 14: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/14.jpg)
Introduction
Example: Borodin – String Quartet No. 2
Pulse level: Quarter note
Tempo: 120-140 BPM (roughly)
Beat tracker without any prior knowledge
Beat tracker with prior knowledge on rough tempo range
![Page 15: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/15.jpg)
Introduction
Pulse level often unclear
Local/sudden tempo changes (e.g. rubato)
Vague information
(e.g., soft onsets, extracted onsets corrupt)
Sparse information
(often only note onsets are used)
Challenges in beat tracking
![Page 16: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/16.jpg)
Onset detection Beat tracking Tempo estimation
Tasks
Introduction
![Page 17: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/17.jpg)
Onset detection Beat tracking Tempo estimation
Tasks
Introduction
![Page 18: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/18.jpg)
periodphase
Onset detection Beat tracking Tempo estimation
Tasks
Introduction
![Page 19: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/19.jpg)
Tempo := 60 / period
Beats per minute (BPM)
Onset detection Beat tracking Tempo estimation
Tasks
period
Introduction
![Page 20: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/20.jpg)
Onset Detection
Finding start times of perceptually relevant acoustic events in music signal
Onset is the time position where a note is played
Onset typically goes along with a change of the signal’s properties:– energy or loudness– pitch or harmony– timbre
![Page 21: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/21.jpg)
Onset Detection
Finding start times of perceptually relevant acoustic events in music signal
Onset is the time position where a note is played
Onset typically goes along with a change of the signal’s properties:– energy or loudness– pitch or harmony– timbre TransientOnset
Attack
Decay
![Page 22: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/22.jpg)
Steps
Time (seconds)
Onset Detection (Energy-Based)
Waveform
![Page 23: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/23.jpg)
Onset Detection (Energy-Based)
Time (seconds)
Squared waveform
Steps1. Amplitude squaring
![Page 24: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/24.jpg)
Onset Detection (Energy-Based)
Time (seconds)
Energy envelope
Steps1. Amplitude squaring2. Windowing
![Page 25: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/25.jpg)
Onset Detection (Energy-Based)
Capturing energy changes
Time (seconds)
Differentiated energy envelope
Steps1. Amplitude squaring2. Windowing3. Differentiation
![Page 26: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/26.jpg)
Onset Detection (Energy-Based)
Time (seconds)
Novelty curve
Steps1. Amplitude squaring2. Windowing3. Differentiation4. Half wave rectification
Only energy increases are relevant for note onsets
![Page 27: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/27.jpg)
Onset Detection (Energy-Based)
Time (seconds)
Steps1. Amplitude squaring2. Windowing3. Differentiation4. Half wave rectification5. Peak picking
Peak positions indicate note onset candidates
![Page 28: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/28.jpg)
Onset Detection (Energy-Based)Example: C4 played by piano
Time (seconds)
![Page 29: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/29.jpg)
Onset Detection (Energy-Based)Example: C4 played by violin
Time (seconds)
![Page 30: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/30.jpg)
Onset Detection (Energy-Based)Example: C4 played by flute
Time (seconds)
![Page 31: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/31.jpg)
Onset Detection
Energy curves often only work for percussive music
Many instruments such as strings have weak note onsets
No energy increase may be observable in complex sound mixtures
More refined methods needed that capture– changes of spectral content– changes of pitch– changes of harmony
![Page 32: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/32.jpg)
Onset Detection (Spectral-Based)
Time (seconds)
Audio recording
![Page 33: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/33.jpg)
1. SpectrogramMagnitude spectrogram || X Steps:
Onset Detection (Spectral-Based)Fr
eque
ncy
(Hz)
Time (seconds)
![Page 34: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/34.jpg)
Compressed spectrogram Y
|)|1log( XCY
Onset Detection (Spectral-Based)
1. Spectrogram2. Logarithmic compression
Steps:
Freq
uenc
y (H
z)
Time (seconds)
![Page 35: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/35.jpg)
Spectral difference
Onset Detection (Spectral-Based)
1. Spectrogram2. Logarithmic compression3. Differentiation &
half wave rectification
Steps:
Freq
uenc
y (H
z)
Time (seconds)
![Page 36: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/36.jpg)
Spectral difference
Novelty curve
Onset Detection (Spectral-Based)
1. Spectrogram2. Logarithmic compression3. Differentiation &
half wave rectification4. Accumulation
Steps:
Time (seconds)
Freq
uenc
y (H
z)
Time (seconds)
0 1 2 3 4 5 60
20
40
60
![Page 37: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/37.jpg)
Onset Detection (Spectral-Based)
1. Spectrogram2. Logarithmic compression3. Differentiation &
half wave rectification4. Accumulation
Steps:
Novelty curve
![Page 38: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/38.jpg)
Substraction of local average
Onset Detection (Spectral-Based)
1. Spectrogram2. Logarithmic compression3. Differentiation &
half wave rectification4. Accumulation5. Normalization
Steps:
Novelty curve
![Page 39: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/39.jpg)
Onset Detection (Spectral-Based)
1. Spectrogram2. Logarithmic compression3. Differentiation &
half wave rectification4. Accumulation5. Normalization
Steps:
Normalized novelty curve
![Page 40: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/40.jpg)
Onset Detection (Spectral-Based)
Spectrogram
Compressed Spectrogram
Novelty curve
![Page 41: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/41.jpg)
Logarithmic Compression
Time (seconds) Time (seconds) Time (seconds)
Freq
uenc
y(H
z)
|)|1log( XCY
No compression C = 1 C = 100
![Page 42: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/42.jpg)
Onset Detection
Time (seconds)
Energy-based novelty curve
Spectral-based novelty curve
Manuel onset annotations
![Page 43: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/43.jpg)
Onset DetectionShostakovich – 2nd Waltz
Time (seconds)
Borodin – String Quartet No. 2
Beethoven – Fifth Symphony
![Page 44: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/44.jpg)
Onset Detection
Drumbeat
Going Home
Lyphard melodie
Por una cabeza
Donau
![Page 45: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/45.jpg)
Beat and Tempo
Steady pulse that drives music forward and provides the temporal framework of a piece of music
Sequence of perceived pulses that are equally spaced in time
The pulse a human taps along when listening to the music
[Parncutt 1994]
[Sethares 2007]
[Large/Palmer 2002]
[Lerdahl/ Jackendoff 1983]
[Fitch/ Rosenfeld 2007]
What is a beat?
The term tempo then refers to the speed of the pulse.
![Page 46: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/46.jpg)
Beat and Tempo
Analyze the novelty curve with respect to reoccurring or quasi-periodic patterns
Avoid the explicit determination of note onsets (no peak picking)
Strategy
![Page 47: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/47.jpg)
Beat and Tempo
[Scheirer, JASA 1998]
[Ellis, JNMR 2007]
[Davies/Plumbley, IEEE-TASLP 2007]
[Peeters, JASP 2007]
Strategy
Comb-filter methods Autocorrelation Fourier transfrom
Methods
[Grosche/Müller, ISMIR 2009]
Analyze the novelty curve with respect to reoccurring or quasi-periodic patterns
Avoid the explicit determination of note onsets (no peak picking)
[Grosche/Müller, IEEE-TASLP 2011]
![Page 48: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/48.jpg)
Definition: A tempogram is a time-tempo representation that encodes the local tempo of a music signalover time.
Tem
po (B
PM)
Time (seconds)
Inte
nsity
Tempogram
![Page 49: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/49.jpg)
Definition: A tempogram is a time-tempo represenationthat encodes the local tempo of a music signalover time.
Compute a spectrogram (STFT) of the novelty curve Convert frequency axis (given in Hertz) into
tempo axis (given in BPM) Magnitude spectrogram indicates local tempo
Fourier-based method
Tempogram (Fourier)
![Page 50: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/50.jpg)
Tem
po (B
PM)
Time (seconds)
Tempogram (Fourier)
Novelty curve
![Page 51: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/51.jpg)
Tem
po (B
PM)
Tempogram (Fourier)
Novelty curve (local section)
Time (seconds)
![Page 52: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/52.jpg)
Tem
po (B
PM)
Windowed sinusoidal
Tempogram (Fourier)
Time (seconds)
![Page 53: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/53.jpg)
Tem
po (B
PM)
Windowed sinusoidal
Tempogram (Fourier)
Time (seconds)
![Page 54: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/54.jpg)
Tem
po (B
PM)
Tempogram (Fourier)
Windowed sinusoidal
Time (seconds)
![Page 55: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/55.jpg)
Definition: A tempogram is a time-tempo represenationthat encodes the local tempo of a music signalover time.
Compare novelty curve with time-lagged local sections of itself
Convert lag-axis (given in seconds) into tempo axis (given in BPM)
Autocorrelogram indicates local tempo
Autocorrelation-based method
Tempogram (Autocorrelation)
![Page 56: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/56.jpg)
Tempogram (Autocorrelation)
Novelty curve (local section)
Lag
(sec
onds
)
Time (seconds)
![Page 57: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/57.jpg)
Tempogram (Autocorrelation)
Windowed autocorrelation
Lag
(sec
onds
)
![Page 58: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/58.jpg)
Tempogram (Autocorrelation)
Lag = 0 (seconds)
Lag
(sec
onds
)
![Page 59: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/59.jpg)
Tempogram (Autocorrelation)
Lag = 0.26 (seconds)
Lag
(sec
onds
)
![Page 60: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/60.jpg)
Tempogram (Autocorrelation)
Lag = 0.52 (seconds)
Lag
(sec
onds
)
![Page 61: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/61.jpg)
Tempogram (Autocorrelation)
Lag = 0.78 (seconds)
Lag
(sec
onds
)
![Page 62: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/62.jpg)
Tempogram (Autocorrelation)
Lag = 1.56 (seconds)
Lag
(sec
onds
)
![Page 63: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/63.jpg)
Tempogram (Autocorrelation)
Time (seconds)
Time (seconds)
Lag
(sec
onds
)
![Page 64: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/64.jpg)
300
60
80
40
30
120
Tempogram (Autocorrelation)Te
mpo
(BPM
)
Time (seconds)
Time (seconds)
![Page 65: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/65.jpg)
600
500
400
300
200
100
Tempogram (Autocorrelation)Te
mpo
(BPM
)
Time (seconds)
Time (seconds)
![Page 66: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/66.jpg)
Time (seconds)
TempogramFourier Autocorrelation
Time (seconds)
Tem
po (B
PM)
![Page 67: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/67.jpg)
TempogramFourier Autocorrelation
210
70
Tem
po (B
PM)
Tempo@Tatum = 210 BPM Tempo@Measure = 70 BPMTime (seconds) Time (seconds)
![Page 68: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/68.jpg)
Tempogram
Fourier Autocorrelation
Time (seconds) Time (seconds)
Tem
po (B
PM)
Time (seconds)
Emphasis of tempo harmonics(integer multiples)
Emphasis of tempo subharmonics (integer fractions)
![Page 69: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/69.jpg)
Tempogram (Summary)
Fourier Autocorrelation
Novelty curve is compared withsinusoidal kernels eachrepresenting a specific tempo
Novelty curve is compared withtime-lagged local (windowed) sections of itself
Convert frequency (Hertz) intotempo (BPM)
Convert time-lag (seconds) intotempo (BPM)
Reveals novelty periodicities Reveals novelty self-similarities
Emphasizes harmonics Emphasizes subharmonics
Suitable to analyze tempo on tatum and tactus level
Suitable to analyze tempo on tactus and measure level
![Page 70: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/70.jpg)
Beat Tracking
Given the tempo, find the best sequence of beats
Complex Fourier tempogram contains magnitude and phase information
The magnitude encodes how well the novelty curve resonates with a sinusoidal kernel of a specific tempo
The phase optimally aligns the sinusoidal kernel with the peaks of the novelty curve
![Page 71: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/71.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
![Page 72: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/72.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
Optimizing local periodicity kernel
![Page 73: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/73.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
Optimizing local periodicity kernel
![Page 74: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/74.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
Optimizing local periodicity kernel
![Page 75: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/75.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
Optimizing local periodicity kernelHalfwave rectificationAccumulation of kernels
Time (seconds)
![Page 76: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/76.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
Optimizing local periodicity kernelHalfwave rectification
![Page 77: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/77.jpg)
Local Pulse Tracking
Novelty Curve
Predominant Local Pulse (PLP)
Time (seconds)
Time (seconds)
![Page 78: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/78.jpg)
Periodicity enhancement of novelty curve Accumulation introduces error robustness Locality of kernels handles tempo variations
Indicates note onset candidates Extraction errors in particular for soft onsets Simple peak-picking problematic
Local Pulse Tracking
Predominant Local Pulse (PLP)
Novelty Curve
![Page 79: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/79.jpg)
Local Pulse Tracking
Local tempo at time : [60:240] BPM
Phase
Sinusoidal kernel
Periodicity curve
![Page 80: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/80.jpg)
Local Pulse Tracking
Novelty Curve
Predominant Local Pulse (PLP)
Time (seconds)
![Page 81: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/81.jpg)
Local Pulse TrackingTe
mpo
(BPM
)
Time (seconds)
Borodin – String Quartet No. 2
![Page 82: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/82.jpg)
Local Pulse TrackingTe
mpo
(BPM
)Borodin – String Quartet No. 2
Strategy: Exploit additional knowledge(e.g. rough tempo range)
Time (seconds)
![Page 83: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/83.jpg)
Pulse Levels
What is the pulse level: Measure – Tactus – Tatum?
1/8
1/4
1/16
Piano Etude Op. 100 No. 2 by Burgmüller
• • • •
•••••••• •••••••• •••••••• ••••••••
• • • • • • • • • • • • • • • •
![Page 84: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/84.jpg)
Pulse LevelsTe
mpo
(BPM
)
![Page 85: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/85.jpg)
Pulse Levels
Time (seconds)
Switching of predominant pulse level
Tem
po (B
PM)
![Page 86: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/86.jpg)
Pulse Levels
1/4 note pulse level
Time (seconds)
Tem
po (B
PM)
![Page 87: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/87.jpg)
Pulse Levels
1/8 note pulse level
Time (seconds)
Tem
po (B
PM)
![Page 88: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/88.jpg)
Pulse Levels
1/16 note pulse level
Time (seconds)
Tem
po (B
PM)
![Page 89: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/89.jpg)
Local Pulse Tracking
Brahms Hungarian Dance No. 5
Tem
po (B
PM)
Time (seconds)
![Page 90: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/90.jpg)
Local Pulse Tracking
Brahms Hungarian Dance No. 5
Time (seconds)
Tem
po (B
PM)
![Page 91: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/91.jpg)
Applications
Feature design (beat-synchronous features, adaptive windowing)
Digital DJ / audio editing(mixing and blending of audio material)
Music classification
Music recommendation
Performance analysis(extraction of tempo curves)
![Page 92: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/92.jpg)
Application: Feature Design
Time (seconds)
Fixed window size
![Page 93: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/93.jpg)
Application: Feature Design
Time (seconds)
Adaptive window size
![Page 94: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/94.jpg)
Application: Feature Design
Freq
uenc
y(p
itch)
Fixed window size
Time (seconds)
![Page 95: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/95.jpg)
Application: Feature Design
Freq
uenc
y(p
itch)
Time (seconds)
Adaptive window size
![Page 96: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/96.jpg)
Application: Feature Design
Denoising by excluding boundary neighborhoods
Time (seconds)
Freq
uenc
y(p
itch)
Adaptive window size
![Page 97: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/97.jpg)
Application: Audio Editing (Digital DJ)
http://www.mixxx.org/
![Page 98: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/98.jpg)
Application: Beat-Synchronous Light Effects
![Page 99: 2018 Mueller MP-BeatTracking · 2018-06-30 · Por una cabeza Donau. Beat and Tempo Steady pulse that drives music forward and provides the temporal framework of a piece of music](https://reader033.vdocuments.us/reader033/viewer/2022042206/5ea846ec43f1214bff3e95fe/html5/thumbnails/99.jpg)
Summary
1. Onset Detection Novelty curve (something is changing) Indicates note onset candidates Hard task for non-percussive instruments (strings)
2. Tempo Estimation Fourier tempogram Autocorrelation tempogram Musical knowledge (tempo range, continuity)
3. Beat tracking Find most likely beat positions Exploiting phase information from Fourier tempogram