Tracking microtiming

variations in the course of a

jazz performance

Christian Dittmar, Martin Pfleiderer, Meinard Müller

christian.dittmar@audiolabs-erlangen.de

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

2

Introduction

Title: Anthropology

Album: The Compelete Galaxy Recordings (1979)

Lineup:

Charlie Haden on bass

Art Pepper playing solo clarinet

Billy Higgins on drums

Music:

B♭-Major

4/4 Signature

~219 BPM

Source: wikipedia

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

3

Typical ride cymbal (RC) pattern

Downbeat

⋮

Backbeat

Offbeat

Downbeat

⋮

Keep tempo

Induce „swing feel“

Introduction

Source: wikipedia

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

4

Introduction: Swing Ratio

𝑠r =0.125

0.125≈ 1

Sig

nal W

ave

form

/

Am

plit

ud

e E

nve

lop

e

Ride

Cymbal

Hi-hat

do

wn

be

at

backb

eat

off

bea

t

do

wn

be

at

backb

eat

off

bea

t Time (Seconds)

240 BPM

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

5

𝑠r =0.167

0.083≈ 2

Introduction: Swing Ratio

Sig

nal W

ave

form

/

Am

plit

ud

e E

nve

lop

e

off

bea

t

do

wn

be

at

backb

eat

do

wn

be

at

backb

eat

Ride

Cymbal

Hi-hat

off

bea

t Time (Seconds)

3 3

240 BPM

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

6

𝑠r =0.188

0.063≈ 3

Introduction: Swing Ratio

Sig

nal W

ave

form

/

Am

plit

ud

e E

nve

lop

e

off

bea

t

do

wn

be

at

backb

eat

do

wn

be

at

backb

eat

Ride

Cymbal

Hi-hat

off

bea

t Time (Seconds)

240 BPM

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

7

Related Work: Jazz Microtiming Analysis

Early investigations into swing ratio:

Kerschbaumer 1978, Reinholdsson 1987, Rose 1989

Manually marked onset times

Measurements using MIDI-fied instruments:

Ellis 1991, Busse 2003

Negative correlation between tempo and swing ratio:

Friberg & Sundström 2002 inspiration for our work

Dittmar et al. 2015 (semi-) automatic swing ratio estimation

No strong correlation, preferred swing ratio ~2.0:

Honing & de Haas 2008 focus on jazz drummers

Marchand & Peeters 2015 swing ratio in diverse genres (GTZAN)

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

8

Swing Ratio Estimation via Log-Lag ACF

Tempo-normalized spectral rhythm patterns

Peeters 2005

Scale transform for rhythmic similarity

Holzapfel & Stylianou 2009, 2011

Marchand & Peeters 2014

Prockup et al. 2015

Log-Lag ACF (LLACF)

Gruhne & Dittmar 2009

Völkl et al. 2010

Dittmar et al. 2015

Autocorrelation (ACF) computed from

onset detection function

Resampled to logarithmically-spaced lag-

axis (resp. tempo-axis)

Rhythmic patterns salient peaks at

distinct periodicities

Tempo factors log-lag offsets

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

9

Swing Ratio Estimation via Log-Lag ACF

𝑠r =0.125

0.125≈ 1

Ride

Cymbal

Hi-hat

do

wn

be

at

backb

eat

off

bea

t

do

wn

be

at

backb

eat

off

bea

t

Pe

rio

dic

ity

Sa

lien

ce

Am

plit

ud

e

En

ve

lop

e

Tempo (BPM)

240 BPM

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

10

𝑠r =0.167

0.083≈ 2

Swing Ratio Estimation via Log-Lag ACF

off

bea

t

do

wn

be

at

backb

eat

do

wn

be

at

backb

eat

Ride

Cymbal

Hi-hat

off

bea

t

Pe

rio

dic

ity

Sa

lien

ce

Am

plit

ud

e

En

ve

lop

e

Tempo (BPM)

3 3

240 BPM

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

11

𝑠r =0.188

0.063≈ 3

Swing Ratio Estimation via Log-Lag ACF

off

bea

t

do

wn

be

at

backb

eat

do

wn

be

at

backb

eat

Ride

Cymbal

Hi-hat

off

bea

t

Pe

rio

dic

ity

Sa

lien

ce

Am

plit

ud

e

En

ve

lop

e

Tempo (BPM)

240 BPM

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

12

Swing Ratio Estimation via Log-Lag ACF

Pattern matching against prototype LLACFs S

win

g r

atio 𝑠

r

Tempo (BPM)

Periodic

ity

Salie

nce

960 480 240 120 601

2

3

4

960 480 240 120 600

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

13

Swing Ratio Estimation via Log-Lag ACF

Pattern matching against prototype LLACFs S

win

g r

atio 𝑠

r

Tempo (BPM)

Periodic

ity

Salie

nce

960 480 240 120 601

2

3

4

960 480 240 120 600

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

14

Swing Ratio Estimation via Log-Lag ACF

Pattern matching against prototype LLACFs S

win

g r

atio 𝑠

r

Tempo (BPM)

Periodic

ity

Salie

nce

960 480 240 120 601

2

3

4

960 480 240 120 600

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

15

Swing Ratio Estimation via Log-Lag ACF:

Friberg & Sundström 2002: Dittmar et al. 2015:

50 100 150 200 250 300 3500.5

1

1.5

2

2.5

3

3.5

4

Sw

ing

ra

tio

𝑠r

50 100 150 200 250 300 3500.5

1

1.5

2

2.5

3

3.5

4

50 100 150 200 250 300 350

1

2

3

4

Art Taylor

Connie Kay

Billy Higgins

Elvin Jones

Tony Williams

Carl Allen

Jeffrey Watts

Jack DeJohnette

Max Roach

Dennis Chambers

Tempo (BPM)

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

16

Tracking Swing Ratio Variations

Weimar Jazz Database:

Soloist swing ratio implicitely given via tone onsets

Automatically estimated swing ratios of RC patterns

Investigations into the interactive art of improvising together

New idea:

Compute segment-wise LLACF accross complete solo

Store matching score for all swing ratios per segment

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

17

10 15 20 25 30 35 40 451

1.5

2

2.5

3

3.5

4

0.82

0.84

0.86

0.88

0.9

0.92

0.94

Tracking Swing Ratio Variations S

win

g r

atio

𝑠r

Time (s)

0.78 0.8 0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.961

1.5

2

2.5

3

3.5

4

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

18

10 15 20 25 30 35 40 451

1.5

2

2.5

3

3.5

4

0.82

0.84

0.86

0.88

0.9

0.92

0.94

Tracking Swing Ratio Variations S

win

g r

atio

𝑠r

Time (s)

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

19

10 15 20 25 30 35 40 451

1.5

2

2.5

3

3.5

4

0.82

0.84

0.86

0.88

0.9

0.92

0.94

Tracking Swing Ratio Variations S

win

g r

atio

𝑠r

Time (s)

Track salient trajectory:

Dynamic Programming (DP) finds optimal paths

Can be parametrized to prevent unreasonable jumps

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

20

Tracking Swing Ratio Variations

Correspondence to ground-truth swing ratio annotations:

Ground-truth based on RC onset transcription

Ground-truth exhibits higher variability

10 15 20 25 30 35 40 451

1.5

2

2.5

3

3.5

4

0.82

0.84

0.86

0.88

0.9

0.92

0.94

Sw

ing

ra

tio

𝑠r

Time (s)

10 15 20 25 30 35 40 45

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

21

Tracking Swing Ratio Variations

Comparison to soloist swing ratio:

Very diverse relation not always lower swing ratio

Selection of swing candidates might need improvement

10 15 20 25 30 35 40 451

1.5

2

2.5

3

3.5

4

0.82

0.84

0.86

0.88

0.9

0.92

0.94

Sw

ing

ra

tio

𝑠r

Time (s)

MID

I p

itch

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

22

Conclusions

Proposed new method to track swing ratio variations

Qualitatively similar results as provided by ground truth

Comparison to soloist swing ratio still has open issues

Next steps?

Which structures are interesting?

Relation to phrases?

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

23

Book: Fundamentals of Music Processing

Meinard Müller

Fundamentals of Music Processing

Audio, Analysis, Algorithms, Applications

483 p., 249 illus., hardcover

ISBN: 978-3-319-21944-8

Springer, 2015

Accompanying website:

www.music-processing.de

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

24

Book: Fundamentals of Music Processing

Meinard Müller

Fundamentals of Music Processing

Audio, Analysis, Algorithms, Applications

483 p., 249 illus., hardcover

ISBN: 978-3-319-21944-8

Springer, 2015

Accompanying website:

www.music-processing.de

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

25

AES Conference: Semantic Audio 2017

Paper deadline: 22.01.2017

© AudioLabs, 2016

Christian Dittmar

Tracking microtiming variations in the course of a jazz performance

26

AES Conference: Semantic Audio 2017

Ready my

carriage! I shall

journey to AES in

Erlangen!