automatic chord recognition using neural networks · automatic chord recognition using neural...
TRANSCRIPT
![Page 1: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/1.jpg)
Automatic Chord Recognition using Neural Networks
Audio Signal Processing
Ayoub Ghriss
MVA, 2017
Computational Statistics MVA, 2017 p 1 / 20
![Page 2: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/2.jpg)
Contents
1 Introduction
2 Features Extraction
3 Features Processing
4 Learning Architecture
5 Conclusion
Computational Statistics MVA, 2017 p 2 / 20
![Page 3: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/3.jpg)
Introduction
Computational Statistics MVA, 2017 p 3 / 20
![Page 4: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/4.jpg)
A bit of music and history
• Automatic Chord Recognition, first start in 1991• Classic methods : KNN, Logistic, HMM• Leap in precision with the introduction of NeuralNetworks
Computational Statistics MVA, 2017 p 4 / 20
![Page 5: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/5.jpg)
Chords and Pitch
• Pitch : subjective perception of a note’s height• The pitch system is composed of 12 classes,• An octave means a doubling of the frequency,each octave : fn = 2
112 fn−1
• Define an equivalence relation between notes• A chord is combination of 2 or more pitches
Computational Statistics MVA, 2017 p 5 / 20
![Page 6: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/6.jpg)
Chords and Pitch
Computational Statistics MVA, 2017 p 6 / 20
![Page 7: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/7.jpg)
Features Extraction
Computational Statistics MVA, 2017 p 7 / 20
![Page 8: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/8.jpg)
Short-Time Fourier Transform
• Discrete FT on equally spaced segments of thesong
• Frequency mapping can be scaled (Mel,Logarithmic)
Drawbacks :• Constant resolution and frequency difference• Not suited to represent the pitch class concept
Computational Statistics MVA, 2017 p 8 / 20
![Page 9: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/9.jpg)
Constant Q Transform
fk = fmin.2kB (1)
where k is the frequency index,B is the number ofbins per octave.
X (k ; r) =1Nk
∑n
x(nr)w(n)e j2πnQ/Nk (2)
Q = 12B−1 , and the resolution−1Nk = [Q fs
fk]
Computational Statistics MVA, 2017 p 9 / 20
![Page 10: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/10.jpg)
PCP, the first ACR System
• Can be seen as 1-bin Constant Q Transform• Uses pitches pattern matching using NN andhand crafted score
Takuya Fujishima, Realtime Chord Recognition ofMusical Sound
Computational Statistics MVA, 2017 p 10 / 20
![Page 11: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/11.jpg)
CQT vs STFT : who’s the best ?
Muhammad Huzaifah, Comparison ofTime-Frequency Representations.
Computational Statistics MVA, 2017 p 11 / 20
![Page 12: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/12.jpg)
Features Processing
Computational Statistics MVA, 2017 p 12 / 20
![Page 13: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/13.jpg)
Pre-processing
• Time slicing : concatenating adjacent frames• First low pass filter : yn = αyn−1 + (1− α)xn• A pair of low pass filters : exponentiallyweighted mean ∑
i=−r
ra−|i |x [.+ r ]
• Other papers : Geometric mean/ Median filter
Computational Statistics MVA, 2017 p 13 / 20
![Page 14: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/14.jpg)
Learning Architecture
Computational Statistics MVA, 2017 p 14 / 20
![Page 15: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/15.jpg)
Computational Statistics MVA, 2017 p 15 / 20
![Page 16: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/16.jpg)
Deep Belief Networks
• Layers of fully connected layers of RestrictedBoltzmann Machines
• A stochastic neural network :• One layer of visible units : chords• One layer of hidden units : latent variables• the hidden units of layer i are the visible for thelayer i + 1
• Seen as an out-dated model in the Deepcommunity
Computational Statistics MVA, 2017 p 16 / 20
![Page 17: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/17.jpg)
Deep architectures
• Recurrent Neural NetworksNicolas Boulanger-Lewandowski, AudioChord Recogntion with Recurent NeuralNetworks.
• Convolutional NN:Anis Rojb al., Music Transcription by DeepLearning with Data and “Artificial Semantic”Augmentation
Computational Statistics MVA, 2017 p 17 / 20
![Page 18: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/18.jpg)
Conclusion
Computational Statistics MVA, 2017 p 18 / 20
![Page 19: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/19.jpg)
Conclusions
• Improvement with the introduction with neuralnetworks
• little difference in performance between differentstructures
• Smoothness of solutions to be improved
Matthias Mauch & Katy Noland & SimonDixon (2009) Using Musical Structure toEnhance Automatic Chord Transcription.
Computational Statistics MVA, 2017 p 19 / 20
![Page 20: Automatic Chord Recognition using Neural Networks · Automatic Chord Recognition using Neural Networks Audio Signal Processing Ayoub Ghriss MVA, 2017 Computational StatisticsMVA,](https://reader035.vdocuments.us/reader035/viewer/2022081514/5fd17f1265ff400be04b026a/html5/thumbnails/20.jpg)
Conclusions
Computational Statistics MVA, 2017 p 20 / 20