melodic style detection in hindustani music - compmusic

Melodic Style Detection in Hindustani Music

Amruta Vidwans Prateek Verma

Preeti Rao

Department of Electrical Engineering Indian Institute of Technology

Bombay

Department of Electrical Engineering , IIT Bombay 2 of 25 of 40

OUTLINE

p  Introduction n  Objective

p  Style Classification in Indian Classical Vocal Music n  Literature Survey

n  Previous Work

n  Feature Description

n  Database and Listening Tests

n  Results and Extention to Turkish Music

p  Style Classification In Flute Alap recordings n  Database and Annotation

n  Signal Characteristics of Discriminatory Ornaments

n  Feature Design

p  Conclusion and Future Work p  References

2/27

Department of Electrical Engineering , IIT Bombay 3 of 25 of 40

INTRODUCTION Objective

p  Melodic features to distinguish n  Hindustani, Carnatic and Turkish music n  Instrument playing styles

p  Basis: Melody line alone suffices for listeners to reliably distinguish musical styles

p  Applications: Extract important metadata automatically

3/27

Department of Electrical Engineering , IIT Bombay 4 of 25

Style Classification in Indian Classical Vocal Music

4/27

Department of Electrical Engineering , IIT Bombay 5 of 25 of 40

INTRODUCTION Literature Survey – Similar work done

p  Timbral features n  Features based on timbre such as MFCC, delta-MFCC, and spectral features used

by Parul et. al (ISMIR13) to distinguish Indian music genres using Adaboost, GMM etc.

p  Timbral + Rhythmic features n  Kini et. al. (NCC10) used these features to do genre classification of North Indian

Classical Music into Quawali, Bhajan, Bollywood etc. n  Liu et. al. (ICASSP09) used timbral, wavelet coefficients and rhythmic features to

classify different styles viz. Arabic, Chinese, Japanese, Indian, Western classical p  Emphasized that the diversity of Indian Classical Music is difficult to model

p  Timbral +Melodic features n  Salamon et al (ICASSP12) gave a large number of pitch based features in

addition to timbral features which improved the performance.

5/27

Department of Electrical Engineering , IIT Bombay 6 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Pre-processing

p  Melody extraction n  Difficulty: Presence of multiple instruments along with voice n  Accuracy of the present state of the art automatic pitch trackers ~80% n  Semi automatic approach for pitch detection is used to achieve best possible

performance

6/27

Carnatic raga Dwijavanti by artiste R Vedavalli

Hindustani raga Jaijaiwanti by artiste Rashid Khan

original resynthesized

Department of Electrical Engineering , IIT Bombay 7 of 25 of 40

p  Localized contour shape-features n  Previous study used Steady Note (SN) and Gamak Measure (GM) n  A steady note: pitch segment of ‘N’ ms with standard deviation less than ‘J’

cents n  Data driven parameters gave highest accuracy (N=400ms J=20cents) n  SN: Normalized total duration of steady regions n  GM: ratio of number of non steady regions(1sec) having oscillations in

3-7.5Hz to the total number of regions

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description

7/27

Department of Electrical Engineering , IIT Bombay 8 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description

p  Primary Shape contour (PS) n  Contour typology proposed by C. Adams [6] for categorizing melodic shapes in 15

types n  Segments taken from silence to silence, assignment done using relation between

Initial, Final, Highest and Lowest pitch values.

8/27

Contour types 12 and 13 in alap section in raga Hindolam by Carnatic artistes (a) T. N. Sheshagopalan

(b)M. S. Subalaxmi

(a) (b)

12

13

Department of Electrical Engineering , IIT Bombay 9 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description

p  Distance of highest peak in unfolded histogram from Tonic (DistTonic) n  In unfolded histogram the Hindustani alaps are concentrated near the tonic,

Carnatic alap pitch distribution is closer to the upper octave tonic n  Distance of the highest peak from the tonic in the unfolded histogram taken as

feature.

9/27

Rashid Khan raga Todi Sudha Raghunathan raga Subhapanthuvarali

Department of Electrical Engineering , IIT Bombay 10 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Feature Description

p  Melodic Transitions (MT) n  The overall progression of the melody can be characterized by this n  Haar wavelet basis function used to represent the melody n  Fifth level approximation is used

p  Lower level approximation captures very minute variation whereas higher level gives a coarse representation.

n  Normalized size of upward jumps(>1semitone) in concatenated pitch contour taken as feature.

10/27

Department of Electrical Engineering , IIT Bombay 11 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Database

p  Choice of alap section used for labeling a track n  Unmetered section always rendered in the start. n  Database Description

p  Widely performed raga pairs of same scale interval from Hindustani and Carnatic p  Ragas belonging to different scales chosen p  Renowned artists of various schools of music chosen

n  Total of 120 alap sections equally distributed across both the styles

11/27

Scale Hindustani Raga (No. of clips)

Carnatic Raga (No. of clips )

Heptatonic Todi (12) Subhapanthuvarali (14)

Pentatonic Malkauns (18) Hindolam (12) Nonatonic Jaijaiwanti (10) Dwijavanthy (14)

Octatonic Yaman and Yaman Kalyan (20) Kalyani (20)

Department of Electrical Engineering , IIT Bombay 12 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Listening Tests

p  Test Design n  Hypothesis : Melody alone is sufficient to carry out style distinction n  Audio clips re-syntheisized with constant timbre sound.

p  Removes bias towards artist identity, pronunciation, voice quality. p  Volume dynamics retained (Sum of vocal harmonics).

n  Interface Description p  User information in terms of training of subject as well as familiarity p  Audios divided in 5 sets --12 clips of each Hindustani (H)-Carnatic (C) in each set. p  Listening of 10 sec audio mandatory with option of pause, Skipping an audio not allowed p  Decision label as H, C or NS (Not Sure) asked for each clip

12/27

Category Accuracy (no of participants) trained <3yrs 74.6% (10) trained 3-10yrs 79.7 % (8) trained >10yrs 89.6 % (2) Amateur 75.2 % (18) Listener 77.5 % (13) Overall Accuracy 76.9 % (51)

Department of Electrical Engineering , IIT Bombay 13 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Automatic classification and feature selection

p  Quadratic classifier used on feature sets n  5 fold CV on 5 different random partitions to avoid any raga bias n  Exhaustive search on all possible feature combinations for feature selection n  Separability of parameterized distribution taken into account for small dataset

p  Distribution of the Log Likelihood ratio (LLR) found for a feature set.

p  F-ratio computed on distribution LLR as a confidence measure.

n  Highest accuracy achieved is 96% for SN, GM, DistTonic, and PS feature set.

13/27

𝐿𝐿𝑅=   ln (𝐿(𝑥∕H )/𝐿(𝑥∕C ) )  𝐹  −𝑅𝑎𝑡𝑖𝑜  = ( 𝜇↓𝐻 − 𝜇↓𝐶 )↑2 /𝜎↓𝐻 ↑2 + 𝜎↓𝐶 ↑2  

Department of Electrical Engineering , IIT Bombay 14 of 25 of 40

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Automatic classification and feature selection

p  Confusion matrix

p  Correlation with the listeners n  Subjective test label across all listeners decided by 60% threshold for labeling H,

C or NS n  For objective measure region around 10% of LLR values around 0 taken as NS n  Cost value of -1,1 or 0 is assigned according to the agreement between

subjective and objective labels. n  Highest value of correlation was found to be 0.79 across all the feature

combinations

14/27

Obs. C H NS True C 45 12 3 H 8 48 4

(a) (b) Confusion matrix for (a) listening tests (b) classifier output The horizontal labels correspond to true labels.

Obs. C H True C 58 2 H 3 57

Department of Electrical Engineering , IIT Bombay 15 of 25

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Extention to Turkish Music

0 15/27

Carnatic raga Dwijavanti by artiste R Vedavalli

Turkish makam Nihavent by artiste Hafiz Kemal Bey

Hindustani raga Jaijaiwanti by artiste Rashid Khan

Department of Electrical Engineering , IIT Bombay 16 of 25 of 40

p  Experimental Setup and Results n  60 Taksims considered for classification using the melodic features n  The features discussed before were applied to 3 classes. n  Highest accuracy of 81.6 % was obtained using SN, GM, MT, DistTonic.

n  Most confusion occurred for T and C which was validate via subjective tests

n  No category of primary shape was discriminating Turkish-Carnatic clips

Obs. H C T True H 56 3 1 C 1 48 11 T 0 17 43

Obs. H C T NS True H 131 13 2 4 C 21 118 7 4 T 5 25 114 6

(a) (b) Confusion matrix for (a) Listening test output (b) Classifier output

16/27

STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC Extention to Turkish Music

Department of Electrical Engineering , IIT Bombay 17 of 25

Style Classification In Flute Alap recordings

17/27

Department of Electrical Engineering , IIT Bombay 18 of 25 of 40

INTRODUCTION

Example: Gayaki vs Tantrakari

18/27

Department of Electrical Engineering , IIT Bombay 19 of 25 of 40

p  Gayaki vs Tantrakari styles n  Gayaki : Vocal characteristics incorporated into instrument play.

p  Singing voice is fluid, glides used often n  Tantrakari : Instrumental characteristics (particularly plucked string)

p  Melodic Leaps eg. Discreteness p  Elements synonymous with pluck adapted in Flute : Tonguing p  Fast Stroke pattern while playing melody adapted in Flute: Fast Tounging

p  Ornament Production n  Glide : Slowly lifting the finger- Rate controlling the glide shape n  Vibrato : Periodic Pulsations in Air Flow n  Fast Tounging : Tongue movement

p  Using tongue to articulate notes differently in a melody p  Synonymous with Sitar-rapid movements of right hand (Tantrakari)

n  Blow Stop : p  Blowing with stops while rendering a note p  Periodic blowing patterns (1-2 or 1-1-2) with same/different note

INTRODUCTION

Tantrakari vs Gayaki Style

19/27

Department of Electrical Engineering , IIT Bombay 20 of 25 of 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Database and Annotation

Artist   Raga   Duration(mins)  

Hari Prasad Chaurasia   Jhinjhoti (T-Metadata)   18:54  

Hansadhwani   1:50  

Yaman   2:29  

Sindh Bhairav   3:30  

Rupak Kulkarni   Hemant (T-Metadata)   6:50  

Ronu Majumdar   Vibhas   5:39  

Pannalal Ghosh   Yaman   2:18  

Shri   2:20  

Pilu   1:30  

Nityanand Haldipur   ShuddhaBasant   2:46  

Four Music Experts unaimously aggred on the two clips as tantrakari

20/27

Department of Electrical Engineering , IIT Bombay 21 of 25 of 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Annotation Rules

p  Five categories used for annotation n  Tantrakari

p  T1 : Blowing same notes with stops p  T2 : Steady notes with abrupt movements p  T3 : Discreteness with silences p  T4 : Discreteness with rapid jumps p  T5 : Fast Tounging with any pitch movement

n  Gayaki p  G1 : Oscillatory repetitive movements p  G2 : Non- Oscillatory repetitive movements p  G3 : Glides p  G4 : Non-specific continuous pitch p  G5 : Vibrato

p  Final characteristic feature taken as the consensus of the feature agreed

by all the musicians

21/27

Department of Electrical Engineering , IIT Bombay 22 of 25 of 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Database and Annotation

Artist   Raga   Duration(mins)   f/T/G(in s)  

Hari Prasad Chaurasia   Jhinjhoti (T-Metadata)   18:54   352/465/267  

Hansadhwani   1:50   43/14/33  

Yaman   2:29   57/12/40  

Sindh Bhairav   3:30   50/20/66  

Rupak Kulkarni   Hemant (T-Metadata)   6:50   124/102/111  

Ronu Majumdar   Vibhas   5:39   47/11/44  

Pannalal Ghosh   Yaman   2:18   39/3/66  

Shri   2:20   45/0/29  

Pilu   1:30   133/35/105  

Nityanand Haldipur   ShuddhaBasant   2:46   94/0/41  

q  G: Gayaki T : Tantrakari f: Niether Tantrakari

or Gayaki

q  Alap-Jod-Jhala composition derived from Sitar. (It mainly contains Tantrakari elements)

q  Two different style artists and their disciple chosen

22/27

Department of Electrical Engineering , IIT Bombay 23 of 25 of 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS

Database and Annotation

Artist   Raga   Vibrato   Fast

Tounging  

Blow -

Stop  

T3/T4   G3/G4  

Hari Prasad Chaurasia   Jhinjhoti(T)   P   P   P   P   P  

Hansadhwani   P   O   P   P   P  

Yaman   O   O   O   P   P  

Sindh Bhairav   P   O   O   P   P  

RupakKulkarni   Hemant(T)   O   P   P   P   P  

RonuMajumdar   Vibhas   P   O   O   P   P  

PannalalGhosh   Yaman   P   O   O   P   P  

Shri   P   O   O   O   P  

Pilu   P   O   O   O   P  

NityanandHaldipur   ShuddhaBasant   P   O   O   O   P  

23/27

Presence of Tantrakari and Gayaki elements

Department of Electrical Engineering , IIT Bombay 24 of 25 of 40

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS Production and Acoustic Characteristics

p  Blow-Stop & Fast Tonguing – Two of the most distinctive features n  Energy Contour different due to difference in articulation n  Higher rate of energy variation due to use of tongue. n  Pitch movements do not influence energy movements. n  Intensity of Tanpura almost constant

24/27

Department of Electrical Engineering , IIT Bombay 25 of 25 of 40

p  Features capturing acoustic characteristics of Fast Tounging

n  Sub-band Peak Ratio (SPR) =

n  Sub-band FFT Ratio (SFR) =

STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS Feature Design

-- 2s segments of data across the two styles taken as data point

-- Ground Truth obtained using manual annotation

25/27

Classifier Accuracy Quadratic 74.07 Linear 73.15

Department of Electrical Engineering , IIT Bombay 26 of 25 of 40

p  Conclusion n  Successfully proposed melodic features to distinguish vocal music styles n  Importance of energy based features brought out for flute style

classification. n  Automatic classification results given with degree of separability

p  Future Work n  Extension of database to non-alap recordings in flute style classification

study. n  Addition of new features to distinguish C and T to improve the

classification accuracy

26/27

Department of Electrical Engineering , IIT Bombay 27 of 25 of 40

1] P. Agarwal, H. Karnick and B. Raj, “A Comparative Study of Indian and Western Music Forms”, ISMIR 2013

2] S. Kini, S. Gulati and P. Rao, “Automatic genre classification of North Indian devotional music” , NCC 2010

3] Y. Liu, Q. Xiang, Y. Wang and L. Cai, “Cultural Style Based Music Classification of Audio Signals,” Acoustics, Speech and Signal Processing, ICASSP, 2009.

4] A. Kruspe et al. “Automatic classification of Music Pieces into Global Cultural Areas ”, In Proc. of 42nd International Conference on Semantic Audio, AES 2011

5] J. Salamon, B. Rocha and E. Gomez, "Musical Genre Classification using Melody Features extracted from polyphonic music signals", International Conference on Acoustics, Speech and Signal Processing, 2012

6] C. Adams, “Melodic Contour Typology” , Ethnomusicology 20.2 : 179-215 (1976) 7] S. Justin, S. Gulati and X. Serra, "A Multipitch Approach to Tonic Identification in

Indian Classical Music", ISMIR, 2012 8] C. Clemments, “Pannalal Ghosh and the Bānsurī in the Twentieth Century” , City

University of New York, New York City, September 2010

27/27

References