jsymbolic

1 / 22

jSymbolicjSymbolic

Jordan Smith – MUMT 611 – 6 March 2008Jordan Smith – MUMT 611 – 6 March 2008

2 / 22

OverviewOverview

jSymbolic extracts high-level features from jSymbolic extracts high-level features from symbolic (MIDI) data.symbolic (MIDI) data.

Walkthrough of the interfaceWalkthrough of the interface Features:Features:

Types of featuresTypes of features Motivation for choice of featuresMotivation for choice of features ExtractionExtraction

Planned improvementsPlanned improvements

3 / 22

4 / 22

5 / 22

6 / 22

7 / 22

8 / 22

9 / 22

OverviewOverview

jSymbolic extracts high-level features from jSymbolic extracts high-level features from symbolic (MIDI) data.symbolic (MIDI) data.

Walkthrough of the interfaceWalkthrough of the interface Features:Features:

Types of featuresTypes of features Motivation for choice of featuresMotivation for choice of features ExtractionExtraction

Planned improvementsPlanned improvements

10 / 22

FeaturesFeatures

3 kinds of features:3 kinds of features: Low-levelLow-level High-levelHigh-level CulturalCultural

11 / 22

FeaturesFeatures

7 categories of high-level features:7 categories of high-level features: Instrumentation (20)Instrumentation (20) Texture (20)Texture (20) Rhythm (35)Rhythm (35) Dynamics (4)Dynamics (4) Pitch statistics (26)Pitch statistics (26) Melody (20)Melody (20) Chords (28)Chords (28)

12 / 22

FeaturesFeatures

Why so many features?Why so many features?

Ensure ability to discriminate as many Ensure ability to discriminate as many different kinds of music as possibledifferent kinds of music as possible

Want features to be as basic as possible, Want features to be as basic as possible, because:because: They are destined for a machine learning They are destined for a machine learning

experimentexperiment Estimating complex features is controversialEstimating complex features is controversial

13 / 22

FeaturesFeatures

Why pick these features?Why pick these features?

Long history of musicological interestLong history of musicological interest Relative ease of extractionRelative ease of extraction

14 / 22

FeaturesFeatures

Why pick these features?Why pick these features?

““The features described above have The features described above have been designed according to those been designed according to those used in musicological studies, but used in musicological studies, but there is no theoretical support for their there is no theoretical support for their … characterization capability.”… characterization capability.”

(Ponce de León. 2004. Statistical (Ponce de León. 2004. Statistical Description Models for Melody Analysis Description Models for Melody Analysis and Characterization. ICMC Proceedings and Characterization. ICMC Proceedings 149-56.)149-56.)

15 / 22

McKay & Fujinaga 2005: Automatic music classification and the McKay & Fujinaga 2005: Automatic music classification and the importance of instrument identification. importance of instrument identification. Proceedings of the Proceedings of the Conference on Interdisciplinary MusicologyConference on Interdisciplinary Musicology..

16 / 22

OverviewOverview

jSymbolic extracts high-level features from jSymbolic extracts high-level features from symbolic (MIDI) data.symbolic (MIDI) data.

Walkthrough of the interfaceWalkthrough of the interface Features:Features:

Types of featuresTypes of features Motivation for choice of featuresMotivation for choice of features ExtractionExtraction

Planned improvementsPlanned improvements

17 / 22

Using the FeaturesUsing the Features

Like jAudio, modular features make it easy Like jAudio, modular features make it easy to add new onesto add new ones

-- ADDING FEATURES --Implement a class for the new feature in the

jAudioFeatureExtractor/MIDIFeatures directory. It must extend the MIDIFeatureExtractor abstract class.

Add a reference to the new class to the populateFeatureExtractors method in the SymbolicFeatureSelectorPanel class.

Features exported to ACE XML or Weka Features exported to ACE XML or Weka ARFFARFF

18 / 22

Feature ExtractionFeature Extraction

Other than jSymbolic, what is the state of Other than jSymbolic, what is the state of the art in symbolic feature extraction?the art in symbolic feature extraction?

Borrow from others or invent your own, and Borrow from others or invent your own, and implement them by yourself.implement them by yourself.

Use MIDI Toolbox.Use MIDI Toolbox.

19 / 22

MIDI Toolbox vs. jSymbolicMIDI Toolbox vs. jSymbolic

ToolboxToolbox

-requires MATLAB-requires MATLAB

-has tools for manipulating -has tools for manipulating and visualizing dataand visualizing data

-analytical goals: estimate a -analytical goals: estimate a musicologically important musicologically important featurefeature

jSymbolicjSymbolic

-requires JAVA-requires JAVA

-is strictly for extracting -is strictly for extracting featuresfeatures

-analytical goals: usefully and -analytical goals: usefully and objectively condense objectively condense informationinformation

20 / 22

Planned ImprovementsPlanned Improvements

Boost number of features from 111 to 160Boost number of features from 111 to 160

Ability to operate on non-MIDI symbolic Ability to operate on non-MIDI symbolic data (MusicXML, GUIDO, kern)data (MusicXML, GUIDO, kern)

Ability to extract over windowsAbility to extract over windows

21 / 22

QuestionsQuestions

22 / 22

ReferencesReferences

jSymbolic overview:jSymbolic overview: McKay, C., and I. Fujinaga. 2006. jSymbolic: A feature extractor for McKay, C., and I. Fujinaga. 2006. jSymbolic: A feature extractor for

MIDI files. MIDI files. Proceedings of the International Computer Music Proceedings of the International Computer Music ConferenceConference. 302-5.. 302-5.

Details of features implemented in jSymbolic:Details of features implemented in jSymbolic: McKay, C. 2004. Automatic genre classification of MIDI recordings. McKay, C. 2004. Automatic genre classification of MIDI recordings.

(M.A. Thesis, McGill University).(M.A. Thesis, McGill University).

Example of jSymbolic’s feature extraction in action:Example of jSymbolic’s feature extraction in action: McKay, C., and I. Fujinaga. 2005. Automatic music classification McKay, C., and I. Fujinaga. 2005. Automatic music classification

and the importance of instrument identification. and the importance of instrument identification. Proceedings of Proceedings of the Conference on Interdisciplinary Musicology.the Conference on Interdisciplinary Musicology.

(This study used a previous version of jSymbolic called Bodhidharma.)(This study used a previous version of jSymbolic called Bodhidharma.)