large-scale capture of producer-defined musical semantics - ryan stables (semantic media @ the...
DESCRIPTION
This talk was given by Ryan Stables (Birmingham City University) at the "Semantic Media @ The British Library" event on 23 September 2013.TRANSCRIPT
![Page 1: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/1.jpg)
Large-Scale Capture of Producer-Defined MusicalSemantics
Ryan StablesSchool of Digital Media Technology
Birmingham City University
![Page 2: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/2.jpg)
Problem...
![Page 3: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/3.jpg)
Problem Definition
Producer:
I Audio effects parametersusually refer to low-levelattributes.
I Professionally produced audiooften requires extensivetraining.
Researcher:
I Lack of semantically annotatedmusic production datasets.
I How can we map low-leveldescriptors to perceivedmuscial timbre?
![Page 4: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/4.jpg)
Problem Definition
I Descriptors need to representthe views of music producers.
I These may change with genre,musical instruments, etc...
I Various terms may be used todefine similar things (colour,texture etc...)
![Page 5: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/5.jpg)
Project Aims
1. Gather large amounts of semantics data during the musiccreation/production process.
I Develop a series of DAW plug-ins.I Extract information and anonymously upload it to a server.
2. Identify correlation and patterns in the semantics data.
3. Use the data to improve/aid music production tasks.
![Page 6: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/6.jpg)
Model...
![Page 7: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/7.jpg)
Project overview
Server
Descriptor name...
Save...Load...
Save...
Semantic Descriptor
Parameter Space
Feature Set
Pre/Post Gain
Analysis...
Natural Language
Processing
Dimensionality
Reduction
Etc...
(1)
(2)(3)
Figure : Schematic Overview of the Semantic Audio Feature Extraction Project.
![Page 8: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/8.jpg)
(1) Plug-in interface
I Parameters can be setexperimentally.
I Semantic descriptors to bestored in text field.
I Descriptors can be loadedthrough same interface.
I Parameters are stored and/orset.
Figure : Semantic Audio plug-in: Multi-band distortion
![Page 9: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/9.jpg)
(2) Feature Extraction
I Features are extracted from theselected region.
I The parameter space is stored.
I Semantic descriptors are sentas targets.
I Additional metadata is sent, ifavailable.
Server
Descriptor name...
Save...Load...
Save...
Semantic Descriptor
Parameter Space
Feature Set Pre/Post Gain
Analysis...
Natural Language Processing
Dimensionality Reduction
Etc...
Figure : Stored attributes.
![Page 10: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/10.jpg)
(3) Mapping
I NLP Algorithms to identifysemantic correlation.
I Dimensionality reduction tofind correlation infeatures/parameters.
I Additional data partitionsbased on metadata (Genre,instrument, etc...)
I Results sent back to userplug-in.
Server
Descriptor name...
Save...Load...
Save...
Semantic Descriptor
Parameter Space
Feature Set Pre/Post Gain
Analysis...
Natural Language Processing
Dimensionality Reduction
Etc...
Figure : Results processing
![Page 11: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/11.jpg)
Design Constraints...
![Page 12: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/12.jpg)
Architecture
I Requirements:I Maximisation of user-base.I Transparency: Access to the processing chain.
I Design decisions:I Stand-alone plug-ins.I MultiFX.I Plug-ins within a plug-in.I Analysis-only.
I Other:I Free field vs. fixed word.I Before and after.I Metadata pane.
![Page 13: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/13.jpg)
Analysis framework
I LibXtract.
I Hard-coded, C library.I Around 400 combined
audio features*.I [Bullock, 2007]
I Vamp.
I Plug-in within a plug-in.I Hosts LibXtract features,
amongst others.I [Cannam et al., 2006]
![Page 14: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/14.jpg)
Mini-Project...
![Page 15: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/15.jpg)
Mini-Project: Aims
I Analyse the production requirements of musicians.I Birmingham ConservatoireI The Music Producers GuildI The Birmingham Music Network
I Build a series of prototype systems for the collection ofmusical semantics data.
I Use these systems to collect data from a small group ofmusicians during the production process.
I Evaluate the results in order to identify a suitable system forfuture research.
I Demonstrate the feasibility of a wider research project in thisarea.
![Page 16: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/16.jpg)
Mini Project: Schematic
Plug-in
development
Interface design
Algorithm
Development
Server, network,
data distribution
User Testing Data
Aquisition
Results
Analysis
Figure : Schematic Overview of the Mini-Project.
![Page 17: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/17.jpg)
Positions and Timescale
I 2 x PhD Students: 1 x C4DM (QMUL) & 1 x DMT (BCU).
I 3 x Advisory roles.
I Timescale: 6-months from September 2013.
I Future: collaborative grant application.
![Page 18: Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)](https://reader034.vdocuments.us/reader034/viewer/2022051819/54c6b7914a795999418b457e/html5/thumbnails/18.jpg)
References
Bullock, J. (2007).Libxtract: A lightweight library for audio feature extraction.In Proceedings of the International Computer MusicConference, volume 43.
Cannam, C., Landone, C., Sandler, M. B., and Bello, J. P.(2006).The sonic visualiser: A visualisation platform for semanticdescriptors from musical signals.In ISMIR, pages 324–327.