melodic features and retrieval ismir graduate school, barcelona 2004 musicology 3-4 frans wiering,...
Post on 16-Jan-2016
213 Views
Preview:
TRANSCRIPT
Melodic Features and Retrieval
ISMIR Graduate School, Barcelona 2004
Musicology 3-4
Frans Wiering, ICS, Utrecht University
Outline
yesterday’s assignment demo: MIR outside academia (7:20; 44:10) one-dimensional melody retrieval Gestalt view of melody advanced melody retrieval assignment
one-dimensional melody retrieval common assumption is (was?) pitch-only retrieval is
sufficient e.g. CCGGAAGGFFEEDDEC mechanisms for fuzzy matching
variants interval (distance between 2 pitches) pitch-contour
same/up/down (Parson’s Code) RURURDRDRDRDRUD
examples: www.musipedia.com (Rainer Typke) www.themefinder.org (CCARH)
Results from Musipedia
query is ranked 3 other hits are
very unlikely unfortunately no
notation/sound available
Haydn: evident false positive why?
Themefinder
Several 1-dimensional search options, e.g. pitch interval contour rhythm
wildcards each theme stored as a
number of strings matching by regular
expressions ca. 40.000 themes
Barlow and Morgenstern (1948)
ESAC encodings Lincoln, 16th Century Motet
(DARMS project)
results from Themefinder
Example from Byrd & Crawford (2001)
other hits not as far-fetched as
musipedia’s different rhythm different meter still not very similar
is this what people have in mind?
Query: +m2 +M2 P1 -M2 -m2 -M2
Nice one we’ve just discovered
www.tuneteller.com Pitch-only search of
MIDI on the internet many more MIR
systems in Rainer Typke’s survey. URL is in your mailbox
Why pitch-only retrieval is unsatisfactory information contribution of other 3 parameters
(estimate for Western music; Byrd & Crawford 2001) pitch: 50% rhythm: 40% timbre + dynamics: 10%
melodic confounds (Selfridge-Field 1998): rests repeated notes grace notes, ornamentation Mozart example
Why pitch-only retrieval is unsatisfactory information contribution of other 3 parameters
(estimate for Western music; Byrd & Crawford 2001) pitch: 50% rhythm: 40% timbre + dynamics: 10%
melodic confounds (Selfridge-Field 1998): rests repeated notes grace notes, ornamentation Mozart example
Gestalt and melody
melody: coherent succession of pitches from New Harvard Dictionary of Music
coherence important for similarity: creates musical meaning bottom-up (pitches and durations) top-down: segmenting, Gestalt
Gestalt theory of perception late 19th/early 20th century, Germany, later US perception of wholes rather than parts explanations: Gestalt principles of grouping application in visual and musical domain
Low-level Gestalt principles
Snyder mentions: proximity
rhythmic intervallic
similarity duration articulation
continuity melodic
these produce closure of wholes
Example: Beethoven 5th symphony: beginning 1st movement also illustrates high-level
principlesfrom Snyder (2001)
Low-level Gestalt principles
Snyder mentions: proximity
rhythmic intervallic
similarity duration articulation
continuity melodic
these produce closure of wholes
Example: Beethoven also illustrates high-level
principlesfrom Snyder (2001)
High-level Gestalt principles
parallellism very strong in Mozart, Ah
vous, second half of melody
intensification important organisational
principle in variations and improvisations
Mozart’s last variation
from Snyder (2001)
Application in analysis and retrieval Gestalt reduces memory
overload: we can ignore the details
Analytical: Schering (1911) 14th century Italian songs basic melodic shape might be nice for retrieval
Problem with Gestalt principles: many different formulations overlap; no rules for conflict intuitive, cannot be
successfully formalized
from New Grove, Music analysis
The cognitive interpretation: chunking what creates a boundary
interval leap long duration tonality (stable chords) etc
Example of quantification: Melucci & Orio (2004) using local boundary detection (Cambouropoulos 1997)
apply weight to intervals and durations boundary after maximum
chunks forther processed for indexing
Organising chunks
STM problem: max. 5-7 different elements very short span
solution: hierarchical grouping
melody schemas contours of melody
cf. Schering ex. examples: axial, arch, gap-
fill Mozart begins with gap-fill
next level: form A-B-A from Snyder (2001)
mental model of a songAh, vous dirai-je maman melody level
phrase level
chunk level
subchunk level
A ABanalysissy
nthe
sis
analysis: from ear to LTM (sub) chunks created by similarity and
continuity a lot of parallellism
boundaries by leaps and harmony chunks may have a harmonic aspect too
(I, V, V->I)
synthesis: from LTM to focus of attention recollection
using general characteristics of phrases and chunks
performance notes are reconstitued through some musical
grammar
Problems of melody retrieval
People remember high-level concepts, not notes often confused with poor performance abilities theme-intensive music (fugues) stimulate formation of such
concepts melodic variability and change
transposition augmentation/diminution ornamentation variation compositional processes: inversion, retrograde
other factors polyphony harmony
Set-based approaches to melody retrieval in polyphony General idea:
compare note sets: find supersets, calculate distance usually take rhythm and pitch into account hopefully more tolerant agains some of the problems of melodic variety
Clausen, Engelbrecht, Meyer, Schmidt (2000): PROMS matches onset times; wildcards elegant indexing
Lemström, Mäkinen, Ukkonen, Turkia (several articles, 2003-4) C-Brahms algorithms for matching line segments
P1: onsets P2: partial match onset times P3: common shared time
attention to time complexity Typke, Veltkamp, Wiering (2003-2004)
Orpheus system
Earth Mover’s Distance
The Earth Mover’s Distance (EMD) measures similarity by calculating a minimum flow that would match two set of weighted points. One set emits weight, the other one receives weight
Y. Rubner (1998); S. Cohen (1999)
Application to music
represent notes as weighted point sets in 2-dimensional space (pitch, time)
weight represents duration other possibilities
contour/metric position etc
other possible application:pitch event + acoustic feature(s)? here, the ‘earth’ is only moved along the temporal axis
Another example
interesting properties tolerant against
melodic confounds
suitable for polyphony
continuous partial matching
disadvantage triangle inequality
doesn’t hold less suitable for
indexing:
after alignment, the ‘earth’ is moved both along the temporal axis and along the pitch axis
Test on RISM A/II
Matching polyphony with the EMD
EMD’s partial matching property is essential MIDI example used as query for RISM database gross errors in playing are ironed out
Proportional Transportation Distance (PTD) Giannopoulos &
Veltkamp (2002) EMD, weigths of sets
normalised to 1 suitable for indexing
triangle inequality holds
no partial matching
Test on RISM A/II
only hits with approximately same length
need 4 queries to find all known items
False positive (EMD)
problems arise when length and/or number of notes differs considerably
Segmenting
overlapping segments of 6-9 consecutive notes
not musical units search results are combined better Recall-Precision
averages
Example of new searchhttp://teuge.labs.cs.uu.nl/Rntt.cgi/mir/mir.cgi
Concluding remarks about melodic retrieval lots of creativity go into melody; difficult to give rules
not a ‘basic musical structure’ (Temperley 2001) essential to use multiple features
pitch, rhythm harmony
segmentation finding perceptually relevant chunks is not easy finding complete melodies may be harder arbitrary segments may also work
indexing strategies for melody melodic change over time several projects have tentative results for polyphony
gut feeling: false positives are big issue notion of salience (Byrd and Crawford)
top related