1 fantastic: a feature analysis toolbox for corpus-based cognitive research on the perception of...
TRANSCRIPT
![Page 1: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/1.jpg)
1
FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music
Daniel Müllensiefen, Psychology Dept, Geraint Wiggins, Computing Dept
Goldsmiths, University of London
Centre for Cognition, Computation and Culture
![Page 2: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/2.jpg)
2
Summary of a Research Project
M4S: Modelling Music Memory and the Perception of Melodic Similarity (2006-2009)
Question: How do Western listeners perceive melody?
Domain: Western commercial pop music
Method: Computational modelling
![Page 3: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/3.jpg)
3
Outline
1. Resultso Music Cognitiono Popular Music Research
2. Methodso Computing Features with FANTASTICo Modelling Music Knowledge from a Corpus (2nd order
features)
3. Backgroundo Similar Approaches/Systemso Questions to be addressed
![Page 4: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/4.jpg)
4
Results: Music Cognition I
Memory for Melodies:
Are there structural features that make melodies more memorable?
How are listeners using musical knowledge to perform implicit and explicit memory tasks?
![Page 5: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/5.jpg)
5
Results: Music Cognition IModelling explicit and implicit memory performance in a
recognition paradigm (Müllensiefen, Halpern & Wiggins, in prep.)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1st ordrerfeatures
2nd orderfeatures(testset)
2nd orderfeatures (pop
corpus)
R^2
Expl. memory(AUC)Impl. memory(old-new)
Results:o Memory performance is
partially explained by musical features
o Implicit memory is better explained by raw features or local context
o Explicit memory draws on domain knowledge and features that are distinctive wrt corpus
![Page 6: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/6.jpg)
6
Results: Music Cognition II
Montreal Battery of Amusia, MBEA, (Peretz et al., 2003):
What makes some test items more difficult than others?
What information do subjects actually use to process tasks?
![Page 7: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/7.jpg)
7
Results: Music Cognition II
Modelling item difficulty in MBEA (Stewart, Müllensiefen & Cooper, in prep)
Results:o 70-80% of item difficulty can
be explained with as few as three musical features
o Relation between item difficulty and features is often non-linear
o Some subtests don’t measure what they are believed to measure (e.g. scale)
![Page 8: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/8.jpg)
8
Results: Pop Music Research I
Court cases of music plagiarism:
Are court decisions predictable from melodic structures?
What musical information is used in court decisions?
![Page 9: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/9.jpg)
9
Results: Pop Music Research I
Model court decisions on melody plagiarism (Müllensiefen & Pendzich, 2009)
Results:o Court decisions can be
closely related to melodic similarity
o Plaintiff’s song is often frame of reference
o Statistical information about commonness of melodic elements is important
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Edit Dist. Tv.equalTv.plaintiff
Tv.defendant
Accuracy
AUC
![Page 10: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/10.jpg)
10
Results: Pop Music Research II
Melodic structure and popularity:
Does popularity correlate with certain structural features of a tune?
![Page 11: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/11.jpg)
11
Results: Pop Music Research IIIdentify features of commercially successful songs on Revolver
(Kopiez & Müllensiefen, 2008)
Results:o 2 features (pitch range and entropy) are sufficient for fully accurate classification
into successful / unsuccessful songso Plausible interpretation as compositional exercise: Invent a chorus melody such
that it has a large range and uses only few pitches much more frequently than the majority of its pitches
€
p (chart_entry =1) =1
1+ e−(772.4 + 141.2 ⋅ pitch_range - 4731.3 ⋅pitch_entropy)
Criterion for commercial success: Entered charts as cover version (yes/no)
![Page 12: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/12.jpg)
12
Method
Two Components
Feature Computation Knowledge from a large corpus of music€
i.abs.std =Δpi − Δp( )
2
i∑
N −1= 2.83
![Page 13: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/13.jpg)
13
Features
Implemented features inspired by concepts fromo Descriptive statisticso Music Theoryo Music Cognitiono Music Information Retrievalo Computational Linguistics
![Page 14: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/14.jpg)
14
Feature ComputationPre-requisite: Transformation from notes to numbers
Melody: Sequence of tuples (notes) with time and pitch information:
),( iii ptn =
![Page 15: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/15.jpg)
15
Summary FeaturesCognitive Hypothesis: Listeners abstract summary
representation of short melodies during listening
Feature format: Value that represents particular aspect of melody
Evidence: o Feature-based description and analysis of tunes since Bartok
(1936) in o Folk song research (Steinbeck, 1982; Jesser, 1990; Sagrillo, 1990) o Computational analysis (Eerola & Toiviainen, 2004; McKay, 2005)
o Summarising mechanism in visual perception (Chong & Treisman, 2005)
![Page 16: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/16.jpg)
16
Summary Features32 summary features based on:
o Pitcho Pitch intervalso Duration ratioso Global extensiono Contouro Implicit tonality
Ex. 1: Pitch range (p.range):
€
p.range = max(p) − min(p)
Ex. 2: Standard deviation of absolute intervals (i.abs.std):
€
i.abs.std =Δpi − Δp( )
2
i∑
N −1
![Page 17: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/17.jpg)
17
Method: Summary Features
Ex. 3: Relative number of direction changes in interpolated contour representation (int.cont.dir.changes)
€
int .cont.dir.changes =
1sgn xi( )≠sgn xi+1( )
∑
1xi ≠xi+1
∑
![Page 18: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/18.jpg)
18
m-type Features
Cognitive Hypothesis: Listeners use literal representation of short subsequences of melody for processing
Inspiration: Text retrieval and computational linguisticsEvidence:
o Sequence learning for pitches and music (e.g. Saffran et al., 1996, 2000; Pearce & Wiggins, 2005, 2006; Loui & Wessel, 2006)
o Sequence learning as a general mechanism
Format of m-type: String of digits (similar to “word type” in linguistics) representing pitch intervals and duration ratios
![Page 19: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/19.jpg)
19
m-type Features• m-types only from within melodic phrases• Pitch intervals: Classified into 19 classes, preserving diatonic
information• Duration ratios: Classified into 3 classes (Drake & Daisy, 2001),
perceptual asymmetric class boundaries (Sadakata et al., 2006)• M-type length: Parallel handling of various lengths (1,…,5)
m-type of length 2:“s1e_s1e”
m-type of length 4:“s1q_s1l_s1q_s1l”
![Page 20: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/20.jpg)
20
m-type FeaturesFormat of m-type feature: Number that represents distributions of m-
types of various lengths in melody
€
mean.Honores.H = 1n⋅100 ⋅
log N
1.01− V 1,N( )V N( )
![Page 21: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/21.jpg)
21
M4S publications on features Melodic Contour (Müllensiefen, Bonometti, Stewart & Wiggins, 2009;
Frieler, Müllensiefen & Riedemann, in press; Müllensiefen & Wiggins, under review)
Phrase segmentation (Pearce, Müllensiefen & Wiggins, 2008; accepted)
Harmonic content (Mauch, Müllensiefen, Dixon & Wiggins, 2008; Rhodes, Lewis & Müllensiefen, 2007)
Melodic accent structure (Pfleiderer & Müllensiefen, 2006; Müllensiefen, Pfleiderer & Frieler, 2009)
![Page 22: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/22.jpg)
22
Method: Using a music corpus
The M4S Corpus of Popular Music (Müllensiefen, Wiggins &
Lewis, 2008): 14,067 high-quality MIDI transcriptions Representative sample of commercial pop songs
from 1950 - 2006 Complete compositional structure (all melodies,
harmonies, rhythms, instrumental parts, lyrics) Some performance information (MIDI patches,
some expressive timing)
![Page 23: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/23.jpg)
23
Using a music corpus: 2nd order summary features
Cognitive Hypothesis: Listeners encode commonness of feature value
Evidence: Statistical learning in music and language (e.g. Huron, 2006; Pearce & Wiggins, 2006; Rohrmeier, 2009)
Method: Replacing feature values by relative frequencies (categorical and discrete features) frequency density from kernel smoothing (continuous features)
![Page 24: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/24.jpg)
24
Using a music corpus: 2nd order summary features
Continuous features Categorical / discrete features
![Page 25: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/25.jpg)
25
Using a music corpus: 2nd order m-type features
Cognitive Hypothesis: Listeners are sensitive to commonness of m-types
Method: Use frequency information on m-types from large corpus
€
mtcf .norm.log.dist =T ′ F τ i
− D ′ F τ iτ i ∈m∑
TFτ ∈m
Example: Normalised distance of m-type frequencies in melody and corpus (mtcf.norm.log.dist)
=> measures whether uncommon m-types are used rather frequently in melody
( )( )∑ ∈
∈∈ =
m
mm
i
i
i TF
TFTF
τ
ττ
2
2'
log
logwith
![Page 26: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/26.jpg)
26
Similarity from features
Three similarity models: Euclidean distance Gower’s coefficient Corpus-based similarity
![Page 27: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/27.jpg)
27
Corpus-based similarity
Cognitive Hypothesis: Similarity perception is based on features of melodies and melody corpus as frame of reference
Idea: Derive similarity from distance on cumulative distribution function
Combine features: By entropy-weighted averaging
( ) ( ) ( )∑ ∑=
=
=
=
−=i jxl
l
xl
llClCjiCk xfxfmm
1 1, ,σ
![Page 28: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/28.jpg)
28
Method: Summary
Feature ANalysis Technology Accessing STatistics In a Corpus:
FANTASTIC
Open source tool box for computational analysis of melodies* 58 features currently implemented Different similarity models based on features Ideas from: Descriptive statistics, music theory, music cognition,
computational linguistics, music information retrieval 2 feature categories: Summary features and m-type features Context modelling via integration of corpus: 2nd order features
*http://www.doc.gold.ac.uk/isms/m4s/?page=Software%20and%20Documentation
![Page 29: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/29.jpg)
29
Method: Yet unexplored
Effects of different corpora Don’t use raw but cognitively weighted pitches (e.g.
perceptual accents) Don’t reduce: Use information from m-types directly for
similarity measurement classification
Create a space / model of feature correlations for corpus (similar to LSA)
![Page 30: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/30.jpg)
30
Background: Similar approaches
Folk Song Research / Ethnomusicology Bartók (1936), Bartók & Lord (1951) Lomax (1977) Steinbeck (1982) Jesser (1992) Sagrillo (1999)
Popular Music Research Moore (2006) Kramarz (2006) Furnes (2006) Riedemann (in prep.)
Computational / Cognitive Musicology Eerola et al. (2001, 2007) McCay (2005) Huron (2006) Frieler (2008)
![Page 31: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/31.jpg)
31
Background: Questions to be addressed
Popular Music ResearchQuestions: How does melodic structure relate to
Popularity and selection processes Style Transmission processes Specific types of behaviour (e.g. singalongability) Value attribution (originality, creativity)
Music Cognition ResearchQuestions: How does melodic structure relate to
Memory performance and memory errors Similarity judgements Expectancy Preference / aesthetic judgements
![Page 32: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/32.jpg)
32
FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music
Daniel Müllensiefen, Psychology Dept Geraint Wiggins, Computing Dept
Goldsmiths, University of London
Centre for Cognition, Computation and Culture
![Page 33: 1 FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music Daniel Müllensiefen, Psychology Dept,Geraint](https://reader036.vdocuments.us/reader036/viewer/2022062511/5515c7fc55034693758b4989/html5/thumbnails/33.jpg)
33
Four main functions
Computing features (summary and m-type): compute.features()
Computing 2nd order summary features: compute.corpus.based feature.frequencies()
Computing 2nd order m-type features: compute.m.type.corpus.based.features()
Computing feature-based similarity: feature.similarity()