presentation 17 may morning casestudy 1 sam davies
TRANSCRIPT
R&D BBC 2012
Automatic Mood Classification of TV
Programmes
Sam Davies, Jana Eggink, Denise Bland
BBC Research & Development
R&D BBC MMVIII
British Broadcasting Corporation Archive
• BBC I&A
– Perivale, London
– >1,000,000 items
• ~ 650,000 TV
• ~ 350,000 Radio
• ~ 1.5 million hours
– Since 1922
• BBC Redux
– online
– 300,000 hours of TV and radio
– Since 2007
• BBC Written Archive
– 4 ½ miles of documents
– Caversham, Reading
R&D BBC MMVIII
Current Programme Retrieval
• Infax
– > 1,500,000 programmes
• LonClasss
– > 52,000 concepts
• “pop music”, “Iraq”, “criticism
of growing plants for biofuels”,
“Dover Castle communications
centre”, “fake psychics”,
“posters of Ariel Sharon”.
– BBC Redux
R&D BBC MMVIII
Current Programme Retrieval – BBC Internal
• Infax
– > 1,500,000 programmes
• LonClasss
– > 52,000 concepts
• “pop music”, “Iraq”, “criticism
of growing plants for biofuels”,
“Dover Castle communications
centre”, “fake psychics”,
“posters of Ariel Sharon”.
– BBC Redux
– BBC Snippets
R&D BBC MMVIII
Current Programme Retrieval – Public facing
• BBC iPlayer
– Catch-up service
– Known item search
– Standard categorisation
R&D BBC MMVIII
Current Programme Retrieval – Public facing
• BBC iPlayer
– Catch-up service
– Known item search
– Standard categorisation
• bbc.co.uk/programmes
– More episodes
– Editorially chosen similar
programmes
R&D BBC MMVIII
Current Programme Retrieval – Public facing
• BBC iPlayer
– Catch-up service
– Known item search
– Standard categorisation
• bbc.co.uk/programmes
– More episodes
– Editorially chosen similar
programmes
• Link key contributors
– Editorially identified
– Automatically linked
R&D BBC MMVIII
Mood Based Classification – System overview
Feature Extraction
• Video & Audio Analysis
– Colour histogram, motion
detection, brightness.
– Spectral audio components
• Object identification
– Faces, animals, objects
(Tardis)
– Gunshots, laughter, screaming
R&D BBC MMVIII
Mood Based Classification: Ground truth collection
• Ground Truth Collection
– Video
• 200 members of public from varied
demographic
• 250 programmes
• Asked to classify programme clips
based around adjectives taken
from Affective Theory
R&D BBC MMVIII
Mood Based Classification - GUI
R&D BBC MMVIII
Mood Based Classification - GUI
R&D BBC MMVIII
Other mood based features - Music
• Ground Truth Collection
– Video
• 200 members of public from
varied demographic
• 250 programmes
– Music
• MusicalMoods
– 20,000 members of
public
– 60 theme tunes
R&D BBC MMVIII
Other mood based features - Text
• Identify mood of any text on three axis:
– Valence (positive/negative) e.g. triumphant, love, paradise
– Arousal (amount of emotion instilled) e.g. rage, thrill, explosion
– Dominance (power) e.g. winner, confident, admired
• Increases dimensionality of sentiment analysis
• Use on large datasets negates requirement for syntactical analysis
• Subtitles are more correct than derived metadata (automatic speech transcripts, machine
vision, machine listening)
• Fast, scalable
• Domain independent
R&D BBC MMVIII
Other mood based features - Text
R&D BBC MMVIII
Other mood based features: Text
Precision 0.95
Recall 0.91
F1 Score 0.93
R&D BBC MMVIII
Future areas - Combination of Affect and Semantic
R&D BBC MMVIII
Future areas - Highlights Generation
• Sports matches
– Audio based analysis
– Two stages
• Live match identification
• Interesting section detected
– Accuracy of 78%
– Looking currently to include
social media to increase
accuracy.
R&D BBC MMVIII
Publications & more info
• Davies, S., Bland, D. & Grafton R (2010) “A Framework for Automatic Mood Classification of TV Programmes”
presented at SAMT 2010.
• Davies, S. (2010) “Interestingness Detection in Sports Audio Broadcasts” presented at IEEE ICMLA 2010
• Knoiusz, P. & Mikolajcyzk, K. (2011) “Soft Assignment Of Visual Words As Linear Coordinate Coding And
Optimisation Of Its Reconstruction Error” presented at ICIP 2011
• Knoiusz, P. & Mikolajcyzk, K. (2011) “Spatial Coordinate Coding To Reduce Histogram Representations, Dominant
Angle and Colour Pyramid Match” presented at ICIP 2011
• Davies, S. & Bland, D. (2011) “An Improved Framework for Affective Classification and Browsing of Large Scale
Broadcast Archives” presented at ACM SIGIR 2011
• Mann, M. & Cox, T. (2011) “Music Mood Classification of Television Theme Tunes” presented at ISMIR 2011
• Davies, S., Mann, M., Cox, T. & Allen, P. (2011) “Musical Moods: A Mass Participation Experiment for Affective Music
Classification” presented at ISMIR 2011
• Eggink, J. Allen, P. & Bland, D. (2011) “A Pilot Study for Mood-Based Classification of TV programmes” presented at
ACM SIGAC 2011
• Eggink, J. & Bland, D. (2012) “A Large Scale Experiment for Mood-Based Classification of TV Programmes”
presented at ICME 2012
• Available at http://www.bbc.co.uk/rd/publications/whitepapers.shtml
