R&D BBC MMXIII

Mood-based Classification of TV Programmes

Jana Eggink, Sam Davies, Denise Bland

BBC R&D

{jana.eggink, sam.davies, denise.bland}@bbc.co.uk

R&D BBC MMXIII

Searching the Archives

• BBC aims to open up its archives for public access by 2022

• Limited metadata available

• Title

• Broadcast date

• Genre (mostly)

• Limited: actors, semantic labels for professional use

• Mood as additional metadata, intuitive understanding

R&D BBC MMXIII

ActivityPotency

Which Moods?

Evaluation

(EPA model based on Osgood et al, 1957)

Interesting – Boring

Happy – Sad

Light-hearted – Dark

Serious – Humorous Fast paced – Slow paced

Exciting – Relaxing

R&D BBC MMXIII

User Trial

• 200 members of the general public

• 544 video clips (3 minutes excerpts)

• Each labelled by at least 6 participants

R&D BBC MMXIII

Inter-rater Agreement

Krippendorff’s Alphae

o

D

D1

Agreement about Mood Labels

random

perfect

R&D BBC MMXIII

Correlation

• Which moods are independent?

• Do observed correlations correspond to the EPA model?

happy humorous exciting interest fast light

happy

humorous

exciting

interest

fast

light

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

-0.6 -0.4 -0.2 0 0.2 0.4 0.6

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

happyhumorous

excitinginterest fast

light-hearted

sadserious

relaxingboringslow

dark

Component 1

Com

pone

nt 2

PCACorrelation

63% variance

24%

var

ianc

e

R&D BBC MMXIII

Video clips

• 444 in development set, 3-fold cross validation

• 100 in holdout set

Features

• Audio (MFCCs, amplitude, zero-crossing, spectral centroid and roll-off)

• Video (face, luminance, cuts, motion)

• Genre (human assigned)

Testing

• Clips with very clear moods only

• Average rates, all clips on a 1 to 5 scale

Classification

R&D BBC MMXIII

Automatic Classification Gives Good Results

Clear moods only

• 2 class problem

• >95% correct for serious/humorous

• ~90% correct for slow/fast-paced

Classification Accuracy

R&D BBC MMXIII

Automatic Classification Gives Good Results

Average rates

• 1-5 scale

• ~0.7 RMSE for serious/humorous

• <0.7 RMSE for slow/fast-paced

RMS Error for detailed moods

R&D BBC MMXIII

Conclusions

• There is general agreement about mood for TV programme clips

• Mood perception is dominated by two dimensions

• Classification for clips with clear moods is very accurate, and still possible on a detailed continuous scale

• Both genre labels and signal processing features are useful

• Humorous-serious is strongly related to genre

• Slow/fast-paced can be better modelled by audio/video features

Eggink & Bland, A Large Scale Experiment for Mood-Based Classification of TV Programmes, IEEE Int. Conf. Multimedia and Expo, ICME2012, also as BBC White Paper Nr. 232

R&D BBC MMXIII

Demo

R&D BBC MMXIII

• Usage data 14th May 2012 to 22nd August 2012

• 3206 unique users, nearly a third (1013) are returning users

Usage of the Redux Mood GUI

R&D BBC MMXIII

Search Behaviour

R&D BBC MMXIII

Frequent Programmes Watched

Never Mind the Buzzcocks 258

Torchwood 90

Dr Finlay`s Casebook 74

An Evening in with David Attenborough 55

Holiday Weatherview 49

Would I Lie to You? 46

Never Mind the Buzzcocks 36

Morecambe and Wise 33

Never Mind the Buzzcocks 32

Till Death Us Do Part 32

Programmes Watched

R&D BBC MMXIII

Outliers attract Attention

R&D BBC MMXIII

• Public facing Mood GUI based on iPlayer

• Available; http://moods.ch.bbc.co.uk

• Requires greater research in UX

Outlook and Future Work

R&D BBC MMXIII

• Integration of pre-existing metadata

Outlook and future work