mediaeval 2016 - emotional impact of movies task
TRANSCRIPT
MediaEval 2016Emotional ImpactofMovies Task
Organizers: Emmanuel Dellandréa, Liming Chen, Yoann Baveye, MatsSjöberg, Christel Chamaret
Contact:EmmanuelDellandréa – [email protected]
1MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Context
Anevolution ofprevious years tasks onviolenceandaffectprediction from videosApplications:Personalized contentdeliveryMovie recommendationVideo editing supervisionVideo summarizationProtectionofchildren from potential harmful content
2MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Representation ofemotions
3MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Task description
Deploy multimedia features andmodels toautomatically predict theemotional impactofmoviesEmotionconsidered interms ofinduced valenceandarousalTwo subtasks:Globalemotion prediction:predict ascoreofinducedvalence(negative-positive)andinduced arousal(calm-excited)forashortvideo clip(~10s)Continuousemotionprediction:Predict thevalenceandarousalcontinuously along longvideos
ØAscoreofinduced valenceandarousalshould be provided foreach 1s-segmentofthevideo
4MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Run submissions andevaluation
Upto5runs foreach subtaskArequired run which usesnoexternal trainingdata,only theprovided development datais allowed
Standardevaluationmetrics:Mean SquareErrorPearson’s Correlation Coefficient
5MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Development dataset
Discrete LIRIS-ACCEDE9,800video clipsfrom 160moviesunder Creative CommonslicensesDurationbetween 8sand12sBasicsetoffeaturesRanking according tovalenceandarousal
Continuous LIRIS-ACCEDE30movies selected among the160Durationbetween 117sand4,566s(totalduration:~7hours)Continuous induced valenceandarousalself-assessments
6
LIRIS-ACCEDE available at: http://liris-accede.ec-lyon.fr
MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Testdataset
From 49newmovies under Creative Commonslicenses1,200additional shortvideo clipsforthefirstsubtask (between 8and12seconds)10additional longmovies (from 25minutesto1.5hours)forthesecondsubtask (foratotaldurationof11.48hours)
7MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Groundtruth
Valenceandarousalranking:Pairwise video comparisonsonCrowdFlower (crowdsourcing)Annotators asked tofocusontheemotion they feltSimpletask:Which oneconveys themostpositiveemotion?Which oneconveys thecalmest emotion?
Quicksort process torank allvideos
8MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Groundtruth
From rankings toratings:Ratingscollected for40video clipsregularly distributed28trained participantsinacontrolled environmentRatingsestimated using Gaussian Process models
9
Arousal Valence
MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Groundtruth
Continuous annotation:Induced valenceandarousalself-assessments16participantsModifiedGtrace interfaceandjoystick
10MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Task participation
13teamregistered,5havesubmitted runsGrandtotalof40run submissionsGlobalsubtask:5 teams,18runs
Continuous subtask:Fullruns:
4 teams,17runs
Lightruns (subset ofthetestset:5movies,totalduration:4.82hours):
1team,5runs
11MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Participants’approaches
FeaturesVisualfeaturesDeep features,SIFTfeatures
AudiofeaturesStatistical acoustic features (OpenSmile),MFCC
TemporalfeaturesImproved DenseTrajectories,Feature Dynamic HistoryHistogram
Feature dimensionreductionSequential Backward Selection,Arousal-ValenceDiscriminantPreserving Embedding
12MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Participants’approaches
Regression modelsLinear regressionSVRNeuralnetworksPartialLeastSquaresLeastSquaresboostingRandom forestsDeep neuralnetworksLongShort-TermMemorymodels
13MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Globalsubtask results
Valence
14MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Globalsubtask results
Arousal
15MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Continuous subtask results
Valence
16MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Continuous subtask results
Arousal
17MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Conclusion
Participants’approaches provided encouraging results,particularly fortheglobalsubtaskArousalbetter predicted than valence(consistentwiththeliterature)Thecontinuous subtask remains particularly challengingOnly oneteamproposed forthecontinuous subtask anapproach tomodeltemporaldependencies (with LSTM)Halfoftheregistered participantshavesubmitted runsè task too difficult ?Thisyear newtestsetasanextensionofLIRIS-ACCEDEdataset available athttp://liris-accede.ec-lyon.fr
18MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
ThefutureoftheEmotionImpacttask
Interesting tocontinue?Anovel orientationofthetask ?
Ø Tobe discussed during thetechnical retreat onFriday
19MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.
Programofthesession
Emotional Impactsession:Friday21st,9:00-9:30BULinMediaEval 2016Emotional ImpactofMoviesTask (Asim JanandYonaFalinie,BrunelUniversity,London,UnitedKingdom)MiningEmotional Features ofMovies (YangLiu,HongKongBaptist University,China)
Technical retreat:Friday21st,9:45-10:45Lessons learnedThefutureofthetaskAny other topics…
20MediaEval 2016 Workshop, Oct. 20-21, 2016, Hilversum, Netherlands.