Mining Actor Correlations with Hierarchical Concurrence Pasring
Reporters: R98922004 Yun-Nung Chen, R98922033 Yu-Cheng Liu
Reference
Ming Actor Correlations with Hierarchical Concurrence Parsing (ICASSP 2010) Kun Yuan, Hongxun Yao, Rongrong Ji,
Xiaoshuai Sun Computer Science & Technology, Harbin
Institute of Technology
Outline
Introduction Actor Indexing Mining Actor Correlations
Context-Based Actor Concurrence Graph Ranking Concurrent Shots Actor Correlation Changes Analysis
Experimental Results
Introduction
Actor correlations graph interfaces
Introduction
Actor correlations graph interfaces
Introduction
Actor correlations graph interfaces Top 20 shots
Overview
Actor Indexing
Locating Actors
Shot boundary detection (SBD) Shots and scenes
Locating actor faces & face tracking algorithm Face set: different poses from the same
actor
Clustering Actors
2D-PCA reduces dimension Features of same person may
distribute discretely in feature space Given 2 face sets Fk and Fl, 2 pose sets
If distance < T, 2 face sets belong to the same person
Clustering Actors
Mining Actor Correlations
Context-Based Actor Concurrence A shot and its surrounding shots may
present a plot between two actors in video Gaussian weight measurement
Context-Based Actor Concurrence
Hierarchical Concurrence
Scene level correlation
Video level correlation
Hierarchical Concurrence
All Actor correlations graph
Construct correlations graph from
Ranking Concurrent Shots
A single character i
Character correlations between i and j
Ranking Concurrent Shots
Given i, j, sort RankScore<i, j>(k) for all k Show top 20 shots
Actor Correlation Changes Analysis Two actors’ correlation changes with
story Analyze the difference of concurrence
R(i, j)A
correlation measure between i and j in the part A
Change ratio Hlp
Experimental Results
Experiment Setup
20 hours video of “Friends” TV series About 4000 shots Over 800 face sets
Clustering into about 60 face sets (T = 0.25)
Manual labeling to 17 actors
Evaluation
The actor concurrence precision in all ranking shots is up to 90%
The precision of each two actor’s co-occurrence in ranking top 20 is up to 98%
Thanks for your attention.