mega-boundaries temporal video boundary detection
TRANSCRIPT
Mega-boundariesTEMPORAL VIDEO BOUNDARY DETECTION
Mega-boundaries
Mega boundaries are defined between macro segments that exhibit different structural and feature consistency.
A good example of mega boundaries application is commercial detection
Commercial detection
Common method is detection of high activity rate and black frame detection coupled with silence detection
Lienhart 1997 Use of monochrome images , scene breaks , and action.
Blum 1992 Use of black frames and activity detector.
Iggulden 1997 Distance between black frame sequences.
Dimitrova 2002 Automatically spots repetitive patterns. Must be identified before
recognizing
Commercial detection
Nafeh 1994 Learning and discerning of broadcast using Neural Network
Bonner 1982
McGee 1999
Novak 1988
Y. Li 2000
Agnihotri 2003
Features for Commercial detection
Mega boundaries detection method’s are based on what features we have on the test video
Unicolor Frame for commercial break
High visual activity
Letterbox format
Dataset of 8 hours of video from TV programs
Feature data consists of 600000 frames
Triggers and Verifiers
Trigger - Features that can aid in determining the location of the commercial break
Verifier – Features that can determine boundaries of the commercial break
We use the time interval between detected unicolor frame as triggers
Presence of a letterbox change or high cut rate expressed in terms of low cut used as verifiers.
Constrains on commercial breaks are longer than 1 minute and shorter than 6 minute.
Bayesian Belief Network Model
Directed acyclical graph (DAG) The nodes correspond to variables
The arcs describe direct casual relationship between linked variables
The strength of these links is given by conditional probability distributions
P(x1,..,xn)=P(Xn|Xn-1,..,x1)*. . . *P(x2|x1)P(x1) Ω(x1,..xn) - Variables define as DAG
P(xi|∏i)=P(xi|x1,..,xn)
P(.|.) is a cpd (conditional probability density)
Using probability density function and chain rule
Bayesian Belief Network Model
Probability for the verification node using pdf and chain rule
Probability for potential commercial
Probability for separator
Bayesian Belief Network Model
Probability for sequence of black frames
Probability for key frame distance
Evolved Algorithm
Challenge to create algorithm for all countries broadcast difference
Genetic algorithms implement a form Darwinian evolution.
Uses chromosome etc..
Eshelman’s CHC algorithm
CHC is general algorithm with 3 features Monotonic
CHC prevents parents from mating if their genetic is too similar.
CHC uses soft restart
CHC for our Experiment
Default Parameters. 50 chromosomes, divergence rate of 35%.
Each parameter was coded as a binary string.
Each chromosome was decoded into set of parameters for the commercial detector and this detector was given a test video stream.
Correct label for the video frames were detected by human
Highest precision and recall was achieved with precision + recall
Results
First data set 8 hours of TV broad cast consisting of 13 TV programs
1.5 hours of 28 different commercials
Second dataset 4 hours of TV broad cast consisting of 11 TV programs
1 hour of 35 different commercials
FN,FP,TP,TN
Recall=TP/(TP+FN)
Precision=TP/(TP+FP)
Results
Using First data set
Results from first 4 experiment (recall and precision) 80.8% and 92.6% , 80.8% and 92.6% , 79.7% and 87.4% , 81.3% and 94.3%
Experiment 5 used experiment 4 and result was 88% and 90 %
Using second data set
This dataset was acquired after the algorithm
Test to the Genetic Algorithm
Results are shown in Figure.
Conclusion
Boundary segmentation in videoVisual scene segmentationMultimodal story segmentationCommercial detection
Questions ?