Presentation on theme: "Mega-boundaries TEMPORAL VIDEO BOUNDARY DETECTION."— Presentation transcript:
Mega-boundaries TEMPORAL VIDEO BOUNDARY DETECTION
Mega-boundaries Mega boundaries are defined between macro segments that exhibit different structural and feature consistency. A good example of mega boundaries application is commercial detection
Commercial detection Common method is detection of high activity rate and black frame detection coupled with silence detection Lienhart 1997 Use of monochrome images, scene breaks, and action. Blum 1992 Use of black frames and activity detector. Iggulden 1997 Distance between black frame sequences. Dimitrova 2002 Automatically spots repetitive patterns. Must be identified before recognizing
Commercial detection Nafeh 1994 Learning and discerning of broadcast using Neural Network Bonner 1982 McGee 1999 Novak 1988 Y. Li 2000 Agnihotri 2003
Features for Commercial detection Mega boundaries detection method’s are based on what features we have on the test video Unicolor Frame for commercial break High visual activity Letterbox format Dataset of 8 hours of video from TV programs Feature data consists of frames
Triggers and Verifiers Trigger - Features that can aid in determining the location of the commercial break Verifier – Features that can determine boundaries of the commercial break We use the time interval between detected unicolor frame as triggers Presence of a letterbox change or high cut rate expressed in terms of low cut used as verifiers. Constrains on commercial breaks are longer than 1 minute and shorter than 6 minute.
Bayesian Belief Network Model Directed acyclical graph (DAG) The nodes correspond to variables The arcs describe direct casual relationship between linked variables The strength of these links is given by conditional probability distributions P(x 1,..,x n )=P(X n |X n-1,..,x 1 )*... *P(x 2 |x 1 )P(x 1 ) Ω(x 1,..x n ) - Variables define as DAG P(x i |∏ i )=P(x i |x 1,..,x n ) P(.|.) is a cpd (conditional probability density) Using probability density function and chain rule
Bayesian Belief Network Model Probability for the verification node using pdf and chain rule Probability for potential commercial Probability for separator
Bayesian Belief Network Model Probability for sequence of black frames Probability for key frame distance
Evolved Algorithm Challenge to create algorithm for all countries broadcast difference Genetic algorithms implement a form Darwinian evolution. Uses chromosome etc.. Eshelman’s CHC algorithm CHC is general algorithm with 3 features Monotonic CHC prevents parents from mating if their genetic is too similar. CHC uses soft restart
CHC for our Experiment Default Parameters. 50 chromosomes, divergence rate of 35%. Each parameter was coded as a binary string. Each chromosome was decoded into set of parameters for the commercial detector and this detector was given a test video stream. Correct label for the video frames were detected by human Highest precision and recall was achieved with precision + recall
Results First data set 8 hours of TV broad cast consisting of 13 TV programs 1.5 hours of 28 different commercials Second dataset 4 hours of TV broad cast consisting of 11 TV programs 1 hour of 35 different commercials FN,FP,TP,TN Recall=TP/(TP+FN) Precision=TP/(TP+FP)
Results Using First data set Results from first 4 experiment (recall and precision) 80.8% and 92.6%, 80.8% and 92.6%, 79.7% and 87.4%, 81.3% and 94.3% Experiment 5 used experiment 4 and result was 88% and 90 % Using second data set This dataset was acquired after the algorithm Test to the Genetic Algorithm Results are shown in Figure.
Conclusion Boundary segmentation in video Visual scene segmentation Multimodal story segmentation Commercial detection Questions ?