1. ICME 2008 Huiying Liu, Shuqiang Jiang, Qingming Huang, Changsheng Xu

2.  Introduction  LAR detection framework  LAR detection for sports video  VCI methods  Experiments  Conclusion

4.  Virtual Content Insertion (VCI) is an emerging application of video analysis and has been studied for several years.  The task of VCI is to make the inserted content attractive to the viewers and meanwhile not intrusive.  Compared with time point, spatial position is even more important as improper placement will make the insertion intrusive.

5.  Framework  In our work we perform LAR detection with shot as the spatio-temporal context.

6. static saliency map + motion saliency map  Attention analysis : static saliency map + motion saliency map  A region contains more perceptive information than a pixel or a block and can be obtained by image segmentation ▪ Image segmentation DENSITY-BASED CLUSTERING [12] [12] Q. Ye, W. Gao, W. Zeng, Color Image Segmentation Using Density-Based Clustering, ICME 2003.

7.  Attention analysis  Static saliency [5] H. Liu, S. Jiang, Q. Huang, C. Xu, and W. Gao. “Region-Based Visual Attention Analysis with Its Application in Image Browsing on Small Displays”. ACM Multimedia,2007 feature contrast area factor spatial distance between the two regions adjacency degree central effect

8.  Attention analysis  Motion saliency [8] ▪ The MVS visualizes the MVF and provides us with a visual aid to understand. [8] L-Y. Duan, M. Xu, Q. Tian, C-S. Xu, J. S. Jin, “A Unified Framework for Semantic Shot Classification in Sports Video” IEEE Multimedia,2005 Angle Magnitude Texture Hue Saturation Value

9. information map + entropy map  Information analysis : information map + entropy map  The necessity of a region is evaluated by its information and entropy in our work.  An LAR must supplies less information and contains less entropy at the same time.  In this paper a simple and effective method is adopted to calculate the information and entropy of each region and the result is used to evaluate the region’s necessity.

10.  Information can be calculated as:  Entropy of the region can be calculated as: The final LAR should be of less information and less entropy. H :the normalized accumulative histogram of a shot h :the normalized histogram of a Region A : the region’s area

11.  Fusing the 4 maps generates the frame attention map. (a): The previous frame (b): Current frame (c): Motion field (d):Segmentation result (e): Static saliency map (f ): Motion intensity (g): Texture by DCT AC energy (h): HSV image of motion vector (i): Motion saliency map (j): Information map. (k): Entropy map (l): Final attentive map

12.  Method Overview  The shots here are classified into field view, player view (coach, referee) and audience view.  Filed view and player view shots take up more than 90% of the video. So we perform LAR detection and VCI only on filed view and player view shots.  For player view shots LAR can be detected by using the bottom up method.

13.  Field View LAR Detection  Playfield detection, line and curve detection, as well as object detection [2] can be combined to detect the LAR.  The GMM based method [9] is adopted to detect the playfield. [2]C. Xu, K. W. Wan, S. H. Bui, Q. Tian, “Implanting Virtual Advertisement into Broadcast Soccer Video”, PCM, 2004 [9] S. Jiang, Q. Ye, W. Gao, T. Huang. “A New Method to segment Playfield and Its Applications in Match Analysis in Sports Video”. ACM Multimedia, 2004

14.  The methods for VCI include  Static insertion ▪ The content floats over the original video.  Dynamic insertion ▪ Merge the content into the background. ▪ Camera parameter must be reconstructed and the content is adapted to the camera metric [10]. input Static insertion dynamic insertion [10] X. Yu, X. Yan, T. T. P. Chi and L. F. Cheong, “Inserting 3D projected virtual content into broadcast tennis video”, ACM Multimedia, 2006

15.  Test sequences (MPEG format)  Broadcast TV play series (22 minutes)  Sports 1 soccer video and (45 minutes)  Sports 2 tennis videos (26minutes)  For the field view shots, we insert the virtual content using dynamic method.  The VCI method presented in before estimates the camera motion and adapts the VC to the camera metric.

16. Static insertion dynamic insertion

17.  In this paper, an LAR detection method is proposed using visual attention analysis and information theory.  In the future we will construct an integrated system of VCI and the temporal position for VCI.