Player Action Recognition in Broadcast Tennis Video with Applications to Semantic Analysis of Sport Game Guangyu Zhu, Changsheng Xu Qingming Huang, Wen Gao Liyuan Xing

Outline Introduction Framework Overview Player Action Recognition Video Analysis Experimental Results

Introduction Semantic gap –between user semantics and low-level feature –Object in sports video can consider as an effective mid-level representation Action Recognition –Far-view –Foreside-swing backside-Swing

Introduction Multimodal Framework –Action recognition method based on motion analysis –High-level analysis Video Indexing Highlight ranking Tactic analysis

Framework Overview Sports video database Low-level analysis Middle-level analysis Fusion scheme High-level analysis

Framework Overview

Low-level Analysis Dominant color-based algorithm in [16] was used to identify all the in-play shots

Player Action Recognition Related Work –Shah[8], Gavrila[9] recognition with close-up views –Motion representation Motion history/energy image [12] Spatial arrangement of moving points [13] Several Constraints –Efroes[11] Motion descriptor in a spatio-temporal volume NNC similarity measure –Miyamori[14][15] Base on silhouette transition Appearance feature is not preserved across videos

Player Action Recognition

Player Tracking and Stabilization Player Tracking –Initial position: detection algo. in [16] –SVR particle filter [24] Player region centroid

Optical Flow Computation Background subtraction

Optical Flow Computation Noise elimination

Local Motion Representation S-OFHs –slice based optical flow histogram The prob. of bin(u) The prob. of bin(u) in slice

Local Motion Representation Two slice of the figure is used Horizontal and vertical optical field is used

Action Classification Using SVM The concatenation of four S-OFHs is fed as feature vector Audio keywords –Silence, hitting ball, applause

Action Classification Action clip window is set to 25 frames Voting Strategy

Video Analysis Fusion of mid-level features Action Based Tennis Video Indexing Highlights Ranking and Browsing Tactics Analysis and Statistics

Video Indexing Based on action recognition and domain knowledge

Highlights Ranking Player action recognition Real-world trajectory computation

Highlights Ranking Affective Features(4 for this paper) Features on action –Swing Switching Rate

Highlights Ranking Features on trajectory –Speed of Player (SOP) –Maximum Covered Court The rectangle shaped with left most, rightmost, topmost, and bottommost points –Direction Switching Rate

Highlights Ranking The feature vector comprised of four affective features is fed into the ranking model Support vector regression User defined threshold

Tactics Analysis and Statictics

Experimental Results Action Recognition (6 seq, 194 clips)

Experimental Results Video Indexing

Experimental Results Highlights ranking

Experimental Results

Future Work More effective slice partition Involve more semantic action –Ex. Overhead-swing Action recognition apply to more applications such as 3-D scene reconstruction Include the ranking accuracy by combining audio features

Thank You