Guillaume-Alexandre Bilodeau Dynamic Selection of Exemplar-SVMs for Watch-List Screening Through Domain Adaptation Saman Bashbaghi Eric Granger Robert Sabourin Guillaume-Alexandre Bilodeau École de technologie supérieure, Université du Québec December 21, 2016

Outline Introduction Application, Challenges, Objectives, and Contributions Ensembles of Exemplar-SVMs Through Domain Adaptation Design phase (pool generation) Operational phase (dynamic selection and integration) Experimental Results COX-S2V data set Experimental protocol Simulation results Conclusions

1. Introduction Application Area Generic system for video-based face recognition

1. Introduction Application Area Video Surveillance applications detect the presence of target individuals of interest (enrolled to the system) based on their facial model types of systems (based on facial model): video-to-video FR (e.g., face re-identification, search and retrieval) still-to-video FR (e.g., watch-list screening) individual-specific detection among a restrained list of individuals [Pagano et al., 2012] Watch-List Screening recognize target individuals from watch-list images, under semi- and unconstrained environments [Bashbaghi et al., 2014] face model is defined as either a set of reference samples (stored in a gallery for template matching), or a set of classifier parameters estimated from reference samples (for pattern classification)

1. Introduction Challenges Limited and Imbalanced Data design with limited number of target reference face captures and an abundance of non-target references small number of target inviduals appearing during operations can estimate of parameters (thresholds, feature subsets, classifiers) with auxiliary data obtained from videos of non- target people in the scene Face Capture and Classification different cameras used to capture reference faces (still camera under controlled conditions) and operational faces (IP video camera under semi- or uncontrolled conditions) nuisance factors – variation in illumination, pose, scale, expression, occlusion, contrast, focus, sharpness, etc.

1. Introduction Objectives Design robust still-to-video FR systems for watch- list screening approaches: provide a novel dynamic ensemble selection method for still-to-video FR through individual-specific multi- classifier system exploit classification systems through domain adaptation with the availability of a single labeled target ROI obtained from a high-quality still reference and unlabeled non-target ROIs captured either from the cohort or low-quality calibration videos evaluate the performance of different training schemes and dynamic selection of ensembles

1. Introduction Contributions A robust still-to-video FR can be designed under SSPP using: Multiple and diverse face representations uniform patches Random Subspace Method provide ensemble diversity and improve robustness to various perturbation factors frequently occur in video surveillance environments Ensemble of e-SVMs through domain adaptation discriminate between a single labeled high-quality target ROI and abundant of unlabeled non-target ROIs from calibration videos Dynamic Classifier Selection An intuitive approach to provide the best separation w.r.t. non-target samples in the feature space that allows the system to select classifiers properly fit the acquisition conditions

2. Dynamic Ensemble of SVM Classifiers Operational system Block diagram of the proposed still-to-video FR system using dynamic ensemble of e-SVMs per target individual

2. Dynamic Ensemble of SVM Classifiers Design Phase (Generating a pool) Multiple Diverse Face Representations Uniform Patches local parts of face to provide diversity for ensembles are suitable for partially occluded faces Feature Extraction Technique HOG, LPQ Random Subspace Method allow for randomly sampling and selection different feature subsets from feature space of input samples Classification Systems Exemplar-SVM relies on a single labeled still face from the enrollment domain along with unlabeled faces in calibration videos from the operational domain

2. Dynamic Ensemble of SVM Classifiers Enrollment Phase (Training schemes) (a) Scheme 1 (b) Scheme 2 (c) Scheme 3 Illustration of e-SVMs in a 2D feature space trained using different classification schemes according. (a) a labeled target still vs labeled non-target still ROIs of Ed, (b) a labeled target still vs unlabeled non-target video ROIs of Od, and (c) a labeled target still vs camera-specific unlabeled non-target video ROIs.

2. Dynamic Ensemble of SVM Classifiers Operational Phase (Dynamic classifier selection) Illustration of the proposed dynamic classifier selection approach in a 2D feature space

3. Experimental Results Dataset COX-S2V dataset [Huang et al., 2013] 1000 subjects, where each subject has a high-quality still image, and four lower-quality facial trajectories (each trajectory has 25 faces of 16x20 and 48x60 resolutions) Example of the reference still of subject ID \#1 and some corresponding videos from the COX-S2V dataset

3. Experimental Results Experimental methodology Protocol for independent 5 trials Design phase (randomly select) 20 individuals in the watch-list (20 high-quality stills) 100 unknown person to form UBM (ROIs video capture) Operational phase (test mode) 100 other unknown persons (ROIs video capture) ROIs video capture of individuals of interest (one at a time) Measures: transaction-based analysis (matching performance) pAUC(20%), AUPROC Time Complexity: Number of dot products required by the ensemble of e-SVMs to process a probe

3. Experimental Results Transaction-level Average transaction-level performance and complexity of the proposed system against state-of-the-art s Average pAUC(20\%) and AUPR performance of different training schemes at transaction-level Average performance of the proposed system with or without dynamic selection

4. Conclusion Training schemes 1 is greatly outperformed by schemes 2 and 3, where calibration videos from operational domains are employed for domain adaptation to train e-SVMs In the scheme 2, videos from all of the cameras are employed to generate an e-SVM pool, while 4 camera- specific e-SVM pools are generated for scheme 3 using videos of each camera Dynamic selection can improve the performance instead of combining all of classifiers, where it is efficient and does not significantly increase the computational burden

Thanks for your attention