Face Processing System Presented by: Harvest Jang Group meeting Fall 2002

Outline Introduction System Architecture Pre-processing Face detection Face tracking Problems Future work

Introduction An automatic system is needed to provide more human-computer interactive services to improve the life style Security System Interactive application/game Robot visualization This is a challenging task for integrating different techniques face detection face tracking face recognition

System Architecture Three main components Pre-processing Face detection Face tracking Fig 1: System architecture

Pre-processing 1. Transform to YCrCb color space 2. Use ellipse color model to locate the fresh color Fig 2: 2D projection in the CrCb subspace (gray dots represent skin color samples and black dots represent non-skin tone color)

Pre-processing 3. Perform morphological operation to reduce noise 4. Skin segmentation to find face candidates Fig 3: Pre-processing step (a) binary skin mask, (b) original images (c) binary skin mask after morphological operation and (d) Face candidates

Comparison on three face detection approach Motivation for the comparison Using different infrastructure systems for detecting faces in a large image Only shows one result pair of detection rate and false alarm rate Using experimental result to compare Sparse Network of Winnows (SNoW) Support Vector Machines (SVM) Neural Network

Comparison on three face detection approach Sparse Network of Winnows (SNoW) Primitive features architecture Winnow update rule learner’s prediction correct response update actionupdate name 1 0 w i :  w i /  if x i  1 w i unchanged if x i  0 demotion step 0 1 w i :  w i if x i  1 w i unchanged if x i  0  >1 fixed parameter promotion step Table 1: Winnow update rule

Comparison on three face detection approach Support Vector Machines (SVM) Find a hyperplane that leaves the maximum margin between two classes which will have the smallest generalization error The margin is defined as the sum of the distances of the hyperplane from the closest point of the two classes Neural Network Back-propagation method x1x1 x2x2 xnxn w1w1 w2w2 wnwn  x 0 = 1 w0w0 Fig 4: SVM Fig 5: Neural Network

Comparison on three face detection approaches CBCL face database from MIT Training set (2429 face pattern, 4548 non-face pattern with 19x19 pixel) Testing set (472 face pattern, non-face pattern with 19x19 pixel) To better represent the detectability of each model, ROC curve is used to replace single point of criterion response

Comparison on three face detection approaches Similar signal for SNoW Neural Network SVM model with linear kernel SVM model with polynomial kernel much stronger signal the classification between face and non- face patterns is much better Fig: 6: ROC curves of different face detection method

Comparison on three face detection approaches In our system, success to detect face is more important than processing more windows Face tracking method to help for tracking the detected face SVM model with polynomial kernel of second degree is chosen. (in sec)NNSNoWSVM with linearSVM with polynomial Mean S.T.D Table 2: The average processing time of each model for testing images ten times

Face detection To detect different size of faces, the region is resized to various scales A 19x19 search window is searching around the re- sized regions Histogram equalization is performed to the search window After the first presence of a face at the larger scale, face recognition using SVM classifier will be performed to retrieve the information of that person.

Face detection - Example Histogram equalization Transform to various scale Apply a 19x19 search window

Face tracking Why using face tracking Face detection method has missing rate and false alarm rate The missing rate and false alarm rate for losing an object will be reduced The performance speed is much better than face detection method Conditional Density Propagation – Condensation algorithm Sampling-based tracking method

The posterior probability density at timestep t using a set of N random samples, denoted as with weights There are three phases to compute the density iteratively at each time step t for the set of random samples to track the movement Face tracking

Selection phase: The element with high weights in the set has a higher probability to be chosen to the predictive steps Predictive phase: The sample setfor the new time-step is generated by independent Brownian motion The weights are approximately from the effective prior density for time-step t+1

Face tracking Measurement (Update) phase Given in terms of likelihood The model is expressed as a histogram for face color HUE (HSV color space) Calculate the measurement where is a normalization factor

Face tracking Fig 7: One time step in the Condensation algorithm

Face tracking The color input image is being masked by the binary skin mask reduce the localization period converge less sensitive from the background noise converge to the face boundary much faster.

Problems The face detection rate is still low The condensation algorithm doesn’t provide verification for the tracking object Two adjacent object with similar color will cause problem

Future work Improve the accuracy of face detection Implement the face tracking verification step using face recognition

Thank you!