1. Silhouette Analysis-Based Gait Recognition for Human Identification
Liang Wang, Tieniu Tan, Huazhong Ning, and Weiming Hu
December 2003
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Presenter:
Xi Chen

2. Background and Research Problem
Biometrics is a technology that makes use of the physiological or behavioral characteristics to authenticate the identities of people.
The interest of vision-based human identification at a distance is driven by the need for automated person identification systems for visual surveillance and monitoring applications in security-sensitive environments such as banks, parking lots, and airports
So far, gait is probably the only perceivable biometric feature from a great distance. In comparison with other first-generation biometric features such as fingerprint and iris, gait has the advantage of being unobtrusive.
Gait recognition, also called gait-based human identifications, is a
relatively new research direction in biometrics, which aims to
discriminate individuals by the way they walk.

3. Related Work
Current gait recognition approaches may be classified into two main classes, model-based and motion-based methods.
Model-based approaches aim to model human body or motion and perform model matching in each frame of a walking sequence.
Motion-based approaches can be divided into two main classes. The first class, state-space method considers gait motion to be composed of a sequence of static body poses. The second class, spatiotemporal methods, characterizes the spatiotemporal distribution generated by gait motion in its continuum.
A number of approaches have already shown that it is possible to recognize people by gait.

4. Overview of Approach

5. Purpose and Contribution
The proposed method has several desirable properties:
It is easy to comprehend and implement.
It is insensitive to the color and texture of cloth as a silhouette-based approach.
Some additional features related to pace, stride, and build are used to improve recognition accuracy.
Experimental results demonstrate that it has a relatively low computational cost.

6. Human Detection and Tracking
Background Modeling:
The main assumption made here is that the camera is static,
and the only moving object in video sequences is the
walker.
Here, the LMedS (Least Median of Squares) method is used to
construct the background. Let I represent a sequence including
N images. The resulting background can be computed by:
Where p is the background brightness value to be determined for the pixel location (x,y), med represents the median value, and t represents the frame index ranging within 1-N.

7. Human Detection and Tracking
Differencing between the Background and the Current Image:
In this paper, the following extraction function is used to indirectly
perform differencing:
Where a(x,y) and b(x,y) are the brightness of current image and the
background at the pixel position (x,y).
This function can detect the change sensitivity of the difference value according to the brightness level of each pixel in the background image.

8. Human Detection and Tracking

9. Human Detection and Tracking
Postprocessing and Tracking:
Differencing process is independently performed for each
component R, G, and B. For a given pixel, if one of the three
components determines it as the changing point, then the pixel
will be set to the foreground.
No change detection algorithm is perfect. So, it is imperative to
remove as much noise and distortion as possible from the segmented
foreground. Morphological operators such as erosion and dilation
are used to further filter spurious pixels, and small holes inside the
extracted silhouettes are then filled.

10. Feature Extraction Silhouette Representation:
For the sake of computational efficiency, we convert these 2D silhouette
changes into an associated 1D signals to approximate temporal pattern
gait.
After the shape centroid is obtained, by choosing it as the reference origin, we can unwrap the outer contour counterclockwise to turn it into
a distance signal.

11. Feature Extraction
Silhouette Representation:

12. Feature Extraction Principal Component Analysis (PCA) Training:
The purpose of PCA training is to obtain several principal components
to represent the original gait features from a high-dimensional
measurement space to a low-dimensional eigenspace.
Given s classes for training, and each one represents a sequence of
distance signals of one subject’s gait.
Let Dij be the jth distance signal in class i and Ni the number of such
distance signals in the ith class. The total number of training samples is

13. Feature Extraction PCA Training:
The mean and the global covariance matrix of such a data set can be
calculated as:

14. Feature Extraction PCA Training:
We can compute the N nonzero eigenvalues and the associated
eigenvectors based on SVD (Singular Value Decomposition).
Considering the memory efficiency in practical applications, only those
bigger eigenvalues and corresponding eigenvectors are kept using a
threshold Ts=0.95.
Where Wk is the accumulated variance of the first k largest eigenvalues with respect to all eigenvalues.

15. Feature Extraction Projection to the Eigenspace:
Taking only k<N largest eigenvalues and their associated eigenvectores,
the transform matrix can be constructed to project an original
distance signal into a point in the k-dimensional eigenspace.
k is usually much smaller than then original data dimension N. That is
to say, eigenspace analysis can drastically reduce the dimensionality
of input samples. For each training sequence, the projection centroid
is given by:

16. Recognition Similarity Measures:
Spatiotemporal Correlation: for two input sequences, they are projected into and in the eigenspace. The similarity between two such vector sequences can be computed by:
Where is a dynamic time warping vector from with respect to time stretching and shifting for an approximation of the temporal alignment between the two sequences. The selection of the parameters a and b depends on the relative stride frequency and phase difference within a gait period respectively.

17. Recognition
Gait Period Analysis: serves to determine the frequency and phase of each observed sequence so as to align sequences before matching.

18. Recognition Similarity Measures:
The selection of the parameters a and b depends on the relative stride frequency and phase difference within a gait period respectively.
Let and denote the frequencies of the two gait sequences, then we can get
By cropping a subsequence of length from the second sequence vector and stretching it with a, we may obtain its correlation with
The minimum of all prominent valleys of the correlation results determines their similarity.

19. Recognition Similarity Measures:
(2) The Normalized Euclidean Distance (NED): the computational cost will increase quickly if the similarity is performed in the spatiotemporal domain, especially when time stretching and shifting is taken into account. The NED is used to measure the similarity of two gait sequences only with the projection centroids.
Each projection centroid implicitly represents a principal structural shape of certain subject in the eigenspace. The normalized Euclidean distance between the two sequential projection centroids can be defined by:

20. Recognition Classifier:
The classification process is carried out via two simple different classification methods, the nearest neighbor classifier (NN) and the nearest neighbor classifier with respect to class exemplars (ENN).
represents a test sequence and represents the ith reference sequence. This test sequence is classified into class that can minimize the similarity distance between the test sequence and all reference patterns by:

21. Experiments Data Acquisition:
A new gait database, called NLPR database, is established for the experiment.
All subjects walk along a straight-line path in three different views with respect to the image plane, laterally, obliquely, and frontally.
The resulting NLPR database includes 20 subjects and 4 sequences for each viewing angle per subject.

22. Experiments Preprocessing and Training:
For each image sequence, motion segmentation and tracking are performed to extract the silhouette of the walking subjects.
The extracted silhouettes are converted into an associated sequence of 1D distance signals before training and projection.

23. Experiments Preprocessing and Training:
The first 15 eigenvalues and their associated eigenvectors are kept to form the eigenspace transformation matrix.
Then each silhouette image can be mapped to one point in a 15-dimensional eigenspace.

24. Results and Analysis Identification Mode:
A useful classification performance measure, defined as the cumulative probability that the real class of a test measurement is among its top k matches, is used. To show the performance graphically, the rank k is plotted in the horizontal axis, and the vertical axis is the match score.
The leave-one-out cross-validation rule is used with the NLPR to estimate the performance of the proposed method.
After computing the similarity between the test sample and the training data, the NN and ENN are then applied for classification.

25. Results and Analysis
Identification Mode:

26. Results and Analysis Identification Mode:
The identification performance using NED is, in general, better than that using STC. In theory, STC should better capture spatiotemporal characteristics of gait motion than NED. This result probably due to the fact that segmentation errors maybe accumulated into a quick-enlarged match error.
The NED based on the exemplar projection centroid performs better than NED using only a single projection centroid.
The recognition performance under frontal walking is the best. This result is probably due to the averaging associated with the silhouette shape analysis because there are less severe variations of silhouette appearances in such gait patterns compared with other views.

27. Results and Analysis Verification Mode:
In this paper, the FAR (False Acceptance Rate) and FRR (False Reject Rate) are also estimated via the leave-one-out rule in verification mode.

28. Results and Analysis Validation Based on Physical Features:
In experiments, recognition errors often occurred when the two smallest values of similarity function are very close.
When the difference between the last two minima is lower than a predefined threshold, some additional features available from the training sequences are introduced to validate the final decision.
The features can be body height, build, and stride length, which can be obtained in the training process.

29. Results and Analysis
Validation Based on Physical Features:

30. Results and Analysis Validation Based on Physical Features:
The experiment results shows that the additional physical features improve the performance of the classifier without validation.

31. Comparisons
Compared with other algorithms, the algorithm proposed has better
performance and the advantage of lowest computational cost.

32. Discussion and Future Work
To provide a general approach to automatic person identification in
unconstrained environments, much remains to be done.
Further evaluation on a much larger and most-varied database is still
needed. We are planning to set up such a database with more subjects,
more sequences and more variations in conditions.
It is more efficient for recognition to extract dynamic information such
as the oscillatory trajectories of joints. Future work may try to combine
both static and dynamic features of gait for recognition.
Also, seeking better similarity measures, designing more sophisticated
classifiers, gait segmentation, and the evaluation of different scenarios
deserve more attention in future work.