1. Recognition of Human Gait From Video Rong Zhang, C. Vogler, and D. Metaxas Computational Biomedicine Imaging and Modeling Center Rutgers University

2. Outline Motivation Distinguishing features Recognition process Silhouette extraction Human model initialization Extracting joint angles over image sequences Recognition Preliminary Results

3. Motivation The goal is to detect and identify humans by the way they walk. The walking pattern (gait) is unique enough to identify a person. Such capabilities will enhance: Human identification. Abnormal behavior detection.

4. Gait Cycle

5. Distinguishing features Features that seem unique to each person: Joint angle between the upper and lower legs Relationship between the knee joints and the feet over time Elevation of knee joint over the ankle (i.e., vertical distance between knee and ankle) shows a distinctive temporal pattern

6. Elevation over ankle is distinctive Transition from swing leg to stance leg is noticeably different across different people over time

7. Gait Recognition Procedure Image sequences Background image Silhouette images Joint angles Human model Recognition

8. Silhouette Extraction Result Image Background After background subtractionFinal result

9. Human Model Human is modeled by five connected trapezoids. Each trapezoid (body part) is represented by lili r 1i r 2i

10. Human Model Each configuration of human body is represented by where, and c as the center of the body.

11. Human Model Initialization – Result

12. 2D Model-based Human tracking Previous methods Cardboard person model Scaled Prismatic Model Twist and exponential maps Condensation-Based Our approach Tracking via Gibbs sampling(probabilistic) Advantages Able to handle occlusion implicitly Has greater chance of avoiding local optima

13. Tracking Results

15. Recognition Collect feature vector with: Elevation over ankle Joint angles between upper and lower leg Use left-right hidden Markov models for recognition One HMM per person, trained on a minimum of 4-5 full step cycles from that person

16. Recognition (continued) Use algorithm similar to isolated speech recognition to identify people: Collect a step cycle from test subject For each HMM in the database, compute likelihood that it matches signal of this step cycle Select HMM with maximum likelihood Person corresponding to that HMM is identified subject

17. Experiment Two sequence sets are taken at two locations: one in a parking lot, one in front of a building Each set contains 3 persons walking sequences

18. Preliminary Results Perfect recognition scores across 3 subjects 5-6 step cycles per subject collected from the computer vision algorithm Use of HMMs with knee elevation and joint angles as features holds promise More work is needed to identify other distinguishing features

19. Thank You!