Recognition of Human Gait From Video Rong Zhang, C. Vogler, and D. Metaxas Computational Biomedicine Imaging and Modeling Center Rutgers University

Outline Motivation Distinguishing features Recognition process Silhouette extraction Human model initialization Extracting joint angles over image sequences Recognition Preliminary Results

Motivation The goal is to detect and identify humans by the way they walk. The walking pattern (gait) is unique enough to identify a person. Such capabilities will enhance: Human identification. Abnormal behavior detection.

Gait Cycle

Distinguishing features Features that seem unique to each person: Joint angle between the upper and lower legs Relationship between the knee joints and the feet over time Elevation of knee joint over the ankle (i.e., vertical distance between knee and ankle) shows a distinctive temporal pattern

Elevation over ankle is distinctive Transition from swing leg to stance leg is noticeably different across different people over time

Gait Recognition Procedure Image sequences Background image Silhouette images Joint angles Human model Recognition

Silhouette Extraction Result Image Background After background subtractionFinal result

Human Model Human is modeled by five connected trapezoids. Each trapezoid (body part) is represented by lili r 1i r 2i

Human Model Each configuration of human body is represented by where, and c as the center of the body.

Human Model Initialization – Result

2D Model-based Human tracking Previous methods Cardboard person model Scaled Prismatic Model Twist and exponential maps Condensation-Based Our approach Tracking via Gibbs sampling(probabilistic) Advantages Able to handle occlusion implicitly Has greater chance of avoiding local optima

Tracking Results

Recognition Collect feature vector with: Elevation over ankle Joint angles between upper and lower leg Use left-right hidden Markov models for recognition One HMM per person, trained on a minimum of 4-5 full step cycles from that person

Recognition (continued) Use algorithm similar to isolated speech recognition to identify people: Collect a step cycle from test subject For each HMM in the database, compute likelihood that it matches signal of this step cycle Select HMM with maximum likelihood Person corresponding to that HMM is identified subject

Experiment Two sequence sets are taken at two locations: one in a parking lot, one in front of a building Each set contains 3 persons walking sequences

Preliminary Results Perfect recognition scores across 3 subjects 5-6 step cycles per subject collected from the computer vision algorithm Use of HMMs with knee elevation and joint angles as features holds promise More work is needed to identify other distinguishing features

Thank You!