1. Video Based Gait Analysis in Biometric person authentication Jani Rönkkönen

2. Gait in general Shortly gait means the walking style of a person Gait signature of each person is unique and thus can be used as a biometric To form a gait signature many different components like cadence or frequency of walking can be used Most common way of receiving gait information is by video cameras (others for example radar, pressure mat or motion sensors)

3. Current status Quite a lot of published articles in recent years Research is still mostly basic research No commercial solutions yet for authentication purposes (May be some medical applications) Most promising areas are medical and surveillance applications Georgia Institute of Technology is developing a method for recognizing people among crowd and estimate it could be commercialised in five years

4. Advantages Can be used with low resolution video sequences Target do not necessarily need to know about the surveillance Sequences can be taken from long distance Non intrusive Gait is not very easily conceivable biometric (although can be altered purposely)

5. Disadvantages The uniqueness of a persons gait signature is not proven with large datasets Not yet clear which components of gait signature are most useful A lot of data usually means high computational cost Gait may be changed purposely Conditions may affect the gait signature more than differences between subjects

6. Conditions Walking speed Affects cadence, stride lenght, frequency, pose and hand swings Walking surface If surface is not smooth and obstacle free gait pattern will no longer be repeatable an periodic Physical Conditions For example pregnancy, drunkness, fatigue or physical injury

7. Conditions Carrying a load Carrying a load affects both the gait dynamics and physical borders of a person Clothes Clothes alter the borders of a person and may hide some movement (a dress for example) footwear affect the gait dynamics (rubber boots vs high heels)

8. Camera conditions Camera angle The gait pattern is very different if looked from different angles Lightning conditions Shadows cause error in border or silhouette extraction Contrast between clothing and backround Too small contrast makes it harder to extract borders or sihouette

9. History G. Johansson used small light bulbs attached to a person to generate 2D motion patterns in 1973 He found out that people could recognize these to represent human movement Later it was discovered that humans could also recognize the gender of the walker or even their friends identity from these patterns A conclusion was drawn that gait could be used as a biometric

10. UMD database University of Maryland (UMD) database First dataset –25 persons and 4 camera angles Second dataset –55 persons and 2 camera angles

11. CMU database Carnegie Mellon University (CMU) database 25 persons and 6 camera angles Slow, fast and inclined walking style A walk holding a ball

12. USF and USH databases University of South Florida (USF) database 71 persons and 2 camera angles 2 different shoetypes and walking surfaces (concrete and grass) Walk holding a briefcase University of Southampton (USH) database 28 persons and 1 camera angle Uniform green backround to help extract clean silhouettes

13. Features Gait signature includes numerous components and even more features can be derived from these Not yet clear which features are most usefulfor authentication purposes Why not just simply use all there is?

14. Why not all features? Would be computationally costly, need much storage space and need complex algorithms Conditions affect different features differently –Some are more robust to changes Using more features may decrease performance –Bad ones only add unwanted noise All features are not always available

15. Methods Another question is how to use our feature(s) of choice for authentication? Typically there is three main steps in a gait recognition algorithm: Extracting the subject from the frames of video sequence (eg. silhouette) Extracting and modifying the wanted features (eg. PCA to simplify the data) Classification based on extracted features and somekind of decision (eg. KNN-classifier)

16. The width of outer contour The basic biometric is the width of the outer contour of binarized silhouette of a walking person Retains physical structure and swings of the limbs during walking The pose information is lost Smoothed and down sampled width vectors are used directly Also a velocity profile is extrated by calculating the difference of subsequent vidth vectors

17. Overlay of width vectors

18. Results UMD database rank 1: 80% and rank 5: 91.2% Velocity profile alone –rank 1: 56% and rank 5: 83% CMU database rank 1: over 95% and rank 5: 100% Fast vs slow –rank 1: 75% and rank 5: 87.5%

19. Results with USF database G = Grass C = Concrete A, B = Footwear R, L = Camera angle

20. Notes Both structural and dynamical information is important fo recognition Leg region is the most important Difference in walking surface causes a lot of problems to the method Walking speed is also an issue

21. Moments from silhouette Silhouette is divided in 7 parts For each part an ellipse is fitted Features: Centroid, Major axis, minor axis and Major axis orientation Height of the body Another testset used gait spectral component features received via fourier transform

22. Divided silhouette

23. Results Tested on dataset consisting of 24 persons Sequences were taken in 4 different days Sequences of one day were compared against other days Rank 1: 30% - 47% and rank 10%: 53%-94% With spectral component features Rank 1: 31% - 82% and rank 10%: 70%-97%

24. Results Also tested with CMU database Results were almost perfect (only one mistake) Third case was gender identification Support vector machines Best results using second degree polynomial kernel 94% correctness

25. Notes Most errors caused by clothing changes Spectral component features were more robust If only sequences taken in a same day were compared, spectral component features were slightly worse Notably good result of gender classification

26. Body shape and gait from silhouette The periodic dimensional changes in silhouette width are used to locate the key frames Key frames are compared to corresponding ones in training data Four subsequent comparison scores are amalgated and used for classification

27. Extracting the key frames

28. Results CMU database Rank 1: at least 92% Slow vs fast rank 1: 76% and rank 10%: 92% Second testset contained 25 persons with sequences taken in different days rank 1: 45% and rank 10%: 77%

29. Notes The main reason for failures (according to the author) were conditions that affected the quality of the silhouette Lightning conditions Clothes Hairstyles

30. Stride length and cadence The method makes following assumptions: Walking velocity is constant Persons walks a straight line for 10-15 seconds Camera is calibrated with respect to the ground plane Frame rate is greater than twice the walking frequency

31. Stride lenght and cadence The key is the periodicity of human walk The width of a sihouette is used to calculate the period From period a number of steps can be calculated Also the distance walked is measured Stride = distance/steps Cadence = steps/time

32. Results A database of 131 sequences from 17 persons

33. Notes The method (according to author) is robust to changes in: Lightning conditions Clothing Tracking error Also it is in principle wiev invariant, but the method used to calculate the frequency works best with fronto parallel sequences

34. Similarity plots A sequence of images of a walking person is mapped to a similarity plot (SP)

35. Similarity plots Each pixel in SP is a result of substracting two blobs SP has dark main diagonal, because comparing a blob to itself results to zero Is symmetric along main dianonal Is periodic, because from key poses A and C and B and D are close to each other

36. Results A dataset of 44 images of 6 persons from one camera angle 90% accuracy with rank 1 The second dataset consisted of 400 sequences of 7 peoples from 8 camera angles and 7 different days 65% accuracy with rank 1

37. Notes The method is view dependent and performs best with fronto-parallel sequences Changes in clothing and lightning affect performance The author used binary blobs and gray level blobs with and without backround Best results with binary blobs, worst with gray level blobs with backround

38. Thigh and lower leg rotation A sobel edge operator is used to obtain the leading edge of a walking person

39. The model A model is then matched to the edge

40. Phase weighted magnitude A phase weighted magnitude (PWM) is calculated from this with the help of fourier transform

41. Results A dataset of 20 persons walkin and running In addition to clean edged images, 25% grey scale random noise were added and also classification was done by decreasing the resolution (from 130*190 pixels to 65*95 pixels) Best results were achieved with clean edged running sequences rank 1: 91.7% and worst with noisy walking sequences 60.8%

42. Notes Reducing resolution did not reduce performance dramatically Adding noise reduced to worse results The reson for running sequences having slightly better identification accuracy was according to author the fact that there is more differeces between humans running styles than walking styles

43. Conclusions Rather good results of identification can already be acquired with small datasets and fixed conditions More robust methods are needed to achieve better accuracy in more general setup A question still remains if gait is unique enough with larger datasets