Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline Rousseau IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 5, MAY 2011

 Introduction  Our System and data set  Falls Characteristics  Shape deformation ▪ mean matching cost ▪ full Procrustes distance  Fall Detection Using GMM  Experimental Results  Conclusion

 Establish new healthcare systems to ensure the safety of elderly people at home.  Falls are one of the major risks for old people living alone.  Fall detection wearable sensor:  Accelerometers or help buttons Problem: -forget to wear -unconscious after the fall -recharged regularly

 Computer vision systems has overcome these problems.  A camera provides a vast amount of information on his/her environment ▪ Monocular Systems ▪ Bounding box[8] ▪ Only placed sideways ▪ Occluding objects ▪ Multi-Camera Systems ▪ Auvinet et al.[17] reconstructed 3-D silhouette of an elderly person ▪ Need to be calibrated ▪ The video sequences need to be synchronize

 Uncalibrated multi-camera system  Low-cost IP cameras, 30 frames/s, 720 × 480 pixels  Wide angle to cover all the room

Total of 75 different events, more than 12 min

1. Lack of significative movement 2. A lying position 3. A person lying on the ground 4. Vertical speed 5. An impact shock 6. Body shape change

 The silhouette is extracted by a background subtraction  N = 250 landmarks * Canny edge detector[25]

 Shape context[20] is a way of describing shapes. Matching cost for pair (p i, q j ):, K=5*12 bins

 Minimizing the total matching cost given a permutation π (i)  Use the Hungarian algorithm[27] for bipartite matching  Time complexity: O(n^3)  Bad landmarks due to segmentation errors or partial occlusions ▪ Add dummy points (not easy to choose). ▪ Match only the most reliable points in our implement (min i Cij = min j Cij)  mean matching cost: ij bipartite graph N ∗ : the total number of best matching points.

 Procrustes analysis [21] has been widely used to compare shapes.  Detect abnormal shape deformation for fall detection ▪ Step1 : image registration(one translation, no rotation, no scaling) ▪ Step2: Compute full Procrustes distance for compare. centered landmarks Zc : Z Zc two centered vectors : v = (v1, · · ·, vk) w = (w1, · · ·,wk). full Procrustes distance :

 mean matching cost  full Procrustes distance  Consider 2 feature (F1, F2) 1) F1 representing the fall : F1 will high in case of fall 1) F1 representing the fall : F1 will high in case of fall 2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low 2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low

 Model normal activity data with a Gaussian Mixture Model(GMM).  GMM: weighted sum of Gaussian(normal) distributions  M : the number of components in the mixture  P (j) : the mixing coefficients  The jth Gaussian probability density function p (x | j) ▪ d is the dimensionality of the input space expectation-maximization (EM) algorithm by maximizing the data likelihood expectation-maximization (EM) algorithm by maximizing the data likelihood GMM Classifier : only tell normal or abnormal! GMM Classifier : only tell normal or abnormal!

 Leave-One-Out Cross-Validation 1. Divided the dataset into N video sequences 2. One sequence is removed 3. Training using the N − 1 remaining sequences (falls are deleted) 4. This sequence is classified with the resulting GMM. 5. Repeat N times 6. Count the number of errors, classification error rate

1. True Positives (TP): falls correctly detected; 2. False Negatives (FN): falls not detected; 3. False Positives (FP): normal activities detected as a fall; 4. True Negatives (TN): normal activities not detected as a fall; 5. Sensitivity: Se = TP/ (TP + FN); 6. Specificity: Sp = TN/ (TN + FP); 7. Accuracy: Ac = (TP+TN) / (TP+TN+FP+FN) ; 8. Classification error rate: Er = (FN+FP) / (TP+TN+FP+FN).

 Shape matching : C++ using the OpenCV library [33]  Fall detection : MATLAB using the NETLAB toolbox [32] to perform the GMM classification.  The original video sequences frame : 30 frames/s  5 frames/s was sufficient to detect a fall  Intel Core 2 Duo processor (2.4 GHz)  The computational time of the shape matching step is about 200 ms

 train a GMM with three components for our experiment.

 Normalize training data.  Detection threshold depends on the sensitivity.

false positives true positives

 Simply majority vote on all cameras (>= 3 vote)  In fig. 9 : error rate 10%  2.7%

 We presented a new GMM classification method to detect falls  By analyzing human shape deformation  Robust to large occlusions and other segmentation difficulties