1. Feature Extraction Spring Semester, 2010

2. Accelerometer Based Gestural Control of Browser Applications M. Kauppila et al., In Proc. of Int. Workshop on Real Field Identification, UCS 2007, pp. 2-17, 2007.

3. Outline 4 Motivation 4 Previous Work 4 H/W Architecture 4 Recognizer 4 Experiments 4 Discussion

4. Motivation 4 Large screen interface –Train, bus stations, marketing places, and other public places –Provide different kinds of interactions between the users and the displays

5. Previous Work 4 DBN, SVM –Samsung AIT, “Two-stage Recognition of Raw Acceleration Signals for 3-D Gesture- Understanding Cell Phones,” 2006. Classify {0~9, O, X} by using DBN SVM is used for the confusing pair of (6, O)

6. Communication Architecture 4 Scenario 4 Data flow

7. Communication Architecture 4 Accelerometers –Developed at our university –Send data at 50Hz 4 UPnP virtual sensor 4 Use a simple asynchronous Java API: decouple the recognizer and its clients

8. Browser Control 4 General browser / photo album app.

9. Recognizer 4 Segmentation 4 Preprocessing and normalization 4 Classification

10. Segmentation 4 Feature vector for segmentation Continuous acceleration signal: Discretized acceleration signal: Approximated derivative: Approximated velocity:  4 Two-state (non-gestural / gestural) HMM Recognizer

11. Segmentation Example Recognizer

12. Preprocessing 4 Sensor model –Dynamic component (gesture): a d (t) –Static component (gravity): a s =(0, 0, g) T –Measured acceleration: R: orthogonal matrix (Describing the orientation of the sensor) 4 Gravity estimation –Ra s : Mean of the measured acceleration Recognizer

13. Preprocessing 4 Tilt Compensation –Let u = v/|v|, where v is the axis of rotation –Remember the measured acceleration –Finally, Recognizer

14. Normalization 4 Power normalization –Frobenius normalization 4 Tempo normalization –Rescale the gesture tempo so that all gestures have 30 samples Downscaling: Box filtering Upscaling: linear interpolation Recognizer

15. Classification 4 Training set –A single person, 16 samples per gesture 4 Recognizer –12-state hidden Markov model per gesture –Choose the gesture class corresponding to the HMM with the highest score Recognizer

16. Experiments 4 11 Subjects 4 Confusion matrix

17. User Study 4 Three stages + a questionnaire (feedback) –Blind stage Freely use the system without any prior training –Task stage Solve a specific browsing task after training –Photo album stage Experiments

18. Subject Feedback 4 Five questions 4 Free-worded feedback –Stress of hands –Unintuitivity and learning overhead Experiments

19. Discussion 4 False positives still pose a problem –Rejection mechanism is needed 4 Recoiling problem –Avoiding the use of overly simplistic gestures

20. Human Activity Recognition with User-Free Accelerometers in the Sensor Networks S. Wang, et al., Int. Conf. Neural Networks and Brain, 2005. pp. 1212-1217, 2005.

21. Outline 4 Motivation 4 Feature Extraction 4 Classification 4 Experiments 4 Summary

22. Motivation 4 Human’s activities can be represented from three aspects –Movements of human bodies –Movements of the objects associated with the activities –Person-object interaction 4 Wearing the sensors is uncomfortable for users

23. Feature Extraction 4 Using a sliding window with 50% overlap 4 19 features –Six features from each of the three axes Acceleration, mean, standard deviation (stability), energy (data periodicity), frequency-domain entropy, correlation  normalized into [-1.1] –One feature represents vibration of the sensor (|a x 2 +a y 2 +a z 2 -g 2 |)

24. Classification 4 Recognition algorithms –C4.5, MLP, SVM 4 Three types of tests –Self-consistency test: Training set = test set –Cross-validated test –Leave-one-subject-out validation

25. Experiments 4 System setup –Accelerometer: KXP74 (32Hz, -2g~+2g) –Fixed to the rear of the telephone receiver, base of the cup, and on the top of the pen 4 SVM-based feature selection

26. Data collection 4 Three activities was performed by four subjects –Drinking, phoning, and writing –Positive actions + negative actions Positive: Write on the table or on the blackboard Negative: Rotate the pen with fingers, … –Each lasted 5 minutes Experiments

27. Results 4 Self-consistency test: Accuracies > 95% 4 Cross-validated test 4 Leave-one-subject-out validation test Experiments

28. Discussion on Feature Selection 4 Attribute sequence –Drinking –Phoning –Writing A: Acceleration, E: Mean, S: Stdev., G: Energy, P: Entropy, C: Correlation, Delta: Vibration Experiments

29. Summary 4 Accelerometer based gestural control of browser applications –Segmentation feature Acceleration, derivative, and velocity –Tilt compensation 4 Human activity recognition with user-free accelerometers in the sensor networks –Feature extraction and selection