1. Activity Recognition from User- Annotated Acceleration Data Presented by James Reinebold CSCI 546

2. Outline Motivation Experiment Design Classification Methods Used Results Conclusion Critique

3. Motivation Can we recognize human activities based on mobile sensor data? Applications – Medicine – Fitness – Security

4. Related Work Recognition of gait pace and incline [Aminan, et. al. 1995] Sedentary vs. vigorous activities [Welk and Differding 2000] Unsupervised learning [Krause, et. al. 2003]

5. Scientifically Meaningful Data Most research is done in highly controlled experiments. – Occasionally the test subjects are the researchers themselves! – Can we generalize to the real world? Noisy Inconsistent Sensors must be practical We need ecologically valid results.

6. Experiment Design Semi-Naturalistic, User-Driven Data Collection – Obstacle course / worksheet – No researcher supervision while subjects performed the tasks Timer synchronization Discard data within 10 seconds of start and finish time for activities

7. Experiment Design (2) Source: Bao 2004

8. Sensors Used Five ADXL210E accelerometers (manufactured by Analog Devices) – Range of +/- 10g – 5mm x 5mm x 2mm – Low Power, Low Cost – Measures both static and dynamic acceleration “Hoarder Board” Source: http://vadim.oversigma.com/Hoarder/LayoutFront.htm

9. Activities Walking Sitting and Relaxing Standing Still Watching TV Running Stretching Scrubbing Folding Laundry Brushing Teeth Riding Elevator Walking Carrying Items Working on Computer Eating or Drinking Reading Bicycling Strength-training Vacuuming Lying down & relaxing Climbing stairs Riding escalator

10. Example Signals Source: Bao 2004

11. Activity Recognition Algorithm FFT-based feature computation – Sample at 76.25 Hz – 512 sample windows – Extract mean energy, entropy, and correlation features Classifier algorithms – All supervised learning techniques

12. Source: Bao 2004

13. Naïve Bayes Classifier Multiplies the probability of an observed datapoint by looking at the priority probabilities that encompass the training set. – P(B|A) = P(A|B) * P(B) / P(A) Assumes that each of the features are independent. Relatively fast. Source: cis.poly.edu/~mleung/FRE7851/f07/naiveBayesianClassifier.pdf

14. Nearest Neighbor Split up the domain into various dimensions, with each dimension corresponding to a feature. Classify an unknown point by having its K nearest neighbors “vote” on who it belongs to. Simple, easy to implement algorithm. Does not work well when there are no clusters. Source: http://pages.cs.wisc.edu/~dyer/cs540/notes/learning.html

15. Nearest Neighbor Example

16. Decision Trees Make a tree where the non-leaf nodes are the features, and each leaf node is a classification. Each edge of the tree represents a value range of the feature. Move through the tree until you arrive at a leaf node Generally, the smaller the tree the better. – Finding the smallest is NP-Hard Source: http://pages.cs.wisc.edu/~dyer/cs540/notes/learning.html

17. Decision Tree Example Weight Friendliness Dog Goat Cat < 20 pounds>= 20 pounds Not friendly Friendly

18. Results Decision tree was the best performer, but… ClassifierUser-specific TrainingLeave-one-subject-out Training Decision Table36.32 +/- 14.50146.75 +/- 9.296 Nearest Neighbor69.21 +/- 6.82282.70 +/- 6.416 Decision Tree71.58 +/- 7.43884.26 +/- 5.178 Naïve Bayes34.94 +/- 5.81852.35 +/- 1.690

20. Trying With Less Sensors Accelerometer (s) Left InDifference in Recognition Activity Hip-34.12 +/- 7.115 Wrist-51.99 +/- 12.194 Arm-63.65 +/- 13.143 Ankle-37.08 +/- 7.601 Thigh-29.47 +/- 4.855 Thigh and Wrist-3.27 +/- 1.062 Hip and Wrist-4.78 +/- 1.331

21. Conclusion Accelerometers can be used to affectively distinguish between everyday activities. Decision trees and nearest neighbor algorithms are good choices for activity recognition. Some sensor locations are more important than others.

22. Critique - Strengths Ecological validity – Devices cannot just work in the lab, they have to live in the real world. Variety of classifiers used Decent sample size

23. Critique - Weaknesses Lack of supervision Practicality of wearing five sensors Post-processing? Why only accelerometers? – Heart rate – Respiration rate – Skin conductance – Microphone – Etc..

24. Sources www.analog.com http://vadim.oversigma.com/Hoarder/Hoarde r.htm http://vadim.oversigma.com/Hoarder/Hoarde r.htm http://pages.cs.wisc.edu/~dyer/cs540/notes/l earning.html http://pages.cs.wisc.edu/~dyer/cs540/notes/l earning.html cis.poly.edu/~mleung/FRE7851/f07/naiveBay esianClassifier.pdf