1. A Practical Approach to Recognizing Physical Activities Jonathan Lester, Tanzeem Choudhury, and Gaetano Borriello In Proceedings of the Fourth International Conference on Pervasive Computing (2006) Benjamin Stokes, Presenter -- 1/24/11 For CS 546: Intelligent Embedded Systems

2. The Challenge Personal activity recognition (in highly constrained use contexts) Healthcare sector: demand growing, currently relies on paid observer or self- reporting; Deficiencies: cost, accuracy, scope, coverage and obtrusiveness.

3. Proposed Solution To investigate three pressing constraints: 1.Unpredictable sensor location (wrist, waist, shoulder) 2.Minimal training across individuals 3.Cost (and sensor) minimizing

4. Technical Preview

5. Related Links MSP Research Initiative @ University of Washington & Intel Research Seattle: http://seattle.intel-research.net/MSP/ http://seattle.intel-research.net/MSP/ Wiki for the Mobile Sensing Platform http://ubi.cs.washington.edu/wiki/index.php/Main_Page http://ubi.cs.washington.edu/wiki/index.php/Main_Page Jonathan Lester research page (with details on the Mobile Sensing Platform) http://www.cs.washington.edu/homes/jlester/research.html http://www.cs.washington.edu/homes/jlester/research.html –Includes a great white paper on their MSP design justification and experience

6. Experimental Design 12 volunteers given a sequence of activities over several days (observer annotates true event) 8 different activities (sitting, standing, walking, walking up/down stairs, riding elevator, brushing teeth – selected as useful for elder care) Recognition trained & tested via activity classification algorithm developed earlier

7. Technical Design

8. Available Sensors

9. Raw Data (12 hrs gathered) (Data from 2nd Data Set)

10. Data Preprocessing 18,000 samples of data per second  so must summarize… Result: 4 Hz @ 651 features

11. Analysis Stage: Classification Window: 15 second sliding (decreases error; reveals transitions) with 5 second overlap For each activity… True activity type is observed by a human All events of that type are divided into 4 “folds” (for training/testing, i.e., 3 :1)

12. Investigating Sensor Location Several options: 1. Any location (of three) = ideal …or… 2.Shoulder only 3.Waist only 4.Wrist only

13. Learning Model Often, learning models are either: a)discriminative: to learn the class boundaries without regard for densities b)generative: to learn the class densities …they have a mix. Specifically: 1.Top 50 features as most discriminating (< 10%) 2.To recognize activities, Hidden Markov Models (HMMs – i.e., a simple Bayesian network); includes “temporal smoothing” (Image Source: Wikipedia)

14. Confusion Matrix (for the “location independent” condition) Precision/recall

15. Variation Across Users Can anyone benefit? How much training? Train on 1-12 users (folds 3:1), test on all 12 Approaches 80% accuracy if testing on outsiders (approaches 95% accuracy if tested within group)

16. Fewer sensors possible? Which are most important? Accelerometer (motion of user) Audio (changing environment) Barometric pressure (env.; in GPS for altitude) Compare best sensor : top three (all locations) 38.96% recall : 81.38% …so use three!!

17. Findings Summary Investigated… (1) location sensitivity, Can recognize context within our constraints! (and works across locations) (2) variations across users, Can be pre-trained by other individuals. (3) which sensor modalities. Can use fewer & cheaper sensors.

18. Critiques, Future Research Curious: Defensive about dual analysis techniques? Limitation: Excluded unclassified activities (Overlooks 5 of 12 hours  low ambiguity tolerance.) Conceptual need: meta-classification to connect activities (e.g., “making the rounds in hospital”) Suggestion: Cluster population groups for performance. (Here it was just “healthy individuals.”) Suggestion: Consider time series data? (e.g., sitting typically followed by standing, which precedes walking)

19. Related Links MSP Research Initiative @ University of Washington & Intel Research Seattle: http://seattle.intel-research.net/MSP/ http://seattle.intel-research.net/MSP/ Wiki for the Mobile Sensing Platform http://ubi.cs.washington.edu/wiki/index.php/Main_Page http://ubi.cs.washington.edu/wiki/index.php/Main_Page Jonathan Lester research page (with details on the Mobile Sensing Platform) http://www.cs.washington.edu/homes/jlester/research.html http://www.cs.washington.edu/homes/jlester/research.html –Includes a great white paper on their MSP design justification and experience