Activity Recognition from User- Annotated Acceleration Data Presented by James Reinebold CSCI 546

Outline Motivation Experiment Design Classification Methods Used Results Conclusion Critique

Motivation Can we recognize human activities based on mobile sensor data? Applications – Medicine – Fitness – Security

Related Work Recognition of gait pace and incline [Aminan, et. al. 1995] Sedentary vs. vigorous activities [Welk and Differding 2000] Unsupervised learning [Krause, et. al. 2003]

Scientifically Meaningful Data Most research is done in highly controlled experiments. – Occasionally the test subjects are the researchers themselves! – Can we generalize to the real world? Noisy Inconsistent Sensors must be practical We need ecologically valid results.

Experiment Design Semi-Naturalistic, User-Driven Data Collection – Obstacle course / worksheet – No researcher supervision while subjects performed the tasks Timer synchronization Discard data within 10 seconds of start and finish time for activities

Experiment Design (2) Source: Bao 2004

Sensors Used Five ADXL210E accelerometers (manufactured by Analog Devices) – Range of +/- 10g – 5mm x 5mm x 2mm – Low Power, Low Cost – Measures both static and dynamic acceleration “Hoarder Board” Source:

Activities Walking Sitting and Relaxing Standing Still Watching TV Running Stretching Scrubbing Folding Laundry Brushing Teeth Riding Elevator Walking Carrying Items Working on Computer Eating or Drinking Reading Bicycling Strength-training Vacuuming Lying down & relaxing Climbing stairs Riding escalator

Example Signals Source: Bao 2004

Activity Recognition Algorithm FFT-based feature computation – Sample at Hz – 512 sample windows – Extract mean energy, entropy, and correlation features Classifier algorithms – All supervised learning techniques

Source: Bao 2004

Naïve Bayes Classifier Multiplies the probability of an observed datapoint by looking at the priority probabilities that encompass the training set. – P(B|A) = P(A|B) * P(B) / P(A) Assumes that each of the features are independent. Relatively fast. Source: cis.poly.edu/~mleung/FRE7851/f07/naiveBayesianClassifier.pdf

Nearest Neighbor Split up the domain into various dimensions, with each dimension corresponding to a feature. Classify an unknown point by having its K nearest neighbors “vote” on who it belongs to. Simple, easy to implement algorithm. Does not work well when there are no clusters. Source:

Nearest Neighbor Example

Decision Trees Make a tree where the non-leaf nodes are the features, and each leaf node is a classification. Each edge of the tree represents a value range of the feature. Move through the tree until you arrive at a leaf node Generally, the smaller the tree the better. – Finding the smallest is NP-Hard Source:

Decision Tree Example Weight Friendliness Dog Goat Cat < 20 pounds>= 20 pounds Not friendly Friendly

Results Decision tree was the best performer, but… ClassifierUser-specific TrainingLeave-one-subject-out Training Decision Table / / Nearest Neighbor / / Decision Tree / / Naïve Bayes / /

Trying With Less Sensors Accelerometer (s) Left InDifference in Recognition Activity Hip / Wrist / Arm / Ankle / Thigh / Thigh and Wrist / Hip and Wrist /

Conclusion Accelerometers can be used to affectively distinguish between everyday activities. Decision trees and nearest neighbor algorithms are good choices for activity recognition. Some sensor locations are more important than others.

Critique - Strengths Ecological validity – Devices cannot just work in the lab, they have to live in the real world. Variety of classifiers used Decent sample size

Critique - Weaknesses Lack of supervision Practicality of wearing five sensors Post-processing? Why only accelerometers? – Heart rate – Respiration rate – Skin conductance – Microphone – Etc..

Sources r.htm r.htm earning.html earning.html cis.poly.edu/~mleung/FRE7851/f07/naiveBay esianClassifier.pdf