1. A two-tier classifier for duty-cycling smartphone activity recognition systems
Vijay Srinivasan
Thomas Phan
ACM PhoneSense, November 6, 2012, Toronto, Canada

2. Background
Walking
Driving
Biking
Idle
Running
Current generation
on-device accelerometers
Sensor sampling
Signal processing
Classification model
Classification engine
Feature extraction
Real-time, on-device activity recognition
Smartphone activity recognition using accelerometers has many exciting use cases
Fitness and health tracking
Context driven UI
Smartphone activity recognition is the process of transforming raw sensor data to user understandable activities such as walking, driving, biking etc using a combination of signal processing and machine learning techniques.
In particular, smartphone activity recognition using accelerometers has many exciting use cases.
It can be used for tracking and visualizing the user’s health and fitness level
It can be used to trigger a change in the UI based on the user’s current activity; for example, driving mode UI can be automatically activated if we detect that the user is driving.

3. Problem statement
Activity recognition in commodity Samsung smartphones is power hungry
Bulk of the power consumption comes from keeping the smartphone awake to perform activity recognition
Problem: How to reduce the amount of time the smartphone stays awake (wake time) to respond to a query for current user activity?
Power consumption breakdown of our accelerometer-based activity recognition system
Power

4. Solution overview: Two-Tier classification approach
Two-Tier classifier
Tier I
Tier II
Two key insights
Performing accurate idle vs. non-idle classification requires a smaller wake time compared to full activity classification
Leverage user’s natural idle periods to save power
Example operation
Walking activity inferred using Tier II
Power consumption of 230 mW
Idle activity inferred using Tier I
Power
Activity recognition ON
Time
Power

5. Contributions
First approach to dynamically change the wake time of the activity recognition system to reduce power
Reduces wake time and power consumption of activity recognition
Wake time reduction of 70.1% and 93.0% compared to fixed wake time scheme for 2 test subjects
At 91% accuracy lower bound
Higher reduction in power under high accuracy or low latency requirements. Enables applications such as:
Context-driven UI
Context-driven reminder applications

6. Prior art
Existing power reduction research in mobile sensing is not very useful for accelerometer-based activity recognition since
Wake lock is the main bottleneck consuming ~200 mW
Sensor sampling and classification only consume ~30-40 mW
Approach
Idea
Reduces power from sensor sampling?
Reduces power from activity classification?
Reduces power from keeping smartphone awake?
Triggered sensing (Lu et al, Sensys 2010)
Trigger high power sensor such as GPS based on low power accelerometer
Yes No
Activity-adaptive approach (Yan et al, ISWC 2012)
Vary sampling rate and set of features used (time-domain vs. frequency domain) to save power
Admission control (Rachuri et al, Ubicomp 2010; Lu et al, Sensys 2010)
Use heuristics to determine if sensor data window should be sent to classifier

7. Prior art Adaptive sensing approach (Rachuri et al, MobiCom 2011)
Does address the high power consumption of keeping smartphone awake
Adapts the sleep period between wake times dynamically to save power
Is being used in combination with the two-tier approach
Is not very useful in reducing power when there are low latency requirements of less than 30 seconds

8. Detail 1/4: Duty-cycling overview
Acquire wake lock to ensure that activity recognition is not interrupted
Sample tri-axial accelerometer data at 32 Hz
Perform feature extraction and two-tier activity classification
Set an alarm for when the smartphone should wake up next
Release wake lock to allow smartphone to enter low power sleep mode
And repeat!
Set alarm
and release wakelock
Alarm triggered
Acquire wakelock
Power
Activity recognition ON
Phone in sleep mode
Time

9. Detail 2/4: Feature extraction
Transform tri-axial accelerometer data into three orientation-independent axes
Segment accelerometer time series into finite sampling windows of duration T seconds
Transform each sampling window to a feature vector for use by classifiers
Time-domain features: Mean, variance etc
Frequency-domain features: Highest magnitude frequency etc.
Data acquisition from accelerometer
Tri-axis normalization
Sliding-window partitioning
FFT computation
Feature extraction
Features
Classification

10. Detail 3/4: Offline classifier training
Use labeled accelerometer data to build a decision tree classifier model using the C4.5 algorithm
Train two classifiers offline with two different sampling windows
Tier I Idle classifier
Use idle window size of seconds
Recognizes only idle vs. non-idle (transform training labels to idle or non-idle)
Tier II activity classifier
Use active window size of 4-8 seconds
Recognizes entire set of physical activities such as walking, driving, idle, biking etc.

11. Detail 4/4: Online activity classification
Use Tier I idle classifier with 1 second sample window to determine if user is idle
If idle, go to sleep immediately to save power
If non-idle, extend wake time to 4-8 seconds and use Tier II activity classifier to identify physical activity
Tier I
Tier II

12. Two additional solutions implemented for comparison
State of the art single-Tier scheme
Example: Always use fixed wake time of 4 seconds
Confidence-based multi-tier scheme
Obtain confidence from decision tree node accuracy
Example: Continue sampling till confidence > 90% or maximum window size is reached

13. Empirical evaluation Power vs. accuracy tradeoff evaluation
Three schemes evaluated on labeled accelerometer data
Scheme I: Single-Tier scheme
Scheme II: Confidence-based multi-Tier Scheme
Scheme III: Two-Tier scheme
Use naturalistic labeled activity data collected from two subjects

14. Evaluation metrics Wake time percentage Classification accuracy
Percentage of time that the two-tier classifier keeps the smartphone awake
Example: 30% wake time percentage means phone is awake 30% of the time due to the activity recognition system alone
Classification accuracy
Percentage of time slices correctly classified by the activity recognition system
Assumes interpolation when sleep periods are introduced
Example: 90% classification accuracy means that the two-tier classifier is correct about the user’s activities 90% of the time

15. Parameter settings for each scheme
Multiple parameter values possible for each of the three schemes
Perform a brute force exploration of the parameter space to determine the best instance of each scheme

16. Experiment #1
Compute the lowest wake time percentage (over all possible parameter values) for each scheme given a lower bound on classification accuracy
Observation: Significant reduction in wake time percentages for two-tier approach at high classification accuracy lower bounds
Subject I results
Subject II results

17. Experiment #2
Compute the lowest wake time percentage (over all possible parameter values) for each scheme given two constraints
Lower bound on classification accuracy (shown along Y-axis)
Upper bound on sleep period in seconds (shown on X-axis)
Observation: About 50% drop in wake time percentage (compared to best of either scheme I and II) for latency bounds less than 60 seconds

18. Effect of lag time and ramp-up on energy consumption - Tier I
Energy consumption of Tier I wakeup spike on galaxy S2with idle window of 1 second: uAh
Tier I classification
Sleep mode
Sleep mode
Lag time

19. Effect of lag time and ramp-up on energy consumption - Tier II
Energy consumption of Tier II wakeup spike on Galaxy S2with active window of 4 seconds: uAh
Tier II classification
Sleep mode
Sleep mode
Lag time

20. Conclusions and future work
First approach to dynamically change the wake time of the activity recognition system to reduce power
Reduces wake time and power consumption of activity recognition
Wake time reduction of 70.1% and 93.0% compared to fixed wake time scheme for 2 test subjects
At 91% accuracy lower bound
Higher reduction in power under high accuracy or low latency requirements. Enables applications such as:
Context-driven UI
Context-driven reminder applications
Ongoing work
Integrate two-tier wakeup with aggressive sleep periods
Incorporate empirical energy models into evaluation
Explore multi-tier confidence based approach with probabilistic classifier