1. William Fadel, Ph.D. August 1, 2018
Classification of Walking and Stair Climbing Based on Raw Accelerometry Data
William Fadel, Ph.D.
August 1, 2018

2. Outline of talk How to measure physical activity (PA)?
available devices
what do the data look like?
how are the data currently reported?
Feature extraction methods
Classification using tree-based methods
How to identify types of PA collected in a free-living environment? (subject-level classifier)
How to identify types of PA collected in a free-living environment? (population-level classifier)
Brief punchline of paper 1, and then focus on paper 2 and 3.
Reduce first 10

3. How to measure physical activity?
Popular approach is to use acceleration measurements
Actigraphy is a non-invasive monitoring of human rest/activity cycles. A small actigraph unit, accelerometer, is worn by an individual to measure motor activity.
Acquired acceleration values form 3-dimensional time series that reflect history of subjects’ real-life activities

4. What do these data look like?
24 hours  60 minutes  60 seconds

5. How are the data reported?
24 hours of data  9000*60min*24hours = 12,960,000 = million observations
504
Activity Count

6. Can we trust our step-count?
On average most devices perform quite well, but we would like to do better! 14 participants (trial x 2) = 28 observations
From: Accuracy of Smartphone Applications and Wearable Devices for Tracking Physical Activity Data
JAMA. 2015;313(6): doi: /jama

7. Can we trust our step-count?
“Fitbit trackers have a finely tuned algorithm for step counting. The algorithm is designed to look for motion patterns that are most indicative of people walking. The algorithm determines whether a motion's size is large enough by setting a threshold...Other factors can create enough acceleration to meet our threshold and cause some over counting of steps, such as riding on a bumpy road. Equally, it's possible for the algorithm to undercount (not meet the required acceleration threshold). Examples here include walking on a very soft surface such as a plush carpet.”
On average most devices perform quite well, but we would like to do better!
From: (accessed 11/30/2016)

8. What your activity tracker sees
Use the blog rather than the video!

9. Research goals
Extract features that quantify the important aspects of walking.
Use extracted features to build an interpretable classification model.
Study the properties of the classification model.
Only focus on papers 2 and 3

10. IU walking and driving study
Motivation:
Walking and driving are the most common activities among the general population
Lack of available data to validate activity recognition methodology
Study Purpose:
Identify patterns of walking, stair climbing, and driving from raw accelerometry data
Collect data that simulates a free-living environment

11. IU walking and driving study
Participants:
We enrolled 32 healthy adults.
19 females and 13 males
Age: Mean = 39 years (SD = 8.9 years)
Range: 23 – 54 years
Data collection:
4 Actigraph GT3X+ accelerometers
left wrist, left hip, left ankle, right ankle
Data collected at sampling frequency of 100 Hz
Participants asked to clap 3 times between types of activity (gold standard)
Internally mark the raw data with 3 consecutive spikes

12. Walking on level ground
𝑣𝑚 𝑡 = 𝑥 𝑡 2 +𝑦 𝑡 2 +𝑧 𝑡 2

13. Down/up/down stairs

14. Goal #1 – feature extraction
Let’s look at a short interval for each activity type
Vector Magnitude, 𝑣𝑚(𝑡))
Tri-axial view, 𝑥 𝑡 , 𝑦 𝑡 , 𝑧 𝑡
Short-time FFT (STFT)

15. Walking
Descending Stairs
Ascending Stairs

16. Feature extraction Divide the signal into windows of a given length
Within each window, extract the features of the signal to describe the activity being performed.
FFT features
DWT features
Time domain features

17. Four FFT features f1 – dominant frequency (cadence)
𝒓𝒂𝒕𝒊𝒐.𝑽𝑴 – ratio of all shaded regions to the total area under spectrum
p1 – squared magnitude (power) at f1
p1_TP – p1 divided by area under total power spectrum ( Hz)

18. Two DWT features 𝑫𝑾𝑻_𝑽𝑴𝟐= 𝑗=𝛼 𝛽 𝑑 𝑗 2 /𝑉 𝑀 2
𝑫𝑾𝑻_𝑽𝑴𝟐= 𝑗=𝛼 𝛽 𝑑 𝑗 2 /𝑉 𝑀 2
where the 𝑑 𝑗 are the detail coefficients from level 𝑗 of the decomposition and
𝑉 𝑀 2 = 𝑡=1 𝑇 𝑣𝑚 𝑡 2
𝑫𝑾𝑻_𝑻𝑷= 𝑗=𝛼 𝛽 𝑑 𝑗 2 / 𝑗=1 𝐽 𝑑 𝑗 2

19. Time domain features 𝑀𝑒𝑎𝑛 𝑉𝑀 = 1 𝑇 𝑡=1 𝑇 𝑣𝑚(𝑡)
𝑀𝑒𝑎𝑛 𝑉𝑀 = 1 𝑇 𝑡=1 𝑇 𝑣𝑚(𝑡)
𝑆𝐷 𝑉𝑀 = 1 𝑇−1 𝑡=1 𝑇 𝑣𝑚(𝑡)−𝑀𝑒𝑎𝑛 𝑉𝑀 2
𝑉𝑀𝐶= 1 𝑇 𝑡=1 𝑇 𝑣𝑚(𝑡)−𝑀𝑒𝑎𝑛(𝑉𝑀)
𝐶𝑜𝑟𝑟 𝑋𝑌 , Corr(XZ), and Corr(YZ)
𝐴𝑐𝑡𝐼𝑛𝑡= 𝑠 𝑥 + 𝑠 𝑦 + 𝑠 𝑧 3
Define notation!

20. Goal #2 – build a classifier (CART)
From: (accessed 12/2/2016)

21. Classification model (subject level)
Data structure:
Response: Activity = {walking, descending stairs, ascending stairs}
Predictors: 13 extracted features
Classification tree for each subject (n = 32):
Train algorithm on a subset of the data
Test algorithm on the remaining data.
Evaluate classifier using 100 iterations of random cross validation to obtain an estimate of classification accuracy and variability.

22. Classification accuracy

23. Feature Importance (10.24s)
Condense these slides. Illustrate what is consistent and what is different across scenarios.

24. Classification model (pop. level)
Goals:
Build population based classifiers based on features extracted previously.
Compare results between normalized and non-normalized features
Difficulties:
Large subject to subject variability in walking characteristics
How to normalize the data?
walking is the most prevalent periodic activity
normalize everything to walking
𝑧 ∗ = 𝑥−𝑚𝑒𝑑𝑖𝑎𝑛 𝑥 𝑀𝐴𝐷 𝑥
where 𝑀𝐴𝐷 𝑥 =1.4826∗𝑚𝑒𝑑𝑖𝑎𝑛 𝑥−𝑚𝑒𝑑𝑖𝑎𝑛 𝑥

25. Why do we normalize?

26. Classification accuracy
This will be a 3x3 panel plot of the normalized vs non-normalized results for the 3 activities!

27. Trees HIP 10.24s ratio.VM < -0.47 SD(VM) >= -0.75 p1_TP >= -1
= Descending stairs
= Walking
= Ascending Stairs
ratio.VM < -0.47
SD(VM) >= -0.75
p1_TP >= -1
Mean(VM) < 4.1

28. Conclusions
We can differentiate between walking on level ground and stair climbing with good accuracy.
Best classification accuracy was achieved with accelero-meters worn at the ankles and features extracted using larger window sizes for both subject- and group-levels.
We lose some accuracy when building a group-level classifier compared to subject-level.
Normalization of features has a greater impact for sensors worn at the hip and wrist versus the ankles.

29. Acknowledgements Indiana University
Jaroslaw Harezlak (Biostatistics, research advisor)
Xiaochun Li (Biostatistics, committee member)
Constantin Yiannoutsos (Biostatistics, committee member)
Andrea Chomistek (Epidemiology, committee member)
Steven Albertson (undergraduate intern)
Johns Hopkins School of Public Health
Jacek Urbanek (Biostatistics, collaborator)

30. Thank you!