1. 1 Challenge the future HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences Omar Oreifej Zicheng Liu CVPR 2013

2. 2 Challenge the future Research Question Input: Depth sequences information (only) of segmented video (1 video for 1 activity) Output: Feature (activity descriptor) for Activity recognition: Classify the activity Why use depth information rather than color information? Depth reflects pure geometry and shape cues. Depth is insensitive to changes in lighting conditions.

3. 3 Challenge the future Two key points about activity recognition feature Capture the shape cues at a specific time instance Capture the motion cues over the time

4. 4 Challenge the future HON4D: Histogram of Oriented 4D Normals Analogous to HOG feature Calculated in 4D space: 2D images (x, y), depth (z), time (t) Surface normals capture the shape cues Change in the surface normals over time capture the motion cues. Normals in 3D example

5. 5 Challenge the future 4D surface normal Depth (z) considered as a function of time (t), space (x, y) A surface S in a 4D space The normal to the surface S is

6. 6 Challenge the future Histogram of 4D normals How to quantize the 4D space, i.e., get the bin of histogram? Polychoron: 4D regular geometric objects, analogous to cube in 3D space Dvide the 4D space uniformly with its vertices use 600-cell polychoron with 120 vertices Each vertex is referred as a projector, i.e., one bin of histogram. Histogram: Project 4D normals into 120 projectors HON4D: 120-dimension feature

7. 7 Challenge the future Histogram of 4D normals Projection of 4D normals set of 120 projectors set of unit normals computed over all depth sequences Projection with inner product 120-dimensional HON4D descriptor feature by Normalization

8. 8 Challenge the future Non-Uniform Quantization == Projectors Refinement Uniform space quantization is not always optimal. A better Non-Uniform Quantization could lead to better classification. = refine the projectors to better capture the distribution of the normals. How to evaluate the importance of each projector?

9. 9 Challenge the future Non-Uniform Quantization == Projectors Refinement First intuition: calculate the projector density is the training video set. High density does not necessarily means high contribution in classification

10. 10 Challenge the future Recall SVM

11. 11 Challenge the future Non-Uniform Quantization == Projectors Refinement Consider a SVM classifier for the training set, is the set of support vectors in the training set. Discriminative projector density 1.High accumulation of normal vectors 2.High contribution in the final classification

12. 12 Challenge the future Projectors Refinement Augment the density-learned projectors In experiments: 120  300

13. 13 Challenge the future Flow chart

14. 14 Challenge the future Experiments