1. Human Action Recognition by Learning Bases of Action Attributes and Parts

2. Outline Introduction Action Recognition with Attributes & Parts Learning Experiments and Results

3. Introduction use attributes and parts for recognizing human actions in still images use the whole image to represent an action treat action recognition as a general image classification problem PASCAL challenge – spatial pyramid – random forest based methods – No explore the semantically meaningful components

5. Introduction some methods rely on labor-intensive annotations of objects and human body parts during training time Inspired by the recent work – using objects and body parts for action recognition – propose an attributes and parts based representation The action attributes are holistic image descriptions of human actions – associated with verbs in the human language – E.g. Riding,sitting,repairing,lifting…

6. Introduction

7. a large number of possible interactions among these attributes parts in terms of co-occurrence statistics. Our challenge is – represent image by using a sparse set of action bases – effectively learn these bases given far-from-perfect detections of action attributes – parts without meticulous human labeling as proposed in previous work

8. Introduction our method has theoretical foundations in sparse coding and compressed sensing. PASCAL action dataset Stanford 40 Actions dataset

9. Attributes and Parts in Human Actions Attribute: – Use are related to verbs in human language – E.x: rinding a bike can be “riding” and “sitting” – attribute to correspond to more than one action Parts: – Composed of objects – Human poses

10. Attributes and Parts in Human Actions an action image consists – the objects that are closely related to the action – The descriptive local human poses. A vector of the normalized confidence scores obtained from these classifiers and detectors is used to represent this image

11. Action Bases of Attributes and Parts Our method learns high-order interactions of image attributes and parts – carry richer information about human actions – improve recognition performance Riding – sitting – bike Using - keyboard - monitor - sitting

12. Action Bases of Attributes and Parts formalize the action bases in a mathematical framework P: attributes and parts 1 Action bases: Coefficients: 4 5

13. Action Classification Using the Action Bases the attributes and parts representation A – reconstructed from the sparse factorization coefficients w. – use the coefficients vector w to represent an image train an SVM classifier for action classification

14. Learning the Dual-Sparse Action Bases and Reconstruction Coefficients 1 Ai is the vector of confidence scores there exists a latent dictionary of bases – frequent co-occurrence of attributes – e.g. “cycling” and “bike” To identify a set of sparse bases Φ = [1..M]

15. Learning the Dual-Sparse Action Bases and Reconstruction Coefficients learn the bases Φ and find the reconstruction coefficients wi for each ai. (2) is non-convex,(3) is convex Eqn.2 is convex with respect to each of the two variables Φ and W when the other one is fixed

16. Learning the Dual-Sparse Action Bases and Reconstruction Coefficients This is called the elastic-net constraint set[29] λ= 0.1 ϒ= 0.15

17. Google, Bing, and Flickr 180 ∼ 300 images for each class

18. Experiments and Results

20. PASCAL Stanford 40 action attributes (A), objects (O), and poselets (P)

21. Experiments and Results

23. Discussion use attributes and parts for action recognition – The attributes are verbs – The parts are composed of objects and poselets reconstructed by a set of sparse coefficients our method achieves state-of-the-art performance on two datasets

24. Future work learned action bases for image tagging explore more detailed semantic understanding of human actions in images