1. Learning an Attribute Dictionary for Human Action Classification
Qiang Qiu
Qiang Qiu, Zhuolin Jiang, and Rama Chellappa, ”Sparse Dictionary-based Representation and Recognition of Action Attributes”, ICCV 2011

2. Action Feature Representation
Shape
Motion
HOG

3. Action Sparse Representation
Sparse code =
0.64
0.53
-0.40
0.35
0.43
0.63
-0.33
-0.36
Action
Dictionary =
0.43 × × × ×
0.64×
+ 0.53× ×
+0.35 ×

4. K-SVD K-SVD Input: signals Y, dictionary size, sparisty T
Sparse codes = Y y2 d1 d2 d3 …
0.64
0.53
-0.40
0.35
0.43
0.63
-0.33
-0.36 x1 x2 y1 D X
Input signals
Dictionary
K-SVD
Input: signals Y, dictionary size, sparisty T
Output: dictionary D, sparse codes X
arg min |Y- DX| s.t i , |xi|0 ≤ T
D,X
[1] M. Aharon and M. Elad and A. Bruckstein, K-SVD: An Algorithm for Designing Overcomplete Dictionries for Sparse Representation, IEEE Trans. on Signal Process, 2006

5. Objective
Compact
Discriminative
and
Dictionary.
Learn a

6. Probabilistic Model for Sparse Representation
A Gaussian Process
Dictionary Class Distribution

7. More Views of Sparse Representation= y2 y1 y4 y3 x1 x2 x3 x4
xd1
0.43
0.63
-0.33
-0.36
0.64
0.53
-0.40
0.35
-0.28
0.698
0.37
0.25
-0.42
0.42
0.47
0.32 l1 l2 d1 d2 d3 … l1 l2

8. A Gaussian Process y2 y1 y4 y3 l1 l2 xd1 x1 x2 x3 x4 … xd2 xd3 xd4 xd5
x1 x2 x3 x4 …
xd2
xd3
xd4
xd5
l1 l1 l2 l2 d1 d2 d3 d4 d5
A Gaussian Process
Covariance function entry: K(i,j) = cov(xdi, xdj)
P(Xd*|XD*) is a Gaussian with a closed-form conditional variance

9. Dictionary Class Distribution
xd1
x1 x2 x3 x4 …
xd2
xd3
xd4
xd5
l1 l1 l2 l2 d1 d2 d3 d4 d5
Dictionary Class Distribution
P(L|di), L [1, M]
aggregate |xdi| based on class labels to obtain a M sized vector
P(L=l1|d5) = ( )/( ) =
P(L=l2|d5) = (0+0.42)/( ) = 0.37

10. Dictionary Learning Approaches
Maximization of Joint Entropy (ME)
Maximization of Mutual Information (MMI)
Unsupervised Learning (MMI-1)
Supervised Learning (MMI-2)

11. Maximization of Joint Entropy (ME)
- Initialize dictionary using k-SVD
Do =
- Start with D* =
Untill |D*|=k, iteratively choose d* from Do\D*,
d* = arg max H(d|D*) d
ME dictionary
where D
A good approximation to ME criteria
arg max H(D)

12. Maximization of Mutual Information for Unsupervised Learning (MMI-1)
- Initialize dictionary using k-SVD
Do =
- Start with D* =
Untill |D*|=k, iteratively choose d* from Do\D*,
MMI dictionary
d* = arg max H(d|D*) - H(d|Do\(D* d)) d
Diversity
Coverage
Closed form:
A near-optimal approximation to MMI
arg max I(D; Do\D)
within (1-1/e) of the optimum D

13. Revisit y2 y1 y4 y3 l1 l2 xd1 x1 x2 x3 x4 … xd2 xd3 xd4 xd5
x1 x2 x3 x4 …
xd2
xd3
xd4
xd5
l1 l1 l2 l2 d1 d2 d3 d4 d5
Revisit
Dictionary Class Distribution
P(L|di), L [1, M]
aggregate|xdi|based on class labels to obtain a M sized vector
P(l1|d5) = ( )/( ) =
P(l2|d5) = (0+0.42)/( ) = 0.37
P(Ld) = P(L|d)
P(LD) = P(L|D) , where

14. Maximization of Mutual Information for Supervised Learning (MMI-2)
- Initialize dictionary using k-SVD
Do =
- Start with D* =
Untill |D*|=k, iteratively choose d* from Do\D*, d
d* = arg max [H(d|D*) - H(d|Do\(D* d))]
+ λ[H(Ld|LD*) – H(Ld|LDo\(D* d))]
- MMI-1 is a special case of MMI-2 with λ=0.

15. Keck gesture dataset

16. Representation Consistency
[1]
[1] J. Liu and M. Shah, Learning Human Actions via Information Maximization, CVPR 2008

17. Recognition Accuracy
The recognition accuracy using initial dictionary Do: (a) 0.23 (b) 0.42 (c) 0.71

18. Recognizing Realistic Actions
150 broadcast sports videos.
10 different actions.
Average recognition rate: 83:6%
Best reported result 86.6%