Sparse Granger Causality Graphs for Human Action Classification Saehoon Yi and Vladimir Pavlovic Rutgers, The State University of New Jersey

Outline  Objective and challenges  Previous work  Sparse Granger Causality Graph Model  Analysis and result  Conclusion

Objective  Classify human action time series data Challenges  High dimensional time series data  Dimensionality reduction  Difficulties in interpretation  Idiosyncratic patterns of same action  Need to find commonality within an action

Previous work  Learning dynamics of joints  Each action is modeled as Linear Dynamic System  C. Bregler, CVPR 97  Align time series data  Dynamic Time Warping  Canonical Time Warping  F. Zhou and F. De la Torre, NIPS 2009  Need to tune parameter for each pair of sequence  Isotonic Canonical Correlation Analysis  S. Shariat and V. Pavlovic, ICCV 2011

Our approach  Robust representation of continuous joint movements using micro event point processes.  Models salient and sparse temporal relations among skeletal joints movements

Overview

Step 1: Generate micro event point processes Continuous time series Joint angles on knees Detect maximal/minimal extreme points as events Micro event point processes

Step 2: Estimate Granger Causality Graph  Granger causality in time  Given two AR time series X, Y  Granger causality

 Granger causality in frequency  Given two point processes,  Estimate power spectrum  Decompose spectrum using Wilson’s algorithm  Granger causality  [A. Nedungadi, G. Rangarajan, N. Jain, and M. Ding ’09]

 Granger causality graph representation  Estimate Granger causality  for each pair of micro events  f frequencies → summarized to 4 bands Granger causality in 128 freqCausality summarized by 4 bands

Step 3: Learn L1 regularized regression  Input : 16M 2 Granger causality features  Output : action category label  Sparse regression coefficient W for each action  Common causality pattern within each class  Positive coefficient W ij  edge i → j have high causality  Negative coefficient W ij  edge i → j have low causality

Experiments  HDM05 dataset  Motion capture sequence of 29 skeletal joints  Each action is performed by 5 subjects  8 action classes are chosen Deposit on the floor w/ R handPunch front w/ L hand Jumping JackPunch front w/ R hand Kick front w/ R legSquat Kick side w/ R legWalk two steps

Experiment settings  Two different cross validation settings  Cut 1  Randomly partition training / testing across all subjects  Cut 2  Test set subjects different from training subjects  To show classification accuracy on unseen data

Example of Sparse Granger Causality Graph Model DEPOSIT FLOOR RIGHT HANDKICK RIGHT SIDE

Example of Sparse Granger Causality Graph Model PUNCH LEFT FRONTPUNCH RIGHT FRONT

Comparative result Confusion matrix of SGCGM

Conclusion  Learn common structure within an action  The sparse regression model chose which pairwise relationship is important for the action  Interpretability of the model  Granger causal graph describes temporal relationship between two joints

Thanks you. Q & A