Outline Introduction Related methods Modeling Human Interactions Single Subject Key Pose Sequence Model Interaction Key Pose Sequence Model Learning the parameters Experiments
Introduction This paper focuses on recognizing interactions between individuals. The sequences of key poses between peoples will be combined into activity level. The activities to be recognized include hugging,shaking hands,pointing, punching, kicking,pushing each others.
Introduction Not every movement or pose by the target is relevant to the activity to be recognized. We use an examplar-based model by giving the pose a score to match the key poses. When people do some activities, they will do the key poses in a chronological order.
Related methods Ryoo and Aggarwal develop a matching kernel that considers spatial and temporal relations between space-time interest points. Yao et al.  use a Hough transform voting scheme from an interest point representation.  M. Ryoo and J. Aggarwal. Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. In ICCV, 2009.  A. Yao, J. Gall, and L. Van Gool. A hough transform-based voting framework for action recognition. In CVPR, 2010. 2, 6
Modeling Human Interactions There are four things to know: 1. Who is involved in the interaction? – Subject or object 2. When do the key poses occur? – The interval of the key poses 3. How are the key poses executed? – With hand or leg, powerful or weak 4. Where are the people when the key poses occur?
Modeling Human Interactions we will assume F maximizes a model G that includes the latent variables H: The variables H are the answer of the four questions above.
We represent each key pose by h: Denote K key poses of a sequense by H
Single Subject Key Pose Sequence Model Exemplar Matching Link: Compute the pose connection strength to the exemplar poses.
Single Subject Key Pose Sequence Model Activity-Key Pose Link: – Compute the sequence similarity in the activity to give a score. Direct Root Model:
Interaction Key Pose Sequence Model Update the model from individual to two people interaction. We should recognize who is subject or object. The scoring function will be:
Learning the parameters Define the scoring function E(x,y): Use multiclass linear SVM classifier to find the best parameters.
Experiments The UT-Interaction dataset contains videos of 6 classes of human-human interactions. Set 1 is with a stationary background,and Set 2 is with slight background movement and camera jitter.