1. Implicit Probabilistic Models of Human Motion for Synthesis and Tracking Hedvig Sidenbladh, KTH, Sweden (now FOI, Sweden) Michael J. Black, Brown University, USA Leonid Sigal, Brown University, USA

2. Articulated 3D tracking  = model parameters I = image Recursive Bayesian formulation:

3. Extreme case Non-linear motion, strong dependencies Model dependencies analytically –e.g. [Hogg, Rohr] for walking Dynamical models –e.g. [Wren&Pentland, Bruderlin&Calvert] Learn from Mocap examples –e.g. [Bowden, Brand, Molina&Hilton] Modeling Human Motion

4. Texture Synthesis Efros & Freeman’01 “Database” Synthetic Texture –e.g. [De Bonnet&Viola, Efros&Leung, Efros&Freeman, Paztor&Freeman, Hertzmann et al] –Image(s) as an implicit probabilistic model.

5. Motion Texture –Motion examples as an implicit probabilistic model.

6. Probabilistic Formulation Generative model: Problem: Model for all motions i in the database! No learning required, all the variability in the data captured. Database example i Sampling from, taking  t =  i

7. Probabilistic Database Search Sort database in some way to enable search in sublinear time Linear search infeasible for large database! No need to visit all database examples - only need to sample from distribution time joint angles c =[c 1, c 2, c 3, c 4 ] Sort into tree-structure according to PCA coefficients

8. Probabilistic Database Search Each level in the tree corresponds to one coefficient l. Sort examples i into tree: Left subtree for negative value of c l,i, right for positive value.

9. Probabilistic Database Search Approximated by sampling from tree iteratively:

10. Synthesis Running Walking Small database with running, walking, skipping, dance etc. Changing color indicates new example sequence. Future work: Add editing possibility, gravity model, goal function. Goal: Generate smooth and plausible-looking motion.

11. Tracking Goal: Efficiently generate samples (image data will sort out which are good). Temperature parameter controls randomness of tree search.

12. Arm Tracking Example Constant velocity model 1000 samples ~1 min/frame Image likelihood model from [Sidenbladh & Black, ICCV 01] Example based model

13. ”Mocap Soup” [Cohen] at SIGGRAPH 02 –Arikan & Forsyth. Interactive motion generation from examples –Li et al. Motion textures: A two-level statistical model for character motion synthesis –Lee et al. Interactive control of avatars animated with human motion data –Kovar et al. Motion graphs –Pullen & Bregler. Motion capture assisted animation: Texturing and synthesis Here we formulate a probabilistic model suitable for stochastic search search and Bayesian tracking.

14. Conclusions Implicit motion model - replace learning with search Analog to example based texture synthesis Larger database - sub-linear time search Tree structure sorted with PCA coefficients Probabilistic tree search - sampling from the tree approximates sampling from