1. Real Time Motion Capture Using a Single Time-Of-Flight Camera
Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun
CVPR 2010
Q 邱碁森

2. Outline Introduction Probabilistic Model Inference Experiments
Conclusions

3. Introduction
Motion capture is used to human-machine interaction, smart surveillance and so on.
Time-of-flight sensors offers rich sensory information, not sensitive to changes in lighting, shadows, and some other problems.
This paper propose an efficient filtering algorithm for tracking human pose for fast operation at video frame.

4. What is Probabilistic Model?
A tree-shaped kinematic chain (skeleton)
Human body is modeled as 15 body parts
The transformations of the body Xt at time t is a set: Xt= {Xi}, i = 1~15
X1: the root of tree → the pelvis part
root(pelvis): could freely rotate and translate
other parts: connected to the their parent, allow to rotate (not to translate)

5. What is Probabilistic Model? (cont.)
The absolute orientation of a body part i: Wi(X)
multiplying the transformations of its ancestors in the kinematic chain
Wi(X) = X1∙X2 · ...· Xparent(i) ∙ Xi

6. Why need the Probabilistic Model?
Determine the most likely state at at time t
the pose set Xt
the first discrete-time derivative set Vt (velocities)
zt: the recorded range measurements
The system is modeled as a dynamic Bayesian network (DBN)

7. Probabilistic Model The measured range scan is denoted by
z = {zk} k=1M
where zk gives the measured depth of the pixel at coordinate k.

8. Probabilistic Model
Assumption: the accelerations in our system are drawn from a Gaussian distribution with zero mean

9. Inference How to perform efficient inference at each frame?
Model Based Hill Climbing Search (HC)
A component locally optimizes the likelihood function
Evidence Propagation (EP)
An inference procedure generate likely states which are used to initialize the HC
Inference n. 推論機

10. Model Based Hill Climbing Search
coarse-to-fine
The procedure can then potentially be applied to a smaller interval about the value chosen at the coarser level
hill-climbing
Start from the base of kinematic chain which includes the largest body parts, and proceed toward the limbs 3 2
sample:
0.5
0.45
0.4
...
-0.35
-0.4
-0.45
-0.5 1
then chose the best one
optimize the X axis

11. Evidence Propagation Problem:
fast motion cause motion blur
occlusion cause the estimate of the state of hidden parts to drift
the likelihood function has ridges (difficult to navigate)
This procedure that identifies promising locations for body parts to find likely poses

12. Evidence Propagation Steps in this procedure:
Body Part Detection: identify possible body part locations from the current range image
Probabilistic Inverse Kinematics: update the body configuration X given possible correspondences between mesh vertices and part detections
Data Association and Inference: determine the best subset of such correspondences

13. Body Part Detection
Five body parts: head, left hand, right hand, left foot and right foot are found from the current range image.
Interest Point(AGEX) Detection
start on the geodesic centroid of the mesh: AGEX1(M)
recursively find the vertex AGEXk(M) which has max geodesic distance to AGEXk-1(M)
Identification of Parts
points are classified as body part by training these data using a marker-based motion capture system( LED mark)
C. Plagemann, V. Ganapathi, D. Koller, and S. Thrun. Realtime
identification and localization of body parts from depth
images. In IEEE Int. Conference on Robotics and Automation
(ICRA), Anchorage, Alaska, USA, 2010.

14. Evidence Propagation

15. Experiments Using a Swissranger SR4000 Time-of-Flight camera
Tracking results on real-world test sequences, sorted from most complex (left) to least complex (right).

16. Experiments A Tennis sequence Only use Model-Based search
Our combined tracker

17. Conclusions
A novel algorithm for combining part detections with local hill-climbing for marker less tracking of human pose.
With the hybrid, GPU-accelerated filtering approach