Presentation is loading. Please wait.

Presentation is loading. Please wait.

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.

Similar presentations


Presentation on theme: "POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of."— Presentation transcript:

1 POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University

2 Objective ImageSegmentationPose Estimate [Images courtesy: M. Black, L. Sigal]

3 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

4 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

5 The Image Segmentation Problem Segments Image

6 Problem – MRF Formulation n Notation Labelling x over the set of pixels The observed pixel intensity values y (constitute data D) n Energy E (x) = - log Pr(x|D) + constant n Unary term Likelihood based on colour n Pairwise terms Prior Contrast term n Find best labelling x* = arg min E(x)

7 MRF for Image Segmentation D (pixels) x (labels) Image Plane i j xixi xjxj Unary Potential i (D|x i ) Pairwise Potential ij (x i, x j ) x i = {segment 1, …, segment k }for instance {obj, bkg}

8 Can be solved using graph cuts MRF for Image Segmentation MAP Solution Pair-wise Terms Contrast Term Ising Model Data (D) Unary likelihood Maximum a-posteriori (MAP) solution x* =

9 MRF for Image Segmentation Pair-wise Terms MAP Solution Unary likelihoodData (D) Unary likelihood Contrast Term Uniform Prior Maximum-a-posteriori (MAP) solution x* = Need for a human like segmentation

10 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

11 Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR 05] – Shape-Prior: Layered Pictorial Structure (LPS) – Learned exemplars for parts of the LPS model – Obtained impressive results Layer 2 Layer 1 Spatial Layout (Pairwise Configuration) + =

12 Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR 05] – Shape-Prior: Layered Pictorial Structure (LPS) – Learned exemplars for parts of the LPS model – Obtained impressive results Shape-Prior Colour + Shape Unary likelihood colour Image

13 Problems in using shape priors n Intra-class variability Need to learn an enormous exemplar set Infeasible for complex subjects (Humans) n Multiple Aspects? n Inference of pose parameters

14 Do we really need accurate models? n Interactive Image Segmentation [Boykov & Jolly, ICCV01] Rough region cues sufficient Segmentation boundary can be extracted from edges additional segmentation cues user segmentation cues

15 Do we really need accurate models? n Interactive Image Segmentation Rough region cues sufficient Segmentation boundary can be extracted from edges

16 Rough Shape Prior - The Stickman Model n 26 degrees of freedom Can be rendered extremely efficiently Over-comes problems of learning a huge exemplar set Gives accurate segmentation results

17 Pose-specific MRF Formulation D (pixels) x (labels) Image Plane i j xixi xjxj Unary Potential i (D|x i ) Pairwise Potential ij (x i, x j ) (pose parameters) Unary Potential i (x i | )

18 Pose-specific MRF Energy to be minimized Unary term Shape prior Pairwise potential Potts model distance transform

19 Pose-specific MRF Energy to be minimized Unary term Shape prior Pairwise potential Potts model += Shape Prior MAP Solution Colour likelihood Data (D) colour+ shape

20 What is the shape prior? Energy to be minimized Unary term Shape prior Pairwise potential Potts model How to find the value of ө ?

21 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

22 Formulating the Pose Inference Problem

23

24 Resolving ambiguity using multiple views Pose specific Segmentation Energy

25 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

26 Solving the Minimization Problem Minimize F( ө ) using Powell Minimization To solve: Let F( ө ) = Computational Problem: Each evaluation of F( ө ) requires a graph cut to be computed. (computationally expensive!!) BUT.. Solution: Use the dynamic graph cut algorithm [Kohli&Torr, ICCV 2005]

27 Dynamic Graph Cuts PBPB SBSB cheaper operation computationally expensive operation Simpler problem P B* differences between A and B similar PAPA SASA solve

28 Dynamic Graph Cuts 20 msec Simpler problem P B* differences between A and B similar xaxa solve xbxb 400 msec

29 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

30 Segmentation Results Colour + Smoothness Colour + Smoothness + Shape Prior Only Colour Image [Images courtesy: M. Black, L. Sigal]

31 Segmentation Results - Accuracy Information used % of object pixels correctly marked Accuracy (% of pixels correctly classified) Colour Colour + GMM Colour + GMM + Shape

32 Segmentation + Pose inference [Images courtesy: M. Black, L. Sigal]

33 Segmentation + Pose inference [Images courtesy: Vicon]

34 Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

35 Conclusions Efficient method for using shape priors for object- specific segmentation Efficient Inference of pose parameters using dynamic graph cuts Good segmentation results Pose inference - Needs further evaluation - Segmentation results could be used for silhouette intersection

36 Future Work Use dimensionality reduction to reduce the number of pose parameters. - results in less number of pose parameteres to optimize - would speed up inference Use of features based on texture Appearance models for individual part of the articulated model (instead of using a single appearance model).

37 Thank You


Download ppt "POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of."

Similar presentations


Ads by Google