Download presentation

Presentation is loading. Please wait.

Published byJaden Dillon Modified over 5 years ago

1
POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University

2
Objective ImageSegmentationPose Estimate [Images courtesy: M. Black, L. Sigal]

3
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

4
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

5
The Image Segmentation Problem Segments Image

6
Problem – MRF Formulation n Notation Labelling x over the set of pixels The observed pixel intensity values y (constitute data D) n Energy E (x) = - log Pr(x|D) + constant n Unary term Likelihood based on colour n Pairwise terms Prior Contrast term n Find best labelling x* = arg min E(x)

7
MRF for Image Segmentation D (pixels) x (labels) Image Plane i j xixi xjxj Unary Potential i (D|x i ) Pairwise Potential ij (x i, x j ) x i = {segment 1, …, segment k }for instance {obj, bkg}

8
Can be solved using graph cuts MRF for Image Segmentation MAP Solution Pair-wise Terms Contrast Term Ising Model Data (D) Unary likelihood Maximum a-posteriori (MAP) solution x* =

9
MRF for Image Segmentation Pair-wise Terms MAP Solution Unary likelihoodData (D) Unary likelihood Contrast Term Uniform Prior Maximum-a-posteriori (MAP) solution x* = Need for a human like segmentation

10
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

11
Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR 05] – Shape-Prior: Layered Pictorial Structure (LPS) – Learned exemplars for parts of the LPS model – Obtained impressive results Layer 2 Layer 1 Spatial Layout (Pairwise Configuration) + =

12
Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR 05] – Shape-Prior: Layered Pictorial Structure (LPS) – Learned exemplars for parts of the LPS model – Obtained impressive results Shape-Prior Colour + Shape Unary likelihood colour Image

13
Problems in using shape priors n Intra-class variability Need to learn an enormous exemplar set Infeasible for complex subjects (Humans) n Multiple Aspects? n Inference of pose parameters

14
Do we really need accurate models? n Interactive Image Segmentation [Boykov & Jolly, ICCV01] Rough region cues sufficient Segmentation boundary can be extracted from edges additional segmentation cues user segmentation cues

15
Do we really need accurate models? n Interactive Image Segmentation Rough region cues sufficient Segmentation boundary can be extracted from edges

16
Rough Shape Prior - The Stickman Model n 26 degrees of freedom Can be rendered extremely efficiently Over-comes problems of learning a huge exemplar set Gives accurate segmentation results

17
Pose-specific MRF Formulation D (pixels) x (labels) Image Plane i j xixi xjxj Unary Potential i (D|x i ) Pairwise Potential ij (x i, x j ) (pose parameters) Unary Potential i (x i | )

18
Pose-specific MRF Energy to be minimized Unary term Shape prior Pairwise potential Potts model distance transform

19
Pose-specific MRF Energy to be minimized Unary term Shape prior Pairwise potential Potts model += Shape Prior MAP Solution Colour likelihood Data (D) colour+ shape

20
What is the shape prior? Energy to be minimized Unary term Shape prior Pairwise potential Potts model How to find the value of ө ?

21
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

22
Formulating the Pose Inference Problem

24
Resolving ambiguity using multiple views Pose specific Segmentation Energy

25
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

26
Solving the Minimization Problem Minimize F( ө ) using Powell Minimization To solve: Let F( ө ) = Computational Problem: Each evaluation of F( ө ) requires a graph cut to be computed. (computationally expensive!!) BUT.. Solution: Use the dynamic graph cut algorithm [Kohli&Torr, ICCV 2005]

27
Dynamic Graph Cuts PBPB SBSB cheaper operation computationally expensive operation Simpler problem P B* differences between A and B similar PAPA SASA solve

28
Dynamic Graph Cuts 20 msec Simpler problem P B* differences between A and B similar xaxa solve xbxb 400 msec

29
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

30
Segmentation Results Colour + Smoothness Colour + Smoothness + Shape Prior Only Colour Image [Images courtesy: M. Black, L. Sigal]

31
Segmentation Results - Accuracy Information used % of object pixels correctly marked Accuracy (% of pixels correctly classified) Colour45.7395.2 Colour + GMM82.4896.9 Colour + GMM + Shape 97.4399.4

32
Segmentation + Pose inference [Images courtesy: M. Black, L. Sigal]

33
Segmentation + Pose inference [Images courtesy: Vicon]

34
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

35
Conclusions Efficient method for using shape priors for object- specific segmentation Efficient Inference of pose parameters using dynamic graph cuts Good segmentation results Pose inference - Needs further evaluation - Segmentation results could be used for silhouette intersection

36
Future Work Use dimensionality reduction to reduce the number of pose parameters. - results in less number of pose parameteres to optimize - would speed up inference Use of features based on texture Appearance models for individual part of the articulated model (instead of using a single appearance model).

37
Thank You

Similar presentations

OK

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google