Download presentation

Presentation is loading. Please wait.

Published byJaden Dillon Modified over 3 years ago

1
POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University

2
Objective ImageSegmentationPose Estimate [Images courtesy: M. Black, L. Sigal]

3
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

4
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

5
The Image Segmentation Problem Segments Image

6
Problem – MRF Formulation n Notation Labelling x over the set of pixels The observed pixel intensity values y (constitute data D) n Energy E (x) = - log Pr(x|D) + constant n Unary term Likelihood based on colour n Pairwise terms Prior Contrast term n Find best labelling x* = arg min E(x)

7
MRF for Image Segmentation D (pixels) x (labels) Image Plane i j xixi xjxj Unary Potential i (D|x i ) Pairwise Potential ij (x i, x j ) x i = {segment 1, …, segment k }for instance {obj, bkg}

8
Can be solved using graph cuts MRF for Image Segmentation MAP Solution Pair-wise Terms Contrast Term Ising Model Data (D) Unary likelihood Maximum a-posteriori (MAP) solution x* =

9
MRF for Image Segmentation Pair-wise Terms MAP Solution Unary likelihoodData (D) Unary likelihood Contrast Term Uniform Prior Maximum-a-posteriori (MAP) solution x* = Need for a human like segmentation

10
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

11
Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR 05] – Shape-Prior: Layered Pictorial Structure (LPS) – Learned exemplars for parts of the LPS model – Obtained impressive results Layer 2 Layer 1 Spatial Layout (Pairwise Configuration) + =

12
Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR 05] – Shape-Prior: Layered Pictorial Structure (LPS) – Learned exemplars for parts of the LPS model – Obtained impressive results Shape-Prior Colour + Shape Unary likelihood colour Image

13
Problems in using shape priors n Intra-class variability Need to learn an enormous exemplar set Infeasible for complex subjects (Humans) n Multiple Aspects? n Inference of pose parameters

14
Do we really need accurate models? n Interactive Image Segmentation [Boykov & Jolly, ICCV01] Rough region cues sufficient Segmentation boundary can be extracted from edges additional segmentation cues user segmentation cues

15
Do we really need accurate models? n Interactive Image Segmentation Rough region cues sufficient Segmentation boundary can be extracted from edges

16
Rough Shape Prior - The Stickman Model n 26 degrees of freedom Can be rendered extremely efficiently Over-comes problems of learning a huge exemplar set Gives accurate segmentation results

17
Pose-specific MRF Formulation D (pixels) x (labels) Image Plane i j xixi xjxj Unary Potential i (D|x i ) Pairwise Potential ij (x i, x j ) (pose parameters) Unary Potential i (x i | )

18
Pose-specific MRF Energy to be minimized Unary term Shape prior Pairwise potential Potts model distance transform

19
Pose-specific MRF Energy to be minimized Unary term Shape prior Pairwise potential Potts model += Shape Prior MAP Solution Colour likelihood Data (D) colour+ shape

20
What is the shape prior? Energy to be minimized Unary term Shape prior Pairwise potential Potts model How to find the value of ө ?

21
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

22
Formulating the Pose Inference Problem

23

24
Resolving ambiguity using multiple views Pose specific Segmentation Energy

25
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

26
Solving the Minimization Problem Minimize F( ө ) using Powell Minimization To solve: Let F( ө ) = Computational Problem: Each evaluation of F( ө ) requires a graph cut to be computed. (computationally expensive!!) BUT.. Solution: Use the dynamic graph cut algorithm [Kohli&Torr, ICCV 2005]

27
Dynamic Graph Cuts PBPB SBSB cheaper operation computationally expensive operation Simpler problem P B* differences between A and B similar PAPA SASA solve

28
Dynamic Graph Cuts 20 msec Simpler problem P B* differences between A and B similar xaxa solve xbxb 400 msec

29
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

30
Segmentation Results Colour + Smoothness Colour + Smoothness + Shape Prior Only Colour Image [Images courtesy: M. Black, L. Sigal]

31
Segmentation Results - Accuracy Information used % of object pixels correctly marked Accuracy (% of pixels correctly classified) Colour Colour + GMM Colour + GMM + Shape

32
Segmentation + Pose inference [Images courtesy: M. Black, L. Sigal]

33
Segmentation + Pose inference [Images courtesy: Vicon]

34
Outline n Image Segmentation Problem n Pose-Specific Segmentation n The Pose Inference Problem n Optimization n Results n Conclusion and Future Work

35
Conclusions Efficient method for using shape priors for object- specific segmentation Efficient Inference of pose parameters using dynamic graph cuts Good segmentation results Pose inference - Needs further evaluation - Segmentation results could be used for silhouette intersection

36
Future Work Use dimensionality reduction to reduce the number of pose parameters. - results in less number of pose parameteres to optimize - would speed up inference Use of features based on texture Appearance models for individual part of the articulated model (instead of using a single appearance model).

37
Thank You

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google