Download presentation

Presentation is loading. Please wait.

Published byAmia Coughlin Modified over 3 years ago

1
Solving Markov Random Fields using Dynamic Graph Cuts & Second Order Cone Programming Relaxations M. Pawan Kumar, Pushmeet Kohli Philip Torr

2
Talk Outline Dynamic Graph Cuts –Fast reestimation of cut –Useful for video –Object specific segmentation Estimation of non submodular MRFs –Relaxations beyond linear!!

3
Example: Video Segmentation

4
Model Based Segmentation ImageSegmentationPose Estimate [Images courtesy: M. Black, L. Sigal]

5
Min-Marginals Image MAP SolutionBelief - Foreground Low smoothness High smoothness Moderate smoothness Colour Scale 1 0 0.5

6
Uses of Min marginals Estimate of true marginals (uncertainty) Parameter Learning. Get best n solutions easily.

7
Dynamic Graph Cuts PBPB SBSB cheaper operation computationally expensive operation Simpler problem P B* differences between A and B similar PAPA SASA solve

8
First segmentation problem MAP solution GaGa Our Algorithm GbGb second segmentation problem Maximum flow residual graph ( G r ) G` difference between G a and G b updated residual graph

9
The Max-flow Problem - Edge capacity and flow balance constraints Computing the st-mincut from Max-flow algorithms Notation - Residual capacity (edge capacity – current flow) - Augmenting path Simple Augmenting Path based Algorithms - Repeatedly find augmenting paths and push flow. - Saturated edges constitute the st-mincut. [Ford-Fulkerson Theorem] Sink (1) Source (0) a1a1 a2a2 2 5 9 4 2 1

10
9 + α 4 + α Adding a constant to both the t-edges of a node does not change the edges constituting the st-mincut. Key Observation Sink (1) Source (0) a1a1 a2a2 2 5 2 1 E (a 1,a 2 ) = 2a 1 + 5ā 1 + 9a 2 + 4ā 2 + 2a 1 ā 2 + ā 1 a 2 E*(a 1,a 2 ) = E(a 1,a 2 ) + α(a 2 +ā 2 ) = E(a 1,a 2 ) + α [a 2 +ā 2 =1] Reparametrization

11
9 + α 4 All reparametrizations of the graph are sums of these two types. Other type of reparametrization Sink (1) Source (0) a1a1 a2a2 2 5 + α 2 + α 1 - α Reparametrization, second type Both maintain the solution and add a constant α to the energy.

12
Reparametrization Nice result (easy to prove) All other reparametrizations can be viewed in terms of these two basic operations. Proof in Hammer, and also in one of Vlads recent papers.

13
s G t original graph 0/9 0/7 0/5 0/20/4 0/1 xixi xjxj flow/residual capacity Graph Re-parameterization

14
s G t original graph 0/9 0/7 0/5 0/20/4 0/1 xixi xjxj flow/residual capacity Graph Re-parameterization t residual graph xixi xjxj 0/12 5/2 3/2 1/0 2/04/0 st-mincut Compute Maxflow GrGr Edges cut

15
Update t-edge Capacities s GrGr t residual graph xixi xjxj 0/12 5/2 3/2 1/0 2/04/0

16
Update t-edge Capacities s GrGr t residual graph xixi xjxj 0/12 5/2 3/2 1/0 2/04/0 capacity changes from 7 to 4

17
Update t-edge Capacities s G` t updated residual graph xixi xjxj 0/12 5/-1 3/2 1/0 2/04/0 capacity changes from 7 to 4 edge capacity constraint violated! (flow > capacity) = 5 – 4 = 1 excess flow (e) = flow – new capacity

18
add e to both t-edges connected to node i Update t-edge Capacities s G` t updated residual graph xixi xjxj 0/12 3/2 1/0 2/04/0 capacity changes from 7 to 4 edge capacity constraint violated! (flow > capacity) = 5 – 4 = 1 excess flow (e) = flow – new capacity 5/-1

19
Update t-edge Capacities s G` t updated residual graph xixi xjxj 0/12 3/2 1/0 4/0 capacity changes from 7 to 4 excess flow (e) = flow – new capacity add e to both t-edges connected to node i = 5 – 4 = 1 5/0 2/1 edge capacity constraint violated! (flow > capacity)

20
Update n-edge Capacities s GrGr t residual graph xixi xjxj 0/12 5/2 3/2 1/0 2/04/0 Capacity changes from 5 to 2

21
Update n-edge Capacities s t Updated residual graph xixi xjxj 0/12 5/2 3/-1 1/0 2/04/0 G` Capacity changes from 5 to 2 - edge capacity constraint violated!

22
Update n-edge Capacities s t Updated residual graph xixi xjxj 0/12 5/2 3/-1 1/0 2/04/0 G` Capacity changes from 5 to 2 - edge capacity constraint violated! Reduce flow to satisfy constraint

23
Update n-edge Capacities s t Updated residual graph xixi xjxj 0/11 5/2 2/0 1/0 2/04/0 excess deficiency G` Capacity changes from 5 to 2 - edge capacity constraint violated! Reduce flow to satisfy constraint - causes flow imbalance!

24
Update n-edge Capacities s t Updated residual graph xixi xjxj 0/11 5/2 2/0 1/0 2/04/0 deficiency excess G` Capacity changes from 5 to 2 - edge capacity constraint violated! Reduce flow to satisfy constraint - causes flow imbalance! Push excess flow to/from the terminals Create capacity by adding α = excess to both t-edges.

25
Update n-edge Capacities Updated residual graph Capacity changes from 5 to 2 - edge capacity constraint violated! Reduce flow to satisfy constraint - causes flow imbalance! Push excess flow to the terminals Create capacity by adding α = excess to both t-edges. G` xixi xjxj 0/11 5/3 2/0 3/04/1 s t

26
Update n-edge Capacities Updated residual graph Capacity changes from 5 to 2 - edge capacity constraint violated! Reduce flow to satisfy constraint - causes flow imbalance! Push excess flow to the terminals Create capacity by adding α = excess to both t-edges. G` xixi xjxj 0/11 5/3 2/0 3/04/1 s t

27
Complexity analysis of MRF Update Operations MRF Energy Operation Graph OperationComplexity modifying a unary term modifying a pair-wise term adding a latent variable delete a latent variable Updating a t-edge capacity Updating a n-edge capacity adding a node set the capacities of all edges of a node zero O(1) O(k)* *requires k edge update operations where k is degree of the node

28
Finding augmenting paths is time consuming. Dual-tree maxflow algorithm [Boykov & Kolmogorov PAMI 2004] -Reuses search trees after each augmentation. -Empirically shown to be substantially faster. Our Idea –Reuse search trees from previous graph cut computation –Saves us search tree creation tree time [O(#edges)] –Search trees have to be modified to make them consistent with new graphs – Constrain the search of augmenting paths New paths must contain at least one updated edge Improving the Algorithm

29
Reusing Search Trees c = measure of change in the energy –Running time Dynamic algorithm (c + re-create search tree ) Improved dynamic algorithm (c) Video Segmentation Example - Duplicate image frames (No time is needed)

30
Dynamic Graph Cut vs Active Cuts Our method flow recycling AC cut recycling Both methods: Tree recycling

31
Experimental Analysis MRF consisting of 2x10 5 latent variables connected in a 4-neighborhood. Running time of the dynamic algorithm

32
Part II SOCP for MRF

33
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Labelling m = {1, -1, -1, 1} Random Variables V = {V 1,..,V 4 } Label Set L = {-1,1}

34
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2

35
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1

36
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1 + 2

37
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1 + 2 + 1

38
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3

39
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3 + 1

40
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3

41
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3 = 13 Minimum Cost Labelling = MAP estimate Pr(m) exp(-Cost(m))

42
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label -1 Label 1 Objectives Applicable to all types of neighbourhood relationships Applicable to all forms of pairwise costs Guaranteed to converge (Convex approximation)

43
Motivation Subgraph Matching - Torr - 2003, Schellewald et al - 2005 G1G1 G2G2 Unary costs are uniform V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D

44
Motivation Subgraph Matching - Torr - 2003, Schellewald et al - 2005 G1G1 G2G2 | d(m i,m j ) - d(V i,V j ) | < 1 2 YESNO Potts Model Pairwise Costs

45
Motivation V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D Subgraph Matching - Torr - 2003, Schellewald et al - 2005

46
Motivation V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D Subgraph Matching - Torr - 2003, Schellewald et al - 2005

47
Motivation Matching Pictorial Structures - Felzenszwalb et al - 2001 Part likelihoodSpatial Prior Outline Texture Image P1P1 P3P3 P2P2 (x,y,, ) MRF

48
Motivation Image P1P1 P3P3 P2P2 (x,y,, ) MRF Unary potentials are negative log likelihoods Valid pairwise configuration Potts Model Matching Pictorial Structures - Felzenszwalb et al - 2001 1 2 YESNO

49
Motivation P1P1 P3P3 P2P2 (x,y,, ) Pr(Cow)Image Unary potentials are negative log likelihoods Matching Pictorial Structures - Felzenszwalb et al - 2001 Valid pairwise configuration Potts Model 1 2 YESNO

50
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

51
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label -1 Label 1 Unary Cost Unary Cost Vector u = [ 5 2 ; 2 4 ] T Labelling m = {1, -1} Label vector x = [ -1 V 1 =-1 1 V 1 = 1 ; 1 -1 ] T Recall that the aim is to find the optimal x

52
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label -1 Label 1 Unary Cost Unary Cost Vector u = [ 5 2 ; 2 4 ] T Labelling m = {1, -1} Label vector x = [ -11; 1 -1 ] T Sum of Unary Costs = 1 2 i u i (1 + x i )

53
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label -1 Label 1 Pairwise Cost Labelling m = {1, -1} 0 Cost of V 1 = -1 and V 1 = -1 0 00 0 Cost of V 1 = -1 and V 2 = -1 3 Cost of V 1 = 0-1and V 2 = 1 10 00 00 10 30 Pairwise Cost Matrix P

54
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label -1 Label 1 Pairwise Cost Labelling m = {1, -1} Pairwise Cost Matrix P 00 00 0 3 10 00 00 10 30 Sum of Pairwise Costs 1 4 ij P ij (1 + x i )(1+x j )

55
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Pairwise Cost Labelling m = {1, 0} Pairwise Cost Matrix P 00 00 0 3 10 00 00 10 30 Sum of Pairwise Costs 1 4 ij P ij (1 + x i +x j + x i x j ) 1 4 ij P ij (1 + x i + x j + X ij )= X = x x T X ij = x i x j

56
Integer Programming Formulation Constraints Each variable should be assigned a unique label x i = 2 - |L| i V a Marginalization constraint X ij = (2 - |L|) x i j V b

57
Integer Programming Formulation Chekuri et al., SODA 2001 x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i {-1,1} X = x x T Convex Non-Convex

58
Key Point In modern optimization the issue is not linear vs non linear but convex vs nonconvex We want to find a convex and good relaxation of the integer program.

59
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

60
Linear Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i {-1,1} X = x x T Chekuri et al., SODA 2001 Retain Convex Part Relax Non-convex Constraint

61
Linear Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X = x x T Chekuri et al., SODA 2001 Retain Convex Part Relax Non-convex Constraint

62
Linear Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] Chekuri et al., SODA 2001 Retain Convex Part X becomes a variable to be optimized

63
Feasible Region (IP) x {-1,1}, X = x 2 Linear Programming Formulation Feasible Region for X.

64
Feasible Region (IP) Feasible Region (Relaxation 1) x {-1,1}, X = x 2 x [-1,1], X = x 2 Linear Programming Formulation Feasible Region for X.

65
Feasible Region (IP) Feasible Region (Relaxation 1) Feasible Region (Relaxation 2) x {-1,1}, X = x 2 x [-1,1], X = x 2 x [-1,1] Linear Programming Formulation Feasible Region for X.

66
Linear Programming Formulation Bounded algorithms proposed by Chekuri et al, SODA 2001 -expansion - Komodakis and Tziritas, ICCV 2005 TRW - Wainwright et al., NIPS 2002 TRW-S - Kolmogorov, AISTATS 2005 Efficient because it uses Linear Programming Not accurate

67
Semidefinite Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i {-1,1} X = x x T Lovasz and Schrijver, SIAM Optimization, 1990 Retain Convex Part Relax Non-convex Constraint

68
x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X = x x T Semidefinite Programming Formulation Retain Convex Part Relax Non-convex Constraint Lovasz and Schrijver, SIAM Optimization, 1990

69
Semidefinite Programming Formulation x1x1 x2x2 xnxn 1... 1x1x1 x2x2... xnxn 1xTxT x X = Rank = 1 X ii = 1 Positive Semidefinite Convex Non-Convex

70
Semidefinite Programming Formulation x1x1 x2x2 xnxn 1... 1x1x1 x2x2... xnxn 1xTxT x X = X ii = 1 Positive Semidefinite Convex

71
Schurs Complement AB BTBT C = I0 B T A -1 I A0 0 C - B T A -1 B IA -1 B 0 I 0 A 0 C -B T A -1 B 0

72
Semidefinite Programming Formulation X - xx T 0 1xTxT x X = 10 x I 10 0 X - xx T IxTxT 0 1 Schurs Complement

73
x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X = x x T Semidefinite Programming Formulation Relax Non-convex Constraint Retain Convex Part Lovasz and Schrijver, SIAM Optimization, 1990

74
x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] Semidefinite Programming Formulation X ii = 1 X - xx T 0 Retain Convex Part Lovasz and Schrijver, SIAM Optimization, 1990

75
Feasible Region (IP) x {-1,1}, X = x 2 Semidefinite Programming Formulation Feasible Region for X.

76
Feasible Region (IP) Feasible Region (Relaxation 1) x {-1,1}, X = x 2 x [-1,1], X = x 2 Semidefinite Programming Formulation Feasible Region for X.

77
Feasible Region (IP) Feasible Region (Relaxation 1) Feasible Region (Relaxation 2) x {-1,1}, X = x 2 x [-1,1], X = x 2 x [-1,1], X x 2 Semidefinite Programming Formulation Feasible Region for X.

78
Semidefinite Programming Formulation Formulated by Lovasz and Schrijver, 1990 Finds a full X matrix Max-cut - Goemans and Williamson, JACM 1995 Max-k-cut - de Klerk et al, 2000 Torr AI Stats for labeling problem (2003 TR 2002) Accurate, but not efficient as Semidefinite Programming algorithms slow

79
Previous Work - Overview LPSDP Examples TRW-S, -expansion Max-k-Cut Torr 2003 AccuracyLowHigh EfficiencyHighLow Is there a Middle Path ???

80
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

81
Second Order Cone Programming Second Order Cone || v || t OR || v || 2 st x 2 + y 2 z 2

82
Minimize f T x Subject to || A i x+ b i || <= c i T x + d i i = 1, …, L Linear Objective Function Affine mapping of Second Order Cone (SOC) Constraints are SOC of n i dimensions Feasible regions are intersections of conic regions Second Order Cone Programming

83
|| v || t tIv vTvT t 0 LP SOCP SDP = 10 vTvT I tI0 0 t 2 - v T v Iv 0 1 Schurs Complement

84
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

85
First quick definition: Matrix Dot Product AB = ij A ij B ij A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 = A 11 B 11 + A 12 B 12 + A 21 B 21 + A 22 B 22

86
SDP Relaxation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X ii = 1 X - xx T 0 We will derive SOCP relaxation from the SDP relaxation Further Relaxation

87
1-D Example X - xx T 0 X - x 2 0 For two semidefinite matrices, the dot product is non-negative A A 0 x 2 X SOC of the form || v || 2 st, s is a scalar constant.

88
Feasible Region (IP) Feasible Region (Relaxation 1) Feasible Region (Relaxation 2) x {-1,1}, X = x 2 x [-1,1], X = x 2 x [-1,1], X x 2 SOCP Relaxation For 1D: Same as the SDP formulation Feasible Region for X.

89
2-D Example X 11 X 12 X 21 X 22 1X 12 1 = X = x1x1x1x1 x1x2x1x2 x2x1x2x1 x2x2x2x2 xx T = x12x12 x1x2x1x2 x1x2x1x2 = x22x22

90
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2. 0 10 00 1 - x 2 2 x 1 2 1 -1 x 1 1 C 1. 0 C 1 0

91
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2 C 2. 0. 0 00 01 X 12 -x 1 x 2 1 - x 2 2 x 2 2 1 LP Relaxation -1 x 2 1 C 2 0

92
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2 C 3. 0. 0 11 11 X 12 -x 1 x 2 1 - x 2 2 (x 1 + x 2 ) 2 2 + 2X 12 SOC of the form || v || 2 st C 3 0

93
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2 C 4. 0. 0 1 1 X 12 -x 1 x 2 1 - x 2 2 (x 1 - x 2 ) 2 2 - 2X 12 SOC of the form || v || 2 st C 4 0

94
General form of SOC constraints Consider a matrix C 1 = UU T 0 (X - xx T ) ||U T x || 2 X. C 1 C 1. 0 Continue for C 2, C 3, …, C n SOC of the form || v || 2 st Kim and Kojima, 2000

95
SOCP Relaxation How many constraints for SOCP = SDP ? Infinite. For all C 0 We specify constraints similar to the 2-D example

96
SOCP Relaxation Muramatsu and Suzuki, 2001 10 00 00 01 11 11 1 1 Constraints hold for the above semidefinite matrices

97
SOCP Relaxation Muramatsu and Suzuki, 2001 10 00 00 01 11 11 1 1 a + b + c+ d a 0 b 0 c 0 d 0 Constraints hold for the linear combination

98
SOCP Relaxation Muramatsu and Suzuki, 2001 a+c+dc-d b+c+d a 0 b 0 c 0 d 0 Includes all semidefinite matrices where Diagonal elements Off-diagonal elements

99
SOCP Relaxation - A x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X ii = 1 X - xx T 0

100
SOCP Relaxation - A x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] (x i + x j ) 2 2 + 2X ij (x i - x j ) 2 2 - 2X ij Specified only when P ij 0 i.e. sparse!!

101
Triangular Inequality At least two of x i, x j and x k have the same sign At least one of X ij, X jk, X ik is equal to one X ij + X jk + X ik -1 X ij - X jk - X ik -1 -X ij - X jk + X ik -1 -X ij + X jk - X ik -1 SOCP-B = SOCP-A + Triangular Inequalities

102
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

103
Robust Truncated Model Pairwise cost of incompatible labels is truncated Potts ModelTruncated Linear Model Truncated Quadratic Model Robust to noise Widely used in Computer Vision - Segmentation, Stereo

104
Robust Truncated Model Pairwise Cost Matrix can be made sparse P = [0.5 0.5 0.3 0.3 0.5] Q = [0 0 -0.2 -0.2 0] Reparameterization Sparse Q matrix Fewer constraints

105
Compatibility Constraint Q(m a, m b ) < 0 for variables V a and V b Relaxation Q ij (1 + x i + x j + X ij ) < 0 SOCP-C = SOCP-B + Compatibility Constraints

106
SOCP Relaxation More accurate than LP More efficient than SDP Time complexity - O( |V| 3 |L| 3 ) Same as LP Approximate algorithms exist for LP relaxation We use |V| 10 and |L| 200

107
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

108
Subgraph Matching Subgraph Matching - Torr - 2003, Schellewald et al - 2005 G1G1 G2G2 Unary costs are uniform V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D Pairwise costs form a Potts model

109
Subgraph Matching 1000 pairs of graphs G 1 and G 2 #vertices in G 2 - between 20 and 30 #vertices in G 1 - 0.25 * #vertices in G 2 5% noise to the position of vertices NP-hard problem

110
Subgraph Matching Method Time (sec) Accuracy (%) LP0.856.64 LBP0.278.6 GBP1.585.2 SDP-A35.093.11 SOCP-A3.092.01 SOCP-B4.594.79 SOCP-C4.896.18

111
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures

112
Pictorial Structures Image P1P1 P3P3 P2P2 (x,y,, ) MRF Matching Pictorial Structures - Felzenszwalb et al - 2001 Outline Texture

113
Pictorial Structures Image P1P1 P3P3 P2P2 (x,y,, ) MRF Unary costs are negative log likelihoods Pairwise costs form a Potts model | V | = 10| L | = 200

114
Pictorial Structures ROC Curves for 450 +ve and 2400 -ve images

115
Pictorial Structures ROC Curves for 450 +ve and 2400 -ve images

116
Conclusions We presented an SOCP relaxation to solve MRF More efficient than SDP More accurate than LP, LBP, GBP #variables can be reduced for Robust Truncated Model Provides excellent results for subgraph matching and pictorial structures

117
Future Work Quality of solution –Additive bounds exist –Multiplicative bounds for special cases ?? –What are good Cs. Message passing algorithm ?? –Similar to TRW-S or -expansion –To handle image sized MRF

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google