1. Presenter ： Kuang-Jui Hsu Date ： 2011/3/24(Thur.)

2. Outline 1.Important Journal & Conference 2.Paper Outline

3. Important Journal & Conference Journal RankingsJournal Rankings: 1. TPAMI: IEEE Transactions on Pattern Analysis and Machine IntelligenceIEEE Transactions on Pattern Analysis and Machine Intelligence 2. IJCV: International Journal of Computer VisionInternational Journal of Computer Vision Conference RankingsConference Rankings : 1. CVPR: IEEE Conference on Computer Vision and Pattern RecognitionIEEE Conference on Computer Vision and Pattern Recognition 2. ICCV: International Conference on Computer VisionInternational Conference on Computer Vision 3. ECCV: European Conference On Computer VisionEuropean Conference On Computer Vision

4. Paper Outline 1. Energy minimization in early vision 2. Energy minimization via graph cuts 3. Finding the optimal swap move 4. Finding the optimal expansion move 5. Experimental results

5. Energy minimization in early vision Goal: find a labeling f that assigns each pixel where f is both piecewise smooth and consistent with the observed data. Energy function: : the extent to which is not piecewise smooth. : the disagreement between f and the observed data. Date energy: : how appropriate a label is for the pixel p given the observed data Ɲ : the set of pairs of adjacent pixels In this paper, authors allow to be arbitrary

6. Energy minimization in early vision Smooth energy: a critical issue, and many different function have been proposed Major difficult: the enormous computation cost, but typically these energy functions have many local minima. Simulated annealing was popularized in computer vision and is widely used since it can optimize an arbitrary energy function. Minimizing an arbitrary requires exponential time, and as consequence simulated annealing is very slow. Annealing is inefficient partly because at each step it changes the value of a single pixel. In special cases such energies can be minimized exactly, but the problem is NP-hard in general.

7. Energy minimization in early vision Authors develop algorithms that approximately minimize energy E(f) for an arbitrary finite set of labels under two fairly general classes of interaction potentials V : a) Semi-metric: on the space, for any pair of labels α, β, it satisfies two properties. 1. 2. b) Metric: the condition of the semi-metric plus the triangle inequality for any α, β, γ in Compute a labeling which is a local minimum even when very large moves are allowed. Move : gives an arbitrary label α to a single pixel

8. Energy minimization in early vision The authors’ algorithms are based on graph cuts. The authors’ two algorithms : a) First algorithm: based on α-β-swap moves and works for any semi-metric b) Second algorithm: based on α-expansion moves but works only for metric Note that α-expansion moves produce a solution within a known factor of the global minimum of E

9. Energy minimization via graph cuts Partitions and moves Algorithms and properties Graph cuts

10. Partitions and moves Any labeling f can be uniquely represented by a partition of image pixels P=, where is a subset of pixels assigned label l. One to one correspondence between labelings f and partitions P α-β swap: given a pairs of labels α,β, a move from a partition P (labeling f ) to a new partition (labeling ) if Difference between P and : some pixels that were labeled α in P are now labeled β in, and some pixels that were labeled β in P are now labeled α in. α-expansion: given a label α, a move from a partition P (labeling f ) to a new partition (labeling ) An α-expansion move allows any set of image pixels to change their labels to α.

11. Algorithms and properties α-β swap move algorithm α-expansion move algorithm

12. Algorithms and properties Three important properties: Guarantee to terminate in a finite number of cycles; Under the fairly general assumptions, terminate in cycles. Once the algorithm has terminated, the energy of the resulting labeling is a local minimum with respect to a swap or an expansion move. Expansion move algorithm produces a labeling f such that where is the global minimum and

13. Graph cuts Key part: step 3.1, where graph cuts are used to efficiently find Let be a weighted graph with two distinguished vertices called terminals. A cut is a set of edges such that the terminals are separated in the induced graph The cost of cut C, denoted |C|, equals the sum of its edge weights. Step 3.1 uses a single minimum cut on a graph whose size is. 1.α, β are terminals. 2., and are weighted edges

14. Finding the optimal swap move Critical step: Given an input labeling f (partition P) and a pair of labels α, β, we wish to find a labeling that minimizes E over all labelings within one α- β swap of f. The technique is based on computing a labeling corresponding to a minimum cut on a graph

15. Finding the optimal swap move 1.include two terminals α, β 2. 3.The set consists of α, β 4. 5.Each pixel is connected to the terminals α, β by edge 6.t-links: terminal links( ) 7.n-links: neighbor links, each pair of pixels which are neighbor connected by an edge 8.The set of edges thus consists of (the t-links) and (the n-links)

16. Finding the optimal swap move Weights assigned to the edges are

17. Finding the optimal swap move Any cut C on must sever (include) exactly one t-link for any pixel Define a natural labeling corresponding to a cut C on Lemma 1: A labeling corresponding to a cut on is one α-β swap away from the initial labeling f.

18. Finding the optimal swap move Property 1: For any cut C and for any n-links

19. Finding the optimal swap move Lemma 2 : For any cut C and for any n-link

20. Finding the optimal swap move Theorem 1: There is a one to one correspondence between cuts C on and labelings that are one α-β swap from f. Moreover, the cost of a cut C on is |C| = plus a constant. Proof : The severed t-links uniquely determine the labels assigned to pixels p and n-links that must to be cut. Compute the cost of a cut C |C|=

21. Finding the optimal swap move Corollary 1 : The optimal α-β swap from f is where C is the minimum cut on a graph

22. Finding the optimal expansion move Given an input labeling f (partition P) and a label α, we wish to find a lebeling that minimizes E over all labelings within one α-expansion of f. Solve the problem assuming each is metric, and thus satisfies the triangle inequality. Based on computing a labeling corresponding to a minimum cut on a graph

23. Finding the optimal expansion move The structure of graph The set of vertices includes the two terminals α and as well as all image pixels For each pair of neighboring pixels separated in the current partition(i.e. ), create an auxiliary vertex Auxiliary nodes are introduced at the boundaries between partition sets. The set of vertices : Each pixel is connected to the terminals α and by t-links, correspondingly. Each pair of neighboring pixels which are not separated by the partition P (i.e. ) is connected by an n-link. For each pair of neighboring pixels {p, q} such that,create a triplet of edges Where a = is the corresponding auxiliary node. The edges and connected pixels p and q to and the t-link connects the auxiliary node to the terminal The set of all edges

24. Finding the optimal expansion move This defines a natural labeling corresponding to a cut C on The weights:

25. Finding the optimal expansion move Any cut C on must sever(include) exactly one t-link for any pixel Lemma 3 A labeling corresponding to a cut C on is one α-expansion away from the initial labeling f.

26. Finding the optimal expansion move Assume that is an auxiliary node between the corresponding pair of neighboring pixels. Property 2 A minimum cut C on satisfies:

27. Finding the optimal expansion move Lemma 4 If and then the minimum cut C on satisfies ( ) Property 1 holds when we substitute for. We will refer to this as property 1( ). property 1( ) : Property 1( ) holds for any cut, and Property 2 holds for a minimum cut. Elementary cut on : A cut that satisfies Properties 1( ) and 2.

28. Finding the optimal expansion move Theorem 2 Let be constructed as above given f and α. Then there is a one to one correspondence between elementary cuts on and labelings within one α-expansion of f. Moreover, for any elementary cut C we have Proof : The label at the pixel p determines which of the t-links to p is in C. Property 1( ) shows which n-links between pairs of neighboring pixels {p, q} such that should be severed. Similarly, Property 2 determines which of the links in corresponding to {p, q} such that should be cut. The cost of an elementary cut C : For any pixel =>

29. Finding the optimal expansion move Lemmas 2 and 4 hold for elementary cuts, since they were based on properties 1 and 2 => => The minimum cut is an elementary cut. Corollary 2 The optimal α-expansion from f is where C is the minimum cut on.

30. Experimental results Use three energy function, each with a quadratic The First energy function, called E1, uses the truncated quadratic as its smoothness term. The second(E2) and third (E3) energy function use the Potts model and the truncated L2 distance, correspondingly, as their smoothness penalty V. Potts model: Truncated distance:

31. Experimental results Simulated annealing Authors’ algorithm

32. Experimental results