Comp 775: Graph Cuts and Continuous Maximal Flows Marc Niethammer, Stephen Pizer Department of Computer Science University of North Carolina, Chapel Hill

2 Representations BackgroundGraph Cuts and Continuous Maximal Flows Classifying individual pixels versus finding an optimal separating curve/surface between object and background.

3 From curve evolution to pixel/voxel labeling BackgroundGraph Cuts and Continuous Maximal Flows Example: Chan-Vese model Indicator function Putting it all together: Relaxed indicator function

4 Convex vs. Non-Convex BackgroundGraph Cuts and Continuous Maximal Flows Non-convex Initial contour Final contour Images: Bresson et al. Convex Prone to locally optimal solutions. Independent of initial condition, optimal solution is guaranteed.

5 From Curve Evolution to Pixel/Voxel Labeling BackgroundGraph Cuts and Continuous Maximal Flows Convex, continuous, constrained optimization problems.... can also include region-based terms, appearance information, orientation-dependency Advantages: - Convex -> Globally optimal - No metrication artifacts - Straightforward parallel implementations convex non-convex

6 Continuous Maximal Flow [Appleton] Continuous Maximal FlowGraph Cuts and Continuous Maximal Flows Continuous version of maximal flow [Appleton]. Results in a PDE and can be solved as such. Energy to be minimized: More general form (can easily include local classifiers):

7 Segmentation with Continuous Max-Flow Continuous Maximal FlowGraph Cuts and Continuous Maximal Flows Images: Unger Thresholding Continuous max-flow with seeds.

8 Continuous Maximal Flow Graph Cuts and Continuous Maximal Flows Energy to be minimized: Introduce the auxiliary variable p, which we maximize for: Now solve this by gradient descent.

9 Continuous Maximal Flow Graph Cuts and Continuous Maximal Flows The variation is The gradient descent scheme becomes

10 Chan-Vese (=Otsu-Thresholding w/ spatial regularity) Continuous Max-FlowGraph Cuts and Continuous Maximal Flows Iterative solution method is related to solving a wave equation.

11 Ex.: Segmentation with Continuous Max-Flow Continuous Max-FlowGraph Cuts and Continuous Maximal Flows

12 3D Example Continuous Max-FlowGraph Cuts and Continuous Maximal Flows

13 Segmentation by Graph-Cut Graph-CutGraph Cuts and Continuous Maximal Flows Example: Binary Segmentation Images from ECCV Tutorial, Kumar/Kohli Need to partition the picture into foreground and background. Alternative approach through graph construction (vs. PDE)

14 Segmentation by Graph-Cut Graph-CutGraph Cuts and Continuous Maximal Flows Graph G=(V,E)‏ Images from ECCV Tutorial, Kumar/Kohli Approach: Interpret the image as a graph, where pixels are connected to its neighbors. Goal: Cut the graph into pieces to obtain the desired image partition. Assign a label to every pixel.

15 Segmentation by Graph-Cut Graph-CutGraph Cuts and Continuous Maximal Flows Optimization Problem: Minimize Looking at it on a pixel-by-pixel basis (where f is the labeling): Problem: We cannot try all possible pairings for the labels f. Need an efficient algorithm to solve this problem.

16 Maximum Flow and Minimum Cut Graph-CutGraph Cuts and Continuous Maximal Flows Solution: 1) Transform problem to maximizing nework flow 2) Use algorithms for network flows on images. Preview: Graph structure for binary labeling Pixels are connected to neighbors (pairwise interaction cost) and to source and target vertices (data cost). Images: Boykov Cut separating source (s) from sink (t) gives the segmentation.

17 Max-Flow/Min-Cut Theorem Graph-CutGraph Cuts and Continuous Maximal Flows For any network having a single origin and a single destination node, the maximum possible flow from origin to destination equals the minimum cut value for all the cuts in the network.

18 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Pixel weights: s to node flow: high if near source (bkg) class, low if not Pixel weights: Node to t flow: high if near target class, low if not Node to neighbor flow: high in both directions if near intensities

19 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows One way to compute maximal flow: 1) Pick any viable path (no zero flow) 2) Subtract minimum flow from each segment on path 3) Add minimum flow on reverse path

20 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 1) Pick path (no zero flow)

21 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 2) Subtract minimum flow from each segment on path 3) Add minimum flow on reverse path

22 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

23 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 1) Pick path (no zero flow)

24 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 2) Subtract minimum flow from each segment on path 3) Add minimum flow on reverse path

25 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

26 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 1) Pick path (no zero flow)

27 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 2) Subtract minimum flow from each segment on path 3) Add minimum flow on reverse path

28 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

29 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 1) Pick path (no zero flow)

30 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows 2) Subtract minimum flow from each segment on path 3) Add minimum flow on reverse path

31 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows

32 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows - Cut at zeros. - Cost of min-cut is 4. - Divides the nodes (pixels) into two groups. - Sum of original values of zeros equals the maximum flow.

33 Example: Ford & Fulkerson Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Look at the difference between initial and final capacities. Ignore negative capacities. This is where everything flows. Flow is conserved at nodes.

34 Why update the backward flow? Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Image: John Chinneck Can undo flows. Final flow does not include the central edge.

35 Image Segmentation Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Allows (amongst many things) to compute binary segmentations. Image: Boykov OriginalNoisyReconstructed

36 Multi-Label Case Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Image: Boykov

37 Multi-Label Case Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Image: Boykov

38 Influence of Neighborhood Choice Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Minimum cost cut (standard 4-neighborhoods)‏ Minimum length geodesic contour (image-based Riemannian metric)‏ Images: Boykov Can choose different weighted neighborhood to reduce metrication errors. (Or use continuous maximal flow.)

39 Interactive Segmentations Segmentation by Graph CutGraph Cuts and Continuous Maximal Flows Images: Boykov