Download presentation

Presentation is loading. Please wait.

Published byMarisol Wafer Modified over 2 years ago

1
Cutting Images: Graphs and Boundary Finding Computational Photography Derek Hoiem, University of Illinois 09/15/11 “The Double Secret”, Magritte

2
Matlab demo: image enhancement

3
Project 1 Results Most people received at least one vote for favorite project or result Class Favorite Projects – Hanna, Jihua Class Favorite Results – Greg: three-source hybrid – Sahil: actor/caricature – Honorable mentions (at least two votes): Kalinka: self/meloncat, Potter/Lennon; Sukolsak: Chewbacca/Leia, Jihua: world/watermelon

4
Project 2: Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collectionProkudin-Gorskii Due Monday (Sept 24), 11:59pm Simple image alignment Coarse-to-fine alignment Adjust colors/contrast Bells and whistles – Automatically crop borders – Align using FFT – More enhancement

5
This month: The digital canvas Camera models, single-view geometry, and 3D reconstruction Image warping and object morphing Cutting and pasting objects, filling holes, and blending

6
This Lecture: Finding Seams and Boundaries Segmentation

7
This Lecture: Finding Seams and Boundaries Retargeting http://swieskowski.net/carve/

8
This Lecture: Finding Seams and Boundaries Stitching

9
This Lecture: Finding seams and boundaries Fundamental Concept: The Image as a Graph Intelligent Scissors: Good boundary = short path Graph cuts: Good region is has low cutting cost

10
Semi-automated segmentation User provides imprecise and incomplete specification of region – your algorithm has to read his/her mind. Key problems 1.What groups of pixels form cohesive regions? 2.What pixels are likely to be on the boundary of regions? 3.Which region is the user trying to select?

11
What makes a good region? Contains small range of color/texture Looks different than background Compact

12
What makes a good boundary? High gradient along boundary Gradient in right direction Smooth

13
The Image as a Graph Node: pixel Edge: cost of path or cut between two pixels

14
Intelligent Scissors Mortenson and Barrett (SIGGRAPH 1995)

15
Intelligent Scissors A good image boundary has a short path through the graph. Mortenson and Barrett (SIGGRAPH 1995) 1 21 4 1 6 9 1 3 1 4 1 1 3 2 3 5 Start End

16
Intelligent Scissors Formulation: find good boundary between seed points Challenges – Minimize interaction time – Define what makes a good boundary – Efficiently find it

17
Intelligent Scissors: method 1.Define boundary cost between neighboring pixels 2.User specifies a starting point (seed) 3.Compute lowest cost from seed to each other pixel 4.Get path from seed to cursor, choose new seed, repeat

18
Intelligent Scissors: method 1.Define boundary cost between neighboring pixels a)Lower if edge is present (e.g., with edge(im, ‘canny’)) b)Lower if gradient is strong c)Lower if gradient is in direction of boundary

19
Gradients, Edges, and Path Cost Gradient Magnitude Edge Image Path Cost

20
Intelligent Scissors: method 1.Define boundary cost between neighboring pixels 2.User specifies a starting point (seed) – Snapping

21
Intelligent Scissors: method 1.Define boundary cost between neighboring pixels 2.User specifies a starting point (seed) 3.Compute lowest cost from seed to each other pixel – Djikstra’s shortest path algorithm

22
Djikstra’s shortest path algorithm Initialize, given seed s: Compute cost 2 (q, r) % cost for boundary from pixel q to neighboring pixel r cost(s) = 0 % total cost from seed to this point A = {s} % set to be expanded E = { } % set of expanded pixels P(q) % pointer to pixel that leads to q Loop while A is not empty 1.q = pixel in A with lowest cost 2.Add q to E 3.for each pixel r in neighborhood of q that is not in E a)cost_tmp = cost(q) + cost 2 (q,r) b)if (r is not in A) OR (cost_tmp < cost(r)) i.cost(r) = cost_tmp ii. P(r) = q iii.Add r to A

23
Intelligent Scissors: method 1.Define boundary cost between neighboring pixels 2.User specifies a starting point (seed) 3.Compute lowest cost from seed to each other pixel 4.Get new seed, get path between seeds, repeat

24
Intelligent Scissors: improving interaction 1.Snap when placing first seed 2.Automatically adjust to boundary as user drags 3.Freeze stable boundary points to make new seeds

25
Where will intelligent scissors work well, or have problems?

26
Grab cuts and graph cuts Grab cuts and graph cuts User Input Result Magic Wand (198?) Intelligent Scissors Mortensen and Barrett (1995) GrabCut Regions Boundary Regions & Boundary Source: Rother

27
Segmentation with graph cuts Source (Label 0) Sink (Label 1) Cost to assign to 0 Cost to assign to 1 Cost to split nodes

28
Segmentation with graph cuts Source (Label 0) Sink (Label 1) Cost to assign to 0 Cost to assign to 1 Cost to split nodes

29
Colour Model Colour Model Gaussian Mixture Model (typically 5-8 components) Foreground & Background Background Foreground Background G R G R Iterated graph cut Source: Rother

30
Graph cuts Boykov and Jolly (2001) Image Min Cut Min Cut Cut: separating source and sink; Energy: collection of edges Min Cut: Global minimal enegry in polynomial time Foreground (source) Background (sink) Source: Rother

31
Graph cuts segmentation 1.Define graph – usually 4-connected or 8-connected 2.Set weights to foreground/background – Color histogram or mixture of Gaussians for background and foreground 3.Set weights for edges between pixels 4.Apply min-cut/max-flow algorithm 5.Return to 2, using current labels to compute foreground, background models

32
What is easy or hard about these cases for graphcut- based segmentation?

33
Easier examples GrabCut – Interactive Foreground Extraction 10

34
More difficult Examples Camouflage & Low Contrast Harder Case Fine structure Initial Rectangle Initial Result GrabCut – Interactive Foreground Extraction 11

35
Lazy Snapping (Li et al. SG 2004)

36
Limitations of Graph Cuts Requires associative graphs – Connected nodes should prefer to have the same label Is optimal only for binary problems

37
Other applications: Seam Carving Demo: http://swieskowski.net/carve/ Seam Carving – Avidan and Shamir (2007)

38
Other applications: Seam Carving Find shortest path from top to bottom (or left to right), where cost = gradient magnitude Demo: http://swieskowski.net/carve/ Seam Carving – Avidan and Shamir (2007) http://www.youtube.com/watch?v=6NcIJXTlugc

39
Other applications: stitching Graphcut Textures – Kwatra et al. SIGGRAPH 2003

40
Other applications: stitching + Graphcut Textures – Kwatra et al. SIGGRAPH 2003 Ideal boundary: 1.Similar color in both images 2.High gradient in both images

41
Summary of big ideas Treat image as a graph – Pixels are nodes – Between-pixel edge weights based on gradients – Sometimes per-pixel weights for affinity to foreground/background Good boundaries are a short path through the graph (Intelligent Scissors, Seam Carving) Good regions are produced by a low-cost cut (GrabCuts, Graph Cut Stitching)

42
Take-home questions 1. What would be the result in “Intelligent Scissors” if all of the edge costs were set to 1? 09/15/11 2. Typically, “GrabCut” will not work well on objects with thin structures. How could you change the boundary costs to better segment such objects?

43
Next class Discussion of project 1 Texture synthesis and hole-filling

Similar presentations

OK

Recap from Monday Frequency domain analytical tool computational shortcut compression tool.

Recap from Monday Frequency domain analytical tool computational shortcut compression tool.

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on standing order template Ppt on importance of drinking water Quiz ppt on india Ppt on loan against property Ppt on topic health and medicine Ppt on fibonacci numbers and nature Ppt on sudoku game project Ppt on chromosomes and genes share Ppt on minimum wages act 2015 Ppt on transcription and translation