1. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 1 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

2. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 2 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

3. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 3 Blocks-World Scene Analysis (Adolfo Guzman, MIT AI Lab, 1967) Input: a line drawing representing a 3D scene consisting of polyhedra Output: lists of faces grouped into objects Method: (a) classify vertices, (b) link faces according to rules, (c) extract connected components of doubly linked regions.

4. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 4

5. 5 Guzman Vertex Categories ell, arrow, fork, tee, kay

6. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 6 Guzman Linking Rules ell, arrow, fork, tee, kay

7. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 7 Guzman Face Grouping Criterion: Two adjacent faces must be doubly linked to be considered part of the same object. E F D G C B A Obj1: (A B C) Obj2: (D E F G)

8. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 8 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

9. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 9 Impossible Figures Locally consistent, globally inconsistent

10. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 10 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

11. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 11 Edge Detection Why? Find boundaries of objects in order to compute their features: shape, area, moments, corner locations, etc. Methods: Compute measures of change in the neighborhood of each pixel. Trace boundaries Transform each neighborhood into a vector space whose basis includes "edge" vectors.

12. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 12 Horizontal differences: a – b Both horiz. & vertical diffs. Roberts' cross operator (a-d) 2 +(b-c) 2 Local Edge Detectors ab ab cd

13. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 13 Express each 3x3 neighborhood as a vector. N = Frei-Chen Edge Detection e h cd fg aab

14. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 14 Frei-Chen Edge Det. (cont) Transform it linearly into the 9D vector space spanned by the following basis vectors (each shown in 3x3 format)

15. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 15 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

16. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 16 Instead of using 1 st derivatives, let's consider 2 nd derivatives. The Laplacian effectively finds subtle changes in intensity.  2 f(x,y) =  2 f(x,y)/  x 2 +  2 f(x,y)/  y 2 discrete approximation: Edge Finding with the Laplacian 0 4 0 00

17. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 17 Remedy: First, smooth the signal with a Gaussian filter. g(x,y) = 1/(2  2 ) exp(-(x 2 +y 2 )/2  2 ) Then take the Laplacian. Or, directly convolve directly with a Laplacian of Gaussian (LOG). Laplacian: Very Sensitive to Noise

18. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 18 The edges tend to be where the second derivative crosses 0. Zero Crossings

19. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 19 Zero Crossings: Example (a) orig. (b) LoG with  = 1, (c) zero crossings. R Fischer et al. http://homepages.inf.ed.ac.uk/rbf/HIPR2/zeros.htm

20. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 20 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

21. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 21 If two neighboring pixels share a side, they are 4-adjacent. d's 4-adjacent neighbors are: b, c, e, g If two neighboring pixels share a corner, they are 8-adjacent d's 8-adjacent neighbors are: a,b,c,e,f,g,h 4- and 8-adjacency e h cd fg aab

22. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 22 A set of pixels R is said to be 4-connected, provided for any p i, p j in R, there is a chain [(p i, p i' ), (p i', p i'' ),..., (p k, p j )] involving only pixels in R, and each pair in the chain is 4- adjacent. The chain may be of length 0. R is 8-connected, iff there exists such a chain where each pair is 8-adjacent. 4- and 8-connectedness

23. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 23 4- and 8-connected components 4-connected components of black: 4 8-connected components of black: 2 4-connected components of white: 2 8-connected components of white: 2

24. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 24 "Dual approach" to edge detection. Let P be a digital image: P = {p 0, p 1,..., p n-1 } A segmentation S = {R 0, R 1,..., R m-1 } is a partition of P, such that For each i, R i is 4-connected, and R i is "uniform", and for all i,j if R i is adjacent to R j, then (R i U R j ) is not uniform. Segmentation into Regions

25. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 25 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

26. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 26 Can compute a segmentation if U(R) means all pixels in R have the same pixel value (e.g., intensity, color, etc.) Algorithm: Initialize a UNION-FIND ADT instance, with each pixel in its own subset. Scan the image. At each pixel p, check its neighbor r to the right. p and r have the same pixel value, determine whether FIND(p) = FIND(r), and if not, perform UNION(FIND(p), FIND(r)). Then check the pixel q below doing the same tests and possible UNION. When the scan is complete, each subset represents one 4- connected component of the image. Image Connected Components

27. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 27 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

28. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 28 Gestalt Grouping Identification of regions using a uniformity predicate based on wider-neighborhood features (e.g., texture).

29. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 29 Gestalt Grouping

30. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 30 Gestalt Grouping Texture element = “texel” Texel directionality Texel granularity Alignments of endpoints Spacing of texels Groups cue for surfaces, objects.

31. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 31 Image Understanding 2 Outline: Guzman Scene Analysis Local and Global Consistency Edge Detection Laplacians and Zero Crossings Segmentation into Regions Image Connected Components Gestalt Grouping Morphology

32. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 32 "Mathematical Morphology" Developed at the Ecoles des Mines, France by J. Serra, from mathematical ideas of Minkowski. Idea is to take two sets of points in space and obtain a new set. Set A: the main set. Set B: the "structuring element" Set C: the result.

33. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 33 Math. Morph. Operation 1: "Erosion" Consider each point b of B to be a vector and use it to translate a copy of A getting A b Take the intersection of all the A b The result is called the erosion of A by B.

34. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 34 Erosion: example A: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 0 0 B: 1 1 C: 0 0 1 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0

35. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 35 Dilation: Take Union instead of Intersection. A: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 0 0 B: 1 1 C: 0 1 1 1 0 1 1 1 1 0 0 1 1 1 1 0 0 1 1 0

36. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 36 Opening: Erode by B, then Dilate by –B (B rotated by  ) A: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 0 0 B: 1 1 C: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 0 0 0

37. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 37 Closing: Dilate by B, then Erode by –B (B rotated by  ) A: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 0 0 B: 1 1 C: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 0 0 (In this example, erosion undoes the dilation.)

38. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 38 Closing – another example A: 0 1 1 0 0 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 B: 1 1 C: 0 1 1 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 0 0 (In this example, closing fills in holes.)

39. CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding II 39 Math. Morphol. Uses Repair regions corrupted by noise; Eliminate small regions Identify holes, bays, protrusions, isthmi, peninsulas, and specific shapes such as crosses. Printed circuit board inspection (find cracks, spurious wires.) Popular primitives for writing industrial machine vision apps.