1. Segmentation (2): edge detection

2. Edge-based segmentation
A large group of methods based on information about previously detected edges in the image
Preprocessing step: edge detection
Motivation: images resulted from edge detection cannot be used as a segmentation result
Edges have to be linked into chains which correspond better with boundaries in an image
Final result: detection of boundaries of objects present in the image
Edge-based segmentation is not about one algorithm, but involves a large group of methods processing information about previously detected edges in the image. So, we need a preprocessing step, which is edge detection, and which can be done with linear filtering (see Laplacian of Gaussian, DoG etc.)
The problem with edge detection, if we remember, is the fact that its results are far from what we want from segmentation. What we want is a set of boundaries corresponding to the set of objects.
We can think about edges as primitives, which have to be grouped into intermediate representations, which are chains. Such chains hopefully correspond to meaningful boundaries in an image.

3. Edge detection with DOG and LOG

4. Edge-based segmentation (cont’d)
Task-oriented: embedding prior information (e.g. shape) into the chain forming process
Most common problems:
- false detections: edge presence in a location where there is no border
- missed detections: no edge presence where a real border exists
- selecting an appropriate threshold for the gradient image; this threshold separates significant from non-significant edge information; alternative?
Edge-based segmentation is most often task-oriented because it embeds prior knowledge into the segmentation process.
I the segmentation process, we’ll have to deal with artefacts caused by edge detection. False detections, where detected edges do not correspond to actual boundaries, and missed detections, which translate into discontinuous boundaries.
Another problem is generated by the fact that edges are obtained by binarizing the gradient image. And we saw in the previous course that binarizing is done by thresholding. Selecting a good threshold is a difficult problem because this threshold separates significant from not significant edge information. From what we have studied until now, can we find alternatives to this problem? Methods that are not based on a simple global threshold? The alternative is non-maxima suppression and hysteresis thresholding (see Canny).

5. Edge-based segmentation
Goal: Connect edges to produce object contours
Approaches:
thresholding with hysteresis
edge relaxation
edge linking
fitting: the Hough transform

6. Edge thresholding Thresholding the gradient magnitude:
{(x, y) : E(x, y) = 1} the set of edge pixels
Problem: No guarantees of continuity
Alternatives :
thresholding with hysteresis
edge relaxation

7. Edge thresholding with hysteresis
Thresholding with hysteresis uses two thresholds:
First identify all definite edge pixels.
Then, add all “maybe” pixels (those greater than t0) only if they are next to an already-labeled edge pixel.
Repeat until convergence.
Thresholding with hysteresis involves local processing and context

8. Edge relaxation Local processing
Evaluates the strength of edges in a local neighborhood for increasing and decreasing the confidence of an edge candidate
Contextual information: a ‘weak’ edge positioned between two ‘strong’ edges should probably be part of a boundary.
Isolated edges do not have supporting context, thus probably do not belong to any border.
Edge context is considered at both ends of an edge in a specified edge neighborhood.

9. Edge relaxation (cont’d)
Central edge e has a vertex at each of its ends and three possible border continuations can be found from both of these vertices

10. Edge relaxation (cont’d)
Edge relaxation is an iterative method, with edge confidences converging either to 0 (edge termination), or one (edge forms a border)

11. Edge linking Link adjacent edge pixels by seeing if they have similar
properties:
1. Similar magnitude:
for some magnitude difference threshold T.
2. Similar orientation
for some angular threshold A.
Once the links established, the sets of linked pixels (chains) can be thought as borders.
Chain linking: postprocessing step to find sets of linked pixels that are separated by small gaps; small gaps can be filled in.

12. Edge linking: Tracing borders of known regions
In some cases, we may already have an image segmented into regions for which we want to calculate boundaries.
In this case, we can simply generate the boundaries by tracing around the region contours. We can do this two ways:
Sequential tracing: Trace around each contour in the image. When we finish tracing one contour, scan the image until we run into another.
Simultaneous tracing: Make one pass through the image, using appropriate data structures to represent the progress of each contour tracing and adding pixels to the appropriate contour as encountered.

13. Fitting: The Hough Transform
The Hough transform is useful for grouping isolated edge points into image structures (patterns, models)
Edges to be grouped are not necessarily adjacenct, connected or close.
The Hough transform requires a parametric representation of the primitive (or model) of interest in the image.
Most simple primitive: a line

14. The Hough Transform for line detection
Consider one edge point (x,y)
There is an infinity of lines passing through this point.
Parametric representation of such lines (slope-intercept form):
For all lines that pass through a given point (x,y), there is a unique value of b for m.
The set of (m,b) values corresponding to the lines passing through point (x,y) form a line in (m,b) space.
Write b=y - mx on the table
Every point in image space (x,y) corresponds to a line in the parameter space (m,b).
Vice versa, every point in parameter space (m,b) corresponds to a line in image space.

15. The Hough transform: correspondence between image and parameter space
Every point in image space (x,y) corresponds to a line in the parameter space (m,b).
Vice versa, every point in parameter space (m,b) corresponds to a line in image space.
(xi, yi) define one line in parameter space
(xj, yj) define another line in parameter space
Suppose that those lines intersect and their intersection is point (a’,b’) in parameter space.
This point defines a line in image space which contains both points (xi, yi) and (xj, yj)

16. The Hough transform: accumulator
The Hough transform lets each point in the image space (x,y) vote in the (m,b) space for each point passing through it.
The votes are summarized into an accumulator.
We have to limit the number of votes → discretize the parameter space

17. The Hough transform: alternate representation
The slope-intercept form of a line cannot represent vertical lines : both m, b→
Another way is expressing a line in (, ) form
Each point in the image space votes for a sinusoid of points in the accumulator. High accumulator values are where sinusoids cross.
Finding maxima in the accumulator still equates to finding lines in the image space.
Drop a perpendicular from the origin to the line.  is the angle that the perpendicular makes with the x axis, and  is the length of the perpendicular.  is bounded by [0, 2pi] and  is bounded by the diagonal of the image.

18. The Hough algorithm
1. Find all of the desired feature points (i.e. edges) in the image
2. For each feature point xi:
3. For each possibility (line, sinusoid) pj in the accumulator that passes through the feature point:
4. Increment that position in the accumulator
5. Find maxima in the accumulator
6. If desired, map each maximum in the accumulator back to image space
We first extract edge points. Every edge point is mapped either to a line or to a sinusoid in the parameter space. In the discrete accumulator, we increment all locations which belong to that line or sinusoid.
Maxima in the accumulator define the most probable locations of the line in the image

19. Hough transform: an example
Initial image: two partially occluding rectangles
In this case, we can use the Hough (line detecting) transform to detect the eight separate straight lines segments of this image and thereby identify the true geometric structure of the subject. The Hough transform offers us the possibility to extract information of higher level for the image.
Edges detected with Canny
Canny detects well the overall boundaries in the image, but this result tells us nothing about the identity (and quantity) of primitives within this boundary description.

20. Hough transform: an example (cont’d)
Note that, although ro and theta are notionally polar coordinates, the accumulator space is plotted rectangularly with as the abscissa and as the ordinate. Note that the accumulator space wraps around at the vertical edge of the image such that, in fact, there are only 8 real peaks.
Contrast enhancement
Hough transform

21. Hough transform: an example
Mapping back from the parameter space into the image space yields a set of line descriptions of the image.
Hough transform found the 8 sides of the two rectangles.
Mapping back from Hough transform space (i.e. de-Houghing) into cartesian space yields a set of line descriptions of the image subject. By overlaying this image on an inverted version of the original, we can confirm the result that the Hough transform found the 8 true sides of the two rectangles and thus revealed the underlying geometry of the occluded scene
Also note that many of the image edges have several detected lines. This arises from having several nearby Hough-space peaks with similar line parameter values. Techniques exist for controlling this effect, but were not used here to illustrate the output of the standard Hough transform.

23. Hough transform for circles
Parametric representation of a circle:
Three parameters: center (xc, yc) and radius r
Only two parameters if size r is known
Same idea: each feature point votes for each circle it could be on.
The accumulator is 3D.

24. The generalized Hough transform
The Hough Transform can be extended to any parametrically defined shape (lines, circles, ellipses etc.)
Problem: as the number of parameters increases, so does the dimensionality of the parameter space
Computational complexity goes up exponentially with the number of parameters