1. Edges

2. University of Missouri at Columbia
Gradients and edges
Points of sharp change in an image are interesting:
change in reflectance
change in object
change in illumination
noise
Sometimes called edge points
General strategy
determine image gradient
now mark points where gradient magnitude is particularly large wrt neighbours (ideally, curves of such points).
CS8690 Computer Vision
University of Missouri at Columbia

3. University of Missouri at Columbia
There are three major issues:
1) The gradient magnitude at different scales is different; which should
we choose?
2) The gradient magnitude is large along thick trail; how
do we identify the significant points?
3) How do we link the relevant points up into curves?
Figures show gradient magnitude of zebra at two different scales
CS8690 Computer Vision
University of Missouri at Columbia

4. Smoothing and Differentiation
Issue: noise
smooth before differentiation
two convolutions to smooth, then differentiate?
actually, no - we can use a derivative of Gaussian filter
because differentiation is convolution, and convolution is associative
Important to point out that the derivative of gaussian kernels I’ve illustrated “look like” the effects
that they’re trying to identify.
CS8690 Computer Vision
University of Missouri at Columbia

5. University of Missouri at Columbia
1 pixel
3 pixels
7 pixels
The scale of the smoothing filter affects derivative estimates, and also
the semantics of the edges recovered.
CS8690 Computer Vision
University of Missouri at Columbia

6. University of Missouri at Columbia
We wish to mark points along the curve where the magnitude is biggest.
We can do this by looking for a maximum along a slice normal to the curve
(non-maximum suppression). These points should form a curve. There are
then two algorithmic issues: at which point is the maximum, and where is the
next one?
CS8690 Computer Vision
University of Missouri at Columbia

7. University of Missouri at Columbia
Non-maximum
suppression
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
CS8690 Computer Vision
University of Missouri at Columbia

8. University of Missouri at Columbia
Predicting
the next
edge point
Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s).
CS8690 Computer Vision
University of Missouri at Columbia

9. University of Missouri at Columbia
Remaining issues
Check that maximum value of gradient value is sufficiently large
drop-outs? use hysteresis
use a high threshold to start edge curves and a low threshold to continue them.
CS8690 Computer Vision
University of Missouri at Columbia

10. University of Missouri at Columbia
Notice
Something nasty is happening at corners
Scale affects contrast
Edges aren’t bounding contours
CS8690 Computer Vision
University of Missouri at Columbia

11. University of Missouri at Columbia
CS8690 Computer Vision
University of Missouri at Columbia

12. University of Missouri at Columbia
fine scale
high
threshold
CS8690 Computer Vision
University of Missouri at Columbia

13. University of Missouri at Columbia
coarse
scale,
high
threshold
CS8690 Computer Vision
University of Missouri at Columbia

14. University of Missouri at Columbia
coarse
scale
low
threshold
CS8690 Computer Vision
University of Missouri at Columbia

15. The Laplacian of Gaussian
Another way to detect an extremal first derivative is to look for a zero second derivative
Appropriate 2D analogy is rotation invariant
the Laplacian
Bad idea to apply a Laplacian without smoothing
smooth with Gaussian, apply Laplacian
this is the same as filtering with a Laplacian of Gaussian filter
Now mark the zero points where there is a sufficiently large derivative, and enough contrast
CS8690 Computer Vision
University of Missouri at Columbia

16. University of Missouri at Columbia
sigma=4
contrast=1
LOG zero crossings
contrast=4
sigma=2
CS8690 Computer Vision
University of Missouri at Columbia

17. University of Missouri at Columbia
We still have unfortunate behaviour
at corners
This is the usual story; if you find edges as a level crossing of a function, then trihedral vertices
are going to cause trouble. Also, notice the bulges at the corners.
CS8690 Computer Vision
University of Missouri at Columbia

18. University of Missouri at Columbia
Filters are templates
Applying a filter at some point can be seen as taking a dot-product between the image and some vector
Filtering the image is a set of dot products
Insight
filters look like the effects they are intended to find
filters find effects they look like
CS8690 Computer Vision
University of Missouri at Columbia

19. University of Missouri at Columbia
Positive responses
Zero mean image, -1:1 scale
Zero mean image, -max:max scale
The filter is the little block in the top left hand corner. Notice this is a fair spot-detector.
CS8690 Computer Vision
University of Missouri at Columbia

20. University of Missouri at Columbia
Positive responses
Zero mean image, -1:1 scale
Zero mean image, -max:max scale
The filter is the little block in the top left hand corner. Notice this is a fair bar-detector.
CS8690 Computer Vision
University of Missouri at Columbia

21. University of Missouri at Columbia
This is a figure from the book, figure Read the caption for the story!
Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications, 1998 copyright 1998, IEEE
CS8690 Computer Vision
University of Missouri at Columbia

22. Normalized correlation
Think of filters of a dot product
now measure the angle
i.e normalised correlation output is filter output, divided by root sum of squares of values over which filter lies
Tricks:
ensure that filter has a zero response to a constant region (helps reduce response to irrelevant background)
subtract image average when computing the normalizing constant (i.e. subtract the image mean in the neighbourhood)
absolute value deals with contrast reversal
CS8690 Computer Vision
University of Missouri at Columbia

23. Region based Similarity Metric
Sum of squared differences
Normalize cross-correlation
Sum of absolute differences
CS8690 Computer Vision
University of Missouri at Columbia

24. Orientation representations
The gradient magnitude is affected by illumination changes
but it’s direction isn’t
We can describe image patches by the swing of the gradient orientation
Important types:
constant window
small gradient mags
edge window
few large gradient mags in one direction
flow window
many large gradient mags in one direction
corner window
large gradient mags that swing
CS8690 Computer Vision
University of Missouri at Columbia

25. University of Missouri at Columbia
Representing Windows
Types
constant
small eigenvalues
Edge
one medium, one small
Flow
one large, one small
corner
two large eigenvalues
CS8690 Computer Vision
University of Missouri at Columbia

26. Point Feature Extraction
Compute eigenvalues of G
If smallest eigenvalue  of G is bigger than  - mark pixel as candidate
feature point
Alternatively feature quality function (Harris Corner Detector)
CS8690 Computer Vision
University of Missouri at Columbia

27. University of Missouri at Columbia
Harris Corner Detector - Example
CS8690 Computer Vision
University of Missouri at Columbia

28. University of Missouri at Columbia
On the left, I’ve plotted the gradients on top of the image. It’s hard to see what’s going on,
which is why there’s more detail in the next slide.
CS8690 Computer Vision
University of Missouri at Columbia

29. University of Missouri at Columbia
Here we see gradient arrows at each pixel of a detailed view. In some regions, the arrows swing,
which should be detectable using our analysis of H above
CS8690 Computer Vision
University of Missouri at Columbia

30. University of Missouri at Columbia
Here H was averaged over a small window, and I’m plotting ellipses x^t H^(-1) x=eps (so if
H has one big and one small eigenvalue, I’ll see a heavily oriented ellipse; if H has two small
eigenvalues, I’ll see a tiny circle, etc.)
CS8690 Computer Vision
University of Missouri at Columbia

31. University of Missouri at Columbia
The average is now over a bigger window.
CS8690 Computer Vision
University of Missouri at Columbia