Hough Transform

Introduction The Hough Transform is a general technique for extracting geometrical primitives from e.g. edge data Performed after Edge Detection It is a technique to isolate the curves of a given shape / shapes in a given image Classical Hough Transform can locate regular curves like straight lines, circles, parabolas, ellipses, etc. –Requires that the curve be specified in some parametric forms

Advantages The Hough Transform is tolerant of gaps in the edges It is relatively unaffected by noise It is also unaffected by occlusion in the image

Why Find Lines, etc? Edge detectors mark interesting features of the world, but produce a binary image By fitting lines, circles, ellipses, etc we group edge points together to give fewer, more significant entities

Why Find Lines, etc? Geometric primitives make important vision tasks easier Recognising repeating patterns Identifying Objects Matching between images Some geometric objects are invariant under projection from the (3D) world to (2D) image Straight lines in the world create straight lines on the image

Projector Camera Systems Camera Projector Screen Computer Cameras used to orient one or more projectors in relation to each other and to elements in the world such as projection surfaces.

Multiple Projectors: Shadows Muted display screen P-2 P-1 [Sukthankar, Cham & Sukthankar, CVPR-2001] Note that the projectors are aligned sufficiently so that the overlap between their output appears visually perfect.

Finding Planes Photos from: Mark Ashdown & Rahul Sukthankar - HP Labs CRL TR ● Lines fall in planes ● Discontinuities fall at plane boundries. ● Identify discontinuities ● Fit line to discontinuities

The Problem Suppose straight lines are important Edge detection provides a set of points (xi, yi) which are likely to lie on those lines But how many lines are there and what are their parameters?

Template Matching One solution is to take a straight line and match it to all possible image positions and orientations Compute a measure of fit to the edge data, e.g. count how many edges are under each possible line Incredibly expensive

Template Matching vs Hough Classic template matching takes a line, lies it on the image data and asks: Does it fit here? Here? How about here? Here then? The Hough takes each data item (edge) and asks: What lines could pass through this? Same effect, but there are many fewer edge points than possible line models

Fitting Choose a parametric object/some objects to represent a set of tokens Most interesting case is when criterion is not local – can’t tell whether a set of points lies on a line by looking only at each point and the next. Three main questions: 1. what object represents this set of tokens best 2. which of several objects gets which token? 3. how many objects are there?

Line Parameters The standard equation of a line y = mx +c If we know the line parameters m,c we can vary x, compute y and draw the line This line represents the set of (x,y) pairs that satisfy the above equation We don’t know the line parameters - we just have some data points (xi,yi)

Parameter Space But if a data point (x i,y i ) lies on a line then that line’s parameters m,c must satisfy y i = mx i + c Thus, we can represent all possible lines through (x i,y i ) by the set of (m,c) pairs that satisfy this equation Rearranging the equation gives c = -mx i + y i This also describes a straight line, but as x i and y i are known and m and c are unknown that line is in m,c space – a parameter space

Parameter Space This straight line in m,c space represents all the values of (m,c) that satisfy yi = mxi+ c, and so the parameters of all the lines that could pass through (x i,y i )

Parameter Space Suppose there are 2 edge points in our image (x 1,y 1 ) and (x 2,y 2 ) They will each generate a line in (m,c) space representing the set of lines that could pass through them

Parameter Space The intersection of our 2 lines is the intersection of 2 sets of line parameters The point of intersection therefore gives the parameters of a line through both (x 1,y 1 ) and (x 2,y 2 )

Parameter Space Taking this one step further All pixels which lie on a line in (x,y) space are represented by lines which all pass through a single point in (m,c) space The point through which they all pass gives the values of m’ and c’ in the equation of the line: y=m’x+c’. To detect lines all we need to do is transform each edge point into m,c space and look for places where lots of lines intersect This is what Hough Transforms do

The Hough Transform 1. Quantize (m,c) space into a two-dimensional array A for appropriate steps of m and c. 2. Initialize all elements of A(m,c) to zero. 3. For each pixel (xi,yi) which lies on some edge in the image, we add 1 to all elements of A(m,c) whose indices m and c satisfy yi=mxi+c. 4. Search for elements of A(m,c) which have large values - each one found corresponds to a line in the original image.

The Hough Transform

A Real Example

Line Parameters The Hough Transform only supplies the parameters of the lines it detects; this can be an advantage or a disadvantage The parametric equation of the line y=mx+c is appropriate only for explanation of the Hough transform principles This representation fails in case of vertical lines, it causes difficulties because m ==> ∞

Another Parameter Space Another equation for a straight line ρ = x cos(θ) + y sin(θ) For an n x n image - ρ is in range [0, 2n] - θ is in range [0, 2π] Each x i, y i generates a sinusoid in ρ, θ space

Polar Space

An Example Brightest point = 20 votes

Brightest point = 6 votes Another Example

Example 3

Real World Example Original Edge Detection Found Lines Parameter Space

EE Computer Vision29 Line Detection by Hough Transform

Fitting other objects The Hough transform is also used to fit points to any object that can be parameterized. (e.g. Circle, ellipse) Objects of arbitrary shape can be parameterized by building a Table. (Assumes orientation information for each token is available.) R and beta value(s) are obtained from the R-Table, based upon omega (orientation)

31 Hough Transform for Detection of Circles The parametric equation of the circle can be written as The equation has three parameters: a, b, r The curve obtained in the Hough Transform space for each edge point will be a right circular cone Point of intersection of the cones gives the parameters a, b, r

32 Hough Transform for Circles Gradient at each edge point is known We know the line on which the center will lie If the radius is also known then center of the circle can be located

Circle Example With no orientation, each token (point) votes for all possible circles. With orientation, each token can vote for a smaller number of circles.

Real World Circle Examples Crosshair indicates results of Hough transform, bounding box found via motion differencing.

Finding Coins Original Canny Edges (note noise)

Finding Coins (Continued) PennyQuarters

Finding Coins (Continued) Coin finding sample images from: Vivik Kwatra Note that because the quarters and penny are different sizes, a different Hough transform (with separate accumulators) was used for each circle size.

38 Recap In detecting lines –The parameters  and  were found out relative to the origin (0,0) In detecting circles –The radius and center were found out In both the cases we have knowledge of the shape We aim to find out its location and orientation in the image The idea can be extended to shapes like ellipses, parabolas, etc.

39 Parameters for analytic curves Analytic FormParametersEquation Line , xcos+ysin= Circle x 0, y 0,  (x-x o ) 2 +(y-y 0 ) 2 =r 2 Parabola x 0, y 0, (y-y 0 ) 2 =4(x-x o ) Ellipse x 0, y 0, a, b, (x-x o ) 2 /a 2 +(y-y 0 ) 2 /b 2 =1

Generalized Hough Transform The Generalized Hough transform can be used to detect arbitrary shapes Complete specification of the exact shape of the target object is required in the form of the R-Table Information that can be extracted are –Location –Size –Orientation –Number of occurrences of that particular shape

Creating the R-table Algorithm –Choose a reference point –Draw a vector from the reference point to an edge point on the boundary –Store the information of the vector against the gradient angle in the R-Table –There may be more than one entry in the R- Table corresponding to a gradient value

Generalized Hough Transform - Algorithm Form an Accumulator array to hold the candidate locations of the reference point For each point on the edge –Compute the gradient direction and determine the row of the R- Table it corresponds to –For each entry on the row calculate the candidate location of the reference point –Increase the Accumulator value for that point The reference point location is given by the highest value in the accumulator array

Generalized Hough Transform – Size and Orientation The size and orientation of the shape can be found out by simply manipulating the R- Table For scaling by factor S multiply the R- Table vectors by S For rotation by angle , rotate the vectors in the R-Table by angle 

Generalized Hough Transform – Advantages and disadvantages Advantages –A method for object recognition –Robust to partial deformation in shape –Tolerant to noise –Can detect multiple occurrences of a shape in the same pass Disadvantages –Lot of memory and computation is required

Improvisation of the Hough transform for detecting straight line segments Hough Transform lacks the ability to detect the end points of lines – localized information is lost during HT Peak points in the accumulator can be difficult to locate in presence of noisy or parallel edges Efficiency of the algorithm decreases if image becomes too large New approach is proposed to reduce these problems

Spatial decomposition This technique preserves the localized information Divide the image recursively into quad- trees, each quad-tree representing a part of the image i.e. a sub-image The leaf nodes will be voted for feature points which are in the sub-image represented by the leaf node

Spatial Decomposition of Hough Transform Parameter space is defined from a global origin rather than a local one Each node contains information about the sub- nodes as well as the number of feature points in the sub-image represented by the node Pruning of sub-trees is done if the number of the feature points falls below a threshold An accumulator is assigned for each leaf node

Some relations involved in spatial decomposition Consider the following –Q – any non-leaf node –F – feature points in the sub-image represented by this node –A – parameter space of the sub-image The following relations hold true

Number of accumulator arrays required Consider the following case –Size of image = N X N –Size of leaf node = M X M –Depth of tree = d (root node = 0) Number of accumulator arrays for only leaf nodes = Number of accumulator arrays for all nodes =

Example

51 References Generalizing The Hough Transform to Detect Arbitrary Shapes – D H Ballard – 1981 Spatial Decomposition of The Hough Transform – Heather and Yang – IEEE computer Society – May 2005 Hypermedia Image Processing Reference 2 – Machine Vision – Ramesh Jain, Rangachar Kasturi, Brian G Schunck, McGraw-Hill, 1995 Machine Vision - Wesley E. Snyder, Hairong Qi, Cambridge University Press, 2004

Conclusion Finding lines and other parameterized objects is an important task for computer vision. The (generalized) Hough transform can detect arbitrary shapes from (edge detected) tokens. Success rate depends directly upon the noise in the edge image. It is robust against noise and occlusion Issues: can be slow, especially for objects in arbitrary scales and orientations (extra parameters increase accumulator space exponentially). Be aware of the properties of the parameter space used, there is often more than one way to apply a Hough