Computer Vision More Image Features Hyunki Hong School of Integrative Engineering

Contents Line & Curve Detection Hough Transform Fitting Ellipses to Image Data S. Seitz 자료 등 참조

Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not vote consistently for any single model Missing data doesn’t matter as long as there are enough features remaining to agree on a good model

Hough transform an early type of voting scheme general outline: – Discretize parameter space into bins. – For each feature point in the image, put a vote in every bin in the parameter space that could have generated this point. – Find bins that have the most votes. P. V. C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959 Image space Hough parameter space

Parameter space representation A line in the image corresponds to a point in Hough space Image spaceHough parameter space

Parameter space representation What does a point (x 0, y 0 ) in the image space map to in the Hough space? Image spaceHough parameter space

Parameter space representation What does a point (x 0, y 0 ) in the image space map to in the Hough space? – Answer: the solutions of b = –x 0 m + y 0 – This is a line in Hough space. Image spaceHough parameter space 점 지나는 선분 → 점

Parameter space representation Where is the line that contains both (x 0, y 0 ) and (x 1, y 1 )? Image spaceHough parameter space (x 0, y 0 ) (x 1, y 1 ) b = –x 1 m + y 1

Parameter space representation Where is the line that contains both (x 0, y 0 ) and (x 1, y 1 )? – It is the intersection of the lines b = –x 0 m + y 0 and b = –x 1 m + y 1 Image spaceHough parameter space (x 0, y 0 ) (x 1, y 1 ) b = –x 1 m + y 1 b = –x 0 m + y 0

Problems with the (m, b) space: – Unbounded parameter domain – Vertical lines require infinite m Parameter space representation tan -1 90° = ∞

Problems with the (m,b) space: – Unbounded parameter domain – Vertical lines require infinite m. Alternative: polar representation Parameter space representation Each point will add a sinusoid in the ( ,  ) parameter space.

Algorithm outline Initialize accumulator H to all zeros. For each edge point (x, y) in the image For θ = 0 to 180 ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end end Find the value(s) of (θ, ρ) where H(θ, ρ) is a local maximum. – The detected line in the image is given by ρ = x cos θ + y sin θ ρ θ

features votes Basic illustration

Square Circle Other shapes

Several lines

A more complicated image

featuresvotes Effect of noise Peak gets fuzzy and hard to locate

Effect of noise Number of votes for a line of 20 points with increasing noise:

Random points Uniform noise can lead to spurious peaks in the array featuresvotes

Random points As the level of uniform noise increases, the maximum number of votes increases too:

Dealing with noise Choose a good grid / discretization – Too coarse: large votes obtained when too many different lines correspond to a single bucket – Too fine: miss lines because some points that are not exactly collinear cast votes for different buckets Increment neighboring bins (smoothing in accumulator array) Try to get rid of irrelevant features – Take only edge points with significant gradient magnitude

Incorporating image gradients Recall: when we detect an edge point, we also know its gradient direction But this means that the line is uniquely determined! Modified Hough transform: Algorithm For each edge point (x, y) θ = gradient orientation at (x, y) ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end

Hough transform for circles How many dimensions will the parameter space have? Given an oriented edge point, what are all possible bins that it can vote for? In a two dimensional space, a circle can by described by: -If a 2D point (x, y) is fixed, then the parameters can be found. -The parameter space would be three dimensional, (a, b, r). -In the 3D space, the circle parameters can be identified by the intersection of many conic surfaces that are defined by points on the 2D circle. 1.The first stage: fixing radius then find the optimal center of circles in a 2D parameter space. 2.The second stage: to find the optimal radius in a one dimensional parameter space. (a, b) : the center of the circle r : the radius Consider 4 points on a circle in the original image and the circle Hough transform. Note that the radius is assumed to be known. For each ( x, y ) of the four points (white points) in the original image, it can define a circle in the Hough parameter space centered at ( x, y ) with radius r Find circle parameter with unknown radius: Since the parameter space is 3D, the accumulator matrix would be 3D, too. We can iterate through possible radiuses and for each radius. Finally, find the local maxima in the 3D accumulator matrix.

Line & Curve Detection Line detection can be performed directly: template matching : look for the peaks of the convolution between the image and a set of masks matched to long, linear edges in all possible orientations Line and curve detection splits logically into two sub-problems. - Grouping: Which image points compose each instance of the target curve in the images? - Model fitting: Given a set of image points probably belonging to a single instance of the target curve, find the best curve interpolating the points.

detect complex patterns of points in binary images mapping a difficult pattern detection problem into a simple peak detection problem in the space of the parameters of the curve two basic steps of HT for lines 1. Transform line detection into a line intersection problem Any line y = mx + n is identified by a unique parameter pair (m, n). → The line is represented by a point in the m, n plane (the parameter space). Conversely, any point p = [x, y] T in the image corresponds to a line n = x(- m) + y in parameter space, which m & n vary, represents all possible image lines through p. → A line defined by N collinear image points, p 1,.., p N, is identified in parameter space by the intersection of the lines associated with p 1,.., p N Hough Transform (HT) ex. Lines and simple curves

The original Hough transform : to detect lines & curves (a) Image space (b) k, q parameter space Any straight line in the image → a single point in the k, q parameter space Accumulator array & threshold

2. Transform line intersection in a simple peak detection problem, or search for a maximum. a. Divide the m, n plane into a finite grid of cells, the resolution of which depends on the accuracy we need. b. For each image pixel p i, increment all counters on the corresponding line in parameter space. c. The image line is identified by the peak of c(m, n) in parameter space. The polar representation solves two problems. 1. Both m and n can take on values in [-∞, ∞], which implies that we cannot sample the whole paramter space. → a limit to the size of the discrete paramter space 2. (m, n) parameterization does not capture the bundle x = k. Hough Transform (HT)

Effect of nonlinear contours: curved contours, or just noise introduced by the edge detector  threshold c(m, n) Because of pixelization and the limited accuracy of edge detection, not all the parameter-space lines corresponding to the points of an image line intersect at the same point. Image(64×64) containing two lines, sampled irregularly, and several random points (left) Plot of the contours in the corresponding parameter space (how many points contribute to each cell( m, n ) (right)

HT Algorithm

The HT generalized to detect curves 1. curves y = f(x, a), where a = [a 1, …, a p ] T is a vector of p parameters.

HT 1. a voting algorithm a. Each point “votes” for all combinations of parameters. b. The array of counters in parameter space can be regarded as a histogram. c. The final total of votes indicates the relative likelihood of the hypothsis. “A curve with paramter set m existis in the image.” 2. pattern (= the class of curves) matching 3. merits: 1) it copes well with occlusion. 2) it’s relatively robust to noise. Major limitations 1. the rapid increase of the search time with the number of parameters in the curve’s representation 2. Non-target shapes can produce spurious peaks in parameter space.

The ellipse detectors take an image of edge points in input, and find the best ellipse fitting the points. Ellipse fitting identifies a set of image points plausibly belonging to a single arc of ellipse. Algorithm 1. Let p 1, …, p N be a set of N image points, p i = [x i, y i ] T. Let x = [x 2, xy, y 2, x, y, 1] T, p = [x, y] T, and f(p, a) = x T a = ax 2 + bxy + cy 2 + dx + ey + f = 0, the implicit equation of the generic ellipse, characterized by the parameter vector a = [a, b, c, d, e, f] T 2. Find the parameter vector, a, associated to the ellipse which fits p 1, …, p N best in the least squares sense, as the solution of, where D(p i, a) is a suitable distance.  What is a suitable distance? ( Euclidean distance & algebraic distance) Fitting Ellipses to Image Data f(p, a) : generic conic

A conic section (or just conic) is a curve obtained as the intersection of a cone with a plane. In the Cartesian coordinate system, the graph of a quadratic equation in two variables is always a conic section: The quadratic can be written as Conic Section Parabola, Circle, ellipse, and Hyperbola

Minimize the Euclidean distance between the ellipse & the measured points., under the constraint that p belongs to the ellipse: Let us try Lagrange multipliers to solve (1). An objective function : and set which yield (2) Since we do not know p, we try to express it as a function of computable quantities.  Use two approximations 1. consider a first-order approximation of the curve (3) 2. assume that the p i are close enough to the curve, so that Euclidean Distance Fit (1) Go to 37 page

Linear Approximation an approximation of a general function using a linear function (more precisely, an affine function) - They are widely used in the method of finite differences to produce first order methods for solving or approximating solutions to equations. - Given a twice continuously differentiable function f of one real variable, Taylor's theorem for the case n = 1 states that, where R 2 is the remainder term. - The linear approximation is obtained by dropping the remainder: - This is a good approximation for x when it is close enough to a; since a curve, when closely observed, will begin to resemble a straight line. Tangent line at (a, f(a)) 예 ) y = x 2

rewrite Eq. (2) as which, plugged into Eq. (3), gives Substituting in Eq. (2), we finally find cf. Another method: Algebraic distance turns into a linear problem with no further approximations

In mathematical optimization, Lagrange multiplier method is a strategy for finding the local maxima and minima of a function subject to equality constraints. For instance, consider the optimization problem maximize f(x, y) subject to g(x, y) = c. Lagrange multiplier Find x and y to maximize f(x, y) subject to a constraint (shown in red) g(x, y) = c. Contour map. The red line shows the constraint g(x, y) = c. The blue lines are contours of f(x, y). The point where the red line tangentially touches a blue contour is our solution. Since d 1 > d 2, the solution is a maximization of f(x, y). when one wishes to maximize or minimize a function subject to fixed outside conditions or constraints.

We need both f and g to have continuous first partial derivatives. Introduce a new variable (λ) called a Lagrange multiplier and study the Lagrange function (or Lagrangian) defined by where λ term may be either added or subtracted. If f(x 0, y 0 ) is a maximum of f(x, y) for the original constrained problem, then there exists λ 0 such that (x 0, y 0, λ 0 ) is a stationary point for the Lagrange function (stationary points: those points where the partial derivatives of Λ are zero, i.e. ∇ Λ = 0). Lagrange multiplier A stationary point or critical point is a point of the domain of a differentiable function, where the derivative is zero.

consider the 2D problem: maximize f(x, y) subject to g(x, y) = c. Lagrange multipliers relies on the intuition that at a maximum f(x, y) cannot be increasing in the direction of any neighboring point where g = c. Suppose we walk along the contour line with g = c. We are interested in finding points where f does not change as we walk, since these points might be a maximum. → we could be following a contour line of f, since by definition f does not change as we walk along its contour lines. This would mean that the contour lines of f and g are parallel here : for some λ where : the respective gradients auxiliary function and solve Lagrange multiplier

Example 1: Suppose we wish to maximize f(x, y) = x + y subject to the constraint x 2 + y 2 =1. The feasible set is the unit circle, and the level sets of f are diagonal lines (with slope -1), so we can see graphically that the maximum occurs at, and that the minimum occurs at. 1. Using the method of Lagrange multipliers, g(x, y) – c = x 2 + y 2 –1. 2. Setting the gradient yields the system of equations: 3. Evaluating the objective function f at these points yields the maximum & the minimum: 대입하면,  stationary points: and

Generalized Hough transform We want to find a shape defined by its boundary points and a reference point. D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981, pp Generalizing the Hough Transform to Detect Arbitrary Shapes a

p Generalized Hough transform We want to find a shape defined by its boundary points and a reference point. For every boundary point p, we can compute the displacement vector r = a – p as a function of gradient orientation θ D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981, pp Generalizing the Hough Transform to Detect Arbitrary Shapes a θ r(θ)

Generalized Hough transform For model shape: construct a table indexed by θ storing displacement vectors r as function of gradient direction Detection: For each edge point p with gradient orientation θ: – Retrieve all r indexed with θ – For each r(θ), put a vote in the Hough space at p + r(θ) Peak in this Hough space is reference point with most supporting edges Assumption: translation is the only transformation here, i.e., orientation and scale are fixed