Image Segmentation And Representation Cosimo Distante.

Slides:



Advertisements
Similar presentations
Feature extraction: Corners
Advertisements

CSE 473/573 Computer Vision and Image Processing (CVIP)
Interest points CSE P 576 Ali Farhadi Many slides from Steve Seitz, Larry Zitnick.
TP14 - Local features: detection and description Computer Vision, FCUP, 2014 Miguel Coimbra Slides by Prof. Kristen Grauman.
Matching with Invariant Features
Features CSE 455, Winter 2010 February 1, Neel Joshi, CSE 455, Winter Review From Last Time.
Computational Photography
Algorithms and Applications in Computer Vision
Interest points CSE P 576 Larry Zitnick Many slides courtesy of Steve Seitz.
EE 7730 Image Segmentation.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Harris corner detector
Lecture 6: Feature matching CS4670: Computer Vision Noah Snavely.
Lecture 4: Feature matching
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Object Recognition A wise robot sees as much as he ought, not as much as he can Search for objects that are important lamps outlets wall corners doors.
Feature extraction: Corners and blobs
Interest Points and Corners Computer Vision CS 143, Brown James Hays Slides from Rick Szeliski, Svetlana Lazebnik, Derek Hoiem and Grauman&Leibe 2008 AAAI.
Fitting a Model to Data Reading: 15.1,
Lecture 3a: Feature detection and matching CS6670: Computer Vision Noah Snavely.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
1 Image Features Hao Jiang Sept Image Matching 2.
CS4670: Computer Vision Kavita Bala Lecture 7: Harris Corner Detection.
CS4670: Computer Vision Kavita Bala Lecture 8: Scale invariance.
October 8, 2013Computer Vision Lecture 11: The Hough Transform 1 Fitting Curve Models to Edges Most contours can be well described by combining several.
Lecture 06 06/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Segmentation Course web page: vision.cis.udel.edu/~cv May 7, 2003  Lecture 31.
Feature Detection and Matching
CSE 185 Introduction to Computer Vision Local Invariant Features.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
Local invariant features 1 Thursday October 3 rd 2013 Neelima Chavali Virginia Tech.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Notes on the Harris Detector
Feature extraction: Corners and blobs. Why extract features? Motivation: panorama stitching We have two images – how do we combine them?
October 16, 2014Computer Vision Lecture 12: Image Segmentation II 1 Hough Transform The Hough transform is a very general technique for feature detection.
Local features and image matching October 1 st 2015 Devi Parikh Virginia Tech Disclaimer: Many slides have been borrowed from Kristen Grauman, who may.
Features Jan-Michael Frahm.
CS654: Digital Image Analysis
Image matching by Diva SianDiva Sian by swashfordswashford TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA.
Announcements Project 1 Out today Help session at the end of class.
Projects Project 1a due this Friday Project 1b will go out on Friday to be done in pairs start looking for a partner now.
Instructor: Mircea Nicolescu Lecture 10 CS 485 / 685 Computer Vision.
CSE 185 Introduction to Computer Vision Local Invariant Features.
Lecture 10: Harris Corner Detector CS4670/5670: Computer Vision Kavita Bala.
Keypoint extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Object Recognition. Segmentation –Roughly speaking, segmentation is to partition the images into meaningful parts that are relatively homogenous in certain.
Invariant Local Features Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging.
Lecture 5: Feature detection and matching CS4670 / 5670: Computer Vision Noah Snavely.
Interest Points EE/CSE 576 Linda Shapiro.
Image Features: Points and Regions
3D Vision Interest Points.
TP12 - Local features: detection and description
Source: D. Lowe, L. Fei-Fei Recap: edge detection Source: D. Lowe, L. Fei-Fei.
Lecture 4: Harris corner detection
Local features: detection and description May 11th, 2017
Feature description and matching
Fitting Curve Models to Edges
Interest Points and Harris Corner Detector
Announcements Project 1 Project 2 (panoramas)
Lecture 5: Feature detection and matching
Announcement Project 1 1.
Local features and image matching
Edge detection f(x,y) viewed as a smooth function
Lecture VI: Corner and Blob Detection
Corner Detection COMP 4900C Winter 2008.
Introduction to Artificial Intelligence Lecture 22: Computer Vision II
Presentation transcript:

Image Segmentation And Representation Cosimo Distante

KMeans Let be the Let us suppose to cluster data inot K classes Let m j, j=1,…,k be the prototype for each class Let m j, j=1,…,k be the prototype for each class Let us suppose having already the K prototypes and we are interested in representing the input data x t with the closest prototype as follow

KMeans How to find the the K prototypes m j ? When x t is represented by m i, there is an error proportional to its distance Out objective is to reduce this distance as much as possible Let us introduce an error function (reconstruction)

KMeans Best performances reached by finding the minimum of E Let us use an iterative procedure to find the K prototypes Let’s start by randomly initialize the K prototypes m i Then we compute the b values (membership) and then minimize the errro E by placing the derivative of E to zero Modifying the m values imply that even b values will vary and then the iterative process is repeated until no changes in m will occur

KMeans

7 K-Means Clustering

8

9

10

11

12

13

14

RGB vector

Clustering Example D. Comaniciu and P. Meer, Robust Analysis of Feature Spaces: Color Image Segmentation, 1997.

K-Means Clustering Example Original K=5 K=11

Bahadir K. Gunturk EE Image Analysis I 18 K-means, only color is used in segmentation, four clusters (out of 20) are shown here.

Bahadir K. Gunturk EE Image Analysis I 19 K-means, color and position is used in segmentation, four clusters (out of 20) are shown here. Each vector is (R,G,B,x,y).

Bahadir K. Gunturk EE Image Analysis I 20 K-Means Clustering: Axis Scaling Features of different types may have different scales.   For example, pixel coordinates on a 100x100 image vs. RGB color values in the range [0,1]. Problem: Features with larger scales dominate clustering. Solution: Scale the features.

KMeans Kmeans results with RGB image in. (a) the original image, (b) K=3; (c) K=5;(d)K=6; (e) K=7 and (f) K=8

Fuzzy C-means 1.Initialize the membership matrix b with values between 0 and 1. such that the following is satisfied 2.Compute centers m i, i=1,…,k 3.Compute cost function Stopping criterion : IF E<threshold or no significative variations between current and previous iteration 4.Compute the new membership matrix And goto STEP 2 And goto STEP 2

10/9/2015 HOUGH Transform

10/9/2015 Contours: Lines and Curves Edge detectors find “ edgels ” (pixel level) To perform image analysis : –edges must be grouped into entities such as contours (higher level). –Canny does this to certain extent: the detector finds chains of edgels.

10/9/2015 Line detection Mathematical model of a line: xy Y = mx + n P(x 1,y 1 ) P(x 2,y 2 ) Y 1 =m x 1 +n Y 2 =m x 2 +n Y N =m x N +n

10/9/2015 Image and Parameter Spaces xy Y = mx + n Y1=m x1+nY1=m x1+nY1=m x1+nY1=m x1+n Y2=m x2+nY2=m x2+nY2=m x2+nY2=m x2+n YN=m xN+nYN=m xN+nYN=m xN+nYN=m xN+n Y = m ’ x + n ’ Image Space Parameter Space interceptslope mn m’m’m’m’ n’n’n’n’ Line in Img. Space ~ Point in Param. Space

10/9/2015 Looking at it backwards … Y 1 =m x 1 +n Image space Fix (m,n), Vary (x,y) - Line Fix (x 1,y 1 ), Vary (m,n) – Lines thru a Point Y = mx + n xy P(x 1,y 1 )

10/9/2015 Looking at it backwards … Can be re-written as: n = -x 1 m + Y 1 Y1=m x1+nY1=m x1+nY1=m x1+nY1=m x1+n Parameter space Fix (-x 1,y 1 ), Vary (m,n) - Line n = -x 1 m + Y 1 interceptslopem n m’m’m’m’ n’n’n’n’

10/9/2015 Img-Param Spaces Image Space –Lines –Points –Collinear points Parameter Space –Points –Lines –Intersecting lines

10/9/2015 Hough Transform Technique H.T. is a method for detecting straight lines (and curves) in images. Main idea: –Map a difficult pattern problem into a simple peak detection problem

10/9/2015 Hough Transform Technique Given an edge point, there is an infinite number of lines passing through it (Vary m and n). –These lines can be represented as a line in parameter space. Parameter Space interceptslopem n xy n = (-x) m + y P(x,y)

10/9/2015 Hough Transform Technique Given a set of collinear edge points, each of them have associated a line in parameter space. –These lines intersect at the point (m,n) corresponding to the parameters of the line in the image space.

10/9/2015 Parameter Space interceptslopem n x y n = (-x1) m + y1 P(x1,y1) Q(x2,y2) n = (-x2) m + y2 p q

10/9/2015 Hough Transform Technique At each point of the (discrete) parameter space, count how many lines pass through it. –Use an array of counters –Can be thought as a “ parameter image ” The higher the count, the more edges are collinear in the image space. –Find a peak in the counter array –This is a “ bright ” point in the parameter image –It can be found by thresholding

10/9/2015 Hough Transform Technique

10/9/2015 Practical Issues The slope of the line is -  <m<  –The parameter space is INFINITE The representation y = mx + n does not express lines of the form x = k

10/9/2015 Solution: Use the “ Normal ” equation of a line: xy Y = mx + n  = x cos  +y sin   P(x,y)x y  Is the line orientation  Is the distance between the origin and the line

10/9/2015Octavia I. Camps38

10/9/2015 New Parameter Space , Use the parameter space ( ,  ) The new space is FINITE –0 <  < D, where D is the image diagonal. –0 <  <  The new space can represent all lines –Y = k is represented with  = k,  =90 –X = k is represented with  = k,  =0

10/9/2015 Consequence: A Point in Image Space is now represented as a SINUSOID –  = x cos  +y sin 

10/9/2015 Hough Transform Algorithm Input is an edge image (E(i,j)=1 for edgels) 1.Discretize  and  in increments of d  and d . Let A(R,T) be an array of integer accumulators, initialized to 0. 2.For each pixel E(i,j)=1 and h=1,2,…T do 1.  = i cos(h * d  ) + j sin(h * d  ) 2.Find closest integer k corresponding to  3.Increment counter A(h,k) by one 3.Find local maxima in A(R,T)

10/9/2015 Hough Transform Speed Up If we know the orientation of the edge – usually available from the edge detection step –We fix theta in the parameter space and increment only one counter! –We can allow for orientation uncertainty by incrementing a few counters around the “ nominal ” counter.

10/9/2015 Hough Transform for Curves The H.T. can be generalized to detect any curve that can be expressed in parametric form: –Y = f(x, a1,a2,…ap) –a1, a2, … ap are the parameters –The parameter space is p-dimensional –The accumulating array is LARGE!

Hough Transform for Circles The above equation can be espressed in parametric form

Hough Transform for Circles

10/9/2015 H.T. Summary H.T. is a “ voting ” scheme –points vote for a set of parameters describing a line or curve. The more votes for a particular set – the more evidence that the corresponding curve is present in the image. Can detect MULTIPLE curves in one shot. Computational cost increases with the number of parameters describing the curve.

Image matching by Diva SianDiva Sian by swashfordswashford TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA

Harder case by Diva SianDiva Sianby scgbtscgbt

Harder still? NASA Mars Rover images

with SIFT feature matches Figure by Noah Snavely Answer below (look for tiny colored squares…)

Local features and alignment [Darya Frolova and Denis Simakov] We need to match (align) images Global methods sensitive to occlusion, lighting, parallax effects. So look for local features that match well. How would you do it by eye?

Local features and alignment Detect feature points in both images [Darya Frolova and Denis Simakov]

Local features and alignment Detect feature points in both images Find corresponding pairs [Darya Frolova and Denis Simakov]

Local features and alignment Detect feature points in both images Find corresponding pairs Use these pairs to align images [Darya Frolova and Denis Simakov]

Local features and alignment Problem 1: – Detect the same point independently in both images no chance to match! We need a repeatable detector [Darya Frolova and Denis Simakov]

Local features and alignment Problem 2: – For each point correctly recognize the corresponding one ? We need a reliable and distinctive descriptor [Darya Frolova and Denis Simakov]

Geometric transformations

Photometric transformations Figure from T. Tuytelaars ECCV 2006 tutorial

And other nuisances… Noise Blur Compression artifacts …

Invariant local features Subset of local feature types designed to be invariant to common geometric and photometric transformations. Basic steps: 1) Detect distinctive interest points 2) Extract invariant descriptors Figure: David Lowe

Main questions Where will the interest points come from? –What are salient features that we’ll detect in multiple views? How to describe a local region? How to establish correspondences, i.e., compute matches?

Finding Corners Key property: in the region around a corner, image gradient has two or more dominant directions Corners are repeatable and distinctive C.Harris and M.Stephens. "A Combined Corner and Edge Detector.“ Proceedings of the 4th Alvey Vision Conference: pages "A Combined Corner and Edge Detector.“ Source: Lana Lazebnik

Corners as distinctive interest points We should easily recognize the point by looking through a small window Shifting a window in any direction should give a large change in intensity “edge”: no change along the edge direction “corner”: significant change in all directions “flat” region: no change in all directions Source: A. Efros

Harris Detector formulation Change of intensity for the shift [u,v]: Intensity Shifted intensity Window function or Window function w(x,y) = Gaussian1 in window, 0 outside Source: R. Szeliski

Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by summing up the squared differences (SSD) this defines an SSD “error” of E(u,v): Feature detection: the math W

Taylor Series expansion of I: If the motion (u,v) is small, then first order approx is good Plugging this into the formula on the previous slide… Small motion assumption

Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by summing up the squared differences this defines an “error” of E(u,v): Feature detection: the math W

This can be rewritten: For the example above You can move the center of the green window to anywhere on the blue unit circle Which directions will result in the largest and smallest E values? We can find these directions by looking at the eigenvectors of M M

Harris Detector formulation This measure of change can be approximated by: where M is a 2  2 matrix computed from image derivatives: M Sum over image region – area we are checking for corner Gradient with respect to x, times gradient with respect to y

Harris Detector formulation where M is a 2  2 matrix computed from image derivatives: Sum over image region – area we are checking for corner Gradient with respect to x, times gradient with respect to y M

Quick eigenvalue/eigenvector review The eigenvectors of a matrix A are the vectors x that satisfy: The scalar is the eigenvalue corresponding to x The eigenvalues are found by solving: In our case, A = M is a 2x2 matrix, so we have The solution: Once you know, you find x by solving

Feature detection: the math Eigenvalues and eigenvectors of H Define shifts with the smallest and largest change (E value) x + = direction of largest increase in E. + = amount of increase in direction x + x - = direction of smallest increase in E. - = amount of increase in direction x - x - x + M

First, consider an axis-aligned corner: What does this matrix reveal?

First, consider an axis-aligned corner: This means dominant gradient directions align with x or y axis If either λ is close to 0, then this is not a corner, so look for locations where both are large. Slide credit: David Jacobs What does this matrix reveal? What if we have a corner that is not aligned with the image axes?

General Case Since M is symmetric, we have We can visualize M as an ellipse with axis lengths determined by the eigenvalues and orientation determined by R direction of the slowest change direction of the fastest change ( max ) -1/2 ( min ) -1/2 Slide adapted form Darya Frolova, Denis Simakov.

Interpreting the eigenvalues 1 2 “Corner” 1 and 2 are large, 1 ~ 2 ; E increases in all directions 1 and 2 are small; E is almost constant in all directions “Edge” 1 >> 2 “Edge” 2 >> 1 “Flat” region Classification of image points using eigenvalues of M:

Corner response function “Corner” R > 0 “Edge” R < 0 “Flat” region |R| small α : constant (0.04 to 0.06)

Harris Corner Detector Algorithm steps: –Compute M matrix within all image windows to get their R scores –Find points with large corner response (R > threshold) –Take the points of local maxima of R

Harris Detector: Workflow Slide adapted form Darya Frolova, Denis Simakov, Weizmann Institute.

Harris Detector: Workflow Compute corner response R

Harris Detector: Workflow Find points with large corner response: R>threshold

Harris Detector: Workflow Take only the points of local maxima of R

Harris Detector: Workflow

Harris Detector: Properties Rotation invariance Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

Harris Detector: Properties Not invariant to image scale All points will be classified as edges Corner !