1. COMP322/S2000/L171 Robot Vision System Major Phases in Robot Vision Systems: A. Data (image) acquisition –Illumination, i.e. lighting consideration –Lenses, and Cameras –Digitizers B. Pre-processing –Enhancement, i.e. smoothing, edge detection –Segmentation (Binarization) C. Recognition –Feature extraction –Pattern matching D. Part Manipulation –Choice of robot, gripper –Orientations of gripper, camera, part w.r.t robot base

2. COMP322/S2000/L172 Vision System: Pre-Processing Two Approaches: Frequency Domain - refers to the collection of pixels (complex) resulting from taking the Fourier transform of an image. (not covered in this course) Spatial Domain - refers to the collection of pixels that forms the image. The methods are procedures that operate directly on these pixels. (this is the approach used in this course) Mathematically, g(x,y) = h [f(x,y)] where f(x,y) is the input image; g(x,y) is the resulting image; and h is an operator on defined over some neighbourhood at pixel (x,y).

3. COMP322/S2000/L173 Vision System: Pre-Processing Spatial Domain Approach - g(x,y) = h [f(x,y)] Usually, h is a square array of numbers called convolution mask, or templates, or windows or filters. Example: h is a 3x3 mask: g(x,y) = w 1 f(x-1,y-1) + w 2 f(x,y-1) + w 3 f(x+1,y-1) + w 4 f(x-1,y) + w 5 f(x,y) + w 7 f(x+1,y) + w 7 f(x-1,y+1) + w 8 f(x,y+1) + w 9 f(x+1,y+1) Example given in class

4. COMP322/S2000/L174 Vision System: Pre-Processing Smoothing: Reduce noise and other effects that are present in the image which are results of sampling, quantization, transmission, etc. Methods: Neighbourhood averaging Given an image f, the resultant smoothed image g is obtained by averaging the intensity values of a neighbourhood centered at pixel (x,y), ie. Where S is the neighbourhood set defined at pixel (x,y); p is the no. of pixels in the neighbourhood. Example given in class

5. COMP322/S2000/L175 Vision System: Pre-Processing - smoothing Median Filtering Recall: the median (m)of a set of values is such that half of the values in the set are less than (and equal) m and half the values are greater than m. e.g. Set = (10, 20, 20, 20, 100, 20, 20, 25, 15) sorted => (10, 15, 20, 20, 20, 20, 20, 25, 100) Idea is to force pixels with very distinct intensities to be more like their neighbours.

6. COMP322/S2000/L176 Vision System: Pre-Processing - smoothing Example: Original: By averaging: Median Filter:

7. COMP322/S2000/L177 Vision System: Pre-Processing - Edge Detection Edge indicates a change in intensities within a window Edge detection techniques involve derivative operators The gradient at pixel (x,y) in an image is defined as a 2-D vector, The magnitude of the gradient = The direction of the gradient = For edge detection, we are interested in the magnitude only.

8. COMP322/S2000/L178 Vision System: Pre-Processing - Edge Detection Common First order Gradient Operators First order difference between adjacent pixels Masks:

9. COMP322/S2000/L179 Vision System: Pre-Processing - Edge Detection Common First order Gradient Operators Sobel Operators Image window centered at (x,y) : Mask_x: Mask_y: Gradient magnitude:

10. COMP322/S2000/L1710 Vision System: Pre-Processing - Edge Detection Laplacian Operator A second order derivative operator, For digital images, Image window centered at (x,y): The Laplacian Mask: The gradient magnitude: Note: Laplacian operator is extremely sensitive to noise

11. COMP322/S2000/L1711 Vision System: Pre-Processing - Thresholding Problem: Given a single value of intensity (L) such that the resultant binary image B, is given by where f(x,y) is the input grey level image. Question is to determine L?

12. COMP322/S2000/L1712 Vision System: Pre-Processing - Thresholding Algorithm: Step 1: Obtain the intensity histogram of the image Step 2: Find the valley of the histogram. The grey level value corresponding to the valley is the value of L Step 3: Assign the value of zero or one to each pixel