Digital image processing Chapter 8 Image analysis and pattern recognition IMAGE ANALYSIS AND PATTERN RECOGNITION Introduction Feature extraction: - spatial feature extraction - transform feature extraction - edge feature extraction; edge detection Objects representation by their boundaries: - contour extraction - contour descriptors Objects representation by their regions: - region extraction - region representation Shapes and structures for region-based object representation: - object skeletons - binary morphology - shape descriptors (numerical shape descriptors) Textures; texture analysis Image segmentation Grey level based segmentation/color based segmentation Connected components analysis Contour-based segmentation Region-based segmentation Mixed techniques

Introduction Fig. 8.1 Image analysis system – block diagram Example image analysis tasks: Digital image processing Chapter 8 Image analysis and pattern recognition License plate recognition

Introduction Example image analysis tasks: Digital image processing Chapter 8 Image analysis and pattern recognition Text detection & recognition or removal Trafic sign recognition Calcite localization/assessmentDetect tissue on the slide

Introduction Example image analysis tasks: Ways to approach the image analysis task? - “straight forward” approach: analyze the content of the whole image => one will eventually find the entire information, including the information of interest – at too high computational complexity/cost - “smart” approach: focus strictly on the information you’re looking for! and extract only this information from the image  focus on the part of the image containing the information of interest. How? 1)describe somehow the main characteristics of the information of interest  describe the discriminative features of the region containing the information you look for, which are not present in the other regions => represent the image in the best feature space, by a feature map Digital image processing Chapter 8 Image analysis and pattern recognition Single object images; defect detection on the object

E.g. on the images with apples – only interested in the defect area (if there is), and for this defect – it appears darker grey than the normal apple coloration, but not black => design a grey scale slicing to set to black everything else 2)Analyze only the information in the region of interest, using the previously defined feature space/feature map e.g. Apples sort: no defects (defect area <2% of the apple); average defects; large defects… Digital image processing Chapter 8 Image analysis and pattern recognition New feature space

Feature extraction: Spatial features extraction: Amplitude features: e.g. the brightness levels can identify regions of interest in the image: Amplitude features may be discriminative enough if intensity is enough to distinguish wanted info from the rest of the scene => defining the best parameters of the transformation for feature extraction – most difficult => amplitude feature space representation is not necessarily binary; just that unwanted parts of the scenes should be represented uniquely (i.e. black) in the feature space => sometimes adaptive thresholding/adaptive grey scale slicing is needed. Tissue identification by color coding (e.g. violet)  Can measure afterwards the area, describe the shape, etc. Digital image processing Chapter 8 Image analysis and pattern recognition

Histogram based features: Local histogram = a local statistical description of the image; If u = an image pixel; x=a grey level => p u (x)=the probability of appearance of the grey level x in the image region = a value in the normalized histogram => One can compute: the standard deviation; the entropy; the median; percentiles, of p u (x). Region of interest (ROI) ROI histogram Measurements Digital image processing Chapter 8 Image analysis and pattern recognition Tissue of interest is well discriminated from the microscopic slide by the standard deviation of the local histogram

Transform features extraction Fig.8.2 Transform features extraction Digital image processing Chapter 8 Image analysis and pattern recognition

Edge features extraction. Edge detection Fig. 8.3 Edge detection with gradient operators (8.5) (8.6) Digital image processing Chapter 8 Image analysis and pattern recognition

Fig. 8.4 Edge detection by compass operators a b c d Fig. 8.5 Compass edge detectors (North direction) N NW W SW S SE E NE Fig. 8.6 Compass operators on different directions Digital image processing Chapter 8 Image analysis and pattern recognition

g k (m,n) –compass gradient on the direction k  0,...,7, The gradient in the spatial position (m,n) is defined as: (8.7) Laplace operators and the zero-crossings edge localization method: (8.8) Fig. 8.7 Edge detection by Laplace operators (the 1-D case) The Laplacian: Digital image processing Chapter 8 Image analysis and pattern recognition

The Laplacian of Gaussian operator (LoG) Discrete implementations: Gaussian filter The derivative of the Gaussian The 2 nd derivative (Laplacian of Gaussian) Digital image processing Chapter 8 Image analysis and pattern recognition

Edge detection by different operators – comparison: Original image Sobel edge detection LoG edge detection; sigma=5 Digital image processing Chapter 8 Image analysis and pattern recognition Roberts edge detection LoG edge detection; Sigma=10

Objects representation by their boundaries: Contour extraction: Fig connectivity; 8-connectivity The Hough transform: Digital image processing Chapter 8 Image analysis and pattern recognition s θ Φ s x y

The Hough transform of lines/random curves: Digital image processing Chapter 8 Image analysis and pattern recognition No convergence point => The points are not on a straight line ss Approx. convergence => approx. straight line Curve convergence on horizontal axis=> Straight line through the origin s

Applications to contour extraction formed by line segments: Digital image processing Chapter 8 Image analysis and pattern recognition Lower left object = Defined by the 3 line intersection points Input image Edge detection (gradient operator) + thresholding => Binary edge map Edge map; Contours not yet extracted/ not yet labeled Apply Hough transform: Extracting the contour of the triangle Contour extraction in 8-connectivity 8-connectivity: Correct labeling 4-connectivity: The contour is “broken” during labeling

Contour representation/contour descriptors: Goal: for a given object, described by its contour, find a compact description, by numerical attributes, able to: -Represent the contour with no significant loss of information (regenerative descriptors) -Generate (by a subset of attributes) descriptions of the contour/shape invariant to: scaling; rotation; translation; mirroring; projection distortions; small (limited) variations of shape (among different individual representations of the same shape)  Using the contour descriptors, one can recognize the shape by template matching or shape classification (contour descriptors classification) General assumption: single object contour (all edge pixels connected); 1-pixel width! Digital image processing Chapter 8 Image analysis and pattern recognition

Chain codes and polygonal approximations: Fig. 8.9 Contour representation by chain codes Fig Polygonal approximation of the contours Digital image processing Chapter 8 Image analysis and pattern recognition Original contour Contour approximation (“quantization”) on a rectangular grid Encode this contour by a chain code with 8 directions, Consider origin= the upper left point; contour following direction = arbitrary; the only possible positions are on the grid corners Start point

Fourier descriptors: (8.12) (8.13) (8.14) (8.15) (8.16) leading to a new contour x'(n) y'(n), given by: (8.17) where (8.18) Table 8.1 Digital image processing Chapter 8 Image analysis and pattern recognition

Shape reconstruction from its Fourier descriptors: OriginalReconstruction using the first 2 descriptors only Reconstruction using the first 6 descriptors Reconstruction using the first 10 descriptors Reconstruction using the first 20 descriptors … Digital image processing Chapter 8 Image analysis and pattern recognition

, then for a given translation u 0, the distance d is minimal when: and where a(k)  b*(k)  c(k)e j  k,  -2  n 0 /N and c(k) is a real valued term. Digital image processing Chapter 8 Image analysis and pattern recognition Shapes dictionary Model shape Macthing results: d<Thd

Objects representation by their regions: Region extraction: same as for contours (use connected components analysis, in 4- or 8- connectivity) Region representation: Fig Quad-tree region representation => the quad-tree code : gbgbwwbwgbwwgbwwb ; decoding: g(bg(bwwb)wg(bwwg(bwwb))) Digital image processing Chapter 8 Image analysis and pattern recognition

 Regions skeletons; medial axis transforms: 8.23)  (m,n) : u k (m,n)  u k (i,j),  (m,n;i,j)  1  (8.24) Fig Skeleton extraction Fig Examples skeletons Fig pass contour thinning: 2 logic rules: Fig Thinning result R1: P1==1 && N(P1)==1 && 2<=NT(P1)<=6 && P2·P4·P6==0 && P2·P4·P8==0 R2: P1==1 && N(P1)==1 && 2<=NT(P1)<=6 && P2·P6·P8==0 && P4·P6·P8==0 Shapes and structures for region-based object representation: Digital image processing Chapter 8 Image analysis and pattern recognition

 Morphological processing. Binary image morphology Fig Algoritmi de transformare morfologica Digital image processing Chapter 8 Image analysis and pattern recognition

 Syntactical representation Fig Syntactical representation of an object Digital image processing Chapter 8 Image analysis and pattern recognition

Shape descriptors Applications: shape recognition; quantitative measurements Def.: The shape of an object = the profile of the object + its physical structure => “shape descriptors” Classification: (1) regenerative descriptors (contours; regions; high order statistics; structural and syntactic descriptors) (2) geometrical shape descriptors (area, perimeter, max-min radii, eccentricity, corners, roundness, symmetry) (3) moments Digital image processing Chapter 8 Image analysis and pattern recognition

Geometrical features (geometrical descriptors):  Perimeter: t – some contour parameter - Discrete => T = count of contour pixels  Area: where: R and  R – the object region and the object contour - Discrete => A = count of pixels inside the object region  Min-max radii, R min and R max – the minimum and maximum distances from the center of mass of the object region to its contour (the R max / R min ratio – gives a measure of the eccentricity or elongation of the object) x y dx dy x(t)x(t+1) y(t) y(t+1) R max R min Digital image processing Chapter 8 Image analysis and pattern recognition

 Roundness or compactness: For a disc -  is minimum, =1. · Symmetry: 2 types of shape symmetry: rotational and mirror Moment-based features: · Center of mass: · The (p,q) order central moments: · Orientation = the angle of the axis of the smallest moment of inertia – found by minimizing: with respect to  : Digital image processing Chapter 8 Image analysis and pattern recognition compactnecompact CompactNon-compact

Textures  The texture = the periodic repetition of some basic structures in an image area; the basic image structure is called texel Texture analysis methods: statistical classification; structural classification  Statistical classification techniques:  The auto-correlation function (ACF): the spatial dimensions of the texels are proportional to the width of the auto- correlation functions:: where  Several measures used to evaluate the distribution of the ACF to describe the texture: Artificial textures Natural textures Digital image processing Chapter 8 Image analysis and pattern recognition

 Image transforms based approaches: (8.41) Fig Various masks in the frequency domain used for texture analysis ACF for “Sand”ACF for “Wool” Digital image processing Chapter 8 Image analysis and pattern recognition

 The edge density – as texture classification feature:  Histogram features for texture analysis: => the co-occurrence histogram: => various features can be extracted from the co-occurrence histogram: - Inertia: - The mean of the distribution: N 0 – the total number of possible orientations. - The variance of the distribution: - The spread of the distribution: Sobel SandRaffia Grass R  Digital image processing Chapter 8 Image analysis and pattern recognition

 Random texture models (8.47) Fig Texture analysis model Digital image processing Chapter 8 Image analysis and pattern recognition

Image segmentation Amplitude thresholding/ grey level window slicing Component labeling  Pixel labeling. Region growing 1) Local maxima detection 2) Local minima detection Objects are found between the local minima Object Digital image processing Chapter 8 Image analysis and pattern recognition

Boundary-based image segmentation Fig Segmentation algorithm based on boundary detection Region-based segmentation; segmentation based on regions and boundaries Fig Region merging Digital image processing Chapter 8 Image analysis and pattern recognition

Fig Segmentation by split and merge algorithm: a. input b. region splitting by quad-trees c. segmented regions Digital image processing Chapter 8 Image analysis and pattern recognition

a) Original image c) Main objects detected, marked on the original image by their contour b) Region-based image segmentation Digital image processing Chapter 8 Image analysis and pattern recognition