Medical Image Analysis Image Representation and Analysis Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Image Representation and Analysis Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, A hierarchical framework of processing steps representing the image (data) and knowledge (model) domains Scenes of specific objects Surface regions (S-regions) Region Contours and edges Pixels

Bottom-Up Scenario Scene-1Scene-I Object-1Object-J S-Region-1S-Region- K Region-1Region-L Pixel (i,j) Edge-MEdge-1 Pixel (k,l) Top-Down Figure 8.1. A hierarchical representation of image features.

Image Reconstruction Image Segmentation (Edge and Region) Feature Extraction and Representation Classification and Object Identification Analysis of Classified Objects Multi-Modality/Multi- Subject/Multi-Dimensional Registration, Visualization and Analysis Raw Data from Imaging System Single Image Understanding Multi-Modality/ Multi-Subject/Multi- Dimensional Image Understanding Scene Representation Models Object Representation Models Feature Representation Models Edge/Region Representation Models Physical Property/ Constraint Models Knowledge Domain Data Domain Figure 8.2. A hierarchical structure of medical image analysis.

Feature Extraction and Representation Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Statistical pixel-level (SPL) features ◦ Mean, variance, histogram, area, contrast of pixels within the region, edge gradient of boundary pixels Shape feature ◦ Circularity, compactness, moments, chain- codes and Hough transform, morphological processing methods

Feature Extraction and Representation Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Texture features ◦ Second-order histogram statistics or co- occurrence matrices, wavelet processing methods for spatio-frequency analysis Relational features ◦ Relational and hierarchical structure of the regions associated with a single or a group of objects

Statistical Pixel-Level (SPL) Features Histogram Mean Variance and central moments

Statistical Pixel-Level (SPL) Features ◦ The third central moment is a measure of noncentrality ◦ The fourth central moment is a measure of flatness of the histogram Energy

Statistical Pixel-Level (SPL) Features Entropy ◦ The entropy Ent is a measure of information represented by the distribution of gray-values in the region

Statistical Pixel-Level (SPL) Features Local contrast Maximum, minimum The mean, variance, energy and entropy of contrast values Gradient information for the boundary pixels

Shape Features Longest axis GE Shortest axis HF Perimeter and area of the minimum bounded rectangle ABCD Elongation ratio: GE/HF Perimeter and the area of the segmented region Hough transform of the region using the gradient information of the boundary pixels of the region

Shape Features Circularity ( = 1 for a circle) of the region computed as Compactness of the region computed as

Shape Features Chain code for boundary contour ◦ Obtained using a set of orientation primitives on the boundary segments derived from a piecewise linear approximation Fourier descriptor of boundary contours ◦ Obtained using the Fourier transform of the sequence of boundary segments derived from a piecewise linear approximation

Shape Features Central moments based shape features for the segmented region Morphological shape descriptors ◦ Obtained through the morphological processing on the segmented region

Boundary Encoding: Chain Code Orientation primitives ◦ 8-connected neighborhood Divide-and-conquer ◦ Curve approximation Maximum-deviation criterion ◦ Perpendicular distance between any point on the original curve segment between the selected vertices and the corresponding approximated straight-line segment

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, xcxc Figure 8.4. The 8-connected neighborhood codes (left) and the orientation directions (right) with respect to the center pixel x c.

F A D C E B A D C E B A D C E B A D C E B A B C D Chain Code: Figure 8.5. A schematic example of developing chain code for a region with boundary contour ABCDE. From top left to bottom right: the original boundary contour, two points A and C with maximum vertical distance parameter BF, two segments AB and BC approximating the contour ABC, five segments approximating the entire contour ABCDE, contour approximation represented in terms of orientation primitives, and the respective chain code of the boundary contour.

Boundary Encoding: Fourier Descriptor Closed boundary of a region Discrete Fourier transform (DFT) of the sequence Rigid geometric transformation of a boundary ◦ Translation, rotation, scaling

Moments for Shape Description Central moments of a segmented image Invariant moments ◦ Shape matching, pattern recognition

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Set A Set B Figure 8.6. A large region with square shape representing the set A and a small region with rectangular shape representing the structuring element set B.

: Dilation of A by B A B: Erosion of A by B ( A B) B A A A B Figure 8.7: The dilation of set A by the structuring element set B (top left), the erosion of set A by the structuring element set B (top right) and the result of two successive erosions of set A by the structuring element set B (bottom).

A B Figure 8.8. Dilation and erosion of an arbitrary shape region A (top left) by a circular structuring element B (top right): dilation of A by B (bottom left) and erosion of A by B (bottom right).

Figure comes from the Wikipedia, Dilation

Figure comes from the Wikipedia, Erosion

Morphological Processing for Shape Description Opening Closing

A B Figure 8.9. The morphological opening and closing of set A (top left) by the structuring element set B (top right): opening of A by B (bottom left) and closing of A by B (bottom right).

Figure comes from the Wikipedia, Opening

Figure comes from the Wikipedia, Closing

Morphological Processing for Shape Description Skeleton Image processing ◦ Erosion can reduce the background noise ◦ Opening can remove the speckle noise and provide smooth contours

Morphological Processing for Shape Description Image processing ◦ Closing preserves the peaks and reduces the sharp variations in the signal such as dark artifacts ◦ Opening followed by closing can reduce the bright and dark artifacts and noise ◦ The morphological gradient image can be obtained by subtracting the eroded image from the dilated image ◦ Edges can also be detected by subtracting the eroded image from the original image

Figure Example of morphological operations on MR brain image using a structuring element of (a) the original MR brain image; (b) the thresholded MR brain image for morphological operations; (c) dilation of the thesholded MR brain image; (d) resultant image after 5 successive dilations of the thresholded brain image; (e) erosion of the thresholded MR brain image; (f) closing of the thesholded MR brain image; (g) opening of the thresholded MR brain image; and (h) morphological boundary detection on the thresholded MR brain image. (b)(a)

(c)(d) (f) (e)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, (g) (h)

Texture Features Texture ◦ Statistical ◦ Structural  A repetitive arrangement of square and triangular shapes ◦ Spectral  Fourier and wavelet transforms Gray-level co-occurrence matrix (GLCM) ◦ is the distribution of the number of occurrences of a pair of gray values and separated by a distance vector

(a) (b) Figure (a) A matrix representation of a 5x5 pixel image with three gray values; (b) the GLCM P(i,j) for d=[1,1].

Texture Features ◦ The probability of occurrence of a pair of gray values and separated by a distance vector, ◦ The probability that a difference in gray-levels exists between two distinct pixels

Second-Order Histogram Statistics Entropy of Angular second moment of

Second-Order Histogram Statistics Contrast of Inverse difference moment of

Second-Order Histogram Statistics Correlation of

Second-Order Histogram Statistics Mean of Deviation of

Second-Order Histogram Statistics Entropy of Angular second moment of

Second-Order Histogram Statistics Mean of

Figures 8.12 (a) A part of a digitized X-ray mammogram showing a region of benign lesion (b) a part of a digitized X-ray mammogram showing a region of malignant cancer of the breast (c). A second-order histograms of (a) computed from the gray-level co-occurrence matrices with a distance vector of [1,1] and (d) A second-order histogram of (b) computed from the gray-level co-occurrence matrices with a distance vector of [1,1]. (a) (b)

(c)

(d)

Relational Features Relational features ◦ Information about adjacencies, repetitive patterns and geometrical relationships among regions of an object Quad-tree representation Tree and graph structures

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, R1R1 R 21 R 22 R 23 R 41 R 43 R 24 R 42 R 44 R3R3 Figure 8.13: A block representation of an image with major quad partitions (top) and its quad-tree representation.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, R R4R4 R3R3 R2R2 R1R1 R 24 R 23 R 22 R 21 R 44 R 43 R 42 R 41 R 14 R 13 R 12 R 11 R 34 R 33 R 32 R 31

A C B D F I E B C A I E D F Figure A 2-D brain ventricles and skull model (top) and region- based tree representation.

Feature and Image Classification Statistical classification methods ◦ Unsupervised: k-means, fuzzy clustering ◦ Supervised Nearest neighbor classifier ◦ Assigned to the class if

Feature and Image Classification Bayes classifier ◦ Risk of wrong classification for assigning the feature vector to the class ◦ Assigned to the class if

Feature and Image Classification Rule-based systems ◦ Analyze the feature vector using multiple sets of rules that are designed to check specific conditions in the database of feature vectors to initiate an action

Strategy Rules A priori knowledge or models Focus of Attention Rules Knowledge Rules Activity Center Input Database Output Database Figure A schematic diagram of a rule-based system for image analysis.

Feature and Image Classification Image and feature classification: neural networks ◦ Backpropagation ◦ Radial basis function ◦ Associative memories ◦ Self-organizing Neuro-fuzzy pattern classification

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Figure A computational neuron model with linear synapses.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Figure The architecture of the Neuro-Fuzzy Pattern Classifier.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Figure The structure of the fuzzy membership function.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, Figure Convex set-based separation of two categories.

Figure (a). Fuzzy membership function M 1 (x) for the subset #1 of the black category. (b). Fuzzy membership function M 2 (x) for the subset #2 of the black category. (a)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, (b)

Figure Fuzzy membership function M 3 (x) (decision surface) for the white category membership.

Figure Resulting decision surface M black (x) for the black category membership function.

Image Analysis Example: Analysis of Difficult-to-Diagnose Mammographic Microcalcification Features ◦ Number of microcalcification ◦ Average number of pixels per microcalcification ◦ … ◦ Entropy of ◦ … ◦ Energy fro the wavelet packet at Level 0 ◦ …