1. 8D040 Basis beeldverwerking Feature Extraction Anna Vilanova i Bartrolí Biomedical Image Analysis Group bmia.bmt.tue.nl

2. N=M=30 What is an image? Image is a 2D rectilinear array of pixels (picture element) N=M=256

3. L=15(4 bits) L=255 (8 bits) What is an image? No continuous values - Quantization 255 170 15 8 Binary Image L=1 (1 bit) L=3 (2 bits)

4. An image is just 2D? No! – It can be in any dimension Example 3D: Voxel-Volume Element

5. Segmentation

6. Reduction of dimensionality Why feature extraction ? Pixel level Image of 256x256 and 8 bits 256 65536 ~ 10 157826 possible images

7. Incorporation of cues from human perception Transcendence of the limits of human perception The need for invariance Why feature extraction ?

8. Apple detection …

9. Transformation (Rotation)

10. How do we transform an image? We transform a point P How do we transform an image f(P) ? How do we know which Q belongs to P ?

11. How do we transform an image? How do we transform an image f(P) ? We know T which is the transformation we want to achieve. How do we know which Q belongs to P ?

12. Apple detection …

13. Feature Characteristics Invariance (e.g., Rotation, Translation) Robust (minimum dependence on) Noise, artifacts, intrinsic variations User parameter settings Quantitative measures

15. We extract features from… Region of Interest Segmented Objects

16. Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

17. Shape Based Features Object based Topology based (Euler Number) Effective Diameter (similarity to a circle to a box) Circularity Compactness Projections Moments (derived by Hu 1962) …

18. 4-neighbourhood of 8-neighbourhood of Adjacency and Connectivity – 2D  Notation: k -Neighbourhood of is

19. Adjacency and Connectivity – 3D 6-neighbourhood 18-neighbourhood 26-neighbourhood

20. Objects or Components (Jordan Theorem) In 2D – (8,4) or (4,8)-connectivity In 3D – (6,26)-,(26,6)-,(18,6)- or (6,18)-connectivity

21. Connected Components Labeling Each object gets a different label

22. Connected Components Labeling A B C Raster Scan Note: We want to label A. Assuming objects are 4-connected B, C are already labeled. Cortesy of S. Narasimhan

23. Connected Components Labeling 1 0 0  label(A) = new label 0 X X  label(A) = “background” 1 0 C  label(A) = label(C) 1 B 0  label(A) = label(B) 1 B C  If label(B) = label(C) then, label(A) = label(B) Cortesy of S. Narasimhan

24. 2 42223 4444? 2 22223 22222 2 22223 22222223 2222222 222  What if label(B) not equal to label(C)? Connected Components Labeling 1 B C

25. Each object gets a different label

26. Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

27. Topology based – Euler Number Euler Number E describes topology. C is # connected components H is # of holes.

28. Euler Number 3D Euler Number E describes topology. C is # connected components Cav is # of cavities G is # of genus E=1+0-1=0 E=1+1-0=2

29. Euler Number 3D E=2+0-0=2 E=1+1-0=2 Euler Number E describes topology. C is # connected components Cav is # of cavities G is # of genus

30. 3D Euler Number The Euler Number in 3D can be computed with local operations Counting number of vertices, edges and faces of the surfaces of the objects

31. Simple Shape Measurements 2D area - 3D volume Summing elements 2D perimeter - 3D surface area Selection of border elements Sum of elemets with weights Error of precision

32. Similarity to other Shape Effective Diameter Circularity (Circle C=1) Compactness – (Actually non-compactness) (Circle Comp= )

33. Moments Definition Order of a moment is Moments identify an object uniquely ? is the Area Centroid

34. Central Moments Moments invariant to position Invariant to scaling

35. Moments to Define Shape and Orientation Inertia Tensor

36. Eigenanalysis of a Matrix Given a matrix S, we solve the following equation we find the eigenvectors and eigenvalues Eigenvectors and eigenvalues go in couples an usually are ordered as follows:

37. Eigenanalysis of the Inertia Matrix Eigenanalysis Sphere Flatness Elongated

38. Eigenanalysis of the Inertia Matrix Eigenanalysis Sphere Flatness Elongated

39. Orientation in 2D Using similar concepts than 3D Covariance or Inertia Matrix Eigenanalysis we obtain 2 eigenvalues and 2 eigenvectors of the ellipse

40. Moments Invariance Translation Central moments are invariant Rotation Eigenvalues of Inertia Matrix are invariant Scaling If moment scaled by (3D) (2D)

41. Moments invariant rotation-translation-scaling For 3D three moments (Sadjadi 1980) For 2D seven moments

42. Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

43. Image Based Features Using all pixels individually Histogram based features −Statistical Moments (Mean, variance, smoothness) −Energy −Entropy −Max-Min of the histogram −Median Co-occurrance Matrix Gonzalez & Woods – Digital Image Processing Chapter 11 – 11.3.3 Texture

44. Histogram L=9 bibi P(b i )

45. How do the histograms of this images look like?

46. Bimodal Histogram

47. Trimodal Features

48. Histogram Features Mean Central Moments

49. Histogram Features Mean Variance Relative Smoothness Skewness

50. Histogram Features Energy (Uniformity) Entropy

51. Examples of Energy and Entropy Energy=1 Entropy=0 Energy=0,111 Entropy=3,327 Energy=0,255 Entropy=2,018 Energy=0,0625 Entropy= 4

52. Examples TextureMeanstdR3rd momentEnergyEntropy 182.6411.790.002-0.1050.0265.434 2143.5674.630.079-0.1510.0057.783 399.7233.730.0170.7500.0136.374 1 2 3

54. The next slides were not given, during the lecture and will not be asked in the exam

55. Intensity Co-occurrance Matrix Operator Q defines the position between two pixels (e.g, pixel to the right) Co-occurance matrix G is ( L+1) x (L+1) (6x6). Counts how often Q occurs 0 0 1 1 4 4 0 0 4 4 5 5 2 2 3 3 1 1 2 2 3 3 4 4 4 4 4 4 1 1 1 1 6 6 3 3 5 5 1 1 1 1 6 6 5 5 5 5 1 1 0123456 00100100 10020101 20002000 30100100 42100011 50201000 60000010 Image G

56. Example L=256 Q “one pixel immediately to the right” Image G - Matrix

57. Features based on the co-ocurrence Matrix The elements of G (g ij ) is converted to probability (p ij ) by dividing by the amount of pairs in G Based on the probability density function we can use Maximum Energy (uniformity) Entropy

58. Features based on the co-ocurrence Matrix Homogenity – closeness to a diagonal matrix Contrast

59. Features based on the co-ocurrence Matrix Correlation – measure of correlation with neighbours

60. Example L=256 Q “one pixel immediately to the right” Image G - Matrix

61. Example Image G - Matrix CorrelationContrastHomogeneity 1- 0.0005108380.0366 2 0.96505700.0824 3 0.879813560.2048

62. Moments Definition Order of a moment is Moments identify an object uniquely Centroid

63. Central Moments Moments invariant to position Normalized central moments

64. Moments invariant rotation-translation-scaling For 3D three moments (Sadjadi 1980) For 2D seven moments (Hu’s 1962)

65. Moments invariant rotation-translation- scaling-mirroring (within minus sign) are all equal Mirroring

67. Projections x y