1. Automated Target Recognition Using Mathematical Morphology Prof. Robert Haralick Ilknur Icke José Hanchi Computer Science Dept. The Graduate Center of CUNY

2. Outline Gray Scale Morphology Converting Images to Datasets Decision Tree Classifier Results / Conclusions

3. Outline Gray Scale Morphology Converting Images to Datasets Decision Tree Classifier Results / Conclusions

4. Mathematical Morphology IS Given an image I  E N and a structuring element S  E N define the morphological operation of Dilation and set translation as Dilation is translation invariant

5. Mathematical Morphology I S Dilation

6. Mathematical Morphology Define the morphological operation of Erosion Erosion is translation invariant

7. Mathematical Morphology If a structuring element can be decomposed as then

8. Basic Morphology Operators Opening Closing

9. Gray Scale Morphology Dilation of f by k Erosion of f by k

10. Gray Scale Morphology Opening of f by k Closing of f by k

11. We have used flat structuring elements of size  { 3,5,7,9,11,13,15,17,19,21 } Structuring Elements Used H w = Horizontal V h =Vertical B w x h = Box h = 5 w = 5 … an illustration

12. Dilation w = 9

13. Erosion w = 9

14. Opening w = 9

15. Closing w = 9

16. The van-Herk-Gil-Werman (HGW) Algorithm—dilation STAGE 1 Given the input signal stream and a flat structuring element of size = 3 x 0, x 1, x 2, x 3, x 4, x 5, x 6, x 7, x 8, x 9, x 10, x 11, x 12, x 13,… 94514211812783 … center segments located at x 0, x 1, x 2, x 3, x 4, x 5 x 6, x 7, x 8 x 9, x 10, x 11 … example: … take the first segment and find the max (i.e. dilation)…

17. The van-Herk-Gil-Werman (HGW) Algorithm—dilation STAGE 1.a 9451421 5 copy 9 max R0R0 5 R1R1 5 R2R2 9 4 preprocess the prefixes x0x0 x4x4 x2x2 x1x1 x3x3

18. The van-Herk-Gil-Werman (HGW) Algorithm—dilation STAGE 1.b 9451421 5 copy 21 max 14 max preprocess the suffixes S0S0 5 S1S1 14 S2S2 21 x0x0 x4x4 x2x2 x1x1 x3x3

19. The van-Herk-Gil-Werman (HGW) Algorithm—dilation STAGE 2 9451421 R0R0 5 R1R1 5 R2R2 9 merging prefixes and suffixes S0S0 5 S1S1 14 S2S2 21 max 9 9 14 max 21 number of max operations per window: x0x0 x4x4 x2x2 x1x1 x3x3

20. The van-Herk-Gil-Werman (HGW) Algorithm—dilation Processing a given input signal for p=3, segment size=5 x 0, x 1, x 2, x 3, x 4, x 5, x 6, x 7, x 8, x 9, x 10, … 91421 18128 … 94514211812783 …

21. Calculating Morphological Features in 2-D The HGW algorithm works on 1-D input To apply it to 2-D images apply –Horizontal Structuring Elements process the image line by line –Vertical Structuring Elements transpose the image process line by line transpose again –Box shaped Structuring Elements horizontal first, then vertical

22. Efficiency of Flat Structuring Elements Given the flat structuring elements H and V Dilation Erosion Opening Closing Since and given w = h Dilation Erosion Opening Closing

23. Original Image Dilation with H 5

24. Erosion With H 5

25. Opening With H 5

26. Closing With H 5

27. Using V 5 Structuring Element erosiondilationopeningclosing

28. Using B 5x5 Structuring Element dilationerosionopeningclosing

29. Outline Gray Scale Morphology Converting Images to Datasets Decision Tree Classifier Results / Conclusions

30. Using Morphological Operations As Features for a Pixel ground truth imageI (3 structural elements) x (10 sizes) x (4morphological operations) = 120 transformed images … …

31. Using Morphological Operations As Features for a Pixel ground truth imageI … … class label f 13 f 14 f 15 f 16 f 17 f 18 f 19 f 20 f 21 f 22 f 23 f 24 {t,c} ={1,0} Given a pixel

32. Ground Truth Image mxn of mxn pixels I (x 1,f1, x 1,f2, x 1, f3,..., x 1, f119, x 1,f120, t) (x 2,f1, x 2,f2, x 2, f3,..., x 2, f119, x 2,f120, t) (x 3,f1, x 3,f2, x 3, f3,..., x 3, f119, x 3,f120, c)... (x N-1,f1,x N-1,f2, x N-1, f3,..., x N-1, f119, x N-1,f120, c) (x N,f1, x N,f2, x N, f3,..., x N, f119, x N,f120, t) data set representation I of I of size N = mxn D Morphological Features Data Set From an Image

33. Preparation Of Data Sets to Train and Test the Classifier D1D1D1D1 D2D2D2D2 … DkDkDkDk D k+1 … DKDKDKDK Create datasets separate vectors I1I1I1I1 I2I2I2I2 … IkIkIkIk I k+1 … IKIKIKIK ground truth images target dataset clutter dataset training dataset test dataset

34. Outline Gray Scale Morphology Converting Images to Datasets Decision Tree Classifier Results / Conclusions

35. Creating a Decision Tree Classifier classify classified dataset evaluateaccuracy create decision tree decision tree training dataset parameters test dataset decision tree test dataset classified dataset

36. Creating a Decision Tree Classifier 1 1 1 1 1 11 1 1 0 0 0 0 0 0 1111 2222 f1 f2 D training f1 1 f1 >  1 true f2 2 f2 >  2 class 0 true 0 0 00 0 0 0 0 3333 4444 f2 3 f2 >  3 class 1 true class 0 f1 4 f1 >  4 class 1 true class 0 threshold decision rule max.entropy = 0.001 max. depth = 20

37. Outline Gray Scale Morphology Converting Images to Datasets Decision Tree Classifier Results / Conclusions

38. Decision Tree Classifier results for test dataset derived from images of resolution = 75mm ClutterTarget Clutter 1,712,09015,422 Target 1,41954,503 train dataset size = 292,831 vectors test dataset size = 1,783,434 vectors true class assigned class accuracy (% correct classification) = 99.046%

39. Decision Tree Classifier results for images of resolution = 75mm 345 images of clutter-only 44 images with mostly target accuracy# clutter images# target images [0.91, 0.92) 4- [0.92, 0.93) 3- [0.93, 0.94) 4- [0.94, 0.95) 12- [0.95, 0.96) 14- [0.96, 0.97) 18- [0.97, 0.98) 315 [0.98, 0.99) 9321 [0.99, 1.00) 16618 Total # images 34544

40. Decision Tree Classifier results for test dataset derived from images of resolution = 200mm ClutterTarget Clutter 231,4951,702 Target 2548,391 train dataset size = 64,127 vectors test dataset size = 241,842 vectors true class assigned class accuracy (% correct classification) = 99.19%

41. Decision Tree Classifier for images with resolution = 200mm 689 images with mostly clutter 34 images with mostly target accuracy# clutter images# target images < 0.90 4- [0.90, 0.92) 1- [0.92, 0.94) 1- [0.94, 0.95) 21 [0.95, 0.96) 3- [0.96, 0.97) 81 [0.97, 0.98) 121 [0.98, 0.99) 7610 [0.99, 1.00] 58221 Total # images 68934