1. James C. Tilton Code 606.3 Computational & Information Sciences and Technology Office NASA Goddard Space Flight Center January 17, 2014 update National Aeronautics and Space Administration www.nasa.gov

2. HSeg Background 2 HSeg produces a hierarchical set of image segmentations with the following characteristics: A set of segmentations that 1.consist of segmentations at different levels of detail, in which 2.the coarser segmentations can be produced from merges of regions from the finer segmentations, and 3.the region boundaries are maintained at the full image spatial resolution The HSeg algorithm is fully described in: James C. Tilton, Yuliya Tarabalka, Paul M. Montesano and Emanuel Gofman, “Best Merge Region Growing Segmentation with Integrated Non-Adjacent Region Object Aggregation,” IEEE Transactions on Geoscience and Remote Sensing, Vol. 50, No. 11, Nov. 2012, pp. 4454-4467. 15 August 2013USGS Global Croplands Working Group Meeting

3. Problem: Large homogeneous regions with gradual gradients aren’t readily formed. 3 Observation: The boundaries between HSeg (or HSWO) subregions of large homogeneous regions do not correspond to any visibly apparent boundary – There is no “edge” between these subregions. Idea: Can edge information be utilized to influence the HSWO/HSeg region growing process to encourage the merging together of large homogeneous regions with gradual gradients? 15 August 2013USGS Global Croplands Working Group Meeting

4. Edge Detection Edge detection aims to identify image points at which the image values change abruptly. The Canny edge detector is considered to be the “state-of-the-art” in edge detection. The Canny edge detector uses a multi-stage process to form a binary edge image. For the purpose of incorporating edge information into HSWO/HSeg, we don’t need such a complicated edge detector – and it would be better to have relative edge information instead of an all or nothing binary edge image. Some simpler choices are the Sobel, Prewitt, Roberts Cross and Frei-Chen edge difference operators. I prefer the Frei-Chen operator because it is the only one that (i) is sensitive to diagonal edges as well as vertical and horizontal edges, and (ii) is normalized to give numeric results in a consistent range (0.0 to 1.0). 4 15 August 2013USGS Global Croplands Working Group Meeting

5. Frei-Chen Edge Difference Operator 5 15 August 2013USGS Global Croplands Working Group Meeting

6. Frei-Chen Edge Difference Operator Result: A true color rendition of a 768x768 pixel section of Ikonos data from the Patterson Park/Inner Harbor area of Baltimore, MD. Frei-Chen Edge Difference Operator Result, maximum over spectral bands, thresholded at 0.07. 6 15 August 2013USGS Global Croplands Working Group Meeting

7. Incorporating Edge Information into HSWO/HSeg/RHSeg 7 Edge information is incorporated at three different stages: 1.An initialization stage in which the edge information directs a fast first-merge region growing process (Muerle-Allen)to quickly merge connected areas with edge values <= E t (set by user), and 2.The normal HSWO/HSeg best merge region growing stage in which the edge information influences the best merge decisions. 3.In performing processing window artifact elimination in RHSeg. J. L. Muerle, D. C. Allen, “Experimental Evaluation of Techniques for Automatic Segmentation of Objects in a Complex Scene,” in G. C. Cheng, et al. (Eds.), Pictorial Pattern Recognition, Thompson, Washington, DC, pp. 3-13, 1968. 15 August 2013USGS Global Croplands Working Group Meeting

8. Incorporating Edge Information into HSWO/HSeg/RHSeg 8 I’ve experimented with three approaches to having the edge information influence the best merge decisions. HSeg Version 1.61: A simple extension of the original HSeg in that one region feature value, E max, is added to each region, which is the maximum value of E value for all pixels in the region. The edge dissimilarity value, E dissim, is taken to be the maximum of E max for the two regions being compared. HSeg Versions 1.71 and 1.80: These versions are a more complicated extension of the original version in that a region data structure is modified to enable the tracking of the value of E value along the mutual boundary between two regions. The value of the edge dissimilarity value, E dissim, is taken to be the average of E value for mutual boundary pixels between the two regions. Versions 1.71 and 1.80 differ in which mutual boundary pixels are used (next slide). 15 August 2013USGS Global Croplands Working Group Meeting

9. RHSeg V1.71 Edge Information Incorporation Scheme: 9

10. RHSeg V1.80 Edge Information Incorporation Scheme 10

11. Influencing the Best Merge Decision with Edge Information: 11 15 August 2013USGS Global Croplands Working Group Meeting

12. Influencing the Best Merge Decision with Edge Information: 12 15 August 2013USGS Global Croplands Working Group Meeting

13. Tests on three GFSAD30 image data sets: 13 Processed with rhseg versions 1.59, 1.71 and 1.80 (rhseg version 1.61 not yet available) with S w = 0.2 and 0.5 and E w = 1.0 on NCCS Discover cluster with 64 CPUs. Quickbird image from South Africa: 3323 columns by 3397 rows by 4 bands. 15 August 2013USGS Global Croplands Working Group Meeting RHSeg version SwSw EtEt (hr:min:sec) 1.590.2-00:02:11 1.590.5-00:03:50 1.710.20.0500:23:30 1.710.50.0500:19:30 1.800.20.0500:20:53 1.800.50.0500:34:16

14. Tests on three GFSAD30 image data sets (cont’d): 14 Processed with rhseg versions 1.59, 1.71 and 1.80 (rhseg version 1.61 not yet available) with S w = 0.2 and 0.5 and E w = 1.0 on NCCS Discover cluster with 64 CPUs. MS image (sensor?) from Iraq: 2004 columns by 3778 rows by 4 bands. 15 August 2013USGS Global Croplands Working Group Meeting RHSeg version SwSw EtEt (hr:min:sec) 1.590.2-00:01:49 1.590.5-00:03:42 1.710.20.0500:34:57 1.710.50.0500:13:06 1.800.20.0500:31:26 1.800.50.0500:36:38

15. Tests on three GFSAD30 image data sets (cont’d): 15 Processed with rhseg versions 1.59, 1.71 and 1.80 (rhseg version 1.61 not yet available) with S w = 0.2 and 0.5 and E w = 1.0 on NCCS Discover cluster with 32 CPUs. MS image (sensor?) from Morocco: 6444 columns by 2137 rows by 4 bands. 15 August 2013USGS Global Croplands Working Group Meeting RHSeg version SwSw EtEt (hr:min:sec) 1.590.2-00:02:42 1.590.5-00:05:22 1.710.20.0300:15:04 1.710.50.0300:20:21 1.800.20.0300:28:13 1.800.50.0300:49:00

16. SignOffPage 16 15 August 2013USGS Global Croplands Working Group Meeting Look at results with HSegViewer and HSegLearn

17. SignOffPage 17 15 August 2013USGS Global Croplands Working Group Meeting