1 Faculty of Information Technology Enhanced Generic Fourier Descriptor for Object-Based Image Retrieval Dengsheng Zhang, Guojun Lu Gippsland School of Comp. & Info Tech Monash University Churchill, VIC 3842 Australia

2 Faculty of Information Technology Outline Motivations Generic Fourier Descriptor (GFD) Problem Enhanced Generic Fourier Descriptor (EGFD) Experimental Results Conclusions Motivations Generic Fourier Descriptor (GFD) Problem Enhanced Generic Fourier Descriptor (EGFD) Experimental Results Conclusions

3 Faculty of Information Technology Motivations Content-based Image Retrieval –Shape is an important image feature along with color and texture Effective and Efficient Shape Descriptor –good retrieval accuracy, compact features, general application, low computation complexity, robust retrieval performance and hierarchical coarse to fine representation Affined Shape Retrieval –Affined shapes are common in nature due to objects being viewed from different angles and objects being stretched, skewed. Content-based Image Retrieval –Shape is an important image feature along with color and texture Effective and Efficient Shape Descriptor –good retrieval accuracy, compact features, general application, low computation complexity, robust retrieval performance and hierarchical coarse to fine representation Affined Shape Retrieval –Affined shapes are common in nature due to objects being viewed from different angles and objects being stretched, skewed.

4 Faculty of Information Technology Affine Distorted Shapes Are Common

5 Faculty of Information Technology Generic Fourier Descriptor Polar Transform –For an input image f(x, y), it is first transformed into polar image f(r,  ): Polar Transform –For an input image f(x, y), it is first transformed into polar image f(r,  ):

6 Faculty of Information Technology Generic Fourier Descriptor-II Polar Raster Sampling Polar Grid Polar image Polar raster sampled image in Cartesian space

7 Faculty of Information Technology Generic Fourier Descriptor-III 2-D Fourier transform on polar raster sampled image f(r,  ): where 0  r<R and  i = i(2  /T) (0  i<T); 0  <R, 0  <T. R and T are the radial frequency resolution and angular frequency resolution respectively. The normalized Fourier coefficients are the GFD. 2-D Fourier transform on polar raster sampled image f(r,  ): where 0  r<R and  i = i(2  /T) (0  i<T); 0  <R, 0  <T. R and T are the radial frequency resolution and angular frequency resolution respectively. The normalized Fourier coefficients are the GFD.

8 Faculty of Information Technology Problem Generally, GFD has good performance on generic shapes. Its overall retrieval precision after full recall is 98.6% for rotated shapes, 90.5% for scaled shapes, 74.1% for perspective transformed shapes and 80.5% for generally distorted shapes. Compared with rotation and scaling invariance test, the retrieval performance on perspective transform and generally distorted shapes are significantly lower. The problem is caused by the polar raster sampling method. Generally, GFD has good performance on generic shapes. Its overall retrieval precision after full recall is 98.6% for rotated shapes, 90.5% for scaled shapes, 74.1% for perspective transformed shapes and 80.5% for generally distorted shapes. Compared with rotation and scaling invariance test, the retrieval performance on perspective transform and generally distorted shapes are significantly lower. The problem is caused by the polar raster sampling method.

9 Faculty of Information Technology Under-sampling Problem Only half the sampled positions contain shape information

10 Faculty of Information Technology Enhanced GFD Normalization –Find major axis –Rotation normalization so that major axis of the shape is horizontal –Scaling normalization so that the ecentricity of the shape is 1. Normalization –Find major axis –Rotation normalization so that major axis of the shape is horizontal –Scaling normalization so that the ecentricity of the shape is 1.

11 Faculty of Information Technology Enhanced GFD-II Optimized Major Axis Algorithm (MAA) –Find the boundary point pairs in a number of directions (e.g. 360). –Find the two points p 1, p 2 with the furthest distance in the found boundary points, then p 1 p 2 is the major axis. The computation of MAA is O(N) rather than O(N 2 ). Optimized Major Axis Algorithm (MAA) –Find the boundary point pairs in a number of directions (e.g. 360). –Find the two points p 1, p 2 with the furthest distance in the found boundary points, then p 1 p 2 is the major axis. The computation of MAA is O(N) rather than O(N 2 ).

12 Faculty of Information Technology Enhanced GFD-III Normalization Result:

13 Faculty of Information Technology Enhanced GFD-IV Applying GFD transform on the rotation and scaling normalized image. The normalized transform coefficients are the enhanced GFD (EGFD). Applying GFD transform on the rotation and scaling normalized image. The normalized transform coefficients are the enhanced GFD (EGFD).

14 Faculty of Information Technology Experiment Dataset –Two datasets from MPEG-7 region shape database CE-2 are used. (CE-2 has been organized by MPEG-7 into six datasets to test a shape descriptor’s behaviors under different distortions) –Set A4 consists of 3101 from the whole database, it is for test of robustness to perspective transform. 330 shapes in Set A4 have been organized into 30 groups (11 similar shapes in each group) which are used as queries. –The whole database consists of 3621 shapes, 651 shapes have been organized into 31 groups (21 similar shapes in each group) which are used as queries. Indexing and automatic retrieval Dataset –Two datasets from MPEG-7 region shape database CE-2 are used. (CE-2 has been organized by MPEG-7 into six datasets to test a shape descriptor’s behaviors under different distortions) –Set A4 consists of 3101 from the whole database, it is for test of robustness to perspective transform. 330 shapes in Set A4 have been organized into 30 groups (11 similar shapes in each group) which are used as queries. –The whole database consists of 3621 shapes, 651 shapes have been organized into 31 groups (21 similar shapes in each group) which are used as queries. Indexing and automatic retrieval

15 Faculty of Information Technology Performance Measurement Recall vs Precision

16 Faculty of Information Technology Results Recall-Precision of EGFD on perspective shapes. –Compared with GFD, the improvement on Set A4 is 15.4%, the overall precision is increased from 74.1% to 89.5%. Recall-Precision of EGFD on perspective shapes. –Compared with GFD, the improvement on Set A4 is 15.4%, the overall precision is increased from 74.1% to 89.5%.

17 Faculty of Information Technology Results Recall-Precision of EGFD on Generally Distorted Shapes –Compared with GFD, the improvement on CE-2 is 12%, the overall precision is increased from 80.5% to 92.5%. Recall-Precision of EGFD on Generally Distorted Shapes –Compared with GFD, the improvement on CE-2 is 12%, the overall precision is increased from 80.5% to 92.5%.

18 Faculty of Information Technology Results EGFD GFD ZMD

19 Faculty of Information Technology Results EGFD GFD ZMD

20 Faculty of Information Technology Application of EGFD The application of the enhancement process is database/ application dependent. If the database has abundant perspective shapes, this technique can be very effective in retrieving similar shapes. If the database does not have perspective shapes, or the user wants finer distinction between similar shapes, the enhanced process may not be desirable. For example, if the user wants to distinguish between rectangles and squares, or to distinguish between ellipses and circles, the enhanced GFD can fail, because it normalizes all the shapes into same eccentricity (=1). In general applications, the enhancement is a useful option to the retrieval system rather than the replacement of GFD. The application of the enhancement process is database/ application dependent. If the database has abundant perspective shapes, this technique can be very effective in retrieving similar shapes. If the database does not have perspective shapes, or the user wants finer distinction between similar shapes, the enhanced process may not be desirable. For example, if the user wants to distinguish between rectangles and squares, or to distinguish between ellipses and circles, the enhanced GFD can fail, because it normalizes all the shapes into same eccentricity (=1). In general applications, the enhancement is a useful option to the retrieval system rather than the replacement of GFD.

21 Faculty of Information Technology Conclusions The proposed EGFD improves GFD significantly. It improves GFD’s relatively lower retrieval performance on severely skewed and stretched shapes. It also improves GFD’s robustness to general shape distortions. A shape normalization method is presented. The shape normalization method can be exploited for general shape representation purposes. An optimized major axis algorithm (MAA) is proposed. MA is a common normalization mechanism in shape modeling and representation. Common MAA is only for finding MA of contour shape. The proposed optimized MAA can be used for finding MA of generic shapes. The proposed EGFD improves GFD significantly. It improves GFD’s relatively lower retrieval performance on severely skewed and stretched shapes. It also improves GFD’s robustness to general shape distortions. A shape normalization method is presented. The shape normalization method can be exploited for general shape representation purposes. An optimized major axis algorithm (MAA) is proposed. MA is a common normalization mechanism in shape modeling and representation. Common MAA is only for finding MA of contour shape. The proposed optimized MAA can be used for finding MA of generic shapes.