Content-based Image Retrieval CE 264 Xiaoguang Feng March 14, 2002 Based on: J. Huang. Color-Spatial Image Indexing and Applications. Ph.D thesis, Cornell Univ., 1998.

Contents Introduction. Color-histogram vs. Correlogram. Implementations and Results. Conclusion.

Introduction Motivation of CBIR Image features for CBIR Low Level:  Color  Texture  Edge/Shape Object Level:  Regions

Color Histogram The histogram of image I is defined as: For a color C i, H ci (I) represents the number of pixels of color C i in image I. OR: For any pixel in image I, H ci (I) represents the possibility of that pixel is in color C i. Most commercial CBIR systems include color histogram as one of the features (e.g., QBIC of IBM). No space information.

Improvement of color histogram There are several techniques proposed to integrate spatial information with color histograms: W.Hsu, et al., An integrated color-spatial approach to content-based image retrieval. 3 rd ACM Multimedia Conf. Nov Smith and Chang, Tools and techniques for color image retrieval, SPIE Proc. 2670, Stricker and Dimai, Color indexing with weak spatial constraints, SPIE Proc. 2670, Gong, et al., Image indexing and retrieval based on human perceptual color clustering, Proc. 17 th IEEE Conf. On Computer Vision and Pattern Recognition, Pass and Zabih, Histogram refinement for content-based image retrieval. IEEE Workshop on Applications of Computer Vision, Park, et al., Models and algorithms for efficient color image indexing. Proc. Of IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997.

Color auto-correlogram Pick any pixel p1 of color C i in the image I, at distance k away from p1 pick another pixel p2, what is the probability that p2 is also of color C i ? P1 P2k Red ? Image: I

Color auto-correlogram The auto-correlogram of image I for color C i, distance k : Integrate both color information and space information.

Color auto-correlogram

Implementations Pixel Distance Measures Use D8 distance (also called chessboard distance): Choose distance k=1,3,5,7 Computation complexity:  Histogram:  Correlogram:

Implementations Features Distance Measures: D( f(I 1 ) - f(I 2 ) ) is small  I 1 and I 2 are similar. Example: f(a)=1000, f(a’)=1050; f(b)=100, f(b’)=150 For histogram: For correlogram:

Test Environment 300 Color Images: flowers, people, scene, etc.

Test Environment Image quantized to 512 and 64 colors. First calculate the correlogram and histogram of the 300 images, saved as data file. For each query, calculate the correlogram and histogram of the query image; compare it with the data file; sort the feature distances. The order of the target image in the sorted searching result measures the performance.

Test Results no difference If there is no difference between the query and the target images, both methods have good performance. Query Image (512 colors) Correlogram method Histogram method 1st2nd3rd4th5th 1st2nd3rd4th5th

Test Results color change The correlogram method is more stable to color change than the histogram method. Query Target Correlogram method: 1 st Histogram method: 48 th

Test Results large appearance change The correlogram method is more stable to large appearance change than the histogram method. Query Target Correlogram method: 1 st Histogram method: 31 th

Test Results contrast & brightness change The correlogram method is more stable to contrast & brightness change than the histogram method. Query 1 Target C: 178 th H: 230 th Query 2 Query 3 Query 4 C: 1 st H: 1 st C: 1 st H: 3 rd C: 5 th H: 18 th

Conclusion The color correlogram describes the global distribution of local spatial correlations of colors. It’s easy to compute. It’s more stable than the color histogram method.

Thanks