Young Deok Chun, Nam Chul Kim, Member, IEEE, and Ick Hoon Jang, Member, IEEE IEEE TRANSACTIONS ON MULTIMEDIA,OCTOBER 2008

 Introduction  Conventional features  Proposed image retrieval method  Experimental results  Conclusion

 Typical CBIR  Extract features related to visual content from a query image  Compute similarity between the features of the query image and target images in DB  Target images are next retrieved which are most similar to the query image  Extraction of good features is one of the important tasks in CBIR  Shape ▪ Describes the contours of objects in an image ▪ Usually extracted from segmenting the image into meaningful regions or objects

 Color ▪ Most widely used visual features ▪ Invariant to image size and orientation  Texture ▪ Visual feature that refers to innate surface properties of an object ▪ Relationship to the surrounding environment  A feature extracted from an image is generally represented as a vector of finite dimension  Feature vector dimension is one of the most important factors that determine ▪ The amount of storage space for the vector ▪ Retrieval accuracy ▪ Retrieval time (or computational complexity)

 Explain the conventional features which are used in the proposed retrieval method  color feature ▪ color autocorrelogram  texture features ▪ BDIP ▪ BVLC

 Color autocorrelogram  Captures the spatial correlation between identical colors only  Probability of finding a pixel p’ of the identical color at a distance k from a given pixel p of the lth color  measure the distance between pixels  As k varies, spatial correlation between identical colors in an image can be obtained in various resolutions

 Block Difference of Inverse Probabilities  Texture feature that effectively extracts edges and valleys  Object boundaries are extracted well, and the intensity variation in dark regions is emphasized Block size (k+1)*(k+1) Intensity of pixel(x,y) Representative (maximum) of intensity variation in a block Representative value in a block Original image BDIP operator

 Block Variation of Local Correlation coefficients  Represents the variation of block-based local correlation coefficients according to four orientations  Local correlation  BVLC standard deviation of the lth block shifted by k Local covariance normalized by local variance Difference between the max and min values of block- based local correlation coefficients (four orientations)

 Overall structure A texture feature vector ft of dimension Nt with BDIP and BVLC moments extracted from Each component image is wavelet decomposed into a wavelet image 4-band wavelet decomposition configuration of 2-level wavelet decomposed images A color feature vector fc of dimension Nc is then formed with color autocorrelograms extracted from the and

 The and are first quantized into and  LL band ▪ uniformly quantized  other subbands ▪ non-uniformly quantized by the generalized Lloyd algorithm  Given the total number of quantization levels L of all subbands, L i is allocated to the ith subband

 Numbers of quantization levels for all subbands are decided  Color autocorrelogram is extracted  To reduce computational complexity, we modify the color autocorrelogram the set of pixels having the lth color

 The color feature vector f c is finally formed with the color autocorrelogram  color autocorrelogram  probability of finding a pixel p’ of the lth color among the two causal neighboring pixels at a distance k from a given pixel p of the lth color in the

 is first divided into nonoverlapping blocks of a given size, where the BDIP and BVLC are computed  BDIP  the denominator in may yield negative BDIP values in wavelet domain, which leads to invalid measurement of intensity variation Pixel values are nonnegative in LL band

 BVLC  Reduce computational complexity ▪ Local covariance ▪ Mean absolute difference of pixels between two blocks ▪ Local variance ▪ Mean absolute difference of four end pixels in the block

 Modified BVLC  Difference between the max and min values of the local correlation coefficients according to two orientations

 The first and second moments of the BDIP and BVLC for each subband are extracted  The texture feature vector

 After color and texture feature vectors are extracted, the retrieval system combines these feature vectors  Each of the color and texture feature components is normalized by its dimension and standard deviation  reducing the effect of different feature vector dimensions and component variances in the similarity computation

 Use generalized Minkowski-form distance of metric order one  Feature dimension  Color feature ▪ N C : determined as the total number of quantization levels L  Texture feature ▪ N T : 16Z  N= N C + N T  The number of additions for a query image in the similarity measurement of the retrieval is given as

 Huge Database  Progressively implement ▪ reduce the computational complexity in CBIR  Set of candidate images is selected by feature matching at the lowest level  Progressive refinement is performed as the level increases  Progressive retrieval is composed of Z+1 steps ▪ First step ▪ the color feature vector f C and the texture feature vector f T are combined for m=LL and n=1 ▪ Second step ▪ m={LL,HL,LH,HH} and n=1 ▪ Z+1 step ▪ m={LL,HL,LH,HH} and n={1,….,Z}

 At each step (jth step)  query ▪ the combined feature vector f j q of dimension N j  Target ▪ the combined feature vector f j t of the same dimension ▪ for each of k j-1 target images  k j target images with the best similarity are retrieved  Total number of additions for a query in the similarity measurement of the progressive retrieval  where k j is determined to decrease and the N j increases as the retrieval step j increases

 Image Database  The Corel DB ▪ 990 RGB color images ▪ 192 x 128 pixels ▪ 11 classes, each of 90 images  VisTex DB ▪ 1200 RGB color images ▪ 128 x 128 pixls ▪ 75 classes, each of 16 images  MPEG-7 common color dataset (CCD) ▪ 5420 color images ▪ 332 ground truth sets (GTS) where the number of images in each GTS varies

 Corel MR DB ▪ directly from a third of all the images for each class in the Corel DB ▪ Ratio of (1.5:1) ▪ Ratio of (2:1)  VisTex MR DB ▪ Ratio of (1:1),(1.5:1),(1.75:1),(2:1)  MPEG-7 CCD MR DB ▪ Ratio of (1:1),(1.5:1),(2:1)  The sizes and numbers of classes of the three derived DBs are the same as those of the original DBs  Contain images of various resolutions

 Performance Measures  For a query q A(q) : a set of retrieved images in a DB B(q) : images relevant to the query q  Precision  Recall  ANMRR (average normalized modified retrieval rank) ▪ measure of retrieval accuracy used in almost all of the MPEG-7 color core experiments ▪ The ANMRR gives just one value for a DB ▪ Lower ANMRR value means more accurate retrieval performance

 Specifications of Retrieval Methods  wavelet decomposition level was chosen as Z=2  total number of quantization levels L for each of the wavelet decomposed H and S component images was also chosen as L=30 ▪ {L m,1 } = {8,4,4,4}, {L m,2 } = {4,2,2,2}  Dimensions of feature vector ▪ N C = 60 ▪ N T = 32  Z=2, Proposed progressive retrieval has 3 steps ▪ At the step of j=1,2,3, the total number of quantization levels for each of the wavelet decomposed H and S component images is given as L = 8,20, and 30  feature vector dimensions ▪ (N1,N2,N3) =(20,56,92)

 The proposed method with nonprogressive scheme and that with progressive scheme (a) Corel DB (b) VisTex DB. former yields the average precision loss of 1.5% and that of 1.1% over the latter for Corel DB and for VisTex DB Proposed retrieval method with progressive scheme is about 1.2 times faster than nonprogressive scheme

 Precision versus recall of the proposed method with progressive scheme according to each step (a) Corel DB (b) VisTex DB.

 Single features and the proposed progressive retrieval method  Combination of color and texture features and the proposed progressive retrieval method

 ANMRR of the retrieval methods The proposed method almost always yields better performance in precision versus recall and in ANMRR over the other methods for the six test DBs

 Retrieval ranks of the relevant images for the query image Query image Resolution is identical with the query image The proposed method is more effective for multiresolution image DBs

 The feature vector is scalable according to the decomposition level Z in the wavelet transform domain  It was found in some experiments that the retrieval accuracies of Z>2 are slightly better than those of Z=2  Experimental results for six test DBs showed that the proposed method yielded higher retrieval accuracy than the other conventional methods  It was all the more so for multiresolution image Databases