1 Embedded colour image coding for content-based retrieval Source: Journal of Visual Communication and Image Representation, Vol. 15, Issue 4, December 2004, pp Author: Guoping Qiu Speaker: Chia-Yi Chuang Date: 2005/03/22

2 Outline Introduction Integrating SBIC, CPAM and VQ Segmentation-based image coding Colored pattern appearance model and vector quantization Statistics Experimental results Conclusions

3 Introduction Image feature extraction Image Database Similar Images Similarity matching Image Query Image Image feature extraction

4 Flow chart Original image I asp,I csp Image Database P asp,P csp 1. SBICCPAM 3. StatisticsStore VQ SS,ASP,CSP 2. CPAM and VQ Image segmented blocks

5 1. SBIC (1/4) Segmentation-Based Image Coding It is often classified as 2nd generation image coding. Idea ： classify image regions into different classes. allocate different number of bits to different regions according to the properties of the region. Restrictions Shapes of the regions must be square Maximum size is N×N Pixels have similar colors in each region

6 1. SBIC (2/4) A constrained adaptive segmentation algorithm (CASA)

7 1. SBIC (3/4)

8 1. SBIC (4/4) Original ImageEL = 10EL = 20 B min =1, B max =16, B step = … … ……… Calculate ：

9 2. CPAM and VQ  Coloured pattern appearance model and vector quantization

CPAM(1/2) Coloured Pattern Appearance Model It is defined as the spatial and spectral characteristics of a (small) block of pixels. A colored image pattern is modeled by three components: the stimulus strength (SS). the achromatic spatial pattern (ASP). the chromatic spatial pattern (CSP). By separating achromatic and chromatic signals, it is possible to work on two low-dimensional vectors rather than one very high-dimensional vector.

11 (121,235,99)(88,1,53)(250,94,1)(82,41,29) (154,198,166)(221,148,69)(20,247,92)(198,23,6) (1,146,195)(164,3,51)(59,61,137)(184,264,21) (257,27,35)(84,62,129)(204,39,47)(91,183,172) Mean = 115 CbCb Y CrCr Asp Csp Y = 0.299R G B Cb = R – 0.331G B Cr = 0.500R – 0.419G – 0.081B 2.1 CPAM(2/2)

Vector Quantization Encoder Map k-dimensional vector x to index i Decoder Map index i to the reproduction vector Codebook Original image Coded image … … … Reconstructed image

13 3. Integrating SBIC, CPAM, and VQ for colour image coding and indexing Since the segmentation is based on the homogeneity of the region, a segmented larger block and a segmented smaller block will roughly have the same level of homogeneity. VQ coding of variable size patterns Construction of image descriptors from SBIC/CPAM/VQ stream

VQ coding of variable size patterns To design one set of codebook at an intermediate block size and which will be used by all the block sizes. Bc - the size of the CPAM pattern (the block size of the codebook) ； Bs - the block size of a segmented block. If Bs < Bc then up-sample Bs, to Bc using bilinear interpolation. If Bs > Bc then subsample Bs to Bc using bilinear interpolation.

Construction of image descriptors from SBIC/CPAM/VQ stream Let P asp (i,j) be the probability of a block of size i and whose ASP vector is encoded by the vector quantizer to the jth codeword of VQ asp. Let P csp (k,l) be the probability of a block of size k and whose CSP vector is encoded into the lth codeword of VQ csp.

16 4. Statistics Image (2000 blocks) I asp (1,0)=15, P asp (1,0)=0.0075I csp (1,0)=20, P csp (1,0)=0.001 I asp (1,1)=30, P asp (1,1)=0.015I csp (1,1)=25, P csp (1,1)= I asp (1,255)=200, P asp (1,255)=0.1I csp (1,255)=60, P csp (1,255)=0.03 I asp (2,0)=150, P asp (2,0)=0.075I csp (2,0)=80, P csp (2,0)=320=0.04 I asp (16,254)=40, P asp (16,254)=0.02I csp (16,254)=75, P csp (16,254)= I asp (16,255)=100, P asp (16,255)=0.05I csp (16,255)=50, P csp (16,255)=0.025 …… …… 4096

17 Similarity measurement Image A : P Aasp (i,j) 、 P Acsp (k,l) Image B : P Basp (i,j) 、 P Bcsp (k,l) The similarity between A and B can be measured by the following distance: where and are relative weights given to the chromatic and achromatic pattern features.

18 Experimental results (1/5) Set ASet B Examples of query image pairs. For each image in set A, there is a corresponding (similar but different) target image in set B, or vice versa.

19 Experimental results (2/5) The trend seemed to be that the higher the error limit, the lower the average ranks of the returned target images. But, if the error limit is too high, the opposite is true. Therefore, EL=7 tends to give very satisfactory image quality and reasonable retrieval performance.

20 Experimental results (3/5) EL=7, B min =4, B max =10, and B step =2  CC and MPEG7 CS had more queries found the target image at lower ranks (better performance).  Both methods had queries which returned the targets at a much higher ranks (worse performance). colour correlogram (cc) ； MPEG7 colour structure (MPEG7 cs)

21 Experimental results (4/5) EL=7, B min =4, B max =10, and B step =2 the average ranking of the new method is much lower (better performance).

22 Experimental results (5/5) The image on the upper left corner is the query, the rest are the returned images arranged in terms of similarity in a canonical order.

23 Conclusions This is a color image coding and indexing method which integrates SBIC, CPAM and VQ. Our objectives are twofolds, i.e., compression and easy content access. The proposed method has at least comparable performances to state of the art methods,such as colour correlogram(cc) and the latest MPEG7 colour structure(MPEG7 cs) descriptor in content-based image retrieval.