1. MPEG-7 DCD Based Relevance Feedback Using Merged Palette Histogram Ka-Man Wong and Lai-Man Po ISIMP 2004 Poly U, Hong Kong Department of Electronic Engineering City University of Hong Kong

2. MPEG-7 Dominant Color Descriptor A compact and effective descriptor Generated by GLA color quantization Maximum of 8 colors in storage Each color have a minimum distance (T d ) of 15 in CIELuv color space

3. Commonly used RF techniques and limitations Feature weighting relevance feedback technique Assumes a fixed feature space (histograms) Taking liner combinations on matching histogram bins. Simple approach: Histogram averaging +() / 2 =

4. Commonly used RF technique and limitations But DCDs of images might have different set of colors, similar images might not have any exactly matched colors. The number of colors in updated query may greatly exceed the limit of the number of colors defined by MPEG-7 Similar colors are separated. By definition of DCD, similar colors should be grouped together. H1H1 H2H2 H’

5. Limitation of feature weighting relevance feedback technique The Merged Palette Histogram Relevance Feedback The updated query contains common colors among selected images Represent the selected images efficiently

6. Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - initialize Obtain all DCD from selected images

7. Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 1 Link all DCD together ++ = 6 colors 8 colors6 colors20 colors

8. Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 2 Palette Merging Find the closest pair of colors based on Euclidian distance in CIELuv If minimum distance smaller than T d merge the color pair and sum up the percentages of merged colors Iterate until minimum distance > T d 20 colors 9 colors

9. Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 3 Approximation Cut the least significant colors if number of colors >8 9 colors8 colors

10. Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 4 Re-normalization Adjust the histogram sum into 1 An updated query is generated Approximated MPHUpdated Query Histogram Sum =1

11. Experimental Results of MPH-RF Experiment Methodology ANMRR Image Database 5466 Image from MPEG-7 common color dataset (CCD) Pre-defined queries and ground truth set Relevance Feedback Ground truth images are selected as relevant images

12. Latest experimental results MPH-RF gives improvement on both similarity measure methods Combination of MPHSM and MPH-RF gives a significant improvement Three iterations of relevance feedback give a significant result *ANMRR – smaller means better InitialAfter 3 RF RF Improvement DCD- MPHSM 0.24750.18380.0637 DCD- QHDM 0.28340.21170.0717

13. Experimental results Visual results – Query #50 from MPEG-7 CCD, MPHSM Visit http://www.ee.cityu.edu.hk/~mirror for more resultshttp://www.ee.cityu.edu.hk/~mirror Query image Ground truth images Initial retrieval, 4 of 8 ground truths hit, NMRR=0.5 First RF retrieval, 6 of 8 ground truths hit, NMRR=0.2782 Second RF retrieval, 7 of 8 ground truths hit, NMRR=0.1541

14. Experimental results Visual results – Query #24 from MPEG-7 CCD, QHDM Visit http://www.ee.cityu.edu.hk/~mirror for more resultshttp://www.ee.cityu.edu.hk/~mirror First RF retrieval, 6 of 12 ground truths hit, NMRR=0.3738 Query image Ground truth images Initial retrieval, 5 of 12 ground truths hit, NMRR=0.5125Second RF retrieval, 9 of 12 ground truths hit, NMRR=0.1963

15. Conclusions on Merged Palette Histogram Relevance Feedback A new MPH-RF for MPEG-7 DCD is proposed MPH-RF generates a new DCD query using palette merging technique Represents the selected relevant images naturally and effectively Experiment result also found that proposed method improve DCD-QHDM by 0.0717 and MPHSM by 0.0637 using MPEG-7 Common Color Dataset The proposed method also provide better perceptually relevant image retrieval.