1 Statistical correlation analysis in image retrieval Reporter : Erica Li 2004/9/30

2 Previous Remark : CBIR, QVM, FRE Content-Based image retrieval Query vector modification Feature relevance estimation

3 Q f1 f2 Q f1 f2 QVM

4 FRE f1 f2 Q w1 = w2 f1 f2 Q w1 > w2

5 1. Introduction This paper propose a statistical correlation model that is able to accumulate and memorize the semantic knowledge learnt from the relevance feedback information of previous queries. This model captures the semantic relationships among images in a database from simple statistics of user-provided relevance feedback information. It is applied in the post-processing of image retrieval results such that more semantically related images are returned to the user.

6 2. Definition of statistical correlation model (1/2) Assumption : two images represent similar semantics if they are jointly labeled as relevant to the same query in a relevance feedback phase. Bigram frequency : the number of times that two images are co-relevant. Unigram frequency : an image is relevant. Maximum frequency : the maximum value of all unigram and bigram frequencies.

7 2. Definition of statistical correlation model (2/2) The correlation strength is within the interval between 0 and 1. I, J : two images. M : maximum frequency. B(I,J) : their bigram frequency U(I) : unigram frequency of image I. R(I,J) : semantic correlation strength between image I and J.

8 3. Training algorithms (1/3)

9 3. Training algorithms (2/3)

10 3. Training algorithms (3/3)

11 4. Image ranking schemes (1/2)

12 4. Image ranking schemes (2/2)

13 5. Experiments (1/3) Image Database : images. Source : 2000 websites. After internal use for months, about 3000 queries with relevance feedbacks were collected. Two experiments were conducted to evaluate the proposed method: text-based image retrieval pure content-based retrieval.

14 5. Experiments (2/3) Each one of them was required to search for images with every query twice and label all relevant and irrelevant images within the top 200 results returned by the system, according to his/her own subjective judgment. The number of iterations : 5 (k = 5) The number of images to propagate relevance scores is set to 30 (m = 30).

15 Fig. 1. Precision vs. scope curve for content-based image retrieval.

16 Fig. 2. Precision vs. scope curve for keyword-based image retrieval

17 5. Experiments (3/3) For CBIR, the precision is improved from 10% to 41% for top 10 images, while from 4.6% to 18.5% for top 100 images.