Presentation in IJCNN 2004 Biased Support Vector Machine for Relevance Feedback in Image Retrieval Hoi, Chu-Hong Steven Department of Computer Science & Engineering The Chinese University of Hong Kong Shatin, Hong Kong Budapest, 25-29 July, 2004

Outline Background & Motivation Biased Support Vector Machine (Biased SVM) Relevance Feedback by Biased SVM Experimental Results Conclusions This is the outline of my presentation. First, I will briefly give the background & motivation of our work. Then, I will present and formulate a modified SVM algorithm, Biased SVM, for attacking the imbalance problem. Followed by the formulation, a relevance feedback scheme by Biased SVM is proposed. After that, experimental results will be presented and conclusions are given at the end.

Background Challenges in Content-based Image Retrieval (CBIR) Semantic gap, low-level features, high-level concepts Subjectivity of human being, … Relevance Feedback (RF) Refine retrieval results by incorporating users’ interactions A technique to narrow down the semantic gap, subjectivity Methods: heuristic weighting [Rui98, MARS99], optimization [MindReader98, Rui00], classification [MacArthur99], other learning techniques [Huang01], … Popular method proposed recently: Support Vector Machines (SVM) [Hong00, Chen01, Tong01, Zhang01] Content-based Image retrieval has been an important topic in visual information retrieval for a long days. However, some problems in CBIR are still challenging, include the semantic gap between low-level features and high-level honcepts, and also the subjectivity of human being. To overcome the challenges, relevance feedback has been proposed as a powerful technique to narrow down the semantic gap. There are many techniques proposed for relevance feedback including heuristic weighting schemes, optimization framework, classification, and many other machine learning techniques. One of the most popular technique engaged for relevance feedback recently is the Support Vector machine, which has excellent performance in many pattern recognition applications.

Motivation A case of regular SVM Imbalance dataset problem? Optimal separating hyperplane margin Here, we briefly show the motivation of our work. We first briefly introduce the basic idea of SVM. In regular binary SVM, there are two classes: positive and negative. The algorithm of SVM is to find the optimal separating hyperplane for separating the data with a maximum margin. There is no imbalance consideration in regular SVM. However, in real-world relevance feedback problem, the number of negative samples is often large than the positive. Positive samples will be overwhelmed by negative ones. The imbalance problem will degrade the learning performance. (Positives clustered, negative scattered) A case of regular SVM Imbalance dataset problem? # negative >> # positive Positive overwhelmed by negative

Motivation Limitation of regular SVMs for RF Our solution: Biased SVM Regular binary SVM Simply treat as a strict binary classification problem without imbalance consideration Regular 1-SVM Exclude negative information Our solution: Biased SVM A modified 1-SVM incorporating negative information with bias control From the above examples, we can see the limitations of regular SVMs techniques for relevance feedback as follows: Regular binary SVM simply regards the relevance feedback problem as a strict binary classification problem without imbalance consideration Regular one class SVM do not consider the negative information. (Do not notice the imbalance problem) To attack the challenge, we proposed a modified one class SVM method incorporating the negative information with bias control.

Biased SVM Problem formulation Training data: The objective function c Now we formulate the Biased support vector machine technique. Suppose we are given the training data: Here, x_i is vector, y_i the label of class, l is the number of training samples, The goal of Biased SVM learning is to find the optimal separating hypersphere which best describes the data. The objective function of Biased SVM is given as follow. Here, we introduce the bias control parameter – b – for imbalance consideration. C here is the centroid of the hypersphere, R here is the radius of the hyersphere ball. R

Biased SVM (cont.) Optimization by Lagrange multipliers Take the partial derivatives of L with respect to R,ξ,c, andρ: In order to solve the optimization problem, we introduce the Lagrange multipliers, We take the partial derivatives of L with respect to the parameters as follows:

Biased SVM (cont.) Dual problem (Quadratic Programming (QP) ) The decision function f (x)<0 f (x)>=0 Then, we can obtain the dual problem. This can be solved by typical quadratic programming technique. After solving the problem, we can acquire the decision function.

Relevance Feedback by Biased SVM One of differences with regular SVM Visual comparison Biased SVM: Regular SVM: Here, we show some differences between Biased SVM and regular SVM. One difference can be seen from the dual optimization function, in Biased SVM, we have a bias factor- b – to estimate the bias between the positive class and negative class, while in regular SVM, there is no imbalance consideration. One the other hand, we can also see the difference from visual comparison: e.g. in the figure, we can see that Biased SVM can better describe the data by incorporating the negative information.

Relevance Feedback by Biased SVM Obtained decision function Simplified evaluation function To formulate the relevance feedback algorithm with Biased SVM, we can derive the simplified evaluation function from original decision function. Here, the evaluation function are obtained by removing the constant values from the original decision function.

Experimental Results Datasets One synthetic dataset: 40-Cat, each contains 100 data points randomly generated by 7 Gaussian in a 40-dimensional space. Two real-world image datasets selected from COREL image CDs 20-Cat: 2,000 images 50-Cat: 5,000 images In the experiments, we evaluate the algorithms on both the synthetic dataset and real-world image datasets. One synthetic dataset: 40-Cat, each contains 100 data points randomly generated by 7 Gaussian in a 40-dimensional space. We pick two real-world datasets by selecting the images from COREL image CDs. One is 20 category dataset containing 2000 images, another is 50 category. Each category has a different semantic meaning. Means and covariance matrices of the Gaussians in each category are randomly generated in the range of [0,10].

Experimental Results (cont.) Image Representation Color Moment 9-dimension Edge Direction Histogram 18-dimension Canny detector 18 bins, each of 20 degrees Wavelet-based texture Daubechies-4 wavelet, 3-level DWT 9 subimages to generate the feature Image representation is an important step for CBIR. Three kinds of features: color, shape, and texture are engaged . For color feature, we use 9-dimensional color moment. For shape, we use 18-dimesion edge direction histogram in which Canny detector is used for edge detection and histogram are divided in 18bins each with 20degrees. For texture, we use 9-dimension wavelet texture, in which DB-4wavlet is used and 3-leve DWT decmposition is performed. 9 subimages are selected to describe the texture.

Experimental Results (cont.) Compared Schemes Relevance Feedback by regular nu-SVM Relevance Feedback with 1-SVM Relevance Feedback with Biased SVM Experimental Setup Metric: Average precision = #relevant / #returned Pick 10 instances, label pos. or neg. First iteration, 2 pos. and 8 neg. Same kernel and settings for compared schemes 200 relevance feedback simulation rounds are executed for each compare scheme. We compare there relevance feedback scheme as follows: The first case is by regular binary SVM (here nu-SVM is engaged) Second scheme is the regular 1-SVM technique. The third one is our Biased SVM. In the experimental evaluation, we employ average precision as the metric. And in each relevance feedback iteration, 10 image instances are picked for lablelling In first iteration, 2 positive and 8 negative samples are given. To enable objective evaluation, kernel and settings are same for the compared scheme.

Experimental Results (cont.) This shows the experimental results of systhetic dataset and real-world image datasets. We can see our proposed method outperform than regular approaches. And it is interested that better results are obtained in the real-world cases. We also see that one-class SVM can perform well initially, but degrade later without including negative information. Synthetic dataset 20-Cat COREL Images

Experimental Results (cont.) In the 50-category dataset, we also see the similar results. Our method have better performance than other two schemes. 50-Cat COREL Images

Experimental Results (cont.) This table show the average precision of three schemes after 10 feedback iterations. From the results, we can Biased SVM outperform than other two schemes both on the 20-cat and 50-cat datasets.

Conclusions Address the imbalance problem of relevance feedback in CBIR. Propose a modified SVM technique, i.e. Biased SVM, to attack the imbalance problem of relevance feedback problem in CBIR. Demonstrate effectiveness of the proposed scheme from experiments. In this presentation, we addressed the imbalance problem of relevance feedback in CBIR. We proposed a modified SVM technique, Biased SVM, to attack the imbalance problem in relevance feedback. We demonstrated the effectiveness of our proposed scheme from convinced CBIR experiments.

Budapest, Hungary, July, 2004 Thank you!

References [Rui98] Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra, “Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval”, IEEE Tran Circuits and Systems for Video Technology, Vol 8 No 5, 1998, 644-655 [MARS99] K. Porkaew, S. Mehrotra, and M. Ortega, “Query Reformulation for Content Based Multimedia Retrieval in MARS”, IEEE Int’l Conf. Multimedia Computing and Systems (ICMCS’99), June, 1999 [MindReader98] Y. Ishikawa, R. Subramanya, and C. Faloutsos, “MindReader: Query databases through multiple examples”, 24th VLDB Conf. (New York), 1998 [Zhang01] L. Zhang, F. Lin, and B. Zhang, “SUPPORT VECTOR MACHINE LEARNING FOR IMAGE RETRIEVAL”, ICIP’2001, 2001 [Rui00] Y. Rui, T. S. Huang, “Optimizing learning in image retrieval”, CVPR’00, Hilton Head Island, SC, June 2000 [MacArthur99] S. MacArthur, C. Brodley, and C. Shyu, “Relevance Feedback Decision Trees in Content-Based Image Retrieval,” workshop CBAIVL, CVPR’00, June 12, 2000. [Tong01] S. Tong, and E. Chang, “Support vector machine active learning for image retrieval”, ACM MM’2001, 2001 [Chen01] Y. Chen, X. S. Zhou, T. S. Huang, “One-class SVM for Learning in Image Retrieval”, ICIP'2001, Thessaloniki, Greece, October 7-10, 2001 [Hong00] P. Hong, Q. Tian, T. S. Huang, "Incorporate Support Vector Machines to Content-Based Image Retrieval with Relevance Feedback", ICIP'2000, Vancouver, Sep 10-13, 2000.