Presentation is loading. Please wait.

Presentation is loading. Please wait.

Relevance Feedback for the Earth Mover‘s Distance / 21 I9 CHAIR OF COMPUTER SCIENCE 9 DATA MANAGEMENT AND EXPLORATION Relevance Feedback for the Earth.

Similar presentations


Presentation on theme: "Relevance Feedback for the Earth Mover‘s Distance / 21 I9 CHAIR OF COMPUTER SCIENCE 9 DATA MANAGEMENT AND EXPLORATION Relevance Feedback for the Earth."— Presentation transcript:

1 Relevance Feedback for the Earth Mover‘s Distance / 21 I9 CHAIR OF COMPUTER SCIENCE 9 DATA MANAGEMENT AND EXPLORATION Relevance Feedback for the Earth Mover‘s Distance Marc Wichterich, Christian Beecks, Martin Sundermeyer, Thomas Seidl Data Management and Data Exploration Group RWTH Aachen University, Germany

2 Relevance Feedback for the Earth Mover‘s Distance / 21 Introduction  Distance-based Adaptable Similarity Search  Similarity of objects defined by distance function  Small distance → similar, large distance → dissimilar  Query by example: user-given object, find similar ones  Query and distance only approximate descriptions of user’s desired result  If delivered result does not meet expectations:  Bad query? Bad distance? Bad database?  How to do it better?  Relevance Feedback attempts to adapt query/similarity model based on simple user input (result relevancy) 1

3 Relevance Feedback for the Earth Mover‘s Distance / 21 2 Relevance Feedback user DB query, feedback results feedback system similarity model Photo: Flickr / Caro WallisCaro Wallis Earth Mover’s Distance RF for EMD RF EMD

4 Relevance Feedback for the Earth Mover‘s Distance / 21 Overview  Introduction  Adaptive Similarity Model  Feature Signatures  The Earth Mover’s Distance  Relevance Feedback for the Earth Mover’s Distance  Experimental Evaluation  Conclusion 3

5 Relevance Feedback for the Earth Mover‘s Distance / 21 Similarity Model – Feature Signatures 4 x y color

6 Relevance Feedback for the Earth Mover‘s Distance / 21 Similarity Model – Earth Mover’s Distance  Introduced in Computer Vision by Rubner et al.  Used in many differing application domains  Idea: transform features of Q into features of P  EMD: minimum of transformation cost 5 Q P x y x y

7 Relevance Feedback for the Earth Mover‘s Distance / 21 Feature Transformation 6

8 Relevance Feedback for the Earth Mover‘s Distance / 21 EMD – Formal Definition  Modeled as linear optimization (transportation problem) 7

9 Relevance Feedback for the Earth Mover‘s Distance / 21 Overview  Introduction  Adaptive Similarity Model  Relevance Feedback for the Earth Mover’s Distance  The Feedback Loop  Query Adaptation  Heuristic EMD Adaptation  Optimization-based EMD Adaptation  Experimental Evaluation  Conclusion 8

10 Relevance Feedback for the Earth Mover‘s Distance / 21 The Feedback Loop 9 user DB query, feedback results feedback system similarity model yes exit start get query adapt distance no get feedback adapt query retrieve results display results satisfied?

11 Relevance Feedback for the Earth Mover‘s Distance / 21 Query Adaptation  Input: signatures from relevant objects  Output: new query signature  Idea: cluster signature elements  Refinements by Rubner:  Only keep clusters with elements from majority of signatures  Reweight resulting signature accordingly  Combine with fixed gd L2 and call it „Query-by-Refinement“  „Query-by-Refinement“ is baseline for our evaluation  We adapt EMD via ground distance 10 satisfie d? retrieve results exit start get query query distance feedback display results

12 Relevance Feedback for the Earth Mover‘s Distance / 21 Heuristic EMD Adaptation 1  Approach: pick gd based on feedback  gd should reflect user preferences:  Don’t care if blue cluster at upper half of image is moved left/right  Do care if it is moved vertically  Use variance information in relevant feedback  Low variance → assume user cares  High variance → assume user does not care  Measure variance in feedback locally around query signature elements c i (Q).  Define gd: c (Q) x FS → R ( ) 11 satisfie d? retrieve results exit start get query query distance feedback display results

13 Relevance Feedback for the Earth Mover‘s Distance / 21 Heuristic EMD Adaptation 2  Not 1 but m distance functions:  gd i (c i (Q),y) = ((c i (Q) - y) V i (c i (Q) - y) T ) ½  Weighted Euclidean Distances (weights on diagonal of V i )  V i : inverted variance for c i (Q) per feature space dimension 12 satisfie d? retrieve results exit start get query query distance feedback display results

14 Relevance Feedback for the Earth Mover‘s Distance / 21 Optimization-Based EMD Adaptation 1  Aim: Pick best possible gd.  Failback: Find a good one.  Q: When is gd good? A: If ranking it produces is good.  New Q: When is a ranking of DB good?  Given ground truth, a number of measures exist  We used “average precision at relevant positions”  We have ground truth for part of the DB: feedback  Idea: test candidates for gd on feedback 13 satisfie d? retrieve results exit start get query query distance feedback display results RankingAvg. Precision

15 Relevance Feedback for the Earth Mover‘s Distance / 21 Optimization-Based EMD Adaptation 2  Optimization:  Optimization variable: gd  Objective function: avgPrec(EMD gd, q, Feedback)  Constraints: m weighted Euclidean distances  Analytic optimization with closed form for weights infeasible (ranking/sorting, EMDs in objective function)  Probabilistic optimization via Simulated Annealing  Start with some initial solution  Move in solution space  Compute objective function  Adopt solution with certain probability  Iterate & turn more greedy 14 satisfie d? retrieve results exit start get query query distance feedback display results

16 Relevance Feedback for the Earth Mover‘s Distance / 21 Optimization-Based EMD Adaptation 3  Optimization for EMD based on Feedback:  Solution: weights for m weighted Euclidean distances  Initial solution: given by heuristic  Moving: redistribute weights per Euclidean distance  Objective function: avgPrec(EMD gd, q, Feedback)  Results for EMD gd on DB? 15 satisfie d? retrieve results exit start get query query distance feedback display results

17 Relevance Feedback for the Earth Mover‘s Distance / 21 Overview  Introduction  Adaptive Similarity Model  Relevance Feedback for the Earth Mover’s Distance  Experimental Evaluation  Conclusion 16

18 Relevance Feedback for the Earth Mover‘s Distance / 21 Experimental Evaluation: Databases 17 72,000 images in ALOI DB~60,000 images in COREL DB

19 Relevance Feedback for the Earth Mover‘s Distance / 21 Experimental Evaluation: ALOI 18 Heuristic Adaptation Optimization-based Query-by-Refinement

20 Relevance Feedback for the Earth Mover‘s Distance / 21 Experimental Evaluation: COREL 19 Heuristic Adaptation Optimization-based Query-by-Refinement

21 Relevance Feedback for the Earth Mover‘s Distance / 21 Experimental Evaluation 20  After 5 iterations of looking for doors in COREL: (a) Query-by-Refinement (b) Heuristic (c) Optimization-Based pos

22 Relevance Feedback for the Earth Mover‘s Distance / 21 Conclusion  Exploited adaptability of the EMD in RF framework  Goal: Improve similarity search results  Techniques:  Baseline: fixed ground distance  Statistics-based heuristic adaptation  Optimization-based adaptation  Evaluation:  Experiments on two image datasets  More relevant objects in fewer iterations  Techniques extensible to other adaptable distance functions 21


Download ppt "Relevance Feedback for the Earth Mover‘s Distance / 21 I9 CHAIR OF COMPUTER SCIENCE 9 DATA MANAGEMENT AND EXPLORATION Relevance Feedback for the Earth."

Similar presentations


Ads by Google