1. NCKU CSIE Visualization & Layout for Image Libraries Baback Moghaddam, Qi Tian IEEE Int’l Conf. on CVPR 2001 Speaker: 蘇琬婷

2. NCKU CSIE Outline System Introduction Visualization and Layout Optimization Context and User Modeling Discussion

3. NCKU CSIE System Introduction-PDH Personal Digital Historian (PDH) Interface Design : Polar coordinate visual layout circular display area touch sensitive table surface top projection table with a whiteboard as the table surface

4. NCKU CSIE PDH Table

5. NCKU CSIE 4W’s: The Organization Principal Who WhatWhereWhen

6. NCKU CSIE

7. Content-based Visualization Contend-based Image Retrieval(CBIR) Images would be indexed by their visual contents Feature(content) extraction Visualization Traditional interfaces PCA Splats Display optimization

8. NCKU CSIE Traditional Systems Visualization Simple 1-D list Sorted by decreasing similarity to the query Drawback Relevant images can appear at separate and distant locations in the list Improvement 2-D display technique

9. NCKU CSIE Top 20 Retrieved Images Ranked top to bottom and left to right

10. NCKU CSIE PCA Splats Principal component analysis(PCA) project the images from the high-dimensional feature space to the 2-D screen 37 visual features(color, texture, structure) on the basis of the first two principal components normalized by the respective eigenvalues The maximum distance preservation from the original high-dimensional feature space to 2-D space

11. NCKU CSIE Display Optimization The drawback of PCA splat images are partially or totally overlapped Optimization Minimizing overlap (decreasing the overlap of the images) Minimizing deviation (deviating as little as possible from their initial PCA splat positions) Minimizing the total cost

12. NCKU CSIE Cost Function F(p) : cost function of the overall overlap G(p) :cost function of the overall deviation from the initial image positions S : scaling factor and S = (N-1)/2 N : the number of images λ: weight and λ ≧ 0

13. NCKU CSIE Minimizing Overlap r i : image size is represented by its radius,i = 1,…,N (x i, y i ) : image center coordinates u : measure of overlapping σ f : curvature-controlling factor range of F(p): (N-1)+(N-2)+…+1 = N(N-1)/2 i j

14. NCKU CSIE Minimizing Deviation ： the optimized and initial center coordinates of the i th image, respectively v : measure of deviation σ g : curvature-controlling factor range of G(p) : N range of F(p) : N(N-1)/2 ∴ S = (N-1)/2

15. NCKU CSIE Optimized PCA Splat

16. NCKU CSIE Context and User Modeling Image content and “meaning” is ultimately based on semantics user’s notion of content : high-level concept visual features : low-level concept

17. NCKU CSIE

18. Context and User Modeling User modeling or “context awareness” constantly be aware of and adapting to the changing concepts and preferences of the users learn from a user-generated layout a novel feature weight estimation scheme : α-estimation α: weighting vector for feature (color, texture, structure) α = (α c, α t, α s ) T α c,t,s : the weight for color, texture, structure α c + α t + α s = 1

19. NCKU CSIE Estimation of Feature Weights X c, t, s : L c, t, s × N matrix where the i th column is the color, text ure, structure feature vector of the i th image, i = 1,…,N L c, t, s : the lengths of color, texture, structure features d ij : the distance Euclidean-based between the i th image and the j th image minimizing with an Lp norm (with p = 2) non-negative least squares solutions

20. NCKU CSIE an example of a user-guided layout αc = 0.3792 αt = 0.5269 αs = 0.1002

21. NCKU CSIE PCA splat on larger set of images estimated weight randomly generated weight

22. NCKU CSIE User Modeling for Automatic Layout user-guided layout computer layout

23. NCKU CSIE Future Work Having the system learn the feature weights from various sample layouts provided by the user Incorporate visual features with semantic labels for both retrieval and layout Incorporation of relevance feedback Automatic “summarization” and display of large image collections