1. Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed

2. Overview of FindPic What it does How it works (from a user standpoint) Modes – grayscale – quantized color – texture for both

3. Technology Objective-C Cocoa framework Local application Self Organizing Maps – T. Kohonen – 1980s

4. Technique: SOM Overview Gather image data (feature vectors) Create matrix (with same size vectors) For each image find best match in matrix Change matrix node and surrounding nodes Reduce neighborhood size and change factor Re-run for a number of epochs

5. Technique: Image Data Reduction Features – e.g. grayscale histogram Storage

6. Setting up the Map 2D Array of matrix nodes Same feature vectors Initialized to random values Mapping begins

7. Finding the Best Match Euclidean distance between the image (input) vectors and the matrix vectors

8. Technique: Mapping Find best match and change that node

9. Technique: Mapping Change the surrounding nodes

10. Technique: Mapping Change the surrounding nodes

11. Making Changes After best match is found change the matrix node is made to be more like the input node e.g. matrixVector[0] = 50, imageVector[0] = 30, cf = 0.9 matrixVector[0] = 50 – 0.9(50 – 30) = 32 e.g. matrixVector[0] = 30, imageVector[0] = 50, cf = 0.9 matrixVector[0] = 30 – 0.9(30 – 50) = 48

12. Finding the Neighbors Inverse parabola Filters for neighbors Also determines change factor

13. Change Over Time Inverse parabola determines neighborhood and change factor Where M=matrix dimension and t = time

14. Sample Neighborhood Best Matching unit is set to position (1,1) with t = 1 0.8 0.9 0.8 0.5 0.0 -0.7 0.9 1.0 0.9 0.6 0.1 -0.6 0.8 0.9 0.8 0.5 0.0 -0.7 0.5 0.6 0.5 0.2 -0.3 -1.0 0.0 0.1 0.0 -0.3 -0.8 -1.5 -0.7 -0.6 -0.7 -1.0 -1.5 -2.2

15. Sample Neighborhood Best Matching unit is set to position (1,1) with t = 0.3 0.1 0.2 0.1 -0.2 -0.7 -1.4 0.2 0.3 0.2 -0.1 -0.6 -1.3 0.1 0.2 0.1 -0.2 -0.7 -1.4 -0.2 -0.1 -0.2 -0.5 -1.0 -1.7 -0.7 -0.6 -0.7 -1.0 -1.5 -2.2 -1.4 -1.3 -1.4 -1.7 -2.2 -2.9

16. End Results Images with similar feature vectors should be grouped into the same matrix cells. Surrounding cells should also contain images that are similar.

17. Conclusions SOM results compared to human categorized results Determining closeness of match Average percentages – Grayscale: 41.46% – Quantized color: 55.14% – Grayscale with texture: 50.01% – Quantized with texture: 56.03%