1. Traditional Database Indexing Techniques for Video Database Indexing Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC 28223 jfan@uncc.edu http://www.cs.uncc.edu/~jfan

2. 1. Why we need indexing? Library: 2000000 books Find the book with title “Multimedia Systems, Standards, and Networks” without indexing! Query: Too hard! 2000000! How we can do this more efficiently?

3. 2. How Library Works? a. Classify these books into several subjects: I get it! Too easy! 11! Books in Library Natural SciencesSocial Sciences Dancing Computer Science Electrical Engineering Computer Languages Researches DatabaseMultimedia

4. 2. How Library Works? b. How they get this good partition and management? Taxonomy !!& Library Science!! Natural Sciences Social Science How we can do this for data & image?

5. 3. Key Problems for Building Indexing? What you can find from this map?

6. 3. Key Problems for Building Indexing? What you can find from this map?

7. 3. Key Problems for Building Indexing?

8. What you can find from this map? 3. Key Problems for Building Indexing?

9. a. Partition b. Representation Database is some tables! Map is similar as tables! Partition the large-scale data set into meaningful & manageable small regions hierarchically! Represent these regions using efficient technique so that they can access very fast!

10. 4. How to build indexing structure for data? a. Space partition approach: Partition the space into regions according to some measure

11. 4. How to build indexing structure for data? a. Space partition approach: Space partition tree is attractive for GIS system.

12. 4. How to build indexing structure for data? Space partition may not work for some case!!

13. 4. How to build indexing structure for data? Partition data based on data distributions!

14. clustering Using clustering to partition data set!! 4. How to build indexing structure for data? b. Data Partition via Clustering

16. 4. How to build indexing structure for data? b. Data Partition via Clustering K-mean data clustering (1) Select K center to start Dark points

17. 4. How to build indexing structure for data? b. Data Partition via Clustering K-mean data clustering (2) Put the testing point into most similar center

18. 4. How to build indexing structure for data? b. Data Partition via Clustering K-mean data clustering (3) Update the corresponding cluster center

19. c. Representation of Data Partition Results: (1) Rectangular box (ID, x 1,y 1,x 2,y 2 ) (2) Sphere (ID, x c,y c, R) (SR-tree)

20. 4. How to build indexing structure for data? A B C D E FG H I J K L M N A B C D E F G H L M N I J K LMN DEFGH IJK Data set Indexing tree Search road R-tree: Minimum Rectangular Box

21. A B C D A B C D ABCD First partition 4. How to build indexing structure for data? R-tree: Minimum Rectangular Box

22. A ab c de f g A B C D ABCD abcdef g Second partition of A 4. How to build indexing structure for data? Data partition approach:

23. 4. How to build indexing structure for data? Data partition approach: B Second partition of B A B C D ABCD h i j k hijk

24. 4. How to build indexing structure for data? Data partition approach: C l m A B C D ABCD l mSecond partition of C

25. 4. How to build indexing structure for data? A B C D BCDA abcdef g lm hijk Final indexing structure Data partition approach:

26. R-tree family A B C F G H D E Root Node ABC DEFGH 4. How to build indexing structure for data?

27. R-tree family A B C F G H D E Root Node ABC DEFGH a. Overlap between A and C! 4. How to build indexing structure for data?

28. X-tree: Minimum Rectangular Box with Fat Node root Normal directory nodes Super-nodes Data nodes

29. 4. How to build indexing structure for data? SR-tree: Minimum Sphere

30. Grid file can be treated as an extended Q-tree with multiple partition at each attribute! salary age 4. How to build indexing structure for data?

31. Grid file can be treated as an extended Q-tree with multiple partition at each attribute! buckets 4. How to build indexing structure for data?

32. primary buckets overflow bucket 4. How to build indexing structure for data?

33. a.Equal query: 1 + M b.Range query: N + N*M c.Insert: 1 + M + 1 d.Delete: 1 + M + 1 Bucket numbers: N; overflow bucket: M; Number of data entries for leaf node: K 4. How to build indexing structure for data?

34. Data distribution information can be used to improve the performance of grid file. salary age 4. How to build indexing structure for data? Dynamic Grid File

35. salary age bucket 4. How to build indexing structure for data?

36. 20* 00 01 10 11 2 2 2 2 LOCAL DEPTH 2 2 DIRECTORY GLOBAL DEPTH Bucket A Bucket B Bucket C Bucket D Bucket A2 (`split image' of Bucket A) 1* 5*21*13* 32* 16* 10* 15*7*19* 4*12* 19* 2 2 2 000 001 010 011 100 101 110 111 3 3 3 DIRECTORY Bucket A Bucket B Bucket C Bucket D Bucket A2 (`split image' of Bucket A) 32* 1*5*21*13* 16* 10* 15* 7* 4* 20* 12* LOCAL DEPTH GLOBAL DEPTH 4. How to build indexing structure for data?

37. a.Equal query: 1 + M b.Range query: N + N*M c.Insert: 1 + M + 1 d.Delete: 1 + M + 1 Bucket numbers: N; overflow bucket: M; Number of data entries for leaf node: K 4. How to build indexing structure for data?

38. Database indexing structure is built for decision making and tries to make the decision as fast as possible! Color = Green? Size = Big? watermelon Size = Medium? apple Grape Color = Yellow? Shape = Round? Size = Big? banana grapefruitlemon Size = small? Taste = sweet? cherrygrape apple yes no yes no yesno yes no yesno yesno yesno 4. How to build indexing structure for data? Decision Tree

39. How to obtain decision for a database? a.Obtain a set of labeled training data set from the database. b.Calculate the entropy impurity: c. Classifier is built by: 4. How to build indexing structure for data?

40. KD-tree By treating query as a decision making procedure, we can use decision to build more effective database indexing! Database root node Salary > $75000? yesno Data table Age > 60? yes no Age > 60? yesno 4. How to build indexing structure for data?

41. Each inter-node, only one attribute is used! It is not balance! Search from different node may have different I/O cost! It can support multiple attribute database indexing like R-tree! It has integrated decision making and database query! 4. How to build indexing structure for data?

42. a.Equal query: N + M b.Range query: N + M c.Insert: N + M + 1 d.Delete: N+ M + 1 Tree levels: N; Leaf nodes: M; Number of data entries for leaf node: K The inter-nodes for kd-tree at the same level are stored on the same page. 4. How to build indexing structure for data?

43. 5. Storage Management for High-Dimensional Indexing Structures Index entries Data entries direct search for (Index File) (Data file) Data Records data entries Data entries Data Records CLUSTERED UNCLUSTERED Index entries Data entries direct search for (Index File) (Data file) Data Records data entries Data entries Data Records CLUSTERED UNCLUSTERED We want to put the similar data in the same page or neighboring pages!

44. 5. Storage Management for High-Dimensional Indexing Structures It is very hard to do multi-dimensional data sorting! 00 01 10 11 Hilbert Curve: scale multi-dimensional data into one dimension.

45. 5. Storage Management for High-Dimensional Indexing Structures 0 1 2 3 4 5 6 7 8 9 10 11 1213 1415 From multi-dimensional indexing to one-dimensional storage in disk!

46. 6. Video Database Indexing Can these technique be used for video database indexing? a. Curse of Dimensions: overlap in high-dimensional space b. Semantic Gap: visual features == semantic concepts What we should do?

47. Color HSV color histogram, dominant color, … Texture Edge histogram, wavelet coefficients, Tamura features, … Motion Directional motion histogram, Camera motion, … Other features Video Sequence Shot 1 Shot i Shot n Visual Representation Schema Determination

48. Color HSV color histogram, dominant color, … Texture Edge histogram, Tamura, …. Shape Rectangular box, moments, ….. Motion Trajectory, motion histogram, … Other features Video Sequence Key Object 1 Key Object i Key Object n Schema Determination

49. 6. Video Database Indexing A B C overlap curse of dimensions

50. 6. Video Database Indexing a. Concept Hierarchy We should try to bridge the semantic gap in the video content partition procedure. Objective: 2000 Olympic Games filed basketball softballsoccer volleyball Team USA Team Norway Team Slovakia Team USA PlayersNews Game Actions PlayersNews Game Actions

51. 6. Video Database Indexing.......... Visual Features...... Semantic Clusters................. Video Contents in Database Weighted mapping? b. Semantic classification

52. 6. Video Database Indexing Video in Database Cluster 1Cluster iCluster n Subcluster 11Subcluster 1j Subcluster n1 Subcluster nl Subregion 11k Subregion nl1Subregion nlm object1111 object nlm1 Disk for Cluster 1 Disk for Cluster i Disk for Cluster n

53. 7. Video Query with Indexing query object feature extraction Cluster 1 Cluster i Cluster n Subcluster i1Subcluster ij Subcluster im Subregion ij1Subregion ijl Subregion ijr Object ijrm Disk for cluster 1Disk for cluster i

58. Video Browsing

62. A* Search Algorithm Video in Database Cluster 1Cluster iCluster n Subcluster 11Subcluster 1j Subcluster n1 Subcluster nl Subregion 11k Subregion nl1Subregion nlm object1111 object nlm1 Disk for Cluster 1 Disk for Cluster i Disk for Cluster n

63. Multimedia Database System Design Access control & rights management Query & Delivery Delivery Query Presentation Query Processing Visual Summarization Indexing Video CollectionsMPEG Encoder Indexing is very important!