1. Introduction to Indexes Rui Zhang http://www.csse.unimelb.edu.au/~rui The University of Melbourne Aug 2006

2. 2 Copyright Many contents are from Database Management Systems, 3rd edition. Raghu Ramakrishnan & Johannes Gehrke McGraw-Hill, 2000. Some slides are from Prof. Beng Chin Ooi’s lecture notes for CS5226 in NUS Some are from the internet

3. 3 Outline Cost Model and File Organization The memory hierarchy Magnetic disk Cost model Heap files The concept of index Indexes : General Concepts and Properties Queries, keys and search keys Simple index files Properties of indexes Indexed sequential access method (ISAM) B + -tree Structure Search Insertion Deletion Other basic indexes Hashing Bitmap

4. 4 The Memory Hierarchy CPU L1 Cache CPU Die L2 Cache Main Memory Hard Disk Registers Intel PIII CPU: 450MHz Memory: 512MB 40 ns 90 ns 14 ms = 14000000 ns Cost is affected significantly by disk accesses !

5. 5 The Hard (Magnetic) disk The time for a disk block access, or disk page access or disk I/O access time = seek time + rotational delay + transfer time IBM Deskstar 14GPX: 14.4GB Seek time: 9.1 msec Rotational delay: 4.17 msec Transfer rate: 13MB/sec, that is, 0.3msec/4KB

6. 6 The Cost Model Cost measure: number of page accesses Objective A simple way to estimate the cost (in terms of execution time) of database operations Reason Page access cost is usually the dominant cost of database operations An accurate model is too complex for analyzing algorithms Note This cost model is for disk based databases; NOT applicable to main memory databases Blocked access: sequential access

7. 7 Basic File Organization : Heap Files File : a logical collection of data, physically stored as a set of pages. Heap File (Unordered File) Linked list of pages The DBMS maintains the header page: Operations Insertion Deletion Search Advantage: Simple Disadvantage: Inefficient Header Page Data Page Data Page Data Page Data Page Data Page Data Page Pages with free space Full pages

8. 8 The Concept of Index Searching a Book… Index A data structure that helps us find data quickly Note Can be a separate structure (we may call it the index file) or in the records themselves. Usually sorted on some attribute Index by Author Index by Title Database by Rao Database by Rui Programming by Alistair CatalogsBooks Where are books by Rui ? Where is “Database” ?

9. 9 Query, Key and Search Key Queries Exact match (point query) Q1: Find me the book with the name “Database” Range query Q2: Find me the books published between year 2003-2005 Searching methods Sequential scan － too expensive Through index – if records are sorted on some attribute, we may do a binary search If sorted on “book name”, then we can do binary search for Q1 If sorted on “year published”, then we can do binary search for Q2 Key vs. Search key Key: the indexed attribute Search key: the attribute queried on

10. 10 Simple Index File (Clustered, Dense) Sorted records 20 10 40 30 60 50 80 70 100 90 Dense index 10 20 30 40 50 60 70 80 90 100 110 120

11. 11 Simple Index File (Clustered, Sparse) Sorted records 20 10 40 30 60 50 80 70 100 90 Sparse index 10 30 50 70 90 110 130 150 170 190 210 230

12. 12 Simple Index File (Clustered, Multi-level) Sorted records 20 10 40 30 60 50 80 70 100 90 Sparse 2nd level 10 30 50 70 90 110 130 150 170 190 210 230 10 90 170 250 330 410 490 570

13. 13 Simple Index File (Unclustered, Dense) Unsorted records 50 30 70 20 40 80 10 100 60 90 Dense index 10 20 30 40 50 60 70... 10 50 90... sparse high level

14. 14 Simple Index File (Unclustered, Sparse ?) 50 30 70 20 40 80 10 100 60 90 30 20 80 100 90... does not make sense! Unsorted records Sparse index

15. 15 Properties of Indexes Alternatives for entries in an index An actual data record A pair (key, ptr) A pair (key, ptr-list) Clustered vs. Unclustered indexes Dense vs. Sparse indexes Dense index on clustered or unclustered attributes Sparse index only on clustered attribute Primary vs. Secondary indexes Primary index on clustered attribute Secondary index on unclustered attribute

16. 16 Indexes on Composite Keys Q3: age=20 & sal=10 Index on two or more attributes: entries are sorted first on the first attribute, then on the second attribute, the third … Q4: age=20 & sal>10 Q5: sal=10 & age>20 Note Different indexes are useful for different queries sue1375 bob cal joe12 10 20 8011 12 nameagesal 12,20 12,10 11,80 13,75 20,12 10,12 75,13 80,11 11 12 13 10 20 75 80 Data records sorted by name Examples of composite key indexes using lexicographic order.

17. 17 Indexed sequential access method (ISAM) Tree structured index Support queries Point queries Range queries Problems Static: inefficient for insertions and deletions Data pages Overflow page

18. 18 The B + -Tree: A Dynamic Index Structure Grows and shrinks dynamically Minimum 50% occupancy (except for root). Each node contains d <= m <= 2d entries. The parameter d is called the order of the tree. Height-balanced Insert/delete at log f N cost (f = fanout, N = No. leaf pages) Pointers to sibling pages Non-leaf pages (internal pages) Leaf pages If directory page, same as non-leaf pages; pointers point to data page addresses If data page P 0 K 1 P 1 K 2 P 2 K 3 … K m P m K 0 D 0 K 1 D 1 K 2 D 2 … K m D m

19. 19 Searching in a B + -Tree Search begins at root, and key comparisons direct it to a leaf (as in ISAM) Search for 5, 15, all data entries >= 24... What about all entries <= 24 ? Root 1724 30 2 35 1416 1920222427 293334 38 39 13

20. 20 Insertion in a B + -Tree Find correct leaf L Put data entry onto L If L has enough space, done! Else, must split L (into L and a new node L2) Redistribute entries evenly, copy up middle key. Insert index entry pointing to L2 into parent of L. This can happen recursively To split index node, redistribute entries evenly, but push up middle key. (Contrast with leaf splits.) Splits “ grow ” tree; root split increases height. Tree growth: gets wider or one level taller at top.

21. 21 Inserting 7 & 8 into the B + -Tree Observe how minimum occupancy is guaranteed in both leaf and index pg splits Root 2 35 71416 1920222427 293334 38 39 1724 30 13 2 35 7 8 5 (Note that 5 is copied up and continues to appear in the leaf.) 1724 30 13

22. 22 Inserting 8 into the B + -Tree (continued) Note the difference between copy up and push up 52430 17 13 (17 is pushed up and only appears once in the index. Contrast this with a leaf split.) 2 35 7 8 5 1724 30 13

23. 23 The B + -Tree After Inserting 8 Note that root was split, leading to increase in height We can avoid splitting by re-distributing entries. However, this is usually not done in practice. Why? 23 Root 17 24 30 1416 1920222427 293334 38 39 135 758

24. 24 Deletion in a B + -Tree Start at root, find leaf L where the entry belongs. Remove the entry. If L is at least half-full, done! If L has only d-1 entries, Try to redistribute, borrowing from sibling (adjacent node with same parent as L). If redistribution fails, merge L and sibling. If merge occurred, must delete entry (pointing to L or sibling) from parent of L. Merge could propagate to root, decreasing height.

25. 25 Deleting 19 & 20 from the B + -Tree Deleting 20 is done with re-distribution. Note how the middle key is copied up 23 Root 17 24 30 1416 1920222427 293334 38 39 135 758 23 Root 17 30 1416 3334 38 39 135 7582224 27 29

26. 26 Deleting 24 from the B + -Tree Merge happens Observe ‘ toss ’ of index entry (on right), and ‘ pull down ’ of index entry (below). Note the decrease in height 30 2227 293334 38 39 2 3 7 1416 22 27 29 3334 38 39 58 Root 30 135 17

27. 27 Summary of the B + -Tree A dynamic structure – height balanced – robustness Scalability Typical order: 100. Typical fill-factor: 67%. average fanout = 133 Typical capacities (root at Level 1, and has 133 entries) Level 5: 1334 = 312,900,700 records Level 4: 1333 = 2,352,637 records Can often hold top levels in buffer pool: Level 1 = 1 page = 8 Kbytes Level 2 = 133 pages = 1 Mbyte Level 3 = 17,689 pages = 133 Mbytes Efficient point and range queries – performance Concurrency Essential properties of a DBMS

28. 28 Other Basic Indexes: Hash Tables Static hashing: static table size, with overflow chains Extendible hashing: dynamic table size, with no overflows Efficient point query, but inefficient range query... M-1 0  (1 +  )

29. 29 Other Basic Indexes: Bitmap Bitmap with bit position to indicate the presence of a value Advantages Efficient bit-wise operations Efficient for aggregation queries: counting bits with 1 ’ s More compact than trees-- amenable to the use of compression techniques Limitations Only good for domain of small cardinality MF 10 10 01 10 cusidnamegenderrating 112JoeM3 115RamM5 119SueF5 120WooM4 12345 00100 00001 00001 00010 genderrating

30. 30 B + -Tree For All and Forever? Can the B+-tree being a single-dimensional index be used for emerging applications such as: Spatial databases High-dimensional databases Temporal databases Main memory databases String databases Genomic/sequence databases... Coming next … Indexing Multidimensional Data