1. Indexing - revisited CS 186, Fall 2012 R & G Chapter 8

2. Index Classification 1.Selections (lookups) supported 2.Representation of data entries in index – what kind of info is the index actually storing? – 3 alternatives here 3.Clustered vs. Unclustered Indexes 4.Single Key vs. Composite Indexes 5.Tree-based, hash-based, other

3. Indexes: Selections supported field constant Equality selections (op is =) – Either “tree” or “hash” indexes help here. Range selections (op is one of, =, BETWEEN) – “Hash” indexes don’t work for these. More exotic selections – multi-dimensional ranges (“east of Berkeley and west of Truckee and North of Fresno and South of Eureka”) – multi-dimensional distances (“within 2 miles of Soda Hall”) – Ranking queries (“10 restaurants closest to Berkeley”) – Regular expression matches, genome string matches, etc. – Keyword/Web search - includes “importance” of words in documents, link structure, …

4. Tree Index: Example Index entries: they direct search for data entries in leaves. In example: Fanout (F) = 3 (note: unrealistic!) – more typical: 16KB page, 67% full, 32Byte entries = approx 300 10*15*20*27*33*37* 40* 46* 51* 55* 63* 97* 2033 5163 40 Root Leaf Level: Nodes contain “Data Entries” Index Levels: Nodes contain “Index Entries”

5. What’s in a “Data Entry”? Question: What is actually stored in the leaves of the index for key value “k”? (a data entry for key “k” is denoted “k*” in book and examples) Three alternatives: 1. Actual data record(s) with key value k 2. { } 3. Choice is orthogonal to the indexing technique. – e.g., B+ trees, hash-based structures, R trees, …

6. Alt 1= “Index-Organized File” Actual data records are stored in leaves. If this is used, index structure becomes a file organization for data records (e.g., a sorted file). At most one index on a given collection of data records can use Alternative 1. This alternative saves pointer lookups but can be expensive to maintain with insertions and deletions.

7. Index-Organized File Leaf Pages Non-leaf Pages Keys and pointers to next level Actual Records Q: How many levels if B leaf blocks and a fanout of F ? A: log F B # Leaf BlocksFanoutLevels 1,0003003 10,0003003 100,0003004 1,000,0003004 10,000,0003004 100,000,0003005 16KB pages, 67%full and 100 byte records = approx 100 recs/page. so, can store 10B rows with 5 levels. Note: All pages at all levels are: “Slotted Pages”

8. Operation Cost Heap FileSorted File (100% Occupancy) Index-Organized File (67% Occupancy) Scan all records BD Equality Search unique key 0.5 BD (log 2 B) * D Range Search BD[(log 2 B) + #match pg]*D Insert 2D((log 2 B)+B)D Delete (0.5B+1) D ((log 2 B)+B)D (because rd,wrt 0.5 file) B: The size of the data (in pages) R: Number of records per page D: (Average) time to read or write disk page 1.5 BD (bcos 67% full) (log F 1.5B) * D ((log F 1.5B) + #match pg)*D ((log F 1.5B)+1)D

9. Alternatives for Data Entries (Contd.) Alternative 2 { } and Alternative 3 Easier to maintain than Index-Organized. On the other hand: Index-organized could be faster for reads. If more than one index is required on a given file, at most one index can use Alt 1; rest must use 2 or 3. Alt 3 more compact than Alt 2, but has variable sized data entries even with fixed-length search keys Even worse, for large rid lists the data entry would have to span multiple blocks!

10. Clustered vs. Unclustered Index “Clustered” Index: the order of data records is the same as, or `close to’, the order of index data entries. A file can be clustered on at most one search key. Cost of retrieving data records through index varies greatly based on whether index is clustered or not! Index-organized implies clustered but not vice-versa. In other words, alt-1 is always clustered alt 2 and alt 3 may or may not be clustered.

11. Example: Alt 2 index for a Heap File For a clustered index: Sort the heap file on the search key column(s) – Leave some free space on pages for future inserts Build the index Use overflow pages in data file if necessary Index entries Data entries direct search for (Index File) (Data file) Data Records data entries Data entries Data Records CLUSTERED UNCLUSTERED

12. Unclustered vs. Clustered Indexes What are the tradeoffs???? Clustered Pros – Efficient for range searches – May be able to do some types of compression – Possible locality benefits (related data?) – ??? Clustered Cons – Maintenance cost (pay on the fly or be lazy with reorganization) – Can only cluster according to a single order

13. Operation Cost Unclustered Alt-2 Tree Idx (Index file: 67% occupancy) (Data file: 100% occupancy ) Clustered Alt-2 Tree Idx (Index and Data files: 67% occupancy) Scan all records BD (ignore index) Equality Search unique key (1+ log F 0.5 B) *D assume an index entry is 1/3 the size of a record so index leaf level =.33 * 1.5B = 0.5B Range Search [(log F 0.5B) + #matching_leaf_pages – 1 + #match records]*D Insert ((log F 0.5B)+3)D Delete same as insert B: The size of the data (in pages) D: (Avg) time to read or write disk page 1.5 BD (ignore index) (1+ log F 0.5B) * D ((log F 0.5B) + #match pages)*D ((log F 0.5B)+3)D same as insert

14. Composite Search Keys Search on a combination of fields. – Equality query: Every field value is equal to a constant value. E.g. wrt index: age=20 and sal =75 – Range query: Some field value is not a constant. E.g.: age > 20; or age=20 and sal > 10 Data entries in index sorted by search key to support range queries. – Lexicographic order – Like the dictionary, but on fields, not letters! sue1320 bob cal joe12 10 20 8011 12 nameagesal 12,20 12,10 11,80 13,20 20,12 10,12 20,13 80,11 11 12 13 10 20 80 Data records sorted by name Data entries in index sorted by Data entries sorted by Examples of composite key indexes using lexicographic order.

15. Index Classification Revisited 1.Selections (lookups) supported 2.Representation of data entries in index – what kind of info is the index actually storing? – 3 alternatives here 3.Clustered vs. Unclustered Indexes 4.Single Key vs. Composite Indexes 5.Tree-based, hash-based, other