Hash-Based Indexes Chapter 11 The slides for this text are organized into chapters. This lecture covers Chapter 11, and discusses hash-based indexing in depth. It should be covered after Chapter 8, which provides an overview of storage and indexing. At the instructor’s discretion, it can also be omitted without loss of continuity in other parts of the text. (In particular, Chapter 20 can be covered without covering this chapter, though covering this chapter will certainly provide a stronger foundation.) 1

Introduction : Hash-based Indexes Best for equality selections. Cannot support range searches. Static and dynamic hashing techniques exist: Trade-offs similar to ISAM vs. B+ trees. 2

Static Hashing h(k) mod N = bucket to which data entry with key k belongs. (N = # of buckets) h(key) mod N 2 key h N-1 Primary bucket pages Overflow pages 3

Static Hashing : h(k) mod N # primary pages fixed (N = # of buckets) allocated sequentially never de-allocated overflow pages if needed. h(key) mod N 2 key h N-1 Primary bucket pages Overflow pages 3

Static Hashing h(k) mod N = bucket to which data entry with key k belongs with N = # of buckets Hash function works on search key of record r. h() must distribute values over range [0 ... N-1]. For example, h(key) = (a * key + b) a and b are constants lots known about how to tune h. 4

Static Hashing Cons Primary pages fixed space  static structure. Fixed # buckets is the problem: Rehashing can be done  Not good for search. In practice, instead use overflow chains. Long overflow chains degrade performance. Solution: Employ dynamic techniques : Extendible hashing, or Linear Hashing 4

Extendible Hashing Problem: Bucket (primary page) becomes full. Solution: Re-organize file by doubling # of buckets? But Reading and writing all pages is expensive! 5

Extendible Hashing Ideas : Details : Trick : Use directory of pointers to buckets instead of buckets Details : Double # of buckets by doubling the directory Split just the bucket that overflowed! Trick : How hash function is adjusted! 5

Example Directory : array[4] LOCAL DEPTH 2 Directory : array[4] To find bucket for r, take last `global depth’ # bits of function h(r) Bucket A 4* 12* 32* 16* GLOBAL DEPTH 2 2 Bucket B 00 1* 5* 21* 13* 01 10 2 Bucket C 11 10* 2 DIRECTORY Bucket D 15* 7* 19* DATA PAGES 6

Example If h(r) = 5 = 101, it is in bucket pointed to by 01. LOCAL DEPTH 2 If h(r) = 5 = 101, it is in bucket pointed to by 01. If h(r) = 4 = 100, it is in bucket pointed to by 00. Bucket A 4* 12* 32* 16* GLOBAL DEPTH 2 2 Bucket B 00 1* 5* 21* 13* 01 10 2 Bucket C 11 10* 2 DIRECTORY Bucket D 15* 7* 19* DATA PAGES 6

Insertion Insert: If bucket is full, split it LOCAL DEPTH 2 Bucket A 4* 12* 32* 16* GLOBAL DEPTH 2 2 Bucket B Insert: If bucket is full, split it (allocate new page, re-distribute content). 00 1* 5* 21* 13* 01 10 2 Bucket C 11 10* 2 DIRECTORY Bucket D 15* 7* 19* DATA PAGES Splitting may : double the directory, or simply link in a new page. To tell what to do : Compare global depth with local depth for split bucket. 6

Insert h(r)= 6 = binary 110 13* 00 01 10 11 2 LOCAL DEPTH GLOBAL DEPTH DIRECTORY Bucket A Bucket B Bucket C Bucket D DATA PAGES 10* 1* 21* 4* 12* 32* 16* 15* 7* 19* 5* 7

Insert h(r)= 6 = binary 110 13* 00 01 10 11 2 LOCAL DEPTH GLOBAL DEPTH DIRECTORY Bucket A Bucket B Bucket C Bucket D DATA PAGES 10* 1* 21* 4* 12* 32* 16* 15* 7* 19* 5* 13* 00 01 10 11 2 LOCAL DEPTH GLOBAL DEPTH DIRECTORY Bucket A Bucket B Bucket C Bucket D DATA PAGES 10* 1* 21* 4* 12* 32* 16* 15* 7* 19* 5* 6* 7

Insert h(r)=20 = binary 10100 13* 00 01 10 11 2 LOCAL DEPTH GLOBAL DEPTH DIRECTORY Bucket A Bucket B Bucket C Bucket D DATA PAGES 10* 1* 21* 4* 12* 32* 16* 15* 7* 19* 5* 7

Insert h(r)=20 Split Bucket A into two buckets A1 and A2. 13* 00 01 10 11 2 LOCAL DEPTH GLOBAL DEPTH DIRECTORY Bucket A Bucket B Bucket C Bucket D DATA PAGES 10* 1* 21* 4* 12* 32* 16* 15* 7* 19* 5* Split Bucket A into two buckets A1 and A2. 3 4* 12* 20* Bucket A2 3 16* 32* Bucket A1 7

Insert h(r)=20 20* 3 Bucket A2 4* 12* Bucket A1 32* 16* LOCAL DEPTH 2 GLOBAL DEPTH 2 2 Bucket B 00 1* 5* 21* 13* 01 10 2 Bucket C 11 10* 2 Bucket D DIRECTORY 15* 7* 19* DATA PAGES 7

Insert h(r)=20 2 LOCAL DEPTH 3 LOCAL DEPTH Bucket A 32* 16* GLOBAL DEPTH 32* 16* Bucket A GLOBAL DEPTH 2 2 3 2 Bucket B 00 1* 5* 21* 13* 000 1* 5* 21* 13* Bucket B 01 001 10 2 2 010 Bucket C 11 10* 011 10* Bucket C 100 2 DIRECTORY 2 101 Bucket D 15* 7* 19* 15* 7* 19* 110 Bucket D 111 2 3 Bucket A2 4* 12* 20* DIRECTORY (`split image' 4* 12* 20* Bucket A2 of Bucket A) (`split image' of Bucket A) 7

Points to Note 20 = binary 10100. Last 2 bits (00) tell us if r belongs in A Last 3 bits needed to tell if r belongs into A1 or A2 8

More Points to Note Bits: Global depth of directory: Max # of bits needed to tell which bucket an entry belongs to Local depth of a bucket: Actual # of bits used to determine if an entry belongs to this bucket. When does bucket split cause directory doubling? Before insert, local depth of bucket = global depth. Insert causes local depth to become > global depth; Directory is doubled by copying it over and `fixing’ pointer to split the one “over-full page”. 8

Extendible Hashing : Delete If removal of data entry makes bucket empty, can be merged with `split image’. If each directory element points to same bucket as its split image, can halve directory. 9

Comments on Extendible Hashing If directory fits in memory, then equality search answered with one disk access; else with two. 100MB file, 100 bytes/rec, 4K pages contain 1,000,000 records (as data entries) and 25,000 directory elements; chances are high that directory will fit in memory. Directory grows in spurts. If the distribution of hash values is skewed, directory can grow large. 9

Summary Hash-based indexes: best for equality searches, cannot support range searches. Static Hashing can lead to long overflow chains. Extendible Hashing avoids overflow pages by splitting full bucket when new data to be added Directory to keep track of buckets, doubles periodically 16