CPSC-608 Database Systems

CPSC-608 Database Systems
Fall 2017 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #18 Notes #7

DBMS graduate database in tables (relations) lock table DDL language
administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager DML (query) language query execution engine DML complier main memory buffers secondary storage (disks) DBMS graduate database

Another Index Structure: Hash Tables
hush function h h(k) buckets search key k A bucket is typically a disk block (probably with overflow blocks) h(k), 0 ≤ k ≤ b-1, gives an easy way to compute the bucket address (direct: address from h(k); indirect: h(k) is the index in a directory. Notes #7

How do we cope with growth?
Overflows and reorganizations Dynamic hashing Extensible Linear

Extensible Hashing: General framework
# bits used by the buckets (block index) # bits used by the directory (hash index) i j1 00…00 00…01 j2 h i k h(x) h(x)i j2 i . 11…11 i directory buckets

Extensible hashing: Searching

Extensible hashing: Searching
input: a search key x \\ h is the hash function, H is the directory, i is the current hash index. m = the first i bits of h(x); read in the disk block B with the address H[m].

Extensible hashing: Insertion

input: a tuple t with search key x \\ h is the hash function, H is the directory, i is the current hash index. m = the first i bits of h(x); let the block with the address H[m] be B; IF B has room THEN add t in B ELSE let j be the block index of B IF i = j THEN {double the size of H to 2i+1, i = i + 1; let the pointers in the new H[2k] and H[2k+1] both equal to that in the old H[k], 0 ≤ k ≤ 2i; } split B U{t} into B0 and B1 (with block index j+1) using the j+1st bit, let H[mj0**] point to B0 and H[mj1**] point to B1. b i m h(x) b i mj h(x)

input: a tuple t with search key x \\ h is the hash function, H is the directory, i is the current hash index. m = the first i bits of h(x); let the block with the address H[m] be B; IF B has room THEN add t in B ELSE let j be the block index of B IF i = j THEN {double the size of H to 2i+1, i = i + 1; let the pointers in the new H[2k] and H[2k+1] both equal to that in the old H[k], 0 ≤ k ≤ 2i; } split B U{t} into B0 and B1 (with block index j+1) using the j+1st bit, let H[mj0**] point to B0 and H[mj1**] point to B1.

input: a tuple t with search key x \\ h is the hash function, H is the directory, i is the current hash index. m = the first i bits of h(x); let the block with the address H[m] be B; IF B has room THEN add t in B ELSE let j be the block index of B IF i = j THEN {double the size of H to 2i+1, i = i + 1; let the pointers in the new H[2k] and H[2k+1] both equal to that in the old H[k], 0 ≤ k ≤ 2i; } split B U{t} into B0 and B1 (with block index j+1) using the j+1st bit, let H[mj0**] point to B0 and H[mj1**] point to B1. b i m h(x)

input: a tuple t with search key x \\ h is the hash function, H is the directory, i is the current hash index. m = the first i bits of h(x); let the block with the address H[m] be B; IF B has room THEN add t in B ELSE let j be the block index of B \\ B has no room for t IF i = j THEN {double the size of H to 2i+1, i = i + 1; let the pointers in the new H[2k] and H[2k+1] both equal to that in the old H[k], 0 ≤ k ≤ 2i; } split B U{t} into B0 and B1 (with block index j+1) using the j+1st bit, let H[mj0**] point to B0 and H[mj1**] point to B1. b i m h(x)

input: a tuple t with search key x \\ h is the hash function, H is the directory, i is the current hash index. m = the first i bits of h(x); let the block with the address H[m] be B; IF B has room THEN add t in B ELSE let j be the block index of B \\ B has no room for t IF i = j THEN {double the size of H to 2i+1, i = i + 1; let the pointers in the new H[2k] and H[2k+1] both equal to that in the old H[k], 0 ≤ k ≤ 2i; } split B U{t} into B0 and B1 (with block index j+1) using the j+1st bit, let H[mj0**] point to B0 and H[mj1**] point to B1. b i m h(x) i > j b i mj h(x)

Insertion in Extensible Hashing

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 1 1 Insert: c1001 1 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 1 1 Insert: g1010 c1001 1 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 1 1 Insert: g1010 c1001 1 g1010 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 1 1 Insert: g1010 c1001 1 g1010 k1100 Split the block, and increase the block index

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 00 01 10 11 i = 1 1 Insert: g1010 c1001 1 g1010 k1100 Split the block, and increase the block index if the block index is equal to the hash index, first double the directory size

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 00 01 10 11 i = 1 1 Insert: g1010 2 c1001 1 g1010 k1100 Split the block, and increase the block index if the block index is equal to the hash index, first double the directory size 2

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 00 01 10 11 i = 1 1 Insert: g1010 2 c1001 1 k1100 g1010 Split the block, and increase the block index if the block index is equal to the hash index, first double the directory size k1100 2

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 00 01 10 11 Insert: g1010 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 00 01 10 11 Insert: g1010 d0111 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 d0111 00 01 10 11 Insert: g1010 d0111 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 1 i = 2 d0111 00 01 10 11 Insert: g1010 d0111 e0000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 b0001 e0000 1 i = 2 d0111 00 01 10 11 Insert: g1010 d0111 e0000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 Split the block, and increase the block index b0001 e0000 1 i = 2 d0111 00 01 10 11 Insert: g1010 d0111 e0000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 Split the block, and increase the block index 2 b0001 e0000 1 2 i = 2 d0111 00 01 10 11 Insert: g1010 d0111 e0000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 Split the block, and increase the block index e0000 2 b0001 d0111 b0001 1 2 i = 2 d0111 00 01 10 11 Insert: g1010 d0111 e0000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 e0000 2 b0001 d0111 2 i = 2 00 01 10 11 Insert: g1010 d0111 e0000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 e0000 2 b0001 d0111 2 i = 2 00 01 10 11 Insert: g1010 d0111 e0000 a1000 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 e0000 2 b0001 d0111 2 i = 2 00 01 10 11 Insert: g1010 d0111 e0000 a1000 2 c1001 g1010 a1000 Split the block, and increase the block index 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 i = 3 e0000 2 000 001 010 011 100 101 110 111 b0001 d0111 2 i = 2 00 01 10 11 if the block index is equal to the hash index, first double the directory size Insert: g1010 d0111 e0000 a1000 2 c1001 g1010 a1000 Split the block, and increase the block index 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 i = 3 e0000 2 000 001 010 011 100 101 110 111 b0001 d0111 2 i = 2 00 01 10 11 3 Insert: g1010 d0111 e0000 a1000 3 2 c1001 g1010 a1000 Split the block, and increase the block index 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 i = 3 e0000 2 000 001 010 011 100 101 110 111 b0001 d0111 2 i = 2 00 01 10 11 3 a1000 Insert: g1010 d0111 e0000 a1000 c1001 3 g1010 2 c1001 g1010 2 k1100

h(b) = 0001 h(c) = 1001 h(d) = 0111 h(e) = 0000 h(g) = 1010 h(k) = 1100 i = 3 e0000 2 000 001 010 011 100 101 110 111 b0001 d0111 2 i = 2 00 01 10 11 3 a1000 Insert: g1010 d0111 e0000 a1000 c1001 g1010 3 2 k1100

Extensible hashing: deletion
No merging of blocks Merge blocks and cut directory if possible (Reverse insert procedure)

Note: Still need overflow chains
Example: many records with duplicate keys insert 1100 1 1101 1100

Example: many records with duplicate keys if we split: insert 1100 2 For 10** 1 1101 1100 2 For 11** 1101 ? 1100 1100

Example: many records with duplicate keys if we split: Even further split: 2 For 10** insert 1100 2 For 10** 1 1101 3 For 110* 1100 1101 2 For 11** 1101 1100 1100 ? ? 1100 1100 3 For 111*

Solution: overflow chains
insert 1100 add overflow block: 1 1 1101 1100 1101 1101 1100

Extensible hashing Summary. + Can handle growing files
- with less wasted space - with no full reorganizations +

Extensible hashing Summary. + - Can handle growing files Indirection
- with less wasted space - with no full reorganizations + Indirection (Not bad if directory in memory) Directory doubles in size (Now it fits, now it does not) -

How do we cope with growth?
Overflows and reorganizations Dynamic hashing Extensible Linear

Linear hashing Another dynamic hashing scheme

Linear hashing Another dynamic hashing scheme Ideas:
Use the same hash function h; Use only part of h when the hash table is smaller (use the i low order bits of h. grows b i h(x) =

Use the same hash function h; Use only part of h when the hash table is smaller (use the i low order bits of h. grows b i h(x) = Similar to Extensible hash

(c) Hash table size n grows linearly
Linear hashing Another dynamic hashing scheme Ideas: Use the same hash function h; Use only part of h when the hash table is smaller (use the i low order bits of h. grows b i h(x) = Similar to Extensible hash (c) Hash table size n grows linearly Main difference n 00..0 (|n| = i)

Use the same hash function h; Use only part of h when the hash table is smaller (use the i low order bits of h. grows b i h(x) = Similar to Extensible hash (c) Hash table size n grows linearly Main difference n 00..0 (|n| = i) (d) Use overflow blocks.

Linear hashing b h(x) = grows i Hash table size n grows linearly (n is a parameter for the hash structure b h(x) = n 00..0 (|n| = i) h(x)i i = |n|

Linear hashing b h(x) = grows i Hash table size n grows linearly (n is a parameter for the hash structure (backet n is the first unused bucket) b h(x) = n 00..0 (|n| = i) h(x)i i = |n|

Where does x go if h(x)i ≥ n?
Linear hashing b h(x) = grows i Hash table size n grows linearly (n is a parameter for the hash structure (backet n is the first unused bucket) b h(x) = n 00..0 (|n| = i) h(x)i i = |n| Where does x go if h(x)i ≥ n?

Where does x go if h(x)i ≥ n?
Linear hashing b h(x) = grows i Hash table size n grows linearly (n is a parameter for the hash structure (backet n is the first unused bucket) b h(x) = n 00..0 (|n| = i) h(x)i i = |n| Where does x go if h(x)i ≥ n? Put x in h(x)i – 2i-1 (< n)!! (h(x)i – 2i-1 = h(x)i with the leading bit 1 replaced with 0)

Linear Hashing: Searching
How Do We Search x? Linear Hashing: Searching input: a search key x \\ h is the hash function, n is the current upper bound, i = |n| m = the last i bits of h(x); IF m ≥ n THEN m = m – 2i-1; read in the disk block(s) with the address m \\ you should check overflow blocks in the address m.

Linear Hashing: Insertion
How Do We Insert x? Linear Hashing: Insertion input: a tuple t with search key x \\ h is the hash function, n is the current upper bound, i = |n| m = the last i bits of h(x); IF m ≥ n THEN m = m – 2i-1; insert t in the disk block B with the address m; \\ If B is full, you need to use an overflow block.

Linear Hashing: Insertion
How Do We Insert x? Delete Linear Hashing: Insertion Deletion input: a tuple t with search key x \\ h is the hash function, n is the current upper bound, i = |n| m = the last i bits of h(x); IF m ≥ n THEN m = m – 2i-1; insert t in the disk block B with the address m; \\ you may need to check overflow blocks. Delete

How Do We Expand the Hash Table?

When Do We Expand the Hash Table?

When Do We Expand the Hash Table? needed space Keep track of: R = available space If R > threshold (e.g., 80%) then increase n

Linear Hashing: Increasing Hash Table Size
How Do We Expand the Hash Table? When Do We Expand the Hash Table? needed space Keep track of: R = available space If R > threshold (e.g., 80%) then increase n Linear Hashing: Increasing Hash Table Size input: the current upper bound n \\ h is the hash function, i = |n| read in the disk block(s) B of address n – 2i -1 ; split (properly) the tuples in B and put them in the block B and the block B’ with address n; n = n + 1;

How Do We Shrink the Hash Table?

How Do We Shrink the Hash Table?
When? When R is smaller than a threshold (e.g., 50%)

Linear Hashing: Decreasing Hash Table Size
How Do We Shrink the Hash Table? When? When R is smaller than a threshold (e.g., 50%) Linear Hashing: Decreasing Hash Table Size input: the current upper bound n \\ h is the hash function, i = |n| n = n − 1; move the tuples in the block(s) of address n to the block(s) of address n – 2i -1 (here i is the length of the new n).

Linear Hashing: General framework
00…00 00…01 x h h(x) i h(x)i . b i no tuples 1*…** = n i grow linearly buckets

Example b=4 bits, 2 keys/bucket
Future growth buckets 0000 0101 1010 1111

Example b=4 bits, 2 keys/bucket n = 10 (1 + the largest index of the used buckets)
future growth buckets 0000 0101 1010 1111 n =10

Example b=4 bits, 2 keys/bucket n = 10 (1 + the largest index of the used buckets) i = |n| = 2 (# used bits) Future growth buckets 0000 0101 1010 1111 n =10

Example b=4 bits, 2 keys/bucket n = 10 (1 + the largest index of the used buckets) i = |n| = 2 (# used bits) Future growth buckets 0000 0101 1010 1111 n =10 Rules: If h(x)i < n, then look at bucket h(x)i

Example b=4 bits, 2 keys/bucket n = 10 (1 + the largest index of the used buckets) i = |n| = 2 (# used bits) Future growth buckets 0000 0101 1010 1111 n =10 Rules: If h(x)i < n, then look at bucket h(x)i If h(x)i ≥ n, then look at bucket h(x)i − 2i -1 (i.e., replacing the leading bit 1 of h(x)i by 0)

Insertion: b=4 bits, 2 keys/bucket, n=10, i=2
1101 (can have overflow chains!) Future growth buckets 0000 0101 1010 1111 n =10 Rules: If h(x)i < n, then look at bucket h(x)i If h(x)i ≥ n, then look at bucket h(x)i − 2i -1 (i.e., replacing the leading bit 1 of h(x)i by 0)

Increase size n: b=4 bits, n=10, i=2
0000 0101 1010 1111 n = 10

0000 0101 1010 1010 1111 n = 10

Future growth buckets 0000 0101 1010 1111 n =11 10

Insert: b=4 bits, n=11, i=2 insert 1101 1101 0000 0101 1010 1111
Future growth buckets 0000 0101 1010 1111 n =11 10

1101 0000 0101 1010 1111 n =11 10

1101 0000 0101 1010 1111 1111 n =11 10

1101 0000 0101 1010 1111 10 n =100

0101 0000 0101 1010 1111 1101 10 n =100

Linear Hashing Summary + + Can handle growing files
- with less wasted space - with no full reorganizations No indirection like extensible hashing + +

Linear Hashing Summary + + ‒ Can handle growing files
- with less wasted space - with no full reorganizations No indirection like extensible hashing + + overflow chains ‒

Example: BAD CASE Very full Very empty Need to move m here…
Would waste space…

Summary Hashing - How it works - Dynamic hashing - Extensible - Linear

Next Algorithms implementing the relational algebraic operations:
Projection and selection Set and bag operations Join operations Grouping, duplicate elimination, sorting

Algorithms Implementing Relational Algebraic Operations
Projection and selection π, σ Set/bag operations US, ∩S, −S, UB, ∩B, −B Join operations Extended operations γ, δ, τ, table-scan × C ,

CPSC-608 Database Systems

Similar presentations

Presentation on theme: "CPSC-608 Database Systems"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

CPSC-608 Database Systems

Similar presentations

Presentation on theme: "CPSC-608 Database Systems"— Presentation transcript:

Similar presentations

About project

Feedback