1. Relational Database Systems 2
Instructor: Prof. James Cheng
Acknowledgement: The slides are extracted from and modified based on the slides provided by Prof. Sourav S. Bhowmick from Nanyang Technological University.

2. Topics to be covered ER model Relational Algebra SQL
Storage and Index Structures
Query Processing and Query Optimization

3. Disks and Files DBMS stores information on “hard” disks
Implications on DBMS Design
READ: transfer data from disk to main memory (RAM) (in msec)
WRITE: transfer data from RAM to disk (in msec)
Both are high-cost operations, relative to in-memory operations, so must be planned carefully!

4. Why Not Store Everything in Main Memory?
Main memory is volatile
We want data to be saved between runs.
Cost too much
Main memory is much more expensive!
Memory Hierarchy
Main memory (RAM) for currently used data
Disk for the main database (secondary storage)
Tapes for archiving older versions of the data (tertiary storage)

5. Disks
Data is stored and retrieved in units called disk blocks or pages
Disk consists of a sequence of blocks
All blocks are of the same size
A disk block is essentially the same size as a virtual memory page
Physical unit of access is always a block even if only a single bit needs to be accessed

6. Pages and Blocks Pages Blocks Data files are decomposed into pages
These are fixed size pieces of contiguous information in the file (usually multiples of 512, 1024, or 2048 bytes)
A page is the unit of exchange between disk and main memory (typical page size is 4096 bytes)
Blocks
Disk are divided into page size blocks of storage
A page can be stored in any disk block

7. ? Accessing a Disk Block I want block x block X in memory
Time to access a disk block
Seek time: Moving arms to position disk head on track (10-20 ms)
Rotational delay: Waiting for block to rotate under head (5-10 ms)
Transfer time: Actually moving data to/from disk surface
Key to lower I/O cost: Reduce seek/rotation delays!

8. Arranging Pages on Disk
Minimizing time cost
Blocks in a file should be arranged sequentially on disk
For a sequential scan, pre-fetching several pages at a time is advantageous!

9. Basic Search Value-based search SELECT * FROM R WHERE A = value;
SELECT * FROM R, S WHERE R.A = S.B;
Range search
SELECT * FROM R WHERE A > value;
Keyword search
SELECT * FROM R WHERE A LIKE value;

10. What is an index? A data structure Input Output What it does?
A property of records (value of one or more fields)
Finds the records with that property “quickly”
What it does?
Index let us find records without having to look at more than a small fraction of all possible records

11. Index
Block
holding
records
Value
Index
Matching
records

12. B-TREES Structure of B-trees Lookup and range queries
Insertion of a node
Deletion of a node

13. B-Trees What is it? Our Focus A type of index (tree)
Nonsequential, “balanced” (access paths to different records of equal length)
Adapts well to insertions & deletions
Consists of blocks holding at most n keys and n+1 pointers, and at least half of this
Our Focus
We consider a variation called a B+ tree

14. Example (B+ tree of order p)
Non-leaf levels
Root
p=4
pleaf=3
Non-leaf node
100
Block
pointers
Leaf
node 11 3 5
100 30 35
101
110
120
130
150
156
179
180
200
205
Leaf level
110
150
180 11
Sequential
link
Pointers in leaf node
Non-leaf node
point to data blocks (except for pointers that link leaf nodes (index blocks) sequentially)
point to nodes (which can be (non-leaf index) blocks once the tree is stored on disk)

15. Block Pointers in a B+-tree
100
110
150
180 11 3 5 30 35
101
120
130
156
179
200
205
Index in memory
Relation on disk … 3
150
100
130
156
120

16. Leaf Nodes of B+ Tree An entry for every value of the search key
If the search key is a primary key
Data pointer to the record or to the block containing the record
Otherwise
Data pointer to a block containing pointers to the data file records
Nodes are linked together to provide ordered access

17. Sample Leaf Node 25 28 Next leaf node 36 To record with k=36 To record

18. Sample Non-Leaf Node 25 28 31 To keys 28<= K < 31 To keys

19. Don’t want nodes to be too empty
Each block has n search keys
Number of pointers
At internal nodes at least (n+1)/2 (to child nodes)
at leaves at least (n+1)/2 (to data records/blocks)

20. 1 if only one record in the file
Summary
Max
Ptr
Keys
Min Ptr->data
Min Keys
Non-leaf (Non-root)
Leaf
(Non-root)
Root
n+1 n
(n+1)/2
(n+1)/2- 1
n+1 n
(n+1)/2
(n+1)/2
1 if only one record in the file
n+1 n
2(*) 1

21. Search 12 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

22. Search 28 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

23. Usefulness in Query Processing
Queries in which a single value of search K is required
Range queries
SELECT *
FROM R
WHERE R.k > 20;
SELECT *
FROM R
WHERE R.k >= 17 AND R.k <= 26 ;

24. R.K > 20 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

25. R.K >= 17 AND R.k <= 26 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

26. B+ Tree Insertion Problem Construct a B+ tree index of order 4
Database Systems
Problem
Construct a B+ tree index of order 4
1, 4, 7, 10, 17, 21, 31, 25, 19, 20, 28, 42
n = 3 25 28 31
To keys
28<= K < 31
To keys
K < 25
To keys
25<= K < 28
To keys
K >= 31

27. Insert 1, 4, 7
Database Systems
Step 1 1
Find a place for the new key in the appropriate leaf and put it there if there is space 1 4 1 4 7

28. Insert 10
Database Systems
Step 2
If there is no space in the leaf we split the leaf into two and divide the keys between the two new nodes so each is half full or just over half full 1 4 7
Overflow
Need to split 10

29. Leaf Node Split Step 1 Step 2 1 4 7 10 1 4 7 10
Database Systems
Step 1
Step 2
Create a new node M, which will be the sibling of N, immediately to its right
The first j = (n + 1)/2) key-pointer pairs, in sorted order, remains with N
Remaining ones are moved to M 1 4 7
Overflow
Need to split 10
J= 4/2
= 2 1 4 7 10

30. Leaf Node Split Step 3 Step 4 7 1 7 10 4
Database Systems
Step 3
Step 4
Extra pointer to the new node is created in the parent
Must be inserted in the parent in the correct sequence
(j+1)th search value is replicated in the parent node
(j+1)th search
value replicated
in parent node 7 1 7 10 4

31. Insert 17
Database Systems 7 1 10 4 17

32. Insert 21 7 1 10 4 17 21 Step 2 Overflow Need to split
Database Systems 7 1 10 4 17 21
Step 2
Overflow
Need to split
The first j = (n + 1)/2) key-pointer pairs, in sorted order, remains with N
Remaining ones are moved to M
J= 4/2
= 2

33. Insert 21 7 1 10 4 17 21 17 Step 3 Step 4 (J+1)th search
Database Systems 7 1 10 4 17 21
(J+1)th search
value replicated
in parent node 17
Step 3
Step 4
Extra pointer to the new node is created in the parent
Must be inserted in the parent in the correct sequence
(j+1)th search value is replicated in the parent node

34. Insert 31
Database Systems 7 1 10 4 17 21 31

35. Insert 25 25 7 1 10 4 17 21 31 Overflow Need to split J= 4/2 = 2
Database Systems 25 7 1 10 4 17 21 31
Overflow
Need to split
J= 4/2
= 2

36. Insert 25 7 1 10 4 17 21 25 31 25 (J+1)th search value replicated
Database Systems 7 1 10 4 17 21 25 31
(J+1)th search
value replicated
in parent node 25

37. Insert 19
Database Systems 7 1 10 4 17 21 25 31 25 19 7 1 10 4 17 19 25 31 25 21

38. Insert 20 7 1 10 4 17 19 25 31 20 25 21 Overflow Need to split J= 4/2
Database Systems 7 1 10 4 17 19 25 31 20 25 21
Overflow
Need to split
J= 4/2
= 2

39. Insert 20 7 1 10 17 19 20 21 25 31 20 4 Principle root needs
Database Systems 7 1 10 17 19 20 21 25 31
root needs
to be split 20 4
Insert a pointer to the new leaf into the node above it. Unfortunately, the parent is full!
Principle
If we try to insert into the root, and there is no room, then we split the root into two nodes and create a new root at the next higher level

40. Non-Leaf Node Split Step 1 (Node creation) Step 2 (Pointer movement)
Database Systems
Step 1 (Node creation)
Step 2 (Pointer movement)
Create a new node M, which will be the sibling of N, immediately to its right
Leave at N the first j=(n + 2)/2) pointers , in sorted order
Move to M the remaining (n+2)/2 pointers
Step 3 (Key movement)
The first n/2 keys stay with N, while the last n/2 keys move to M 7 17 25 1 7 10 17 19 20 21 25 31 4

41. Non-Leaf Node Split Step 4 (Key Movement to the Parent)
Database Systems
Step 4 (Key Movement to the Parent)
One key (say K) always in the middle left over
Goes neither to M nor to N
K -> smallest key reachable via the first of M’s children
K will be used by the parent of N and M to divide searches between these two nodes 20 7 17 25 1 7 10 17 19 20 21 25 31 4

42. Insert 28
Database Systems 20 7 17 25 1 7 10 17 19 20 21 4 25 31 28

43. Insert 42 (Leaf node split)
Database Systems 20 42 7 17 25 1 7 10 17 19 20 21 4 25 28 31
Overflow
Need to split
J= 4/2
= 2

44. Insert 42 (Leaf Node Split)
Database Systems 20 7 17 25 31 1 17 19 20 21 31 42 4 7 10 25 28

45. B+-Tree Deletion (Delete 5)
Database Systems
Step 1
Step 2
Locate the record to be deleted and its key-pointer pair in the leaf
Delete the record itself from the data file and delete the key-pointer pair from the tree
Step 3
If the node from which a deletion occurred still has at least the minimum number of keys and pointers then there is nothing more to be done 20 7 17 25 1 7 10 17 19 20 21 4 5 25 31

46. Minimum-key rule violation
Database Systems
Case 1
If one of the adjacent siblings of node N has more than the minimum number of keys and pointers then one key-pointer pair can be moved to N
Adjust parent key
Delete 17 20 7 17 16 25 1 7 10 20 21 4 5 16 17 19 25 31

47. Minimum-key rule violation
Database Systems
Case 2
Neither adjacent sibling can be used
Merge the two nodes, deleting one of them
Adjust the parent
Delete 4 20 7 17 25 1 7 10 17 19 20 21 4 7 10 25 31

48. Efficiency of B+-tree No. of records No. of Disk I/O
Assume an average node has 255 pointers
A three-level B-tree has 2552 = leaves with total of 2553 or about 16.6 million pointers to records
No. of Disk I/O
If root block kept in main memory
Each record can be accessed with 2+1 disk I/Os;
If all 256 internal nodes are in main memory, record access requires 1+1 disk I/Os
(256 x 4 KB = 1 MB; quite feasible!)

49. HASHING
Static Hashing
Dynamic Hashing

50. Hashing What does the hash function do? key  h(key)
A hash function takes a search key (hash key) as an input
Computes an integer between 0 to B – 1 (B is the number of buckets)
key  h(key)
<key>
Buckets
(typically 1
disk block) .

51. Main-memory Hash Table
Bucket Array
An array indexed from 0 to B – 1
Holds the header of B linked lists one for each bucket
Storage of Record
If a record has a search key K then we store the record by linking it to the bucket list for the bucket numbered h(K)

52. Main memory Hash Table 1 2 3 4 5

53. Secondary-storage Hash Table
Bucket Array
Consists of blocks rather than pointers to the header of lists
Storage of Record
Records that are hashed by the hash function h to a certain bucket are put in the block for that bucket
What happens when buckets overflow?
Chain of overflow blocks can be added to the bucket to
hold more records

54. Classification Static hash table Dynamic hash table Extensible hashing
Number of buckets is fixed and does not change
Number of buckets is allowed to vary
About one block per bucket
Extensible hashing
Linear hashing
Grows the number of buckets by doubling it
Number of buckets grow by 1

55. Static Hashing - Example
Assume that index structure has 8 buckets reserved
Each bucket takes only 2 search keys
Hashing function: h(k) = h mod 8 1 2 3 4 5 6 7

56. Insert 3, 23, 19 1 2 3 4 5 6 7 3 23 19
3%8 = 3
23%8 = 7
19%8 = 3

57. Insert 27, 31, 39 1 2 3 4 5 6 7 3 23 19 31 27 39

58. Find the record with search key 271 2 3 4 5 6 7 3 23 19 31 27 39
Two blocks

59. Find the record with key >= 271 2 3 4 5 6 7 3 23 19 31
Does not
Support range
queries 27 39
10 blocks

60. Deletion 27, 29, 3 1 2 3 4 5 6 7 3 20 5 23 19 29 31 39 27 21

61. Rule of Thumb for Static Hashing
Utilization
Try to keep space utilization between 50% and 80%
Utilization = # keys used
total # keys that fit
Rule of thumb
If < 50%, wasting space
If > 80%, overflows significant depends on how good hash function is & on # keys/bucket

62. How do we cope with growth?
Overflows and reorganizations
Dynamic hashing: # of buckets may vary
Extensible
Linear
also others ...

63. Extensible hashing What is it? Rules
Array of pointers to blocks represents the buckets, instead of arrays containing data blocks
Rules
Array of pointers can grow and its length is always a power of 2
There does not have to have a data block for each bucket
Certain buckets can share a block

64. Key ideas 00110101 Principle h(K)
The hash function h computes for each key a sequence of k bits for some large k
The bucket numbers will at all times use some smaller number of bits, say i bits, from the beginning of the sequence k
For example, k=32
h(K)
i (grows with time)

65. Key ideas Principle h(K)[i ] to bucket
The bucket array will have 2i entries when i is the number of bits used
h(K)[i ] to bucket .
Each bucket stores j, indicating #bits used for placing the
records in this block (j  i) .

66. Insertion of a record with key K
Step 1
Step 2
Compute h(K)
Take the first i bits of this
bit sequence
Step 3
Go to the entry of the bucket array indexed by these i bits
Follow the pointer and arrive at block B
Step 4
If B is not full then insert the record

67. i = 1 001 1 Block is full 100 101 Variable j 111
001 1 1
Block is full
100
101
Variable j
111
Let j indicates how many bits of the hash value are used to determine membership in block B

68. Case 1 (j < i) Step 1 Step 2 Step 3 Split B into two
Distribute records in B to the two blocks based on the value of their (j+1)st bit
Record whose key has 0 in that bit stays in B
Those with 1 there goes to the new block
Step 2
Put j+1 in each block’s “nub” to indicate number of bits used to determine membership
Step 3
Adjust the pointers in the bucket array so entries that formerly point to B now point either to B or to new block depending on their (j+1)st bit
Recursively if necessary

69. Case 2 (j = i) Step 1 Step 2 Step 3 Step 4 Step 5 Increment i by 1
Double the length of the bucket array (2i+1)
Step 3
Step 4
Suppose w is a sequence of i bits indexing one of the entries in the previous bucket array
In the new bucket array entries are indexed by w0 and w1
Each point to the same block that w used to point to
Membership in the block is still determined by whatever number of bits was previously used
Step 5
We proceed to split B as in case 1 as (i > j)

70. Insertion of 111 j=i=1 Step 3 Step 1 Step 2 001 1 100 101 00 001 01
001
j=i=1 1 1
100
101
Step 3 00 1
Step 1
001
Suppose w is a sequence of i bits indexing one of the entries in the previous bucket array
In the new bucket array entries are indexed by w0 and w1
Each point to the same block that w used to point to
Step 2
Increment i by 1 01 1
100
101
Double the length of the bucket array (2i+1) 10
i=2
j< i
Case 1 is true 11

71. Insertion of 111 Step 3 Step 1 Step 2 00 001 01 i=2 100 101 j=1 10 111
Split B into two
Distribute records in B to the two blocks based on the value of their (j+1)st bit
Record whose key has 0 in that bit stays in B
Those with 1 there goes to the new block
Adjust the pointers in the bucket array so entries that formerly point to B now point either to B or to new block depending on their (j+1)st bit
Recursively if necessary
Put j+1 in each block’s “nub” to indicate number of bits used to determine membership 1 00
001 01
i=2 2
100
101
j=1 10 2
111 11

72. Deletion (Eg: 0001) Cannot be combined 0001 1001 1010 1100 00 01 10 112
0001
1001
1010
1100 00 01 10
Cannot be combined 1 11 2
1001
1010 2
1100

73. Deletion (Eg: 1100) 00 1 1 1 01 1 1
1001
1010 2 2
1001
1001
1010
1010 10 2
1100 11
Steps
Delete value and block pointer
Empty bucket can be combined with neighbour
Size of array is reduced

74. Pros and Cons
Pros
Can handle growing files - without full reorganizations
Only one data block examined
Cons
Indirection (Not bad if directory in memory)
Array doubles in size(First it fits in memory, then it does not -> sudden performance degradation)

75. Indexing vs Hashing Hashing Indexing Hashing good for probes given key
SELECT …
FROM R
WHERE R.A = 5
Indexing
Good for range searches
SELECT ….
FROM R
WHERE R.A > 5