1. CS 540 Database Management Systems
Lecture 5: DBMS Architecture, storage, and access methods

2. Database System Implementation
User Requirements
Conceptual Design
Physical Storage
Schema
Entity Relationship(ER) Model
Relational Model
Files and Indexes

3. The advantage of RDBMS
It separates logical level (schema) from physical level (implementation).
Physical data independence
Users do not worry about how their data is stored and processes on the physical devices.
It is all SQL!
Their queries work over (almost) all RDBMS deployments.

4. Challenges in physical level
Processor: – MIPS
Main memory: around 10 Gb/ sec.
Secondary storage: higher capacity and durability
Disk random access
Seek time + rotational latency + transfer time
Seek time: 4 ms ms!
Rotational latency: 2 ms – 7 ms!
Transfer time: at most 1000 Mb/ sec
Read, write in blocks.

5. Gloomy future: Moor’s law
Speed of processors and cost and maximum capacity of storage increase exponentially over time.
But storage (main and secondary) access time grows much more slowly.

6. Random access versus sequential access
Disk random access : Seek time + rotational latency + transfer time.
Disk sequential access: reading blocks next to each other
No seek time or rotational latency
Much faster than random access

7. DBMS Architecture User/Web Forms/Applications/DBA query transaction
Process manager
Query Parser
Transaction Manager
Query Rewriter
Logging &
Recovery
Query Optimizer
Lock Manager
Query Executor
Files & Access Methods
Lock Tables
Buffers
Buffer Manager
Main Memory
Storage Manager
Storage

8. DBMS Architecture User/Web Forms/Applications/DBA query transaction
Process manager
Query Parser
Transaction Manager
Query Rewriter
Logging &
Recovery
Query Optimizer
Lock Manager
Query Executor
Files & Access Methods
Lock Tables
Buffers
Buffer Manager
This lecture
Main Memory
Storage Manager
Storage

9. A Design Dilemma To what extent should we reuse OS services?
Reuse as much as we can
Performance problem (inefficient)
Lack of control (incorrect crash recovery)
Replicating some OS functions (“mini OS”)
Have its own buffer pool
Directly manage record structures with files …

10. OS vs. DBMS Similarities?
What do they manage?
What do they provide?

11. OS vs. DBMS: Similarities
Purpose of an OS:
managing hardware
presenting interface abstraction to applications
DBMS is in some sense an OS?
DBMS manages data
Both as API for application development!

12. OS vs. DBMS: Related Concepts
Process Management  What DB concepts?
process synchronization
deadlock handling
Storage management  What DB concepts?
virtual memory
file system

13. OS vs. DBMS: Differences?

14. OS vs. DBMS: Differences
DBMS: Top-down to encapsulate high-level semantics!
Data
data with particular logical structures
Queries
query language with well defined operations
Transactions
transactions with ACID properties
OS: Bottom-up to present low-level hardware

15. Problems with DBMS on top of OS
Buffer pool management
File system
Process management
Consistency control
Paged virtual memory

16. Buffer Pool Management
Performance of system calls
LRU replacement
Query-aware replacement needed for performance
Circular access: 1, 2, …, n, 1, 2, ..
Prefetching
DBMS knows exactly which block is to be fetched next
Crash recovery
Need “selected force out”

17. Relations vs. File system
Data object abstraction
file: array of characters
relation: set of tuples
Physical contiguity:
large DB files want clustering of blocks
sol1: managing raw disks by DBMS
sol2: simulate by managing free spaces in DBMS
Multiple trees (access methods)
file access: directory hierarchy (user access method)
block access: inodes
tuple access: DBMS indexes
- Sol2: DBMS asks OS for large-than-needed-now chunks, and manage space within DBMS

18. Process management Reuse OS process management
One process for each user
Problem: DB processes are large
long time to switch between processes
Problem: critical sections
Processes may have to wait for a descheduled process that has locks.
n server processes that handle users’ requests
duplication of OS multi-tasking inside servers!
communication between processes:
Message passing is not efficient
Solutions: OS implements
favored processes
not forced out, relinquish the control voluntarily.
faster message passing methods.

19. Consistency control OS provides some support for locking and recovery.
OS provides lock on files
DB requires lock on smaller units like tuples
Commit point
Buffer manager ensures all changes are flushed on disk.
Buffer manager must know the inside of transactions.

20. State of the art DBMSs duplicate some OS functionalities.
OS customized for DBMS

21. Access methods The methods that RDBMS uses to retrieve the data.
Attribute value(s)  Tuple(s)

22. Types of search queries
Point query over Product(name, price)
Select *
From Product
Where name = ‘IPad-Pro’;
Range query over Product(name, price) Select *
Where price > 2 AND price < 10;

23. Types of access methods
Full table scan
Inefficient for both point and range queries.
Sequential access
Efficient for both point and range queries.
Should keep the file sorted.
Inefficient to maintain
Middle ground?

24. Indexing
An old idea

25. Index
A data structure that speeds up selecting tuples in a relation based on some search keys.
Search key
A subset of the attributes in a relation
May not be the same as the (primary) key
Entries in an index
(k, r)
k is the search key.
r is the pointer to a record (record id).

26. Index Data file stores the table data.
Index file stores the index data structure.
Index file is smaller than the data file.
Ideally, the index should fit in the main memory.
Index File
Data File 10 20 10 20 30 40 30 40 50 60 70 80 50 60

27. Well known index structures
B+ trees:
very popular
Hash tables:
Not frequently used

28. B+ trees The index of a very large data file gets too large.
How about building an index for the index file?
A multi-level index, or a tree

29. B+ trees Degree of the tree: d
Each node (except root) stores [d, 2d] keys:
Non-leaf nodes 10 32 94
[A , 10)
[10, 32)
[32, 94)
[94, B)
Leaf nodes 12 28 32 39 41 65
Records 12 28 32

30. Example
d = 2 60 19 50 80 90
110 12 13 17 19 21 30 40 50 52 60 65 72 12 13 17 19 21 30 40 50 52 60 65 72

31. Retrieving tuples using B+ tree
Point queries
Start from the root and follow the links to the leaf.
Range queries
Find the lowest point in the range.
Then, follow the links between the nodes.
The top levels are kept in the buffer pool.

32. Inserting a new key Pick the proper leaf node and insert the key.
If the node contains more than 2d keys, split the node and insert the extra node in the parent.
If leaf level, add K3 to the right node
(K3, ) parent K1 K2 K3 K4 K5 R0 R1 R2 R3 R4 R5 K1 K2 R0 R1 R2 K4 K5 R3 R4 R5

33. Example
Insert K = 18 60 19 50 80 90
110 12 13 17 19 21 30 40 50 52 60 65 72 12 13 17 19 21 30 40 50 52 60 65 72

34. Insertion
Insert K = 18 60 19 50 80 90
110 12 13 17 18 19 21 30 40 50 52 60 65 72 12 13 17 18 19 21 30 40 50 52 60 65 72

35. Insertion
Insert K= 20 60 19 50 80 90
110 12 13 17 18 19 20 21 30 40 50 52 60 65 72 12 13 17 18 19 20 21 30 40 50 52 60 65 72

36. Insertion Need to split the node 60 19 50 80 90 110 12 13 17 18 19 2021 30 40 50 52 60 65 72 12 13 17 18 19 20 21 30 40 50 52 60 65 72

37. Insertion Split and update the parent node.
What if we need to split the root? 60 19 21 50 80 90
110 12 13 17 18 19 20 21 30 40 50 52 60 65 72 12 13 17 18 19 20 21 30 40 50 52 60 65 72

38. Deletion
Delete K = 21 60 19 21 50 80 90
110 12 13 17 18 19 20 21 30 40 50 52 60 65 72 12 13 17 18 19 20 21 30 40 50 52 60 65 72

39. Deletion Note: K = 21 may still remain in the internal levels 60 19 2150 80 90
110 12 13 17 18 19 20 30 40 50 52 60 65 72 12 13 17 18 19 20 30 40 50 52 60 65 72

40. Deletion
Delete K = 20 60 19 21 50 80 90
110 12 13 17 18 19 20 30 40 50 52 60 65 72 12 13 17 18 19 20 30 40 50 52 60 65 72

41. Deletion We need to update the number of keys on the node:
Borrow from siblings: rotate 60 19 21 50 80 90
110 12 13 17 18 19 30 40 50 52 60 65 72 12 13 17 18 19 30 40 50 52 60 65 72

42. Deletion We need to update the number of keys on the node:
Borrow from siblings: rotate 60 19 21 50 80 90
110 12 13 17 18 19 30 40 50 52 60 65 72 12 13 17 18 19 30 40 50 52 60 65 72

43. Deletion We need to update the number of keys on the node:
Borrow from siblings: rotate 60 18 21 50 80 90
110 12 13 17 18 19 30 40 50 52 60 65 72 12 13 17 18 19 30 40 50 52 60 65 72

44. Deletion What if we cannot borrow from siblings?
Example: delete K = 30 60 18 21 50 80 90
110 12 13 17 18 19 30 40 50 52 60 65 72 12 13 17 18 19 30 40 50 52 60 65 72

45. Deletion What if we cannot borrow from siblings? Merge with a sibling.60 18 21 50 80 90
110 12 13 17 18 19 40 50 52 60 65 72 12 13 17 18 19 40 50 52 60 65 72

46. Deletion What if we cannot borrow from siblings? Merge siblings! 60 1821 50 80 90
110 12 13 17 18 19 40 50 52 60 65 72 12 13 17 18 19 40 50 52 60 65 72

47. Deletion
What to do with the dangling key and pointer? simply remove them 60 18 21 50 80 90
110 12 13 17 18 19 40 50 52 60 65 72 12 13 17 18 19 40 50 52 60 65 72

48. Deletion
Final tree 60 18 50 80 90
110 12 13 17 18 19 40 50 52 60 65 72 12 13 17 18 19 40 50 52 60 65 72

49. What You Should Know
What are some major limitations of services provided by an OS in supporting a DBMS?
In response to such limitations, what does a DBMS do?
B+ tree indexing