CPSC-608 Database Systems Fall 2015 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #5

secondary storage (disks) database administrator DDL language database programmer DML (query) language DBMS file manager buffer manager main memory buffers index/file manager DML complier DDL complier query execution engine transaction manager concurrency control lock table logging & recovery Graduate Database

secondary storage (disks) database administrator DDL language database programmer DML (query) language DBMS file manager buffer manager main memory buffers index/file manager DML complier DDL complier query execution engine transaction manager concurrency control lock table logging & recovery Graduate Database

Computer Memory Hierarchy

CPU main memory Disk controller Secondary Storage... A Typical Computer bus disks

Main Memory fast small capacity (gigabytes) volatile Disks slow large capacity (100’s gigabytes) non-volatile

Typical Disk Terms: Platter, Head, Cylinder, Track, Sector, Gap …

Top View Track Sector Gap

A “typical” disk 5 platters (thus 10 surfaces) A surface has 20,000 tracks A track has 500 sectors (million bytes) A sector has several thousand bytes Disk makes 5000 revolutions per minute (so about 10 millisecond per rotation)

Blocks A (logic) block = one or several sectors (typical size 16KB) Block address Physical device Cylinder # Surface # Sector

Disk Access Time block X in memory ? I want block X Time = Seek Time + Rotational Delay + Transfer Time + Other

Seek Time 3 or 5x x 1N Cylinders Traveled Time

Average Random Seek Time   SEEKTIME (i  j) S = N(N-1) N N i=1 j=1 j  i typical seek time: 10 ms  40 ms

Rotational Delay Head here Block I want Average Rotational Delay R = 1/2 revolution typical rotational delay = 8 ms

Transfer Rate: typical: t = 80 MB/second = 80 KB/millisecond transfer time: block size / t ~ 10/80 < 1 ms

Other Delays CPU time to issue I/O Contention for controller Contention for bus, memory Typical value: ≈ 0

Thus, reading a block of 16K bytes: Time = Seek Time + Rotational Delay + Transfer Time + Other ~ 30 ms + 8 ms + 16/80 ms + 0 ~ 40 ms

Main Memory fast (read/write: nanosecond) small capacity (gigabytes) volatile Disks slow (read/write: 1~40 millisecond) large capacity (100’s gigabytes) non-volatile Disks are about 10 5 ~10 6 times slower than main memory

I/O Model of Computation Dominance of I/O cost: if a block needs to be moved between disk and main memory, then the time taken to perform the read/write is much larger than the time likely to be used to manipulate that data in main memory. The number of disk block reads/writes is a good approximation to the entire computation.

Example. Sorting on disk Each tuple (with a key) takes 160 bytes Each block holds 100 tuples (16KB) A relation R has 10M tuples (1.6 GB, 100K blocks) Main memory has 100MB (6400 blocks) A disk read/write: 40 ms

Main memory sorting algorithms heap sort: 10M * log 2 (10M) = 230M disk block read/write = 9200M ms = seconds > 100 day quick sort and merge sort: 2 * 100K (blocks) * log 2 (10M) = 4.6M disk block read/write = 184M ms = seconds > 2 day

Two-phase Multiway MergeSort Phase 1. making sorted sublist repeat fill the main memory with remaining tuples in R and sort them; write the sorted sublist (of 6400 blocks) back to disk Phase 2. Merging repeat bring in a block from each of the sorted sublist; merge them and put in an “output” block; write the “output” block back to disk when it is full

Main memory Disk Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk Sort it First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk Sort it First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk Sort it First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk Sort it First Phase Two-phase Multiway MergeSort

Main memory Disk First Phase Two-phase Multiway MergeSort

Main memory Disk Second Phase

Main memory Disk Second Phase One block per sublist Two-phase Multiway MergeSort

Main memory Disk merge Two-phase Multiway MergeSort One block per sublist

Main memory Disk merge Two-phase Multiway MergeSort One block per sublist

Main memory Disk merge Two-phase Multiway MergeSort One block per sublist

Main memory Disk merge Two-phase Multiway MergeSort One block per sublist

Main memory Disk merge Two-phase Multiway MergeSort One block per sublist

Main memory Disk merge Two-phase Multiway MergeSort

# sublists = 100K/6400 = 16 thus, in phase 2, we can easily hold a block for each sublist in the main memory Disk block read/write: 100K (blocks) * 4 = 400K disk block read/write = 16M ms = seconds < 4.5 hours

secondary storage (disks) database administrator DDL language database programmer DML (query) language DBMS file manager buffer manager main memory buffers index/file manager DML complier DDL complier query execution engine transaction manager concurrency control lock table logging & recovery Graduate Database

Read Chapter 13 for more details on memory structures