1. CS 440 Database Management Systems RDBMS Architecture and Data Storage 1

2. Announcements Normal form and FD practice session on Feb 4 th in the class. Assignment 1 due on Feb 7 th – Submission through TEACH. Project progress report due on Feb 11 th – 1 – 2 pages of status report – Submission through TEACH 2

3. Database Implementation 3 Conceptual Design Physical Storage Schema Entity Relationship(ER) Model Relational Model Files and Indexes User Requirements SQL Data

4. The big advantage of RDBMS It separates logical level (schema) from physical level (implementation). Physical data independence – Users do not worry about how their data is stored and processes on the physical devices. – It is all SQL! – Their queries work over (almost) all RDBMS deployments. 4

5. Issues in logical level Data models – Relational, XML, … Query language Data quality – normalization Usability... 5

6. Issues on physical level Processor: 100 – 1000 MIPS Main memory: 1μs – 1 ns Secondary storage: higher capacity and durability Disk random access : Seek time + rotational latency + transfer time – Seek time: 4 ms - 15 ms! – Rotational latency: 2 ms – 7 ms! – Transfer time: around 1000 Mb/ sec – Read, write in blocks. 6

7. 7 Storage capacity versus access time 10 -9 10 -6 10 -3 10 -0 10 3 access time (sec) 10 15 10 13 10 11 10 9 10 7 10 5 10 3 cache electronic main electronic secondary magnetic optical disks online tape nearline tape & optical disks offline tape typical capacity (bytes) from Gray & Reuter updated in 2002

8. 8 Storage cost versus access time 10 -9 10 -6 10 -3 10 -0 10 3 access time (sec) 10 4 10 2 10 0 10 -2 10 -4 cache electronic main electronic secondary magnetic optical disks online tape nearline tape & optical disks offline tape dollars/MB from Gray & Reuter

9. Gloomy future: Moors law Speed of processors and cost and maximum capacity of storage increase exponentially over time. But storage (main and secondary) access time grows much more slowly. This is why managing and analyzing big data is hard. 9

10. Issues in physical level Three things are important in the database systems: performance, performance, and performance! ( Bruce Lindsay, co-creator of System R) 10

11. Issues in physical level Other things also matter – Reliability when it comes to transactions. – … But performance is still a big deal. 11

12. Is it easy to achieve good performance? Lets build an RDBMS. It supports core SQL. No stored procedure for this version! 12

13. Storing Data Store each relation in an ASCII file: Person (SSN, Name, Age) person.txt: 111222333 - John - 24 444222111 - Charles - 43 13

14. Storing Data Store schema information in a catalogue relation: Catalogue (AttrName, Type, RelName, Position) catalogue.txt: SSN - String – Person - 1 Name - String - Person - 2 Age – Integer – Person - 3 14

15. SQL Support SQL compiler Like any other computer language compiler. SELECT SSN FROM Person; SSN 111222333 444222111 15

16. Query Execution: Selection 1.Find the selection attribute position from the catalogue. 2.Scan the file that contains the relation. 3.Show the tuples that satisfy the condition. SELECT * FROM Person WHERE SSN = 111222333; 16

17. Query Execution: Join 1.Read the catalogue to find the info on join attributes. 2.Read the first relation, for each tuple: a.Read the second relation, for each tuple: b.Assemble the join tuple c.Output if they satisfy the condition. SELECT * FROM Person, PersonAddr WHERE Person.SSN = PersonAddr.SSN and Person.SSN = 111222333; 17

18. Performance Issues: Storing Data Update John to Sheldon – Rewrite the whole file very slow – Type conversion slow Delete the tuple with SSN of 111222333. Person (SSN, Name, Age) person.txt: 111222333 - John - 24 444222111 - Charles - 43 18

19. Performance Issues: Selection We have to scan the whole relation to select some tuples very slow We can use an index to find the tuples much fasters. SELECT * FROM Person WHERE SSN = 111222333; 19

20. Performance Issues: Selection Read tuples one by one – Much faster if we read a whole bunch of them together: caching SELECT * FROM Person WHERE SSN = 111222333; 20

21. Performance Issues: Join Quadratic I/O access – Very slow for large relations SELECT * FROM Person, PersonAddr WHERE Person.SSN = PersonAddr.SSN and Person.SSN = 111222333; 21

22. Performance Issues: Query Execution Two ways of executing the query – First join, then select – First select, then join much faster Query (execution) optimization. SELECT * FROM Person, PersonAddr WHERE Person.SSN = PersonAddr.SSN and Person.SSN = 111222333; 22

23. Reliability Update the name in person – Power outage is the operation done? – Disk crash Update Person SET Name = Smith WHERE Person.SSN = 111222333; 23

24. Probably not that many people download our RDBMS Lets redesign the components of our RDBMS 24

25. Database Implementation 25 Conceptual Design Physical Storage Schema Entity Relationship(ER) Model Relational Model Files and Indexes User Requirements Data storage

26. Random access versus sequential access Disk random access : Seek time + rotational latency + transfer time. Disk sequential access: reading blocks next to each other No seek time or rotational latency Much faster than random access 26

27. Units of data on physical device Fields: data items Records Blocks Files 27

28. Fields Fixed size – Integer, Boolean, … Variable length – Varchar, … – Null terminated – Size at the beginning of the string 28

29. Records: Sets of Fields Schema – Number of fields, types of fields, order, … Fixed format and length – Record holds only the data items Variable format and length – Record holds fields and their size, type, … information Range of formats in between 29

30. Record Header Pointer to the record schema ( record type) Record size Timestamp Other info … 30

31. Blocks Collection of records Reduces number of I/O access Different from OS blocks – Why should RDBMS manage its own blocks? – It knows the access pattern better than OS. Separating records in a block – Fixed size records: no worry! – Markers between records – Keep record size information in records or block header. 31

32. Spanned versus un-spanned Unspanned – Each records belongs to only one block Spanned – Records may store across multiple records – Saves space – The only way to deal with large records and fields: blob, image, … 32

33. Heap versus Sorted Files Heap files – There is not any order in the file – New blocks (records) are inserted at the end of file. Sorted files – Order blocks (and records) based on some key. – Physically contiguous or using links to the next blocks. 33

34. Average Cost of Data Operations Insertion – Heap files are more efficient. – Overflow areas for sorted files. Search for a record – Sorted files are more efficient. Search for a range of records – Sorted files are more efficient. Deletion – Heap files are more efficient – Although we find the record faster in the sorted file. 34

35. Indirection The address of a record on the disk Physical address – Device ID, Cylinder #, Track #, … Map physical addresses to logical addresses – Flexible in moving records for insertion and deletion – Costly lookup – Many options in between, tradeoff 35 Rec IDPhysical Address Logical address Physical address on disk

36. Block Header Data about block File, relation, DB IDs Block ID and type Record directory Pointer to free space Timestamp Other info … 36

37. Row and Column Stores We have talked about row store – All fields of a record are stored together. 37

38. Row and Column Stores We can store the fields in columns. – We can store SSNs implicitly. 38

39. Row versus column store Column store – Compact storage – Faster reads on data analysis and mining operations Row store – Faster writes – Faster reads for record access (OLTP) Further reading – Mike Stonebreaker, et al, C-Store, a column oriented DBMS, VLDB05. 39