Chapter 111 Chapter 11: Hardware (Slides by Hector Garcia-Molina,

Slides:



Advertisements
Similar presentations
CPS216: Data-Intensive Computing Systems Data Access from Disks Shivnath Babu.
Advertisements

- Dr. Kalpakis CMSC Dr. Kalpakis 1 Outline In implementing DBMS we need to answer How should the system store and manage very large amounts of data?
CS 277 – Spring 2002Notes 21 CS 277: Database System Implementation Notes 02: Hardware Arthur Keller.
CS 245Notes 21 CS 245: Database System Principles Notes 02: Hardware Hector Garcia-Molina.
Data Storage John Ortiz. Lecture 17Data Storage2 Overview  Database stores data on secondary storage  Disk has distinct storage and access characteristics.
The Memory Hierarchy fastest, perhaps 1Mb
Storage. The Memory Hierarchy fastest, but small under a microsecond, random access, perhaps 2Gb Typically magnetic disks, magneto­ optical (erasable),
CS4432: Database Systems II Data Storage - Lecture 2 (Sections 13.1 – 13.3) Elke A. Rundensteiner.
1 Advanced Database Technology February 12, 2004 DATA STORAGE (Lecture based on [GUW ], [Sanders03, ], and [MaheshwariZeh03, ])
CPSC-608 Database Systems Fall 2008 Instructor: Jianer Chen Office: HRBB 309B Phone: Notes #6.
1 Chapter 6 Storage and Multimedia: The Facts and More.
Disk Drivers May 10, 2000 Instructor: Gary Kimura.
CS 4432lecture #31 CS4432: Database Systems II Lecture #3 Professor Elke A. Rundensteiner.
1 Storage Hierarchy Cache Main Memory Virtual Memory File System Tertiary Storage Programs DBMS Capacity & Cost Secondary Storage.
SECTIONS 13.1 – 13.3 Sanuja Dabade & Eilbroun Benjamin CS 257 – Dr. TY Lin SECONDARY STORAGE MANAGEMENT.
CS4432: Database Systems II Lecture 2 Timothy Sutherland.
CMSC424: Database Design Data Storage. Storage Hierarchy.
CPSC-608 Database Systems Fall 2009 Instructor: Jianer Chen Office: HRBB 309B Phone: Notes #5.
CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #5.
Physical Storage Organization. Advanced DatabasesPhysical Storage Organization2 Outline Where and How data are stored? –physical level –logical level.
CS 4432lecture #31 CS4432: Database Systems II Lecture #3 Using the Disk, and Disk Optimizations Professor Elke A. Rundensteiner.
1 CS222: Principles of Database Management Fall 2010 Professor Chen Li Department of Computer Science University of California, Irvine Notes 01.
1 Today I/O Systems Storage. 2 I/O Devices Many different kinds of I/O devices Software that controls them: device drivers.
SECTIONS 13.1 – 13.3 Sanuja Dabade & Eilbroun Benjamin CS 257 – Dr. TY Lin SECONDARY STORAGE MANAGEMENT.
CPSC 231 Secondary storage (D.H.)1 Learning Objectives Understanding disk organization. Sectors, clusters and extents. Fragmentation. Disk access time.
Secondary Storage Management Hank Levy. 8/7/20152 Secondary Storage • Secondary Storage is usually: –anything outside of “primary memory” –storage that.
1 Introduction to Computers Day 4. 2 Storage device A functional unit into which data can be –placed –retained(stored) –retrieved(accessed)
Introduction to Database Systems 1 The Storage Hierarchy and Magnetic Disks Storage Technology: Topic 1.
CS4432: Database Systems II Data Storage (Better Block Organization) 1.
Chapter 2 Data Storage How does a computer system store and manage very large volumes of data ?
Lecture 11: DMBS Internals
Chapter 3 Data Storage. Media Storage Main memory (Electronic Memory): Stores data currently being used Is made of semiconductor chips. Secondary Memory.
Physical Storage Organization. Advanced DatabasesPhysical Storage Organization2 Outline Where and How are data stored? –physical level –logical level.
CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 29 Database Systems II Secondary Storage.
1 Chapter 17 Disk Storage, Basic File Structures, and Hashing Chapter 18 Index Structures for Files.
Disks Chapter 5 Thursday, April 5, Today’s Schedule Input/Output – Disks (Chapter 5.4)  Magnetic vs. Optical Disks  RAID levels and functions.
1) Disk Storage, Basic File Structures, and Hashing This material is a modified version of the slides provided by Ramez Elmasri and Shamkant Navathe for.
Chapter 2. Data Storage Chapter 2.
2.1 Operating System Concepts Chapter 2: Computer-System Structures Computer System Operation Storage Structure Storage Hierarchy Hardware Protection General.
File Processing : Storage Media 2015, Spring Pusan National University Ki-Joune Li.
1 Data Storage (Chap. 11) Based on Hector Garcia-Molina’s slides.
Physical Storage Organization. Advanced DatabasesPhysical Storage Organization2 Outline Where and How data are stored? –physical level –logical level.
Programming for GCSE Topic 5.1: Memory and Storage T eaching L ondon C omputing William Marsh School of Electronic Engineering and Computer Science Queen.
11.1Database System Concepts. 11.2Database System Concepts Now Something Different 1st part of the course: Application Oriented 2nd part of the course:
DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.
CS4432: Database Systems II Data Storage 1. Storage in DBMSs DBMSs manage large amounts of data How does a DBMS store and manage large amounts of data?
CPSC-608 Database Systems Fall 2015 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #5.
CS 101 – Sept. 28 Main vs. secondary memory Examples of secondary storage –Disk (direct access) Various types Disk geometry –Flash memory (random access)
DBMS 2001Notes 2: Hardware1 Principles of Database Management Systems Pekka Kilpeläinen (after Stanford CS245 slide originals by Hector Garcia-Molina,
DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.
CPSC 231 Secondary storage (D.H.)1 Learning Objectives Understanding disk organization. Sectors, clusters and extents. Fragmentation. Disk access time.
1 CSE232A: Database System Principles Hardware. Data + Indexes Database System Architecture Query ProcessingTransaction Management SQL query Parser Query.
CPS216: Advanced Database Systems Notes 03: Data Access from Disks Shivnath Babu.
1 Lecture 16: Data Storage Wednesday, November 6, 2006.
CS 245Notes 21 Database System Principles Notes 02: Hardware.
1 Query Processing Exercise Session 1. 2 The system (OS or DBMS) manages the buffer Disk B1B2B3 Bn … … Program’s private memory An application program.
CS422 Principles of Database Systems Disk Access Chengyu Sun California State University, Los Angeles.
Data Storage and Querying in Various Storage Devices.
File organization Secondary Storage Devices Lec#7 Presenter: Dr Emad Nabil.
File Organization Record Storage and Primary File Organization
TYPES OF MEMORY.
Chapter 2: Computer-System Structures
Storage and Disks.
Computer Science 210 Computer Organization
CS 554: Advanced Database System Notes 02: Hardware
CPSC-608 Database Systems
Lecture 11: DMBS Internals
CSE 451: Operating Systems Autumn 2004 Secondary Storage
CPS216: Advanced Database Systems Notes 04: Data Access from Disks
CS 245: Database System Principles Notes 02: Hardware
Presentation transcript:

Chapter 111 Chapter 11: Hardware (Slides by Hector Garcia-Molina,

Chapter 112 Outline Hardware: Disks Access Times Optimizations Other Topics: –Storage costs –Using secondary storage –Disk failures

Chapter 113 The Big Picture: DBMS Data Storage

Chapter 114 P M Typical Computer Secondary Storage...

Chapter 115 Processor Fast, slow, reduced instruction set, with cache, pipelined… Speed: 100  500  1000 MIPS Memory Fast, slow, non-volatile, read-only,… Access time:  sec. 1  s  1 ns

Chapter 116 Secondary storage Many flavors: - Disk: Floppy (hard, soft) Removable Packs Winchester Ram disks Optical, CD-ROM… Arrays - TapeReel, cartridge Robots

Chapter 117 Focus on: “Typical Disk” Terms: Platter, Head, Cylinder, Track Sector (physical), Block (logical), Gap …

Chapter 118 Top View

Chapter 119 “Typical” Numbers Diameter: 1 inch  15 inches Cylinders:100  2000 Surfaces:1 (CDs)  (Tracks/cyl) 2 (floppies)  30 Sector Size:512B  50K Capacity:360 KB (old floppy)  30 GB (I use)

Chapter 1110 Disk Access Time block x in memory ? I want block X

Chapter 1111 Time = Seek Time + Rotational Delay + Transfer Time + Other

Chapter 1112 Rotational Delay Head Here Block I Want

Chapter 1113 Average Rotational Delay R = 1/2 revolution “typical” R = 8.33 ms (3600 RPM)

Chapter 1114 Transfer Rate: t “typical” t: 1  3 MB/second transfer time: block size t

Chapter 1115 Other Delays CPU time to issue I/O Contention for controller Contention for bus, memory “Typical” Value: 0

Chapter 1116 So far: Random Block Access What about: Reading “Next” block?

Chapter 1117 If we do things right (e.g., Double Buffer, Stagger Blocks…) Time to get = Block Size + Negligible block t - skip gap - switch track - once in a while, next cylinder

Chapter 1118 Rule ofRandom I/O: Expensive Thumb Sequential I/O: Much less Ex:1 KB Block »Random I/O:  20 ms. »Sequential I/O:  1 ms.

Chapter 1119 Cost for Writing similar to Reading …. unless we want to verify! need to add (full) rotation + Block size t

Chapter 1120 To Modify a Block? To Modify Block: (a) Read Block (b) Modify in Memory (c) Write Block [(d) Verify?]

Chapter 1121 Block Address: Physical Device Cylinder # Surface # Sector

Chapter 1122 Outline Hardware: Disks Access Times Optimizations Other Topics –Storage Costs –Using Secondary Storage –Disk Failures here

Chapter 1123 Optimizations (in controller or O.S.) Double Buffering Disk Scheduling Algorithms: sec –e.g., elevator algorithm

Chapter 1124 Double Buffering Problem: Have a File » Sequence of Blocks B1, B2 Have a Program » Process B1 » Process B2 » Process B3...

Chapter 1125 Single Buffer Solution (1) Read B1  Buffer (2) Process Data in Buffer (3) Read B2  Buffer (4) Process Data in Buffer...

Chapter 1126 SayP = time to process/block R = time to read in 1 block n = # blocks Single buffer time = n(P+R)

Chapter 1127 Double Buffering Memory: Disk: ABCDGEFA B done process A C B done

Chapter 1128 Say P  R What is processing time? P = Processing time/block R = IO time/block n = # blocks Double buffering time = R + nP Single buffering time = n(R+P)

Chapter 1129 Block Size Selection? Big Block  Amortize I/O Cost Big Block  Read in more useless stuff! and takes longer to read Unfortunately...

Chapter 1130 Trend As memory prices drop, blocks get bigger... Trend

Chapter 1131 Storage Cost access time (sec) cache electronic main electronic secondary magnetic optical disks tape optical disks tape typical capacity (bytes) from Gray & Reuter

Chapter 1132 Storage Cost access time (sec) cache electronic main electronic secondary magnetic optical disks tape optical disks tape dollars/MB from Gray & Reuter

Chapter 1133 Using secondary storage effectively Example: Sorting data on disk Conclusion: –I/O costs dominate –Design algorithms to reduce I/O Also: How big should blocks be?

Chapter 1134 Disk Failures (Sec 11.6) Partial  Total Intermittent  Permanent

Chapter 1135 Coping with Disk Failures Detection –e.g. Checksum Correction  Redundancy

Chapter 1136 At what level do we cope? Single Disk –e.g., Error Correcting Codes Disk Array Logical Physical

Chapter 1137 Operating System e.g., Stable Storage Logical BlockCopy A Copy B

Chapter 1138 Database System e.g., Log Current DBLast week’s DB

Chapter 1139 Summary Secondary storage, mainly disks I/O times I/Os should be avoided, especially random ones….. Summary

Chapter 1140 Outline Hardware: Disks Access Times Optimizations Other Topics –Storage Costs –Using Secondary Storage –Disk Failures here