1. Logging in Flash-based Database Systems Lu Zeping 2009-06-27

2. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 2

3. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 3

4. Characteristics of flash memory No in-place update 4 × Out – place update

5. Characteristics of flash memory No mechanical latency 5 Media Random-to-Sequential Ratio Read workloadWrite workload Magnetic Disk4.3-12.34.5-10.0 Flash1.1-1.22.4-14.2

6. Characteristics of flash memory Asymmetric speed of read/write MediaReadWriteErase Magnetic disk 12.7ms (2KB) 13.7ms (2KB) N/A Flash80μs (2KB) 200μs (2KB) 1.5ms (128KB) 6

7. The flash memory organization Device : 1G Block : 1024 Page : 64 2K Bytes64 Bytes Page Register 7

8. The flash memory organization Device : 1G Block : 1024 Page : 64 2K Bytes64 Bytes Page Register 8

9. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 9

10. Design of IPL IPL ： In-Page Logging 10S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 Database Buffer in-memory data page (8KB) in-memory log sector (512B)

11. Design of IPL S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 11 Flash Memory log area (8KB): 16 sectors Erase unit (Block): 128KB 15 data pages (8KB each) …. ……

12. IPL Write S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 12 Database Buffer in-memory data page (8KB) in-memory log sector (512B) Flash Memory log area (8KB): 16 sectors Erase unit (Block): 128KB 15 data pages (8KB each) …. ……

13. IPL Read S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 13 Database Buffer in-memory data page (8KB) in-memory log sector (512B) Flash Memory log area (8KB): 16 sectors Erase unit (Block): 128KB 15 data pages (8KB each) …. ……

14. IPL Read S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 14 Database Buffer in-memory data page (8KB) in-memory log sector (512B) Flash Memory log area (8KB): 16 sectors Erase unit (Block): 128KB 15 data pages (8KB each) …. ……

15. IPL Merge 15 S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 Flash Memory …. …… …. Merge ….

16. IPL Merge 16 S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007 Flash Memory …. …… …. Merge ….

17. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 17

18. SSD SSD = Flash memory + FTL （ Flash Translation Layer ） 18

19. To be used in…… Accessing data in secondary storage is the main source of bottlenecks in database processing. Database table, index, log, temporary table and rollback segment could be created on physically separate storage devices.(SSD+ HDD) Log records are always appended to the end of log. 19

20. Simple SQL Transactions B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.20 TPS: transactions-per-seconds This embedded SQL program is multi-threaded and simulates concurrent transactions. Each thread updates a single record and commits, and repeats this cycle of update and commit continuously.

21. TPC-B benchmark performance B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.21 We used TPC-B to stress-test the log storage part of the commercial database server. Due to the group commit mechanism, the average size of a single log write was slightly different between the two storage media.

22. TPC-B benchmark performance B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.22 In the case of SSD, the bottleneck in transaction throughput is the processing speed of CPU, while it was not in the case of disk.

23. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 23

24. USB flash memory advantages USB ports are widely available. USB bus bandwidth is good for logging. The hot-plug capability allows easy replacement of drives. The price of an individual drive is low. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.24

25. Flash devices and disk performance 25Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

26. Four key challenges Efficiently exploiting an array of flash drives. Coping with large variance of write latencies. Efficient recovery processing. Combining USB flash drives with disks for better logging and recovery performance. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 26

27. Exploiting an array of flash drives What’s wrong with the conventional array organization? Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 27 In-memory log buffer …… Devices ： N the stripe unit size is S Data at address A are found on the k-th device, where k = ⌊ A/S ⌋ mod N.

28. Unconventional array design Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 28 In-memory log buffer Request queue Worker …… LSN ： Log Sequence Number

29. Coping with outlier Writes Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 29 Sequential writes have large variance Outlier prediction is difficult.

30. Outlier Detection and Hiding Measure the average sequential write latencies for different request sizes for a device. Upon detecting an outlier, we would like to reissue the same request to a different ready device. Reserve an area on each device for outlier hiding ， during recovery, treat these areas as “pseudo” devices. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 30

31. Outlier Detection and Hiding 31 Worker In-memory log buffer Worker Measure the average sequential write latencies for different request sizes for a device. Longer than twice the average latency? Reissue the same request to a different ready device. Non-decreasing LSN? Outlier hiding area

32. Efficient recovery processing Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 32 Two main steps of recovery processing (i)Locating the log record corresponding to the LSN in the last checkpoint. (ii) Scanning the log sequentially until the crash point.

33. Flash devices and disk performance 33Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

34. FlashLogging design Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 34 Request queue Worker …… In-memory log buffer

35. Using a disk for better performance Our goal to optionally include a disk : The disk can serve as a large capacity archival device to complement our online logging design. A magnetic disk may provide larger recovery scan bandwidth. Magnetic disks are competitive to USB flash drives for writes that are 32KB or larger. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09. 35

36. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 36

37. Several recovery technologies LogGroup commit PTLogShadow page TxFlash Update Pattern In-Place Out-Place Concurrency Granularity Record PageSegment/ file Page Atomicity Type Write atomicity Propagate atomicity Commit Overhead HighLow HighLow Rollback Overhead High None High Logging I/O Type Synchronous None GC overhead None Low High 37

38. Outline Characteristics of flash memory Log records in different kinds of Flash devices – In Flash chips - IPL – In SSD – In USB – FlashLogging Future work – Recovery processing – Experiments of PTLog 38

39. Experiments of PTLog Berkeley DB 39 Linux Nandsim nand_base.c 、 nand_bbt.c 、 nand_ids.c 、 nand_ecc.c 、 nandsim.c 、 diskonchip.c 、 cafe_nand.c 、 s3c2410.c

40. 40 Thank you ！ Q&A ？