Hybrid Buffer Pool The Good, the Bad and the Ugly =tg= Thomas Grohser Hybrid Buffer Pool The Good, the Bad and the Ugly

select * from =tg= where topic = =tg= Thomas Grohser, NTT Data Senior Director Technical Solutions Architecture email: tg@grohser.com Focus on SQL Server Security, Performance Engineering, Infrastructure and Architecture Wrote some of Close Relationship with SQLCAT (SQL Server Customer Advisory Team) SCAN (SQL Server Customer Advisory Network) TAP (Technology Adoption Program) Product Teams in Redmond Active PASS member and PASS Summit Speaker @@Version Remark SQL 4.21 First SQL Server ever used (1994) SQL 6.0 First Log Shipping with failover SQL 6.5 First SQL Server Cluster (NT4.0 + Wolfpack) SQL 7.0 2+ billion rows / month in a single Table SQL 2000 938 days with 100% availability SQL 2000 IA64 First SQL Server on Itanium IA64 SQL 2005 IA64 First OLTP long distance database mirroring SQL 2008 IA64 First Replication into mirrored databases SQL 2008R2 IA64 SQL 2008R2 x64 First 256 CPUs & >500.000 STMT/sec First Scale out > 1.000.000 STMT/sec First time 1.2+ trillion rows in a table SQL 2012 > 220.000 Transactions per second > 1.3 Trillion Rows in a table SQL 2014 > 400.000 Transactions per second Fully automated deploy and management SQL 2016 AlwaysOn Automatic HA and DR, crossed the PB in storage SQL 2017 In production 3 minutes after RTM, HA for Replication 3+ Trillion Rows and > 1 PB of data in single DB SQL 2019 Can’t wait to push the limits even further

Shameless Plug This is my 49th SQL Saturday I am speaking at ... The 50th will be the first I am also organizing …

Agenda How SQL Server IO Works Storage Technology Over Time SQL Server Buffer Pool Features Over Time Hybrid Buffer Pool Questions

Learning How to Compare Stuff Marketing

The Bad First …. All features in this talk require Enterprise Edition

SQL Server IO - Select SELECT … Query Processor Buffer Pool Data File

SQL Server IO - Update Update … Query Processor Buffer Pool Log Buffer Log Writer Log File Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

SQL Server IO – Buffer Pool is Full SELECT … Query Processor Buffer Pool P1 P4 P7 P9 P1 P3 P5 P6 P13 P33 P17 P28 P21 P67 P42 P34 P54 P78 P52 P54 P88 P49 P41 P77 P99 P29 P16 P37 P94 P59 P66 P38 Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

Legacy Storage Technology Over Time 40 MB in about 10 minutes 40 MB 600 kb/s 27 ms 5 MB/s Hard Disk RAID SAN Storage Array 1+ PB 8 GB/s 0.2 ms 2 GB/s RAM SAN Fusion IO SSD All Flash Array 1 PB in about 1 ½ days

Current Storage Options NVMe Disk PCIe Card NVDIMM PMEM – Optane DC Persistent Memory

Current Storage Options NVMe & PCIe cards limited only by PCIe bus up to 16 TB NVMe up to 7 GB/s PCIe up to 14 GB/s PMEM up to (1…8) x (1…12) x (128…512) GB = up to 48 TB NVDIMM up to (1…8) x (1…12) x (8…16) GB = up to 1.5 TB 1 PB in about 2.5 hours 16 TB in about 40 minutes 16 TB in about 20 minutes 48 TB in about 300 seconds 1.5 TB in about 3 seconds Latency measured in µs and ns

PMEM – Two Options 6 x 512 GB = 6144 GB 6 x 128 GB = 768 GB Memory Mode OS sees: 6144 GB RAM CPU sees: 6144 GB RAM with 768 GB L4 cache Application Direct Mode OS sees: 6144 GB DAX Storage 768 GB RAM DO NOT USE FOR SQL or he will never talk to you again

Buffer Pool Features over time SQL 4.21 up to 2 GB SQL 7.0 8k instead of 2k pages, PAE up to 64 GB SQL 2000 NUMA, IA64 on Itanium (2 TB) Marry Go Round Piggyback Scan SQL 2005 x64 (2 TB) SQL2008 Large Page Allocations, up to 4 TB SQL2014 Buffer pool extensions SQL2016 NVDIMM support for tail of the log SQL2019 Hybrid Buffer Pool

Buffer Pool Extensions Data File Buffer Pool (L1) Buffer Pool Extension (L2) P1 P1 P1 Up to 32 x max server memory Up to 16 x physical RAM Unfortunate release time, during development of feature RAM and SSD price fell and made is by and large obsolete on day one

NVDIMM – Tail of the log Update … Query Processor Buffer Pool Log Buffer L1 Log Writer Log File Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

Hybrid Buffer Pool SELECT … Query Processor Buffer Pool Data File = If the page is modified it is still copied to RAM and then written back to PMEM during checkpoint or eager write. Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

Requirements for hybrid buffer pool Server with PMEM Windows Server 2019 SQL Server 2019 Volume on PMEM with Filesystem that supports DAX and formatted with 2MB cluster size Data File that is multiple of 2MB in size on the volume (create new or move existing)

TSQL Commands (SQL 2019 and up) ALTER SERVER CONFIGURATION SET MEMORY_OPTIMIZED HYBRID_BUFFER_POOL = ON; SET MEMORY_OPTIMIZED HYBRID_BUFFER_POOL = OFF; ALTER DATABASE <databaseName> SET MEMORY_OPTIMIZED = ON; ALTER DATABASE <databaseName> SET MEMORY_OPTIMIZED = OFF; SELECT * FROM sys.configurations WHERE name = 'hybrid_buffer_pool'; SELECT name, is_memory_optimized_enabled FROM sys.databases;

Hybrid Buffer Pool Speed Sorry Some Algebra and Calculus needed  If 𝑥 = speed of RAM then 4𝑥≈ speed of PMEM and 16𝑥≈ speed of NVMe Disks Size of 1st Subsequent Average becomes Data Scan Scans < Buffer Pool 16𝑥+𝑥 𝑥 16𝑥+𝑥+𝑥(𝑛−1) 𝑛 = (16+𝑛)𝑥 𝑛 →lim 𝑛→∞ (16+𝑛)𝑥 𝑛 =𝒙 > Buffer Pool 16𝑥+𝑥 16𝑥+𝑥 𝟏𝟕𝒙 < Hybrid Buffer Pool 4𝑥 4𝑥 𝟒𝒙

Conclusion If the workload does not fit into RAM but fits into PMEM it makes sense RAM RAM PMEM

tg@grohser.com Questions?