1. Hybrid Buffer Pool The Good, the Bad and the Ugly
=tg= Thomas Grohser
Hybrid Buffer Pool The Good, the Bad and the Ugly

2. select * from =tg= where topic =
=tg= Thomas Grohser,
NTT Data
Senior Director Technical Solutions Architecture
Focus on SQL Server Security,
Performance Engineering,
Infrastructure and Architecture
Wrote some of
Close Relationship with
SQLCAT (SQL Server Customer Advisory Team)
SCAN (SQL Server Customer Advisory Network)
TAP (Technology Adoption Program)
Product Teams in Redmond
Active PASS member and
PASS Summit Speaker
Remark
SQL 4.21
First SQL Server ever used (1994)
SQL 6.0
First Log Shipping with failover
SQL 6.5
First SQL Server Cluster (NT4.0 + Wolfpack)
SQL 7.0
2+ billion rows / month in a single Table
SQL 2000
938 days with 100% availability
SQL 2000 IA64
First SQL Server on Itanium IA64
SQL 2005 IA64
First OLTP long distance database mirroring
SQL 2008 IA64
First Replication into mirrored databases
SQL 2008R2 IA64
SQL 2008R2 x64
First 256 CPUs & > STMT/sec
First Scale out > STMT/sec
First time 1.2+ trillion rows in a table
SQL 2012
> Transactions per second
> 1.3 Trillion Rows in a table
SQL 2014
> Transactions per second
Fully automated deploy and management
SQL 2016
AlwaysOn Automatic HA and DR, crossed the PB in storage
SQL 2017
In production 3 minutes after RTM, HA for Replication
3+ Trillion Rows and > 1 PB of data in single DB
SQL 2019
Can’t wait to push the limits even further

3. Shameless Plug This is my 49th SQL Saturday I am speaking at ...
The 50th will be the first I am also organizing …

4. Agenda How SQL Server IO Works Storage Technology Over Time
SQL Server Buffer Pool Features Over Time
Hybrid Buffer Pool
Questions

5. Learning How to Compare Stuff
Marketing

6. The Bad First ….
All features in this talk require
Enterprise Edition

7. SQL Server IO - Select SELECT … Query Processor Buffer Pool Data File

8. SQL Server IO - Update Update … Query Processor Buffer Pool Log Buffer
Log Writer
Log File
Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

9. SQL Server IO – Buffer Pool is Full
SELECT …
Query Processor
Buffer Pool P1 P4 P7 P9 P1 P3 P5 P6
P13
P33
P17
P28
P21
P67
P42
P34
P54
P78
P52
P54
P88
P49
P41
P77
P99
P29
P16
P37
P94
P59
P66
P38
Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

10. Legacy Storage Technology Over Time
40 MB in about 10 minutes
40 MB
600 kb/s
27 ms
5 MB/s
Hard Disk
RAID
SAN
Storage Array
1+ PB
8 GB/s
0.2 ms
2 GB/s
RAM SAN
Fusion IO
SSD
All Flash Array
1 PB in about 1 ½ days

11. Current Storage Options
NVMe Disk
PCIe Card
NVDIMM
PMEM – Optane DC Persistent Memory

12. Current Storage Options
NVMe & PCIe cards limited only by PCIe bus up to 16 TB NVMe up to 7 GB/s PCIe up to 14 GB/s PMEM up to (1…8) x (1…12) x (128…512) GB = up to 48 TB NVDIMM up to (1…8) x (1…12) x (8…16) GB = up to 1.5 TB
1 PB in about 2.5 hours
16 TB in about 40 minutes
16 TB in about 20 minutes
48 TB in about 300 seconds
1.5 TB in about 3 seconds
Latency measured in µs and ns

13. PMEM – Two Options
6 x 512 GB = 6144 GB
6 x 128 GB = GB
Memory Mode OS sees: 6144 GB RAM CPU sees: 6144 GB RAM with 768 GB L4 cache Application Direct Mode OS sees: 6144 GB DAX Storage 768 GB RAM
DO NOT USE FOR SQL or he will never talk to you again

14. Buffer Pool Features over time
SQL up to 2 GB
SQL k instead of 2k pages,
PAE up to 64 GB
SQL NUMA, IA64 on Itanium (2 TB)
Marry Go Round Piggyback Scan
SQL x64 (2 TB)
SQL Large Page Allocations, up to 4 TB
SQL Buffer pool extensions
SQL NVDIMM support for tail of the log
SQL Hybrid Buffer Pool

15. Buffer Pool Extensions
Data File
Buffer Pool
(L1)
Buffer Pool Extension
(L2) P1 P1 P1
Up to 32 x max server memory
Up to 16 x physical RAM
Unfortunate release time, during development of feature RAM and SSD
price fell and made is by and large obsolete on day one

16. NVDIMM – Tail of the log Update … Query Processor Buffer Pool
Log Buffer L1
Log Writer
Log File
Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

17. Hybrid Buffer Pool SELECT … Query Processor Buffer Pool Data File =
If the page is modified it is still copied to RAM and then written back to PMEM during checkpoint or eager write.
Data File P1 P1 P2 P2 P3 P3 P4 P4 P5 P5 P6 P6 …

18. Requirements for hybrid buffer pool
Server with PMEM
Windows Server 2019
SQL Server 2019
Volume on PMEM with Filesystem that supports DAX and formatted with 2MB cluster size
Data File that is multiple of 2MB in size on the volume
(create new or move existing)

19. TSQL Commands (SQL 2019 and up)
ALTER SERVER CONFIGURATION SET MEMORY_OPTIMIZED HYBRID_BUFFER_POOL = ON; SET MEMORY_OPTIMIZED HYBRID_BUFFER_POOL = OFF; ALTER DATABASE <databaseName> SET MEMORY_OPTIMIZED = ON; ALTER DATABASE <databaseName> SET MEMORY_OPTIMIZED = OFF; SELECT * FROM sys.configurations WHERE name = 'hybrid_buffer_pool'; SELECT name, is_memory_optimized_enabled FROM sys.databases;

20. Hybrid Buffer Pool Speed
Sorry Some Algebra and Calculus needed 
If 𝑥 = speed of RAM then 4𝑥≈ speed of PMEM and 16𝑥≈ speed of NVMe Disks
Size of 1st Subsequent Average becomes Data Scan Scans
< Buffer Pool 16𝑥+𝑥 𝑥 𝑥+𝑥+𝑥(𝑛−1) 𝑛 = (16+𝑛)𝑥 𝑛
→lim 𝑛→∞ (16+𝑛)𝑥 𝑛 =𝒙
> Buffer Pool 16𝑥+𝑥 16𝑥+𝑥 𝟏𝟕𝒙
< Hybrid Buffer Pool 𝑥 4𝑥 𝟒𝒙

21. Conclusion
If the workload does not fit into RAM but fits into PMEM it makes sense
RAM
RAM
PMEM

22.
Questions?