1. Vedran Kesegić

3. About me  M.Sc., FER, Zagreb  HRPro d.o.o. Before: Vipnet, FER  13+ years with SQL Server (since SQL 2000)  Microsoft Certified Master: SQL 2008  Author: XDetails – addin for SQL Studio  Hobby: Photography vedran.kesegic@gmail.com

4. What you will hear  What is new in SQL2014 ?  What is In-Memory OLTP ?  New DB engine and RAM structures  RAM - do we have Durability? How?  Memory-optimized tables  Memory-optimized types (and table variables)  New Indexes: hash index, range index  Natively compiled stored procedures  Demo

5. SQL Server 2014 (Hekaton) Released date: 1.4.2014 (RTM)  In-memory tables and table types (table variables)  Compiled procedures (eXtreme Transaction Processing)  Buffer Pool Extension (to SSD)  Delayed Durability (WRTELOG wait eliminator)  Rebuild Partition ONLINE  Partition-level statistics  Updatable Clustered Columnstore Index (INS/UPD/DEL)  Resource governor can now limit IO (IOPS)  AlwaysOn enhanced (8 replicas, better algorithms)  Standard Edition max. RAM raised from 64 to 128GB

6. What is In-Memory OLTP ?  Today servers: 1TB RAM, 32 CPU < $50 000  It is cheaper to buy RAM than pay CPU licences  NOT cached tables!  New database engine in SQL 2014  Built for speed, not comfort (lots of limitations)  Extreme concurrency (no locking – no blocking)  XTP (eXtreme Transaction Processing) – machine code compiled SP’s

7. How ?  Tables in RAM  Faster and smaller transaction log  MVCC (MultiVersion Concurrency Control)  No locks, no latches, no spinlocks  New indexes: HASH index, RANGE index  Natively compiled SP (machine code)

8. Overview

9. Memory-optimized Tables  Disk-based tables: 8KB pages, with rows in them  Memory-optimized tables: linked ROW structures in RAM, no pages!  ROW limited to 8060 bytes (no row overflow, no LOB)  DLL per table generated for table access and traverse rows In-memory ROW structure

10. Memory-optimized Tables No locks, latches, spinlocks - really? Really!  What about writing EndTs? Lock must exist!?  hardware CAS instructions (compare and switch)

11. Indexes  The only thing that connects rows into table (table MUST have min. ONE index!)  Max 8 indexes per table  Created together with the table (cannot be created later)  Not durably stored, built on db start  Two types: HASH indexes RANGE indexes

12. HASH Index  Array of pointers  Key value is hashed = location  BUCKET_COUNT = #distinct key values

13. HASH Index  Two indexes = Two pointers  Max 8 indexes per table (8 pointers)

14. RANGE Index  When no idea about bucket count  When you need a range scan (not just equality)  Bw-tree (new data struture)  Lock-free, latch-free variation of B-tree  Envisioned by Microsoft Research 2011  Variable-length pages, immutable  Pointers are not direct memory address: PID + Page Mapping Table = address

15. RANGE Index

16. Durability  Checkpoint – stored as FILESTREAM Data stream (INSERTS) Delta stream (DELETES – int filter)  Transaction log Written into regular transaction log Only REDO info (UNDO not needed) Uncommitted transactions are not logged Index operations are not logged Grouped into one big IO = very efficient  All indexes rebuilt on recovery, during table data loading into RAM

17. Durability SCHEMA_AND_DATA (default) SCHEMA_ONLY Schema is durable Rows are durable  Is in checkpoint files  Is logged (Tran. Log)  Is in backup  Survives AG failover  Blazing fast

18. Natively Compiled SP  TSQL (CREATE TABLE, SELECT,...) converted to C source code  C compiled to DLL (by Visual C compiler)  One machine-code DLL per SP  DLL is in RAM, not durably stored. Only metadata (TSQL) needed for DLL creation  DLL is generated (compiled) on every start of a database or instance, from metadata

19. Natively Compiled SP -- name = "...\xtp_t_ _.dll" SELECT m.name, m.description FROM sys.dm_os_loaded_modules m WHERE m.description='XTP Native DLL'  Stored in file system for debug purposes  C source code also

20. Natively Compiled SP  Marked „WITH NATIVE_COMPILATION”  SCHEMABINDING is required  EXECUTE AS is required (CALLER not supported. Only OWNER, SELF, user)  One ATOMIC block required: TRANSACTION ISOLATION LEVEL (SNAPSHOT, REPEATABLEREAD, SERIALIZABLE) LANGUAGE (sys.syslanguages)  Cannot partially succeed (rollbacked to savepoint at the beginning)

21. Natively Compiled SP Limitations:  „two.part” names must be used  Not all TSQL can be used  Restrictions are documented, will change http://msdn.microsoft.com/en- us/library/dn246937(v=sql.120).aspx http://msdn.microsoft.com/en- us/library/dn246937(v=sql.120).aspx

22. Natively Compiled SP CREATE PROCEDURE SP_Example(@id INT) WITH NATIVE_COMPILATION, SCHEMABINDING, EXECUTE AS OWNER AS BEGIN ATOMIC WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE = 'English' ) SELECT Name, Address, Phone FROM dbo.Customers WHERE Id = @id END GO

23. Interop TSQL  Interpreted TSQL accessing memory-optimized tables  Interop can access both memory-optimized and disk-based tables  Natively compiled can access only memory- optimized tables  NOT as fast as native (but still VERY fast!)  Useful as a step in migration to native

25. Out of presentation scope  Transaction isolation  Garbage collection  Checkpoint files automerging  New DMV’s (%XTP%) and other changes  Special Transaction log records (XTP) inspection

26. Summary  In-Memory OLTP is a major advance  100x faster on the same hardware is achievable  Speed has tradeoff in restrictions (less restrictions in the future)  Migrate only performance critical tables and procedures

27. Literature  Kalen Delaney: „SQL Server In-Memory OLTP Internals Overview for CTP2”, SQL 2014 CTP2 Whitepaper, October 2013  Microsoft (multiple authors): „ Hekaton: SQL Server’s Memory-Optimized OLTP Engine”