Statistics for beginners – In-Memory OLTP Lies, a blatant lie, statistics. Demystification of the statistics. In-Memory OLTP Андрій Зробок azrobok@gmail.com

Database creation USE [master] GO /****** Object: Database [IMstat] Script Date: 3/4/2015 3:58:01 PM ******/ CREATE DATABASE [IMstat] CONTAINMENT = NONE ON PRIMARY ( NAME = N'IMstat', FILENAME = N'D:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER2014\MSSQL\DATA\IMstat.mdf' , SIZE = 5120KB , MAXSIZE = UNLIMITED, FILEGROWTH = 1024KB ) LOG ON ( NAME = N'IMstat_log', FILENAME = N'D:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER2014\MSSQL\DATA\IMstat_log.ldf' , SIZE = 2048KB , MAXSIZE = 2048GB , FILEGROWTH = 10%) ALTER DATABASE [IMstat] SET COMPATIBILITY_LEVEL = 120 -- enable for in-memory OLTP - change file path as needed ALTER DATABASE IMstat ADD FILEGROUP IMstat_memory CONTAINS MEMORY_OPTIMIZED_DATA GO ALTER DATABASE IMstat ADD FILE (name='IMstat_memory1', filename='D:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER2014\MSSQL\DATA\IMstat_memory1.mdf') TO FILEGROUP IMstat_memory 2 | 1/14/2019 | Statistics for beginners - 2

Database creation 3 | 1/14/2019 | Statistics for beginners - 2

Table creation -- non-clustered hash (equal operation) CREATE TABLE [dbo].[TransactionHistory]( [TransactionID] [int] NOT NULL primary key nonclustered hash with (bucket_count=1000000), [ProductID] [int] NOT NULL index ix_ProductId nonclustered hash with (bucket_count=1000), [ReferenceOrderID] [int] NOT NULL, [ReferenceOrderLineID] [int] NOT NULL , [TransactionDate] [datetime] NOT NULL , [TransactionType] [nchar](1) NOT NULL, [Quantity] [int] NOT NULL, [ActualCost] [money] NOT NULL, [ModifiedDate] [datetime] NOT NULL) WITH (MEMORY_OPTIMIZED=ON) GO -- auto-generated script CREATE TABLE [dbo].[TransactionHistory] ( [TransactionID] [int] NOT NULL, [ProductID] [int] NOT NULL [ReferenceOrderID] [int] NOT NULL, [ReferenceOrderLineID] [int] ,NOT NULL, [TransactionDate] [datetime] NOT NULL, [TransactionType] [nchar](1) COLLATE Latin1_General_CI_AS NOT NULL, [Quantity] [int] NOT NULL, [ActualCost] [money] NOT NULL, [ModifiedDate] [datetime] NOT NULL, INDEX [ix_ProductId] NONCLUSTERED HASH ([ProductID])WITH ( BUCKET_COUNT = 1024), PRIMARY KEY NONCLUSTERED HASH ([TransactionID])WITH ( BUCKET_COUNT = 1048576) )WITH ( MEMORY_OPTIMIZED = ON , DURABILITY = SCHEMA_AND_DATA ) GO -- non-clustered b-tree (aggregation/diapason -- operation) CREATE TABLE [dbo].[TransactionHistory_1]( [TransactionID] [int] NOT NULL primary key nonclustered hash with (bucket_count=1000000), [ProductID] [int] NOT NULL index ix_ProductId nonclustered , [ReferenceOrderID] [int] NOT NULL, [ReferenceOrderLineID] [int] NOT NULL , [TransactionDate] [datetime] NOT NULL , [TransactionType] [nchar](1) NOT NULL, [Quantity] [int] NOT NULL, [ActualCost] [money] NOT NULL, [ModifiedDate] [datetime] NOT NULL ) WITH (MEMORY_OPTIMIZED=ON) GO A memory-optimized table can have up to 8 indexes, including the index created with the primary key. 4 | 1/14/2019 | Statistics for beginners - 2

Btw. Tables: Disk based VS in-memory Disk primary, memory secondary -“live” on disk - Data save as page (8k) - Buffer pull has the same structure (mirror of disk) - Blocking for logical data protection - Latches, spinlock for memory protection (waits) Memory primary, disk secondary - “live” in memory - Data save as linked list (one index required) of records - Saved on disk as file stream - No blocking (versioning) - No latches, no spinlock - No waits Must be loaded into memory Can be saved on the disk 5 | 1/14/2019 | Statistics for beginners - 2

Btw. Tables: Disk based VS in-memory 6 | 1/14/2019 | Statistics for beginners - 2

Data loading -- Productid - b-tree index USE [IMstat] GO SELECT [TransactionID] ,[ProductID] ,[ReferenceOrderID] ,[ReferenceOrderLineID] ,[TransactionDate] ,[TransactionType] ,[Quantity] ,[ActualCost] ,[ModifiedDate] into #t1 FROM [AdventureWorks2014].[Production].[TransactionHistory] -- ProductID – hash index INSERT INTO [dbo].[TransactionHistory] ([TransactionID] ,[ProductID] ,[ReferenceOrderID] ,[ReferenceOrderLineID] ,[TransactionDate] ,[TransactionType] ,[Quantity] ,[ActualCost] ,[ModifiedDate]) SELECT [TransactionID] ,[ModifiedDate] FROM #t1 -- Productid - b-tree index insert into [dbo].[TransactionHistory_1] select * from [dbo].[TransactionHistory] go Msg 41317, Level 16accesses memory optimized tables or natively compiled proc, State 5, Line 4 A user transaction that edures cannot access more than one user database or databases model and msdb, and it cannot write to master. 7 | 1/14/2019 | Statistics for beginners - 2

Default Statistics select * from [dbo].[TransactionHistory] where productid = 324 go -- SQL Server assumes that the table will have a number of rows equal to the number of buckets on the index -- No auto update statistics DBCC SHOW_STATISTICS ('[dbo].[TransactionHistory]', ix_ProductID) WITH STAT_HEADER go 8 | 1/14/2019 | Statistics for beginners - 2

Update Statistics 9 | 1/14/2019 | Statistics for beginners - 2 UPDATE STATISTICS [dbo].[TransactionHistory] WITH FULLSCAN, NORECOMPUTE -- fullscan - Memory-Optimized tables don't support sampled statistics -- norecompute - Memory-Optimized tables do not automatically update statistics DBCC SHOW_STATISTICS ('[dbo].[TransactionHistory]', ix_ProductID) dbcc freeproccache go select * from [dbo].[TransactionHistory] where productid = 321 9 | 1/14/2019 | Statistics for beginners - 2

Aggregation (hash vs b-tree) dbcc freeproccache go select productid, count(*) from [dbo].[TransactionHistory] where productid between 802 and 850 group by productid select productid, count(*) from [dbo].[TransactionHistory_1] where productid between 802 and 850 group by productid 10 | 1/14/2019 | Statistics for beginners - 2

Diapason (hash vs b-tree) dbcc freeproccache go select * from [dbo].[TransactionHistory] where productid between 802 and 850 select * from [dbo].[TransactionHistory_1] 11 | 1/14/2019 | Statistics for beginners - 2

Without index: does not created > does not used? -- non indexed column CREATE STATISTICS sta_referenceorderid ON [dbo].[TransactionHistory_1] (referenceorderid) WITH FULLSCAN, NORECOMPUTE -- disk based table (without any indexes and constraints) select * into dbo.transactionhistory_2 from [AdventureWorks2014].production.transactionhistory dbcc freeproccache dbcc DROPCLEANBUFFERS set statistics time on go -- in memory OLTP table print 'in-memory' select productid, count(*) from dbo.transactionhistory_1 where ReferenceOrderID = 53465 group by productid -- disk based table print 'disc based' select productid, count(*) from dbo.transactionhistory_2 where ReferenceOrderID = 53465 group by productid 12 | 1/14/2019 | Statistics for beginners - 2

Without index: does not created > does not used? DBCC execution completed. If DBCC printed error messages, contact your system administrator. SQL Server parse and compile time: CPU time = 0 ms, elapsed time = 2 ms. in-memory SQL Server Execution Times: CPU time = 0 ms, elapsed time = 0 ms. (72 row(s) affected) (1 row(s) affected) SQL Server Execution Times: CPU time = 31 ms, elapsed time = 70 ms. disc based SQL Server Execution Times: CPU time = 0 ms, elapsed time = 61 ms. SQL Server parse and compile time: CPU time = 0 ms, elapsed time = 0 ms. 13 | 1/14/2019 | Statistics for beginners - 2

Without index: does not created > does not used? 14 | 1/14/2019 | Statistics for beginners - 2

Natively Compiled Stored Procedures CREATE PROCEDURE dbo.get_transactionHistory ( @productid int ) WITH NATIVE_COMPILATION, SCHEMABINDING, EXECUTE AS OWNER AS BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE = N'us_english') SELECT transactionID, transactiondate, transactiontype, quantity, actualcost FROM [dbo].[TransactionHistory] WHERE productid = @productid END GO Natively Compiled Stored Procedures do not use the plan cache. The query plan is made at compile time when the procedure's DLL is created. So to use a new plan on a natively compiled stored procedure you must drop and recreate the natively compiled stored procedure in question. 15 | 1/14/2019 | Statistics for beginners - 2

Natively Compiled Stored Procedures SET SHOWPLAN_XML ON GO exec dbo.get_transactionHistory 321 go SET SHOWPLAN_XML OFF Nested Loops is the only join operator supported in natively compiled stored procedures. All plans that contain joins will use the Nested Loops operator, even if the plan for same query executed as interpreted Transact-SQL contains a hash or merge join. 16 | 1/14/2019 | Statistics for beginners - 2

Guidelines for Statistics To ensure that the query optimizer has up-to-date statistics when creating query plans, deploy memory-optimized tables using these five steps: Create tables and indexes. Indexes are specified inline in the CREATE TABLE statements. Load data into the tables. Update statistics on the tables. Create stored procedures that access the tables. Run the workload, which can contain a mix of natively compiled and interpreted Transact-SQL stored procedures, as well as ad hoc batches. Creating natively compiled stored procedures after you load the data and update the statistics ensures that the optimizer has statistics available for the memory-optimized tables. This will ensure efficient query plans when the procedure is compiled. https://msdn.microsoft.com/en-us/library/dn205319.aspx https://msdn.microsoft.com/en-us/library/dn232522.aspx 17 | 1/14/2019 | Statistics for beginners - 2

Statistics for beginners – in-Memory OLTP Performance depends on correct index type but not up-to-date statistics Statistics on non-indexed column does not matter Q&A The end 18 | 1/14/2019 | Statistics for beginners -2