Memory-optimized Table Filegroup Data Filegroup SQL Server.exe Hekaton Engine: Memory_optimized Tables & Indexes TDS Handler and Session Management Natively Compiled SPs and Schema Buffer Pool for Tables & Indexes Proc/Plan cache for ad-hoc T- SQL and SPs Client App Transaction Log Query Interop Non-durable Table T1 T4 T3 T2 T1 T4 T3 T2 T1 T4 T3 T2 T1 T4 T3 T2 Tables Indexes Interpreter for TSQL, query plans, expressions T1 T4 T3 T2 T1 T4 T3 T2 Checkpoint & Recovery Access Methods Parser, Catalog, Algebrizer, Optimizer Hekaton Compiler Hekaton Component Key Existing SQL Component Generated.dll
Memory-optimized Table Filegroup Data Filegroup SQL Server.exe Hekaton Engine for Memory_optimized Tables & Indexes TDS Handler and Session Management Natively Compiled SPs and Schema Buffer Pool for Tables & Indexes Proc/Plan cache for ad- hoc T-SQL and SPs Client App Transaction Log Query Interop Interpreter for TSQL, query plans, expressions Access Methods Parser, Catalog, Algebrizer, Optimizer Hekaton Compiler 10-30x more efficient Reduced log bandwidth & contention. Log latency remains Checkpoints are background sequential IO No improvements in communication stack, parameter passing, result set generation Hekaton Component Key Existing SQL Component Generated.dll
OperationFactor faster (slower) than regular SQL Comments InteropNative Select count(*) 1 (2.5)= No clustered index scan in Hekaton Hash Join 1 (1.3)N/A Uses index scan Nested-loop Join 1 4.0N/A Probes into hash index Single-row selects 1 1.340 SP doing selects in loop Single-row selects 1 1.217 Native compiled SP calls SQL’s rand() Single-row updates 1 N/A10 SP doing update in loop Bwin Session State6 Version M4 Expectation for OLTP workloads Advantage of pushing work to SPs Interop targets app migration, not perf (1) 1 million rows accessed in single query or SP
T-SQL Compiled to Machine Code T-SQL compiled to machine code via C code generator and VC Invoking a procedure is just a DLL entry-point Aggressive optimizations @ compile-time Stalling CPU clock rate Hardware trends Efficient, business- logic processing Customer Benefits Hekaton Tech Pillars Drivers Native Compiled Stored Procedures Non-Native Compilation Performance High. Significantly less instructions to go through No different than T-SQL calls in SQL Server today Migration StrategyApplication changes – development overhead Easier app migration as can still access Memory Optimized (MO) tables Access to objectsCan only interact with Memory Optimized tables All Objects. Access for transactions across MO and b-tree tables Support for T-SQL Constructs Limited.T-SQL surface area (limit on MO interaction) Optimization/Stats and Query Plan Statistics utilized at CREATE -> Compile time Statistics updates can be utilized to modify plan at runtime FlexibilityLimited (e.g., no ALTER procedure, compile-time isolation level) Ad-hoc query patterns
Memory Optimized Data Filegroup Range 300- 399 Range 100- 199 Range 200- 299 Range 400- 499 Range 500- offline checkpoint Thread A transaction with a commit timestamp of 600 inserts one new row and deletes rows inserted by transactions with a commit timestamp of 150, 250 and 420
Memory Optimized Data Filegroup Key Range 100- 199 Range 200- 299 Range 300- 399 Range 400- 499 Range 100- 199 Range 200- 399 Range 400- 499 Range 500- 599 Data file with rows generated in timestamp range a-b Delta file with IDs of Deleted Rows Merge 200-399 Range 200- 299 Range 300- 399 Deleted Files
sys.dm_db_xtp_checkpointReturns database that has one or more IM objects sys.dm_db_xtp_checkpoint_filesDisplays information about checkpoint files sys.dm_db_xtp_hash_index_statsUseful for understanding and tuning the bucket counts and duplicates for index key sys.dm_db_xtp_index_statsReports statistics about scans on an index sys.dm_db_xtp_memory_consumersReports the database-level memory consumers in the IM database engine. sys.dm_db_xtp_object_statsReports statistics about operations on a memory optimized object. sys.dm_db_xtp_table_memory_statsReturns memory usage statistics for each IM table (user and system) in the current database. sys.dm_db_xtp_transactionsReports the active transactions in the IM database engine. sys.dm_xtp_consumer_memory_usageReports memory usage for all memory consumers including @database level and @system level. sys.dm_xtp_gc_statsReports information about the current behavior of the IM garbage-collection process. sys.dm_xtp_system_memory_consumersReports information about memory usage. sys.dm_xtp_threadsReports the threads that the IM database engine has started internally. sys.dm_xtp_transaction_statsReports statistics about transactions that have run since the server started.
New or updated property, system view, stored procedures, or DMV Change OBJECTPROPERTYEXNew property: TableIsMemoryOptimized. SERVERPROPERTYNew property: IsXTPSupported. sys.data_spacesThe following columns display additional values: type and type_desc sys.indexesThe following columns display additional values: type and type_desc. sys.parametersNew column: is_nullable. sys.all_sql_modulesNew column: uses_native_compilation. sys.sql_modulesNew column: uses_native_compilation. sys.table_typesNew column: is_memory_optimized. sys.tablesNew columns: durability, durability_desc, and is_memory_optimized. sys.hash_indexesNew: Shows the current hash indexes and the hash index properties sp_xtp_merge_checkpoint_filesNew stored procedure: Merges all data and delta files in the transaction range specified.
Column Store (CS) Deleted Bitmap Row Store (RS) Partition
CREATE TABLE ( … ) CREATE CLUSTERED COLUMNSTORE INDEX on Converts entire table to CS format Take care of memory needed and parallelism (MAXDOP 1) BULK INSERT, SELECT INTO on Creates new CS row groups INSERT/UPDATE Store in Row Store Tuple Mover When RS reaches 1M rows, convert to a CS row group Runs every 5 minutes by default Started explicitly by ALTER INDEX ON REORGANIZE Partitioning works on clustered columnstores Just like any other table The motivation is manageability more than performance