SQL 2014 In-Memory OLTP What, Why, and How Andy Vold Thursday June 19, 2014

About me…Andy Vold Architect with RevGen Partners Work primarily in business intelligence, SSIS/ETL, and database development and design. Telemark skier Mountain biker Twitter: @volda2000 Email: avold@revgen.com Feel free to ask questions as we go! Nov-18

In-Memory OLTP: Introduction Announced at SQL PASS 2012 as code-name “Hekaton” New in SQL Server 2014 Is a memory-optimized database engine integrated into the SQL Server engine Can significantly improve OLTP database application performance 64-bit Enterprise, Developer, or Evaluation edition of SQL Server 2014. Tonight we will only scratch the surface. Nov-18

In-Memory OLTP: What Memory-optimized data access. Fully ACID compliant data storage structure, which stores it’s data mainly within the system’s memory Lock and latch free algorithms. Threads that perform transactional work don’t use locks or latches for concurrency control. Native compilation of business logic Natively compiled stored procedures are Transact-SQL stored procedures compiled to native code that access memory-optimized tables. The output of the stored procedure compiler is C code and the compiler leverages Microsoft’s Visual C/C++ compiler to convert the C code into machine code. Memory-optimized tables are backed up as part of regular database backups Tables and sprocs become DLLs on the server. This is totally new. Nov-18

In-Memory OLTP: What Storage for memory-optimized tables has following key attributes: All memory-optimized tables are mapped to a memory-optimized filegroup. This filegroup is built using the filestream filegroup. There are no pages and the data is persisted as a row. All changes to memory-optimized tables are stored by appending to active files. Both reading and writing to files is sequential. This means that there are different versions of the data (versioning) An update is implemented as a delete followed by an insert. The deleted rows are not immediately removed from the storage. The deleted rows are removed by a background process, called MERGE. Unlike disk-based tables, storage for memory-optimized tables is not compressed. When migrating a compressed (ROW or PAGE) disk-based table to memory-optimized table, you will need to account for the change in size. Nov-18

In-Memory OLTP: What There are two durability options for memory-optimized tables: SCHEMA_ONLY (non-durable table) This option ensures durability of the table schema, including indexes When SQL Server is restarted, the non-durable table is recreated, but starts with no data. SCHEMA_AND_DATA (durable table) This option provides durability of both schema and data. The level of data durability depends on whether you commit a transaction as fully durable or with delayed durability. There are two types of memory-optimized indexes: Nonclustered hash indexes, which are made for point lookups. Nonclustered indexes, which are made for range scans and ordered scans. Memory-Optimized Table Variables With memory-optimized table variables, there is no tempdb utilization. Table variables are not stored in tempdb and do not use any resources in tempdb. Durable vs. non-durable is important. Schema vs schema and data…explain. Hash index is built in memory vs. other on disk Nov-18

In-Memory OLTP: Why Implementation Scenario Benefits of In-Memory OLTP High data insertion rate from multiple concurrent connections. Primarily append-only store. Unable to keep up with the insert workload. Eliminate contention. Reduce logging. Read performance and scale with periodic batch inserts and updates. High performance read operations, especially when each server request has multiple read operations to perform. Unable to meet scale-up requirements. Eliminate contention when new data arrives. Lower latency data retrieval. Minimize code execution time. Intensive business logic processing in the database server. Insert, update, and delete workload. Intensive computation inside stored procedures. Read and write contention. Minimize code execution time for reduced latency and improved throughput. Low latency. Require low latency business transactions which typical database solutions cannot achieve. Low latency code execution. Efficient data retrieval. Session state management. Frequent insert, update and point lookups. High scale load from numerous stateless web servers. Optional IO reduction or removal, when using non-durable tables The following table summarizes the workload patterns that may benefit most by using In-Memory OLTP. Really it comes down to “Eliminating contention” which happens due to the Lock and latch free algorithms Nov-18

In-Memory OLTP: How Built in tools The transaction performance collector in SQL Server Management Studio helps you evaluate if In-Memory OLTP will improve your database application’s performance. http://msdn.microsoft.com/en-us/library/dn205133.aspx The transaction performance analysis report also indicates how much work you must do to enable In-Memory OLTP in your application. After you identify a disk-based table to port to In-Memory OLTP, you can use the Memory Optimization Advisor, to help you migrate the table. http://msdn.microsoft.com/en-us/library/dn284308.aspx Nov-18

In-Memory OLTP: How DMVs Introduction of many SQL Server Dynamic Management views are new for In-Memory OLTP (xtp = eXtreme Transaction Processing): sys.dm_db_xtp_checkpoint_stats sys.dm_db_xtp_checkpoint_files sys.dm_db_xtp_gc_cycle_stats sys.dm_db_xtp_hash_index_stats sys.dm_db_xtp_index_stats sys.dm_db_xtp_memory_consumers sys.dm_db_xtp_merge_requests sys.dm_db_xtp_object_stats sys.dm_db_xtp_nonclustered_index_stats sys.dm_db_xtp_table_memory_stats sys.dm_db_xtp_transactions sys.dm_xtp_gc_queue_stats sys.dm_xtp_gc_stats sys.dm_xtp_system_memory_consumers sys.dm_xtp_transaction_stats And others (most all have “xtp” somewhere in the name) Nov-18

In-Memory OLTP: How Nov-18

References MSDN (In-Memory OLTP (In-Memory Optimization)) http://msdn.microsoft.com/en-us/library/dn133186.aspx MSDN (Introduction to Memory-Optimized Tables) http://msdn.microsoft.com/en-us/library/dn133165.aspx TechNet Blog (Architectural Overview of SQL Server 2014’s In-Memory OLTP Technology) http://blogs.technet.com/b/dataplatforminsider/archive/2013/07/22/architectural-overview-of-sql-server-2014-s-in-memory-oltp-technology.aspx Nov-18

Thank You. Andy Vold avold@revgen.com