Architectures and Case Studies with In-Memory OLTP in SQL Microsoft Ignite 2016 5/7/2018 4:37 AM BRK3103 Architectures and Case Studies with In-Memory OLTP in SQL Jos de Bruijn <jodebrui@microsoft.com> Senior Program Manager © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Features and Durability

SQL Server In-Memory OLTP Optimized for OLTP workloads Memory-optimized data structures for efficient data access Lock/Latch free access for linear scaling Native Compilation of T-SQL Modules for efficient execution Fully integrated into SQL Server No new license or product Familiar tools Integrated Security and High Availability No/Limited Application Changes Available in SQL Server and Azure SQL DB Performance numbers Up to 30X performance improvement compared with traditional tables and stored procs Actual gain depends on app pattern; typical gains: 2X-10X Perf numbers in testing (4S server): 10M rows ingested per second 1GB/s of log generation 360K transactions per second for order processing workload (most transactions are mixed read/write) bwin.party experiences with ASP.NET session state Consolidate 18 servers -> 1 in production 1M requests/sec in testing

In-Memory OLTP features TechReady 23 5/7/2018 4:37 AM In-Memory OLTP features Memory-Optimized Tables Data storage in memory-optimized structures Compiled into DLLs for efficient data access Memory-Optimized Table Types Memory-optimized table variables live in memory, in user DB (no tempdb, no IO) Can be used for TVPs and table variables in all stored procs (not just native) Natively Compiled Stored Procedures Optimize performance of OLTP-style operations Not suitable for reporting/analytics style queries Natively Compiled Scalar User-Defined Functions Do not require memory-optimized tables if there is no data access © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Create Table DDL CREATE TABLE [Customer]( [CustomerID] INT NOT NULL PRIMARY KEY NONCLUSTERED, [Name] NVARCHAR(250) NOT NULL, [CustomerSince] DATETIME2 NULL, INDEX [ICustomerSince] NONCLUSTERED (CustomerID, CustomerSince) ) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA); This table is durable (default). Non-durable tables: DURABILITY=SCHEMA_ONLY

Create Table Type DDL CREATE TYPE [Sales].[SalesOrderDetailType_inmem] AS TABLE( [OrderQty] [smallint] NOT NULL, [ProductID] [int] NOT NULL, [SpecialOfferID] [int] NOT NULL, INDEX [IX_ProductID] NONCLUSTERED HASH ([ProductID]) WITH ( BUCKET_COUNT = 8) ) WITH ( MEMORY_OPTIMIZED = ON )

Create Procedure DDL CREATE PROCEDURE [dbo].[InsertOrder] @id INT, @date DATETIME2 WITH NATIVE_COMPILATION, SCHEMABINDING AS BEGIN ATOMIC (TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE = N‘Dutch') -- insert T-SQL here END

Create Function DDL CREATE FUNCTION [dbo].[ufnGetAccountingEndDate_native]() RETURNS [datetime] WITH NATIVE_COMPILATION, SCHEMABINDING AS BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL=SNAPSHOT, LANGUAGE=N'us_english') RETURN DATEADD(millisecond, -2, CONVERT(datetime, '20160701', 112)); END

Durability options Durability level Configuration Pros/Cons Scenarios Full durability Default DURABILITY=SCHEMA_AND_DATA Pro: Every committed change is guaranteed to survive failure Con: Latency impact: every commit requires log IO Default: most scenarios need full durability Delayed durability Transaction commit time COMMIT WITH (DELAYED_DURABILITY=ON) Atomic block of native procedure BEGIN ATOMIC WITH (DELAYED_DURABILITY=ON, …) Database level ALTER DATABASE CURRENT SET DELAYED_DURABILITY=FORCED Low latency due to no log IO in transaction execution path Efficient log IO due to batching Limited data loss on failure (usually ~60K or ~1ms worth) Low latency requirements Can accept some data loss OR Copy of recent data exists elsewhere in case of failure AlwaysOn auto-failover (sync replicas) with low latency Non-durable tables Table creation DURABILITY=SCHEMA_ONLY No IO at all Lose data on failure Transient data such as session state Caching ETL (staging tables)

In-Memory OLTP Features and Durability Options Sources: In-Memory OLTP perf demo Memory-optimized table variables and temp tables

When to use

When to use (high-level) Yes Increase transaction throughput Improve data ingestion rate Reduce latency Remove latency spikes – need consistent low latency Transient data scenarios No Improve perf of analytics/BI – use Columnstore instead Perf bottleneck outside SQL App is chatty – many short requests – network communication dominates Bottleneck is in the app Low transaction volume Resource limitations – not enough available memory for perf-critical data

When to use (low-level) Yes Optimize DML (insert/update/delete) High degree of concurrency Latency spikes due to latch/lock waits Transient and temporary data Tempdb bottlenecks SQL Server: contention on PFS/SGAM pages Azure SQL DB: tempdb counts partly toward log IO cap Reduce log throughput Logging more efficient due to no logging of indexes Azure SQL DB: more data fits in log IO cap Data is temporal in nature Temporal memory-optimized tables to manage history Historical data lives on-disk [native compilation]: “OLTP-like” operations DML (insert/update/delete) Queries touch limited number of rows Query plans favor nested loops over hash join The more statements and the more complex the logic in a single proc -> the better the perf benefits No Optimize large range/table scans – use Columnstore instead Can use combination of memory-optimized and columnstore Short transactions with slow log IO Log IO device becomes perf bottleneck, so no gain from optimizing data processing Can potentially work around using DELAYED_DURABILITY Tables don’t fit in memory Note: for data that is temporal in nature, can use temporal memory-optimized tables to limit memory footprint [native compilation]: “Analytics-style” queries Queries touching large nrs of rows (e.g., table scans) Parallel queries (native modules are single-threaded) Query plans favor hash join over nested-loops (native support only nested-loops)

Scenarios and Case Studies

In-Memory OLTP Scenarios 5/7/2018 In-Memory OLTP Scenarios High throughput OLTP with low latency requirements E.g., financial trading, gaming Shock-absorber for data load (high data ingestion rate) Memory-optimized table as “buffer”, with offload to clustered columnstore Remove mid-tier caching Internet of Things (IoT) Ingesting sensor data (e.g., oil&gas) Consumer device telemetry (e.g., manufacturing) Temporal to manage sensor/device history Session state and caching E.g., ASP.NET session state Replace NoSQL caching (e.g., Redis) with better perf and reduced complexity Tempdb replacement Memory-optimized table-valued parameters (TVPs) Memory-only (SCHEMA_ONLY) tables to replace temp tables ETL Use SCHEMA_ONLY staging tables © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

High throughput OLTP with low latency Verticals: financials (trading), gaming/betting Scenarios: Currency exchange (SBI LM, Bittrex*), financial derivatives (CMC Markets*), sports betting (Derivco*), mobile games (CJ&E), mobile ad delivery High-throughput OLTP Extreme throughput needs – In-Memory OLTP sustains up to 360K transactions per second for representative order processing workload (read/write) Scaling bottlenecks with traditional tables – e.g., latch or lock contention In-Memory OLTP scales linearly, due to lock-free architecture Low latency Extreme low latency requirements – transactions can be processed in <0.5ms, including commit to log on disk (provided log IO device is fast enough). Native compilation reduces CPU cycles required to process a transaction by an order of magnitude. Consistent low latency React to time-sensitive events: e.g., financial trades based on market data, bets on sports events like, in tennis, “is the next serve a fault?” Traditional tables: bursts in activity result in latency spikes (e.g., 10ms->200ms) due to latching/locking, violates SLA to customer In-Memory OLTP retains low latency even with activity spikes, due to lock-free implementation Implementation Pinpoint hot tables and stored procedures (transaction performance analysis report in SSMS) Migrate hot tables and stored procedures using SSMS (memory-optimized advisor) or SSDT * Case study pending

Shock-absorber for data load (#1) Before (disk-based): Domain controllers Events Disk-based table Collecting all security events from Active Directory Domain Controllers to report on and assess any threats or unusual security activity in the environment (healthcare – Beth Israel) 5 million messages / day Bursts of activity (“shocks”) lead to increased latency, database running behind Latch/lock contention on batch transforms As a result, transforms are done hourly Causes large data lag for reports Hourly Batch Move With Transforms Reporting Queries (60-90s response time) Even details Disk-based table

Shock-absorber for data load (#1) After (memory-optimized): Domain controllers Data Ingestion: Data no longer delayed into database Can handle high spikes in workload Parse/Cleanse/Aggregate processing: No more contention allows for more frequent job execution Steps removed (aggregation) with SQL 2016 Query Performance: Between 4X and 6X gains (queries executing 60-90 seconds reduced to 10-20 sec) Reporting queries run in near real-time (only 5 min data lag) Events Memory-optimized table Move With Transforms Every 5 mins Reporting Queries (10-20s response time) Event details Clustered Columnstore

Shock-absorber for data load (#2) Before (disk-based): Google/Bing search Mobile apps EdgeNet maintains inventory and pricing for stores of large retailers Used by internet search and mobile apps Data is read mostly, with 1-2X daily batch updates Updates come from retailers based on their own inventory information Desire to support more frequent updates, to have data more current Update takes 40 minutes Latching/locking during updates blocks reads EdgeNet implemented mid-tier cache to support read workload during batch updates Queries Mid-tier cache Disk-based table Periodic batch updates Updated inventory & pricing

Shock-absorber for data load (#2) After (memory-optimized): Google/Bing search Mobile apps Update time reduced 10X (4 minutes) Can support much more frequent batch updates, allow for more current information No impact on read workload during update Removed mid-tier caching Queries Memory-optimized table Periodic batch updates Updated inventory & pricing

IoT – Cyber security Internet New app – requirements: SCHEMA_ONLY Memory-optimized table SQL Server running in Azure VM Excel Excel queries the memory-optimized table Hadoop/Azure table storage APS / Azure DW New app – requirements: Ingest 100K messages/sec – generated by machines across the internet Perform real-time analytics over last several minutes – assess results of certain actions Data must remain available for later offline analytics Only SQL Server with In-Memory OLTP could support 100K messages/sec ingestion with real- time analytics. NoSQL solutions do not come close.

IoT – Oil and Gas (Quorum Business Solutions – case study pending) Requirements: Ingest sensor data in an Azure SQL Database Perform historical analysis of sensor readings Reduce CPU and log IO to save cost (lower pricing tier) Limit memory utilization Sensors Mid-tier app Memory-optimized TVP Native compiled stored procedure Temporal memory-optimized table Current state (in-memory) History (on-disk CCI) UPDATE Azure SQL Database Solution: Ingest into temporal memory-optimized table Each row represents a sensor A new sensor reading is an UPDATE Use time travel queries for historical analysis Reduced CPU: Memory-optimized TVP and table optimize data access Native proc optimizes execution of business logic Use Columnstore index for history for efficient analytics Reduced log IO Memory-optimized TVPs do not incur log IO (tempdb-based TVPs do) Memory-optimized tables are logged more efficiently (e.g., no index logging) Clustered Columnstore for history has more efficient log IO pattern (only compressed segments are logged) Limited in-memory storage footprint Only latest sensor reading needs to fit in memory Size in-memory storage based on sensor count, not number of readings Sample: https://github.com/Microsoft/sql-server-samples/tree/master/samples/applications/iot-smart-grid

Session state and caching …or, how to use SQL Server/DB as an extremely fast key-value store Applications / Web servers SQL Database (memory-optimized tables with native procs) ASP.NET Session State Sessions stored as BLOB – SQL Server used as key-value store Sessions are transient, no strong durability guarantee (can use SCHEMA_ONLY) Session count is limited – UPDATE-heavy workload Other Caching Scenarios Simple key-value BLOBs like in session state, similar to Redis Semi-structured data with flexible schema using JSON Full relational cache Results 1M requests/sec on single 4S machine Reduce 18 servers -> 1 server (bwin.party) Use one server to replace caching solutions for hundreds of apps, with consistent <1ms latency

Tempdb replacement Scripts: Memory-optimized table variables and temp tables

ETL – staging for data load Requirements Load data from operational system into data warehouse (ex. customer: ATTOM) Perform transformations before loading into the fact and dimension tables Reduce time taken for ETL process Data Warehouse – SQL Server 2016 Operational DB SCHEMA_ONLY tables (optional) Transformations Using native procs Dimension & Fact tables (Columnstore) SSIS INSERT … SELECT Advantages SCHEMA_ONLY tables remove IO for staging tables – reduces time taken Native procs limit CPU utilization for transformations – reduces time taken

Free IT Pro resources To advance your career in cloud technology Microsoft Ignite 2016 5/7/2018 4:37 AM Free IT Pro resources To advance your career in cloud technology Plan your career path Microsoft IT Pro Career Center www.microsoft.com/itprocareercenter Cloud role mapping Expert advice on skills needed Self-paced curriculum by cloud role $300 Azure credits and extended trials Pluralsight 3 month subscription (10 courses) Phone support incident Weekly short videos and insights from Microsoft’s leaders and engineers Connect with community of peers and Microsoft experts Get started with Azure Microsoft IT Pro Cloud Essentials www.microsoft.com/itprocloudessentials Demos and how-to videos Microsoft Mechanics www.microsoft.com/mechanics Connect with peers and experts Microsoft Tech Community https://techcommunity.microsoft.com © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please evaluate this session 5/7/2018 4:37 AM Please evaluate this session Your feedback is important to us! From your PC or Tablet visit MyIgnite at http://myignite.microsoft.com From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting https://aka.ms/ignite.mobileapp © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5/7/2018 4:37 AM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.