1. In-Memory OLTP: Concepts and Improvements in SQL Server 2016
Sunil Agarwal
Principal Program Manager (SQL Tiger Team)
Microsoft Inc.

2. SQL Server Tiger Team Proactive customer engagement
3/12/2018 7:36 PM
SQL Server Tiger Team
Proactive customer engagement
Customer delighting enhancements
Servicing excellence (SPs/CUs/HotFixes)
We are the engineering team (a.k.a Tiger Team) within the Data Platform Product Group that is in charge of delighting SQL Server on-premises customers for in-market releases. Our main areas of focus are:
Proactive Customer Engagement
Reach out to our Tier 1 customers virtually or onsite and prevent issues before they happen
Proactive issue and break-fix Management for critical SQL Server customers
Social engagement with broader customer base using Twitter, MSDN Blogs, Stack Overflow etc.
Key presentations, EBCs, conferences for SQL Server on-premises
Customer Delighting Enhancements
Providing enhancements for in-market releases rather than waiting for v.next for critical feature completeness, performance, supportability and hardware adoption improvements
Servicing Excellence
Delivering the results of proactive customer engagement and resulting customer delighting enhancements via high quality, zero regression, on-time servicing vehicles like hotfixes, cumulative updates and service packs
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. Session Objectives and Takeaways
Tech Ready 15
3/12/2018
Session Objectives and Takeaways
Session objective(s):
In-Memory OLTP overview
Identify the scenarios where In-Memory OLTP provides benefit
Assess whether and how In-Memory OLTP can benefit a customer scenario
Key takeaways:
SQL In-Memory OLTP offers incredible performance gains for transactional and data ingestion workloads
Many non-core transactional scenarios can benefit from In-Memory OLTP, e.g., temp tables, caching, ETL
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4. Hardware Trends New CPU won’t run a short transaction much faster
Moore’s Law on total CPU processing power holds but in parallel processing…
New CPU won’t run a short transaction much faster
CPU clock rate stalled…
Because processors would melt…
Meanwhile RAM cost continues to drop
Actually while some of these are increasing the key point is that they are not increasing exponentially any longer.

5. In-Memory OLTP Architectural Pillars
Main-Memory Optimized
Direct pointers to rows
Indexes exist only in memory
No buffer pool
No write-ahead logging
Stream-based storage
Architectural Pillars
Results
In-memory cache speed with capabilities of a database
Principles
Performance-critical data fits in memory
T-SQL Compiled to Native Machine Code
T-SQL compiled to machine code leveraging VC compiler
Procedure and its queries, becomes a C function
Aggressive compile- time optimizations
Queries & business logic run at native-code speed
Push decisions to compilation time
Non-Blocking Execution
Multi-version optimistic concurrency control, full ACID support
Lock-free data structures
No locks, latches or spinlocks,
No I/O in transaction
Transactions execute to completion without blocking
Conflicts are Rare
SQL Server Integration
Same manageability, administration & development experience
Integrated queries & transactions
Integrated HA and backup/restore
Hybrid engine and integrated experience
Built-In

6. Key lookup: Btree vs Memoty-Optimized Table
TechEd 2013
3/12/2018 7:36 PM
Key lookup: Btree vs Memoty-Optimized Table
Non clustered index
Hash index on Name R1 R2 R3
Key points
Show the searching of a row with SQL and with HK
Do mention the point of covering index…if the index leaf row does not have all the requested columns, SQL traverses the clustered index to get to the data row
Matching index record
Clustered Index
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

7. In-Memory OLTP: Integration and Application Migration
Few improvements in comm layers
TDS Handler and Session Management
Key
SQL Server.exe
Parser, Catalog, Optimizer
Existing SQL Component
Execution Plan cache for ad-hoc T-SQL and SPs
Hekaton Compiler
Native-Compiled SPs and Schema
20-40x more efficient
Real Apps see 2-30x
T-SQL Interpreter
Hekaton Component
Access Methods
Hekaton Engine: Memory_optimized Tables & Indexes
Non-durable Table option
Query Interop
Generated .dll
Buffer Pool T1 T2 T3
Tables
Reduced log contention; Low latency still critical for performance T1 T2 T3
Indexes
Memory-optimized Table Filegroup
Transaction Log
Data Filegroup T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3

8. Features and Durability

9. SQL Server In-Memory OLTP – Overview
Optimized for OLTP workloads
Memory-optimized data structures for efficient data access
Lock/Latch free access for linear scaling
Native Compilation of T-SQL Modules for efficient execution
Fully integrated into SQL Server
No new license or product
Familiar tools
Integrated Security and High Availability
None or Limited Application Changes
Available in SQL Server and Azure SQL DB
Performance numbers
Up to 30X performance improvement compared with traditional tables and stored procs
Actual gain depends on app pattern; typical gains: 2X-10X
Perf numbers in testing (4S server):
10M rows ingested per second
1GB/s of log generation
360K transactions per second for order processing workload (most transactions are mixed read/write)
BWIN.PARTY experiences with ASP.NET session state
Consolidate 18 servers -> 1 in production
1M requests/sec in testing

10. In-Memory OLTP features - Overview
TechReady 23
3/12/2018 7:36 PM
In-Memory OLTP features - Overview
Memory-Optimized Tables
Data storage in memory-optimized structures
Compiled into DLLs for efficient data access
Memory-Optimized Table Types
Memory-optimized table variables live in memory, in user DB (no tempdb, no IO)
Can be used for TVPs and table variables in all stored procs (not just native)
Natively Compiled Stored Procedures
Optimize performance of OLTP-style operations
Not suitable for reporting/analytics style queries
Natively Compiled Scalar User-Defined Functions
Do not require memory-optimized tables if there is no data access
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11. Create Table DDL CREATE TABLE [Customer]( [CustomerID] INT NOT NULL
PRIMARY KEY NONCLUSTERED,
[Name] NVARCHAR(250) NOT NULL,
[CustomerSince] DATETIME2 NULL,
INDEX [ICustomerSince] NONCLUSTERED
(CustomerID, CustomerSince) )
WITH (MEMORY_OPTIMIZED = ON,
DURABILITY = SCHEMA_AND_DATA);
This table is Memory Optimized
This table is durable (default).
Non-durable tables:
DURABILITY=SCHEMA_ONLY

12. Create Table Type DDL
CREATE TYPE [Sales].[SalesOrderDetailType_inmem] AS TABLE(
[OrderQty] [smallint] NOT NULL,
[ProductID] [int] NOT NULL,
[SpecialOfferID] [int] NOT NULL,
INDEX [IX_ProductID] NONCLUSTERED HASH
([ProductID]) WITH ( BUCKET_COUNT = 8)
) WITH ( MEMORY_OPTIMIZED = ON )

13. Create Procedure DDL
CREATE PROCEDURE DATETIME2
WITH
NATIVE_COMPILATION,
SCHEMABINDING AS
BEGIN ATOMIC
(TRANSACTION
ISOLATION LEVEL = SNAPSHOT,
LANGUAGE = N‘Dutch')
-- insert T-SQL here
END

14. Create Function DDL
CREATE FUNCTION [dbo].[ufnGetAccountingEndDate_native]()
RETURNS [datetime]
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMIC WITH (TRANSACTION ISOLATION LEVEL=SNAPSHOT, LANGUAGE=N'us_english')
RETURN DATEADD(millisecond, -2, CONVERT(datetime, ' ', 112));
END

15. Durability options Durability level Configuration Pros/Cons Scenarios
Full durability
Default
DURABILITY=SCHEMA_AND_DATA
Pro:
Every committed change is guaranteed to survive failure
Con:
Latency impact: every commit requires log IO
Default: most scenarios need full durability
Non-durable tables
Table creation
DURABILITY=SCHEMA_ONLY
No IO at all
Lose data on failure
Transient data such as session state
Caching
ETL (staging tables)
Delayed durability
Transaction commit time
COMMIT WITH (DELAYED_DURABILITY=ON)
Atomic block of native procedure
BEGIN ATOMIC WITH (DELAYED_DURABILITY=ON, …)
Database level
ALTER DATABASE CURRENT SET DELAYED_DURABILITY=FORCED
Low latency due to no log IO in transaction execution path
Efficient log IO due to batching
Limited data loss on failure (usually ~60K or ~1ms worth)
Low latency requirements
Can accept some data loss OR
Copy of recent data exists elsewhere in case of failure
AlwaysOn auto-failover (sync replicas) with low latency

16. In-Memory OLTP Performance
Sunil Agarwal
Principal Program Manager
Microsoft SQL Server Tiger Team
Sources:
In-Memory OLTP perf demo
Memory-optimized table variables and temp tables

17. When to use In-Memory OLTP

18. When to use (high-level)
In-Memory OLTP Recommended
Increase transaction throughput
Improve data ingestion rate
Reduce latency
Remove latency spikes – need consistent low latency
Transient data scenarios
Don’t Use In-Memory OLTP
Improve perf of analytics/BI – use Columnstore instead
Perf bottleneck outside SQL
App is chatty – many short requests – network communication dominates
Bottleneck is in the app
Low transaction volume
Resource limitations – not enough available memory for perf-critical data

19. When to use (low-level)
In-memory OLTP Recommended
Optimize DML (insert/update/delete)
High degree of concurrency
Latency spikes due to latch/lock waits
Transient and temporary data
Tempdb bottlenecks
SQL Server: contention on PFS/SGAM pages
Azure SQL DB: tempdb counts partly toward log IO cap
Reduce log throughput
Logging more efficient due to no logging of indexes
Azure SQL DB: more data fits in log IO cap
[native compilation]: “OLTP-like” operations
DML (insert/update/delete)
Queries touch limited number of rows
Query plans favor nested loops over hash join
The more statements and the more complex the logic in a single proc -> the better the perf benefits
Don’t use In-Memory OLTP
Optimize large range/table scans – use Columnstore instead
Can use combination of memory-optimized and columnstore
Short transactions with slow log IO
Log IO device becomes perf bottleneck, so no gain from optimizing data processing
Can potentially work around using DELAYED_DURABILITY
Tables don’t fit in memory
Note: for data that is temporal in nature, can use temporal memory-optimized tables to limit memory footprint
[native compilation]: “Analytics-style” queries
Queries touching large nrs of rows (e.g., table scans)
Parallel queries (native modules are single-threaded)
Query plans favor hash join over nested-loops (native support only nested-loops)

20. Scenarios and Case Studies

21. In-Memory OLTP Scenarios
3/12/2018
High throughput OLTP with low latency requirements
E.g., financial trading, gaming
Shock-absorber for data load (high data ingestion rate)
Memory-optimized table as “buffer”, with offload to clustered columnstore
Remove mid-tier caching
Session state and caching
E.g., ASP.NET session state
Replace NoSQL caching (e.g., Redis) with better perf and reduced complexity
ETL
Use SCHEMA_ONLY staging tables
Tempdb replacement
Memory-optimized table-valued parameters (TVPs)
Memory-only (SCHEMA_ONLY) tables to replace temp tables
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. High throughput OLTP with low latency
Extreme throughput needs – In-Memory OLTP sustains up to 360K transactions per second for representative order processing workload (read/write)
Scaling bottlenecks with traditional tables – e.g., latch or lock contention
In-Memory OLTP scales linearly, due to lock-free architecture
Low latency
Extreme low latency requirements – transactions can be processed in <0.5ms, including commit to log on disk (provided log IO device is fast enough).
Native compilation reduces CPU cycles required to process a transaction by an order of magnitude.
Consistent low latency
React to time-sensitive events: e.g., financial trades based on market data, bets on sports events like, in tennis, “is the next serve a fault?”
Traditional tables: bursts in activity result in latency spikes (e.g., 10ms->200ms) due to latching/locking, violates SLA to customer
In-Memory OLTP retains low latency even with activity spikes, due to lock-free implementation
Verticals: financials (trading), gaming/betting
Scenarios: Currency exchange (SBI LM, Bittrex*), financial derivatives (CMC Markets*), sports betting (Derivco*), BWIN, mobile games (CJ&E), mobile ad delivery
Implementation
Pinpoint hot tables and stored procedures (transaction performance analysis report in SSMS)
Migrate hot tables and stored procedures using SSMS (memory-optimized advisor) or SSDT
* Case study pending

23. Shock-absorber for data load (#1)
Before (disk-based):
Domain controllers
Events
Disk-based table
Collecting all security events from Active Directory Domain Controllers to report on and assess any threats or unusual security activity in the environment (healthcare – Beth Israel)
5 million messages / day
Bursts of activity (“shocks”) lead to increased latency, database running behind
Latch/lock contention on batch transforms
As a result, transforms are done hourly
Causes large data lag for reports
Hourly
Batch
Move
With Transforms
Reporting
Queries
(60-90s response
time)
Event details
Disk-based table

24. Shock-absorber for data load (#1)
After (memory-optimized):
Domain controllers
Data Ingestion:
Data no longer delayed into database
Can handle high spikes in workload
Parse/Cleanse/Aggregate processing:
No more contention allows for more frequent job execution
Steps removed (aggregation) with SQL 2016
Query Performance:
Between 4X and 6X gains (queries executing seconds reduced to sec)
Reporting queries run in near real-time (only 5 min data lag)
Events
Memory-optimized table
Move
With
Transforms
Every 5 mins
Reporting
Queries
(10-20s response
time)
Event details
Clustered Columnstore

25. Shock-absorber for data load (#2)
Before (disk-based):
Google/Bing search
Mobile apps
EdgeNet maintains inventory and pricing for stores of large retailers
Used by internet search and mobile apps
Data is read mostly, with 1-2X daily batch updates
Updates come from retailers based on their own inventory information
Desire to support more frequent updates, to have data more current
Update takes 40 minutes
Latching/locking during updates blocks reads
EdgeNet implemented mid-tier cache to support read workload during batch updates
Queries
Mid-tier cache
Disk-based table
Periodic batch
updates
Updated inventory & pricing

26. Shock-absorber for data load (#2)
After (memory-optimized):
Google/Bing search
Mobile apps
Update time reduced 10X (4 minutes)
Can support much more frequent batch updates, allow for more current information
No impact on read workload during update
Removed mid-tier caching
Queries
Memory-optimized table
Periodic batch
updates
Updated inventory & pricing

27. IoT – Cyber security Internet New app – requirements:
SCHEMA_ONLY Memory-optimized table
SQL Server running in Azure VM
Excel
Excel queries the memory-optimized table
Hadoop/Azure table storage
APS / Azure DW
New app – requirements:
Ingest 100K messages/sec – generated by machines across the internet
Perform real-time analytics over last several minutes – assess results of certain actions
Data must remain available for later offline analytics
Only SQL Server with In-Memory OLTP could support 100K messages/sec ingestion with real- time analytics. NoSQL solutions do not come close.

28. IoT – Oil and Gas (Quorum Business Solutions – case study pending)
Requirements:
Ingest sensor data in an Azure SQL Database
Perform historical analysis of sensor readings
Reduce CPU and log IO to save cost (lower pricing tier)
Limit memory utilization
Sensors
Mid-tier app
Memory-optimized TVP
Native compiled stored procedure
Temporal memory-optimized table
Current state
(in-memory)
History
(on-disk CCI)
UPDATE
Azure SQL Database
Solution:
Ingest into temporal memory-optimized table
Each row represents a sensor
A new sensor reading is an UPDATE
Use time travel queries for historical analysis
Reduced CPU:
Memory-optimized TVP and table optimize data access
Native proc optimizes execution of business logic
Use Columnstore index for history for efficient analytics
Reduced log IO
Memory-optimized TVPs do not incur log IO (tempdb-based TVPs do)
Memory-optimized tables are logged more efficiently (e.g., no index logging)
Clustered Columnstore for history has more efficient log IO pattern (only compressed segments are logged)
Limited in-memory storage footprint
Only latest sensor reading needs to fit in memory
Size in-memory storage based on sensor count, not number of readings
Sample:

29. Session state and caching
…or, how to use SQL Server/DB as an extremely fast key-value store
Applications / Web servers
SQL Database
(memory-optimized tables with native procs)
ASP.NET Session State
Sessions stored as BLOB – SQL Server used as key-value store
Sessions are transient, no strong durability guarantee (can use SCHEMA_ONLY)
Session count is limited – UPDATE-heavy workload
Other Caching Scenarios
Semi-structured data with flexible schema using JSON
Full relational cache
Results
1M requests/sec on single 4S machine
Reduce 18 servers -> 1 server (bwin.party)
Use one server to replace caching solutions for hundreds of apps, with consistent <1ms latency

30. ETL – staging for data load
Requirements
Load data from operational system into data warehouse
Perform transformations before loading into the fact and dimension tables
Reduce time taken for ETL process
Data Warehouse – SQL Server 2016
Operational DB
SCHEMA_ONLY
tables
(optional)
Transformations
Using native procs
Dimension & Fact tables (Columnstore)
SSIS
INSERT … SELECT
Advantages
SCHEMA_ONLY tables remove IO for staging tables – reduces time taken
Native procs limit CPU utilization for transformations – reduces time taken

31. TempDB replacement Sources: In-Memory OLTP perf demo
Sunil Agarwal
Principal Program Manager
Microsoft SQL Server Tiger Team
Sources:
In-Memory OLTP perf demo
Memory-optimized table variables and temp tables

32. SQL Server 2016: Performance improvements
Throughput:
Sustained log generation rate 1GB/s
Numerous improvements under the hood
Parallel plans and parallel scan of memory-optimized tables and indexes
Heap scan – no traversal of index structures is required

33. SQL Server 2016: Programmability improvements
Native Compilation
Full collations support in native modules
Query surface area
{LEFT|RIGHT} OUTER JOIN
Disjunction (OR, NOT)
UNION [ALL]
SELECT DISTINCT
Subqueries (EXISTS, IN, scalar)
Nested Stored procedures (EXECUTE)
Natively Compiled Scalar UDFs
Access from both native and interop
Improves perf of scalar UDFs in traditional workload if no table access is required
Natively Compiled Inline TVFs
EXECUTE AS CALLER
Query Store
Tables
Collations:
Full collations support in index key columns
All code pages supported for (var)char columns
Constraints
FOREIGN KEY
CHECK
UNIQUE constraints and indexes
LOB types: [n]varchar(max) and varbinary(max)
Row size >8060 bytes
NULLable index key columns
DML triggers on memory-optimized tables (AFTER)
Row-Level Security
Always Encrypted
Temporal Tables (history lives on-disk)

34. SQL Server 2016: Manageability improvements
Improved SSMS experience
Lightweight Performance Analysis in SSMS
Generate migration checklists to evaluate all tables/procedures for migration
ALTER support for procedures and tables
Full ALTER TABLE/PROCEDURE support
Index management through ALTER TABLE … ADD/ALTER/DROP INDEX
Limitation: ALTER TABLE offline for now; ALTER of 10GB table takes 1.5 minutes
No ALTER between disk-based and memory-optimized
TDE (Transparent Data Encryption)
All on-disk data files are now encrypted once TDE is enabled
Stats improvements: auto-update and sampled stats

35. In-Memory OLTP for Azure SQL DB
Now in Public Preview
Supported in all new Premium databases
No need to manage filegroup
No need to bind database to a resource pool for memory management
Data size:
P1 – 1GB user data size
P2 – 2GB user data size
P4 – 3GB* user data size
P6 – 8GB user data size
P11 – 10GB* user data size
* Will change as follows: P4 – 4GB; P11 – 14GB
Full In-Memory OLTP Surface Area
Memory-optimized tables and table types
Natively compiled stored procedures and scalar UDFs
New features are released to Azure SQL DB as soon as they are completed

36. In review: session objectives and takeaways
Tech Ready 15
3/12/2018
In review: session objectives and takeaways
Session objective(s):
Identify the scenarios where In-Memory OLTP provides benefit
Assess whether and how In-Memory OLTP can benefit a customer scenario
Key takeaways:
SQL In-Memory OLTP offers incredible performance gains for transactional and data ingestion workloads
Many non-core transactional scenarios can benefit from In-Memory OLTP, e.g., temp tables, caching, ETL
This slide is recommended as a final slide to recap the objectives of the session to remind attendees what you said would be covered and to highlight that you did indeed cover those points. The objectives listed on this slide should match the objectives covered on the required Objective slide at the beginning of your presentation.
If you have questions, please contact your Track PM. See slide 5 in this template for a complete list of Tracks and TPMs.
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

38. Connect us with us: @SQLServerGeeks facebook.com/SQLServerGeeks
@SQLServerGeeks
facebook.com/SQLServerGeeks
facebook.com/groups/theSQLGeeks