1. Building Modern Transaction Systems on SQL Server
Bill Graziano
Building Modern Transaction Systems on SQL Server

2. Scenario: You’ve just been hired…
Process 100 “Business” Transactions per Second
Keep the average latency under 100ms…
and the 99.9th percentile latency under 500ms
Limitations
Need to process on a single box
Each transaction is SQL statements
You get to pick the hardware … but with a budget

3. SQL Server 2016 Enterprise Edition
$12,000 for a 2 core license pack
$200/month for two cores over five years
$800/month for eight cores over five years
Standard Edition limited to 24 cores and 128GB of RAM
Focus on frequency optimized chips up to 8 cores
Things I like:
Read-Committed Snapshot
In-Memory OLTP
Availability Groups
Meta data schema changes
Hot-add memory and CPU in VMs
Plan Guides
Online Index Rebuild

4. Read Committed Snapshot Isolation
Writers block Writers.
Readers don’t take read locks.
Requires different coding practices
Generates additional I/O to keep Version Store in tempdb
Can be enabled for existing workloads but best to start from scratch
Demo!

5. HADR - Availability Groups
Shared nothing
Each node has all databases
Multi-subnet Listeners
Easy to switch data centers
Fail databases together
Scale reads across replicas
Hekaton startup is faster
Can spread reads
Remote and local failover is the same
Listener name follows the active node
In-Memory OLTP startup is faster
Option to use cheaper local storage

6. SSD: Getting Cheaper tempdb should be on SSD for RCSI
Storage
Cost (1)
$ / GB
Read IOPS (2)
$ / IOPS
4TB 15K SAS in RAID10
$2,100
$0.52
~ 4,200
$0.50
4TB SSD SAS in RAID5
$7,000
$1.75
~ 50,000+
$0.14
4TB SSD NVMe (local)
~$6,000
$1.50
Yikes!
Drive only costs as of September 2016
Assumes reads spread over both sides of mirror. SSD IOPS range from 30K to 100K plus per- drive.
SSD is 50% cheaper than Summer 2015
SSD aren’t that much more expensive
tempdb should be on SSD for RCSI
Use SSD drives for buffer pool in SQL Server 2014+

7. In-Memory OLTP (Hekaton)
Lock-free, latch-free fast super-fast relational tables w/natively compiled stored procedures
10x performance increase is easy, 30x is achievable
Few limitations in SQL Server 2016
Please note the “In-Memory” part
Use cases
Fast transactions
Near real-time reporting
Session state
Poker hands 
Write intensive logging
Transaction logging

8. Reporting Production Database! Availability Group Replica Replication
Shift read queries to a synchronous read-only replica
Replication
More indexes than production
Different permissions than production
Use Read Committed Snapshot Isolation
Subset of the data
Add a table updated every minute with a timestamp
Can’t replicate from In-Memory OLTP
Data Warehouse
Specialized data structures for reporting
Can be near-real time
Build from an AG replica

9. (Realish-Time) Analytics
Build non-clustered column store indexes on your transaction tables
Read-committed snapshot isolation prevent blocking
Run your reporting out of an AG replica
Near-real-time analytics:
Capture changes from CDC or Change Tracking
In-Memory OLTP structure
Non-clustered column store indexes
Clustered column store indexes are replacing cubes

10. SQL like it’s NoSQL
Adding column with NOT NULL DEFAULT is a meta data change
Changing data types is HARD
INT -> BIGINT
Store schema-less data in XML or key-value tables
JSON in SQL Server 2016
Availability Groups and Replication support adding and removing columns
Avoid SELECT * as this will break when columns change

11. Database Design Hints Use BIGINTs
All tables should have a primary key that is a clustered index
Unless you have a specific reason to break that rule
Prefer stored procedures
Remember indexed persisted computed columns
Use schemas to group like permissions together
Just enough indexes … but not too many
* Too many is any index for reporting

12. Process Asynchronously
Faster!
Less
Blocking!
Table as Queue
Service Broker
Change Tracking
Change Data Capture
Custom change tracking
Hourly job -> 10 minute job -> 1 minute job
Demo!

13. T-SQL Tricks Demo! Consider SEQUENCEs over IDENTITY
Get the value before the INSERT
Multiple tables share a sequence
Use the OUTPUT clause
Process small batches or in loops
Read-Only Cursors!
Demo!

14. Client Applications CACHE! Use connection pooling
Multiple connection strings
Production
Read-Only – Synchronous replica
Reporting – Asynchronous replica or schema identical replication
Set the Application Name in the connection string
Use small transactions

15. Plan for Shards
Think about a Transaction Identifier that has multiple components
Version (8 bits)
Server Identifier (8 bits)
Time or date stamp (16 bits)
Sequence (32 bits)
Create a computed column of four columns?
Can still use IDENTITY for local identifier
How do you push configuration to each database?
How do you process refunds or adjustments?
Lots already written
Start with Twitter’s Snowflake

16. Planning for the Future
Archive, Archive, Archive!
Design around your data retention. Enforce it now!
Plan a maintenance window and stick to it!
Upgrades at half-life
Initial purchase is half of CPU and RAM capacity
Availability Groups allow different hardware
Linked servers are not your friends
Security issues
Query plan issues

17. THANK YOU!

18. Query Plan Stability Keep statistics up to date
Prefer stored procedures
Plan Guides over index hints
Be aware of parameter sniffing
OPTION RECOMPILE and OPTIMIZE FOR
Especially around date parameters
Regular traces can capture run times
DMVs to look at most run queries
Dynamic SQL for good query plans
Nightly or more often
Especially early when data can be in flux

19. Monitoring Capture the “Business Transaction” duration
Traces for > 500K Reads and Duration > 1 second
Run and analyze traces regularly
Alerting for blocking
Monitor lock waits, WRITELOG waits, Page Latch IO waits
Event Tracking for DDL events
Extended Events for exceptions
Code is in Github under billgraziano

20. Leftovers Use separate database for staging Reduce linked servers
Push final values into production concurrently
Reduce linked servers
Limit how they’re used
Can generate crazy query plans and unexpected SQL
Audit configuration table changes
Commonly triggers
SQL Server 2016 offers temporal tables
Performance test harness