SQL Server Internals & Architecture Naomi Williams, SQL DBA LinkedIn

2 Tips After Party / Networking Event 5:30pm -8:30pm at Jefferson’s 618 Georgia Avenue Chattanooga, TN #SQLHELP Get a copy, study it and become a master!

3 Agenda  ACID  SQLOS  Components of database engine  Components of storage engine  Cache aging  Differences of Hekaton

Why is there ACID in my SQL Server?  ACID properties of Transactions  Atomic  Consistent  Isolated  Durable 4

Welcome to SQL Server! Where does the path begin? 5

6 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) SQLOS

SQLOS, Workers and Schedulers 7  SQLOS creates a set of schedulers upon start equal to the number of logical CPU’s.  Each scheduler manages workers, for every task to execute it’s assigned a worker that is in an idle state.  Workers do not move between schedulers, tasks are never moved between workers. However SQLOS can create child tasks and assign them different workers.  Tasks can have one of six states  Pending; task is waiting for an available worker  Done; task is completed  Running; task is currently executing  Runnable; task is waiting for the scheduler  Suspended; task is waiting for external event or resource  Spinloop; task is processing a spinlock

SQLOS, Workers and Schedulers, cont  Running state  Suspended queue and suspended state  Runnable queue and runnable state 8

9 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Buffer Pool Data Cache Language Hash of execution plan Does the hash exist in the Plan Cache? If query hash does not exist in Plan Cache YES! Query has plan exists! Plan Cache SQLOS

Project (*) Filter (o.date=‘ ’) Inner Join c.cid=o.cid Get (customers) as C Get (orders) as O Logical Query Tree Select * from Customers c inner join Orders o on c.cid=o.cid where o.date=‘ ’; 10

11 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Buffer Pool Data Cache Plan Cache SQLOS

Query Optimizer Parse and logical query tree Does it make sense and how can I get there? Binding, do they exist? Do the objects exist? Create path in binary from algebrizer for optimizer. Generate physical query plan Most efficient plan (path) to the data. 12

Physical Query Plan 13

14 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access Methods Transaction Manager Buffer Pool Data Cache OLE DB Does Data exist in Cache? Retrieves data from disk to cache Plan Cache SQLOS

15 Insert, Update, Delete SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access MethodsTransaction Manager Lock and Log manager Buffer Pool Data Cache OLE DB Plan Cache SQLOS CheckPoint

16 WAIT, WAIT WAIT SOS_Scheduler_Yield Async_Network_IO Writelog, LogBuffer PageIOLatch_x, Async_IO_Completion, IO_Completion Pagelatch_x, Latch_x, Resource_Semaphore Locks LCK_x, LCK_M_x Latches OLEDB BackupIOCXPacket IO_Completion MSQL_XP MSQL_DQ PreEmptive_OS_ ThreadPool

17 WAIT, WAIT WAIT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access MethodsTransaction Manager Lock and Log manager Buffer Pool Data Cache OLE DB Plan Cache SOS_Scheduler_Yield, CXPacket Async_Network_IO SQLOS Writelog, LogBuffer PageIOLatch_x, Async_IO_Completion, IO_Completion Pagelatch_x, Latch_x, Resource_Semaphore Locks LCK_x, LCK_M_x Latches

Data Cache and LRU-K 18 ABC AB C LAZY WRITER Data File A BC Buffer Pool Data Cache Lazy Writer 0 0 SQLOS Free Buffer List  Every buffer has a header  Last two times the page was referenced  Status information

Plan Cache and LRU 19 ABC AB C Buffer Pool Plan Cache Resource Monitor 0 0 SQLOS ABC

20 Hekaton SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface)  Protocols  TCP/IP  Shared Memory  Named Pipes  Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access MethodsTransaction Manager Lock and Log manager Buffer Pool Data Cache OLE DB Plan Cache SQLOS Natively compiled stored procedures and schema In-Memory OLTP Compiler Memory Optimized Tables and Indexes

Summary 21  ACID  SQLOS  Components of database engine  Components of storage engine  Cache aging  Differences of Hekaton