1. SQL Saturday Cleveland Peter Shore
Wait, Wait Do Tell Me
SQL Saturday Cleveland
Peter Shore

2. About Me SQL Server Database Administrator – ThirtyOne Gifts
Intentionally Accidental DBA
Over 25 years IT experience
Server Engineer
Desktop Engineer
Network Infrastructure
Desk side support
President CBusPASS
Co-Organizer SQL Saturday Columbus
How to find me

3. Introduction
In 2003 Tom Davidson published “SQL Server Performance Tuning Using Waits and Queues”
Stresses not looking at waits individually or in a vacuum
Greg Gonzalez published “Why Wait Stats Alone Are Not Enough” for SentryOne in 2020

4. What is wait time?
The time elapsed between the thread exiting and returning to the running state. The time elapsed between initiation and completion of a thread, when the thread is not running. The time spent waiting to return to running.

5. What are wait stats?
The accumulated time spent waiting broken down per resource. All time spent not running is tracked by SQL Server Everything the thread is waiting for is a resource

6. Threads and SQLOS Threads are a subset of a process
Shared address space
Context switching is faster between threads than between processes
Threads are the smallest sequence of instruction that can be managed
SQL Server has a maximum number of threads based on the number of logical processors
Threads are managed by a scheduler
More on threads ocesses

7. Threads and SQLOS
Scheduler is a piece of software that manages threads or processes
SQL Server has a 1:1 correspondence of scheduler to core
Scheduler is not bound to a CPU core
Changing cores is called context switching
Windows and Linux schedulers are preemptive
SQLOS schedulers are non-preemptive
CPU schedulers are managed, along with memory and disk access through SQLOS

8. Schedulers and Workers Demo

9. Threads and SQLOS Threads can be in only of six states
Pending – waiting for worker
Runnable – waiting for a quantum
Running – currently running
Suspended – officially “has a worker but waiting for an event” simply “waiting for a resource”
Done – completed
Spinloop – stuck in spinlock

10. Threads and SQLOS Running Thread States 73 87 103 72 66 97 151 62
Runnable
Suspended/Waiter

11. Threads and SQLOS Running Thread States 87 103 72 66 151 97 62 73
Runnable
Suspended/Waiter

12. Threads and SQLOS Running Thread States 103 72 66 151 73 97 62 87
Runnable
Suspended/Waiter

13. Threads and SQLOS Threads leave the runnable state due to: Completion
Resource Need
Page from memory
Page from disk
Results from another thread
Acknowledgement from caller
End of quantum
4 milliseconds
Hard coded

14. Threads Demo

15. Waiting
SQL Server measures wait time by design for every resource wait
There will always be waits
There will always be a top wait
Waits should not be analyzed in isolation
Look at the data
Baseline waits to define “normal” and be able to identify changes

16. General Waits Demo

17. SOS_SCHEDULER_YIELD
Moves from RUNNING to RUNNABLE Queue, never on the waiter list
Two meanings
Thread has exhausted 4ms quantum
Thread is spinning on a spinlock and has to backoff
Not a sign of CPU pressure
Look for plans with long scans
Low to no PAGEIOLATCH_XX waits
Remediate by reducing the work for the query

18. Parallelism Waits CXPACKET and CXCONSUMER
Presence of these waits can simply mean parallelism is happening
CXCONSUMER
Control thread wait
Introduced in SQL Server 2016 SP2 and SQL Server 2017 CU3
Generally can be ignored
CXPACKET
Prior to CXCONSUMER, was both PRODUCER and CONSUMER side wait
Generally can be ignored in SQL Server versions without CXCONSUMER
May indicate skewed parallelism

19. Parallelism Waits Avoid MAXDOP = 1 as a response
Change MAXDOP but be cognizant of the affect
Change Cost Threshold of Parallelism
Keep statistics up to date

20. Page Access Waits PAGEIOLATCH_XX PAGELATCH_XX
Waiting for a page to be read from disk into memory
XX is the mode of the latch to be acquired
SH (shared) page that will be read
EX (exclusive) page that will be changed
PAGELATCH_XX
Waiting to read page already in memory

21. Page Access Waits Avoid confusing them for each other
For PAGEIOLATCH_XX do not assume I/O subsystem or I/O path is the problem
For PAGELATCH_XX do not assume more memory or faster I/O will help

22. Page Access Waits PAGEIOLATCH_XX Analysis Potential Solutions
Determine which table/index is being read
DBCC PAGE
Investigate I/O subsystem latency with dm_os_virtual_file_stats and Avg Disk sec/read PerfMon Count
If coupled with CXPACKET, look for parallel scans (table scans)
Look for implicit conversions
Check for low Page Life Expectancy or other memory pressure
Potential Solutions
Indexing strategy
Up to date statistics
Update hardware (faster disk, more memory)

23. Page Access Waits PAGELATCH_XX Analysis Potential Solutions
Determine which table/index is being read
DBCC PAGE
Possible TempDB access
Look at query plans
Look for page splits
Potential Solutions
TempDB contention (TF 1118, number of files)
Reduce page splits
Increase FILLFACTOR
Change clustering key

24. Transaction Log IO Wait
WRITELOG
Extremely common
Waiting for log block buffer to be flushed to disk
Avoid the temptation to add more log files, SQL Server will not do parallel transaction log writes
Avoid assuming the I/O subsystem is an issue
Use sys.dm_io_virtual_file_stats to check for latency in I/O subsystem
Average disk queue length for the log drive, 32 hard coded max queue length
Look for page splits
Fix with faster disk
Larger transactions
Remove unused nonclustered indexes
Reduce page splits, increase FILLFACTOR

25. Windows Waits PREEMPTIVE_OS_XX Thread call to OS
Thread switches to PREEMTPIVE mode
State will be running rather than suspended
Remove “PREEMPTIVE_OS” prefix to get the name of the Windows API
Resolution of wait varies by API

26. Network Wait ASYNC_NETWORK_IO
Occurs when the calling machine/app cannot process data as fast as SQL Server sends it
Avoid thinking this is a network problem, it almost never is
Capture device executing query

27. Wrapping Up SQL Server 2017 has 1,140 defined wait types
New wait types are added with each version of SQL Server
Many of them are able to be ignored

28. References & Resources
SQL Server Performance Tuning Using Waits and Queues
7C43AE954BF4/SQLServer2000_SQL_Performance_Tuning_using_Waits_and_ Queues.pdf
Paul Randall Waits Library
Paul Randall’s blog
Greg Gonzalez Waits Are Not Enough
enough

29. Questions?
Thank you!