1. Transaction Log Deep Dive
8/31/2018 8:30 PM
Transaction Log Deep Dive
Kevin G. Boles
SQL Server Consultant Extraordinaire “The Wolf” for the Relational Engine Indicium Resources, Inc. AIM/Twitter: TheSQLGuru YIM: SR20Pilot MSM: Gtalk: MCITP, MCT, MVP, yada-yada-yada
Deck by Chirag Roy
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Who Am I World-class relational engine expert
8/31/2018 8:30 PM
Who Am I
World-class relational engine expert
Almost 40,000 man-hours invested in SQL Server
Absolutely LOVE doing targeted performance analysis and tuning work
VERY good at making others better at interacting with SQL Server
Couldn’t SSAS my way out of a paper bag
. . . with both ends open! 
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. Agenda
Transaction Log Architecture
Design Options for Performance
Hardware Options for Performance
Transaction Log Troubleshooting
Summary

4. Transaction Log architecture

5. Transaction Log Architecture
Physical/Logical Architecture
VLF1
VLF2
VLF3
VLF4
VLF5
Logical Log File
The transaction log is used to guarantee the data integrity of the database and for data recovery.
Virtual Log Files *

6. Transaction Log Architecture
Lazy Writer
Checkpoint
Buffer Pool
Dirty Page
Plan Cache
Transaction Log
The Buffer manager will tell the transaction manager to log an update, the transaction manager then passes down to the log manager and the change is written to the log. Once this is committed, the transaction manager will tell the buffer manager that the transaction is persisted - buffer manager then changes the pages in memory (a dirty page). As the change is always made to the log first, we called it a write-ahead log.
The check point process, running roughly every minute will scan through the buffer pool and flush dirty pages to disk - then writes a time stamp to the transaction log - anything behind this can then be dropped. Once on disk the page is added to the free list. An additional process in buffer management is the lazy writer which checks through the buffer pool and ages pages. Once at 0, a page is added to the free list and can be reused.
Note the checkpoint process timing is modified. Checkpoints occur periodically based on the number of log records generated by data modifications, or when requested by a user (issue the command ‘CHECKPOINT’) or a system shutdown. When a database is recovered at startup, transactions are rolled forward (ie, changes made it to the log, but not to disk), and rolled back (uncommitted transactions are ‘undone’). The time between checkpoints is calculated to attempt to ensure that the database is recovered within the recovery interval specified by sp_configure.
Simple Recovery Mode
Full Recovery Mode
Data Cache
Data file

7. Transaction Log Architecture
Recovery Type Considerations -
Simple Recovery - Log file CAN BE cleared on checkpoint
Full/Bulk Logged Recovery – Log file cleared on Log Backup (log fills up if do not back it up!!!)
Bulk Logged Recovery
Potentially Larger Log Backups when running -
ALTER INDEX REORGANIZE
DBCC INDEXDEFRAG

8. Transaction Log Architecture
Tools to Check T-LOG -
DBCC LOGINFO

9. Transaction Log Architecture
Tools to Check T-LOG -
DBCC SQLPERF(LOGSPACE)

10. Transaction Log Architecture
Tools to Check T-LOG -
Disk Usage Report
TRACE FLAG 3004

11. Design Options for Performance

12. Design options for performance
VLF Design
Too few Large VLF’s due to poor design
Too many Small VLF’s in case of Autogrow
Smallest Log File Size can be 512KB on creation
VLF Sizing should be carefully planned according to environment needs
Talk about Checkpoint process, Lazy writer

13. Design options for performance
VLF Design
Chunk Size
Number of VLFs
<= 1MB 2
>=1MB and < 64MB 4
>=64MB and < 1GB 8
1GB and larger 16
Talk about Checkpoint process, Lazy writer

14. Design options for performance
VLF Design
If log file designed for VLDBs > 8GB, expand Log File in Increments of 8GB (8000MB due to bug for exact 4GB increments) on DB Creation to create 512MB VLFs
If log file designed < 8GB, size Log File as per requirements
Talk about Checkpoint process, Lazy writer

15. Design options for performance
Considerations -
Autoshrink is Evil – Switch OFF
Autogrowth by % is Evil’er, causes VLF Fragmentation
VLF Fragmentation -
Leads to I/O overhead
Affects Redo/Undo phase performance
Increases database recovery/restore time
Cluster Failover Timing

16. Design options for performance
Considerations -
Place Data and Log files on separate LUNS to distribute I/O
Data Files experience Random Read/Writes
Log Files experience Sequential Read/Writes
SAN Admins need to provision LUNS optimized for the type of load

17. Design options for performance
Considerations -
Change Model Database Recovery Mode to Simple
Full Recovery Database in Pseudo Simple Until First Full Backup
Runaway Log file if subsequently no Log backups are taken
Instant File Initialization does not work with Log Files

18. Design options for performance
Considerations -
Log clearing can be affected by –
Long running transactions
Recovery Model
Replication
Database Mirroring
Switch on Backup Compression in SQL 2008/R2

19. Design options for performance
TempDB - Special Case
Always size Tempdb data and log file appropriately
Test using Autogrow (or max out drive?)
Size before going into production
Checkpoint occurs when Log File is 70% Full
Slow Disk I/O can cause delayed checkpoint
Mitigate using Alerts to notify
Manual Checkpoint precedes over System Checkpoint

20. Hardware Options for Performance

21. Hardware Options RAID 1 Good Read, Slower Write Performance
Good Redundancy
Data Availability
Expensive *

22. Hardware Options RAID 10 Good Read/Write Performance
Very Good Potential Redundancy
Data Availability
More Expensive *

23. Hardware Options SSD Extremely Good Read + Good Write Performance
Redundancy??
Data Availability?
Very Expensive *

24. Hardware Options Disk Sector Alignment
Still on Windows 2003 make sure to use disk sector alignment
Read Jimmy May’s blogs or whitepaper
In Windows 2008, disk sectors are aligned to 1MB by default for disks larger than 4GB (OEM Partition affect??)
Kendal Van Dyke blog series

25. Transaction Log troubleshooting

26. hardware utilisation and performance
Storage
Check the file latency within SQL Server using
sys.dm_io_virtual_file_stats (db_id,file_id)
My tuning session!! 
Use this script to get the latency for each file:
select db_name(database_id),
io_stall_read_ms/num_of_reads AS 'Disk Read Transfer/ms',
io_stall_write_ms/num_of_writes AS 'Disk Write Transfer/ms'
from sys.dm_io_virtual_file_stats (2,1)

27. Dynamic management views
sys.dm_os_waiting_tasks
Wait information
Task level
Very accurate
Transient data

28. Dynamic management views
sys.dm_os_wait_stats
Wait information
Cumulative by wait type
Persistent data
Transient data

29. Dynamic management views
Log_reuse_wait_desc in sys.databases
NOTHING
CHECKPOINT
LOG_BACKUP
ACTIVE_BACKUP_OR_RESTORE
ACTIVE_TRANSACTION
DATABASE_MIRRORING
REPLICATION
DATABASE_SNAPSHOT_CREATION
LOG_SCAN
OTHER_TRANSIENT

30. Common wait types ASYNC_IO_COMPLETION WRITELOG LOGBUFFER
Can be for "zeroing" out a transaction log file during log creation or growth
WRITELOG
Writing transaction log to disk
LOGBUFFER
Indicates worker thread is waiting for a log buffer to write log blocks for a transaction *

31. summary

32. SUMMARY VLF Design Switch of AutoShrink Use Autogrow as last resort
Enable Compression in SQL 2008/R2
Log files on Faster Dedicated Disks
significant resource waits
ASYNC_IO_COMPLETION
WRITELOG
LOGBUFFER

33. THANK YOU & Questions Recommended Reading:
THANK YOU & Questions