2. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Session 306 Advanced SQL Profiler Bill Graziano ClearData Consulting, Inc. SQLTeam.com 2003 PASS Community Summit November 11-14, 2003

3. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Agenda Introduction / Overview Introduction / Overview Performance Monitoring Performance Monitoring Security Monitoring Security Monitoring Scripting Traces Scripting Traces Blackbox Traces Blackbox Traces Replaying Traces Replaying Traces

4. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Profiler A Quick Demo

5. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. SQL Server Trace Architecture Producer Filter #1 Duration: > 10 seconds Filter #2 App Like MyApp% Queue Filtered Data Destination File or Recordset

6. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Trace Architecture Producer Producer Generates events to be monitored Generates events to be monitored Ex: Lock Manager produces lock events Ex: Lock Manager produces lock events Each trace can have multiple producers Each trace can have multiple producers Resides on the server Resides on the server Filter Filter Filters events based on criteria Filters events based on criteria Resides on the server Resides on the server

7. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Trace Architecture Queue Queue Single queue per *SERVER* Single queue per *SERVER* Again resides on the server Again resides on the server Destination Destination Usually External to SQL Server Usually External to SQL Server Profiler Profiler Profiler -> Table Profiler -> Table Trace File Trace File

8. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Trace Templates Predefined set of Predefined set of Events Events Data columns Data columns Filters Filters Demo Demo

9. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Handy Tips Data Column Groups Data Column Groups Really just sorts Really just sorts Event / Data Column Mapping Event / Data Column Mapping Not all events create all data columns Not all events create all data columns Store traces in a database table Store traces in a database table Not the server youre tracing! Not the server youre tracing!

10. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. CPU Monitoring Events Events TSQL TSQL SQL:BatchCompleted SQL:BatchCompleted SQL:StmtCompleted SQL:StmtCompleted StoredProcedures StoredProcedures RPC:Completed RPC:Completed A few other miscellaneous events A few other miscellaneous events AuditLogout, DTCTransaction, TransactionLog AuditLogout, DTCTransaction, TransactionLog

11. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. CPU Monitoring Data Columns Data Columns CPU CPU Milliseconds of CPU time Milliseconds of CPU time Catches queries that have high CPU usage Catches queries that have high CPU usage Duration Duration Milliseconds of duration Milliseconds of duration Catches long queries regardless of reason Catches long queries regardless of reason

12. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Disk Monitoring Reads – Logical Disk Reads Reads – Logical Disk Reads Writes – Logical Disk Writes Writes – Logical Disk Writes Logical <> Physical Logical <> Physical Duration minus CPU => Disk Time Duration minus CPU => Disk Time Except for locks and other things (waits) Except for locks and other things (waits) Demo Demo

13. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Memory Difficult to track using Profiler Difficult to track using Profiler Server : Server Memory Change Server : Server Memory Change Indicates memory change by 1 MB or 5% (which ever is greater) Indicates memory change by 1 MB or 5% (which ever is greater) Stored Procedures : SP:CacheMiss Stored Procedures : SP:CacheMiss Stored Procedures : SP:Recompile Stored Procedures : SP:Recompile

14. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Lock Events Acquired & Released Acquired & Released Filter on object – still impractical Filter on object – still impractical Deadlock Deadlock Deadlock Chain Deadlock Chain Indicates SPIDs involved in a deadlock Indicates SPIDs involved in a deadlock Timeout Timeout Demo Demo

15. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Other Events Cursors : CursorOpen Cursors : CursorOpen Database : Data File Auto Grow Database : Data File Auto Grow Database : Log File Auto Grow Database : Log File Auto Grow Objects: Auto Stats Objects: Auto Stats Scans : Scan Started Scans : Scan Started Can be misleading as many small tables are scanned (Demo) Can be misleading as many small tables are scanned (Demo)

16. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Profiling Security Focus on logins and permissions Focus on logins and permissions Can monitor Can monitor Adding and removing logins and users Adding and removing logins and users Granting and revoking permissions Granting and revoking permissions Changing passwords Changing passwords Changing role membership Changing role membership Checking object permissions Checking object permissions

17. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Profiling Security Audit all Security Audit class events EXCEPT: Audit all Security Audit class events EXCEPT: Audit Object Derived Permission Audit Object Derived Permission Audit Object Permission Audit Object Permission Audit Statement Permission Audit Statement Permission These will fire each time permission is checked on an object or statement These will fire each time permission is checked on an object or statement

18. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Scripting Traces Effective Security Tracing Effective Security Tracing Automatic Automatic Continuous Continuous Demo Demo

19. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Creating Traces First, we create the trace First, we create the trace sp_trace_create @TraceID output, @options = 0, @tracefile = Nc:\work\trace1', @maxfilesize = 5, @stoptime = NULL Returns a TraceID as int Returns a TraceID as int Used to control the trace Used to control the trace

20. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Create Trace Options Produce Row Set (1) Produce Row Set (1) This is what Profiler uses This is what Profiler uses File Rollover (2) File Rollover (2) Rolls to new file when max file size is reached Rolls to new file when max file size is reached Each new file has a sequential number appended to it Each new file has a sequential number appended to it Tracefile_1.trc Tracefile_1.trc

21. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Create Trace Options Shutdown on Error (4) Shutdown on Error (4) Shuts down SQL Server if unable to write trace file Shuts down SQL Server if unable to write trace file Used for C2 certification Used for C2 certification Black Box Traces (8) Black Box Traces (8) Always holds last 5MB worth of data Always holds last 5MB worth of data Incompatible with other options Incompatible with other options

22. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Adding Events Adds a data column to the trace Adds a data column to the trace sp_trace_setevent @TraceID, @eventid = 10, @columnid = 1, @on = 1 One entry for each event / column combination One entry for each event / column combination

23. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Adding Filters Sets a filter on the trace Sets a filter on the trace sp_trace_setfilter @TraceID, @columnid = 10, @logical_operator = 0, @comparison_operator = 7, @value = N'SQL Profiler i.e. - ApplicationName not like SQL Profiler i.e. - ApplicationName not like SQL Profiler

24. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Starting Traces Starts the trace Starts the trace sp_trace_setstatus @TraceID, @status = 1 Other Statuses Other Statuses 0 – Stops the trace 0 – Stops the trace 2 – Deletes the trace definition 2 – Deletes the trace definition A trace must be stopped AND deleted when youre done with it A trace must be stopped AND deleted when youre done with it

25. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Autorun a Stored Procedure Can automatically start a stored procedure at the time the service starts Can automatically start a stored procedure at the time the service starts sp_procoption @ProcName = usp_BaseSecurity', @OptionName = 'startup', @OptionValue = true

26. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Processing Trace Files Profiler can save an existing trace file to a table (Demo) Profiler can save an existing trace file to a table (Demo) Or use fn_trace_gettable to query a trace file Or use fn_trace_gettable to query a trace file Select * from ::fn_trace_gettable( BaseSecurity.trc, default) Demo Demo

27. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Processing Trace Files Can easily INSERT into a table Can easily INSERT into a table Second parameter indicates to read all trace files (trace.trc, trace_1.trc, etc.) Second parameter indicates to read all trace files (trace.trc, trace_1.trc, etc.) Scheduled job to import then archive the files Scheduled job to import then archive the files

28. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Blackbox Traces Special trace type Special trace type exec @rc = sp_trace_create @TraceID output, 8 Automatically captures the last 5-10MB of data Automatically captures the last 5-10MB of data Reuses two rollover files Reuses two rollover files

29. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Blackbox Traces File is stored in the default DATA directory File is stored in the default DATA directory SQL Server determines the events and columns to track SQL Server determines the events and columns to track Data written in 128KB chunks Data written in 128KB chunks

30. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Handy UDFs fn_trace_getinfo fn_trace_getinfo Returns information about a trace or all active traces Returns information about a trace or all active traces fn_trace_getfilterinfo fn_trace_getfilterinfo fn_trace_geteventinfo fn_trace_geteventinfo

31. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Alerts & Errors Event classes to capture errors Event classes to capture errors Exception Events Exception Events T-SQL exceptions T-SQL exceptions ErrorLog Events ErrorLog Events EventLog Events EventLog Events Attention Events Attention Events Broken client connections Broken client connections

32. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. User Configurable Events T-SQL can create events that appear in Profiler T-SQL can create events that appear in Profiler sp_trace_generateevent @eventid = 82, @userinfo = N'My Information, @userdata = @SomeVarBinaryData Demo Demo

33. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Replaying Traces Profiler can replay traces Profiler can replay traces Requires specific events to be captured Requires specific events to be captured Best to use the provided template Best to use the provided template Can replay sequentially or multi- threaded Can replay sequentially or multi- threaded Poor mans load tool Poor mans load tool Capture a training session and replay it Capture a training session and replay it Demo Demo

34. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Performance Performance Impact Difficult to Quantify Performance Impact Difficult to Quantify Greatly depends on server load Greatly depends on server load Try to limit traces to least amount of information needed Try to limit traces to least amount of information needed Information vs. Performance Information vs. Performance Profiler is slower than SQL Server tracing to a file Profiler is slower than SQL Server tracing to a file

35. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Summary Performance Monitoring Performance Monitoring Security Monitoring Security Monitoring Scripting Traces Scripting Traces Blackbox Traces Blackbox Traces Replaying Traces Replaying Traces

36. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Presentation Download Available tomorrow at Available tomorrow at www.cleardata.biz www.cleardata.biz www.sqlteam.com www.sqlteam.com Contains Contains Current slides Current slides Demo files Demo files graz@SQLTeam.com graz@SQLTeam.com

37. Advanced SQL Profiler 306 November 12 th, 2003 3:00 PM Copyright 2003 ClearData Consulting, Inc. Thank you! Thank you for attending this session and the 2003 PASS Community Summit in Seattle! Please help us improve the quality of our conference by completing your session evaluation form. Completed evaluation forms may be given to the room monitor as you exit or to staff at the registration desk.