SQL Server Data Collector From Every Angle Gianluca Sartori sqlconsulting.it

Sponsors & Media Partners

Organizers

Gianluca Sartori Independent SQL Server consultant Works with SQL Server since version 7 MCTS, MCITP, MCT DBA @ Scuderia Ferrari Blog: spaghettidba.com Twitter: @spaghettidba

Agenda What is the Data Collector Why Data Collection? How it works Troubleshooting Baselining Monitoring How it works SQL Server Data Collector at work Setup Reports Querying Extending

What is the Data Collector? Framework and tools for: Low overhead collection Persistence of performance data Diagnostic analysis Monitoring Reporting

Why Data Collection? Goals: Troubleshooting What happened tonight at 03:40? Tune performance Which queries consume most resources? Improve management How fast are my databases growing? Monitoring Is the system behaving normally?

Baselining What is a baseline? Describes performance under “normal” conditions Useful for: Comparing performance Monitoring Troubleshooting Trend Analysis and Capacity Planning

Challenges What to collect? How to collect? How often? Where to persist? Which retention policy? How to analyze?

What to collect? Performance indicators Dynamic Management Objects DMV DMF Performance Counters Query Stats Wait Stats

Dynamic Management Objects sys.dm_os_ OS information sys.dm_io_ Disk performance sys.dm_exec_ Sessions, requests, connections sys.dm_db_ Database info sys.dm_tran_ Transactions, locks sys.dm_... Audits, broker, CDC, FileStream, FTS, HADR, Repl, RG, XE… Which DMOs? SQL Server 2012 contains 141 DMVs and DMFs!

Performance Counters CPU Memory Disk SQL - specific … Which counters? SQL Server 2012 exposes more than 400 counters!

Challenges What to collect  System Collection Sets How to collect  Data Collector Infrastructure How often  Default Schedules Where to persist  Management Datawarehouse Which retention policy  Cleanup jobs How to analyze  Canned reports

Data Collector Overview Data Collection UI (Object Explorer) Management Data Warehouse Data Collection Configuration Collection Set Reports Target Data Collector Target Data Collector

Components Data Collector Extensible data collection infrastructure Low overhead (approx. 5% CPU overhead) Includes system collection sets Supports multiple data providers Support for SQL Server relational engine only other services can be added through extension

Components Management Data Warehouse Data repository for baseline and historical comparisons Central database for multiple SQL Server instances Size for around 300 Mb per day per instance Automatically purges old data Aggregated “olystic” repository for Utility Control Point

Components Management Studio components: Data collection configuration UI Setup Properties Configuration Data collector reports Canned reports based on system collection sets Entry point for monitoring and troubleshooting

Concepts Data Provider Collector Type Collection Item Collection Set Source of information Collector Type Reads data from data providers Collection Item Instance of a collector type Data input, frequency and target table Collection Set Group of collection items Logic unit of data collection

Data Collector (dcexec.exe) Architecture msdb SSMS SQL Agent SQL Server MDW Data Collector (dcexec.exe) Operating System Data Collector Cache

Architecture Collected Collection Job Data (SSIS runtime) Data Collector Cache MDW Upload Job (SSIS runtime)

DEMO Identifying Data Collector components

Data Collector SSMS UI Setup Configuring collection sets Create a Management Datawarehouse Set up data collection Configuring collection sets Starting and stopping Collect and upload Configure properties View log

DEMO Setting up data collection

Data Collector Reports System reports Based on system collection sets Server Activity Performance dashboard Many drill down reports Query Statistics Most expensive queries Drill down Disk Usage Monitor growth trends

DEMO Using system reports

Management Datawarehouse Schemas: core Internal usage snapshots Data gathered by system collection sets custom_snapshots Data gathered by custom collection sets

DEMO Querying the Management Datawarehouse: Who was active?

Extending the Data Collector Custom collection sets Based on generic T-SQL collector Some limitations apply Based on custom SSIS packages Complicated

DEMO Creating a custom collection set

Extending the Data Collector Custom reports Integrate in SSMS Distribute to all DBAs? Deploy to Reporting Services Bill Ramos’ MDW reports Analyze with PowerBI tools PowerQuery PowerView

DEMO Using Bill Ramos’ MDW reports

Bottom line Data collection provides out-of-the-box tools to collect and analyze performance data Free - Ships with SQL Server Some limitations apply Steep learning curve Viable alternative to commercial monitoring tools when money is a constraint Always test first!

Q&A Questions?

Resources Bill Ramos – MDW Reports Bart Duncan – Query Hash Statistics http://blogs.msdn.com/b/billramo/archive/tags/mdw+reports/ Bart Duncan – Query Hash Statistics http://code.msdn.microsoft.com/QueryHashStatistics

#sqlsat264 #sqlsatancona Thanks!