Presentation on theme: "Performance-Tuning for Extract/Transform/Load (ETL) Operations Presented by : Ken Stuber."— Presentation transcript:
Performance-Tuning for Extract/Transform/Load (ETL) Operations Presented by : Ken Stuber
About Ken Data Architect at Jewelers Mutual Insurance Has worked with SQL Server since 1997 Has supported Data Warehousing on SQL platform for 7+ years Missed MCM by one lab exam question. Grrr…. Will try again with whatever new advanced cert MS comes up
First… know your goals BASELINE SLAs Refresh Frequency Process Prioritization
Then … Design for Performance Data Grain Source Data Local Re-startable Parallel processing Merge or Rebuild Dims?
Database and Server Settings Recovery Model Isolation Max Memory Power Option MAXDOP
Getting to your Source Data If Local… Backup & Restore SAN Snap or Clone AA Readable 2 nd Snapshot of Mirror Replication Log Shipping CDC? Performance Trade-off of local vs Remote Always TEST!
Bulk Loading Data Reduce IO w/ Minimal Logging Methods sys.dm_io_virtual_file_stats TABLOCK vs Trace Flag 610
Functions CPU killer UDF vs CLR vs In-Line
Indexing Source Staging Dest.
Statistics Bad Stats = Catastrophic Performance Update after every build FULLSCAN if you can afford it
Compression? I/O Reduction at cost of CPU
Partitioning? Can Eliminate the L. of E.T.L. Increases Complexity Can Benefit Query performance too
Other Advanced Features Hekaton? Updatable Columnstore Indexes?
Hardware Considerations More RAM!!! IO or CPU bound? Know your IO Subsystem SSDs Multiple files TempDB local
Tuning existing ETL
Resources Microsoft CAT Teams Data Loading Performance Guide Microsoft CAT Teams Data Loading Performance Guide SQL Magazine: How to find your most expensive queriesHow to find your most expensive queries SAN Performance Tuning with SQLIO Columnstore Insert/Update Performance