Honest Bob’s Cube Processing Bob Duffy Database Architect Prodata SQL Centre of Excellence 18 th July, 2014

Bob Duffy 20 years in database sector, 250+ projects 20 years in database sector, 250+ projects Senior Consultant with Microsoft Senior Consultant with Microsoft One of about 25 MCA for SQL Server globally (aka SQL Ranger) One of about 25 MCA for SQL Server globally (aka SQL Ranger) SQL MCM on SQL 2005 and 2008 SQL MCM on SQL 2005 and 2008 SQL Server MVP SQL Server MVP SSAS Maestro SSAS Maestro Database Architect at Prodata SQL Centre of Excellence Database Architect at Prodata SQL Centre of Excellence @bob_duffy

What we Will Cover How Processing Works Tips to Optimise Processing Benchmarking Tools Processing Options and Strategies Tips for Tabular Models

What's a cube ?

Dimension Processing

Measure Group Processing

Recap: Tuning Attribute Properties Key Column Name Column AttributeHierarchyEnabledAttributeHierarchyOptimizedAttributeHierarchyOrdered Add correct relationships Avoid Large Dimensions Or DELETE THEM

Determining Size of Objects 

Slow Execute SQL ? 1. Check WAIT Statistics What does ASYNC_NETWORK mean ? What does ASYNC_NETWORK mean ? 2. For Dimensions Optimise for Select Distinct Optimise for Select Distinct Or Switch “by Table Mode” Or Switch “by Table Mode” 3. For Facts 1. Ensure narrow base table (no joins!) 2. Optimise Scan speed 512k Read Ahead 512k Read Ahead Fast Disks Fast Disks Good File Group Layout Good File Group Layout No Extent Fragmentation (-E trace flag) No Extent Fragmentation (-E trace flag)

Processing Tuning Approach Measure Object Sizes Who is hogging space Measure Process Time Where is Timing Being Spent Attributes/Dimensions Process Data Bitmap Indexes Aggregations Optimise the Cube/Dimensions Tune the Engine/Hardware Repeat Baseline

Process Data Review Grain Increase BufferRecordLimit Or Aggregate in SQL Too many Measures ? Use Calculated Measures Do we have disk bottleneck Do we have Network bottleneck Are we consuming a lot of CPU Match SSAS to SQL Types Idle Server Resources Increase Parallelism

Optimising Bitmap Indexes 1. Optimise Attributes on Dimensions 2. Ensure good CPUs ! 3. Ensure max parallelism 4. Don’t over partition 5. Will sorting and Compression Help ?

Optimising Compression by Sorting Each Segment in MOLAP is 64k multidimensional-it-is-the-order-of-things/ multidimensional-it-is-the-order-of-things/

Optimising Aggregations Use correct Attribute Relationships Don’t over aggregate (say 20) Consider consolidating aggregations Avoid spilling to disk Ensure max parallelism CPU should be saturated!

Benchmarking Processing Time Use XMLA to Process interesting objects Use XMLA to capture a trace file Use logman to capture perfmon counters Disk throughput and latency CPU SSAS Threads Automate with a batch file

Analysing the Trace File 

Case Study – Tuning AW Size Before 8.8 GB Data, 1.2 GB Index Size After 8.5 GB, 0.8 GB Index

Dimension / Fact Structures ExtensionDescription Kstore, Asstore, Ahstore, bStore, Ahstore Attribute Stored DstoreHierarchy Decode Store lStoreHierarchy Child/Parent Structure SstoreSet Store. Ancestors of a level oStoreOrder Store. Contains Position of each level. BstoreBlob Store MapBitmap Index ExtensionDescription Fact.dataRaw Fact Data Map, HdrBitmap index Files Agg.flex.dataFlexible Aggregations Aff.Rigid.dataRigid Aggregations

Processing Options/Strategy Process Full Drops related Fact data! Process Update May drop flexible Aggregations and Indexes Will bloat over time Process Add Fastest Processing Option Not available via GUI Other Process Commands ProcessData/IndexProcessStructureProcessClear

Perfmon Counters ObjectCounter Processor% Processor Time Logical DiskAvg. Disk sec/Transfer Disk Transfers/Sec ProcessingRows Read/sec Proc IndexRows/sec Proc AggregationsRows Created/Sec Temp file bytes written/sec Current Partitions ThreadsProcessing pool idle threads Processing pool job queue length Processing pool busy threads MemoryMemory Usage Kb High/Low too

ENGINE TUNING TIPS

Connection String Check Data Source Try OLEDB driver instead of SqlClient Max TCP/IP Packet Size set to 32,756 Up default connections from 10 Don’t use TCP/IP on shared server

INI Settings See Analysis Services Performance Tuning paper INI SettingImpact OLAP\Process\ AggregationMemoryLimitMin AggregationMemoryLimitMax Increase Max if spill to disk Decrease Min if lots of partitions ThreadPool\Process\MaxThreadsIncrease if maxed out CoordinatorExecutionModeIncrease Threads per core BufferMemoryLimitImprove data grouping TempDirCan place on faster disk

Processing Options & Strategy Dimensions Process Full ProcessDataProcessIndex Process Update ProcessAdd Cube/Measure Group ProcessFullProcessDataProcessIndexProcessAdd Can we use ROLAP ?

THE TABULAR MODEL Cube Processing Deep Dive

Tabular Processing Events

Tabular Processing Types ProcessFullProcessDataProcessAddProcessRecalcProcessDefragProcessClear

Baselining Tabular Processing 

Sizing Tool 

Optimising Columns Don’t store unused columns like fact keys Reduce Precision Split high cardinality columns Sort we improve sorting ProcessingTimeBoxSecPerMRow Review Segment Size DefaultSegmentRowCount Smaller may process faster in parallel Larger may compress better

Other Tips for tabular Replace simple calculated columns with DAX Optimise the ExecuteSQL for table scan Replace multiple ProcessFull with ProcessData and Process Recalc Avoid NUMA hardware

Should we Partition ? CANNOT be processed in parallel No support for partition elimination

Thank You

THE CRAIC SQL SATURDAY #310 DUBLIN STAY Hilton Hotel Charlemont – rooms from €159 B&B REGISTER com/310/ LEARN FROM INTERNATIONAL EXPERTS - -Sept 19th: 6 x Precons - -Sept 20th: 6 x Tracks – BI,DEV,DBA,2014,Cloud 20th Sept DUBLIN PARTY After party and entertainment during the event

Tabular Disk Structures ExtensionDescription XMLMeta Data IDFCompressed Column Store or Index DICTIONARYMap of DataID to Value HIDXAttribute Hierarchy

HARDWARE Selecting hardware for Analysis Servics

Considerations for HW SSAS does not perform well on NUMA Very sensitive to CPU and memory FE is single threaded Disk speed not as important for Tabular The intel E5/7 v2 chips are almost a must!  

Scale Out MOLAP does not really “scale up” for querying