1. Honest Bob’s Cube Processing Bob Duffy Database Architect

2. Thank you to our sponsors!

3. Speaker - Bob Duffy  20 years in database sector, 250+ projects  Senior Consultant with Microsoft 2005-2008  One of about 25 MCA for SQL Server globally (aka SQL Ranger)  SQL MCM on SQL 2005 and 2008  SQL Server MVP 2009+  SSAS Maestro  Database Architect at Prodata SQL Centre of Excellence  http://blogs.prodata.ie/author/bob.aspx http://blogs.prodata.ie/author/bob.aspx  bob@Prodata.ie bob@Prodata.ie

4. What we Will Cover  Processing Internals  Dimensions Deep Dive  Measure Group Deep Dive  Tuning Patterns  Baseline Harness  Case Study

5. What's a cube ?

6. Dimension Processing

7. Measure Group Processing

8. Recap: Tuning Attribute Properties  Key Column  Name Column  AttributeHierarchyEnabled  AttributeHierarchyOptimized  AttributeHierarchyOrdered  Add correct relationships  Avoid Large Dimensions  Or  DELETE THEM

9. Dimension / Fact Structures ExtensionDescription Kstore, Asstore, Ahstore, bStore, Ahstore Attribute Stored DstoreHierarchy Decode Store lStoreHierarchy Child/Parent Structure SstoreSet Store. Ancestors of a level oStoreOrder Store. Contains Position of each level. BstoreBlob Store MapBitmap Index ExtensionDescription Fact.dataRaw Fact Data Map, HdrBitmap index Files Agg.flex.dataFlexible Aggregations Aff.Rigid.dataRigid Aggregations

10. Determining Size of Objects http://tinyurl.com/MolapTools

11. Slow Execute SQL ? 1.Check WAIT Statistics What does ASYNC_NETWORK mean ? 2.For Dimensions Optimise for Select Distinct Or Switch “by Table Mode” 3.For Facts 1.Ensure narrow base table (no joins!) 2.Optimise Scan speed 512k Read Ahead Fast Disks Good File Group Layout No Extent Fragmentation (-E trace flag)

12. Process Data  Review Grain  Increase BufferRecordLimit  Or Aggregate in SQL  Too many Measures ?  Use Calculated Measures  Do we have disk bottleneck  Do we have Network bottleneck  Are we consuming a lot of CPU  Match SSAS to SQL Types  Idle Server Resources  Increase Parallelism

13. Optimising Bitmap Indexes 1.Optimise Attributes on Dimensions 2.Ensure good CPUs ! 3.Ensure max parallelism 4.Don’t over partition 5.Will sorting and Compression Help ?

14. Optimising Compression by Sorting  Each Segment in MOLAP is 64k http://dennyglee.com/2013/09/30/analysis-services-multidimensional-it-is-the-order-of-things/

15. Optimising Aggregations  Use correct Attribute Relationships  Don’t over aggregate (say 20)  Consider consolidating aggregations  Avoid spilling to disk  Ensure max parallelism  CPU should be saturated!

16. Benchmarking Processing Time  Use XMLA to Process interesting objects  Use XMLA to capture a trace file  Use logman to capture perfmon counters  Disk throughput and latency  CPU  SSAS Threads  Automate with a batch file http://blogs.prodata.ie/post/capturing-a-service-side-trace-for-mdx.aspx

17. Analysing the Trace File http://blogs.prodata.ie/post/Tool-for-baselining-MOLAP-Processing.aspx

18. Processing Options/Strategy  Process Full  Drops related Fact data!  Process Update  May drop flexible Aggregations and Indexes  Will bloat over time  Process Add  Fastest Processing Option  Not available via GUI  Other Process Commands  ProcessData/Index  ProcessStructure  ProcessClear

19. Perfmon Counters ObjectCounter Processor% Processor Time Logical DiskAvg. Disk sec/Transfer Disk Transfers/Sec ProcessingRows Read/sec Proc IndexRows/sec Proc AggregationsRows Created/Sec Temp file bytes written/sec Current Partitions ThreadsProcessing pool idle threads Processing pool job queue length Processing pool busy threads MemoryMemory Usage Kb High/Low too

20. ENGINE TUNING TIPS

21. Connection String  Check Data Source  Try OLEDB driver instead of SqlClient  Max TCP/IP Packet Size set to 32,756  Up default connections from 10  Don’t use TCP/IP on shared server

22. INI Settings (before 2012)  See Analysis Services Performance Tuning paper INI SettingImpact OLAP\Process\ AggregationMemoryLimitMin AggregationMemoryLimitMax Increase Max if spill to disk Decrease Min if lots of partitions ThreadPool\Process\MaxThreadsIncrease if maxed out CoordinatorExecutionModeIncrease Threads per core BufferMemoryLimitImprove data grouping TempDirCan place on faster disk

23. Partitioning Strategies  For Processing  At least one partition per core  Try and make even size  Ensure fact.data smaller than index/agg data  Use Partitions to avoid updating all data

24. Case Study – Tuning AW  Size Before  8.8 GB Data, 1.2 GB Index  Size After  8.5 GB, 0.8 GB Index

25. Case Study II – Retail Cube Baseline  2.5 Hours Starting Point for 350GB cube

26. Case Study II – Data Types  All Numeric Converted to money or float in sql view.  30% decrease in time for ProcessData

27. Case Study II – Attribute/Aggs  53% overall reduction in cube size  55% reduction in Aggregations  73% reduction in size of Bitmap Indexes

28. Case Study II – Tuning the TSQL  Need to denormalise Fact Table

29. References  Excel Tool for Analysing Process Time  http://tinyurl.com/MolapTools http://tinyurl.com/MolapTools  Excel tool for MOLAP disk space analysis  http://tinyurl.com/MolapDiskSize http://tinyurl.com/MolapDiskSize  Scripts for baselining Cube Processing  http://tinyurl.com/MolapHarness http://tinyurl.com/MolapHarness

30. Your feedback is important! Please let us know what you thought of this session http://sqlsaturday.com/414/sessions/sessionevaluation.aspx

31. Thank You

32. Near Real Time Processing  Process recent partitions frequently  Queries can Block Process Commit  Consider adjusting “ForceCommitTimeout”  Or “Cube flipping”  Tip: Only Consider ROLAP as a last resort

33. Scale Out  MOLAP does not really “scale up” for querying