Power BI Architecture, Best Practices, and Performance Tuning Microsoft Ignite 2016 6/3/2018 7:51 PM Power BI Architecture, Best Practices, and Performance Tuning DA331a Sirui Sun © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Motivation Why think about performance and best practices? Performant reports are delightful to use No one will use a report with bad performance How to ensure good performance? Understand the tools Use them properly

Agenda 50,000 ft. view – where is the time spent Choosing how Power BI connects to your data Import DirectQuery Live connection Gateway best practices Network latency All-up best practices MICROSOFT CONFIDENTIAL

The view from 50,000 feet Cloud data sources On-prem data sources Azure Analysis Services, SQL Azure, etc. On-prem data sources Analysis Services, SQL Server, etc. Scheduled refresh/ Live connection/ DirectQuery Consume Consumption by end users PBI Desktop Group OneDrive PBI Service Datasets Reports Dashboards Scheduled refresh/ Live connection/ DirectQuery On-Premises Gateway

Where is the time spent? Data sources (and how they’re connected) Cloud data sources Azure Analysis Services, SQL Azure, etc. PBI Desktop On-Premises Gateway Group OneDrive On-prem data sources Analysis Services, SQL Server, etc. PBI Service Scheduled refresh/ Live connection/ DirectQuery Datasets Reports Dashboards Consume Consumption by end users

Where is the time spent? Gateway Cloud data sources Azure Analysis Services, SQL Azure, etc. PBI Desktop On-Premises Gateway Group OneDrive On-prem data sources Analysis Services, SQL Server, etc. PBI Service Scheduled refresh/ Live connection/ DirectQuery Datasets Reports Dashboards Consume Consumption by end users

Where is the time spent? Network latency Cloud data sources Azure Analysis Services, SQL Azure, etc. Consume Scheduled refresh/ Live connection/ DirectQuery Consumption by end users PBI Service Group OneDrive PBI Desktop Datasets Reports Dashboards Scheduled refresh/ Live connection/ DirectQuery On-prem data sources Analysis Services, SQL Server, etc. On-Premises Gateway

Where is the time spent? Power BI Service, Clients Cloud data sources Azure Analysis Services, SQL Azure, etc. PBI Desktop On-Premises Gateway Group OneDrive On-prem data sources Analysis Services, SQL Server, etc. PBI Service Scheduled refresh/ Live connection/ DirectQuery Datasets Reports Dashboards Consume Consumption by end users Scheduled refresh/ Live connection/ DirectQuery

Choosing how Power BI connects to your data 6/3/2018 7:51 PM Choosing how Power BI connects to your data © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Where is the time spent? Data sources (and how they’re connected) Cloud data sources Azure Analysis Services, SQL Azure, etc. PBI Desktop On-Premises Gateway Group OneDrive On-prem data sources Analysis Services, SQL Server, etc. PBI Service Scheduled refresh/ Live connection/ DirectQuery Datasets Reports Dashboards Consume Consumption by end users

Review: How Power BI views your BI data model Imported Models Aka Cloud Models Dataset metadata and data stored in Power BI External Models Data stored in external data source Metadata stored in Power BI i.e. DirectQuery, Live connection MICROSOFT CONFIDENTIAL

Key factors to consider when scaling with Power BI Simple Moderate Complex Model size and complexity Data update frequency Active concurrent users Number of queries Query latency time

Cloud Models Best for: Small-to-medium models with moderate concurrent usage Microsoft Confidential MICROSOFT CONFIDENTIAL

Cloud data sources On-prem data sources SQL Azure, Google Analytics, etc. Consume Consumption by end users Scheduled refresh Data is copied to Power BI at regular intervals PBI Service PBI Desktop Datasets Reports Dashboards Dashboard/Reports are updated with latest data at the end of scheduled refresh. Queries are made against the copy of the data in Power BI. Scheduled refresh Data is copied to Power BI at regular intervals On-prem data sources SQL Server, SAP HANA, etc. On-Premises Gateway

No. of concurrent dashboard/report users Microsoft Ignite 2016 Cloud Models 6/3/2018 7:51 PM Target Scenario Small to medium models with moderate concurrent usage Query latency Low (Same DC) (In-Memory) Data freshness Data updated up to 8 times/day Data model size limit 1GB No. of concurrent dashboard/report users Throttled © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Live connection to SSAS Best for: Large models Frequent data updates (>8x day) High concurrent usage Microsoft Confidential MICROSOFT CONFIDENTIAL

Live connection to SSAS Azure Analysis Services Live connection Power BI queries Analysis Services to populate visuals Queries are made against Analysis Services Consumption by end users PBI Service Group OneDrive PBI Desktop Datasets Reports Dashboards Dashboards are auto-updated every 15m with latest data for fast load. Analysis Services On-premises On-Premises Gateway

No. of concurrent dashboard/report users Microsoft Ignite 2016 6/3/2018 7:51 PM Live connection to SSAS Target Scenario Workloads that have one or more of the following characteristics: Large models Frequent data updates (<3 hrs) High concurrent usage On-Prem Azure Query latency Medium (Multi-Hop) (In-Memory) Low (Same DC) Data refreshness Depends on how Analysis Services configuration Data model size limit Limited by data source or Analysis Services model size No. of concurrent dashboard/report users Scales with machine configuration © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Specific guidance for SSAS Microsoft Ignite 2016 6/3/2018 7:51 PM Specific guidance for SSAS Scale up SSAS instances RAM >= 2.5 * size(models) for full processing © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Specific guidance for SSAS Microsoft Ignite 2016 6/3/2018 7:51 PM Specific guidance for SSAS Scale up SSAS instances RAM >= 2.5 * size(models) for full processing Be sure to use SSAS 2016 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Specific guidance for SSAS Microsoft Ignite 2016 6/3/2018 7:51 PM Specific guidance for SSAS Scale up SSAS instances RAM >= 2.5 * size(models) for full processing Be sure to use SSAS 2016 DAX is being sent by PBI to SSAS 2016 is significantly optimized Set up performance monitoring © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

DirectQuery Best for: Workloads that have a near real-time requirement Microsoft Confidential MICROSOFT CONFIDENTIAL

DirectQuery Azure SQL SQL Server On-premises Consumption by end users On report view/exploration: queries are made against underlying data source DirectQuery Power BI queries data source to populate visuals Direct Query PBI Service Datasets Reports Dashboards PBI Desktop Every 15m: Dashboards are auto-updated every with latest data for fast load. SQL Server On-premises On-Premises Gateway

Max active concurrent report users DirectQuery Target Scenario Workloads that have a near real-time requirement On-Prem Azure Query Latency High (Multi-Hop) (DQ) (Same DC) Data model size limit Limited only by data source Max active concurrent report users Varies depending on data source limits Microsoft Confidential

General guidance for DirectQuery Microsoft Ignite 2016 6/3/2018 7:51 PM General guidance for DirectQuery Ensure that in Desktop, the performance of refreshing/interacting with visuals is reasonable Recommended: <5-10s © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

General guidance for DirectQuery Microsoft Ignite 2016 6/3/2018 7:51 PM General guidance for DirectQuery Ensure that in Desktop, the performance of refreshing/interacting with visuals is reasonable Recommended: <5-10s Understand DQ-specific constraints E.g. no time intelligence, limited to single data source, limited DAX functions, no bi-directional filtering © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

General guidance for DirectQuery Microsoft Ignite 2016 6/3/2018 7:51 PM General guidance for DirectQuery Ensure that in Desktop, the performance of refreshing/interacting with visuals is reasonable Recommended: <5-10s Understand DQ-specific constraints E.g. no time intelligence, limited to single data source, limited DAX functions, no bi-directional filtering Understand Data source limitations and load implications E.g. SQL Data Warehouse has a limit on concurrent queries, based on deployment size Source: https://docs.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-develop-concurrency © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Summary Import: Live connection to SSAS DirectQuery Small model size (<1 GB) Medium concurrency needs Infrequent data updates (<8x day) Live connection to SSAS Medium-to-large model size (>1 GB) High concurrency needs More frequent data updates DirectQuery Only if you have near-real-time requirement Be aware of the caveats!

Additional Scalability Considerations Guidelines and data points in the previous table are based on typical usage and may be affected by a number of factors, including: Query complexity Row level security Network latency and/or data center location Usage patterns (e.g. dashboards vs reports vs interactive filtering)

Additional Scalability Considerations Remember that these factors can be multiplicative (Many Visuals) X (Frequent Data Updates) Consumption by end users PBI Service Datasets Reports Dashboards Tile 1 Tile 2 Tile 3 On-premise model Analysis Services On-Premises Gateway

Additional Scalability Considerations Remember that these factors can be multiplicative With RLS: (Many Visuals) X (Frequent Data Updates) X (Number of users) Consumption by end users PBI Service Datasets Reports Dashboards Tile 1A Tile 2A Tile 3A On-premise model Tile 1B Tile 2B Tile 3B Tile 1C Tile 2C Tile 3C Analysis Services On-Premises Gateway

6/3/2018 7:51 PM Gateways © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Gateways Cloud data sources On-prem data sources Azure Analysis Services, SQL Azure, etc. Consume Scheduled refresh/ Live connection/ DirectQuery Consumption by end users PBI Service Group OneDrive PBI Desktop Datasets Reports Dashboards On-Premises Gateway Scheduled refresh/ Live connection/ DirectQuery On-prem data sources Analysis Services, SQL Server, etc.

Gateways Best Practices Use the On-premises Data Gateway

Gateways Best Practices Use the On-premises Data Gateway Min system recommendation: 8 GB RAM/8 CPU cores

Gateways Best Practices Use the On-premises Data Gateway Min system recommendation: 8 GB RAM/8 CPU cores, Windows Server 2012 R2 Set up performance monitoring on the gateway

Gateways Best Practices Use the On-premises Data Gateway Min system recommendation: 8 GB RAM/8 CPU cores, Windows Server 2012 R2 Set up performance monitoring on the gateway Scale up/out as needed

Gateways Best Practices Use the On-premises Data Gateway Min system recommendation: 8 GB RAM/8 CPU cores, Windows Server 2012 R2 Set up performance monitoring on the gateway Scale up/out as needed Different gateways for DQ vs. import On-Premises Gateway DQ Import On-Premises Gateway

6/3/2018 7:51 PM Network Latency © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Network Latency Cloud data sources On-prem data sources Azure Analysis Services, SQL Azure, etc. Consume Scheduled refresh/ Live connection/ DirectQuery Consumption by end users PBI Service Group OneDrive PBI Desktop Datasets Reports Dashboards Scheduled refresh/ Live connection/ DirectQuery On-prem data sources Analysis Services, SQL Server, etc. On-Premises Gateway

Network Latency Eastern US Australia Cloud data sources Azure Analysis Services, SQL Azure, etc. Consume Scheduled refresh/ Live connection/ DirectQuery Consumption by end users PBI Service Group OneDrive PBI Desktop Datasets Reports Dashboards Eastern US Scheduled refresh/ Live connection/ DirectQuery On-prem data sources Analysis Services, SQL Server, etc. On-Premises Gateway Australia

Network/Client Latency Best Practices Use PBI Desktop/Azure Speed Test to gauge query latency

Network/Client Latency Best Practices Use PBI Desktop/Azure Speed Test to gauge query latency Make sure data sources, gateway and Power BI region are located as close together as possible

Network/Client Latency Best Practices Use PBI Desktop/Azure Speed Test to gauge query latency Make sure data sources, gateway and Power BI region are located as close together as possible Consider Azure ExpressRoute

6/3/2018 7:51 PM All-up Guidance © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

All-up guidance Limit the number of visuals in dashboards/reports Put custom visuals through their paces Set and save filters in reports Limit complicated measures and aggregates in data models Perform load testing on your backend

Additional Resources Walkthrough: how to set up On-Premises Gateway performance monitoring Gateway troubleshooting documentation Gateway Perf counters and logging Power BI security whitepaper (includes architectural deep-dive) Power BI governance and deployment whitepaper SSAS Performance Tuning documentation DirectQuery documentation Azure Speed Test

Continue your Ignite learning path 6/3/2018 7:51 PM Continue your Ignite learning path Visit Channel 9 to access a wide range of Microsoft training and event recordings https://channel9.msdn.com/ Head to the TechNet Eval Centre to download trials of the latest Microsoft products http://Microsoft.com/en-us/evalcenter/ Visit Microsoft Virtual Academy for free online training visit https://www.microsoftvirtualacademy.com © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6/3/2018 7:51 PM Thank you Chat with me in the Speaker Lounge Find me on LinkedIn: sirui-sun © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Appendix

Query latency w.r.t DQ/live connection