Real Time Data with Azure and Power BI Warner Chaves Principal Consultant @ Pythian

Meet Your Presenter Data Pro and Consultant for 11 years. Started at HP in Costa Rica, now work at Pythian in Ottawa, Ontario. Microsoft Data Platform MVP. Blog: Sqlturbo.com Email: warner@sqlturbo.com Company: Pythian.com

Agenda GOAL: to go over the building blocks to build a robust real time data consumption architecture with Azure and Power BI. We will cover: Why Real Time? Architecture Demos

Why Real Time?

What do we mean? Data delivered to the end consumer at the same rate as it is generated. In realistic terms, we’re looking at generation to consumption delay of 1 minute or less. < 1 min

The business driver Modern business competitiveness and user expectations demand faster time to insight Traditionally Batch style processing Hours / Day delay Infrastructure expertise Reporting / Visualization tools gated by IT or BI Dev teams Modern Streaming data Minutes / Seconds delay Cloud turnkey components Dynamic self-service BI

End to End Architecture

< 1 min We’re going to look at one (not the only) reference architecture to do this in Azure and Power BI

Event Hub Stream Analytics Power BI Blob Storage HDInsight SQL Data Warehouse Analysis Services

Real-time datasets Streaming Dataset Push Dataset Supports streaming tiles Limited graphics options Very fast refresh rate Redis cache under the covers (1 hour) Push Dataset Supports regular report creation All visuals supported Lower refresh rate Azure SQL Db under the covers (200K records if FIFO is enabled)

Push or Streaming? You can create either one through the REST API. You can also create both at once ‘PushStreaming’. StreamAnalytics creates a PushStreaming one with a FIFO retention policy. For most cases I would recommend PushStreaming or Push since Streaming only is very limited on visuals.

Demos!

Recap Demand for real-time analysis is growing. Azure provides all the building blocks for a robust, performant end to end architecture. There are many ways to tune and tweak, use this knowledge and apply to your use case. Go forth and build!

Questions?

Thank you for Attending! Don’t forget to join your local PUG to enjoy year-round networking and learning. www.pbiusergroup.com/Toronto