1. BI Workloads and How They Change in the Cloud
Trey Johnson
Chief Evangelist, ZAP

2. Sponsors
A quick comment about sponsors. SQL Saturdays cannot take place without the funding provided by sponsors.
The speakers are not paid. The organizers and other folks running around making sure this event runs smoothly are all volunteers.
However, his facility, the food, and other expenses that go into putting on an event of this magnitude requires money. Sponsors provide that money. So, show your appreciation by saying hi and thank you when you stop by the sponsor tables to stuff your raffle ticket into the box. You might even take a couple of minutes to ask about their product and services. You may learn something valuable that you can bring back to your work, or that might become a career opportunity. It's all part of the very important networking you should be doing while you are here.

3. About Me Career Built on BI/DW/Data Science (25 years)
Chief Evangelist for one of the World’s Greatest Data Management Automation Products, ZAP Data Hub (10 Years) 
Speaker and Member of the SQL Server/BI Community (19 Years)
Former Board Member of PASS (6 Years)
on Twitter

4. What this session is…
A chance to talk through the BI workloads most people face On Premises today
A chance to discuss how those workloads change as we move to the cloud
A conversation and not a bunch of demonstrations (It’s okay if we abandon the slides at some point!)
A chance to win a small gift (if you are on the right team) 

5. So before we begin, a few questions….
How many of you:
are willing to participate in the conversation?
are predominantly On Premises with your BI/DW/Data Science Data Platform?
have been directed to move BI to the cloud?
would NOT consider yourselves BI/DW/Data Science type of people?

6. Join a Team Every other person is Team Red or Team Blue.
Teams earn points through responses.
I’m keeping score 
The more you participate, the more you help your team and the more we ALL win!

7. Variables which drive the move to Azure
Infrastructure Age & Infrastructure Costs
New Sources of Data
Other Cloud Investments
Change in Staff (New Leadership, Different Personnel)
Perception of being more Nimble/Agile “Off Prem”

8. Your Decision Points about what part of the Data Platform to Use

9. “On Prem” Architecture – ETL or ELT

10. “On Prem” Architecture – What are your Data Sources…
…and where do they end up, today?
Data Mart
Data Warehouse
A Relational Data Store or ?

11. “On Prem” Architecture - Transform and Analyze
Depends on the “T” in ETL or ELT
Traditionally structured as a dimensional model
Model materializes somewhere in either the Relational Data Store or a downstream Multi-Dimensional/Tabular Data Store

12. “On Prem” Architecture - Typical Transforms
Lookup Transformations
SCD Transformations
Row Transformations
Data Enrichment (External Lookups, Derived Columns, etc…)
Rowset Transformations (Aggregate, Sampling, Sort, Pivot, etc…)
Split Transformations (Conditional, Multicast)
Join Transformations (Union, Merge, Join)

13. “On Prem” Architecture - Analysis Services / BISM

14. Got Dimensional Model?

15. “On Prem” Architecture - Transform and Analyze
Are you using SSIS or something else to populate your current “On Prem”?
Is Analysis Services in use or do you simply provide data relationally from a warehouse/mart DB?

16. “On Prem” Architecture - Visualize and Decide
Your Visualization Platform might be the MOST insulated architecturally as you look to move to Azure.
Of Course, there are caveats with this, too!

17. Workloads Being Managed – On Prem
Applications – ETL/ELT, Models and Database Engines
Data – Raw Sourced Data through Curated Data Stores
Runtime – Execution of all Processes PLUS Performance of Platform
O/S - Patching, Firewalls and Security
DB/OLAP DB Engines – Especially DBAs
Networking – Include Security here, too

18. The Typical Pro’s and Con’s of On Prem PRO CON
Speed of Local Data Transfer
High Degree of Oversight on Executions
Inherent Security (Local Traffic)
Non-Throttled Connectivity
Flexibility of BI Tools
Management Overhead
Higher Ongoing Ownership Costs
Scalability
Ease of Sharing (if everything is behind the firewall)
Patching (O/S, Apps)
Still Latency around Realtime
Maintaining Rigid Data Models due to complex ELT/ETL

19. The Typical Pro’s and Con’s of On Prem PRO CON
Speed of Local Data Transfer
High Degree of Oversight on Executions
Inherent Security (Local Traffic)
Non-Throttled Connectivity
Flexibility of BI Tools
Management Overhead
Higher Ongoing Ownership Costs
Scalability
Ease of Sharing (if everything is behind the firewall)
Patching (O/S, Apps)
Still Latency around Realtime
Maintaining Rigid Data Models due to complex ELT/ETL
What PROs and CONs do you have?
Is this driving you to consider MORE of an Azure footprint?

20. Is it an absence of “Pros” or the presence of “Cons” which are driving you to look to Azure? Do you have other concerns about moving to Azure?

21. Source 1: James Serra - Slideshare

23. The Modern SQL Data Warehouse
1 Combine all your structured, unstructured and semi-structured data (logs, files, and media) using Azure Data Factory to Azure Blob Storage.
2 Leverage data in Azure Blob Storage to perform scalable analytics with Azure Databricks and achieve cleansed and transformed data.
3 Cleansed and transformed data can be moved to Azure SQL Data Warehouse to combine with existing structured data, creating one hub for all your data. Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
4 Build operational reports and analytical dashboards on top of Azure Data Warehouse to derive insights from the data, and use Azure Analysis Services to serve thousands of end users.
5 Run ad hoc queries directly on data within Azure Databricks.

24. Your Data Warehouse in Azure?
1 Combine all your structured, unstructured and semi-structured data (logs, files, and media) using Azure Data Factory to Azure Blob Storage.
2 Leverage data in Azure Blob Storage to perform scalable analytics with Azure Databricks and achieve cleansed and transformed data.
3 Cleansed and transformed data can be moved to Azure SQL Data Warehouse to combine with existing structured data, creating one hub for all your data. Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
4 Build operational reports and analytical dashboards on top of Azure Data Warehouse to derive insights from the data, and use Azure Analysis Services to serve thousands of end users.
5 Run ad hoc queries directly on data within Azure Databricks.

25. Which are you considering for moving your BI Workloads to Azure?
Are you “Lifting” and “Shifting” or going through a Re-Engineering Exercise, too?

26. Azure Data Factory vs SSIS
Graphic courtesy of Reza Rad

27. Azure Data Factory PLUS SSIS

28. Azure Data Factory PLUS SSIS
Can you actually do what you need to in ADF, easily?
If you can’t, you probably will be able to sooner.

29. Key Thoughts about ADF
You may be best updating your SSIS packages initially to validate your decisions about Azure Data Platform before trying to convert to ADF.
No doubt that ADF plays best with the rest of the Azure Platform (HDInsight, ML, Data Lake, Databricks, etc…)
Will continue to be invested in and that’s why the story around Connectors, Control Flow, Transforms AND SSIS integration has gotten better!
It feels like a move in the “Developer” direction and there are multiple language dependencies for the transforms/activities (Pig, MapReduce, Spark, U-SQL, Jar, Python, Custom [.NET], Machine Learning, etc…)
Stick close to smart folks like Andy Leonard! (has a session here tomorrow)

30. Some Cool ADF Reference Material
together-with-adf-v2-preview/
expression-language-functions

31. Azure Analysis Services

32. Previously used Analysis Services?
What’s in Azure AS (PaaS)?
Tabular Models
Partitions
Perspectives
Row-Level Security
Translations
What’s Not (only in SQL Server)?
Multidimensional Models (Cubes)
Named Sets
Attach/Detach
MSMDPUMP
Are you on a multidimensional cube now?
Need advise on the transition?

33. Analysis Services : Move to Azure?
Challenges
Feature Differences between SSAS Tabular and SSAS MD
Migrations are “Hand Crafted” for the most part
Re-Writing of Calcs in DAX vs MDX
A good reference…
overview/

34. Any Other Questions?
Feel free to reach out to me at: Or follow on Or Learn more about ZAP Data Hub as part of your path to Azure at: