1. Introduction to Microsoft Azure SQL Data Warehouse www.Henkvandervalk.com

2. Thanks to our platinum sponsors : PASS SQL Saturday – Holland - 20152 |

3. Thanks to our silver and gold sponsors : PASS SQL Saturday – Holland - 20153 |

4. Speaker Introduction  10 years active in SQLPass community!  10 years of Unisys-EMEA Performance Center  2002- Largest SQL DWH in the world (SQL2000)  Project Real – (SQL 2005) ETL WR - loading 1TB within 30 mins (SQL 2008) Contributor to SQL performance whitepapers Perf Tips & tricks: www.henkvandervalk.comwww.henkvandervalk.com  Schuberg Philis- 100% uptime for mission critical apps  Since april 1st, 2011 – Microsoft ! All info represents my own personal opinion (based upon my own experience) @HenkvanderValk

5. Agenda  Intro / SQLDW in 5 minutes  Getting started  Create your own SQLDW via Azure Portal  SQL SMP Migration Utility  Load data BCP / Polybase  Tables  Monitoring MPP query execution  Backup/Restore

6. Microsoft Analytics Suite Transform Data into Intelligent Action Business Scenarios DATAINTELLIGENCE ACTION Big Data Stores

7. Introducing SQL Data Warehouse  Fully managed relational data warehouse-as-a-service  The first elastic cloud data warehouse with enterprise-grade capabilities  Supports the smallest to largest data sets Elastic scale & performance Market Leading Price & Performance

8. Elastic Scale  Spin up for heavy workloads, cycle down for daily activity  Buy time to insight based on what you need, when you need it  Choose the combo of compute and storage that meets your needs

9. Pause  Data remains in place – no reloading / restoring of data  When paused, cloud-scale storage is at minimal cost  Automate via PowerShell/REST API $$$$

10. Its all about flex- & scalability

11. Data Warehouse Unit (DWU) Simply buy the query performance you need, not just hardware Quantified by workload objectives: how fast rows are scanned, loaded, copied Measure of Power Transparency First DW service to offer compute power on demand, independent of storage On Demand Scan 1B rows 100 DWU=xx sec 400 DWU=xx sec 800 DWU=xx sec 1,600 DWU=xx sec Scan Ratexx M row/sec Loading Rate xx K row/sec Table Copy Rate xx K row/sec 100 DWU

12. Market-Leading Price/Performance  Best On-Demand Price/Performance  Advantages in elasticity and pause to reduce customer cost  SQL DW start small, can grow to PB+  Pay for performance by scaling compute against storage 100GB1TB2TB 1+PB SQL DW High Performance SQL DW High Capacity SQL DW Balanced Performance

13. Azure Security Configure Server level Firewall for remote access

14. Demos  Create a SQLDW DB via Azure portal  DWH Migration utility  Lets Scale SQL DW up& down!

16. Signup for Preview

17. Create a V12 SQL DB

19. SQL DB Logins In Master DB:  CREATE LOGIN [cloudsa] WITH PASSWORD = 'SQLSAT-434‘ In SQLDW DB:  CREATE USER CloudSA FOR LOGIN CloudSA;  EXEC sp_addrolemember 'db_datareader', 'CloudSA'; -- allows ApplicationUser to read data  EXEC sp_addrolemember 'db_datawriter', 'CloudSA'; -- allows ApplicationUser to write data

20. Loading data with BCP BCP sqldw.dbo.lineitem_CCI_RR in C:\Toolbox-SQL\dbgen\lineitem.tbl -c -U cloudsa -P SQLSAT-434 -S henks-sqldw.database.windows.net -q -t"|" 600572 rows copied. Network packet size (bytes): 4096 Clock Time (ms.) Total : 72875 Average : (8241.13 rows per sec.)

21. Scale #DWU’s via PowerShell

22. Scale #DWU’s via TSQL Via master DB: ALTER DATABASE [dbo].[SQLSATDW] { SET SERVICE_OBJECTIVE = 'DW1000' };

23. MPP vs. SMP Architecture vsvs

24. SQL DW: Building on SQL DB Foundation Elastic, Petabyte Scale DW Optimized 99.99% uptime SLA, Geo-restore Azure Compliance (ISO, HIPAA, EU, etc.) True SQL Server Experience; Existing Tools Just Work SQL DW SQL DB Service Tiers

25. Broad Ecosystem Leverage Azure ML,a Power BI, ADF, Stream Analytics and more Streamlined deployment with Azure Portal Deep tool integration with top partners Azure SQL DW Azure ML Azure Event Hub Azure HDInsight

26. Query Unstructured Data via Polybase T-SQL query SQL Server Hadoop Quote: ************************ ********************** ********************* ********************** *********************** $658.39 Jim Gray Name 11/13/58 DOB WA State Ann Smith04/29/76ME

27. Polybase demo Direct Read/Write Into SQLDW From/to WASB (Windows Azure blob storage data)

28. SQLDW Backup & Restore

29. Monitoring SQLDW Tracing exact query response timings: select command, * from sys.dm_pdw_exec_requests where submit_time > '2015-09-25 00:00:01.953' and command not like '%SELECT SERVERPROPERTY(%' and command not like '%DECLARE @edition sysname% ' Find the longest running queries: SELECT * FROM sys.dm_pdw_exec_requests ORDER BY total_elapsed_time DESC

30. Check if queries are waiting for resources SELECT waits.session_id, waits.request_id, requests.command, requests.status, requests.start_time, waits.type, waits.object_type, waits.object_name, waits.state FROM sys.dm_pdw_waits waits JOIN sys.dm_pdw_exec_requests requests ON waits.request_id=requests.request_id WHERE waits.request_id = 'QID33188' ORDER BY waits.object_name, waits.object_type, waits.state;

31. Example – Setup Keys & Credentials  -- This is only one time for a database  CREATE MASTER KEY;  CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'SecurePassword';  -- This is required once per a new credential.  CREATE CREDENTIAL ProdCreds ON DATABASE WITH IDENTITY = ‘StorageName', Secret = 'StorageKeyValueGoesHere';

32. Example – Data Source  -- Create a data source  CREATE EXTERNAL DATA SOURCE [WebLogSource]  WITH  (  TYPE = HADOOP,  LOCATION = 'wasbs://weblogs@sqldwdemo.blob.core.windows.net/',  CREDENTIAL = ProdCreds  );

33. Example – File Format  -- Create the file format  CREATE EXTERNAL FILE FORMAT [WebLogFormat] WITH  (  FORMAT_TYPE = DELIMITEDTEXT,  FORMAT_OPTIONS  (  FIELD_TERMINATOR = ‘|',  DATE_FORMAT = 'yyyy-MM-dd HH:mm:ss'  )  );

34. Example – External Table  -- Create the external Table  CREATE EXTERNAL TABLE [WebLogs]  (  url VARCHAR(50),  event_date DATE,  user_IP VARCHAR(50)  )  WITH  (  LOCATION='weblogs',  DATA_SOURCE = WebLogData,  FILE_FORMAT = WebLogFormat  );

35. Symmetry  Integrated with Power BI, Azure Machine Learning, and Azure Data Factory App ServiceAzure SQL Database Azure Machine Learning Intelligent App Hadoop Azure SQL Data Warehouse Power BI

36. Online Resources  SQLDW Landing page: https://azure.microsoft.com/en-us/documentation/articles/sql-data-warehouse- get-started-provision/  Subscribe to fortnightly newsletter  Channel 9 Sessions on Demand  MVA Free Online Learning Henkvandervalk.com http://aka.ms/technetnzhttp://aka.ms/technetnz http://aka.ms/msdnnzhttp://aka.ms/msdnnz http://aka.ms/ch9nz http://aka.ms/mva

37. Please review the event & sessions!  http://www.sqlsaturday.com/434/eventeval.aspx http://www.sqlsaturday.com/434/eventeval.aspx  http://www.sqlsaturday.com/434/sessions/sessionevaluation.aspx http://www.sqlsaturday.com/434/sessions/sessionevaluation.aspx PASS SQL Saturday – Holland - 201537 |

38. Appendix - Demo scripts ----------------------------------------------------------------------------------- --- SQLSATURDAY434 - Create distributed CCI table ---- ----------------------------------------------------------------------------------- --- Setup notes: --- Server: henks-sqldw.database.windows.net ----------------------------------------------------------------------------------- select @@VERSION CREATE TABLE [dbo].[lineitem1] ( [l_shipdate] DateTime NOT NULL,[l_orderkey] BigInt NOT NULL,[l_discount] Float NOT NULL,[l_extendedprice] Float NOT NULL,[l_suppkey] Int NOT NULL,[l_quantity] Float NOT NULL,[l_returnflag] Char(1) NOT NULL,[l_partkey] Int NOT NULL,[l_linestatus] Char(1) NOT NULL,[l_tax] Float NOT NULL,[l_commitdate] DateTime NOT NULL,[l_receiptdate] DateTime NOT NULL,[l_shipmode] Char(10) NOT NULL,[l_linenumber] Int NOT NULL,[l_shipinstruct] Char(25) NOT NULL,[l_comment] VarChar(132) NOT NULL ) WITH (CLUSTERED COLUMNSTORE INDEX, DISTRIBUTION = HASH([l_orderkey])) ------------------------------------------------------------------------------------------------------------------------- --- SQLSATURDAY434 - Polybase CETAS reading from Flatfile on Azure Blob Storage ---- ------------------------------------------------------------------------------------------------------------------------- SELECT * FROM sys.external_tables SELECT * FROM sys.external_data_sources SELECT * FROM sys.external_file_formats CREATE EXTERNAL DATA SOURCE dws_wasb_storage WITH ( TYPE = HADOOP, LOCATION = 'wasbs://SQLSAT434_Container@henks_storage_account.blob.core.windows.net/' ); CREATE EXTERNAL FILE FORMAT pipe_delimited_text_file_format WITH ( FORMAT_TYPE = DELIMITEDTEXT ) CREATE EXTERNAL TABLE [stage].[FactSalesStage_FULL_WASB] ( [OnlineSalesKey] bigint NULL, [DateKey] datetime NULL, [StoreKey] int NULL, [ProductKey] int NULL, [PromotionKey] int NULL, [CurrencyKey] int NULL, [CustomerKey] int NULL, [SalesOrderNumber] varchar(28) COLLATE SQL_Latin1_General_CP1_CI_AS NULL, [TotalCost] money NULL, [UnitCost] money NULL, [UnitPrice] money NULL, [ETLLoadID] int NULL, [LoadDate] datetime NULL, [UpdateDate] datetime NULL ) WITH (LOCATION='Fact.txt', DATA_SOURCE = dws_wasb_storage, FILE_FORMAT = pipe_delimited_text_file_format, REJECT_TYPE = VALUE, REJECT_VALUE = 0 ) select count_big (*) from stage.[FactSalesStage_WASB]

39. Appendix – Demo scripts (cont.) ----------------------------------------------------------------- --- SQLSATURDAY434 - Monitoring SQLDW ---- ----------------------------------------------------------------- SELECT * from sys.tables -- List all tables with distribution types: SELECT a.name AS [Table Name], b.distribution_policy_desc AS [Distribution Type] FROM sys.tables AS a JOIN sys.pdw_table_distribution_properties AS b ON a.object_id = b.object_id --Data alignment over all distributions: DBCC PDW_SHOWSPACEUSED("dbo.FactOnlineSales"); -- Active Sessions and connections:: SELECT top (10) * FROM sys.dm_pdw_nodes_exec_connections; SELECT top (10) * FROM sys.dm_pdw_nodes_exec_sessions where login_time > '2015-09-26' -- Info on current connection: SELECT * FROM sys.dm_pdw_nodes_exec_connections AS c JOIN sys.dm_pdw_nodes_exec_sessions AS s ON c.session_id = s.session_id WHERE c.session_id = @@SPID; -- Query History: select command, * from sys.dm_pdw_exec_requests where submit_time > '2015-09-25 00:00:00.953' and command not like '%SELECT SERVERPROPERTY(%' and command not like '%DECLARE @edition sysname% ' --- Query Execution: -- Monitor running queries: SELECT * FROM sys.dm_pdw_exec_requests WHERE status = 'Running'; -- Find the longest running queries: SELECT * FROM sys.dm_pdw_exec_requests ORDER BY total_elapsed_time DESC; -- Sample : QID152698 -- Zoom into MPP execution plan: -- Find the distribution run times for a SQL step. SELECT * FROM sys.dm_pdw_sql_requests WHERE request_id = 'QID152698' --AND step_index = 2; -- Find the information about all the workers completing a Data Movement Step. -- (Rows processed/ node ID/ duration! -- USE Explain to see MPP execution plan: explain SELECT * FROM sys.dm_pdw_dms_workers WHERE request_id = 'QID152698'