1. SQLintersection Do you know Data in the Microsoft Cloud? Upgrade Your Life Bob Ward, CTO CSS, Microsoft bobward@microsoft.com http://aka.ms/bobwardms Tuesday, 3:45-5:00pm The Experts: Jason Howell, Azim Uddin, Lisa Liu, Adam Saxton, Rohit Nayak

2. © SQLintersection. All rights reserved. http://www.SQLintersection.com What is Microsoft Azure? Microsoft Azure is a growing collection of integrated cloud services— analytics, computing, database, mobile, networking, storage, and web— for moving faster, achieving more, and saving money….. Use an open and flexible platform Extend your existing IT Scale as you need, pay as you go Protect your data Run your apps anywhere Make smarter decisions Rely on a trusted cloud The Cloud for Modern Business

3. © SQLintersection. All rights reserved. http://www.SQLintersection.com The “No Excuse” Cloud Checklist Internal security measures, isolation and control, and Compliancemeasuresisolation and controlCompliance “I’m concerned about cloud security and privacy” Azure Virtual Machines allow you complete control of your SQL Server instance and configuration “The Cloud doesn’t give me the control I need” Azure ExpressRoute to get WAN network speeds up to 10GbpsExpressRoute Data centers in 20 regions around the worldData centers “I can’t rely on the speed of the public Internet” Azure SQL Database now contains features such as full-text search, TDE, and row-level security “The cloud doesn’t offer the features I need” Hybrid is one of the key diffenteriators for Azure including Azure Backup, SQL Stretch Database, and secondary replicas in Azure VMAzure BackupSQL Stretch Databasesecondary replicas “I can’t move everything to the cloud” Azure Trust Center

4. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data and Microsoft Azure SQL in Azure Virtual Machine Azure SQL Database Azure SQL Data Warehouse Preview Azure DocumentDB Azure HDInsight Azure Data Lake Preview coming Azure Data Factory Azure Machine Learning Power BI DataAnalytics Stream AnalyticsStream Analytics, Search, and Data Catalog PreviewSearchData Catalog

5. © SQLintersection. All rights reserved. http://www.SQLintersection.com SQL Server in Azure Virtual Machine Complete SQL Server “box” running in a VMThe VM is hosted in a Microsoft data center with a variety of sizessizesSQL Server license is “subscription based” or “bring your own”VM usage is “pay as you use it”pay as you use itUse the Marketplace to avoid the pain of setup and configurationMarketplaceHybrid scenarios available and welcome DB YouMicrosoft VM Infrastructure as a Service (IAAS)

6. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure SQL Database  Deploy a database and application. A managed service  Built-in High Availability, Geo-Replication, Restore, and Auditing  Basic, Standard, and Premium Service Tiers (determine features, price, and performance)Service Tiers  Predictable Performance via DTU that can be dynamically adjustedDTU  Close to box parity and new features are here first (row level security and full-text)row level security full-text  Elastic pools to manage large number of databases  Elastic scale-out for distributed performance Elastic scale-out  Stretch Database in SQL Server 2016 for historical archive scenarios Stretch Database DB YouMicrosoft VM Platform as a Service (PAAS)

7. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure SQL Database is Ready Premium now supports 1TB database Point in Time Restore, Geo- Restore, Geo and Active Geo-Replication Application isolation and predictable performance with Service Tiers CLR, XML, Change Tracking, and Full-Text Search now supported Azure Active Directory support as alternative for SQL authentication Columnstore index support Security features such as row-level security, auditing, TDE Monitor and troubleshoot with Query Store, Index Advisor, and Extended Events

8. Demo Let’s explore Azure SQL Database

9. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure SQL Data Warehouse Preview Think APS and SQL Server together in the cloud Fully managed (PAAS) like Azure SQL Database Petabytes storage of data along with MPP for scale-out queries Separate usage and costs for compute (DWU) and storage Full Polybase support for queries that span SQL and Hadoop environments Power BI direct connectivity Solution Partners Solution Partners for ETL solutions

10. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure DocumentDB A fully-managed NoSQL document database service  Why NoSQL document?  Schema-free data storage (no tables and columns predefined)  JSON document storage (preferred by many web developers) JSON  Why database?  Supports transaction processing  Supports database consistency  Automatically indexes JSON documents  Engine programming via procs, triggers, and UDF written in JavaScript  A SQL language query interface built-in  Who would use this?  JSON has become a popular choice for schema-free, unstructured data  Natural choice for any developer working with JSON formatted data Don’t think Word document { "id": “1", “Team": “Dallas Cowboys", “Players": [ { “Name": “Tony Romo“, “Jersey”: 9, “Position” : “QB” }, { “Name": “Dez Bryant“, “Jersey”: 88, “Position” : “WR” }, ], “Conference": “NFC“, “Prediction”: 1 } A JSON DBMS Try Query PlaygroundQuery Playground JSON support JSON support In SQL 2016

11. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure HDInsight Distributed processing of large data (Big) across a cluster of computers Support both Windows and Linux environments all running in Azure Storage based on HDFS that is “schema on read” Job scheduling and resource management through YARN Parallel processing of data through MapReduce (think Batch processing) A semi-managed Hadoop based data and analytics serviceHadoop We simplify the deployment of Hadoop clusters Hortonworks Data Platform (HDP) Core Engine – HDFS and YARN Batch – MapReduce Script – PigSQL – HiveNoSQL – Hbase Streaming – Storm In-Memory - Spark AmbariMahoutOoziePhoenixSqoopTezZookeeper Azure Storage Blob Azure Storage Blob (WASB)

12. © SQLintersection. All rights reserved. http://www.SQLintersection.com MapReduce Explained Map Take input and “map” this into an output. Imagine multiple parallel “map” tasks doing this work. Reduce Take the output of the map tasks and combine or process them into the final output Your schema is in your code on “read” ERRORLOG.1ERRORLOG.100ERRORLOG.1000 “Show me the count of ERRORLOG entries for spid3s” HIVE provides a SQL “like” experience Spid3s:5 Spid3s:10 Spid3s:2 17 Think “batch jobs”

13. Demo Using HDInsight HIVE to analyze SQL ERRORLOG files Azim Uddin Big Data Support Blog

14. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure Data Lake Preview Coming A fully managed store and analytics service Based on YARN and HDFS Rich Visual Studio Development Environment U-SQLU-SQL makes it easy for the SQL Professional @t = EXTRACT date string, time string, author string, tweet string FROM "/input/MyTwitterHistory.csv" USING Extractors.Csv(); @res = SELECT author, COUNT(*) AS tweetcount FROM @t GROUP BY author; OUTPUT @res TO "/output/MyTwitterAnalysis.csv" ORDER BY tweetcount DESC USING Outputters.Csv(); Jobs instead of clusters Used by many services Built on Azure Blob Storage

15. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure Data Factory  Think SSIS and SQLAgent capabilities as a managed cloud service  Linked Services – Data stores or compute service  Datasets – Input Data Source or Output dest  Pipelines – Orchestrated activities using datasets and linked services  Develop in Azure Portal, PowerShell, or Visual Studio  Schedule and monitor your pipelines Convert ERRORLOG files to UTF-8 Copy to Azure Storage Execute HIVE DDL Execute HIVE Queries Copy results into Azure Database Defined with JSON Copy Activity HIVE Query Activity HIVE Query Activity Linked Service: onPremisesFileServer Linked Service: onPremisesFileServer Linked Service: AzureStorage Linked Service: AzureStorage Linked Service: HDInsightOnDemand Linked Service: HDInsightOnDemand Linked Service: AzureSQLDatabase Linked Service: AzureSQLDatabase DataSet: FileShare DataSet: AzureBlob Copy Activity DataSet: AzureBlob DataSet: AzureSQLTable Power BI Dashboard Direct Connect Power BI Dashboard Direct Connect OnDemand – Data Factory creates and deletes HDInsight clusters as needed OnDemand – Data Factory creates and deletes HDInsight clusters as needed

16. © SQLintersection. All rights reserved. http://www.SQLintersection.com Azure Machine Learning  Azure Machine Learning = cloud computing platform to build, test, deploy, and publish predictive models  A predictive model contains…  Define your problem (“I want to predict the winner of Super Bowl 50”)  Ingest, clean, and aggregate data from past experiences and current data  Build a model using “learning tasks”, algorithms, modules, and a “flow”  Deploy and run your model  Machine Learning Studio allows you to build and deploy prediction models Machine Learning Studio  Reuse libraries of algorithms and modules  Enhance with custom R and Python scripts  Publish your work as a web service to be consumed in the MarketplaceMarketplace  Use Cortana Analytics Gallery to get jump startedCortana Analytics Gallery  T-SQL R language integration coming to SQL Server 2016 “Learning is any process by which a system improves performance from experience” – Herbert Simon “Learning is any process by which a system improves performance from experience” – Herbert Simon Azure ML Cheat Sheet Try it freefree Try it freefree experiments

17. © SQLintersection. All rights reserved. http://www.SQLintersection.com Power BI Based on our own data and analysis technologies running in Azure Focuses on dashboards, self-service reports, and Q&A A true self-service BI and reporting solution Gateways to schedule refresh for on-premise data sources Azure stores and services directly connected. Use Stream Analytics for a “live” feed Get Data from on-premise, file, Azure, or SAAS stores and services Power BI Desktop app allows you to develop offline and publish Content Packs make it easy to distribute, share, and consume Want more visuals. Check out the new community gallery.gallery A rich development and publishing system IPhone and Android Windows Application Power BI Mobile Free or Power BI Pro

18. Demo Using PowerBI and SQL Server

19. © SQLintersection. All rights reserved. http://www.SQLintersection.com Does this stuff work together?

20. © SQLintersection. All rights reserved. http://www.SQLintersection.com References  Understanding Azure SQL Database and SQL Server in Azure VMs Understanding Azure SQL Database and SQL Server in Azure VMs  Spotlight on SQL Database Active Geo-Replication Spotlight on SQL Database Active Geo-Replication  Azure SQL Database Benchmark Overview Azure SQL Database Benchmark Overview  Data Analytic Scenarios Data Analytic Scenarios  Getting Started with Azure SQL Data Warehouse Getting Started with Azure SQL Data Warehouse  Working with NoSQL Data in DocumentDB Working with NoSQL Data in DocumentDB  Hadoop tutorial: Get started using Hadoop with Hive in HDInsight on Linux Hadoop tutorial: Get started using Hadoop with Hive in HDInsight on Linux  Building Big Data Applications Using Azure HDInsight Service Building Big Data Applications Using Azure HDInsight Service  Build your first pipeline using Azure Data Factory Build your first pipeline using Azure Data Factory  Build your first Machine Learning Experiment Build your first Machine Learning Experiment  Azure Friday Videos Azure Friday Videos

21. © SQLintersection. All rights reserved. http://www.SQLintersection.com Review  Azure has the security, privacy, scale, flexibility, and speed to meet the needs of production applications  Azure SQL Database is a fully managed database service using the features of SQL Server  Azure SQL Data Warehouse combines the power of APS and SQL Server  Azure HDInsight provides a semi-managed Hadoop suite of services  Azure Data Lake is a fully-managed store and analytics service for big data at scale  Azure Data Factory provides data pipeline and orchestration services  PowerBI brings a true self-service analytics and reporting solution  Azure Machine Learning is a fully managed predictive model cloud service Sign-up free today and get $200 credit on Azure Servicesfree MSDN subscribers get free Azure benefitsAzure benefits Sign-up free today and get $200 credit on Azure Servicesfree MSDN subscribers get free Azure benefitsAzure benefits

22. Don’t forget to complete an online evaluation on EventBoard! Your evaluation helps organizers build better conferences and helps speakers improve their sessions. Questions? Thank you! Do you know Data in the Microsoft Cloud?