Azure SQL DB Edge - tiny but mighty - Julie Koesmarno (jukoesma@microsoft.com) Principal Program Manager Business Analytics for Azure Data at Microsoft Sydney #875, 3 Aug 2019

Edge

The Basics of Edge … Other useful things to know - … Fog Computing decentralizes computing infra in which data, compute, storage, and applications are distributed in the most logical, efficient place between the data source and the cloud. IoT = uniquely identifiable, trackable and internet-connected objects … … … … Edge Computing pushes intelligence and computing power closer to the source of the data, i.e. machines such as pumps, turbines, sensors, and more. Edge Device also referred as Edge Computing Device, such as sensors or other connected devices (gateways) that processes or analyzes data, instead of sending it back to the cloud or remote data center. It has also been referred to as an apparatus that acts as an entry point to the core networks of an enterprise or a service provider Connected Semi-connected Offline / Disconnected

Data Landscape is Changing Top trends driving the change Latency, security, bandwidth/connectivity continue to gravitate data compute needs to edge New sensors and chip innovation with analytics and lower cost enable more Edge compute   5G and IoT Mesh architecture accelerate distributed computing Cost Inefficiency to store everything in Cloud By 2025, 75% of enterprise-generated data will be created and processed outside the datacenter or cloud, up from less than 20% today Source: Gartner” “Edge Computing Solutions for Industrial IoT, July 2018, Gartner” “Top Strategic IoT Trends and Technologies Through 2023. September 2018, Gartner”

Volume of Captured IoT Data (in Exabytes, 1018 bytes) Edge data grows significantly w/ potential for analytics Volume of Captured IoT Data (in Exabytes, 1018 bytes) Less than 50% of structured data is actively used in decision making Less than 1% of unstructured data is analyzed or used at all Challenges for data engines running on Edge Limitations on size and processing power Limited database choices Security and compliance concern Complexity of management Data siloed on the edge Learning curve for developers and operators Cost of licensing CAGR 35.7% 21.9% Source: ABI Research, Harvard Business Review, Cisco, Constat Edge Data Research

Roadmap

Microsoft - Intelligent cloud and intelligent edge 9/3/2019 12:50 PM Microsoft - Intelligent cloud and intelligent edge Consistent Microsoft Azure App Platform Azure Services Security Intelligent cloud Identity Azure Security Center Intelligent edge Management a b c d e The intelligent cloud is ubiquitous computing enabled by the public cloud and powered with AI. In this way, we refer to Azure as the world’s computer, powering every type of intelligent application and system our customers envision. The intelligent edge is the continually expanding set of connected systems and devices that gather and analyze information close to the physical world where data resides, to deliver real-time insights and immersive experiences that are highly responsive and contextually aware. Enabling intelligent cloud and intelligent edge solutions requires a new class of distributed, connected applications and will ultimately deliver break-through business outcomes. These cloud/edge applications are built as a single solution yet run in a distributed fashion – optimized to take advantage of both robust cloud capabilities and edge locality. At the edge, the application is contextually aware and can run in both connected and disconnected states. Azure Active Directory Azure Management Artificial Intelligence Edge MCU Interactive devices Edge Gateways/Servers e.g. Azure Sphere e.g. Drones, industrial assets e.g. Azure Data Box Edge Azure Stack Azure IoT Edge Comprehensive Source: Julia White Blogpost outlining Microsoft’s Intelligent Cloud, Intelligent Edge POV © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Consistent Data & Analytics Engine running natively across Edge, Data Center and Cloud Edge MCU Interactive Devices SQL Server Azure SQL Database Data Center Cloud Industry-leading data engine with full AI/ML capability for mission-critical workloads Edge Gateways Edge Servers Azure SQL Database Edge A small-footprint, turn-key, ML-capable data engine running connected or offline for Edge Introduce new Edge analytics, data movement and management capabilities Complement existing Azure IoT Edge services by providing local data storage across device tiers

Azure SQL Database Edge Microsoft Data Amp 9/3/2019 12:50 PM Azure SQL Database Edge Small footprint, AI built-in, optimized for Edge Choice of platform Time-series built-in AI built-in Turn-key management Unparalleled performance and security SQL Time-Series Arm64 x64 Light-weight SQL Engine <500 MB tuned to run on ARM and Intel devices Built-In Time-Series Streaming and Analytics Machine Learning over all data with SQL In-Database ML and Spark Single management plane for deployment & updates Most secure with industry leading performance benchmarks Local streaming, storage, analytics all-in-one Develop once, deploy anywhere for your apps Competitive TCO Edge Data Center/Cloud © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Key Capabilities Native data movement Complete analytics platform Ease of management Management Plane (e.g. Azure IoT) Data Center/Cloud Storage Data Center/Cloud ML Services Create, deploy, update Runtime, Messages Device Sync Data Sync Trained Models Predictions Results Gateway Gateway Interactive Dev. Gateway Interactive Dev. Native-scoring / Predictions from ML models Data Streaming Edge Devices Optimize network bandwidth for native data sync with cloud/enterprise portal Stream IoT data from Edge with a high performant engine Process time-series data directly in Azure SQL Database Edge Use SQL in-database machine learning and Time- Series analytics for predictions and anomaly detection etc. in low-latency on Edge Single management plane for deployment, remote updates/re-initialize for feature and security Consistent security; flexible HA/DR built-in Device and data relationship mapping

Azure SQL Database Edge architecture Edge compute autonomy with bi-directional to cloud Data streaming for aggregation and association; anomaly detection, classification while streaming Data compute + storage In-DB parallel ML processing using R/Python/ Spark/Java Built-in connectors to storage targets on data sync Reference data device sync Feed local ML results back to central ML model Push down ML model to SQL DB Edge container Create, deploy, update SQL DB Edge container from single management plane BI at both edge and cloud Azure SQL DB SQL DW Cosmos DB SQL Server Azure ML Azure IoT Hub Custom Apps Analytics BI 9 4 6 7 8 5 Cloud / On-premises Edge Small-footprint container on Arm, x64 devices w/ Linux, Windows OS Sensor 1 Sensor 2 … Relational data Time-series data 2 3 1

Demo: Robot Telemetry Analysis with SQL DB Edge Data streaming for aggregation and association; anomaly detection, classification while streaming Data compute + storage In-DB parallel ML processing using R/Python/ Spark/Java Built-in connectors to storage targets on data sync Reference data device sync Feed local ML results back to central ML model Push down ML model to SQL DB Edge container Create, deploy, update SQL DB Edge container from single management plane BI at both edge and cloud Azure IoT Hub Azure SQL DB BI at (local) Edge 9 BI on Cloud 8 4 Cloud / On-premises Edge Small-footprint container on Arm device w/ Linux Sensor 1 Sensor 2 … Relational data Time-series data 2 3 1 Simulated Robot sensors container on Arm

sudo docker logs RobotTelemetry -f --tail 25 A bunch of telemetry recorded, there is no smartness from the Robot sensor itself. No anomaly detection.

sudo docker logs AzureSQLDatabaseEdge -f --tail 25 A bunch of telemetry recorded, there is no smartness from the Robot sensor itself. No anomaly detection.

Azure SQL Database Edge

Azure SQL Database

Use cases

Use case: data processing + movement in Agriculture Microsoft Data Amp 9/3/2019 12:50 PM Use case: data processing + movement in Agriculture Business Challenges: Able to monitor soil, humidity data detect anomaly for real-time actions Optimize for network bandwidth by pre-process massive raw data locally before sync (aggregates) to cloud A credible vendor that can address multiple edge scenarios Azure SQL Database CosmosDB Azure SQL Data Warehouse Azure Machine Learning Azure Analytics Predictions/ results Aggregated Data Trained ML Gateway in farms Data Streaming Data Processing Local BI Aggregates Filtering Anomaly detection on raw data … Azure SQL Database Edge Azure SQL Database Edge Streaming, storage, ML all-in-one Native Edge/Cloud movement for data and analytics Credibility of Microsoft and SQL brand and franchise it brings 3 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Use case: low-latency analytics in utilities Microsoft Data Amp 9/3/2019 12:50 PM Use case: low-latency analytics in utilities Business Challenges: Able to monitor pump pressure, detect anomaly for real-time actions A solution that could be bi-directional b/t Edge & Cloud A credible vendor that can address multiple edge scenarios Operational data in SQL Server Azure ML Azure SQL Database / CosmosDB Predictions/ Other data Sensitive Data Trained ML Machine in field Data Streaming Anomaly Detection Real-time actions Maintenance Shut-off Azure SQL Database Edge Azure SQL Database Edge Streaming, storage, ML all-in-one Native Edge/Cloud movement for data and analytics Credibility of Microsoft and SQL brand and franchise it brings 2 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Take-Aways Data landscape gravitating towards Edge Intelligent Cloud and Intelligent Edge Solutions bring robustness of cloud and edge locality, expanding real time insights where the data physically lives, built as a single solution and run in a distributed fashion Azure SQL DB Edge brings a small-footprint, turn-key, ML-capable data engine running connected or offline for Edge Consistent data and analytics engine running natively across Edge, Data Center and Cloud Sign up for Early Adoption Program & learn more! Take-Aways

Microsoft Data Amp 9/3/2019 12:50 PM Getting started today Learn more on Azure.com for Azure SQL Database Edge Download Azure SQL Database Edge Infographic – http://aka.ms/asde-infographic Check the demo on Arm container and streaming - http://aka.ms/asde-demo-video Read and follow our blog for more updates – http://aka.ms/asde-blog Join Azure SQL Database Edge Early Adopter Program http://aka.ms/sqldbedge-eap Gain access to the product bits Get engineering support on your edge project Be a champion for edge solutions and share your success 2 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

GOLD Silver Bronze