Getting to know U-SQL Azhagappan Arunachalam

 Sr Applications Database Architect 

Agenda  What is U-SQL?  Where does it fit in?  A brief history of U-SQL  Getting started with U-SQL  Demo  … and more demo

What is U-SQL?  It’s a new query processing language that leverages T-SQL with.net (C#)  Aligns U-SQL metadata system, syntax and language semantics with T-SQL/ANSI SQL  Uses C# data types and expression language  Integrates features available in Hive and other big data languages to identify patterns and data processing requirements

Brief History of U-SQL  Came out of Microsoft internal development – SCOPE (Structured Computations Optimized for Parallel Execution)  Currently being used for Big Data analytics at Microsoft, for their Bing, XBox, and Office 365 product lines  Supposedly 100,000 of batch jobs are being run based on U-SQL  Introduced to the public at Build 2015 alongside Azure Data Lake Services  Still in public preview…

Connected World...  World is moving towards a weightless economy  Purchasing is being replaced by subscription based services  We have lots of data points with IP address (IoT devices)  Collect everything  Analyze anything  & Predict the foreseeable future

Getting Started with ADL/U-SQL  What you need to get started?  An azure data lake analytics account (along with an azure data lake storage account)  Works with Visual Studio 2012, 2013 & 2015 – even Community Edition  Install the Azure SDK (pre-requisite to installing the ADL components/plug-ins)  Install ADL Tools for Visual Studio plug-in ( )

Microsoft Visual Studio Dev Essentials Program  essentials-vs.aspx essentials-vs.aspx  Some of the benefits:  6-months of pluralsight training free ($29 x 6 = $174)  VSTS team services - basic level (small team, upto 5 people)  Visual Studio Community Edition  Visual Studio Code (Mac/Linux anyone?) -  Microsoft R Server – Developer edition  Monthly Azure Credit – ($25 x 12 = $300)  Hack Hands credit - $25  Xamarin University Free training  Microsoft SQL Server Developer Edition – Used to be $60  3-months Wintellect Training ~ $90

Demo  Signing up for Azure Data Lake service accounts  Data Lake Analytics Account  Data Lake Storage Account

Demo  U-SQL scripting

Summary We saw…  What is U-SQL?  Where does it fit in?  A brief history of U-SQL  Getting started with U-SQL  Demo  … and addl demos  And remember to save ~$500 by signing up for the Visual Studio Dev Essentials program

References  Lay of the land - paths/data-lake-analytics-self-guided-training/ paths/data-lake-analytics-self-guided-training/  U-SQL on MSDN (  ADL Public Preview (   - to download the latest azure data lake tools for visual studio

Evaluations…   …. and thank you for filling out your evaluations…