Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc.

Agenda - Overview - Management Portal / Tools - Windows Azure Storage - Tables, Blobs, Queues - Windows Azure SQL Database - Putting them together with Cloud Design Pattern Examples

Windows Azure Storage Overview Management Portal Storage Emulator / Tools

What is Windows Azure Storage? - Storage as a service – Blobs, Tables, and Queues - Lots of it (up to 200 TB) – pay as you go - Highly Available - Redundancy - Highly Scalable - Partitions - Blazing Speed - Performance “Targets”

Getting Started with Storage - Create a storage account in the portal - Connection string Account Name Account Key

Getting Started with Storage - Use storage emulator for development - Connection string UseDevelopmentStorage=true

Windows Azure SDK Windows Azure SDK Visual Studio 2013 Visual Studio 2012 Visual Studio Oct2013Supported Not Supported 2.1 Jul2013Not SupportedSupported 2.0 Apr2013Not SupportedSupported - SDK contains client libraries - Committed to backwards compatibility

Storage Client Library - SCL 2.1 included with SDK SCL 3.0.x upgrade available with NuGet

Tools to interact with storage Visual Studio Server Explorer CloudXplorer/TableXplorer AzureXplorer (VS Extension)

Demo Management Portal Storage Tools

Windows Azure Storage Tables Blobs Queues

Tables Concept Overview AccountTableEntity gwabjax racers results Name=Ricky Status=Active Name=Dale Status=Active Race=Jax 500 Date=3/29/14

- Structured storage in the form of tables -1MB max size -252 properties - Quick queries by clustered index (only index) - 2,000 entities per second per partition - Data-defined partition scheme -Table name + PK Table Details

- Tables store data more like key-value pairs - No schema - De-normalized structure optimized for performance Table Details First NameLast NameDate of BirthLast Win RickyBobby1/1/1971 DaleEarnhardt2/2/1972Daytona 500 CalNaughtonMarch 3, 1973

Table Entities - Property Types byte[] bool DateTime double Guid Int32 Int64 String

Table Entities - Required properties PartitionKey, RowKey, Timestamp - Operations on entities Delete Insert Upsert (Insert + Merge/Replace) Update (Merge or Replace) Retrieve Query

Using Tables 1.Get table reference 2.Execute operations on entities CloudStorageAccount storageAccount = CloudStorageAccount.Parse( CloudConfigurationManager.GetSetting("DefaultConnection")); CloudTableClient tableClient = storageAccount.CreateCloudTableClient(); CloudTable table = tableClient.GetTableReference("racers"); TableOperation upsertOperation = TableOperation.InsertOrReplace(racer); table.Execute(upsertOperation);

Using Tables with DTOs/POCOs 1.Get table reference (like before) 2.Execute operations on entities via EntityAdapter With TableEntity – RacerEntity var operation = TableOperation.InsertOrReplace(racerEntity); table.Execute(operation); With DTO – Racer var adapter = new RacerAdapter(racer); var operation = TableOperation.InsertOrReplace(adapter); table.Execute(operation);

Table Best Practices 1.Avoid querying across partitions 2.Batch inserts (within partition) - Order and details in same table/partition 3.Storing aggregate copies - Ensures 1 query  1 partition - Format data up front 4.Use intelligent PKs

Demo Tables Blobs Queues

Blobs Concept Overview AccountContainerBlob gwabjax photos videos DSC j pg teamlogo.png 1stPlace.avi /

Blobs - Types of Blobs - Page - Block - Operations on blobs Exists List Upload from Stream, ByteArray, Text, File Download to Stream, ByteArray, Text, File Delete

Using Blobs 1.Get blob container reference 2.Perform operations with blobs CloudStorageAccount account = CloudStorageAccount.Parse( CloudConfigurationManager.GetSetting("DefaultConnection")); CloudBlobClient client = account.CreateCloudBlobClient(); CloudBlobContainer container = client.GetContainerReference("photos"); CloudBlockBlob blockBlob = container.GetBlockBlobReference(refId); blockBlob.UploadFromStream(stream); blockBlob.Metadata["FileName"] = fileName; blockBlob.SetMetadata();

Demo Tables Blobs Queues

Queues Concept Overview AccountQueueMessages gwabjax orders s-to- send Bulk Small

Queues - Operations on queues Add Peek Get Update Delete - Queue naming - All lowercase - Alpha-numeric and hyphen “-” characters

ClientWeb RoleStorage Queues in the wild Worker Role Browse r Web App Get Survey Complete Survey Post Results Surveys Queue Proces s Task Thank you Add Get Delete Process*

ClientWeb RoleStorage Queues in the wild Worker Role Blob Store Browse r Web App Get Survey Complete Survey Post Results Small Surveys Large Surveys Small Task Big Task Small Survey Get Delete Process* Add Thank you

ClientWeb RoleStorage Queues in the wild Worker Role Blob/Tabl e Store Browse r Web App Get Survey Complete Survey Post Results Small Surveys Queue Large Surveys Queue Small Task Large Task Large Survey Get Delete Process* Add Thank you Upload Download

Using Queues 1.Get queue reference 2.Perform operations with messages CloudStorageAccount account = CloudStorageAccount.Parse( CloudConfigurationManager.GetSetting("DefaultConnection")); CloudQueueClient client = account.CreateCloudQueueClient(); CloudQueue queue = client.GetQueueReference("orders"); CloudQueueMessage message = new CloudQueueMessage("ticket-order-17"); queue.AddMessage(message);

Demo Tables Blobs Queues

Windows Azure SQL Database (formerly SQL Azure) Management Portal Data Access

Windows Azure SQL Database Tools - SSMS - Visual Studio/SSDT - Modeling tools Code -ADO.NET -Enterprise Library -ORMs (e.g. Entity Framework, nHibernate)

Key Features -High Availability/Durability -Three Replicas (1 Primary – 2 Secondary) -99.9% SLA -Manageability -Portal, PowerShell, SSMS, REST API -Predictable Performance -Scale-out*

What’s Different – Maybe Bad? Missing -Extended stored procedures -SQL-CLR -Service Broker -Table partitioning Important: All tables must have a clustered index

Data Types? Data Type CategoryWindows Azure SQL Database Support Exact numericsbigint, bit, decimal, int, money, numeric, smallint, smallmoney, tinyint Approximate numericsfloat, real Date and timedate, datetime2, datetime, datetimeoffset, smalldatetime, time Character stringschar, varchar, text Unicode character strings nchar, nvarchar, ntext Binary stringsbinary, varbinary, image Spatial data typesgeography, geometry Other data typescursor, hierarchyid, sql_variant, table, timestamp, uniqueidentifier, xml

Development Story -Local SQL 2012/Express database -SSDT Database projects -Deploy directly to Azure from Visual Studio -Create deployment package -Use Release Management -Azure Connection String <add name="DefaultConnection" connectionString="Server=tcp:[server].database.windows.net,1433; Database=[database]; User Password=[password]; Encrypt=true;" providerName="System.Data.SqlClient"/>

Demo Management Portal Create Server Create Database Firewall Rules

Patterns at Work Cache-aside Pattern Materialized View Pattern Competing Consumers Pattern Compensating Transaction Pattern

Cache-Aside Pattern -Build cache on-demand -Read cache first -If not there -Get from data store -Store in cache

Materialized View Pattern -Improve performance in systems with difficult queries -Pre-populated views -Not updated by app, can be entirely rebuilt -Inherent delays -Doesn’t query well

Competing Consumers Pattern -Protect against a large influx (burst) of requests -Balanced workload -Scalable

Compensating Transaction Pattern -Eventual consistency -Series of autonomous steps -Intermediate steps appear inconsistent -Compensating Transaction -Intelligent process to undo succeeded steps

Compensating Transaction Pattern -Example: Itinerary Creation -Book 3 flights and 2 hotel rooms -If one or more are unavailable, we start over

Compensating Transaction Pattern -Example: Itinerary Creation

What did we learn? -Azure data storage options/uses -Each has pros/cons/best fit -Management Portal and Tools -Design patterns

Questions? Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc.

Thank you! Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc.

References Cloud Design Patterns (P&P) Data Services – Storage Storage Differences – Emulator vs Cloud Data Services – SQL Database