Presentation is loading. Please wait.

Presentation is loading. Please wait.

Windows Azure Tables: Programming Cloud Table Storage

Similar presentations


Presentation on theme: "Windows Azure Tables: Programming Cloud Table Storage"— Presentation transcript:

1 Windows Azure Tables: Programming Cloud Table Storage
4/1/2017 2:37 PM ES07 Windows Azure Tables: Programming Cloud Table Storage  Niranjan Nilakantan Development Lead Microsoft Corporation  Pablo Castro Software Architect Microsoft Corporation © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2 Windows Azure Windows Azure is the foundation of Microsoft’s Cloud Platform It is an “Operating System in the Cloud” and provides Essential Services for the cloud Virtualized Computation Scalable Storage Automatic Management Developer SDK

3 Fundamental Data Abstractions
Blobs – Provide a simple interface for storing named files along with metadata for the file Tables – Provide structured storage; A Table is a set of entities, which contain a set of properties Queues – Provide reliable storage and delivery of messages for an application

4 Windows Azure Tables Provides Structured Storage
Massively Scalable Tables Billions of entities (rows) and TBs of data Can use thousands of servers as traffic grows Highly Available Can always access your data Durable Data is replicated several times Familiar and Easy to use API ADO.NET Data Services – .NET 3.5 SP1 .NET classes and LINQ REST – with any platform or language

5 Agenda Data model Demo – creating tables
Basic CRUD using .NET and REST Demo – a sample application Advanced features

6 Data Model Table Data is stored in Tables Entity
A Storage Account can create many tables Table name is scoped by Account Data is stored in Tables A Table is a set of Entities (rows) An Entity is a set of Properties (columns) Entity Two “key” properties that together are the unique ID of the entity PartitionKey – enables scalability RowKey – uniquely identifies the entity within the partition

7 Partitioning Partition Key Document Row Key Version Property 3 ChangedOn Property 4 Description Examples Doc V1.0 8/2/2007 Committed version V2.0.1 9/28/2007 Alice’s working version FAQ Doc 5/2/2007 V1.0.1 7/6/2007 V1.0.2 8/1/2007 Sally’s working version Partition 1 Partition 2 Table Partition – all entities in table with same partition key value Application controls granularity of partition

8 Partitioning Guidelines
Performance Use a PartitionKey that is common in your queries Entities with same partition key value are clustered Scalability We monitor partition traffic Automatically load balance partitions Each partition can potentially be served by a different storage node Scale to meet the traffic needs of your application More partitions – makes it easier to balance load

9 Entity Locality Getting all versions of FAQ is fast
Partition Key Document Row Key Version Property 3 ChangedOn Property 4 Description Examples Doc V1.0 8/2/2007 Committed version V2.0.1 9/28/2007 Alice’s working version FAQ Doc 5/2/2007 V1.0.1 7/6/2007 V1.0.2 8/1/2007 Sally’s working version Partition 1 Partition 2 Getting all versions of FAQ is fast Access single partition Get all documents before 5/30/2008 takes longer Have to traverse all partitions

10 Entities and Properties
Each Entity can have up to 255 properties Every Entity has fixed key properties Partition key Row key No fixed schema for rest of properties 2 entities in the same table can have different properties Properties stored as <Name, TypedValue> pairs Each entity has a system maintained version

11 Property Types Partition key and Row key Other properties
String (up to 64KB) Other properties Binary (up to 64KB) Bool DateTime GUID Int Int64 Double

12 Sample Scenario Micro-blogging application Scalability challenges
Post messages to channels Subscribe to channels Retrieve channel messages Scalability challenges Large volume of new messages daily Need to retrieve recent messages quickly Messages permanently addressable Name – PK Author CreatedDate Channels Channel – PK PostedDate – RK Text Rating Messages Name – PK Updates Channels Users

13 MicroBlogging Application
4/1/2017 2:37 PM demo MicroBlogging Application © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14 Table Schema Define the schema as a .NET class
[DataServiceKey("PartitionKey", "RowKey")] public class Message { // ChannelName public string PartitionKey { get; set; } // PostedDate public string RowKey { get; set; } // User defined properties public string Text { get; set; } public int Rating { get; set; } }

15 Insert Entity using .NET
Create a new Message and Insert into Table Message message = new Message { PartitionKey = "Channel9", // ChannelName RowKey = DateTime.UtcNow.ToString(), // PostedDate Text = "Hello PDC", Rating = 3 }; serviceUri = new Uri("http://<account>.table.core.windows.net"); var context = new DataServiceContext(serviceUri); context.AddObject("Messages", message); DataServiceContext response = context.SaveChanges();

16 Insert Entity using REST
Create an Atom XML payload and POST it POST ... <!– Atom envelope --> <m:properties> <d:PartitionKey>Channel9</d:PartitionKey> <!-- ChannelName --> <d:RowKey>Oct-29</d:RowKey> <!-- PostedDate --> <d:Text>Hello PDC</d:Text> <d:Rating>3</d:Rating> </m:properties>

17 Create Table Every Account has a table called “Tables”
To use a table it has to be inserted into “Tables” [DataServiceKey("TableName")] public class TableStorageTable { public string TableName { get; set; } } // Service Uri is “http://<Account>.table.core.windows.net/” DataServiceContext context = new DataServiceContext(serviceUri); TableStorageTable table = new TableStorageTable("Messages"); context.AddObject("Tables", table); DataServiceResponse response = context.SaveChanges();

18 Query Entities .NET – LINQ REST
serviceUri = new Uri("http://<account>.table.core.windows.net"); DataServiceContext context = new DataServiceContext(serviceUri); var messages = from message in context.CreateQuery<Message>("Messages") where message.Rating == 3 select message; foreach (Message message in messages) { } REST GET Rating eq 3

19 Update a property in an entity
Using the .NET API Message message = (from message in context.CreateQuery<Message>("Messages") where message.PartitionKey == "Channel9"   && message.RowKey == "Oct-29" select message).FirstOrDefault(); message.Text = "Hi there"; context.UpdateObject(message); DataServiceResponse response = context.SaveChanges(); Using the REST API PUT ‘Oct-29’) <m:properties> <d:Text>Hi there</d:Text> <!-- Other properties are the same --> </m:properties>

20 Delete an Entity Using the .NET API Using the REST API
// Get the Message object for ("Channel9", "Oct-29") context.DeleteObject(message); DataServiceResponse response = context.SaveChanges(); Using the REST API DELETE ‘Oct-29’)

21 .NET vs. REST ADO.NET Data Services REST Interface
4/1/2017 2:37 PM .NET vs. REST ADO.NET Data Services REST Interface Client included in .NET Framework 3.5 SP1 Data represented as .NET objects DataServiceContext methods for updates Use LINQ to define queries Use any HTTP stack Data represented in Atom (XML) Standard HTTP verbs for updates Use URLs to define queries © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22 MicroBlogging Application
4/1/2017 2:37 PM demo MicroBlogging Application © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23 Windows Platform Integration
Reuse .NET skills Fully compatible with ADO.NET data services .NET client included in .NET 3.5 SP1 LINQ support ASP.NET integration for website authoring Sample data source control for data binding ASP.NET dynamic data for instant front-ends ASP.NET providers for membership, roles, etc.

24 Advanced Topics Concurrent updates Pagination Table consistency

25 Concurrent Updates ClientB Client A
1: Ch9, Jan-2, 5 2: Ch9, Jan-2, 5 Version Rating 1: Ch9, Jan-2, 4 Error: 412 5 : Ch9, Jan-1, 3 If-Match: 1 Ch9, Jan-2, 4 If-Match: 1 Ch9, Jan-2, 5 1 : Ch9, Jan-2, 2 1 : Ch9, Jan-2, 2 2: Ch9, Jan-2, 5 1 : Ch9, Jan-2, 2 9 : Ch9, Jan-3, 6 Use standard HTTP mechanisms – Etag and If-Match Get entity – get system maintained version as ETag Update Entities Locally – change rating Send Update with version check - IF-Match with Etag Success if version matches, and update version on Client-A Precondition failed (412) if version does not match

26 Getting the Top N entities
.NET: LINQ Take(N) function serviceUri = new Uri("http://<account>.table.core.windows.net"); DataServiceContext context = new DataServiceContext(serviceUri); var allMessages = context.CreateQuery<Message>("Messages"); foreach (Message message in allMessages.Take(100)) { Console.WriteLine(message.Name); } REST: $top=N query string option GET

27 Pagination – Continuation Tokens
Send a request GET Get continuation token in response header x-ms-continuation-NextPartitionKey: Channel9 x-ms-continuation-NextRowKey: Date101 Messages 100 Ch9, Date1, Ch9, Date2, Ch9, … Set HTTP query parameters Ch9,Date100, GET &NextPartitionKey=Channel9 &NextRowKey=Date101 Ch9,Date101, Ch9, …

28 Single Table Consistency
ACID transactions for single entity CUD Insert/update/delete Snapshot isolation for query within a single partition Consistent view from start time of the query No dirty (uncommitted) reads Does not block concurrent updates No snapshot isolation across partitions No snapshot isolation across different continuations of a query

29 Cross Table Consistency
Application is responsible for maintaining consistency Example When a channel is deleted, delete all the messages for that channel Failures can occur in the middle Example - Application fails after deleting some messages Use Windows Azure Queues to help ensure completion of operation

30 Cross Table Consistency
Delete channel Delete messages worker Front end Worker Queue Messages Ch1, Msg1 Del Ch5 Del Ch11 Del Ch1 Del Ch1 2 Ch1, Msg2 Ch1, Msg3 Front End Ch2, Msg1 Ch2, Msg2 Channels Ch1,… Ch3, Msg1 Ch2,… 1. Dequeue DelCh1 3. Delete from Messages 4. Delete queue entry 2. Delete Ch1 from Channels

31 Resuming After Failure
Delete channel Delete messages worker Front end Worker 1 Del Ch1h Queue Messages Ch1, Msg1 Del Ch5 Del Ch11 Del Ch1 Del Ch1 2 Ch1, Msg2 Ch1, Msg3 Front End Ch2, Msg1 Worker2 Ch2, Msg2 Channels Ch1,… Ch3, Msg1 Ch2,… 2. Fails after deleting Ch1 and Msg1 3. DelCh1 is visible again 1. Dequeue DelCh1 and start delete 4. Dequeue DelCh1 again 5. Repeat delete operations

32 Future Windows Azure Table Support
At PDC Single Index Query and retrieve results sorted by PartitionKey and RowKey Future Support for Secondary Indexes Query and retrieve results sorted by other properties Single Entity Transactions Single Entity Insert, Update, or Delete Atomically Entity Groups Atomic transactions across multiple entities within same partition

33 Summary Azure tables are Simple familiar API
Massively Scalable Highly Available Simple familiar API Use .NET –- ADO.NET Data Services and LINQ Or use REST Leverage your .NET expertise ASP .NET integration for instant front ends

34 Resources Links Windows Azure ADO.NET Data Services
ADO.NET Data Services

35 Evals & Recordings www.microsoftpdc.com
Please fill out your evaluation for this session at: This session will be available as a recording at:

36 Please use the microphones provided
4/1/2017 2:37 PM Q&A Please use the microphones provided © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

37 4/1/2017 2:37 PM © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

38 4/1/2017 2:37 PM © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.


Download ppt "Windows Azure Tables: Programming Cloud Table Storage"

Similar presentations


Ads by Google