1. Windows Azure Tables: Programming Cloud Table Storage
4/1/2017 2:37 PM
ES07
Windows Azure Tables: Programming Cloud Table Storage
 Niranjan Nilakantan
Development Lead
Microsoft Corporation
 Pablo Castro
Software Architect
Microsoft Corporation
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Windows Azure
Windows Azure is the foundation of Microsoft’s Cloud Platform
It is an “Operating System in the Cloud” and provides Essential Services for the cloud
Virtualized Computation
Scalable Storage
Automatic Management
Developer SDK

3. Fundamental Data Abstractions
Blobs – Provide a simple interface for storing named files along with metadata for the file
Tables – Provide structured storage; A Table is a set of entities, which contain a set of properties
Queues – Provide reliable storage and delivery of messages for an application

4. Windows Azure Tables Provides Structured Storage
Massively Scalable Tables
Billions of entities (rows) and TBs of data
Can use thousands of servers as traffic grows
Highly Available
Can always access your data
Durable
Data is replicated several times
Familiar and Easy to use API
ADO.NET Data Services – .NET 3.5 SP1
.NET classes and LINQ
REST – with any platform or language

5. Agenda Data model Demo – creating tables
Basic CRUD using .NET and REST
Demo – a sample application
Advanced features

6. Data Model Table Data is stored in Tables Entity
A Storage Account can create many tables
Table name is scoped by Account
Data is stored in Tables
A Table is a set of Entities (rows)
An Entity is a set of Properties (columns)
Entity
Two “key” properties that together are the unique ID of the entity
PartitionKey – enables scalability
RowKey – uniquely identifies the entity within the partition

7. Partitioning
Partition Key
Document
Row Key
Version
Property 3
ChangedOn
Property 4
Description
Examples Doc
V1.0
8/2/2007
Committed version
V2.0.1
9/28/2007
Alice’s working version
FAQ Doc
5/2/2007
V1.0.1
7/6/2007
V1.0.2
8/1/2007
Sally’s working version
Partition 1
Partition 2
Table Partition – all entities in table with same partition key value
Application controls granularity of partition

8. Partitioning Guidelines
Performance
Use a PartitionKey that is common in your queries
Entities with same partition key value are clustered
Scalability
We monitor partition traffic
Automatically load balance partitions
Each partition can potentially be served by a different storage node
Scale to meet the traffic needs of your application
More partitions – makes it easier to balance load

9. Entity Locality Getting all versions of FAQ is fast
Partition Key
Document
Row Key
Version
Property 3
ChangedOn
Property 4
Description
Examples Doc
V1.0
8/2/2007
Committed version
V2.0.1
9/28/2007
Alice’s working version
FAQ Doc
5/2/2007
V1.0.1
7/6/2007
V1.0.2
8/1/2007
Sally’s working version
Partition 1
Partition 2
Getting all versions of FAQ is fast
Access single partition
Get all documents before 5/30/2008 takes longer
Have to traverse all partitions

10. Entities and Properties
Each Entity can have up to 255 properties
Every Entity has fixed key properties
Partition key
Row key
No fixed schema for rest of properties
2 entities in the same table can have different properties
Properties stored as <Name, TypedValue> pairs
Each entity has a system maintained version

11. Property Types Partition key and Row key Other properties
String (up to 64KB)
Other properties
Binary (up to 64KB)
Bool
DateTime
GUID
Int
Int64
Double

12. Sample Scenario Micro-blogging application Scalability challenges
Post messages to channels
Subscribe to channels
Retrieve channel messages
Scalability challenges
Large volume of new messages daily
Need to retrieve recent messages quickly
Messages permanently addressable
Name – PK
Author
CreatedDate
Channels
Channel – PK
PostedDate – RK
Text
Rating
Messages
Name – PK
Updates
Channels
Users

13. MicroBlogging Application
4/1/2017 2:37 PM
demo
MicroBlogging Application
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. Table Schema Define the schema as a .NET class
[DataServiceKey("PartitionKey", "RowKey")]
public class Message {
// ChannelName
public string PartitionKey { get; set; }
// PostedDate
public string RowKey { get; set; }
// User defined properties
public string Text { get; set; }
public int Rating { get; set; } }

15. Insert Entity using .NET
Create a new Message and Insert into Table
Message message = new Message {
PartitionKey = "Channel9", // ChannelName
RowKey = DateTime.UtcNow.ToString(), // PostedDate
Text = "Hello PDC",
Rating = 3 };
serviceUri = new Uri("
var context = new DataServiceContext(serviceUri);
context.AddObject("Messages", message);
DataServiceContext response = context.SaveChanges();

16. Insert Entity using REST
Create an Atom XML payload and POST it
POST
... <!– Atom envelope -->
<m:properties>
<d:PartitionKey>Channel9</d:PartitionKey> <!-- ChannelName -->
<d:RowKey>Oct-29</d:RowKey> <!-- PostedDate -->
<d:Text>Hello PDC</d:Text>
<d:Rating>3</d:Rating>
</m:properties>

17. Create Table Every Account has a table called “Tables”
To use a table it has to be inserted into “Tables”
[DataServiceKey("TableName")]
public class TableStorageTable {
public string TableName { get; set; } }
// Service Uri is “
DataServiceContext context = new DataServiceContext(serviceUri);
TableStorageTable table = new TableStorageTable("Messages");
context.AddObject("Tables", table);
DataServiceResponse response = context.SaveChanges();

18. Query Entities .NET – LINQ REST
serviceUri = new Uri("
DataServiceContext context = new DataServiceContext(serviceUri);
var messages =
from message in context.CreateQuery<Message>("Messages")
where message.Rating == 3
select message;
foreach (Message message in messages) { }
REST
GET Rating eq 3

19. Update a property in an entity
Using the .NET API
Message message =
(from message in context.CreateQuery<Message>("Messages")
where message.PartitionKey == "Channel9"  
&& message.RowKey == "Oct-29"
select message).FirstOrDefault();
message.Text = "Hi there";
context.UpdateObject(message);
DataServiceResponse response = context.SaveChanges();
Using the REST API
PUT ‘Oct-29’)
<m:properties>
<d:Text>Hi there</d:Text>
<!-- Other properties are the same -->
</m:properties>

20. Delete an Entity Using the .NET API Using the REST API
// Get the Message object for ("Channel9", "Oct-29")
context.DeleteObject(message);
DataServiceResponse response = context.SaveChanges();
Using the REST API
DELETE ‘Oct-29’)

21. .NET vs. REST ADO.NET Data Services REST Interface
4/1/2017 2:37 PM
.NET vs. REST
ADO.NET Data Services
REST Interface
Client included in .NET Framework 3.5 SP1
Data represented as .NET objects
DataServiceContext methods for updates
Use LINQ to define queries
Use any HTTP stack
Data represented in Atom (XML)
Standard HTTP verbs for updates
Use URLs to define queries
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. MicroBlogging Application
4/1/2017 2:37 PM
demo
MicroBlogging Application
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. Windows Platform Integration
Reuse .NET skills
Fully compatible with ADO.NET data services
.NET client included in .NET 3.5 SP1
LINQ support
ASP.NET integration for website authoring
Sample data source control for data binding
ASP.NET dynamic data for instant front-ends
ASP.NET providers for membership, roles, etc.

24. Advanced Topics
Concurrent updates
Pagination
Table consistency

25. Concurrent Updates ClientB Client A
1: Ch9, Jan-2, 5
2: Ch9, Jan-2, 5
Version
Rating
1: Ch9, Jan-2, 4
Error: 412
5 : Ch9, Jan-1, 3
If-Match: 1
Ch9, Jan-2, 4
If-Match: 1
Ch9, Jan-2, 5
1 : Ch9, Jan-2, 2
1 : Ch9, Jan-2, 2
2: Ch9, Jan-2, 5
1 : Ch9, Jan-2, 2
9 : Ch9, Jan-3, 6
Use standard HTTP mechanisms – Etag and If-Match
Get entity – get system maintained version as ETag
Update Entities Locally – change rating
Send Update with version check - IF-Match with Etag
Success if version matches, and update version on Client-A
Precondition failed (412) if version does not match

26. Getting the Top N entities
.NET: LINQ Take(N) function
serviceUri = new Uri("
DataServiceContext context = new DataServiceContext(serviceUri);
var allMessages = context.CreateQuery<Message>("Messages");
foreach (Message message in allMessages.Take(100)) {
Console.WriteLine(message.Name); }
REST: $top=N query string option
GET

27. Pagination – Continuation Tokens
Send a request
GET
Get continuation token in response header
x-ms-continuation-NextPartitionKey: Channel9
x-ms-continuation-NextRowKey: Date101
Messages
100
Ch9, Date1,
Ch9, Date2,
Ch9, …
Set HTTP query parameters
Ch9,Date100,
GET
&NextPartitionKey=Channel9
&NextRowKey=Date101
Ch9,Date101,
Ch9, …

28. Single Table Consistency
ACID transactions for single entity CUD
Insert/update/delete
Snapshot isolation for query within a single partition
Consistent view from start time of the query
No dirty (uncommitted) reads
Does not block concurrent updates
No snapshot isolation across partitions
No snapshot isolation across different continuations of a query

29. Cross Table Consistency
Application is responsible for maintaining consistency
Example
When a channel is deleted, delete all the messages for that channel
Failures can occur in the middle
Example - Application fails after deleting some messages
Use Windows Azure Queues to help ensure completion of operation

30. Cross Table Consistency
Delete channel
Delete messages worker
Front end
Worker
Queue
Messages
Ch1, Msg1
Del Ch5
Del Ch11
Del Ch1
Del Ch1 2
Ch1, Msg2
Ch1, Msg3
Front End
Ch2, Msg1
Ch2, Msg2
Channels
Ch1,…
Ch3, Msg1
Ch2,…
1. Dequeue DelCh1
3. Delete from Messages
4. Delete queue entry
2. Delete Ch1 from Channels

31. Resuming After Failure
Delete channel
Delete messages worker
Front end
Worker 1
Del Ch1h
Queue
Messages
Ch1, Msg1
Del Ch5
Del Ch11
Del Ch1
Del Ch1 2
Ch1, Msg2
Ch1, Msg3
Front End
Ch2, Msg1
Worker2
Ch2, Msg2
Channels
Ch1,…
Ch3, Msg1
Ch2,…
2. Fails after deleting Ch1 and Msg1
3. DelCh1 is visible again
1. Dequeue DelCh1 and start delete
4. Dequeue DelCh1 again
5. Repeat delete operations

32. Future Windows Azure Table Support
At PDC
Single Index
Query and retrieve results sorted by PartitionKey and RowKey
Future Support for
Secondary Indexes
Query and retrieve results sorted by other properties
Single Entity Transactions
Single Entity Insert, Update, or Delete Atomically
Entity Groups
Atomic transactions across multiple entities within same partition

33. Summary Azure tables are Simple familiar API
Massively Scalable
Highly Available
Simple familiar API
Use .NET –- ADO.NET Data Services and LINQ
Or use REST
Leverage your .NET expertise
ASP .NET integration for instant front ends

34. Resources Links Windows Azure ADO.NET Data Services
ADO.NET Data Services

35. Evals & Recordings www.microsoftpdc.com
Please fill out your evaluation for this session at:
This session will be available as a recording at:

36. Please use the microphones provided
4/1/2017 2:37 PM
Q&A
Please use the microphones provided
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

37. 4/1/2017 2:37 PM
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

38. 4/1/2017 2:37 PM
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.