virtual techdays INDIA │ november 2010 Windows Azure Storage – Deep Dive Gaurav Mantri │ Founder, Cerebrata Software

 Overview of Windows Azure Storage  Blob Containers & Blobs  Tables & Entities  Queues & Messages  API & Tools  Resources  Q&A virtual techdays INDIA │ november 2010 S E S S I O N A G E N D A

 Please provide feedback for the session. Again, the feedback console for this session will be live till 5:00pm today. Your feedback is considered in deciding the content for the next event.  In case you are facing any technical problems, please change your status on the top-right corner of your screen, and someone will assist you.  If you have questions in between the presentation, use the “Q&A panel” on Live Meeting console to ask your question. I will try to answer as many questions as possible. virtual techdays INDIA │ november 2010 Housekeeping

virtual techdays INDIA │ november 2010 Windows Azure Storage Storage Options Blob StorageTable Storage Queue Storage SQL Azure

 Azure storage is URL accessible  – Blob storage end point  – Table storage end point  – Queue storage end point  Each component is scoped by account name which needs to be unique across Azure storage.  Access is restricted by using an account key virtual techdays INDIA │ november 2010 Azure Storage Overview

 Overview of Windows Azure Storage  Blob Containers & Blobs  Tables & Entities  Queues & Messages  API & Tools  Resources  Q&A virtual techdays INDIA │ november 2010 S E S S I O N A G E N D A

 Think of it as file system in the cloud.  Similar to Amazon S3.  2 Level hierarchy.  No limit of number of blob containers & blobs.  Size limited to the size of Storage Account (currently 100 TB). virtual techdays INDIA │ november 2010 Blob Storage – Containers & Blobs Overview

 Think of it as “Folders” in Cloud file system.  Similar to “Buckets” in Amazon S3.  Scoped by storage account.  No limit on number of blobs.  Nested blob containers not supported.  Supports metadata (up to 8K in size).  Supports ACL & Signed URL.  Typical URL:  “$root” blob container:  Use “$root” for storing client access policy file. virtual techdays INDIA │ november 2010 Blob Containers Overview

 Create  Delete  Get/set metadata  Get/set ACL & access policies  List blobs virtual techdays INDIA │ november 2010 Blob Containers Operations

 The blob container name must be a valid DNS name, confirming to the following naming rules:  Blob container names must start with a letter or number, and can contain only letters, numbers, and the dash (-) character.  Every dash (-) character must be immediately preceded and followed by a letter or number e.g. my-blob-container.  All letters in a container name must be lowercase e.g. siteimages.  Blob container names must be from 3 through 63 characters long.  Valid blob container names: abc, myblobcontainer, videos-and-images  Invalid blob container names: a, MyBlobContainer, videos_and_images virtual techdays INDIA │ november 2010 Blob Containers Naming

 ACL & Access Policies provide a mechanism to provide additional access security on blob containers and blobs.  Account owner will always have complete control over blob container.  Provide 3 kinds of access control over blob container:  Full public read access (Container)  Public read access for blobs only (Blob)  No public read access (Private)  Up to 5 access policies can be defined for a blob container.  Provide read/write/delete/list access on blob container and blobs without giving away account credentials.  Use “Signed URL” to provide time limited access to blob container and blobs. virtual techdays INDIA │ november 2010 Blob Containers ACL & Access Policies

virtual techdays INDIA │ august 2010 DEMO: Blob Container ACL

 Ability to assign name/value pairs as blob container metadata.  Metadata size is limited to 8K.  Metadata name must be a valid C# identifier. virtual techdays INDIA │ november 2010 Blob Containers Metadata

 Think of them as “Files” in the cloud.  URI accessible e.g. /myblob.txt  Scoped by blob container.  Flat hierarchy however an illusion of folder hierarchy is achievable.  Size limited by blob type and storage account capacity.  Supports metadata (up to 8K in size).  Two kinds of blobs – Block blob & Page blob. virtual techdays INDIA │ november 2010 Blobs Overview

 Create/Upload/Download  Delete  Copy  Get/Set Metadata  Snapshot  Lease virtual techdays INDIA │ november 2010 Blobs Operations

 Ideal for streaming purposes e.g. images, documents, videos etc.  Each block blob can be up to 200GB in size.  Consists of a sequence of blocks where each block can be of a maximum of 4 MB size. virtual techdays INDIA │ november 2010 Blobs Block Blobs

 Ideal for random read/write workloads.  Each page blob can be up to 1TB in size.  Consists of a sequence of pages where each page is of 512 bytes.  Pay only for the pages used. Empty pages are not charged. virtual techdays INDIA │ november 2010 Blobs Page Blobs

 Azure drive is a page blob mounted as an NTFS drive.  Provides a durable NTFS volume for Windows Azure applications.  Mounted by one VM at a time for read/write.  Contents of an Azure drive is only visible to the VM which mounted it. virtual techdays INDIA │ november 2010 Blobs Azure Drive

 Overview of Windows Azure Storage  Blob Containers & Blobs  Tables & Entities  Queues & Messages  API & Tools  Resources  Q&A virtual techdays INDIA │ november 2010 S E S S I O N A G E N D A

 Use it to store structured data in the cloud.  NOT a relational database rather it’s a NO-SQL implementation by Microsoft.  Similar to Amazon SDB.  Possible to store huge amounts of data.  No limit of number of tables & entities.  Size limited to the size of Storage Account (currently 100 TB). virtual techdays INDIA │ november 2010 Table Storage – Tables & Entities Overview

 NOT a relational table.  Think of it as a property bag where disparate data can be stored.  For example, in one table you can store Customer and Order data.  Data in a table is “Partitioned” to support massive storage. virtual techdays INDIA │ november 2010 Tables Overview

 Create  Delete  Query entities  Entity group transaction  Insert  Update – Merge/Replace virtual techdays INDIA │ november 2010 Tables Operations

 The table name must be a valid DNS name, confirming to the following naming rules:  Table names may contain only alphanumeric characters.  A table name may not begin with a numeric character.  Table names are case-insensitive.  Table names must be from 3 through 63 characters long.  Valid table names: Abc, Customer, Account1  Invalid blob container names: A, 1Temp virtual techdays INDIA │ november 2010 Tables Naming

 NOT a row in a table.  Think of it as a property bag.  A maximum of 255 properties per entity.  Mandatory properties – PartitionKey, RowKey, and Timestamp (system generated).  Each entity can be of a maximum of 1MB in size.  Each property can be of following data type virtual techdays INDIA │ november 2010 Entities Overview

 PartitionKey & RowKey are of “String” type.  Once created, PartitionKey & RowKey are not editable.  An entity can be uniquely identified by it’s PartitionKey and RowKey value.  Timestamp value is system generated and is always in UTC. virtual techdays INDIA │ november 2010 Entities PartitionKey, RowKey and Timestamp

 Supports the query options defined by the ADO.NET Data Services Framework.  Not all query options supported.  Querying table will return all attributes of matching entities i.e. you can’t select attributes to return.  For best performance, one must always include “PartitionKey” in query criteria.  Example (PartitionKey eq ‘MyPartitionKey’) and (RowKey eq ‘MyRowKey’) virtual techdays INDIA │ november 2010 Tables Querying tables

virtual techdays INDIA │ august 2010 DEMO: Tables & Entities

 Overview of Windows Azure Storage  Blob Containers & Blobs  Tables & Entities  Queues & Messages  API & Tools  Resources  Q&A virtual techdays INDIA │ november 2010 S E S S I O N A G E N D A

 Use it as reliable messaging infrastructure in the cloud.  Similar to Amazon SQS.  No limit of number of queues and messages.  Set a limit for message expiration (currently a message can stay for a maximum of 7 days). virtual techdays INDIA │ november 2010 Queues & Messages Overview

 The queue name must be a valid DNS name, confirming to the following naming rules:  A queue name must start with a letter or number, and may contain only letters, numbers, and the dash (-) character.  The first and last letters in the queue name must be alphanumeric. The dash (-) character may not be the first or last letter.  All letters in a queue name must be lowercase.  A queue name must be from 3 through 63 characters long.  Valid queue names: abc, myqueue, poison-messages  Invalid queue names: Abc, MYQUEUE, -poison_messages virtual techdays INDIA │ november 2010 Queues Naming

 A message can be 8K in size. For larger messages use Blob or Table storage.  No limit on a number of messages in a queue.  Message expiration date can be specified. A message can stay in a queue for a maximum of 7 days.  Up to a maximum of 32 messages can be fetched at a time.  Peek/Get message semantics. virtual techdays INDIA │ november 2010 Messages Overview

 Overview of Windows Azure Storage  Blob Containers & Blobs  Tables & Entities  Queues & Messages  API & Tools  Resources  Q&A virtual techdays INDIA │ november 2010 S E S S I O N A G E N D A

 Azure Storage is accessible via REST API.  Microsoft has provided a.Net based storage client library which is built on top of REST API.  Open source libraries are available for Java & PHP support. virtual techdays INDIA │ november 2010 API & Tools API

 Storage can also be managed by a number of tools both from Microsoft & ISVs.  Tools range from open source to commercial.  Partial list of available tools:  Cerebrata Cloud Storage Studio - $70.00  ClumsyLeaf Azure Explorer - $50.00  Microsoft MMC Plugin – Free  Microsoft Visual Studio Plugin – Free  Microsft MyAzureStorage.com – Free  Azure Storage Explorer – Free  storage-explorers.aspx storage-explorers.aspx virtual techdays INDIA │ november 2010 API & Tools Tools

 Overview of Windows Azure Storage  Blob Containers & Blobs  Tables & Entities  Queues & Messages  API & Tools  Resources  Q&A virtual techdays INDIA │ november 2010 S E S S I O N A G E N D A

 Windows Azure Home Page:   Windows Azure Storage Team Blog:   Windows Azure MSDN Forums:   Neil Mackenzie Blog:   MSDN Documentation:  virtual techdays INDIA │ november 2010 Resources Useful links

virtual techdays THANKS │ november 2010 Questions??

virtual techdays THANKS │ november 2010 │