1. Building Low Latency Web Applications
9/29/ :57 AM
Building Low Latency Web Applications
Name
Title
Company
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Low Latency Web Applications
What do I mean?
Global user base, snappy response times
Scale out to handle many concurrent requests
Heavy transaction load & bandwidth requirements
How does this apply to the cloud?
Windows Azure applications are mostly like on-premises applications…
… but some things are different (and even easier) in the cloud.
I will assume you already know the basics of Windows Azure
Agenda
Asynchronous design patterns
Managing data access
Tuning application performance

3. Synchronous Design Pattern
Handle one request at a time for each thread
Block on “the work” done for each request, then respond & repeat
Thread stacks are the data structures used to track client requests
Threads spend most of their time blocking
Increase thread count for greater throughput – heavyweight approach
Web App Front End
SQL Azure
Client Request #1
“The Work” #1
Middle Tier
Client Response #1
Thread
Thread
Response #1
WA Storage
blocks
Client Request #2
Waiting…
Time passes…

4. Asynchronous Design Pattern
Handle one request at a time for each thread
Queue “the work”, then handle the next request or post back results
Client requests tracked explicitly in app’s data structures
Local state is not durable
External state operations must be idempotent or transactional
Threads block less so fewer threads provide greater throughput
Throughput gated primarily on duration of “The Work”
Web App Front End
SQL Azure
Client Request #1
“The Work” #1
Middle Tier
Client Response #1
Response #1
WA Storage
Client Request #2
Thread
Thread
“The Work” #2
Context
Client Response #2
Response #2

5. Asynchronous Cloud Apps
Async design applies to both on-premise and cloud apps equally
Windows Azure Storage Queues are useful for async communication between role instances
Built-in load balancing
Handles loss of individual role instances gracefully
SQL Azure and Windows Azure Storage both support asynchronous calls
ADO.NET Entity Framework
WCF Data Services
LINQ to SQL
Plain old ADO.NET

6. Managing Data Access
How to transfer data efficiently to and from clients?
There are different kinds of data; each has its own tricks
Trick #1: Get out of the way when you can
Send clients directly to blob storage for static content
Media (e.g. images, video)
Binaries (e.g. XAP, MSI, ZIP)
Data files (e.g. XML)
Blob
Storage
Hosted Compute
Hosted Compute

7. Shared Access Signatures
Trick #2: Shared access signatures provide direct access to ACLed content
Can be time-bound or revoked on demand
Also works for write access (e.g. user-generated content)
2. Service prepares a Shared Access Signature (SAS) to X using the securely stored storage account key
1. “I am Bob & I want X”
Hosted Compute
Stg Key
3. Service returns SAS (signed HTTPS URL)
Blob Storage
4. Bob uses SAS to access X directly from Blob Storage for reduced latency & compute load
Non-public blob
(e.g. paid or ad-funded content) X

8. Serve Blobs from the Edge
Trick #3: Serve public blobs from the edge with the Windows Azure CDN
Reduces latency and central storage load
Use the CDN when you expect multiple accesses before content expiration
Possibly many hops or slow/lossy links
Few hops
CDN
Blob Storage
Closest Point of Presence
Public container X X
DNS name resolves to closest POP
Blob header determines time-to-live at the edge

9. Windows Azure Content Delivery Network
>20 global locations with 99.95% availability
Enabling CDN access for your Windows Azure storage account
Enable the CDN in the dev portal
It will generate a new URL for CDN-based access to your account
Same content, 2 URLs with different access patterns
CDN URL:
WA Storage URL:
CNAME mappings to CDN URLs
Adaptive Streaming can be made to work with the CDN too
Will not require cleverness soon

10. Managing CDN Content Expiration
Default behavior is to fetch once and cache for up to 72 hrs
Modify max-age cache control blob header to control the TTL
x-ms-blob-cache-control: public, max-age=<value in seconds>
Think hours, days or weeks
Higher numbers reduce cost and latency via CDN, proxy & browser caches
Use versioned URLs to expire content on-demand
Enables easy rollback and A/B testing
HTML Served by App
CDN
Blob Storage
… <img src=" />…
… <img src=" />…
logo png
logo png
logo png
logo png

11. In-Memory Caching
Trick #4: Cache hot data in memory to avoid slower data-tier access
Session state (e.g. shopping cart) & immutable reference data (e.g. product catalog entries)
Caching tier will help you reduce latency and cost
Lower latency/higher throughput than data tier, especially under load
In-Memory Caching
SQL Azure
SQL Azure
Hosted Compute
Table Storage
Table Storage

12. Anatomy of A Distributed Cache
Cache footprint or bandwidth requirement may grow beyond a single VM
Distributed caches scale out
Multiple role instances may be cache clients
Clients access the cache as if it was a single large namespace
Unified Cache View
Cache layer distributes data across the various cache instances

13. Windows Azure AppFabric Caching
What is it?
Windows Server AppFabric Cache is an on-premise distributed in-memory cache
Windows Azure AppFabric Caching is a new, hosted in-memory caching service
CTP release at PDC
Roadmap to full parity between on-premise and cloud where it makes sense
Excellent performance
Highly scalable, 64-bit service
Low latency, will be hosted per subregion for app affinity
Highly-available
AuthN/AuthZ integrated with Access Control Service
You may have heard of memcached too
Latest memcached works on Windows Azure with 1.3 SDK
You manage instances and pay for compute hours

14. AppFabric Caching Advantages
Simple to administer
No need to manage and host a distributed cache yourself
Integrates easily into existing applications
ASP.NET session state and output cache providers enable no-code integration
Same managed interfaces as Windows Server AppFabric Cache
Caches any serializable managed object
No object size limits
Near cache (client-local) for hot data without serialization costs
On-Premises App
Windows Azure App
Windows Server AppFabric Cache
Windows Azure AppFabric Caching
Core Logic
AppFabric Cache APIs
Core Logic
AppFabric Cache APIs

15. AppFabric Caching Security Configuration
<configuration> <dataCacheClient deployment="Simple"> <hosts> <host name="<your URI>" cachePort="22233" /> </hosts> <securityProperties mode="Message"> <messageSecurity authorizationInfo="<your authentication token>" /> </securityProperties> </dataCacheClient> </configuration>

16. AppFabric Caching Session State Provider
<configuration> <system.web> <sessionState mode="Custom" customProvider="DistributedSessionProvider" compressionEnabled="false"> <providers> <add name="DistributedSessionProvider" type="Microsoft.Web.DistributedCache.DistributedCacheSessionStateStoreProvider,Microsoft.Web.DistributedCache" cacheName="default" applicationName="Contoso" useBlobMode="false"/> </providers> </sessionState> </system.web> </configuration>

17. AppFabric Caching Output Provider
<system.web> <caching> <outputCache defaultProvider="DistributedCache"> <providers> <add name="DistributedCache" type="Microsoft.Web.DistributedCache.DistributedCacheOutputCacheProvider,Microsoft.Web.DistributedCache" cacheName="default" applicationName="Contoso" /> </providers> </outputCache> </caching> </system.web>

18. AppFabric Caching Code
// Use cache configuration from app config DataCacheFactory CacheFactory = new DataCacheFactory(); // Get cache client for cache DataCache myCache = CacheFactory.GetDefaultCache(); // Add an object to the cache. myCache.Put(“myKey", myObject); retrievedObject = myCache.Get("myKey"); if (retrievedObject == null) { // Cache miss }

19. SQL Azure & Sharding
Trick #5: Partition (or shard) your SQL Azure data across databases
Spreads load across multiple database instances
Avoid hitting database size limits
Parallelized queries across more nodes
Improved query performance on commodity hardware
Partitioning scheme varies per data set
A-M
A-Z
Hosted Compute
N-Z

20. Basic Performance Tuning
Tune Windows Azure applications just as you would on-premises applications
Measure
Optimize where it makes a difference
Windows Azure uses Full IIS for web roles starting with the 1.3 SDK
Startup admin tasks using WebPI can install up-to-date extensions as desired
Startup admin tasks using AppCmd can configure IIS as desired
Basic tips
Build in release mode (not debug mode) for production
Do not enable IntelliTrace or Failed Request Tracing in production
Tune role instance counts and consider dynamic scaling

21. Advanced Performance Tuning
Enable compression for additional dynamic content types
<add mimeType="application/json" enabled="true" />
<add mimeType="application/json; charset=utf-8" enabled="true" />
Office document formats
Tune application pool recycling to suit your workload
Modify default schedule to avoid peak times
Measure and eliminate memory leaks if footprint grows over time
New IIS 7.5 module to warm up app upon init at
Move ASP.NET cache to the resource disk for more space
Tune Windows Azure Diagnostics settings

22. Summary Approach WA apps like you would on-premises apps
Blob Storage
Cache control
Versioned URLs
CDN
Public
Public
Private
Synchronous Hosted Compute
Asynchronous Hosted Compute
AppFabric Caching
Shared Access Signatures
Table Storage
Table Storage
Stg Key
Tuning
SQL Azure
SQL Azure
Approach WA apps like you would on-premises apps
Use rich platform features in Windows Azure to tune for the cloud too
SQL Azure
Sharding

23. © 2010 Microsoft Corporation. All rights reserved
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.