1. Matthew Kerner Principal Program Manager Lead Microsoft Corporation
COS401
Tips & Tricks: Low Latency Web Applications on the Windows Azure Platform
Matthew Kerner
Principal Program Manager Lead
Microsoft Corporation
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Low Latency Web Applications
What do I mean?
Global user base, snappy response times
May scale out to handle many concurrent requests
Heavy CPU, memory, disk and network bandwidth requirements
How does this apply to the cloud?
Windows Azure applications are mostly like on-premises applications…
… but some things are different (and even easier) in the cloud.
I will assume you already know the basics of Windows Azure
Agenda
Part 1: Asynchronous design patterns
Part 2: Managing data access
Part 3: Tuning application performance

3. Part 1: Asynchronous Design Patterns

4. Synchronous Design Pattern
Handle one request at a time for each thread
Block on “the work” done for each request, then respond & repeat
Each thread services next request only after first completes
The thread count is the scalability bottleneck
Poor scaling: adding another thread enables only one additional concurrent transaction
Web App Front End
SQL Azure
Client Request #1
“The Work” #1
Middle Tier
Client Response #1
Thread
Thread
Response #1
WA Storage
blocks
Client Request #2
Waiting…
Time passes…

5. Asynchronous Design Pattern
Handle one request at a time for each thread
Queue “the work”, then either handle the next request or return results to the client
Threads can handle additional requests while previous requests are in-flight
The bottleneck is no longer the thread count
Good scaling: adding additional threads enables > 1 concurrent transaction
Caveat: client requests tracked explicitly in app’s data structures
Consider context overhead and contention on thread-safe data structures
Web App Front End
SQL Azure
Client Request #1
“The Work” #1
Middle Tier
Client Response #1
Response #1
WA Storage
Client Request #2
Thread
Thread
“The Work” #2
Context
Client Response #2
Response #2

6. Asynchronous All the Way
Every tier of your application should be asynchronous
Each tier pushes the bottleneck deeper into the application
SQL Azure and Windows Azure Storage both support asynchronous I/O
ADO.NET Entity Framework
OData (WCF Data Services)
LINQ to SQL
Plain old ADO.NET

7. A Note on Failures
Windows Azure is designed to run on commodity hardware
Eliminate hardware redundancy to reduce costs
Failures can happen at any time
Local in-memory or on-disk data is not durable
You must design redundancy into software
Windows Azure Storage Queues are useful for async communication between role instances
Gracefully handle loss of role instances with message invisibility support
Built-in load balancing
Work items can be executed multiple times: must be idempotent 1
Queue BE FE 3 3 2 2 1 1 BE FE 2 1 3

8. Part 2(a): Managing Client Data Access

9. Get Out of the Way
How to transfer data efficiently to and from clients?
There are different kinds of data; each has its own tricks
Trick #1: Get out of the way when you can
Send clients directly to blob storage for public static content
Media (e.g. images, video)
Binaries (e.g. XAP, MSI, ZIP)
Data files (e.g. XML)
Blob
Storage
Hosted Compute
Hosted Compute

10. Shared Access Signatures
What about private content?
Trick #2: Shared access signatures provide direct access to ACLed content
Can be time-bound or revoked on demand
Beware leaking the SAS to adversaries
Also works for write access (e.g. user-generated content)
2. Service prepares a Shared Access Signature (SAS) to X using the securely stored storage account key
[The caller who presents this URL is allowed to access X between times A and B] Storage Key
1. “I am Bob & I want X”
Hosted Compute
Stg Key
3. Service returns SAS (signed HTTPS URL)
Blob Storage
4. Bob uses SAS to access X directly from Blob Storage for reduced latency & compute load
Non-public blob
(e.g. paid or ad-funded content) X

11. Serve Blobs from the Edge
Trick #3: Serve public blobs from the edge with the Windows Azure CDN
Use the CDN if you expect multiple reads before content expiration
Reduces cost, latency and load on central storage account
Possibly many hops or poor links
Few hops
CDN
Blob Storage
Closest Point of Presence
Public container X X
DNS name resolves to closest POP
Blob header determines time-to-live at the edge

12. Windows Azure Content Delivery Network
24 global locations
Enabling CDN access for your Windows Azure storage account
Enable the CDN in the dev portal
It will generate a new URL for CDN-based access to your account
CDN URL served from edge:
WA Storage URL served from source:
CNAME mappings supported
Smooth streaming CTP is coming this summer

13. Effective Caching vs. Stale Content
Will the CDN serve stale content?
Default behavior is to fetch once and cache for up to 72 hrs
Modify cache control blob header to control the TTL
x-ms-blob-cache-control: public, max-age=<value in seconds>
Think hours, days or weeks
Higher numbers reduce cost and latency via CDN & downstream caches
Use versioned URLs to expire content on-demand
Enables easy rollback and A/B testing
HTML Served by App
CDN
Blob Storage
… <img src=" />…
… <img src=" />…
logo png
logo png
logo png
logo png

14. CDN for Web Apps What about dynamic content?
CDN now works for web apps, not just for public blobs
Same model as before, but source is your web app
Useful for stable dynamic content
Source URL:
CDN URL:
Cached URL:
CNAMEs & HTTPS supported
Choose to index with or without query string

15. Windows Azure Traffic Manager
What about non-stable dynamic content, and writes?
Trick #4: Direct users to the service in the closest region with the Windows Azure Traffic Manager
CNAMEs supported
Useful for performance, business continuity, price, compliance & tax
Not the same as CDN
Not serving from the edge
Only DNS is cached (at client)
foo-us.cloudapp.net
foo.cloudapp.net
Traffic Manager
DNS response
foo-europe.cloudapp.net
Policies
Monitoring
foo-asia.cloudapp.net

16. Traffic Manager Details
Multiple factors determine DNS resolution
Configured by Microsoft
Geo-IP mapping
Periodic performance measurement
Configured by service owner
Policy: Performance, Failover, Geo, Ratio
Monitoring
Currently in CTP
Only works in “production” slot but not yet intended for production use
No SLA or billing, and the Traffic Manager domain name will change
foo-us.cloudapp.net
Periodic GETs
foo-europe.cloudapp.net
Monitoring
/monitoring/testme.aspx
foo-asia.cloudapp.net

17. Part 2(b): Managing Server Data Access

18. In-Memory Caching
Trick #5: Cache hot data in memory to avoid data-tier access
Session state (e.g. shopping cart) & immutable reference data (e.g. product catalog entries)
Caching tier will help you reduce latency and cost
Lower latency/higher throughput than data tier, especially under load
In-Memory Caching
SQL Azure
SQL Azure
Hosted Compute
Table Storage
Table Storage

19. Windows Azure AppFabric Caching
What is it?
AppFabric Caching is a “Distributed In-Memory Cache As A Service”
Similar to on-premises Windows Server AppFabric Cache
Roadmap to full parity between on-premise and cloud where it makes sense
Excellent performance
Highly scalable, 64-bit service
Low latency, hosted per subregion for app affinity
Highly-available
AuthN/AuthZ integrated with Access Control Service
You may have heard of memcached too
Latest memcached works on Windows Azure with 1.3+ SDK
You manage instances and pay for compute hours

20. AppFabric Caching Advantages
Simple to administer
No need to manage and host a distributed cache yourself
Integrates easily into existing applications
ASP.NET session state and output cache providers enable no-code integration
Same managed interfaces as Windows Server AppFabric Cache
Near cache (client-local) for hot data without serialization costs
On-Premises App
Windows Azure App
Windows Server AppFabric Cache
Windows Azure AppFabric Caching
Core Logic
AppFabric Cache APIs
Core Logic
AppFabric Cache APIs

21. SQL Azure & Sharding
Trick #6: Partition (or shard) your SQL Azure data across databases
Spreads load across multiple database instances
Avoid hitting database size limits
Parallelized queries across more nodes
Improved query performance on commodity hardware
Partitioning scheme varies per data set
My sample scheme is not a good one! Why not?
A-M
A-Z
Hosted Compute
N-Z

22. Part 3: Performance Tuning

23. Basic Performance Tuning
Tune Windows Azure applications just as you would on-premises applications
Measure & optimize where it makes a difference
Windows Azure uses Full IIS for web roles in the 1.3+ SDK
Startup admin tasks using WebPI can install up-to-date extensions as desired
Startup admin tasks using AppCmd can configure IIS as desired
Basic tips
Build in release mode (not debug mode) for production
Do not enable IntelliTrace or Failed Request Tracing in production
Tune role instance counts and consider dynamic scaling

24. Advanced Performance Tuning
Enable compression for additional dynamic content types
<add mimeType="application/json" enabled="true" />
<add mimeType="application/json; charset=utf-8" enabled="true" />
Office document formats
Tune application pool recycling to suit your workload
Modify default schedule to avoid peak times
Measure and eliminate memory leaks if footprint grows over time
Move ASP.NET cache to the resource disk for more space
Tune Windows Azure Diagnostics settings

25. Summary Blob Storage Approach WA apps like you would on-premises apps
Cache control
Versioned URLs
CDN
Public
Public
Web App
Private
Traffic Mgr
Asynchronous Hosted Compute
Asynchronous Hosted Compute
Synchronous Hosted Compute
AppFabric Caching
Shared Access Signatures
Table Storage
Table Storage
Stg Key
Tuning
SQL Azure
SQL Azure
Approach WA apps like you would on-premises apps
Use rich platform features in Windows Azure to tune for the cloud too
SQL Azure
Sharding

26. Required Slide
Speakers, please list the Breakout Sessions, Interactive Discussions, Labs, Demo Stations and Certification Exam that relate to your session. Also indicate when they can find you staffing in the TLC.
Tech Ed North America 2010
5/21/ :15 AM
Related Content
BOF10-DEV Developing for Windows Azure: Real-World Lessons from Real-World Projects
BOF12-DEV How Do I Design Scalable Cloud Applications
COS301 Inside Windows Azure, the Cloud Operating System with Mark Russinovich
COS318 A Lap Around Windows Azure AppFabric
COS276-INT Windows Azure Traffic Manager: Improve Application Performance and Availability Using Global Load Balancing
COS275-HOL Windows Azure Content Distribution Network (CDN)
CS285-HOL Microsoft SQL Azure Tips and Tricks
Find Me Later At the Windows Azure booth or Cloud Power Lounge
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

27. Tech Ed North America 2010
5/21/ :15 AM
Track Resources
Don’t forget to visit the Cloud Power area within the TLC (Blue Section) to see product demos and speak with experts about the Server & Cloud Platform solutions that help drive your business forward.
You can also find the latest information about our products at the following links:
Cloud Power -
Private Cloud -
Windows Server -
Windows Azure -
Microsoft System Center -
Microsoft Forefront -
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

28. Resources Learning http://northamerica.msteched.com
Tech Ed North America 2010
5/21/ :15 AM
Resources
Connect. Share. Discuss.
Learning
Sessions On-Demand & Community
Microsoft Certification & Training Resources
Resources for IT Professionals
Resources for Developers
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

29. Complete an evaluation on CommNet and enter to win!
Tech Ed North America 2010
5/21/ :15 AM
Complete an evaluation on CommNet and enter to win!
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

30. Tech Ed North America 2010 5/21/2018 10:15 AM
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

31. 5/21/ :15 AM
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.