Building Low Latency Web Applications 9/29/2017 11:57 AM Building Low Latency Web Applications Name Title Company © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Low Latency Web Applications What do I mean? Global user base, snappy response times Scale out to handle many concurrent requests Heavy transaction load & bandwidth requirements How does this apply to the cloud? Windows Azure applications are mostly like on-premises applications… … but some things are different (and even easier) in the cloud. I will assume you already know the basics of Windows Azure Agenda Asynchronous design patterns Managing data access Tuning application performance

Synchronous Design Pattern Handle one request at a time for each thread Block on “the work” done for each request, then respond & repeat Thread stacks are the data structures used to track client requests Threads spend most of their time blocking Increase thread count for greater throughput – heavyweight approach Web App Front End SQL Azure Client Request #1 “The Work” #1 Middle Tier Client Response #1 Thread Thread Response #1 WA Storage blocks Client Request #2 Waiting… Time passes…

Asynchronous Design Pattern Handle one request at a time for each thread Queue “the work”, then handle the next request or post back results Client requests tracked explicitly in app’s data structures Local state is not durable External state operations must be idempotent or transactional Threads block less so fewer threads provide greater throughput Throughput gated primarily on duration of “The Work” Web App Front End SQL Azure Client Request #1 “The Work” #1 Middle Tier Client Response #1 Response #1 WA Storage Client Request #2 Thread Thread “The Work” #2 Context Client Response #2 Response #2

Asynchronous Cloud Apps Async design applies to both on-premise and cloud apps equally Windows Azure Storage Queues are useful for async communication between role instances Built-in load balancing Handles loss of individual role instances gracefully SQL Azure and Windows Azure Storage both support asynchronous calls ADO.NET Entity Framework WCF Data Services LINQ to SQL Plain old ADO.NET

Managing Data Access How to transfer data efficiently to and from clients? There are different kinds of data; each has its own tricks Trick #1: Get out of the way when you can Send clients directly to blob storage for static content Media (e.g. images, video) Binaries (e.g. XAP, MSI, ZIP) Data files (e.g. XML) Blob Storage Hosted Compute Hosted Compute

Shared Access Signatures Trick #2: Shared access signatures provide direct access to ACLed content Can be time-bound or revoked on demand Also works for write access (e.g. user-generated content) http://blog.smarx.com/posts/shared-access-signatures-are-easy-these-days 2. Service prepares a Shared Access Signature (SAS) to X using the securely stored storage account key 1. “I am Bob & I want X” Hosted Compute Stg Key 3. Service returns SAS (signed HTTPS URL) Blob Storage 4. Bob uses SAS to access X directly from Blob Storage for reduced latency & compute load Non-public blob (e.g. paid or ad-funded content) X

Serve Blobs from the Edge Trick #3: Serve public blobs from the edge with the Windows Azure CDN Reduces latency and central storage load Use the CDN when you expect multiple accesses before content expiration Possibly many hops or slow/lossy links Few hops CDN Blob Storage Closest Point of Presence Public container X X DNS name resolves to closest POP Blob header determines time-to-live at the edge

Windows Azure Content Delivery Network >20 global locations with 99.95% availability Enabling CDN access for your Windows Azure storage account Enable the CDN in the dev portal It will generate a new URL for CDN-based access to your account Same content, 2 URLs with different access patterns CDN URL: http://azXXXX.vo.msecnd.net/images/myimage.png WA Storage URL: http://myacct.blob.core.windows.net/images/myimage.png CNAME mappings to CDN URLs http://blog.smarx.com/posts/using-the-new-windows-azure-cdn-with-a-custom-domain Adaptive Streaming can be made to work with the CDN too http://blog.smarx.com/posts/smooth-streaming-with-windows-azure-blobs-and-cdn Will not require cleverness soon

Managing CDN Content Expiration Default behavior is to fetch once and cache for up to 72 hrs Modify max-age cache control blob header to control the TTL x-ms-blob-cache-control: public, max-age=<value in seconds> Think hours, days or weeks Higher numbers reduce cost and latency via CDN, proxy & browser caches Use versioned URLs to expire content on-demand Enables easy rollback and A/B testing HTML Served by App CDN Blob Storage … <img src="http://azXXXX.vo.msecnd.net/images/logo.2010-10-29.png" />… … <img src="http://azXXXX.vo.msecnd.net/images/logo.2010-08-01.png" />… logo.2010-08-01.png logo.2010-08-01.png logo.2010-10-29.png logo.2010-10-29.png

In-Memory Caching Trick #4: Cache hot data in memory to avoid slower data-tier access Session state (e.g. shopping cart) & immutable reference data (e.g. product catalog entries) Caching tier will help you reduce latency and cost Lower latency/higher throughput than data tier, especially under load In-Memory Caching SQL Azure SQL Azure Hosted Compute Table Storage Table Storage

Anatomy of A Distributed Cache Cache footprint or bandwidth requirement may grow beyond a single VM Distributed caches scale out Multiple role instances may be cache clients Clients access the cache as if it was a single large namespace Unified Cache View Cache layer distributes data across the various cache instances

Windows Azure AppFabric Caching What is it? Windows Server AppFabric Cache is an on-premise distributed in-memory cache Windows Azure AppFabric Caching is a new, hosted in-memory caching service CTP release at PDC Roadmap to full parity between on-premise and cloud where it makes sense Excellent performance Highly scalable, 64-bit service Low latency, will be hosted per subregion for app affinity Highly-available AuthN/AuthZ integrated with Access Control Service You may have heard of memcached too Latest memcached works on Windows Azure with 1.3 SDK http://code.msdn.microsoft.com/winazurememcached You manage instances and pay for compute hours

AppFabric Caching Advantages Simple to administer No need to manage and host a distributed cache yourself Integrates easily into existing applications ASP.NET session state and output cache providers enable no-code integration Same managed interfaces as Windows Server AppFabric Cache Caches any serializable managed object No object size limits Near cache (client-local) for hot data without serialization costs On-Premises App Windows Azure App Windows Server AppFabric Cache Windows Azure AppFabric Caching Core Logic AppFabric Cache APIs Core Logic AppFabric Cache APIs

AppFabric Caching Security Configuration <configuration> <dataCacheClient deployment="Simple"> <hosts> <host name="<your URI>" cachePort="22233" /> </hosts> <securityProperties mode="Message"> <messageSecurity authorizationInfo="<your authentication token>" /> </securityProperties> </dataCacheClient> </configuration>

AppFabric Caching Session State Provider <configuration> <system.web> <sessionState mode="Custom" customProvider="DistributedSessionProvider" compressionEnabled="false"> <providers> <add name="DistributedSessionProvider" type="Microsoft.Web.DistributedCache.DistributedCacheSessionStateStoreProvider,Microsoft.Web.DistributedCache" cacheName="default" applicationName="Contoso" useBlobMode="false"/> </providers> </sessionState> </system.web> </configuration>

AppFabric Caching Output Provider <system.web> <caching> <outputCache defaultProvider="DistributedCache"> <providers> <add name="DistributedCache" type="Microsoft.Web.DistributedCache.DistributedCacheOutputCacheProvider,Microsoft.Web.DistributedCache" cacheName="default" applicationName="Contoso" /> </providers> </outputCache> </caching> </system.web>

AppFabric Caching Code // Use cache configuration from app config DataCacheFactory CacheFactory = new DataCacheFactory(); // Get cache client for cache DataCache myCache = CacheFactory.GetDefaultCache(); // Add an object to the cache. myCache.Put(“myKey", myObject); retrievedObject = myCache.Get("myKey"); if (retrievedObject == null) { // Cache miss }

SQL Azure & Sharding Trick #5: Partition (or shard) your SQL Azure data across databases Spreads load across multiple database instances Avoid hitting database size limits Parallelized queries across more nodes Improved query performance on commodity hardware Partitioning scheme varies per data set A-M A-Z Hosted Compute N-Z

Basic Performance Tuning Tune Windows Azure applications just as you would on-premises applications Measure Optimize where it makes a difference Windows Azure uses Full IIS for web roles starting with the 1.3 SDK Startup admin tasks using WebPI can install up-to-date extensions as desired Startup admin tasks using AppCmd can configure IIS as desired Basic tips Build in release mode (not debug mode) for production Do not enable IntelliTrace or Failed Request Tracing in production Tune role instance counts and consider dynamic scaling

Advanced Performance Tuning Enable compression for additional dynamic content types <add mimeType="application/json" enabled="true" /> <add mimeType="application/json; charset=utf-8" enabled="true" /> Office document formats Tune application pool recycling to suit your workload Modify default schedule to avoid peak times Measure and eliminate memory leaks if footprint grows over time New IIS 7.5 module to warm up app upon init at http://www.iis.net/download/ApplicationWarmUp Move ASP.NET cache to the resource disk for more space Tune Windows Azure Diagnostics settings

Summary Approach WA apps like you would on-premises apps Blob Storage Cache control Versioned URLs CDN Public Public Private Synchronous Hosted Compute Asynchronous Hosted Compute AppFabric Caching Shared Access Signatures Table Storage Table Storage Stg Key Tuning SQL Azure SQL Azure Approach WA apps like you would on-premises apps Use rich platform features in Windows Azure to tune for the cloud too SQL Azure Sharding

© 2010 Microsoft Corporation. All rights reserved © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.