4. … Application /Web Tier UsersUsers DatabaseDatabase Data tier Application Asp.Net app Session cart hosted in memory Asp.Net app Session cart hosted in memory Load balancer Sticky routing Load balancer Sticky routing Catalog sits in database

5. Gas Prices Go up!! Dow Goes Down!!

6. … Application / Web Tier Application UsersUsers Database Data Tier Application Web site’s too slow!! Servers are crashing Database is hot!! Where did my shopping cart go?

7. There is hope..

9. Unified cache view Velocity callers can be spread across machines or processes Callers access the cache as if it was a large single cache Cache layer distributes data across the various cache nodes

10. … Application /Web tier Application UsersUsers DatabaseDatabase CloudCloud Data tier Application Velocity access layer Velocity service Cache tier

12. OperationThroughputLatency Read 2k30,000 / sec3 – 4 ms Write 2k18,000 / sec3 ms OperationServersThroughput Read 2k130,000 / sec Read 2k258, 600 / sec Read 2k385, 500 / sec Velocity

13. * Requires windows server. Final SKUs/product positioning hasn’t been figured out and some capabilities/number of nodes may require additional licenses

15. CTP1 TechedTeched CTP2 PDC 2008 CTP3 End of Mar ‘09 RTM Mid 2009

16. SectorDescription MediaCache all media files and metadata; big media provider in US ContentBig content provider in Europe ContentMSDN forums is live on CTP2 HealthcareOne of the large health care ISV in the world; US FinanceBig brokerage firm/banking; high performance computation GamingNotifications for presence; EnterpriseLargest mainframe migration to Windows ManufacturingManufacturing process monitoring InsuranceSession data for insurance purchase

19. Caching Reference Data Web tier Clients Local cache (in Proc) Velocity Data tier User profile threads… User profile threads… Scenario: MSDN forums

20. Caching Activity-Oriented Data Data tier Application tier Distributed cache Integration hub Integration hub Clients Web tier Vendor services Pricing Vendor sources Order, invoice, payment Order, invoice, payment Aggregated vendor catalogs Aggregated vendor catalogs Scenario: Insurance

21. Caching Resource-Oriented Data Scenario: Flight Inventory and Pricing Application tier American Distributed cache Flight Itinerary Flight Itinerary Flight segment Flight price Flight segment Flight price Airlines Seat inventory Booking service

22. Catalog Shopping cart Shopping cart Inventory

25. Velocity Service Cache Tier Configuration Store (Can be database, File share, etc.) Stores Global Cache Policies Stores Current Partitioning Information Velocity Service Install Windows Cache Service Copy Client DLLs and add app.config … UsersUsers Application / Web Tier Application Application Velocity Client Application Velocity Client Application Velocity Client <section name="dcacheClient" type=“Microsoft.Data.Caching.DCacheClientSection,CacheBaseLibrary" allowLocation="true" allowDefinition="Everywhere"/> <host name="BL1CDB8083714" cachePort="22233" cacheHostName="DistributedCacheService"/>

26. // Simple Get/Put catalog.Put("toy-101", new Toy("Puzzle",.,.)); // From the same or a different client Toy toyObj = (Toy)catalog.Get("toy-101"); // Simple Get/Put catalog.Put("toy-101", new Toy("Puzzle",.,.)); // From the same or a different client Toy toyObj = (Toy)catalog.Get("toy-101"); // Region based Get/Put catalog.CreateRegion("toyRegion"); // Both toy and toyparts are put in the same region catalog.Put("toy-101", new Toy(.,.), “toyRegion”); Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”); Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion"); // Region based Get/Put catalog.CreateRegion("toyRegion"); // Both toy and toyparts are put in the same region catalog.Put("toy-101", new Toy(.,.), “toyRegion”); Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”); Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");

27. Regions Region A Key Payload Tags 121 xxxx “Toy” “Child” 123 yyyy “Toy” “Chair”.. Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects Velocity service Named cache : product catalog Named cache : electronics inventory Velocity service

28. // Add Tags Tag hotItem = new Tag("hotItem"); Tag discItem = new Tag("discountItem"); catalog.Put("toyRegion", "toy-101", new Toy("Puzzle"), new Tag[]{hotItem}); catalog.Put("toyRegion", "toy-102", new Toy("Bridge"), new Tag[]{hotItem,discItem}); // From the same or a different client List > toys = catalog.GetAnyMatchingTag("toyRegion", hotItem); // Add Tags Tag hotItem = new Tag("hotItem"); Tag discItem = new Tag("discountItem"); catalog.Put("toyRegion", "toy-101", new Toy("Puzzle"), new Tag[]{hotItem}); catalog.Put("toyRegion", "toy-102", new Toy("Bridge"), new Tag[]{hotItem,discItem}); // From the same or a different client List > toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);

29. // Read from Cache Toy toyObj = (Toy)catalog.Get("toy-101"); // Read from Cache Toy toyObj = (Toy)catalog.Get("toy-101"); Application Velocity Service // If Not present in the cache if (toyObj == null) { // Read from backend.. toyObj = ReadFromDatabase(); // Populate Cache catalog.Put("toy-101", toyObj); return toyObj; } // If Not present in the cache if (toyObj == null) { // Read from backend.. toyObj = ReadFromDatabase(); // Populate Cache catalog.Put("toy-101", toyObj); return toyObj; }

33. … Application / Web tier Application Cache tier Velocity service UsersUsers Application Velocity client local cache Velocity service Application Velocity client local cache Application Velocity client local cache AA CC EEDD BB GGIIHH FF

34. RegionKey "Cust1" "Cust2" "Cust33" "ToyRegion " "Toy101" "ToyRegion " "Toy102" "BoxRegio n" "Box101 " Partition (Range of Ids) 0 – 1000 1001 - 2000 … ….. xxx - Maxint Region (Name) Default Region 1 Default Region 2 … Default Region 256 ToyRegion BoxRegion Velocity Service Region Name Hashed into Region Id Keys Bucketize d into Regions ID Ranges mapped to Nodes

35. Application Cache2 Cache1 Primary for K2,v2 Primary for K2,v2 Primary for K1,V1 K1, v1 Cache3 Primary for K3,V3 K3, v3 Velocity client2 Get(k2) Routing table (K2, v2) Velocity client1 Routing table Put Using the routing table client routes The PUT to cache2 (primary) node Routing table K2, v2 Operations queue for notifications, to bring up a new secondary, etc.

38. Load test controller … Server 1 Server 2 Server 3 Cache client1 Cache client 2 Cache client n Load controller spawns test clients Client machines run test Cache service cluster Partitioned cache

39. Single Server Throughput Increases with Increasing Load Until Server Saturation Single Server Throughput Increases with Increasing Load Until Server Saturation LoadLoad ThroughputThroughput LatencyLatency Server 2 Added Throughput Increases Latency Decreases Until Server Saturation Server 2 Added Throughput Increases Latency Decreases Until Server Saturation Server 3 Added Throughput Increases Latency Decreases Server 3 Added Throughput Increases Latency Decreases

41. Velocity client Local cache Put(K2, V2) Routing table Cache2 Cache1 Primary for K2,V2 K2, v2 Primary for K1,V1 K1, v1 Cache3 Primary for K3,V3 K3, v3 Velocity client Local cache Routing table K2, V2 Get(K2)

42. Application (K2, V2) Cache2 Cache1 Cache3 Primary for (K2,V2) Get(K2) Primary for (K1,V1) Primary for (K3,V3) K3, v3 Velocity client1 Routing table K2, v2 Using the routing table client routes The PUT to cache2 (primary) node Using the routing table client routes The PUT to cache2 (primary) node Put Queues the PUT operation puts locally propagates operation to secondary Returns control Queues the PUT operation puts locally propagates operation to secondary Returns control Secondary for (K2,V2), (K3,V3) K2, v2 K1, v1 K3, v3 Secondary for (K1,V1), (K3,V3) K3, v3 K1, v1 Secondary for (K1,V1), (K2,V2) K1, v1 Velocity client Routing table K2, v2 Replication agent

43. Cache4 Primary for (K4,V4) Primary for (K4,V4) K4, V4 Secondary for K1, V1 K3, V3 Partition Manager Global Partition Map Cache2 Cache1 Cache3 Primary for (K2,V2) Primary for (K3,V3) K3, V3 Routing Table Secondary for K2, V2 K1, V1 K3, V3 Secondary for K3, V3 K1, V1 Secondary for K4, V4 K2, V2 Detects Cache 2 failure. Notifies PM (on Cache4) Replication Agent Reconfiguration Agent Reconfiguration Agent Local Partition Map Picks Cache1 as the primary for (K2,V2). Picks Cache 3 as new secondary PM analyzes the info on secondaries of all primary partitions of Cache2 to elect the primaries. Cache1 creates new secondary on Cache2 Cache1 initiates reconfiguration. After reconfig, Cache1 is primary for (K1, V1) and (K2, V2)

44. … Session state stored in velocity Velocity service Load balance requests No more sticky routing Load balance requests No more sticky routing Application Velocity sessionstoreprovid er Application Velocity sessionstoreprovide r Application Velocity sessionstoreprovider Velocity service Scale your session store dynamically dynamically Highly available Drop in velocity sessionstoreprovider <add name="SessionStoreProvider" type=" System.Data.Caching.SessionStoreProvider, ClientLibrary" cacheName=" "/> <add name="SessionStoreProvider" type=" System.Data.Caching.SessionStoreProvider, ClientLibrary" cacheName=" "/>

46. Cache cluster velocity with ASP.Net application Cache cluster velocity with ASP.Net application Northwind Application Northwind Application Asp.Net session Cache Asp.Net session Cache VELOCITY Velocity session Store provider Velocity session Store provider Northwind Application Northwind Application Asp.Net session Cache Asp.Net session Cache VELOCITY Velocity session Store provider Velocity session Store provider VELOCITY Partitioned cache

49. Version Based Update TimeClient1Client2 (Different Thread or process) T0 CacheItem item = catalog.GetCacheItem(“PlayerRegion ”, ”Zune”); CacheItem item = catalog.GetCacheItem(“PlayerRegion”, ”Zune”); T1((ZuneObject)item.Object).inventory - -; T2 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version); T3 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version); // Version mismatch // Client must retry again Two clients access the same item Both update the item Second Client gets in first; put succeeds because item version matches; atomically increments the version First client tries put; Fails because the versions don’t match First client tries put; Fails because the versions don’t match

50. K1 Client1: GetAndLock ("k1") Client1: GetAndLock ("k1") Client2: GetAndLock ("k1") Client2: GetAndLock ("k1") Client3: Get ("k1") Client3: Get ("k1") Regular Get succeeds GetAndLock gets lock handle Other GetAndLock on same item fails

52. Cache.RegisterCacheLevelCallback( int filter, CacheChangeCallback delegate); Cache.RegisterRegionLevelCallback( String region, int filter, CacheChangeCallback delegate); Cache.RegisterKeyLevelCallback( String region, String key, int filter, CacheChangeCallback delegate); public delegate void CacheChangeCallback( String CacheName, String regionName, String key, CacheItemVersion version, CacheEventType event, NotificationDescriptor nd ); Cache.RegisterCacheLevelCallback( int filter, CacheChangeCallback delegate); Cache.RegisterRegionLevelCallback( String region, int filter, CacheChangeCallback delegate); Cache.RegisterKeyLevelCallback( String region, String key, int filter, CacheChangeCallback delegate); public delegate void CacheChangeCallback( String CacheName, String regionName, String key, CacheItemVersion version, CacheEventType event, NotificationDescriptor nd );

53. Application Cache2 Cache1 Primary for K2,V2 K2, V2 Primary for K1,V1 K1, V1 Cache3 Primary for K3,V3 K3, V3 Velocity client1 Routing table Register notification for key "a", "b" Map keys to partition ranges Poll required nodes Nodes return list of changes LSN order Nodes return list of changes LSN order Partition: P1 Last LSN: 20 Partition: P1 Last LSN: 20 Partition: P2 Last LSN: 21 Partition: P2 Last LSN: 21 Call Delegate Store Last LSN Call Delegate Store Last LSN Change Log 1Add K2 2Del K32 3Del K43 Change Log 1Add K2 2Del K32 3Del K43 Change Log 21 Add K3 22 Del K4 23 Upd K9 Change Log 21 Add K3 22 Del K4 23 Upd K9 Change Log 33 Add K1 34 Del K22 35 Upd K93 Change Log 33 Add K1 34 Del K22 35 Upd K93 Change Log 1Add K2 2Del K32 3Del K43 Change Log 1Add K2 2Del K32 3Del K43 Change Log 1Add K2 2Del K32 3Del K43 Change Log 1Add K2 2Del K32 3Del K43 Change Log 33 Add K1 34 Del K22 35 Upd K93 Change Log 33 Add K1 34 Del K22 35 Upd K93

62. … Application / web tier Application UsersUsers Application Velocity access layer Velocity service Cache tier Storage/Microsoft SDS

63. … ApplicationrolesApplicationroles UsersUsers Application Velocity access layer Velocity service Velocity roles Storage/Microsoft SDS

67. Cache2 Cache1 Primary regions Toy1, 500 Cache3 Primary regions Toy2, 350 Toy3, 400 Cache API Local cache Velocity client Dispatch manager Federated query processor Object manager In-memory Data Manager Query processor Object manager In-memory Data Manager Query processor Object manager In-memory Data Manager Query processor from toy in catalog () where toy.ToyPrice > 300 select toy; ToyRegion Toy4, 100 from toy in catalog () where toy.ToyPrice > 300 select toy;

69. Scratch Velocity Intermediate Store Velocity Intermediate Store Rollup Operation Rollup Operation Final Results Final Results Keys Split Method Split Method Market Data Central Market Data Store (~1 TB Tick Data) Central Market Data Store (~1 TB Tick Data) Final Results Store Job Input Job Input Scratch Calculation Operation Calculation Operation Velocity Data Cache Market Data

70. Scratch Velocity Intermediate Store Velocity Intermediate Store Rollup Operation Rollup Operation Final Results Final Results Final Results Store Keys Split Method Split Method Central Market Data Store (~1 TB Tick Data) Central Market Data Store (~1 TB Tick Data) Job Input Job Input Scratch Calculation Operation Velocity Node Calculation Operation Velocity Node Market Data

73. Web Scenarios Distributed/Global object cache Low latency access High scale Availability Cache for reference and activity data Scale IIS/ASP.Net applications Enterprise/ HPC Scenarios LINQ enabled cache Co-locate computation and data Integrate with HPC server Persistence Heterogeneous client support Software + Services Scenarios Application Cache for cloud Storage In tegration with SSDS, Windows Azure More data services BI, St reaming, Reporting REST and SOA access

76. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.