2. 2 Microsoft project code named for Scalable and Available Applications Anil Nori, Distinguished Engineer Muralidhar Krishnaprasad (“MK”), Principal Architect Microsoft Session Code: DAT202 “Velocity”

3. 3 Outline Need for Distributed Caches Usage Scenarios Project “Velocity” - Introduction and Vision Key Concepts APIs Cache Configurations Architecture Release Schedule Demos

4. 4 Rapidly dropping memory prices, enabling huge amounts of data to reside in-memory Availability of cheap commodity hardware that can be easily replaced Multi-core CPUs, cheap flash memory Rapidly dropping memory prices, enabling huge amounts of data to reside in-memory Availability of cheap commodity hardware that can be easily replaced Multi-core CPUs, cheap flash memory Most composite applications designed for serious scale and availability are using mid-tier, distributed caching technologies today. Data is sourced from all over the place: local, federated, cloud An abundance of applications working together, each with different shapes for the same data More workloads require increasing scale, availability, and performance as they become mission critical Applications must be cheap to scale Data is sourced from all over the place: local, federated, cloud An abundance of applications working together, each with different shapes for the same data More workloads require increasing scale, availability, and performance as they become mission critical Applications must be cheap to scale Evolving requirements Logic and data distributed, with data living closer to mid-tier logic Global shared data structures enable application productivity and acceleration Recognition of different types of data with different semantics Deployment of high-speed data storage where feature set is not “one size fits all” Logic and data distributed, with data living closer to mid-tier logic Global shared data structures enable application productivity and acceleration Recognition of different types of data with different semantics Deployment of high-speed data storage where feature set is not “one size fits all” Evolving application architectures being solved by Underlying trends enabled by

5. 5 Distributed Cache Usage Commonalities: Extreme transaction volume, store for transient processing Horizontal Verticals Scenario Web User-specific HTTP session and shared state across web farm In-flight shopping carts for web retail Enabling online self-service applications Explicit storage of pre-computed or highly-accessed data LOB Enterprise-wide product catalog for POS, analytics Caching frequently used reference data for a ERP application Telco Cellular/VOIP: compute utilization, prepay charges, call routing and session info SMS: message content / notification / receipt, billing Travel Aggregated flight pricing / availability retrieved from airlines Defense Sensor network data processing and threat detection Financial Per-user portfolio data and delayed quote storage for trading Aggregate and process ticker stream for algorithmic trading

6. 6 A version of the authoritative data Aggregated or transformed Each version is unique Refreshed periodically Examples Web and Enterprise (Product) Catalogs User, Employee data Access pattern Mostly read Shared & Concurrent Access Scale Large number of accesses Functionality Key based Access Query & Filtering Loading Caching Reference Data Web Tier Clients Local Cache (in Proc) Distributed Cache Servers Data Tier Usernames, Name-> ID Mapping Usernames, Name-> ID Mapping Friend Lists Usernames Friend Lists Usernames Scenario: Social Networking

7. 7 Data typically generated as part of the application activity Active during business transactions Typically logged to a backend data source Historical data Examples Shopping Cart Session State Enterprise LOB app (Purchase Order) Access pattern Read and write Primarily exclusive access Scale High data (and access) scale Functionality Key based access Transactions Caching Activity-oriented Data Data Tier Mid Tier Rich Clients Distributed Cache Integration Hub Integration Hub Thin Clients Web Tier Vendor services, Pricing Vendor Sources Order, Invoice, Payment External Systems… Aggregated Vendor Catalogs Aggregated Vendor Catalogs Scenario: Enterprise LOB Application

8. 8 Caching Resource-oriented Data Authoritative data Modified by transactions; spans transactions Examples Flight Inventory Access pattern: Read and write Shared access Functionality Key based access Transactions Scale Large number of concurrent accesses Relaxed consistency for scale Scenario: Flight Inventory and Pricing App Logic Continent al America n United Distributed Cache Flight Routing Itinerary Flight Routing Itinerary Flight Segment Flight Price Flight Segment Flight Price Airlines Inventory Booking Service

9. 9 Application Cache Requirements Performance Millisecond access 10s of 1000s of accesses Scale 10s – 100s of nodes in enterprise 100s – 1000s in web applications Functionality Key based and Query based access Transactions, Optimistic concurrency control Invalidations Availability Certain enterprise and e-commerce data (e.g. shopping cart, inventory) Consistency Relaxed consistency for scale and performance

10. 10 Unified Cache View What is Velocity? An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.) Fuse "memory" across machines into a unified cache Clients can be spread across machines or processes Clients Access the Cache as if it was a large single cache Cache Layer distributes data across the various cache nodes

11. 11 Where does Velocity fit? App (Mid) Tier IIS apps, ASP.NET,.NET Apps, PHP Apps etc. Cache Tier Velocity Data Tier SQL Server, SSDS, etc. Client Desktop, Mobile, etc. Web Client AJAX, Silverlight etc.

12. 12

13. 13 Host Physical processes hosting Velocity instance. Named Caches Can span across machines Defined in the configuration file Regions Physically co-located Container of Cache Items May be implicit or explicitly created Cache Item Key, Payload (Object ), Tags, TTL, Timestamps, Version Velocity Logical Hierarchy Machine 2 Machine 1 Cache Host A Cache Host B Cache Host C Machine 3 Cache Host D Named Cache : Product Catalog Named Cache : Electronics Inventory Regions Region A Key Payload Tags 121 xxxx “Toy” “Child” 123 yyyy “Toy” “Chair”.. Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects

14. 14 Velocity – Example Usage // Create instance of cachefactory (reads appconfig) CacheFactory fac = new CacheFactory(); // Get a named cache from the factory Cache catalog = fac.GetCache("catalogcache"); // ------------------------------------------------------ // Region based Get/Put catalog.CreateRegion("toyRegion"); // Both toy and toyparts are put in the same region catalog.Put("toyRegion", "toy-101", new Toy(.,.)); Catalog.Put("toyRegion", "toypart-100", new ToyParts(…)); Toy toyObj = (Toy)catalog.Get("toyRegion", "toy-101"); // ------------------------------------------------------ // Region based Get/Put catalog.CreateRegion("toyRegion"); // Both toy and toyparts are put in the same region catalog.Put("toyRegion", "toy-101", new Toy(.,.)); Catalog.Put("toyRegion", "toypart-100", new ToyParts(…)); Toy toyObj = (Toy)catalog.Get("toyRegion", "toy-101"); //------------------------------------------------------- // Simple Get/Put catalog.Put("toy-101", new Toy("thomas",.,.)); // From the same or a different client Toy toyObj = (Toy)catalog.Get("toy-101"); //------------------------------------------------------- // Simple Get/Put catalog.Put("toy-101", new Toy("thomas",.,.)); // From the same or a different client Toy toyObj = (Toy)catalog.Get("toy-101");

15. 15 Velocity Access APIs Simple Access Get – Returns object or entire Cache item Add - Creates new entry else exception if entry exists Put - Replaces existing entry or creates a new one Remove - Removes existing entry Tag Based Access Search limited to a specific region Search on "all" or "any" equality match of tags Enumeration Enumerate entire region or result from tag search

16. 16 Velocity Client Demo

17. 17 Types of Caches Distributed Configurations Partitioned Cache Data Partitioned across all nodes in the named cache Used for Scale & Availability Replicated Cache Data replicated across all nodes of the named cache Used for scaling (read) requests Local Cache Cache in the application process space for frequently accessed items Payload kept in object form

18. 18 Velocity Deployment Velocity Cache Service Cluster Service Application Velocity Client Local Cache Caching Tier Application Tier

19. 19 Velocity Client2 Velocity Client1 Partitioned Cache Put (K2, V2) on Client1 Routing Layer routes to Cache2 (primary) Cache2 performs local operations & returns back to client Get (K2) on Client2 Client2 routes to Cache2 for K2 & gets value from there Put(K2, v2) Get(K2) Routing layer Cache2 Cache1 Primary Regions K2, V2 Primary Regions K1, V1 Cache3 Primary Regions K3, V3

20. 20 Velocity Client Local Cache Velocity Client Local Cache Partitioned Cache fronted by Local Cache Local Cache can help speed up access on clients Put(K2, v2) Get(K2) Routing layer Cache2 Cache1 Primary Regions K2, V2 Primary Regions K1, V1 Cache3 Primary Regions K3, V3 K2, V2

21. 21 Achieving Scale and Performance Scale on Data Size More machines => More memory to cache Scale on Cache Throughput Cache Throughput: Operations/sec from entire cache More machines => keys distributed across more machines => better throughput Performance High Performance by scaling out data and processing Increased memory Increased processing Workload distributed across multiple cache nodes

22. 22 Velocity Scale Demo

23. 23 Basic Layout Load Test Controller … Server 1 Server 2 Server 3 Cache Client1 Cache Client 2 Cache Client n Load Controller Spawns Test Clients Client Machines run test Cache Service Cluster Partitioned Cache

24. 24 Demo Output Single Server Throughput Increases with Increasing Load Until Server Saturation Single Server Throughput Increases with Increasing Load Until Server Saturation LoadLoad ThroughputThroughput LatencyLatency Server 2 Added Throughput Increases Latency Decreases Until Server Saturation Server 2 Added Throughput Increases Latency Decreases Until Server Saturation Server 3 Added Throughput Increases Latency Decreases Server 3 Added Throughput Increases Latency Decreases

25. 25

26. 26 Velocity Client Cache2 Secondary Regions Cache1 Secondary Regions Cache3 Secondary Regions Primary Regions Availability Put(K2, v2) K2, V2 Get(K2) Primary Regions K3, V3K1, V1 K2, V2 K3, V3 K1, V1 Routing layer Put (K2, V2) on Cache1 Cache2 replicates changes to secondaries Cache1 and Cache3 Waits for acks from Cache1 and Cache2; returns back to client Get (K2) on Cache3 Cache3 routes to Cache2 primary for K2 & gets value from there

27. 27 Cache2 Cache1 Cache3 Replicated Regions Replicated Cache Put (K2, V2) on Cache1 Cache2 performs local operation & Acks back to Cache1 Cache2 Asynchronously replicates changes to all machines Get on cache3 Cache 3 reads from local store and returns the result Put(K2, v2) K2, V2 Get(K2) Replicated Regions K3, V3K1, V1 K3, V3 K1, V1 K2, V2 Routing layer

28. 28 Velocity Availability Demo

29. 29 Availability Demo Cache Cluster Velocity Embedded in ASP.Net Application Cache Cluster Velocity Embedded in ASP.Net Application Northwind Application Northwind Application ASP.net Session Cache ASP.net Session Cache VELOCITY Velocity Session Store Provider Velocity Session Store Provider Northwind Application Northwind Application ASP.net Session Cache ASP.net Session Cache VELOCITY Velocity Session Store Provider Velocity Session Store Provider CacheViewer Application CacheViewer Application VELOCITY Partitioned Cache

30. 30

31. 31 Velocity Service Architecture … Server 1 Server 2 Server 3 Application / Web Tier Application Cache Tier Configuration Store (Can be database, File share etc.) Stores Global Cache Policies Stores Current Partitioning Information Configuration Manager Velocity Service Replication Substrate Communication Substrate (Cluster Ring Management) Communication Substrate (Cluster Ring Management) One of the Velocity Service Hosts the Configuration Manager Velocity Client UsersUsers Data manage, Object Manager

32. 32 Velocity V1 Features Usage Key Based Caching with simple Get/Put Access and Tag based lookup Optimistic Version Based and Pessimistic Lock Based Updates Notification Support Highly Available Cache with configurable number of secondaries Enterprise and Web Scale 10s – 100s of machines Integration with distributed management and monitoring tools Consistency Models Strong/Synchronous – Synchronous communication with secondaries Weak/Asynchronous – Async communication for Higher performance Topologies Embedded configuration – ASP.Net Cache Service Cache Types Partitioned – For Activity & Resource Data Replicated – For Reference Data Local Cache – Client cache for speeding up access Hosting Integration with ASP.Net for session caching Hostable on IIS, Biztalk etc.

33. 33 ASP.Net Session Integration SessionStoreProvider class Plugs in to the ASP.Net Session store Stores session state in “Velocity” Scale Session information accessible from all ASP.NET nodes Enables non-sticky routing Dynamically scale by adding new nodes High Availability Session data is backed up on to additional machines Resilient to machine or process failures

34. 34 Administration & Monitoring Logging Provider model; Default ETW, support for file logs Administration Admin tool Command Line Tool Admin operations Start and stop cluster or service, configure named caches.. Integrate with Management Tools (e.g. IIS, Microsoft System Center) Monitoring tool Monitor Cache statistics Perfmon integration

35. 35 Velocity Vision In-memory DM + QP Componentized, embeddable Query and Search In-memory DM + QP Componentized, embeddable Query and Search Consistent Cache TransactionsInvalidationsSynchronization TransactionsInvalidationsSynchronization Distributed Cache Distribute Hash Table Scale-outFailover Distributed Cache Distribute Hash Table Scale-outFailover Application Cache CLR objects, rows, etc. ID-based access: Get/Put Distribute Hash Table Application Cache CLR objects, rows, etc. ID-based access: Get/Put Distribute Hash Table Global Queryable, Transactional Cache Scalable Application Cache In-Memory Cache In-Memory Data Manager Distributed Data Manager Distributed Data Manager, Query Processor Enterprise Application Cache Available Application Cache Search and LINQ queries across the distributed cache

36. 36

37. 37 Schedule June 2008 (CTP1) (TechEd 2008) Oct 2008 (CTP2) (PDC 2008) RTM Distributed, partitioned cache serviceRegions, Tags Local CacheScale, Performance ASP.Net IntegrationMonitoring Tools AvailabilityReplicated cache Administration ToolsInvalidation & Events Multiple hosting environments H1 CY 2009

38. 38

39. Microsoft.com Velocity site http://www.microsoft.com/sql/velocity.aspx MSDN Data Platform Development Center http://msdn.microsoft.com/data MSDN Code Gallery http://code.msdn.microsoft.com/velocity Forums – Access from http://msdn.microsoft.com/data Track Resources

40. www.microsoft.com/teched Tech·Talks Tech·Ed Bloggers Live SimulcastsVirtual Labs http://microsoft.com/msdn Developer’s Kit, Licenses, and MORE! Resources for Developers

41. Complete an evaluation on CommNet and enter to win! 1 Year Subscription!

42. 42 © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.