1. Internet Service Migration and Placement Part 1 Instructor: Xiaodong Zhang Xiaoning Ding 11/08/2004

2. Outline  Background  OPUS: An Overlay Peer Utility Service Overview Architecture Research issues  Model-based resource provisioning Overview Web service model Model-based resource allocator

3. Outsourcing Services & Utility-based services Outsourcing services  Customer-owned or leased system.  The service provider takes responsibility for managing the customer ’ s IT and network system – the computing infrastructure – based on customer-defined service level agreements (SLA).  Billed on a monthly or fixed-fee basis. Utility-based services  The service provider owns the infrastructure  leases the infrastructure to the customers  pay for what you use  Example: Internet data center enabling ASPs to deliver ASP services

4. Utility & SLA  Utilities deliver IT resources (CPU, storage, and bandwidth) to hosted application and, ultimately, end users  much as the electric utility transparently delivers power on demand to customers.  Applications agree to Service Level Agreements (SLAs) with the utility

5.  Dedicate fixed resources per application  Reprovision manually as needed  Overprovision for surges High variable cost of capacity Static Provisioning

6. Load Is Dynamic World Cup soccer site May-June 1998 Seasonal fluctuations Event surges (11x) ita.ee.lbl.gov M T W Th F S S M T W Th F S S Week 6 7 8 Week 6 7 8 ibm.com external site February 2001 Daily fluctuations Workday cycle Weekends off

7. Adaptive Provisioning offer economies of scale  Network access  Power and cooling  Administration and security  Surge capacity

8. Overlay network and Mobile code  Increasing number of important network services are deploying overlays CDN, Replicated services, Storage systems... Dynamically map data and functions onto network resources  Programs and data will adaptively migrate and replicate in response to changing network conditions, client access characteristics,... Programs dynamically run at optimal network locations Data dynamically flow to where it is required.

9. Outline  Background  OPUS: An Overlay Peer Utility Service Overview Architecture Research issues  Model-based resource provisioning Overview Web service model Model-based resource allocator

10. OPUS: An Overlay Utility Service App demand (per network region) Overlay node Peering Allocate nodes to services based on current demand

11. OPUS: Overview  targeting utilities consisting of a distributed set of thousands of server sites, each with potentially 1000's of individual machines, cooperating together to fulfill aggregate SLAs  Simultaneously hosts multiple distributed applications replicated web services application-layer multicast content distribution networks....

12. Opus tasks  Resource allocation Allocate resources among competing applications Maximize aggregate performance Based on changing application and network characteristics, SLAs  Replica placement Closely related to resource allocation Where to place individual application replicas Consider dynamically changing client access patterns, network failures, etc.

13. Opus tasks  Overlay topology construction create overlays that meet application requirements of performance, delay, and reliability minimize consumed network resources  Request routing discover the service replica capable of delivering the highest quality of service

14. OPUS: Architecture

15. The service overlay  Each Opus site runs an instance of site manager coordinating resource usage at that site and exchange status summaries with other opus sites.  Interconnects all active nodes and provides overlay services  “ Backbone ” for coordinated, decentralized resource allocation and resource control

16. The service overlay  Assist the construction and maintenance of application overlay  Dynamic and self-healing  Scalability issue Hierarchical data dissemination in dicast Think globally but act locally

17. Adaptive per-application overlay  Each application uses its application overlay to Route internal application traffic Disseminate content Synchronize state information …  The topology and site allotments are subject to change by resource allocator

18. Security and isolation  Allocating resources to applications at the granularity of individual nodes  Future plan: using virtual machine  Using VLAN to isolate traffic on the wire

19. Research Issues  Overlay topology construction  Resource allocation  Scalable tracking of system characteristics  Reliability QoS guarantees

20. Overlay topology construction  Emphasize scalability Quantify the benefits of competing structures Develop scalable distributed constructing algorithms  Initial work A general overlay topology that enables dynamic tradeoffs between network performance/reliability and cost Focus on network cost and relative delay penalty (RDP) to characterize overlay topology Two candidate overlay topologies: K-spanner and LAST.

21. Overlay topology construction

22.  Distributed algorithms for building and maintaining the topology Selectively probing using probabilistic techniques and hierarchy Using partial, approximate and probabilistic knowledge of network infomation Having each node gradually migrate to its “ proper ” location in the overlay.

23. Resource allocation classical economic model  Customers are associated with utility functions specifying the value of the services result from a allotment. (concave functions)  Opus maximizes global value across all applications.  Optimal solution: the marginal value of an additional resource unit is in equilibrium across all customers.

24. Resource allocation Allocated Resources Throughput (Value) App2 App1 Gradient 2 Gradient 1

25. Resource allocation  Scalability consideration Adapt from economic resource allocation  Decentralized federation of autonomous local “ markets ” exchanging information to converge toward a global equilibrium Celluar structure  Cell: an entire Opus site or a portion of large site  Cells plan their internal allocation locally  Cells operate to trade load or resources

26. Tracking system characteristics  Nodes are partitioned into clusters of size d.  Each cluster elects an agent responsible for disseminating local cluster information  Agents from d adjacent clusters form second-level clusters  All nodes are organized into a tree called dicast tree. Height=log d N

27. Tracking system characteristics Hierarchical data dissemination in dicast

28. Tracking system characteristics  Data travels up the tree, and may be aggregated with data from the nodes  At each level of the tree, an overlay propagates the data among all participating cluster members  Updates are buffered awaiting the arrival of further updates until a threshold is reached, and updates are aggregated  Each node may has exact information of “ nearby ” nodes in the same cluster Aggregate information of remote cluster

29. Reliability QoS Guarantees Address network level failures  Restricted flooding Redundantly transmit the same data over multiple logical path  Minimizing the overhead Intermediate nodes re-evaluate the reliability of the remainder of the path, and choose between forwarding redundant data and suppressing duplicate data

30. Reliability QoS Guarantees 0.96 A B JD S 0.97 0.98 0.97 0.99 S  A  D: 0.96*0.98*0.99=0.931 S  A  D: 0.97*0.97*0.99=0.931 S  A and B  D: (1-(1-0.96*0.98)*(1-0.97*0.97)) *0.99=0.987

31. Reliability QoS Guarantees  To match the overlay topology with the failure characteristics of underlying network Construct overlays with disjoint paths to lower the failure correlation among logical overlay links Collect statistical information about loss correlation Use network topology information

32. Outline  Background  OPUS: An Overlay Peer Utility Service Overview Architecture Research issues  Model-based resource provisioning Overview Web service model Model-based resource allocator

33. Overview  Addresses the provisioning problem Multiple competing services hosted by a shared server cluster (utility) How much resource does a service need to meet SLA targets  Applications Static web content Heavily resource-intensive Predictable in average per-request resource demands

34. System architecture

35. The use of the model

36. Web service model

37. α Zipf locality parameter λ Offered load in requests/s S Average object size T Total number of objects M Memory size for object cache RpRp CPU response time H Object cache hit ratio λsλs Storage request load in IOPS RsRs Average storage response time

38. Web service model μs,φμs,φ Peak storage throughput in IOPS RpRp CPU response time H Object cache hit ratio λsλs Storage request load in IOPS RsRs Average storage response time R Average total response time

39. Model-based resource allocator  Periodically invoked by the utility OS executive to adjust the allotments  Focus on memory and storage resources, ignore CPU constraints  Output an allotment vector for each service CPU share,Memory and storage allotment [M, φ ]

40. Model-based resource allocator  Resource provisioning primitives Candidate plans initial candidate allotment vectors LocalAdjust modifies a candidate vector to adapt to local resource constraint or surplus GroupAdjust modifies a set of candidate vectors to adapt to a resource constrait or surplus

41. Model-based resource allocator Generating Initial Candidates Ρ target  R p Φ=μ s Rp, Φ, ρ target  R s (4) λ,H  λ s (2) Φ=λs / ρ target |Φ-Φ desired |<ε H  M (1)

42. References  Utility Computing White Paper: http://www.sun.com/service/utility/FINAL_UC_WP.pdf http://www.sun.com/service/utility/FINAL_UC_WP.pdf  Service Utilities: http://issg.cs.duke.edu/utilies.htmlhttp://issg.cs.duke.edu/utilies.html  D. G. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris, "Resilient Overlay Networks," in 18th ACM Symposium on Operating Systems Principles (SOSP), October 2001, pp. 131- 145.  "OPUS: Overlay Utility Service", Rebecca Braynard, Dejan Kostic, Adolfo Rodriguez, Jeff Chase and Amin Vahdat, poster at 18th ACM Symposium on Operating System Principles (SOSP), Banff, Canada, October 2001. ( poster)  R. Braynard, D. Kostic, A. Rodriguez, J. Chase, and A. Vahdat. Opus: an Overlay Peer Utility Service. IEEE OPENARCH 2002.  Ronald P. Doyle, et. al., ``Model-based resource provisioning in a Web service utility", Proceedings of the 4th USENIX Symposium on Internet Technology, 2003.