Load Balancing in Distributed Systems Nalini Venkatasubramanian nalini@ics.uci.edu

Global Distributed Systems and Multimedia Motivation A given computer is overloaded Must decrease load and maintain certain characteristics Scalability Performance Throughput Ideally, this should be done transparently Solution: Load Balancing Global Distributed Systems and Multimedia

Introduction to Load Balancing Distributed resource allocation Can be thought of as “distributed scheduling” Deals with distribution of processes among processors connected by a network Handles issues such as deciding which process should be handled by a given processor Can also be influenced by “distributed placement” Important in data intensive environments and applications Data placement may force process placement Global Distributed Systems and Multimedia

Load Balancing Relationships Load Balancer Manages resources Resource assignment depends on policy or policies in effect Client Requests resources Requests services Global Distributed Systems and Multimedia

Load Balancing Overhead Satisfy client resource access without imposing large amounts of overhead Performance How well resources are managed Efficiency Cost of accessing and using a resource obtained through a load balancer Global Distributed Systems and Multimedia

Global Distributed Systems and Multimedia Load Balancing Issues When to migrate processes or forward requests Which processor should be chosen to handle a given process or request Should processes be moved off a computer How should searching for lightly loaded computer be performed Global Distributed Systems and Multimedia

Load Balancing Issues (cont.) When should load balancing decisions be made What should be taken into account when making the above decisions How should old data be handled Should load balancing data be stored centrally, or in a distributed manner Global Distributed Systems and Multimedia

Load Balancing Issues (cont.) Should computers make decisions together What is the performance/overhead tradeoff incurred by load balancing Prevention of overloading a lightly loaded computer Global Distributed Systems and Multimedia

Load Balancing Techniques Basically two ways to perform load balancing Statically Resource is allocated once Dynamically Resource is allocated and managed (possibly dynamically reallocated) to ensure balanced load Global Distributed Systems and Multimedia

Global Distributed Systems and Multimedia Static Load Balancing Resource allocation is performed once Once resource is allocated it remains allocated (for what duration??) Scheduling decisions are made Deterministically Probabilistically Global Distributed Systems and Multimedia

Static Load Balancing – cont’d Advantages State generally need not be stored Simplifies implementation Less network traffic due to load balancing related messages Disadvantages Poor resource utilization A given resource may be used much more than others Does not adjust to fluctuations in the load Possible for resource to become overloaded Global Distributed Systems and Multimedia

Static Load Balancing – cont’d Example of Static Load Balancing Forwarding processes/requests to a given computer based on dynamically assigned addresses (e.g. via DNS) Web servers (e.g. CNN) UCI host ea.uci.edu is load balanced Global Distributed Systems and Multimedia

Dynamic Load Balancing Attempts to maintain a balanced load by managing resources while a resource is in use May involve the following Process migration Disabling further access to a resource until a later time Adding new resources “on-the-fly” Global Distributed Systems and Multimedia

Dynamic Load Balancing Strategies Distributed versus non-distributed Should load information be stored centrally or across several hosts Simplicity versus overhead and reliability Cooperative versus non-cooperative Should decisions be made by a single load balancer or several Globally managed resources versus locally managed resources Global Distributed Systems and Multimedia

Dynamic Load Balancing Strategies - cont’d Adaptive versus non-adaptive Should previous data effect scheduling decisions Preemptive versus non-preemptive Should a running process be preempted in favor of another process, or for migration to another resource Global Distributed Systems and Multimedia

Global Distributed Systems and Multimedia Case Studies Load Balancing for Web Servers Load Balancing for Parallel Computers Load Balancing for Multimedia Applications Global Distributed Systems and Multimedia

Multimedia Applications Electronic Commerce Video Servers                                                  Global Entertainment Network Web Servers Distance Learning Graphics Processing Tele-medicine Global Distributed Systems and Multimedia Requirements - Availability, Reliability, Quality-of-Service, Cost-effectiveness, Security

Multimedia Load Management Primary focus resource optimization across streams and resource management across servers. Quality of Service continuous delivery requirement minor violations of performance requirements Admission Control - resource reservation/negotiation Media Delivery - Resource scheduling (CPU,Disk) Resource Mgmt. Implies Admission Control Caching, VCR Control, Server Selection, Data Placement Global Distributed Systems and Multimedia

Distributed MM Servers Video Server Topology Partitioned Server Externally Switched Fully Switched Video File Placement Heterogeneous Workload - large/small, hot/cold Online Placement Good placement is important dynamic replication time-consuming, dynamic load-bal complementary Global Distributed Systems and Multimedia

Dynamic Load Balancing Adapts to statistical fluctuations and changing access patterns Dynamic Migration Deals with poor initial placement Replication Dynamic Segment Replication partial replication (quick response, less expensive) Total Replication on-demand vs. predictive Global Distributed Systems and Multimedia

Load Management of Distributed MM Servers Adaptive Scheduling Assigns requests to servers based on demand and load factors. Invokes replication-on-demand, request migration Predictive Placement Invokes dereplication Optimizations Eager Replication Lazy Dereplication Global Distributed Systems and Multimedia

A Scalable Video Server Architecture Distribution Network requests data Distribution Controller Data Source Data Source Data Source Tertiery Storage ... control Global Distributed Systems and Multimedia

Architectural View of a Networked MM System Qos Broker and Load Management System Node Manager Node Manager Node Manager Node Manager ... Local Data Streaming Global Distributed Systems and Multimedia

Resources in a Video Server Client Client Network Processing Module Communication Modules Data Manipulation Modules Storage Modules Global Distributed Systems and Multimedia

Load Placement Scenario Data Source S2 Data Source S1 Storage: 8 objects Bandwidth: 3 requests Storage: 2 objects Bandwidth: 8 requests Access Network ... Clients Global Distributed Systems and Multimedia

Characterizing Server Resource Usage Ability to service a request on a server depends on: resource available characteristics of a request Load factor(LF) for a request: represents how far a server is from request admission threshold. LF (Ri, Sj) = max (Dbi/DBj , Mi/Mj , CPUi/CPUj , Xi/Xj) Global Distributed Systems and Multimedia

Global Distributed Systems and Multimedia Adaptive Scheduling When the broker receives a request Ri for a video object Vi : Consider only data sources that have a copy of Vi. Consider only data sources tha have sufficient resources to support Ri. Chooser server for which LF (Ri, Sj) is a minimum. If no such server exists Reject request. Perform replication-on-demand. Perform request migration. Global Distributed Systems and Multimedia

Predictive Data Placement Determines when, where and how many replicas of a video object. Initiated periodically. Results in an assignment of replicas to data sources. Greedy algorithm that uses revenue generated as a metric. Global Distributed Systems and Multimedia

The Greedy Cost Placement Matrix PM(Vi, Sj) is the maximum revenue that can accrue from allocating Vi to Sj. Greedy heuristic: Map(Vi,Sj) = 1 if PM(Vi,Sj) = a b max(PM(Va,Sb)) Global Distributed Systems and Multimedia

Global Distributed Systems and Multimedia Optimizations To minimize the overhead of replication Eager replication Replication of video object in anticipation Performed when server resources are free Lazy Dereplication Critical nature of storage resources Mark reusable resources, reclaim disk space later If disk blocks are not overwritten, can be reclaimed Global Distributed Systems and Multimedia

Global Distributed Systems and Multimedia Life of a video object Global Distributed Systems and Multimedia

Performance Evaluation Policies Global Distributed Systems and Multimedia

Performance Evaluation - Startup Latencies Global Distributed Systems and Multimedia

Performance of the basic configuration Global Distributed Systems and Multimedia

Performance Evaluation - Varying Replication BW Global Distributed Systems and Multimedia

Performance Evaluation Summary P1: entails high startup latency, requires high storage and replication bandwidth. P2: Unacceptably poor performance. P3: Similar performance to P4 in many cases. At low transfer bandwidths, P4 outperforms P3. P4: Performs well in all cases. Global Distributed Systems and Multimedia