1. Transparent Adaptive Resource Management for Middleware Systems
Jaiganesh Balasubramanian
Ossama Othman
Dr. Douglas C. Schmidt
{jai, ossama,
Department of Electrical Engineering and Computer Science
Vanderbilt University, Nashville

2. Distributed Systems Typical issues with distributed systems
Heterogeneous environments
Concurrency
Large bursts of client requests
24/7 availability
Stringent QoS requirements
Examples of distributed systems
E-commerce
Online trading systems
Mission critical systems
Vanderbilt University

3. Motivation
Development and maintenance of QoS-enabled distributed systems
Non-trivial
Requires expertise that application developers often lack
Solution: Middleware (e.g. CORBA)
Can shield distributed system developers from the complexities involved with developing distributed applications
Can facilitate manipulation of QoS requirements and management of resources
Vanderbilt University

4. Load Balancing
Load balancing can improve the efficiency and scalability of a distributed system
Load balancing service can be implemented in the following layers
Network layer
OS layer
Middleware layer
Why choose middleware-layer load balancing?
Allows additional components to be added to distributed systems with minimal impact to performance
Improves availability due to inherent redundancy
Can be composed with other services (e.g. fault tolerance, security, etc)
Can take into account request content
Vanderbilt University

5. Distributed System Resource Management
Managing resources transparently is important for legacy DRE systems
“Transparency” is hard
“Transparency is essential
Middleware in use may need to be enhanced to support this functionality
Such an enhancement can be found in a middleware-based load balancing service
Vanderbilt University

6. Common Scenario Multiple clients making request invocations
Potentially non-deterministic
Members
Multiple instances of the same object implementation
Object groups
Collections of members among which loads will be distributed equitably
Logically a single object
Load balancer
Transparently distributes requests to members within an object group
Vanderbilt University

7. TAO Load Balancer (Cygnus)
Cygnus makes it easier to develop distributed applications by providing
application transparency,
scalability,
flexibility,
adaptability and
interoperability.
Has the following built-in strategies:
- RoundRobin
- Random
- LeastLoaded
EVENT SERVICE
TRANSACTIONS
SECURITY
AV STREAMS
DYNAMIC/STATIC
SCHEDULING
LOAD
BALANCING
RT-CORBA 1.0
Vanderbilt University

8. Cygnus Load Balancing Strategies
Client binding granularity
Per-session, client permanently forwarded to a replica
Per-request, requests forwarded on client’s behalf
On-demand, client can be rebound to another replica when necessary
Vanderbilt University

9. Cygnus Load Balancing Architectures
Load balancing architecture comprised of a combination of client binding granularity and balancing policy
Given the strategies just described, there are six possible architectures
Three common architectures
Non-adaptive per-session
Adaptive per-request
Adaptive on-demand
Vanderbilt University

10. Cygnus Load Balancer Components
Load Monitor
Provides load feedback
Load Analyzer
Determines location and member load conditions
Member Locator
Binds client to appropriate object group member
Load Alert
Facilitates load control (load shedding)
Load Manager
Mediates all interactions between load balancer components
Vanderbilt University

11. Cygnus Load Monitor Facilitates feedback Monitor and report loads
Load monitoring is location-oriented
The loads at a given location are monitored and reported
Contrast with loads on a given object group member
Vanderbilt University

12. Cygnus Load Analyzer Load Analyzer
Decides which member will receive the next client request
Determines load condition at each location
Induces load shedding
Extensible load balancing strategies
Set via the PropertyManager interface
Each object group may use a different strategy
Vanderbilt University

13. Cygnus Member Locator Implements the Interceptor design pattern
Transparently forwards client requests to member retrieved from the load analyzer
In CORBA, redirection induced via standard GIOP LOCATION_FORWARD message
Conforming client side ORB will transparently re-issue request to member chosen by load balancer
Vanderbilt University

14. Cygnus Load Alert Facilitates “control” aspect of load balancing
Load shedding
Only used for adaptive load balancing strategies
Forwards client requests back to load balancer
Vanderbilt University

15. Cygnus Load Manager
Mediates interactions between all load balancer components
Manages object groups
Manages load monitor and load alert location maps
Acts as a specialized event channel
Load events are published by load monitors
Load events are consumed by load analyzers
Vanderbilt University

16. Adaptive Load Balancing Interactions
Vanderbilt University

17. Load Balancing Strategies (1/2)
Adaptive Load Balancing Strategies
Use run-time system information, example host load information to make load balancing decisions.
Adaptive strategies integrated in Cygnus are
LeastLoaded, which allows locations to receive loads until a threshold value is reached. Once a threshold is reached , subsequent requests are transferred to the location with the lowest load.
LoadMinimum, which calculates the average load at all locations and if the load at a particular location is greater than the average load and greater than the least loaded location by a certain threshold, then the load at the particular location is transferred to the least loaded location.
Vanderbilt University

18. Load Balancing Strategies (2/2)
Non-adaptive load balancing strategies
Makes load balancing decisions based on some static information.
Do not use run-time system state information while making load balancing decisions.
Non-adaptive load balancing strategies integrated into Cygnus are RoundRobin and Random.
RoundRobin strategy keeps a list of locations and iterates through that list to select the next location that will receive requests from the client session.
Random strategy keeps a list of locations and selects a random member to receive the requests from the next client session.
Vanderbilt University

19. Load Metrics Cygnus supports the following load metrics
Request-per-second, which calculates the average number of requests arriving per second at each server.
CPU run queue length, which returns the load as the average number of processes waiting in the OS run time queue over a configurable time period in seconds, normalized over the number of processors.
CPU utilization, which returns the load as the CPU usage percentage.
Vanderbilt University

20. Evaluating load balancing strategies
Load balancing strategies can be of the type adaptive or non-adaptive
Load balancing strategies can be employed various run time parameters and load metrics.
Determining the appropriate load balancing strategies for different classes of distributed applications is hard without the guidance of comprehensive performance evaluation models, systematic benchmarks and empirical results.
Vanderbilt University

21. Introducing LBPerf
LBPerf is an open source benchmarking tool suite for evaluating middleware load balancing strategies.
Extensible C++ middleware framework.
Supports range of adaptive and non-adaptive load balancing strategies.
Tune different configurations of middleware load balancing , including choosing different load balancing strategies and run-time parameters associated with those strategies.
Evaluate strategies using different metrics like throughput, latency, CPU utilization.
Vanderbilt University

22. Workload Characterization (1/2)
Empirical evaluations not useful, unless the workload is representative of the context in which the load balancer is deployed.
Workload model could be of the type:
Closed analytical network models
Simulation models
Executable models
Our benchmarking experiments are based on the executable workload models.
Vanderbilt University

23. Workload Characterization (2/2)
Executable models can be classified as :
Resource type, where workloads are characterized by the type of resource being consumed.
Service type, where the workloads can be characterized by the type of service being performed on behalf of the clients.
Session type, where the workloads are characterized by the type of requests initiated by a client and serviced by a server in the context of a session.
Our benchmarking experiments focused on generating session type workloads.
Vanderbilt University

24. Experiment
Single threaded clients generating CPU intensive requests on the servers
Experiments repeated for different strategies including RoundRobin, Random, LoadMinimum and LeastLoaded.
Measure throughput, which is the number of requests processed per second by the server and the CPU utilization, which is the CPU usage percentage.
Vanderbilt University

25. Run-time configurations
Load reporting interval, which is the time interval between successive load information updates from the server to the load balancer.
Reject threshold value of the LeastLoaded strategy, which is the load at which the servers will stop receiving extra loads.
Critical threshold value of the LeastLoaded strategy, which is the load at which the servers will start shedding loads.
Migration threshold value of the LoadMinimum strategy, which is the difference between the most loaded machine and the least loaded machine to trigger a migration of load from the most loaded machine to the least loaded machine.
Dampening value, which is the fraction of the newly reported load that will be considered for fresh load balancing decisions.
Vanderbilt University

26. Vanderbilt University

27. Vanderbilt University

28. Vanderbilt University

29. Vanderbilt University

30. Vanderbilt University

31. Results Analysis Our results indicate that:
Adaptive load balancing strategies perform better than non-adaptive load balancing strategies in the presence of non-uniform loads.
LeastLoaded strategy is better than the other three strategies under such loading conditions
Reducing the number of client migrations is necessary for achieving maximum performance. In other words, client session migrations are not always effective.
Need to maintain system utilization to enable more predictive behavior of the strategies.
Vanderbilt University

32. Future Work Extend LBPerf to evaluate other types of workloads.
Use the empirical results as a learning process to understand the nuances of the different run-time behaviors of these adaptive load balancing strategies.
Use observations from the learning process to train an operational phase by developing self-adaptive load balancing strategies that can dynamically tune the run-time parameters according to the load being experienced.
Define stateful load balancing model
Decentralized load balancing
Reduced network overhead
Improved reliability
Integrating load balancer service into the CCM model.
Vanderbilt University

33. Concluding Remarks Cygnus load balancing architecture and model is
Flexible
Extensible load balancing strategies
Allows both non-adaptive and adaptive (including self-adaptive) strategies to be used
Freedom to implement in a variety of ways
Centralized
Decentralized / Federated / Cooperative
Hierarchical
Generic
Supports multiple object groups
Not application-specific
Familiar
Uses many of the same group management concepts found in existing fault tolerance models
Vanderbilt University

34. References
Ossama Othman, Jaiganesh Balasubramanian, Douglas C. Schmidt
“ The Design of an Adaptive Load Balancing and Monitoring Service”
Jaiganesh Balasubramanian, Ossama Othman, Douglas C. Schmidt
“Evaluating the performance of middleware load balancing strategies”
Download Cygnus, LBPerf and TAO from
Vanderbilt University