© 2009 IBM Corporation 1 Improving Consolidation of Virtual Machines with Risk-aware Bandwidth Oversubscription in Compute Clouds Amir Epstein Joint work with David Breitgand

© 2009 IBM Corporation 2 Motivation  Network Bandwidth is a critical Data Center resource  Network Bandwidth may become a bottleneck for consolidation  Accurate and efficient network bandwidth demand estimation is difficult  Common practice: fully provision for peak loads  Consequences: resource waste

© 2009 IBM Corporation 3 Full Provisioning VS. Multiplexing  The aggregate demand of VMs may be much smaller than the sum of the maximum demand of each VM: ∑ i max t d i (t) >> max t ∑ i d i (t) Max(VM1)+Max(VM2)=110

© 2009 IBM Corporation 4 Full Provisioning VS. Multiplexing Max(VM1+VM2)=71 < Max(VM1)+Max(VM2)=110

© 2009 IBM Corporation 5 Statistical Multiplexing  Consider each VM dynamic bandwidth demands as a random variable  Consider the aggregate bandwidth demand which is a sum of the random variables representing VMs Bandwidth demands  As the number of VMs increases: –The ratio between standard deviation of the aggregate bandwidth demand and the mean decreases

© 2009 IBM Corporation 6 Overcommit  Cloud provider aims at improving cost-efficiency  Overcommit resources using statistical multiplexing  Our focus is bandwidth

© 2009 IBM Corporation 7 Stochastic Bin Packing Problem (SBP)  S={X 1,…, X n } – Set of items  X i – random variable representing the size (bandwidth demand) of item i  p – overflow probability  Goal: Partition the set S into the smallest number of subsets (bins) S 1,…,S k such that p represents a probabilistic SLA / policy

© 2009 IBM Corporation 8 SBP with Normal Distribution  We assume that each item i independently follows normal distribution N( μ i, σ i 2 ).  When σ i,=0, for all i, then X i = μ i and the problem reduces to the classical bin packing problem  The focus of this work is SBP with normal variables

© 2009 IBM Corporation 9 Related Work – Bin Packing  The problem is NP-hard  Bin packing is hard to approximate to a factor better than 3/2 unless P=NP.  First Fit Decreasing (FFD) has asymptotic approximation ratio of 11/9 and (absolute) approximation ratio of 3/2.  MFFD algorithm has asymptotic approximation ratio of 71/60.  AFPTAS exists.  Online bin packing – First Fit (FF) has competitive ratio of 17/10. – Best upper and lower bounds are and , respectively.

© 2009 IBM Corporation 10 Related Work – Stochastic Bin Packing  -approximation for SBP with Bernoulli variables [Kleinberg et. al 1997]  SBP with Poisson, Exponential and Bernoulli variables [Goel and Indik 1999] – PTAS exists for Poisson and exponential distributions. – Quasi-PTAS exists for Bernoulli variables. – These results relax bin capacity and overflow probability constraints by a factor 1+ε.  - competitive algorithm for SBP with normal variables [Wang et. al 2011]

© 2009 IBM Corporation 11 Our Results  2-approximation algorithm for SBP with normal variables  (2+ε)-competitive algorithm for online SBP with normal variables  Observe the existence of a dual PTAS for SBP with normal variables.

© 2009 IBM Corporation 12 Definitions  Definition: The effective load of bin j is where and the quantile function is the inverse function of the CDF Ф of N(0,1).  Observation: A packing is feasible for a given overflow probability p iff for every bin j, The load of bin j is normally distributed with mean and variance

© 2009 IBM Corporation 13 Simple solution approach  Reduce the problem to the classical bin packing problem with item sizes, thus  A feasible solution to the classical bin packing problem is a feasible solution SBP, since  The optimum for the classical bin packing instance with the new sizes may be significantly larger than the optimum for SBP.

© 2009 IBM Corporation 14 Effective Size   Thus, the effective size of item i on bin j can be viewed as

© 2009 IBM Corporation 15 Approximation Algorithm Algorithm 1: First Fit VMR decreasing  Order the items in non-increasing order of VMR  Place the next item in the first bin into which it can be feasibly packed  If no such bin exists, open a new bin to pack this item Variance to Mean Ratio (VMR) is

© 2009 IBM Corporation 16 Approximation Algorithm Theorem 1: Algorithm 1 is a 2-approximation algorithm for SBP with normal variables.

© 2009 IBM Corporation 17 Integer Program for SBP

© 2009 IBM Corporation 18 Mathematical Program Relaxation

© 2009 IBM Corporation 19 Fractional Algorithm (Algorithm 2)  Order the items in non-increasing order of VMR  Place the next item in the bin with remaining capacity. If the item causes an overflow to the bin, assign maximum fraction of this item to the bin. Then, open a new bin to pack the remaining part of this item. Variance to Mean Ratio (VMR) is

© 2009 IBM Corporation 20 Analysis Lemma: There exists a feasible solution to the MP with the following property. For any pair of items k,l and a pair of bins i 0 and x li >0, then d l ≥ d k. Observation: Fractional algorithm produces a feasible fractional solution to the MP.  This implies that collocating items with high VMR (bursy) minimizes the total effective size of the items Variance to Mean Ratio (VMR) is

© 2009 IBM Corporation 21 Proof Outline  Consider a feasible solution to the MP with lexicographically maximal standard deviation (STD) vector of the bins S=(S 1,…,S m ), where  Assume by contradiction that the items are not packed into the bins according to non-increasing order of VMR  Thus, there exists at least one pair of items that are not placed in this order (i.e., item with smaller VMR is packed to a bin with smaller index than the other item).  We show that we can exchange fractions of these items between the bins, such that –the new solution is feasible –The STD vector of the bins in the solution is lexicographically greater than the one in the original solution  Contradiction

© 2009 IBM Corporation 22 Online Algorithm  VMR  Let  Class 0:  Class 1≤k≤C:  Class C+1:

© 2009 IBM Corporation 23 Online Algorithm Algorithm 3:  Classify next item according to the VMR classes  Place the next item in the first bin of its class into which it can be feasibly packed  If no such bin exists, open a new bin to pack this item Theorem 2: Algorithm 3 is a (2+O(ε))-approximation algorithm for SBP with normal variables.

© 2009 IBM Corporation 24 Simulation Study  Compare our proposed algorithms to previous reported ones  Data set –Real trace from production data center used to compute mean and standard deviation of bandwidth consumption of 6000 VMs over a few hours period. –Synthetic traces with statistical properties similar to those of the real traces

© 2009 IBM Corporation 25 Algorithms  Algorithms 1-3  First Fit (FF) with deterministic item sizes μ i +βσ i  First Fit Decreasing (FFD) with deterministic item sizes μ i +βσ i  Group Packing (GP) [Wang et. al 2011] For the online algorithms (Algorithm 3 and Group Packing), we set ε=0.1.

© 2009 IBM Corporation 26 Real Instance (Online) (Approx.) (L.B)

© 2009 IBM Corporation 27 Real Instance (L.B) (Approx.) (Online)

© 2009 IBM Corporation 28 Real Instance (L.B) (Approx.) (Online)

© 2009 IBM Corporation 29 Online Algorithms  Large synthetic instances 8% 9%

© 2009 IBM Corporation 30 Summary  We studied SBP under the assumption that virtual machines bandwidth demand obeys normal distribution  We showed a 2-approximation algorithm  We showed (2+ε)-competitive algorithm  We observed the existence of a dual PTAS for SBP  We studied the performance and applicability of our algorithms using synthetic and real data  The performance evaluation showed that our proposed algorithms considerably reduce the number of bins compared to the best known algorithms for the problem