1 Finding Constant From Change: Revisiting Network Performance Aware Optimizations on IaaS Clouds Yifan Gong, Bingsheng He, Dan Li Nanyang Technological University Tsinghua University

Outline Introduction RPCA-based approach Evaluation results Conclusions 2

Outline Introduction RPCA-based approach Evaluation results Conclusions 3

4 Infrastructure as a Service is Pervasive Many cloud providers supply IaaS clouds Amazon EC2 Google Compute Engine Rackspace IaaS clouds have emerged as a popular computing infrastructure for many distributed applications Life Science [Nagavaram 2011] Physics [Nunez 2010] Big data processing [He 2013]

5 Network Heterogeneity in IaaS cloud Topology of data center in IaaS cloud switch machines rack Network Heterogeneity switch machine rack Low Performance High Performance Heterogeneity

Network Performance-aware Optimization: State-of-the-art Network Performance-aware Optimization is an effective approach to optimize distributed applications on cloud MPI Collective Operation [Gong 2013] Topology Mapping [Hoefler 2011] Workflow Management [Spooner 2005 ] 6 Definition of Network Performance-aware Optimization Estimating or measuring the all-link network performance and carefully selecting communication links for minimizing the network transfer time of the application

Example of MPI Broadcast MPI Broadcast Binominal Tree Algorithm Root Machine data 7

Network Performance-aware Optimization of MPI Broadcast m0 Topology m1 m6 m7 m2 m3 m4 m Total Cost=6+6+6= Total Cost= =12 m0m1m2m3m4m5m6m7 m m m m m m m m Performance Matrix

Network Interference Problem 9 Machine 0 Application Machine 1 Application Background Application Network Interference Interference Network performance is changing as the time goes by –network interference

Distribution of Latency and Bandwidth between a pair of instance (virtual machine) in a long period Cloud Performance Dynamics: Latency/Bandwidth The range of Bandwidth(45~110MB/s) 10 Network performance is changing as the time goes by Interference largely impacts the network performance Network performance-aware optimization could not work

Our Work 11 Our solution: Robust Principal Component Analysis  Decouple the constant component from the dynamic network performance while minimizing the difference between the network performance and the constant component  Use the interference component to determine the effectiveness of network performance aware optimizations  Utilize the constant component to guide network performance aware optimizations Problem  How to distinguish network interference in the cloud?  How to enable existing network performance-aware optimizations in the dynamic cloud environment?

12 Robust Principal Component Analysis(RPCA) Frame 1 at t 1 Frame 2 at t 2 RPCA Low rank matrix D Sparse matrix E Mathematic Problem minimize rank(D) + λ ∥ E ∥ subject to A = D + E Video surveillance Figures are cited from [Wright 2009] * Frame 3 at t 3 Frame 4 at t 4 + Video

Outline Introduction RPCA-based approach Evaluation results Conclusions 13

Definition (b)TP-Matrix A Performance Matrix We can measure the performance of each link for a set of instances and the all-link network performance are defined as Performance Matrix Temporal Performance Matrix (TP-matrix) Performance matrix can only reflect a snapshot of network performance at a certain point of time. We define Temporal Performance Matrix as the combination of Performance matrix in a continuous time interval.

15 Analogy to RPCA in computer vision Frame 1 at t 1 Frame 2 at t 2 Frame 3 at t 3 Frame 4 at t 4 Performance Matrix at t Reshape t1t1 Performance Matrix at t Reshape t2t2 Combine t 1 t 2, …, t t1t1 t3t3 t2t2 t4t t3t t4t4 Temporal Performance Matrix Video

RPCA Video (Original matrix A) Analogy to RPCA in computer vision (con’t) RPCA Temporal Performance Matrix A t1t1 t3t3 t2t2 t4t4 matrix D (Low rank ) matrix E (Sparse) + Matrix D (Constant) Matrix E (Interference)

17 RPCA-based Approach Reshape Matrix D (Constant) Matrix E (Interference) Performance Matrix 0.1 Network Performance-aware Optimization Calculate quantity of interference

A is the temporal performance matrix from direct measurement Use Robust PCA to distinguish the interference minimize rank(D) + λ ∥ E ∥ subject to A = D + E D is constant and E represents the interference Use matrix E to calculate quantity of interference in the cluster and predict the efficiency of network performance- aware optimizations Norm (E) = ∥ E ∥ / ∥ A ∥ Use matrix D to optimize some applications 18 Basic procedure

Model calibration How to reduce the calibration overhead for each performance matrix? How many performance matrix to calibrate? Update maintenance When to re-calibrate the network performance? 19 Implementation Details Detailed techniques are in the paper.

Outline Introduction RPCA-based approach Evaluation results Conclusions 20

Experiment Setup 21 Experiments with Amazon EC2 196 instances m1.medium Simulation with ns-2 Tree-structured topology 1024 machines: totally 32 racks (10Gb/s) and each rack contains 32 servers (1Gb/s) Application MPI Broadcast/Scatter Topology Mapping Conjugate gradient (CG) N-body

Comparisons 22 Baseline This simulates the scenario of running directly in the cloud environment, essentially without network performance aware optimizations Heuristics We capture the TP-matrix and use the average value of each link to optimize the applications. RPCA The network performance aware optimizations are guided by the constant part captured by our RPCA approach.

23 The impact of interference on different applications in the virtual cluster (RPCA approach) The impact of interference on different approaches in the virtual cluster (MPI Scatter) 25% 32% 37% 24% 32% Norm(E) The impact of interference The interference in Amazon EC2 is relatively low (about 0.1) Performance improvement in Amazon EC2 is about 25%-37%

Overall Performance of CG 24 31%14% The trend of improvement When the vector size is small, our algorithm is slower than MPICH2-based CG, due to the calibrating and calculation overheads. As the vector size increases, the performance gain compensates the overhead, with 31% and 14% performance improvement over baseline and Heuristics.

Background traffic simulation 25 Choose the links and then vary two parameters to control the background traffic  The distribution of waiting time between sending the message  poisson distribution and the expected value is λ  Message size The impact of network interference on Norm(E) Norm(E) (a) Different expected values (λ) of Poisson Distribution (b) Different message size Norm(E) clearly has positive correlations with the background traffic in the simulated cluster.

Outline Introduction RPCA-based approach Evaluation results Conclusions 26

Conclusions Network is dynamic in the cloud and network performance- aware optimizations could not be directly used because of network interference. We distinguish the constant and interference from the network and calculate the quantity of interference Norm (E) = ∥ E ∥ / ∥ A ∥ We study the relationship between the interference and the network aware optimization efficiency We propose approaches to enable existing/new network performance-aware optimizations 27

Thank you!