Jennifer Rexford Fall 2010 (TTh 1:30-2:50 in COS 302) COS 561: Advanced Computer Networks Data Center Traffic Engineering

Cloud Computing 2

Elastic resources –Expand and contract resources –Pay-per-use –Infrastructure on demand Multi-tenancy –Multiple independent users –Security and resource isolation –Amortize the cost of the (shared) infrastructure Flexibility service management –Resiliency: isolate failure of servers and storage –Workload movement: move work to other locations 3

Cloud Service Models Software as a Service –Provider licenses applications to users as a service –E.g., customer relationship management, ,.. –Avoid costs of installation, maintenance, patches, … Platform as a Service –Provider offers software platform for building applications –E.g., Google’s App-Engine –Avoid worrying about scalability of platform Infrastructure as a Service –Provider offers raw computing, storage, and network –E.g., Amazon’s Elastic Computing Cloud (EC2) –Avoid buying servers and estimating resource needs 4

Multi-Tier Applications Applications consist of tasks –Many separate components –Running on different machines Commodity computers –Many general-purpose computers –Not one big mainframe –Easier scaling 5 Front end Server Aggregator … Aggregator Worker … …

Enabling Technology: Virtualization Multiple virtual machines on one physical machine Applications run unmodified as on real machine VM can migrate from one computer to another 6

Data Center Network 7

Status Quo: Virtual Switch in Server 8

Top-of-Rack Architecture Rack of servers –Commodity servers –And top-of-rack switch Modular design –Preconfigured racks –Power, network, and storage cabling Aggregate to the next level 9

Modularity, Modularity, Modularity Containers Many containers 10

Data Center Network Topology 11 CR AR... S S S S Internet S S S S A A A A A A … S S S S A A A A A A …... Key CR = Core Router AR = Access Router S = Ethernet Switch A = Rack of app. servers ~ 1,000 servers/pod

Capacity Mismatch 12 CR AR S S S S S S S S A A A A A A … S S S S A A A A A A …... S S S S S S S S A A A A A A … S S S S A A A A A A … ~ 5:1 ~ 40:1 ~ 200:1

Data-Center Routing 13 CR AR... S S S S DC-Layer 3 Internet S S S S A A A A A A … S S S S A A A A A A …... DC-Layer 2 Key CR = Core Router (L3) AR = Access Router (L3) S = Ethernet Switch (L2) A = Rack of app. servers ~ 1,000 servers/pod == IP subnet SSSS SS

Reminder: Layer 2 vs. Layer 3 Ethernet switching (layer 2) –Cheaper switch equipment –Fixed addresses and auto-configuration –Seamless mobility, migration, and failover IP routing (layer 3) –Scalability through hierarchical addressing –Efficiency through shortest-path routing –Multipath routing through equal-cost multipath So, like in enterprises… –Data centers often connect layer-2 islands by IP routers 14

Load Balancers Spread load over server replicas –Present a single public address (VIP) for a service –Direct each request to a server replica 15 Virtual IP (VIP)

Data Center Costs (Monthly Costs) Servers: 45% –CPU, memory, disk Infrastructure: 25% –UPS, cooling, power distribution Power draw: 15% –Electrical utility costs Network: 15% –Switches, links, transit 16

Wide-Area Network 17 Router DNS Server DNS-based site selection... Servers Internet Clients Data Centers

Wide-Area Network: Ingress Proxies 18 Router Data Centers... Servers Clients Proxy

Data Center Traffic Engineering Challenges and Opportunities 19

Traffic Engineering Challenges Scale –Many switches, hosts, and virtual machines Churn –Large number of component failures –Virtual Machine (VM) migration Traffic characteristics –High traffic volume and dense traffic matrix –Volatile, unpredictable traffic patterns Performance requirements –Delay-sensitive applications –Resource isolation between tenants 20

Traffic Engineering Opportunities Efficient network –Low propagation delay and high capacity Specialized topology –Fat tree, Clos network, etc. –Opportunities for hierarchical addressing Control over both network and hosts –Joint optimization of routing and server placement –Can move network functionality into the end host Flexible movement of workload –Services replicated at multiple servers and data centers –Virtual Machine (VM) migration 21

VL2 Paper 22 Slides from Changhoon Kim (now at Microsoft)

Virtual Layer 2 Switch L2 semantics 2. Uniform high capacity 3. Performance isolation A A A A A A … A A A A A A …... A A A A A A … A A A A A A … CR AR S S S S S S S S S S S S S S S S S S S S S S S S A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A...

VL2 Goals and Solutions 24 Solution Approach Objective 2. Uniform high capacity between servers Enforce hose model using existing mechanisms only Employ flat addressing 1. Layer-2 semantics 3. Performance Isolation Guarantee bandwidth for hose-model traffic Flow-based random traffic indirection (Valiant LB) Name-location separation & resolution service TCP “Hose”: each node has ingress/egress bandwidth constraints

Name/Location Separation 25 payload ToR 3... y x Servers use flat names Switches run link-state routing and maintain only switch-level topology Cope with host churns with very little overhead yz payload ToR 4 z ToR 2 ToR 4 ToR 1 ToR 3 y, z payload ToR 3 z... Directory Service … x  ToR 2 y  ToR 3 z  ToR 4 … Lookup & Response … x  ToR 2 y  ToR 3 z  ToR 3 … Allows to use low-cost switches Protects network and hosts from host-state churn Obviates host and switch reconfiguration Allows to use low-cost switches Protects network and hosts from host-state churn Obviates host and switch reconfiguration

Clos Network Topology TOR 20 Servers Int... Aggr K aggr switches with D ports 20*(DK/4) Servers Offer huge aggr capacity & multi paths at modest cost D (# of 10G ports) Max DC size (# of Servers) 4811, , ,680

Valiant Load Balancing: Indirection 27 xy payload T3T3 y z T5T5 z I ANY Cope with arbitrary TMs with very little overhead Links used for up paths Links used for down paths T1T1 T2T2 T3T3 T4T4 T5T5 T6T6 [ ECMP + IP Anycast ] Harness huge bisection bandwidth Obviate esoteric traffic engineering or optimization Ensure robustness to failures Work with switch mechanisms available today [ ECMP + IP Anycast ] Harness huge bisection bandwidth Obviate esoteric traffic engineering or optimization Ensure robustness to failures Work with switch mechanisms available today 1. Must spread traffic 2. Must ensure dst independence 1. Must spread traffic 2. Must ensure dst independence Equal Cost Multi Path Forwarding

VL2 vs. Seattle Similar “virtual layer 2” abstraction –Flat end-point addresses –Indirection through intermediate node Enterprise networks (Seattle) –Hard to change hosts  directory on the switches –Sparse traffic patterns  effectiveness of caching –Predictable traffic patterns  no emphasis on TE Data center networks (VL2) –Easy to change hosts  move functionality to hosts –Dense traffic matrix  reduce dependency on caching –Unpredictable traffic patterns  ECMP and VLB for TE 28

Ongoing Research 29

Research Questions What topology to use in data centers? –Reducing wiring complexity –Achieving high bisection bandwidth –Exploiting capabilities of optics and wireless Routing architecture? –Flat layer-2 network vs. hybrid switch/router –Flat vs. hierarchical addressing How to perform traffic engineering? –Over-engineering vs. adapting to load –Server selection, VM placement, or optimizing routing Virtualization of NICs, servers, switches, … 30

Research Questions Rethinking TCP congestion control? –Low propagation delay and high bandwidth –“Incast” problem leading to bursty packet loss Division of labor for TE, access control, … –VM, hypervisor, ToR, and core switches/routers Reducing energy consumption –Better load balancing vs. selective shutting down Wide-area traffic engineering –Selecting the least-loaded or closest data center Security –Preventing information leakage and attacks 31

Discuss 32