15-744: Computer Networking L-12 Middleboxes and NFV

Middleboxes and NFV Overview of NFV Challenge of middleboxes Middlebox consolidation Outsourcing middlebox functionality Readings: ETSI Network Functions Virtualization White Paper Design and Implementation of a Consolidated Middlebox Architecture Optional reading Making Middleboxes Someone Else’s Problem

Middleboxes and NFV Overview of NFV Challenge of middleboxes Middlebox consolidation Outsourcing middlebox functionality Readings: ETSI Network Functions Virtualization White Paper Design and Implementation of a Consolidated Middlebox Architecture Optional reading Making Middleboxes Someone Else’s Problem

ETSI NFV

Vision Why cant networking get same benefits as IT and cloud world? Commodity hardware? Virtualization? Consolidation

Network Functions Virtualisation

Benefits Reduced Capex Reduced time to market Elastic scaling Targeted services “Openness” (vendor neutrality..) Streamlining operations

Key Enablers Cloud + Virtualization Commodity/high volume servers

Big Picture Network Functions Virtualisation aligns closely with the SDN objectives to use commodity servers and switches

SDN vs NFV Complementary SDN is all about “control” plane NFV can happen w/o SDN Natural allies SDN enables orchestration, routing NFV can be the “substrate” over which SDN runs

New use cases Virtualized services for enterprises Virtual CDNs Virtualized mobile core networks Cloud bursting Integrate production/testing

What are the grand challenges? High performance virtual appliances? Isolation/coexistence Management solutions Fault tolerance Vendor independence/multi-vendor

What’s missing? What functions yield most benefits? Can it really replace hardware acceleration? Is virtualization necessary? What novel services can be developed? How much benefit is “enough” to motivate adoption?

Some references on NFV/SDN in real world AT&T Domain 2.0 Vision White Paper ONS 2014 Keynote: John Donovan, Senior EVP, AT&T https://www.youtube.com/watch?v=tLshR-BkIas Verizon-Carrier Adoption of Software-defined Networking, Stuart Elby, VP, Verizon https://www.youtube.com/watch?v=WVczl03edi4

Middleboxes and NFV Overview of NFV Challenge of middleboxes Middlebox consolidation Outsourcing middlebox functionality Readings: ETSI Network Functions Virtualization White Paper Design and Implementation of a Consolidated Middlebox Architecture Optional reading Making Middleboxes Someone Else’s Problem

Network “101” vs. Reality Traditional view: “Dumb” network application gateways Reality: Lots of in-network processing Appliances or Middleboxes: IDS, Firewall, Proxies, Load balancers….

Need for Network Evolution New applications Evolving threats Policy constraints Performance, Security, Compliance New devices

Middleboxes Galore! Data from a large enterprise Survey across 57 network operators Type of appliance Number Firewalls 166 NIDS 127 Media gateways 110 Load balancers 67 Proxies 66 VPN gateways 45 WAN Optimizers 44 Voice gateways 11 Total Middleboxes 636 Total routers ~900 APLOMB (SIGCOMM’13)

Middlebox management is hard! Middleboxes are widely used today to meet security, performance and compliance requirements. Even though they are a critical piece of the network infrastructure, there are expensive, complex and difficult to manage. A survey across 57 network operators showed that middlebox management is complex as significant manual effort is required for configuring them, as a result middleboxes are prone to failures due to misconfiguration and overload. Critical for security, performance, compliance But expensive, complex and difficult to manage

Middleboxes and NFV Overview of middleboxes and NFV Middlebox consolidation Outsourcing middlebox functionality Readings: Design and Implementation of a Consolidated Middlebox Architecture ETSI Network Functions Virtualization White Paper Optional reading Making Middleboxes Someone Else’s Problem

Can NFV/SDN help middlebox management? Centralized Controller Firewall IDS Proxy Web OpenFlow “Flow” FwdAction … “Flow” FwdAction … Proxy IDS … Necessity and an Opportunity: Incorporate functions markets views as important

Key “pain points”  Narrow interfaces Specialized boxes Management Management Narrow interfaces Specialized boxes Increases capital expenses & sprawl Increases operating expenses Limits extensibility and flexibility “Point” solutions! 

High-level idea: Consolidation Outline Motivation High-level idea: Consolidation System design Implementation and Evaluation

Key idea: Consolidation Two levels corresponding to two sources of inefficiency Network-wide Controller 2. Consolidate Management 1. Consolidate Platform

Consolidation at Platform-Level Today: Independent, specialized boxes AppFilter Proxy Firewall IDS/IPS Decouple Hardware and Software Commodity hardware: e.g., PacketShader, RouteBricks, ServerSwitch, SwitchBlade Consolidation reduces capital expenses and sprawl

Consolidation reduces CapEx Multiplexing benefit = Max_of_TotalUtilization / Sum_of_MaxUtilizations

Consolidation Enables Extensibility VPN Web Mail IDS Proxy Firewall Protocol Parsers Session Management Contribution of reusable modules: 30 – 80 %

Management consolidation enables flexible resource allocation Today: All processing at logical “ingress” Process (0.4 P) Process (P) Process (0.3 P) Simplify this slide! Process (0.3 P) N3 N1 Overload! P: N1 N3 N2 Network-wide distribution reduces load imbalance

High-level idea: Consolidation Outline Motivation High-level idea: Consolidation CoMb: System design Implementation and Evaluation

CoMb System Overview Network-wide Controller Logically centralized e.g., NOX, 4D General-purpose hardware: e.g., PacketShader, RouteBricks, ServerSwitch, Middleboxes: complex, heterogeneous, new opportunities

CoMb Management Layer Goal: Balance load across network. Leverage multiplexing, reuse, distribution Policy Constraints Resource Requirements Routing, Traffic Network-wide Controller Processing responsibilities

Capturing Reuse with HyperApps HTTP: 1+2 unit of CPU 1+3 units of mem HyperApp: find the union of apps to run CPU HTTP: IDS & Proxy 4 3 Memory HTTP UDP HTTP NFS IDS Proxy 2 UDP: IDS NFS: Proxy 1 3 4 CPU Memory 3 1 1 common CPU Memory Footprint on resource Need per-packet policy dependencies! Policy dependency are implicit

Modeling Processing Coverage HTTP: Run IDS  Proxy IDS  Proxy 0.4 IDS  Proxy 0.3 IDS  Proxy 0.3 HTTP N1  N3 N1 N2 N3 What fraction of traffic of class HTTP from N1 to N3 should each node process?

Network-wide Optimization Minimize Maximum Load, Subject to Processing coverage for each class of traffic  Fraction of processed traffic adds up to 1 No explicit Dependency Policy Load on each node  sum over HyperApp responsibilities per-path

Network-wide Optimization A simple, tractable linear program Very close (< 0.1%) to theoretical optimal

CoMb System Overview Network-wide Controller Logically centralized e.g., NOX, 4D General-purpose hardware: e.g., PacketShader, RouteBricks, ServerSwitch, Existing work: simple, homogeneous routing-like workload Middleboxes: complex, heterogeneous, new opportunities

CoMb Platform Applications Policy Enforcer IDS Proxy NIC Challenges: Performance Parallelize Isolation … Core1 … Core4 Challenges: Lightweight Parallelize Policy Enforcer Policy Shim (Pshim) IDS Proxy NIC Challenges: No contention Fast classification Classification: HTTP Traffic

Parallelizing Application Instances ✔ App-per-core HyperApp-per-core M2 M3 PShim M1 Core1 Core2 M1 M2 M3 Core1 Core2 Core3 PShim PShim Inter-core communication More work for PShim + No in-core context switch + Keeps structures core-local + Better for reuse - But incurs context-switch HyperApp-per-core is better or comparable Contention does not seem to matter!

CoMb Platform Design Core-local processing Workload balancing Hyper App1 Hyper App2 Hyper App3 Hyper App4 Hyper App3 PShim PShim PShim PShim PShim Q1 Q2 Q3 Q4 Q5 NIC hardware Parallel, core-local Contention-free network I/O

Outline Motivation High-level idea: Consolidation System design: Making Consolidation Practical Implementation and Evaluation

Consolidation is Practical Low overhead for existing applications Controller takes < 1.6s for 52-node topology

Benefits: Reduction in Maximum Load MaxLoadToday /MaxLoadConsolidated Consolidation reduces maximum load by 2.5-25X

Benefits: Reduction in Provisioning Cost ProvisioningToday /ProvisioningConsolidated Consolidation reduces provisioning cost 1.8-2.5X

Discussion Changes traditional vendor business Isolation Already happening (e.g., “virtual appliances”) Benefits imply someone will do it! May already have extensible stacks internally! Isolation Current: rely on process-level isolation Get reuse-despite-isolation?

Conclusions Network evolution occurs via middleboxes Today: Narrow “point” solutions High CapEx, OpEx, and device sprawl Inflexible, difficult to extend Our proposal: Consolidated architecture Reduces CapEx, OpEx, and device sprawl Extensible, general-purpose More opportunities Isolation APIs (H/W—Apps, Management—Apps, App Stack)

Middleboxes and NFV Overview of NFV Challenge of middleboxes Middlebox consolidation Outsourcing middlebox functionality Readings: ETSI Network Functions Virtualization White Paper Design and Implementation of a Consolidated Middlebox Architecture Optional reading Making Middleboxes Someone Else’s Problem

Typical Enterprise Networks Introduce components, really…, describe what mboxes are. Internet

Typical Enterprise Networks These have been a topic of considerable interest in the research community. Our intuition seeing these devices has been “it’s complicated”, but we wanted to test this claim and get to the bottom of how middleboxes impact network administration… Internet

Recap of middleboxes pain points High Capital and Operating Expenses Time Consuming and Error-Prone Physical and Overload Failures

How can we improve this? One option would be to fix the way that we manage and provision middleboxes in enterprise networks – e.g. CoMB in NSDI, ETTM in last year’s NSDI. A completely different track is to just take them out of the hands of the enterprise entirely, and place them in the cloud

Our Proposal There are also a number of middleboxes… (1) Firewalls… Internet

Our Proposal Internet Cloud Provider Introduce components, really…, describe what mboxes are. Internet

Economies of scale and pay-per use A move to the cloud High Capital and Operating Expenses Time Consuming and Error Prone Physical and Overload Failures Economies of scale and pay-per use Simplifies configuration and deployment Redundant resources for failover Use the word service / interface / abstract / policy re: configuration

Challenges Minimal Complexity at the Enterprise Functional Equivalence Low Performance Overhead “Can we really???”

APLOMB “Appliance for Outsourcing Middleboxes”

Outsourcing Middleboxes with APLOMB Cloud Provider Let’s start with a simple scenario --- one of these host guys wants to send traffic out to the Internet…. NAT APLOMB Gateway Internet

Inbound Traffic “Sure, that works great for end-user services – typically NATed anyway for security – but what about web-facing services? Consider this guy…” “Much like CDN services today…”

Inbound Traffic “Sure, that works great for end-user services – typically NATed anyway for security – but what about web-facing services? Consider this guy…” “Much like CDN services today…”

Inbound Traffic “Sure, that works great for end-user services – typically NATed anyway for security – but what about web-facing services? Consider this guy…” “Much like CDN services today…”

Inbound Traffic Internet Web Server: www.enterprise.com 192.168.1.100 Cloud Provider “Sure, that works great for end-user services – typically NATed anyway for security – but what about web-facing services? Consider this guy…” “Much like CDN services today…” Register: www.enterprise.com 192.168.1.100 Internet Enterprise Network Admin.

Inbound Traffic Internet Cloud Provider Register: enterprise.com “Sure, that works great for end-user services – typically NATed anyway for security – but what about web-facing services? Consider this guy…” “Much like CDN services today…” DNS Register: enterprise.com 98.76.54.32 Internet

Choosing a Datacenter External Client Route through cloud datacenter that minimizes end to end latency. Cloud Provider East Enterprise One option would be to map everyone to the cloud provider that’s closest to them; a better option….. APLOMB Gateway keeps a “routing table” to select best tunnel for every Internet prefix. Cloud Provider West External Client

Caches and “Terminal Services” Traffic destined to services like caches should be redirected to the nearest node. Cloud Provider West

APLOMB “Appliance for Outsourcing Middleboxes” Place middleboxes in the cloud. Use APLOMB devices and DNS to redirect traffic to and from the cloud. That’s it.

Can we outsource all middleboxes? Firewalls IDSes Load Balancers VPNs Proxy/Caches WAN Optimizers ✔ ✔ ✔ ✔ ✗ Bandwidth? ✗ Compression?

APLOMB+ for Compression Add generic compression to APLOMB gateway to reduce bandwidth consumption. I …I’ll show some numbers from a case study of an enterprise on how well this works. Cloud Provider Internet

Can we outsource all middleboxes? Firewalls IDSes Load Balancers VPNs Proxy/Caches WAN Optimizers ✔ ✔ ✔ ✔ ✗ Bandwidth? ✗ Compression? ✔ ✔

Does it work? Can such a simple design really work? Will it introduce a heavy performance overhead?

Our Deployment Cloud provider: EC2 – 7 Datacenters OpenVPN for tunneling, Vyatta for middlebox services Two Types of Clients: Software VPN client on laptops Tunneling software router for wired hosts

Evaluation Implementation & Deployment Wide-Area Measurements Performance metrics Wide-Area Measurements Network latency Mention two key questions we want to ask…. APPLICATION PERFORMANCE;ARCHITECTURAL/FEASIBILITY. Is such a simple solution really practical? Case Study of a Large Enterprise Impact in a real usage scenario

Does APLOMB inflate latency?

Drop explanations “poorly connected universities” For PlanetLab nodes, 60% of pairs’ latency improves with redirection through EC2.

Latency at a Large Enterprise Measured redirection latency between enterprise sites. Median latency inflation: 1.13 ms Sites experiencing inflation were primarily in areas where EC2 does not have a wide footprint. Asia has sites…. “areas where EC2

How does APLOMB impact other quality metrics, like bandwidth and jitter?

Bandwidth: download times with BitTorrent increased on average 2.3% Jitter: consistently within industry standard bounds of 30ms

Does APLOMB negate the benefits of bandwidth-saving devices?

Median case penalty – 3.8%; worst 8% APLOMB+ incurs a median penalty of 3.8% bandwidth inflation over traditional WAN Optimizers.

Does “elastic scaling” at the cloud provide real benefits? Start with overload

Some sites generate as much as 13x traffic more than average at peak hours.

Latency median inflation 1.1ms Download times increased only 2.3% Recap Good application performance Latency median inflation 1.1ms Download times increased only 2.3% Generic redundancy elimination saves bandwidth costs Strong benefits from elasticity

Conclusion Moving middleboxes to the cloud is a practical and feasible solution to the complexity of enterprise networks. I showed you APLOMB – a very simple architecture for outsourcing middlebox processing to the cloud, yet, despite it’s simplicity it achieves good very good performance – and as a result of its simplicity, we consider it easily deployable.

Summary NFV Middlebox Challenges of the traditional “appliances” Using NFV to consolidate middleboxes (CoMb) Outsource middleboxes to the could (APLOMB)

Next Lecture Data Center Networks 1 Readings: Portland: Read in full DCTCP: Read intro VL2, Incast: Optional reading

AT&T Domain 2.0

Why is AT&T writing this .. Reassessing future network technology Operations methods “Sourcing” methods i.e., call for arms to vendors! Bit of frustration with existing vendors Lock in, cost, inflexibility Experiment with startups/disruptive innovations

AT&T Domain 2.0 Draw ideas from cloud computing Architecture Open Provide APIs, enable better participation of third parties, and improve visibility Simplify Weed out complexity from services and operations; support more nimble business models. Scale traffic growth diversity of traffic types diversity of performance and reliability expectations.

Natural enablers SDN-like packet forwarding on merchant silicon NFV substrate Cloud/datacenter

Caveats Not a completed architecture or technology plan; sets direction! Networking hardware + software Software engineering Carrier operations models Cloud “DevOps” models.

Cloud networking infrastructure

Some use cases Flexible fabric inside datacenter Simplify customer premise equipment Internet of Things? Multi service access networks

AT&T and SDN Interesting footnote: AT&T accepts that proprietary interfaces fit the architecture described by SDN, there is a strong aversion to being locked-in to a vendor-specific protocol as it’s unlikely to allow us to reach our white box vision

Orchestration Control and Policy Management Service composition, instantiation & activation Dynamic creation, modification, customization & release of virtualized resources Run-time management, monitoring, Security Analytics, trending, & prediction feedback into optimization Audits & diagnostics ….

Operation transformation Physical to virtual Hardware to software

Some transitions Separate IT/data center & Network/CO  Common technology & technical plant Quarterly software releases  Continuous software process Tight coupling of NE, generic, EMS & NMS/OSS  Separation of physical & logical components. Optimized provider network & ops process Optimized customer experience. Static billing and charging  Granular and dynamic usage-based charging, billing, financial management, subscription

Business side of things Google/FB/Amazon – “insourcing” AT&T – traditionally 1-2 vendors New approach openly used, and cannot be lost through the acquisition or insolvency of a vendor do business with startups and small businesses that might have been too risky

Open source/cooperation etc draw from a much broader set of software sources, and attract top-tier software architects and developers. will not accept a proprietary information model, API makes business sense to work with others who have the same needs and desires on common technologies and component