A Narrow Waist for Multipath Routing Murtaza Motiwala Bilal Anwer, Mukarram bin Tariq David Andersen, Nick Feamster

Many Threats to Availability Natural disasters Physical failures (node, link) Router software bugs Misconfiguration Mis-coordination Denial-of-service (DoS) attacks Changes in traffic patterns (e.g., flash crowd) …

Idea: Backup/Multipath For intradomain routing –IP and MPLS fast re-route –Packet deflections [Yang 2006] –ECMP, NotVia, Loop-Free Alternates [Cisco] For interdomain routing –MIRO [Rexford 2006] Problem –Complexity and Scale: Protecting against arbitrary failures requires storing lots of state, exchanging lots of messages –Control: End systems cant signal when they think a path has failed

Two Questions What is the appropriate mechanism for achieving multiple paths? –One example: Path Splicing What is the appropriate interface for allowing end systems access to multiple paths? –Path Bits: A narrow waist for Internet routing

Backup Paths: Promise and Problems Bad: If any link fails on both paths, s is disconnected from t Want: End systems remain connected unless the underlying graph has a cut ts

Path Splicing: Main Idea Step 1 (Generate slices): Run multiple instances of the routing protocol, each with slightly perturbed versions of the configuration Step 2 (Splice end-to-end paths): Allow traffic to switch between instances at any node in the protocol t s Compute multiple forwarding trees per destination. Allow packets to switch slices midstream.

Generating Slices Goal: Each instance provides different paths Mechanism: Each edge is given a weight that is a slightly perturbed version of the original weight –Two schemes: Uniform and degree-based ts Base Graph ts Perturbed Graph

How to Perturb the Link Weights? Uniform: Perturbation is a function of the initial weight of the link Degree-based: Perturbation is a linear function of the degrees of the incident nodes –Intuition: Deflect traffic away from nodes where traffic might tend to pass through by default

Forwarding Traffic One approach: shim header with forwarding bits Routers use lg(k) bits to index forwarding tables –Shift bits after inspection To access different (or multiple) paths, end systems simply change the forwarding bits –Incremental deployment is trivial –Persistent loops cannot occur Other variations are possible (2 nd half of talk)

Forwarding: Putting It Together End system sets forwarding bits in packet header –Forwarding bits specify slice to be used at any hop Router examines/shifts bits, and forwards t s

Reliability Approaches Optimal Sprint (Rocketfuel) topology 1,000 trials p indicates probability edge was removed from base graph Reliability approaches optimal Average stretch is only 1.3 Sprint topology, degree-based perturbations

Simple Recovery Strategies Work Well Which paths can be recovered within 5 trials? –Sequential trials: 5 round-trip times –…but trials could also be made in parallel Recovery approaches maximum possible Adding a few more slices improves recovery beyond best possible reliability with fewer slices.

Two Questions What is the appropriate mechanism for achieving multiple paths? –One example: Path Splicing What is the appropriate interface for allowing end systems access to multiple paths? –Path Bits: A narrow waist for Internet routing

Allow for Innovation Above & Below Different applications & uses for multipath –Performance and load balancing –Availability Different mechanisms –Routing protocols: path splicing, ECMP, R-BGP, NIRA, MIRO, … –Implementation platforms: proprietary solutions, Click, NetFPGA, OpenFlow Need: Common interface for multipath routing.

A Narrow Waist for Multipath Routing

Decouple End Hosts from Protocols Easy access to multiple paths Consistent path selection per-host –Same encoding should always yield the same path Interoperation among many different multipath routing protocols –Different switches and different networks may have different implementations

Simple Interface and Implementation Ease of use at end systems Efficiently implementable in the network Scalable access to a large number of paths –End systems should not have to store state to represent all paths –Routers should not have to store forwarding table state for all paths

Path Bits: Three-Part Design Path bits interface –Level of control –Number of bits End system support –Monitoring framework –Packet interception Network control –Indexing into forwarding tables

Decision: Level of Control Path bits are opaque –Carry no explicit semantics –Instead, provide the following: Changing the bits, will, with high probability, change the forwarding path Satisfies the two design goals –End-host interface is decoupled from implementation –Interface is simple: few changes required

Decision: Number of Bits A small number of bits can provide sufficient flexibility. –Example: Path splicing Possible to map many existing multipath implementations to the interface

End-System Implementation Socket capture library –Intercept connect() and sendto() Kernel interface –Click module –Keeps track of active flows & bits for those flows Monitoring daemon

Network Implementations Software implementations in Click –Splicing, Deflections, ECMP++ Hardware implementations –Intel IXP: Splicing (~30 lines of C++ plug-in) –OpenFlow: kN entries per slice, ~20-25 lines at NOX controller –NetFPGA: Small additional overhead beyond base router implementation Key Idea: Multiple forwarding table entires with bits to index. Different routing protocols simply change what is in the tables and how the bits are mapped to this interface.

Path Splicing in Click Eight lines of C++.

Routing Deflections in Click Nine lines of C++.

NetFPGA Implementation Eight slices About 69% of available BRAM on NetFPGA (base router uses 53% of available BRAM)

NetFPGA Forwarding Performance No noticeable performance penalty with four tables.

Applications with Path Bits: Recovery Single-Path Routing (Baseline) Path SplicingECMP++

Conclusion Many uses and applications for multipath routing –Availability –Performance Many implementations of protocols –Path splicing: Simple, scalable, stable –Routing deflections Path bits: narrow waist allows applications and implementations to evolve independently –Decouples end host request from network implementation/protocol –Affords very simple implementations –Many environments: Data center, interdomain, intradomain, etc.