11 Tesseract* A 4D Network Control Plane Hong Yan, David A. Maltz, T. S. Eugene Ng Hemant Gogineni, Hui Zhang, Zheng Cai *Tesseract is a 4-dimensional cube

22 Split load between S5 and S6Shut down S6 for maintenance on May 1 forwarding state Ideally… Managing network in a simple way Directly and explicitly apply policies to network accurate network view S1 S2S3S4 S5 S6 Internet

33 Probe routers to fetch configuration Monitor control traffic (e.g., LSAs, BGP update) probe routers and guess network view S1 S2S3S4 S5 S6 Internet Indirect Control - Fact #1: Infer network view by reverse engineering ? ? ? ??

44 Change OSPF link weights on S2, S3, S4.. Modify routing policies on S2, S3, S4… configuration commands Many knobs to tune Trial and error probe routers and guess network view S1 S2S3S4 S5 S6 Internet ? ? ? ?? Indirect Control - Fact #2: Policies buried in box-centric configuration

55 Complex configuration is error-prone and is causing network outages interface Ethernet0 ip address interface Serial1/0.5 point-to-point ip address ip access-group 143 in frame-relay interface-dlci 28 router ospf 64 redistribute connected subnets redistribute bgp metric 1 subnets network area 0 router bgp redistribute ospf 64 match route-map 8aTzlvBrbaW neighbor remote-as neighbor distribute-list 4 in access-list 143 deny /16 access-list 143 permit any route-map 8aTzlvBrbaW deny 10 match ip address 4 route-map 8aTzlvBrbaW permit 20 match ip address 7 ip route /

66 Indirect Control - Fact #3: Indirect Control Creates Subtle Dependencies Example:  Policy #1: use C as egress point for traffic from AS X  Policy #2: enable ECMP for A-C flow AS Y AS X 1 4 DesiredUnexpected! CB A D

77 Direct Control: A New World Express goals explicitly  Security policies, QoS, egress point selection  Do not bury goals in box-specific configuration  Make policy dependencies explicit Design network to provide timely and accurate view  Topology, traffic, resource limitations  Give decision maker the inputs it needs Decision maker computes and pushes desired network state  FIB entries, packet filters, queuing parameters  Simplify router functionality  Add new functions without modifying/creating protocols or upgrading routers

88 D How can we get there? Routing Table Access Control Table NAT Table Tunnel Table Decision Computation Service Generating table entries Data Plane Modeled as a set of tables Install table entries Discovery Dissemination Service D D D 4D

99 Tesseract: A 4D System Decision Element Dissemination R2 R1 Hello from R1 Hello from R2 Discovery

10 Bootstrapping Dissemination R1 R3 R2 R4 R5 Beac1: DE1 R3 Beac1: DE1 R3 R2 Beac1: DE1 R3 R2 R4 Beac1: DE1 Beac1: DE1 R3 R2 R4 R5 DE1 DE2

11 Bootstrapping Dissemination R1 R3 R2 R4 R5 DE1 DE2 DE beacons establish ctrl topology LSAs flow back from routers over ctrl topology After link/switch crash, next beacon heals topology

12 Making Decision R2 DE ’ s input includes TE goals, reachability matrix DE creates tables for each router (FIB, filters) Tables source-routed to destination via dissemination R2’s Routing Table: /24: R /24: R /24: eth0 0/0: R5

13 Decision/Dissemination Interface R1 DE1 Decision Plane Dissemination Plane Flood (pkt) Send (pkt, dst) RegisterUpCall (*fun) LinkFailure(link) PreferredRoute(dst, route) LSA

14 Reusable Decision Algorithms

15 Code Snippet Floyd-Warshall for (unsigned k = 0; k < num; k++) for (unsigned i = 0; i < num; i++) for (unsigned j = 0; j < num; j++) { if (CostMatrix[i][k] != -1 && CostMatrix[k][j] != -1) if (CostMatrix[i][j] == -1 || CostMatrix[i][j] > CostMatrix[i][k] + CostMatrix[k][j] ) { CostMatrix[i][j] = CostMatrix[i][k] + CostMatrix[k][j]; FirstHopMatrix[i][j] = FirstHopMatrix[i][k]; LastHopMatrix[i][j] = LastHopMatrix[k][j]; }

16 DE1 is alive DE1 is boss DE Robustness R1 DE1 DE2 All DEs send beacons  Routers send state updates to all DEs on network  DEs can see each others ’ beacons DE with lowest ID is only one to write configs to routers If active DE crashes, its beacons stop  Next highest ranking DE takes over DE1 heard too long ago I becoming boss

17 Evaluation Emulab Topologies  Rocketfuel backbone network (114 nodes, 190 links) with a maximum round trip delay of 250 ms  Production enterprise network (40 nodes, 60 links)

18 Routing Convergence Experiments On both backbone and enterprise topologies Failure scenarios  Single link failures  Single node failures  Regional failures for backbone (failing all nodes in one city)  Link flapping Tesseract versus Aggressively Tuned OSPF (Fast OSPF)

19 Enterprise Network, Switch Failures Tesseract Fast OSPF

20 Backbone Network, Switch Failures Tesseract Fast OSPF

21 Backbone Network, Regional Failures Tesseract Fast OSPF

22 Microbenchmark Experiments A subset of Rocketfuel topologies with varying sizes Independently fail each link Measure:  DE computation time  Control traffic volume

23 DE Computation Time

24 Control Traffic Volume

25 Tesseract Applications Joint Control of Packet Routing and Filtering  Problem: dynamic routing but static packet filter placement  Solution: in addition to computing routes, DE computes filter placement based on a reachability matrix Link Cost Driven Ethernet Switching  Problem: Spanning tree switching makes inefficient use of available links  Solution: DE computes both spanning tree and shortest paths

26 Link Cost Driven Ethernet Switching: Multi-Tree

27 Revisiting Randomize Equal-Cost Shortest Path Selection for (unsigned k = 0; k < num; k++) for (unsigned i = 0; i < num; i++) for (unsigned j = 0; j < num; j++) { if (CostMatrix[i][k] != -1 && CostMatrix[k][j] != -1) if (CostMatrix[i][j] == -1 || CostMatrix[i][j] > CostMatrix[i][k] + CostMatrix[k][j] || CostMatrix[i][j] == CostMatrix[i][k] + CostMatrix[k][j] && rand() > RAND_MAX/2 ) { CostMatrix[i][j] = CostMatrix[i][k] + CostMatrix[k][j]; FirstHopMatrix[i][j] = FirstHopMatrix[i][k]; LastHopMatrix[i][j] = LastHopMatrix[k][j]; }

28 Link Cost Driven Ethernet Switching: Multi-Tree

29 Throughput Comparison

30 Related Work Separation of forwarding elements and control elements  IETF: FORCES, GSMP, GMPLS  SoftRouter [Lakshman] Centralization of decision making logic  RCP [Feamster], SANE [Casado] Alternative frameworks for network control  Tempest [Rooney], FIRE [Partridge]

31 Summary Direct control is desirable  Make sophisticated control policies easier to understand and deploy  Simplify router software  Enable easy innovation Direct control is implementable  Tesseract as proof-of-concept  Sufficiently scalable  Fast convergence

32 Future Work Formulate models that establish bounds of Tesseract  Scale, latency, stability, failure models, objectives Structuring decision logic  Arbitrate among multiple, potentially competing objectives  Unify control when some logic takes longer than others Protocol improvements  Better dissemination planes Tesseract Router Deployment in today’s networks  Data center, enterprise, campus, backbone

33 Reality TE/Security Policy Reverse-engineer Routing Logic Convert to Control plane configuration Config commands Access Control NAT Table Tunnel Table EIRGPOSPFBGP Configuration File Forwarding Table Access Control NAT Table Tunnel Table EIRGPOSPFBGP Configuration File Forwarding Table Access Control NAT Table Tunnel Table EIRGPOSPFBGP Configuration File Forwarding Table Indirect control with primitive configuration interface

34 Link Cost Driven Ethernet Switching: Mesh

35 Effects of Switch Failure on Aggregated Throughputs