1. Sponsored by the National Science Foundation GEC16: OpenFlow Switches in GENI Marshall Brinn, GPO March 21, 2013

2. Sponsored by the National Science Foundation2January, 2013 Outline Nick Bastin, Big Switch –Introduction to Hardware Switch Architectures Marshall Brinn, GPO –Network Slicing and Programming with VLAN’s and OpenFlow

3. Sponsored by the National Science Foundation3January, 2013 NICK BASTIN

4. Sponsored by the National Science Foundation GENI: Network Slicing and Programming with VLANs and OpenFlow Marshall Brinn, GPO March 21, 2013

5. Sponsored by the National Science Foundation5January, 2013 Introduction GENI has focused on specifying requirements on Aggregates for resource allocation through the Aggregate Manager (AM) API But there are GENI requirements about network slicing and programmability that aren’t specified in the AM API. –Specifically, how to support the common (though not universal) use case of network management with VLAN’s and OpenFlow These slides propose a set of standards for GENI aggregates with respect to network slicing and programmability using VLANs and using OpenFlow –And describe some simple examples and possible engineering approaches. By establishing these standards, we can then assess existing and developing aggregates to make the experimenter experience more uniform and reliable over time.

6. Sponsored by the National Science Foundation6January, 2013 Slicing and Programming the Network in GENI GENI network slicing will be done by VLAN tags –Why? The simplest, standard way to partition L2 traffic GENI network programming may be done by OpenFlow –Note: It isn’t a requirement that GENI aggregates use OpenFlow for network programming. –But if they DO use OpenFlow, we would like there to be common conventions for that use, particularly wrt. slicing the network by VLAN tags –We are particularly considering the case of GENI racks, which we expect will use OpenFlow to provide network programmability We aren’t saying one couldn’t also slice or program a network in other ways. But these slides focus on the case of using OF to program the VLAN-sliced network.

7. Sponsored by the National Science Foundation7January, 2013 Preliminaries GENI networking operates on two distinct planes: –The Control/Management Plane (L3+) for: Traffic between tools and aggregates (AM API) Intra-aggregate control traffic Extra-aggregate control traffic (to GMOC, CH) OpenFlow Control traffic SSH to log into resources –The Data Plane (L2) for experimenter traffic Each slice has one or more VLAN’s uniquely assigned to it. Slice traffic is VLAN tagged and (therefore) segregated across slices [Note: Some deployments will require sharing VLAN’s across slices] An Aggregate should provide two different network interfaces to support and segregate these two different kinds of network traffic.

8. Sponsored by the National Science Foundation8January, 2013 GENI OpenFlow Networking: The cast of characters Switch: The point of network ingress/egress for an aggregate [Ignore, for now, any aggregate-internal switches] Controller: Experimenter-provided OF Controller Proxy-Controller: Managing interface between Switch and Controller [Think: FlowVisor or similar] Host: Network-addressable ‘edge node’ compute resource in a topology Obviously, a given topology may have many instances of these, configured in arbitrary ways. But these are the Lego-pieces from which we build a sliced, stitched, programmable network topology.

9. Sponsored by the National Science Foundation9January, 2013 The Simple Case Experimenter Controller (VLAN=v) Experimenter Controller (VLAN=v) Proxy-Controller Switch Host 1) A packet comes into the switch. 2) IF the packet doesn’t match any current switch flow rules, it passes the packet to the Proxy Controller. 3) IF the packet is associated with an experimenter-provided Controller (based on VLAN of packet and slice), the packet is dispatched to the experimenter Controller. 4) The Controller may drop the packet, or pass back a modified packet, or propose flow rules to install in Switch. 5) The Proxy Controller may allow the packets/rules to flow to the switch, or may filter or modify them to protect the segregation of slice traffic. 6) The packet is (possibly) passed along to host.

10. Sponsored by the National Science Foundation10January, 2013 But things aren’t always so simple… Different classes of OF switches VLAN translation Special topologies require special tagging and control

11. Sponsored by the National Science Foundation11January, 2013 Three Classes of OF Switches Pure OF SwitchPort Hybrid SwitchVLAN Hybrid Switch OF GranularityEach port is OF enabled Some ports are OF enabled, some aren’t Some VLAN’s are OF enabled, some aren’t DPID’sSingle DPID for entire switch Single DPID for all OF-enabled ports One per VLAN ControllersOne (proxy-) controller for entire switch One (proxy-) controller for all OF- enabled ports One (proxy-) controller per DPID But could use same (proxy-) controller for multiple DPID’s Traffic to Controller VLAN-tagged Not VLAN-tagged Proxy- Controller Discriminant Dispatch by VLAN- tag Dispatch by DPID Think of the Port Hybrid as two switches: An OF switch with fewer ports, and a non-OF switch for the rest of the ports. To handle the general set of switches, Slices and Experimenter controllers must be tagged by a unique VLAN/DPID tuple.

12. Sponsored by the National Science Foundation12January, 2013 Switch: Description and Requirements There may be one or more outward-facing (linked to resources and networks outside the aggregate) ports on the switch –As well as one or more inward-facing ports (linked to aggregate resources) OF-enabled Switches must provide an OpenFlow datapath (DPID) or multiple OF DPID’s –Supporting OF V1.0 Not every Switch must be OF-enabled (on all or any ports). But consider those Switches that are OF-enabled.

13. Sponsored by the National Science Foundation13January, 2013 Switch: Description and Requirements [2] The Switch should support VLAN translation –To translate external VLAN tags to aggregate-internal VLAN tags as needed. Why? –Traffic that never reaches ION or another translation service (e.g. traffic between two campuses of the same regional, or traffic between two aggregates on the same campus) have no default VLAN translation mechanism –Making stitching a manual and less-likely prospect. [Note: We recognize that some campuses may connect to GENI in other ways that will require special engineering (e.g. tunneling).] This is a key enabler of GENI scalability and new racks must provide this capability

14. Sponsored by the National Science Foundation14January, 2013 Controller: Description and Requirements The Controller may create any flow entry or packet –But only flow entries and packets for VLAN’s owned by the slice associated with the controller will be forwarded to the switch by the proxy-controller –That is, the controller can only program traffic for the DPID(s) or VLAN(s) of the associated slice Traffic reaching the controller will be tagged with a sliver-unique ‘discriminant’: either VLAN or DPID (or both) –Depending on the slice topology and switch configuration

15. Sponsored by the National Science Foundation15January, 2013 Proxy-Controller: Description and Requirements The Proxy-Controller performs several functions: –Multiplexes multiple experimenter controllers, based on VLAN –Distributes OF messages (including packets) from switches to experimenter controllers based on discriminant [VLAN, DPID] –Monitors and filters data from experimenter controllers to OF switch Making sure packet VLAN is properly set for slice traffic Adding VLAN match criteria on any flow entries provided by experimenter controller Note: I intentionally avoid specifying FlowVisor here. While it is a perfectly acceptable implementations of the Proxy-Controller, an aggregate can implement these requirements as it chooses.

16. Sponsored by the National Science Foundation16January, 2013 Proxy-Controller: Description and Requirements [2] For slices for which no controller is supplied, Proxy- Controller operates as standard L2 learning switch Learning port  MAC mapping for nodes on that VLAN by flooding/remembering when an unknown MAC destination is encountered –Writing this mapping into OF switch An experimenter should not create a topology with a loop without providing a controller –Though the Proxy-controller could use spanning tree algorithms to detect and avoid bad consequences. Note: The Proxy-Controller is not necessarily an Aggregate Manager and doesn’t need to speak the AM API. It is the job of an aggregate (be it FOAM or the ‘compute resource’ aggregate) to inform the Proxy- Controller about new flow space requirements.

17. Sponsored by the National Science Foundation17January, 2013 Proxy-Controller: Example Operations Controller (VLAN=v) Proxy- Controller Switch Flow Entries provided by Controller have VLAN entries added to match clauses {Match: DEST=a, Action: out=p} {Match: DEST=a, VLAN=v Action: out=p} Controller (VLAN=v) Proxy- Controller Switch Flow Entries tagged with wrong VLAN dropped {Match: DEST=a, VLAN=w Action: out=p} Controller (VLAN=v) Proxy- Controller Switch Packets tagged with wrong VLAN dropped {SRC=s, VLAN=w}

18. Sponsored by the National Science Foundation18January, 2013 Proxy-Controller: Example Operations Controller (VLAN=v) Proxy- Controller Switch Unmatched packets dispatched to Controller by VLAN {VLAN=v, SRC=s, DST=d, …|} {VLAN=v, SRC=s, DST=d, …|} Proxy- Controller Switch No Controller: Act as L2 learning switch Receive unknown packet, flood and learn PORT  MAC rules Controller (DPID=d) Proxy- Controller VLAN Hybrid Switch Unmatched packets dispatched to Controller by DPID {DPID=d, SRC=s, DST=d, …|} {DPID=d, SRC=s, DST=d, …|}

19. Sponsored by the National Science Foundation19January, 2013 VLAN Hybrid Switches and Controllers In the case of VLAN Hybrid Switches, there are many individual DPID’s provided and each can be associated with a controller. It is still desirable to interpose a proxy-controller between the controller and the switch: –To protect against controllers that don’t reliably drop or fix improper VLAN tagging on packets or flows –To protect against unreliable switch firmware

20. Sponsored by the National Science Foundation20January, 2013 Ports/VLANs/DPIDs are the Unique Tuple In the general case, OF rules discriminate traffic on the basis of a unique [PORT, VLAN-tag, DPID] tuple –There are potentially multiple ingress/egress ports on a switch (especially beyond edge nodes, at backbones or regionals) –There are potentially multiple paths for L2 traffic between two edge nodes –There are potentially multiple VLAN’s per slice spanning multiple aggregates Consider the case of three switches connected in a triangular topology: S1 S2 S3 Traffic from a node on S1 to a node on S3 cannot be uniquely specified by a VLAN, nor by an output port, but by the pairing of the two

21. Sponsored by the National Science Foundation21January, 2013 Some Engineering Details: An interesting example GA Tech SOX (OF) U. FLA Clemson Juniper (non- OF) Juniper (non- OF) A controller managing SOX switch MUST write VLAN-tagged packets: Juniper switch is invisible to GENI (not in stitching manifest). SOX indicates that it has traffic going out same port but different VLAN’s. VLAN=6 VLAN=7 Port=1, VLAN=6Port=1, VLAN=7 VLAN=100

22. Sponsored by the National Science Foundation22January, 2013 Some Engineering Details: Stitching From the AGG’s perspective, the act of “creating a stitch” is precisely the act of establishing VLAN translation between external VLAN tags/ports and internal VLAN tags/ports Agg 1 Switch 1 Agg 2 Switch 2 Topology with VLAN=v1 Topology with VLAN=v2 Extra-aggregate traffic on VLAN=v0 Switch 0 Switch Rule “Map V0=>V2 incoming, V2=>V0 outgoing” is the stitch Switch Rule “Map V0=>V1 incoming, V1=>V0 outgoing” is the stitch

23. Sponsored by the National Science Foundation23January, 2013 Stitching to non-GENI Campus resources This same approach to stitching allows aggregates to stitch non-GENI campus resources into a given slice. –Administrators arrange for VLAN-tagged traffic to appear on a particular port of aggregate switch Avoiding conflicts on a shared VLAN is a human activity. –The aggregate maps this traffic into the slice topology Campus Resource Agg 2 Switch 2 Topology with VLAN=v2 Extra-aggregate traffic on VLAN=v0 Switch Rule “Map V0=>V3 incoming, V3=>V0 outgoing” is the stitch

24. Sponsored by the National Science Foundation24January, 2013 Summary The different kinds of OpenFlow switches (pure, VLAN- hybrid, PORT-hybrid) have different semantics and require different handling In the general case, OpenFlow controllers need to manage a unique tuple of [PORT, DPID, VLAN] to manage (route, distinguish) traffic The Proxy-Controller must, in addition to filtering improper rules and packets, add VLAN, DPID or PORT match criteria to controller-provided rules. There are configurations for which a GENI aggregate must perform VLAN translation (or fail to stitch) The main ‘take away’ points from this brief which we’d like your help refining.

25. Sponsored by the National Science Foundation25January, 2013 Conclusion These slides try to lay out some principles for providing network programmability and slicing using OpenFlow and VLAN tags I hope that over time we can flesh these out to be more correct and complete Then I expect we can use these to assess current and developing aggregates in terms of the OpenFlow network programmability capability they may provide