Nick McKeown (Many thanks to Lisa and Lavanya) CS244 Programmable Switches Forwarding metamorphosis: fast programmable match-action processing … Pat Bosshart, et al.

Open Compute Project: Wedge switch (open-source design from Facebook) 2 Switch chip! State of the art about 3.2Tb/s (32x100GE) Switch chip! State of the art about 3.2Tb/s (32x100GE)

Linux Feature Code Feature Code Control Plane 2014: The bare-metal switch Feature Code Feature Code Feature Code Feature Code

Now I can tailor my network to meet my needs! I can…. 1.Quickly deploy new protocols. 2.See what my forwarding plane is doing. 3.Put expensive middlebox functions into the network. 4.Try out beautiful new ideas. Tailor my network to meet my needs. 5.Differentiate. Now I own my intellectual property.

“Beautiful ideas” 1.Deploy new protocols and new headers 2.Simplify the data plane. Throw out unused protocols. 3.Reallocate resources in switches: tables, packet buffers, etc. 4.Add new telemetry for debugging and diagnostics 5.Verify network behavior 6.Embed “middlebox” functions into the network: load-balancing, gateways and firewalls. 7.In-network congestion control 8.New routing and reliability algorithms 9.…

Linux Feature Code Feature Code Control Plane Driver Tailoring my network switches today Feature Code Feature Code Feature Code Feature Code New Feature Code New Feature Code

Can a CPU forward all my packets?

Packet Forwarding Speeds 8 Gb/s (per chip) 3.2Tb/s

Packet Forwarding Speeds 9 50x Gb/s (per chip) 3.2Tb/s

Conventional Wisdom: “Programmable devices are x slower. They consume much more power and area.”

Wedge 11 Whitebox CPU Blackbox switch

My whitebox switch has a blackbox switch inside

Fixed-Function Switch Chips Queues L2 Stage IPv4 Stage Parser IPv6 Stage ACL Stage L3 L2 Packet 13

Domain Specific Processors GPU Graphics Compiler Applications DSP Signal Processing Compiler Applications My codec My renderer

Conventional wisdom said: programmability too expensive Then, someone identified: 1.The right model for data-parallelism 2.Basic underlying processing primitives Domain-specific processors were built Domain-specific languages, compilers and tool-chains

Control Flow Graph Queues L2 Stage IPv4 Stage Parser IPv6 Stage ACL Stage L2 Table IPv4 Table IPv6 Table ACL Table L2 v4 v6 ACL Control Flow Graph Switch Pipeline Fixed Action Action Fixed Action 16

Fixed-Function Switch Chips Are Limited 1.Can’t add new forwarding functionality 17

Fixed-Function Switch Chips Queues L2 Stage IPv4 Stage Parser IPv6 Stage ACL Stage L2 Table IPv4 Table IPv6 Table ACL Table Fixed Action Action Fixed Action L2 v4 v6 ACL Control Flow Graph Switch Pipeline MyEncap 18

19 Fixed-Function Switch Chips Are Limited 1.Can’t add new forwarding functionality 2.Can’t move resources between functions Queues L2 Stag e IPv4 Stag e Parser IPv6 Stag e ACL Stag e Fixed Action Action L2 Table Fixed Action IPv4 Table IPv6 Table ACL Table

Control Flow Graph Switch Pipeline Programmable Switch Chips Queues Parser Fixed Action L2 Table Fixed Action IPv4 Table IPv6 Table ACL Table Match Table L2 v4 v6 ACL Action Macro 20

Match Table Action Macro Mapping Control Flow to Programmable Switch Chip. Queues Parser Match Table L2 Table IPv4 Table IPv6 Table ACL Table Action Macro L2 v4 v6 ACL Control Flow Graph Switch Pipeline L2 v6 ACL v4 L2 Action Macro v4 Action Macro v6 Action ACL Action Macro 21

Control Flow Graph Switch Pipeline Programmable Switch Chips Queues Parser L2 Table IPv4 Table ACL Table IPv6 MyEncap L2 v4 v6 ACL MyEncap L2 Action Macro v4 Action Macro ACL Action Macro Action MyEncap Action IPv4 Action IPv4 Action 22 IPv6 IPv4 Action

RMT: Reconfigurable Match + Action (Now more commonly called “PISA”) 23

PISA: Protocol Independent Switch Architecture 24 Programmable Parser Programmable Parser Memory Match+Action ALU

Programmable Parser Programmable Parser Match+Action

P4 Programming P4 code Compiler Programmable Parser Programmable Parser Memory Match+Action ALU

Questions 1.Why does the PISA (“RMT”) switch in the paper have ingress and egress pipeline stages? 2.Is the switch IQ, OQ or something else? 3.Why do we really need to add any new functionality to a switch? 27

Why now?

Switching Chip Area Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Serial I/O Parser

40% Serial I/O Switching Chip Area 40% Memory 10% Wire 10% Logic Wire Logic

2013: programmability cost 10-15% increase in area/power By 2018: Difference will be negligible

Language and compiler 32

Emre Orbay — The proposed compiler has to decide on a memory allocation from logical stages to physical stages involving match tables across stages whilst adhering to dependency constraints between stages. A random allocation would probably perform poorly for a sufficiently complicated configuration, so the compiler is now required to solve a bin-packing problem subject to additional constraints! If an exact solution is required (doubtful), compilation could take days! But that’s an issue for the compiler, not subject of the paper. 33

Queues Parser Fixed Action L2 Table Fixed Action IPv4 Table IPv6 Table ACL Table Match Table P4 ( parser parse_ethernet { extract(ethernet); select(latest.etherType) { 0x800 : parse_ipv4; 0x86DD : parse_ipv6; } parser parse_ethernet { extract(ethernet); select(latest.etherType) { 0x800 : parse_ipv4; 0x86DD : parse_ipv6; } table ipv4_lpm { reads { ipv4.dstAddr : lpm; } actions { set_next_hop; drop; } table ipv4_lpm { reads { ipv4.dstAddr : lpm; } actions { set_next_hop; drop; } control ingress { apply(l2_table); if (valid(ipv4)) { apply(ipv4_table); } if (valid(ipv6)) { apply(ipv6_table); } apply (acl); } control ingress { apply(l2_table); if (valid(ipv4)) { apply(ipv4_table); } if (valid(ipv6)) { apply(ipv6_table); } apply (acl); } L2 v4 v6 ACL Action Macro 34 Parser Match Action Tables Control Flow Graph

Question How can we exploit the parallelism within each stage? 35

Naïve Mapping: Control Flow Graph Parser Match Table Action Macro L2 v4 v6 ACL Control Flow Switch Pipeline Queues L2 Table IPv4 Table IPv6 Table ACL Table L2 v6 ACL v4 Action v4 Action Macro v6 Action Macro Action 36

Control Flow Graph L2 Table Dependency Graph (TDG) v4 v6 ACL L2 v4 v6 ACL Table Dependency Graph 37

Switch Pipeline Efficient Mapping: TDG Queues Parser L2 Table IPv4 Table IPv6 Table Table Dependency GraphControl Flow Graph L2 v4 v6 ACL L2 v4 v6 ACL Action v4 Action Macro v6 Action Macro 38 ACL Table Action

L2 Control Flow Graph Switch Pipeline Resource constraints v4 v6 ACL Queues Parser L2 Table IPv6 IPv4 L3 L2 v6 v4 L2 Action Macro v4 Action Macro v6 Action Macro Action ACL Table 39

Step 1: P4 Program Step 2: Control Flow Graph L2 v4 v6 ACL 40 Step 3: Table Dependency Graph L2 v4 v6 ACL Step 4: Table Configuration

RMT Switch … RMT 32 Stages

Example A Typical TDG 42 IPv6- Mcast EG-ACL1 EG-Phy- Meta IG-Agg- Intf IG-Dmac IPv4- Mcast IPv4- Nexthop IPv6- Nexthop IG-Props IG- Router- Mac Ipv4- Ecmp IG-Smac Ipv4- Ucast- LPM Ipv4- Ucast- Host Ipv6- Ucast- Host Ipv6- Ucast- LPM Ipv6- Ecmp IG_ACL2 IG_Bcast _Storm Ipv4_Ur pf Ipv6_Ur pf IG_ACL1 EG_Prop s IG_Phy_ Meta Configuration for RMT

Area Comparison with Fixed Function Switches SectionArea % of chipExtra Cost I/O, buffer, queue, CPU, etc37%0.0% Match memory & logic54.3%8.0% VLIW action engine7.4%5.5% Parser + deparser1.3%0.7% Total extra area cost14.2%

Area increase: the cost of flexibility 44 Serial I/O CPU Queues +8% Match Memory and Logic Match Memory and Logic +5.5% Action Engine +0.7% Parser Total: +14.2%

45 SectionPower % of chipExtra Cost I/O26.0%0.0% Memory leakage43.7%4.0% Logic leakage7.3%2.5% RAM active2.7%0.4% TCAM active3.5%0.0% Logic active16.8%5.5% Total extra power cost12.4% 45 Power Comparison with Fixed Function Switches