1. E-VPN on UW System Network Michael Hare

2. Purpose of presentation A high level introduction to E-VPN A simple lab demonstration For our documentation, see https://kb.wisc.edu/uwsysnet/internal/page.php?id=56133 https://kb.wisc.edu/uwsysnet/internal/page.php?id=56133 For a deeper dive, see http://www.juniper.net/us/en/training/jnbooks/day-one/proof- concept-labs/using-ethernet-vpns/

3. E-VPN MPLS service for loop free multipoing bridging using BGP as a control plane Similar deliverable as VPLS but with some additional features such as support for active/active CE multihoming Supports features such as default gateway sync, which is not discussed in this presentation

4. E-VPN basic theory of operation EVI: E-VPN instance, as defined on a PE. ES: Ethernet Segment. Ethernet link(s) between customer and provider network. When LAN is multihomed (multiple CE or multiple PE), the set of links is considered a single ES and must be assigned a network-wide unique ESI (Ethernet Segment Identifier). Many multihoming examples show a single CE directly connected to two PEs. However, an ES is also multihomed if a CE has a path to more than one PE through the CE's switched network. In latter case, single-active must be used (PEs for this ES cannot do active/active load balancing). ETI: Ethernet Tag Identifier, describes a unique broadcast domain in an EVI. For our purposes, ETI is a vlan ID. Note that an ETI inside an EVI may contain multiple unique ESI. DF: Designated Forwarder. Only for a multihomed ESI, DF replaces spanning tree and is responsible for forwarding multi-destination traffic (Broadcast, unknown unicast, and multicast traffic, or BUM). Non-DFs, or Backup Forwarders, drop BUM traffic on ingress.

6. Learning MAC addresses When a PE router detects a new MAC address on its EVI access interface, it adds the address to its appropriate local Layer 2 forwarding table, or MAC-VRF. The PE then transmits a MAC Advertisement route using MP-BGP to all remote PEs. The inclusion of the ESI in the MAC Advertisement route is critical for implementing aliasing, or load balancing. Multi-homed PEs advertise their connectivity to a common ESI by transmitting Auto-Discovery routes to all remote PEs. When a given remote PE subsequently learns of a MAC address from that ESI, it knows that the destination is reachable via the set of multi-homed PEs. The PE can then load balance traffic to the multiple PEs connected to the common ES. 2:143.235.32.38:700::200::00:24:97:32:a7:42/304 (1 entry, 0 announced) *BGP Preference: 170/-101 Route Distinguisher: 143.235.32.38:700 Task: BGP_65010.143.235.32.112+179 Communities: target:65010:700 Import Accepted Route Label: 306720 ESI: 00:00:07:00:00:00:00:00:00:00 Secondary Tables: EVPN-2.evpn.0

7. Looking at the bridging/forwarding tables From the bridging perspective m7h@r-mx104-lab-ac-re0> show bridge mac-table instance EVPN-2 … … MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Routing instance : EVPN-2 Bridging domain : V200, VLAN : 200 MAC MAC Logical NH RTR address flags interface Index ID 00:11:20:3e:3e:81 DC 1048594 1048594 00:11:20:3e:3e:c2 DC 1048594 1048594 00:24:97:32:a7:42 DC 1048577 1048577 … …

8. Looking at the bridging/forwarding tables (2) From the E-VPN database perspective m7h@r-mx104-lab-ac-re0> show evpn database instance EVPN-2 extensive Instance: EVPN-2 VLAN ID: 200, MAC address: 00:11:20:3e:3e:81 Source: 143.235.32.113, Rank: 1, Status: Active Timestamp: Sep 11 10:18:23 (0x55f2f0bf) State: VLAN ID: 200, MAC address: 00:11:20:3e:3e:c2 Source: 143.235.32.113, Rank: 1, Status: Active Timestamp: Sep 11 15:29:13 (0x55f33999) State: VLAN ID: 200, MAC address: 00:24:97:32:a7:42 Source: 00:00:07:00:00:00:00:00:00:00, Rank: 1, Status: Active Remote origin: 143.235.32.38 Timestamp: Sep 11 15:29:13 (0x55f33999) State:

9. Routing tables bgp.evpn.0: contains all EVPN related routes carried in BGP. You’ll see this on P and PE. To decode, see https://tools.ietf.org/html/rfc7432#section-7.https://tools.ietf.org/html/rfc7432#section-7 __default_evpn__.evpn.0: carries multihomed ES routes. You’ll see this on P and PE. $EVI.evpn.0 is like bgp.evpn.0 but is routing-instance specific. You’ll only see this in PE’s that are part of the given EVI.

10. E-VPN NLRI In E-VPN routes, the first integer is the E-VPN NLRI Route type. 1 - Ethernet Auto-Discovery (A-D) route [used for multihoming] 2 - MAC/IP Advertisement route [endpoint reachability info] 3 - Inclusive Multicast Ethernet Tag route [for BUM forwarding] 4 - Ethernet Segment route [DF/multihoming election]

11. Looking at the bridging/forwarding tables (3) m7h@r-mx104-lab-ac-re0> show route table EVPN-2.evpn.0 evpn-mac-address 00:24:97:32:a7:42 EVPN-2.evpn.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both … 2:143.235.32.38:700::200::00:24:97:32:a7:42/304 *[BGP/170] 00:03:47, localpref 100, from 143.235.32.112 AS path: I, validation-state: unverified > to 143.235.33.217 via ae0.3477 to 143.235.33.144 via xe-0/3/0.3475, Push 299888 …

12. Determining the DF for an ESI DF election is performed ESI per EVI. This facilitates load balancing of BUM traffic amongst PEs, a feature known as Service Carving. See https://tools.ietf.org/html/rfc7432#section-8.5 m7h@r-mx104-lab-ac-re0> show evpn instance EVPN-2 esi 00:00:07:00:00:00:00:00:00:00 extensive Instance: EVPN-2 … … Local interface: ae1.200, Status: Up/Blocking Number of remote PEs connected: 1 Remote PE MAC label Aliasing label Mode 143.235.32.38 306720 306720 single-active Designated forwarder: 143.235.32.38 Backup forwarder: 143.235.32.106 … …https://tools.ietf.org/html/rfc7432#section-8.5 Here is another way to check e-vpn forwarding state on a CE link m7h@r-mx104-lab-ac-re0# run show interfaces ae1.200 detail | match EVPN Protocol bridge, MTU: 1522, Generation: 222, Route table: 4, Mesh Group: __all_ces__, EVPN multi-homed status: Blocking

13. Config snippets: client handoff interface m7h@r-mx104-lab-ac-re0# show interfaces ae1 apply-groups-except ethernet-standards; description "s-lab-4 Port-channel1 uwplatteville switch 2 simulator"; enable; per-unit-scheduler; flexible-vlan-tagging; encapsulation flexible-ethernet-services; esi { <---------- omit this if single homed 00:00:00:00:00:00:00:00:07:00; single-active; } aggregated-ether-options { link-speed 1g; lacp { active; } } unit 200 { description "s-lab-4 Port-channel1 uwplatteville switch 2 simulator"; family bridge { interface-mode trunk; vlan-id-list [ 200 201 202 ]; } }

14. Config snippets: E-VPN instance m7h@r-mx104-lab-ac-re0# show routing-instances EVPN-2 instance-type virtual-switch; interface ae1.200; route-distinguisher 143.235.32.106:700; vrf-target target:65010:700; protocols { evpn { extended-vlan-list 200-202; default-gateway advertise; } } bridge-domains { V200 { vlan-id 200; } V201 { vlan-id 201; } V202 { vlan-id 202; } } Tracking ESI/RD/vrf targets: https://kb.wisc.edu/uwsysnet/internal/page.php?id=55002

15. Config snippets: BGP set protocols bgp group iBGP-reflector family evpn signaling m7h@r-mx2010-lab-re0# show policy-options policy-statement select-iBGP- reflector-routes-mx2010 … … term bgp-evpn { from { protocol bgp; rib bgp.evpn.0; } then next policy; }

16. Keeping it running Monitoring We track BGP NLRI counts and watch syslog.

17. That’s all, folks FIN