INTER-AUTONOMOUS SYSTEM MPLS VPN: ADVANCED CONCEPTS December 2003

Agenda Routing between sub-autonomous systems Inter-AS scaling Inter-AS filtering and route distribution Load balancing RT rewrite Services in Inter-AS Inter-AS and CSC comparison Inter-AS Summary

ROUTING BETWEEN SUB-AUTONOMOUS SYSTEMS Presentation_ID © 2003 Cisco Systems, Inc. All rights reserved. 3 3

Confederation Multiple IGP Domains Separate IGPs Each sub-confederations runs a single IGP Route-reflectors are used as peering points between sub-confederations for better scaling Next-hop self done by border routers on eBGP and iBGP sessions towards intra-confederation peers

Confederation Multiple IGP Domains Sub-AS1 with IGP-1 Sub-AS2 with IGP-2 Core of P LSRs Core of P LSRs MP-eBGP intra confederation for VPNv4 routes with label distribution MP-iBGP PE-1 PE-2 PE-3 CEGBP-1 CEGBP-2 PEs exchange VPNv4 addresses with labels Next-hop and labels are changed (next-hop self is used) PE1 and PE-2 addresses are known in both IGPs CE-1 CE-2 CE-5 CE-4 CE-3

Confederation Multiple IGP Domains (Cont.) Sub-AS1 with IGP-1 Sub-AS2 with IGP-2 Core of P LSRs Core of P LSRs Network=RD1:N Next-hop=PE1 Label=L1 Network=RD1:N Next-hop=RR2 Label=L3 PE-1 Network=RD1:N Next-hop=RR1 Label=L2 PE-2 PE-3 CEGBP-1 CEBGP-2 Network=N Next-hop=PE3 Network=N Next-hop=CE2 CE-2 CE-1 CE-5 CE-3 CE-4

Confederation Multiple IGP Domains: Important Points Route reflectors exchange routes Using Route reflectors is a natural approach since they already have all VPN routes Next-hop-self choices Option-1: eBGP only Option-2: eBGP and iBGP on border routers When next-hop self is used on both iBGP and eBGP sessions (in CEBGP-1 and CEBGP-2) the topology is similar to a Multi-provider-VPN topology

Confederation Multiple IGP Domains: Important Points (Cont.) CEBGP-1 and CEBGP-2 each need to be known in both IGPs CEBGP-1 and CEBGP-2 use interface addresses for their BGP session Label has to be bound on peer address; single label is used between sub-confederations Neighbor route needs to be known either a static router, or by using PPP neighbor-route discovery Implementation will create a neighbor route for the BGP peer address

SCALING INTER-PROVIDER SOLUTIONS Presentation_ID © 2003 Cisco Systems, Inc. All rights reserved. 9 9

PE-ASBR Memory Consumption VPNv4 MP-iBGP Sessions No. VPN Routes Memory Consumption

PE-ASBR Memory Scaling Potentially large amounts of VPN routing information that may not need to be carried on PE-ASBRs Large percentage will be local VPN prefixes PE-ASBRs must hold relevant VPN routing information such as VPN prefix details Two methods available to aid scaling ARF with local VRF import ARF disabled with inbound filtering

ARF with Local VRF Import Automatic Route Filtering (ARF) for non-imported routes If RT does not match locally configured import statement then drop the route Each PE-ASBR holds VRFs for Inter-AS VPNs and imports routes based on RT values PE-ASBR acts like normal PE routers with MP-eBGP sessions

ARF with Local VRF Import (Cont.) BGP Memory VRFs CEF Memory MP-iBGP VPNv4 Automatic Route Filtering MPLS Memory Routing Table Memory BGP, CEF, MPLS & RT Memory per-VRF

ARF Disabled With Inbound Filtering Automatic Route Filtering (ARF) enabled by default if no VRFs are configured then ALL VPN routes are dropped by the PE-ASBR Automatic Route Filtering may be disabled with no default BGP route-target filter command within the BGP configuration Disabling of ARF will cause ALL routes to be accepted by the PE-ASBR Additional filtering mechanisms should be used to drop unwanted routes

ARF Disabled With Inbound Filtering (Cont.) router bgp 1 ! no bgp default route-target filter address-family vpnv4 neighbor 154.27.0.134 activate neighbor 154.27.0.134 send-community extended neighbor 154.27.0.134 route-map vpn-routes-filter in BGP Memory MP-iBGP VPNv4 LFIB Memory NO Automatic Route Filtering VRF & CEF memory not required Routing Table memory not required NO per-VRF CEF or RT Memory, only BGP & LFIB

Next-Hop-Self Effect On LFIB BGP Memory 1000 prefixes BGP Memory 1000 prefixes BGP Memory 1000 prefixes LFIB Memory 1000 prefixes LFIB Memory 1000 prefixes LFIB memory 1 prefix for BGP next-hop With NHS Without NHS MP-iBGP VPNv4 Note that route/path combination in LFIB requires 168 bytes of memory – with 1000 prefixes this is 164k 1000 prefixes in total Next-hop-self increase amount of LFIB entries on receiving PE-ASBR

FILTERING AND ROUTER DISTRIBUTION MECHANISMS

Various Filtering Points In Inter-AS 4. Inbound filtering per-peer OR rr-group 3. Automatic route filtering inbound RR 1. Inbound filtering on PE-ASBR AS #200 RR 2. Outbound filtering per-peer PE PE AS #100 RR AS #300 PE 5. Automatic route filtering inbound

Inbound Filtering On PE-ASBR router bgp 1 ! no bgp default route-target filter address-family vpnv4 neighbor 154.27.0.134 activate neighbor 154.27.0.134 send-community extended neighbor 154.27.0.134 route-map vpn-routes-filter in ip extcommunity-list 1 permit rt 214:27 rt 214:94 route-map vpn-routes-filter permit 10 match extcommunity 1 Blue VPN routes discarded RT 214:129 NO Automatic Route Filtering BGP Memory RT 214:27 LFIB Memory RT 214:94 NO ARF – Filter inbound on per-peer basis

Outbound Filtering On PE-ASBR address-family vpnv4 neighbor 157.27.0.132 route-map MPeBGP-2 out neighbor 149.27.0.142 route-map MPeBGP-3 out ! route-map MPeBGP-2 permit 10 match extcommunity 214:27 route-map MPeBGP-3 permit 10 match extcommunity 214:94 RED VPN AS #200 BGP Table AS #300 GREEN VPN

Downstream RT Allocation Inbound and outbound filtering are restrictive with a large number of VPN clients Each RT must be known, and the filters must be established Changes to VPN client membership will cause configuration changes on PE-ASBRs Each filter must be updated to reflect the addition/deletion of VPN clients Simplified filtering scheme is needed with a large number of clients Provided with “downstream provider RT allocation” scheme

Downstream RT Allocation (Cont.) address-family vpnv4 neighbor 154.27.0.134 activate neighbor 154.27.0.134 send-community extended neighbor 154.27.0.134 route-map asbr-routes-filter in neighbor 157.27.0.132 route-map MPeBGP-2 out neighbor 149.27.0.142 route-map MPeBGP-3 out ! ip extcommunity-list 1 permit rt 129:101 rt 129:102 ip extcommunity-list 16 permit rt 129:101 ip extcommunity-list 17 permit rt 129:102 route-map asbr-routes-filter permit 10 match extcommunity 1 ! route-map MPeBGP-2 permit 10 match extcommunity 16 route-map MPeBGP-3 permit 10 match extcommunity 17 AS #200 RT 129:101 RED VPN Note that you need two export statements or you can use export maps without any route-target export commands within the VRF configuration AS #100 Export RT 129:12001 RT 129:101 Export RT 129:12090 RT 129:102 AS #300 RT 129:102 GREEN VPN GREEN VPN RT 129:12001 RED VPN RT 129:12090

LOAD BALANCING: DISTRIBUTION OF TRAFFIC LOAD BETWEEN PROVIDERS

Load Balancing Between Backbones Balancing of Inter-AS traffic is an important issue for distribution of traffic and redundancy of network design All Inter-AS traffic must pass through PE-ASBRs As BGP next-hops are reachable via these routers Multiple links provide traffic distribution These do not provide redundancy due to single point of failure of the PE-ASBR

VPN Client Traffic Flow PE-ASBR-1 VPN-v4 updates: NH=PE-ASBR-1 PE-ASBR-2 VPN-v4 update: RD:1:27:152.12.4.0/24, NH=PE-1 RT=1:222, Label=(L1) VPN-v4 updates: NH=PE-ASBR-2 ALL Inter-AS traffic flows across PE-ASBR-2 to PE-ASBR-1 link PE-1 PE-2 BGP, OSPF, RIPv2 152.12.4.0/24,NH=CE-2 CE-2 CE-3 VPN-B VPN-B 152.12.4.0/24 VPN Client to VPN Client traffic flow via Inter-AS Link

Load Balancing Between PE-ASBRs Network Y BGP NH=PE-ASBR-2 LO0 PE-ASBR-1 PE-ASBR-2 Routing Table PE-ASBR-2 LO0 via 193.1.1.9 via 193.1.1.13 via 193.1.1.17 193.1.1.9 Network Y 193.1.1.13 193.1.1.17 Static’s or IGP AND LDP Loopback Interface Loopback Interface Note that this requires Multi-HOP MP-eBGP between PE-ASBR routers. Also note CSCdt70169 as this configuration may be broken in current releases. BGP peering (Multi-HOP MP-eBGP) between loopbacks Load Balancing across multiple PE-ASBR links

Redundant PE-ASBR Connections RR will choose BGP best path and advertise only this path to receiving clients PE-ASBR-1 VPN-v4 updates: NH=PE-ASBR-1 PE-ASBR-2 VPN-v4 updates: NH=PE-ASBR-2 VPN-v4 update: RD:1:27:152.12.4.0/24, NH=PE-1 RT=1:222, Label=(L1) VPN-v4 updates: NH=PE-ASBR-4 PE-ASBR-3 VPN-v4 updates: NH=PE-ASBR-3 PE-ASBR-4 VPN-v4 updates: NH=PE-ASBR-4 PE-1 Note that local-preference or weight could also be used as a tool at PE-ASBR-2 and PE-ASBR-4. Inter-site traffic flow VPN-B VPN-B Redundant PE-ASBR used purely for backup

Redundant PE-ASBR Load Balancing VPN-v4 updates: NH=PE-ASBR-1 PE-ASBR-2 iBGP multipath support provides ability to load balance between two exit points VPN-v4 updates: NH=PE-ASBR-2 VPN-v4 update: RD:1:27:152.12.4.0/24, NH=PE-1 RT=1:222, Label=(L1) PE-ASBR-3 VPN-v4 updates: NH=PE-ASBR-3 VPN-v4 updates: NH=PE-ASBR-4 PE-1 This slide shows how to load balance across multiple PE-ASBR links but shows that route reflectors cannot be used in this case which 99% of the time is not going to be an acceptable deployment solution – therefore the ability to propagate multiple exit points via the route reflectors is a requirement Network 152.12.4.0/24 BGP NH=PE-ASBR-2 PE-ASBR-4 PE-ASBR-4 VPN-B VPN-B Load balancing PE-ASBR links without Route Reflectors

RT REWRITE

RT Rewrite RTs identify the VRF routing tables into which the prefix carried by the update is to be imported Carried as extended community attributes in bgp-vpnv4 updates RT Rewrites Supported for VRF export-maps Allow the replacement of route-targets on incoming and outgoing BGP updates Enables Service Providers to customize Route Targets within their network RT replacement can be performed at ASBRs exchanging VPNv4 prefixes RTs can also be replaced by PEs or RRs Both the above changes will require a slight modification of the BGP update processing path - Support for imposing RTs on incoming/outgoing updates. This is currently supported for VRF export-maps, but support needs to be extended to BGP inbound/outbound route-maps. The memory impact of this feature should be insignificant as it modifies the update itself without requiring the storage of the update. Also, other transient memory requirements are minimal. The performance impact will depend on the length of the route-map. For each update, to perform RT replacement, each extended-community list being used will have to be walked twice, once while matching and once while deleting the RT. The performance impact will depend on the product of the number of updates and the size of the route-map. This is true for any route-map related feature.

RT Rewrite Memory and Performance Impact Memory impact should be insignificant, as it modifies the update itself without requiring storage Other transient memory requirements are minimal Performance impact will depend on the product of the number of updates and the size (length, depth) of the route-map To perform RT replacement, each extended-community list is examined while matching and again while deleting the RT The memory impact of this feature should be insignificant as it modifies the update itself without requiring the storage of the update. Also, other transient memory requirements are minimal. The performance impact will depend on the length of the route-map. For each update, to perform RT replacement, each extended-community list being used will have to be walked twice, once while matching and once while deleting the RT. The performance impact will depend on the product of the number of updates and the size of the route-map. This is true for any route-map related feature.

RT Rewrite Sample Configuration Replace RT X with Y Use BGP inbound or outbound route-map at the receiving PE(ASBR, RR): ip extcommunity-list <X> permit rt c:d ! route-map extmap permit <#1> match extcommunity X set extcomm-list <X> delete set extcomm-list <Y> additive <!continue #2 to the next route-map if have more RT to change. Can use c:* for additional RTs> address family vpnv4 neighbor <ASBR IP#> route-map extmap <in/out>

RT Rewrite Verification Commands Verify route target replacement show ip bgp vpnv4  [all] Verifying the Route Target Replacement Policy debug ip bgp updates <ASBR IP Address> Router# <B&NBSP;STYLE="FONT-WEIGHT:&NBSP;BOLD">show ip bgp vpnv4 all 172.16.17.17 BGP routing table entry for 100:1:172.16.17.17/32, version 6 Paths: (1 available, best #1, no table)   Advertised to update-groups:      1            100 300     192.168.1.1 from 192.168.1.1 (172.16.13.13)       Origin incomplete, localpref 100, valid, external, best       Extended Community: RT:100:1 The following examples verify route target replacement on PE1 and PE2. Verify route target on PE1: Router# <B&NBSP;STYLE="FONT-WEIGHT:&NBSP;BOLD">show ip bgp vpnv4 all 172.16.17.17 BGP routing table entry for 100:1:172.16.17.17/32, version 13 Paths: (1 available, best #1, table vpn1)   Advertised to update-groups:      1            300     192.168.2.1 (via vpn1) from 192.168.2.1 (172.16.19.19)       Origin incomplete, metric 0, localpref 100, valid, external, best       Extended Community: RT:200:1 Verify route target on PE2: Router# <B&NBSP;STYLE="FONT-WEIGHT:&NBSP;BOLD">show ip bgp vpnv4 all 172.16.17.17 BGP routing table entry for 100:1:172.16.17.17/32, version 13 Paths: (1 available, best #1, table vpn1)   Advertised to update-groups:      3            100 300     192.168.1.1 (metric 20) from 172.16.16.16 (172.16.16.16)       Origin incomplete, localpref 100, valid, internal, best       Extended Community: RT:100:1 ========================================================

SHARED SERVICES IN INTER-AS

Supported Shared Services in Inter-AS Network Address Translation Address Translation at the egress point of the peering Service Provider is possible Redundancy (HSRP, VRRP, GLBP) Two ASBRs will reside in a single SP network IP Address Management and assignment DHCP, ODAP will be supported for Inter-AS Security AAA Servers Troubleshoot/Management Ping, Traceroute, SAA, Netflow

INTER-AS VERSUS CARRIER SUPPORTING CARRIER

CSC versus Inter-AS Carrier Supporting Carrier Inter-Provider Access Opportunity: Offer backbone services to peer or smaller carriers Inter-Provider Access Opportunity: Provide carrier services on behalf of other carriers Carrier A Backbone Carrier Customer Carrier A POP1 Customer Carrier A POP2 Carrier B

CSC versus Inter-AS (Cont.) Client-server topologies Peer-to-peer topologies ISP or MPLS VPN provider is a customer of another MPLS VPN backbone provider Two ISPs peer up providing services to some of the common customer base MPLS VPN backbone services needed between the same carrier POPs Single SP POPs not available in all geographical areas required by their customers Subscribing service provider may or may not have MPLS enabled Participating Providers must support MPLS VPNs Customers sites do not distribute reachability information to the backbone carrier Customers sites distribute reachability information directly to the participating service providers MPLS VPN in a BGP confederation

INTER-AS SUMMARY

Inter-AS Summary Service Providers have deployed Inter-AS for: Scalability purposes Partitioning the network based on services or management boundaries Some contract work is in progress amongst Service Providers to establish partnership and offer end-end VPN services to the common customer base Service Provider networks are completely separate Do not need to exchange internal prefix or label information Each Service Provider establishes a direct MP-eBGP session with the others to exchange VPN-IPv4 addresses with labels /32 route to reach the ASBR is created by default so ASBRs can communicate without a need for IGP Must be redistributed in the receiving Service Provider’s IGP

Inter-AS Summary (Cont.) IGP or LDP across ASBR links is not required Labels are already assigned to the routes when exchanged via MP-eBGP Interface used to establish MP-eBGP session does not need to be associated with a VRF Direct eBGP routes and labels can be exchanged. Next-Hop self can be turned on on ASBRs, enabling the ASBR to use its own address for next-hop Using the next-hop self requires an additional entry in the TFIB for each VPNv4 route (about 180) bytes If the Service Provider wishes to hide the Inter-AS link then use the next-hop-self method otherwise use the redistribute connected subnets method

Inter-AS Summary (Cont.) Multi-hop MP-eBGP sessions can be passed between Service Providers without conversions to VPNv4 routes Configuration of VRFs is not required on the ASBRs because bgp default route-target filter (automatic route filtering feature) has been disabled To conserve memory on both sides of the boundary and implement a simple form of security, always configure inbound route-maps to filter only routes that need to be passed to the other AS

References Inter-AS for MPLS VPNs CCO Documentation: www.cisco.com/univercd/cc/td/doc/product/software/ios121/ 121newft/121t/121t5/interas.htm MPLS and VPN architectures Jim Guichard/Ivan Pepelnjak ISBN 1-58705-002-1: www.ciscopress.com/book.cfm?book=168 Support for Inter-provider MPLS VPN ENG-48803 Dan Tappan, (internal only)