1. INTER-AUTONOMOUS SYSTEM MPLS VPN: ADVANCED CONCEPTS
December 2003

2. Agenda Routing between sub-autonomous systems Inter-AS scaling
Inter-AS filtering and route distribution
Load balancing
RT rewrite
Services in Inter-AS
Inter-AS and CSC comparison
Inter-AS Summary

3. ROUTING BETWEEN SUB-AUTONOMOUS SYSTEMS
Presentation_ID
© 2003 Cisco Systems, Inc. All rights reserved. 3 3

4. Confederation Multiple IGP Domains
Separate IGPs
Each sub-confederations runs a single IGP
Route-reflectors are used as peering points between sub-confederations for better scaling
Next-hop self done by border routers on eBGP and iBGP sessions towards intra-confederation peers

5. Confederation Multiple IGP Domains
Sub-AS1 with
IGP-1
Sub-AS2 with IGP-2
Core of P LSRs
Core of P LSRs
MP-eBGP intra confederation for VPNv4 routes with label distribution
MP-iBGP
PE-1
PE-2
PE-3
CEGBP-1
CEGBP-2
PEs exchange VPNv4 addresses with labels
Next-hop and labels are changed (next-hop self is used) PE1 and PE-2 addresses are known in both IGPs
CE-1
CE-2
CE-5
CE-4
CE-3

6. Confederation Multiple IGP Domains (Cont.)
Sub-AS1 with
IGP-1
Sub-AS2 with IGP-2
Core of P LSRs
Core of P LSRs
Network=RD1:N Next-hop=PE1 Label=L1
Network=RD1:N Next-hop=RR2 Label=L3
PE-1
Network=RD1:N Next-hop=RR1 Label=L2
PE-2
PE-3
CEGBP-1
CEBGP-2
Network=N Next-hop=PE3
Network=N Next-hop=CE2
CE-2
CE-1
CE-5
CE-3
CE-4

7. Confederation Multiple IGP Domains: Important Points
Route reflectors exchange routes
Using Route reflectors is a natural approach since they already have all VPN routes
Next-hop-self choices
Option-1: eBGP only
Option-2: eBGP and iBGP on border routers
When next-hop self is used on both iBGP and eBGP sessions (in CEBGP-1 and CEBGP-2) the topology is similar to a Multi-provider-VPN topology

8. Confederation Multiple IGP Domains: Important Points (Cont.)
CEBGP-1 and CEBGP-2 each need to be known in both IGPs
CEBGP-1 and CEBGP-2 use interface addresses for their BGP session
Label has to be bound on peer address; single label is used between sub-confederations
Neighbor route needs to be known either a static router, or by using PPP neighbor-route discovery
Implementation will create a neighbor route for the BGP peer address

9. SCALING INTER-PROVIDER SOLUTIONS
Presentation_ID
© 2003 Cisco Systems, Inc. All rights reserved. 9 9

10. PE-ASBR Memory Consumption
VPNv4 MP-iBGP Sessions
No. VPN Routes
Memory Consumption

11. PE-ASBR Memory Scaling
Potentially large amounts of VPN routing information that may not need to be carried on PE-ASBRs
Large percentage will be local VPN prefixes
PE-ASBRs must hold relevant VPN routing information such as VPN prefix details
Two methods available to aid scaling
ARF with local VRF import
ARF disabled with inbound filtering

12. ARF with Local VRF Import
Automatic Route Filtering (ARF) for non-imported routes
If RT does not match locally configured import statement then drop the route
Each PE-ASBR holds VRFs for Inter-AS VPNs and imports routes based on RT values
PE-ASBR acts like normal PE routers with MP-eBGP sessions

13. ARF with Local VRF Import (Cont.)
BGP Memory
VRFs
CEF Memory
MP-iBGP VPNv4
Automatic Route Filtering
MPLS Memory
Routing Table Memory
BGP, CEF, MPLS & RT Memory per-VRF

14. ARF Disabled With Inbound Filtering
Automatic Route Filtering (ARF) enabled by default
if no VRFs are configured then ALL VPN routes are dropped by the PE-ASBR
Automatic Route Filtering may be disabled with no default BGP route-target filter command within the BGP configuration
Disabling of ARF will cause ALL routes to be accepted by the PE-ASBR
Additional filtering mechanisms should be used to drop unwanted routes

15. ARF Disabled With Inbound Filtering (Cont.)
router bgp 1 !
no bgp default route-target filter
address-family vpnv4
neighbor activate
neighbor send-community extended
neighbor route-map vpn-routes-filter in
BGP Memory
MP-iBGP VPNv4
LFIB Memory
NO Automatic Route Filtering
VRF & CEF memory not required
Routing Table memory not required
NO per-VRF CEF or RT Memory, only BGP & LFIB

16. Next-Hop-Self Effect On LFIB
BGP Memory 1000 prefixes
BGP Memory 1000 prefixes
BGP Memory 1000 prefixes
LFIB Memory 1000 prefixes
LFIB Memory 1000 prefixes
LFIB memory 1 prefix for BGP next-hop
With NHS
Without NHS
MP-iBGP VPNv4
Note that route/path combination in LFIB requires 168 bytes of memory – with 1000 prefixes this is 164k
1000 prefixes in total
Next-hop-self increase amount of LFIB entries on receiving PE-ASBR

17. FILTERING AND ROUTER DISTRIBUTION MECHANISMS

18. Various Filtering Points In Inter-AS
4. Inbound filtering per-peer OR rr-group
3. Automatic route filtering inbound RR
1. Inbound filtering on PE-ASBR
AS #200 RR
2. Outbound filtering per-peer PE PE
AS #100 RR
AS #300 PE
5. Automatic route filtering inbound

19. Inbound Filtering On PE-ASBR
router bgp 1 !
no bgp default route-target filter
address-family vpnv4
neighbor activate
neighbor send-community extended
neighbor route-map vpn-routes-filter in
ip extcommunity-list 1 permit rt 214:27 rt 214:94
route-map vpn-routes-filter permit 10
match extcommunity 1
Blue VPN routes discarded
RT 214:129
NO Automatic Route Filtering
BGP Memory
RT 214:27
LFIB Memory
RT 214:94
NO ARF – Filter inbound on per-peer basis

20. Outbound Filtering On PE-ASBR
address-family vpnv4
neighbor route-map MPeBGP-2 out
neighbor route-map MPeBGP-3 out !
route-map MPeBGP-2 permit 10
match extcommunity 214:27
route-map MPeBGP-3 permit 10
match extcommunity 214:94
RED VPN
AS #200
BGP Table
AS #300
GREEN VPN

21. Downstream RT Allocation
Inbound and outbound filtering are restrictive with a large number of VPN clients
Each RT must be known, and the filters must be established
Changes to VPN client membership will cause configuration changes on PE-ASBRs
Each filter must be updated to reflect the addition/deletion of VPN clients
Simplified filtering scheme is needed with a large number of clients
Provided with “downstream provider RT allocation” scheme

22. Downstream RT Allocation (Cont.)
address-family vpnv4
neighbor activate
neighbor send-community extended
neighbor route-map asbr-routes-filter in
neighbor route-map MPeBGP-2 out
neighbor route-map MPeBGP-3 out !
ip extcommunity-list 1 permit rt 129:101 rt 129:102
ip extcommunity-list 16 permit rt 129:101
ip extcommunity-list 17 permit rt 129:102
route-map asbr-routes-filter permit 10
match extcommunity 1 !
route-map MPeBGP-2 permit 10
match extcommunity 16
route-map MPeBGP-3 permit 10
match extcommunity 17
AS #200
RT 129:101
RED VPN
Note that you need two export statements or you can use export maps without any route-target export commands within the VRF configuration
AS #100
Export RT 129:12001 RT 129:101
Export RT 129:12090 RT 129:102
AS #300
RT 129:102
GREEN VPN
GREEN VPN RT 129:12001
RED VPN RT 129:12090

23. LOAD BALANCING: DISTRIBUTION OF TRAFFIC LOAD BETWEEN PROVIDERS

24. Load Balancing Between Backbones
Balancing of Inter-AS traffic is an important issue for distribution of traffic and redundancy of network design
All Inter-AS traffic must pass through PE-ASBRs
As BGP next-hops are reachable via these routers
Multiple links provide traffic distribution
These do not provide redundancy due to single point of failure of the PE-ASBR

25. VPN Client Traffic Flow
PE-ASBR-1
VPN-v4 updates: NH=PE-ASBR-1
PE-ASBR-2
VPN-v4 update: RD:1:27: /24, NH=PE-1 RT=1:222, Label=(L1)
VPN-v4 updates: NH=PE-ASBR-2
ALL Inter-AS traffic flows across PE-ASBR-2 to PE-ASBR-1 link
PE-1
PE-2
BGP, OSPF, RIPv /24,NH=CE-2
CE-2
CE-3
VPN-B
VPN-B
/24
VPN Client to VPN Client traffic flow via Inter-AS Link

26. Load Balancing Between PE-ASBRs
Network Y
BGP NH=PE-ASBR-2 LO0
PE-ASBR-1
PE-ASBR-2
Routing Table
PE-ASBR-2 LO0 via
via
via
Network Y
Static’s or IGP AND LDP
Loopback Interface
Loopback Interface
Note that this requires Multi-HOP MP-eBGP between PE-ASBR routers. Also note CSCdt70169 as this configuration may be broken in current releases.
BGP peering (Multi-HOP MP-eBGP) between loopbacks
Load Balancing across multiple PE-ASBR links

27. Redundant PE-ASBR Connections
RR will choose BGP best path and advertise only this path to receiving clients
PE-ASBR-1
VPN-v4 updates: NH=PE-ASBR-1
PE-ASBR-2
VPN-v4 updates: NH=PE-ASBR-2
VPN-v4 update: RD:1:27: /24, NH=PE-1 RT=1:222, Label=(L1)
VPN-v4 updates: NH=PE-ASBR-4
PE-ASBR-3
VPN-v4 updates: NH=PE-ASBR-3
PE-ASBR-4
VPN-v4 updates: NH=PE-ASBR-4
PE-1
Note that local-preference or weight could also be used as a tool at PE-ASBR-2 and PE-ASBR-4.
Inter-site traffic flow
VPN-B
VPN-B
Redundant PE-ASBR used purely for backup

28. Redundant PE-ASBR Load Balancing
VPN-v4 updates: NH=PE-ASBR-1
PE-ASBR-2
iBGP multipath support provides ability to load balance between two exit points
VPN-v4 updates: NH=PE-ASBR-2
VPN-v4 update: RD:1:27: /24, NH=PE-1 RT=1:222, Label=(L1)
PE-ASBR-3
VPN-v4 updates: NH=PE-ASBR-3
VPN-v4 updates: NH=PE-ASBR-4
PE-1
This slide shows how to load balance across multiple PE-ASBR links but shows that route reflectors cannot be used in this case which 99% of the time is not going to be an acceptable deployment solution – therefore the ability to propagate multiple exit points via the route reflectors is a requirement
Network /24
BGP NH=PE-ASBR-2
PE-ASBR-4
PE-ASBR-4
VPN-B
VPN-B
Load balancing PE-ASBR links without Route Reflectors

29. RT REWRITE

30. RT Rewrite
RTs identify the VRF routing tables into which the prefix carried by the update is to be imported
Carried as extended community attributes in bgp-vpnv4 updates
RT Rewrites
Supported for VRF export-maps
Allow the replacement of route-targets on incoming and outgoing BGP updates
Enables Service Providers to customize Route Targets within their network
RT replacement can be performed at ASBRs exchanging VPNv4 prefixes
RTs can also be replaced by PEs or RRs
Both the above changes will require a slight modification of the BGP update processing path
- Support for imposing RTs on incoming/outgoing updates. This is currently supported for VRF export-maps, but support needs to be extended to BGP inbound/outbound route-maps.
The memory impact of this feature should be insignificant as it modifies the update itself without requiring the
storage of the update. Also, other transient memory requirements are minimal. The performance impact will depend on the length of the route-map. For each update, to perform RT replacement, each extended-community list being used will have to be walked twice, once while matching and once while deleting the RT. The performance impact will depend on the product of the number of updates and the size of the route-map. This is true for any route-map related feature.

31. RT Rewrite Memory and Performance Impact
Memory impact should be insignificant, as it modifies the update itself without requiring storage
Other transient memory requirements are minimal
Performance impact will depend on the product of the number of updates and the size (length, depth) of the route-map
To perform RT replacement, each extended-community list is examined while matching and again while deleting the RT
The memory impact of this feature should be insignificant as it modifies the update itself without requiring the storage of the update. Also, other transient memory requirements are minimal. The performance impact will depend on the length of the route-map. For each update, to perform RT replacement, each extended-community list being used will have to be walked twice, once while matching and once while deleting the RT. The performance impact will depend on the product of the number of updates and the size of the route-map. This is true for any route-map related feature.

32. RT Rewrite Sample Configuration Replace RT X with Y
Use BGP inbound or outbound route-map at the receiving PE(ASBR, RR):
ip extcommunity-list <X> permit rt c:d !
route-map extmap permit <#1>
match extcommunity X
set extcomm-list <X> delete
set extcomm-list <Y> additive
<!continue #2 to the next route-map if have more
RT to change. Can use c:* for additional RTs>
address family vpnv4
neighbor <ASBR IP#> route-map extmap <in/out>

33. RT Rewrite Verification Commands
Verify route target replacement
show ip bgp vpnv4  [all]
Verifying the Route Target Replacement Policy
debug ip bgp updates <ASBR IP Address>
Router# <B&NBSP;STYLE="FONT-WEIGHT:&NBSP;BOLD">show ip bgp vpnv4 all  BGP routing table entry for 100:1: /32, version 6 Paths: (1 available, best #1, no table)   Advertised to update-groups:      1            100 300       from   ( )       Origin incomplete, localpref 100, valid, external, best       Extended Community: RT:100:1 The following examples verify route target replacement on PE1 and PE2.
Verify route target on PE1:
Router# <B&NBSP;STYLE="FONT-WEIGHT:&NBSP;BOLD">show ip bgp vpnv4 all  BGP routing table entry for 100:1: /32, version 13 Paths: (1 available, best #1, table vpn1)   Advertised to update-groups:      1            300       (via vpn1) from   ( )       Origin incomplete, metric 0, localpref 100, valid, external, best       Extended Community: RT:200:1 Verify route target on PE2:
Router# <B&NBSP;STYLE="FONT-WEIGHT:&NBSP;BOLD">show ip bgp vpnv4 all  BGP routing table entry for 100:1: /32, version 13 Paths: (1 available, best #1, table vpn1)   Advertised to update-groups:      3            100 300       (metric 20) from   ( )       Origin incomplete, localpref 100, valid, internal, best       Extended Community: RT:100:1
========================================================

34. SHARED SERVICES IN INTER-AS

35. Supported Shared Services in Inter-AS
Network Address Translation
Address Translation at the egress point of the peering Service Provider is possible
Redundancy (HSRP, VRRP, GLBP)
Two ASBRs will reside in a single SP network
IP Address Management and assignment
DHCP, ODAP will be supported for Inter-AS
Security
AAA Servers
Troubleshoot/Management
Ping, Traceroute, SAA, Netflow

36. INTER-AS VERSUS CARRIER SUPPORTING CARRIER

37. CSC versus Inter-AS Carrier Supporting Carrier Inter-Provider Access
Opportunity: Offer backbone services to peer or smaller carriers
Inter-Provider Access
Opportunity: Provide carrier services on behalf of other carriers
Carrier A
Backbone
Carrier
Customer
Carrier A
POP1
Customer
Carrier A
POP2
Carrier B

38. CSC versus Inter-AS (Cont.)
Client-server topologies
Peer-to-peer topologies
ISP or MPLS VPN provider is a customer of another MPLS VPN backbone provider
Two ISPs peer up providing services to some of the common customer base
MPLS VPN backbone services needed between the same carrier POPs
Single SP POPs not available in all geographical areas required by their customers
Subscribing service provider may or may not have MPLS enabled
Participating Providers must support MPLS VPNs
Customers sites do not distribute reachability information to the backbone carrier
Customers sites distribute reachability information directly to the participating service providers
MPLS VPN in a BGP confederation

39. INTER-AS SUMMARY

40. Inter-AS Summary Service Providers have deployed Inter-AS for:
Scalability purposes
Partitioning the network based on services or management boundaries
Some contract work is in progress amongst Service Providers to establish partnership and offer end-end VPN services to the common customer base
Service Provider networks are completely separate
Do not need to exchange internal prefix or label information
Each Service Provider establishes a direct MP-eBGP session with the others to exchange VPN-IPv4 addresses with labels
/32 route to reach the ASBR is created by default so ASBRs can communicate without a need for IGP
Must be redistributed in the receiving Service Provider’s IGP

41. Inter-AS Summary (Cont.)
IGP or LDP across ASBR links is not required
Labels are already assigned to the routes when exchanged via MP-eBGP
Interface used to establish MP-eBGP session does not need to be associated with a VRF
Direct eBGP routes and labels can be exchanged.
Next-Hop self can be turned on on ASBRs, enabling the ASBR to use its own address for next-hop
Using the next-hop self requires an additional entry in the TFIB for each VPNv4 route (about 180) bytes
If the Service Provider wishes to hide the Inter-AS link then use the next-hop-self method otherwise use the redistribute connected subnets method

42. Inter-AS Summary (Cont.)
Multi-hop MP-eBGP sessions can be passed between Service Providers without conversions to VPNv4 routes
Configuration of VRFs is not required on the ASBRs because bgp default route-target filter (automatic route filtering feature) has been disabled
To conserve memory on both sides of the boundary and implement a simple form of security, always configure inbound route-maps to filter only routes that need to be passed to the other AS

43. References Inter-AS for MPLS VPNs CCO Documentation:
121newft/121t/121t5/interas.htm
MPLS and VPN architectures Jim Guichard/Ivan Pepelnjak ISBN :
Support for Inter-provider MPLS VPN ENG Dan Tappan, (internal only)