MPLS VPN TOI eosborne@cisco.com.

MPLS VPN TOI

Agenda How MPLS VPN works What Code Is MPLS VPN In?
Platform Issues in Implementation Lab Demo - config

How MPLS-VPN Works Concepts and goals Terminology Connection model
Forwarding Mechanisms Topologies Scaling Configuration

MPLS-VPN What is a VPN ? An IP network infrastructure delivering private network services over a public infrastructure Use a layer 3 backbone Scalability, easy provisioning Global as well as non-unique private address space QoS Controlled access Easy configuration for customers

VPN Models - The Overlay model
Private trunks over a TELCO/SP shared infrastructure Leased/Dialup lines FR/ATM circuits IP (GRE) tunnelling Transparency between provider and customer networks Optimal routing requires full mesh over over backbone

VPN Models - The Peer model
Both provider and customer network use same network protocol CE and PE routers have a routing adjacency at each site All provider routers hold the full routing information about all customer networks Private addresses are not allowed May use the virtual router capability Multiple routing and forwarding tables based on Customer Networks

VPN Models - MPLS-VPN: The True Peer model
Same as Peer model BUT !!! Provider Edge routers receive and hold routing information only about VPNs directly connected Reduces the amount of routing information a PE router will store Routing information is proportional to the number of VPNs a router is attached to MPLS is used within the backbone to switch packets (no need of full routing)

Agenda Concepts and goals Terminology Connection model Forwarding
Mechanisms Topologies Scaling Configuration

MPLS-VPN Terminology Provider Network (P-Network)
The backbone under control of a Service Provider Customer Network (C-Network) Network under customer control CE router Customer Edge router. Part of the C-network and interfaces to a PE router

MPLS-VPN Terminology Site PE router P router
Set of (sub)networks part of the C-network and co-located A site is connected to the VPN backbone through one or more PE/CE links PE router Provider Edge router. Part of the P-Network and interfaces to CE routers P router Provider (core) router, without knowledge of VPN

MPLS-VPN Terminology Border router Extended Community
Provider Edge router interfacing to other provider networks Extended Community BGP attribute used to identify a Route-origin, Route-target Site of Origin Identifier (SOO) 64 bits identifying routers where the route has been originated

MPLS-VPN Terminology Route-Target Route Distinguisher
64 bits identifying routers that should receive the route Route Distinguisher Attributes of each route used to uniquely identify prefixes among VPNs (64 bits) VRF based (not VPN based) VPN-IPv4 addresses Address including the 64 bits Route Distinguisher and the 32 bits IP address

MPLS-VPN Terminology VRF VPN-Aware network
VPN Routing and Forwarding Instance Routing table and FIB table Populated by routing protocol contexts VPN-Aware network A provider backbone where MPLS-VPN is deployed

MPLS VPN Connection Model
A VPN is a collection of sites sharing a common routing information (routing table) A site can be part of different VPNs A VPN has to be seen as a community of interest (or Closed User Group) Multiple Routing/Forwarding instances (VRF) on PE routers

Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B A site belonging to different VPNs may or MAY NOT be used as a transit point between VPNs If two or more VPNs have a common site, address space must be unique among these VPNs

The VPN backbone is composed by MPLS LSRs PE routers (edge LSRs) P routers (core LSRs) PE routers are faced to CE routers and distribute VPN information through MP-BGP to other PE routers VPN-IPv4 addresses, Extended Community, Label P routers do not run BGP and do not have any VPN knowledge

VPN_A VPN_B CE PE iBGP sessions VPN_A CE VPN_A P P PE CE P P VPN_B PE CE P routers (LSRs) are in the core of the MPLS cloud PE routers use MPLS with the core and plain IP with CE routers P and PE routers share a common IGP PE router are MP-iBGP fully meshed

PE CE Site-2 Site-1 EBGP,OSPF, RIPv2,Static PE and CE routers exchange routing information through: EBGP, OSPF, RIPv2, Static routing CE router run standard routing software

CE Site-1 PE EBGP,OSPF, RIPv2,Static CE VPN Backbone IGP (OSPF, ISIS) Site-2 PE routers maintain separate routing tables The global routing table With all PE and P routes Populated by the VPN backbone IGP (ISIS or OSPF) VRF (VPN Routing and Forwarding) Routing and Forwarding table associated with one or more directly connected sites (CEs) VRF are associated to (sub/virtual/tunnel)interfaces Interfaces may share the same VRF if the connected sites may share the same routing information

CE Site-1 PE CE Site-2 Different site sharing the same routing information, may share the same VRF Interfaces connecting these sites will use the same VRF Sites belonging to the same VPN may share same VRF

CE Site-1 PE EBGP,OSPF, RIPv2,Static VPN Backbone IGP CE Site-2 The routes the PE receives from CE routers are installed in the appropriate VRF The routes the PE receives through the backbone IGP are installed in the global routing table By using separate VRFs, addresses need NOT to be unique among VPNs

The Global Routing Table is populated by IGP protocols. In PE routers it may contain the BGP Internet routes (standard BGP-4 routes) BGP-4 (IPv4) routes go into global routing table MP-BGP (VPN-IPv4) routes go into VRFs

PE PE VPN Backbone IGP P P iBGP session PE and P routers share a common IGP (ISIS or OSPF) PEs establish MP-iBGP sessions between them PEs use MP-BGP to exchange routing information related to the connected sites and VPNs VPN-IPv4 addresses, Extended Community, Label

MPLS VPN Connection Model MP-BGP Update
VPN-IPV4 address Route Distinguisher 64 bits Makes the IPv4 route globally unique RD is configured in the PE for each VRF RD may or may not be related to a site or a VPN IPv4 address (32bits) Extended Community attribute (64 bits) Site of Origin (SOO): identifies the originating site Route-target (RT): identifies the set of sites the route has to be advertised to

Any other standard BGP attribute Local Preference MED Next-hop AS_PATH Standard Community ... A Label identifying: The outgoing interface The VRF where a lookup has to be done (aggregate label) The BGP label will be the second label in the label stack of packets travelling in the core

MPLS VPN Connection Model MP-BGP Update - Extended community
BGP extended community attribute Structured, to support multiple applications 64 bits for increased range General form <16bits type>:<ASN>:<32 bit number> Registered AS number <16bits type>:<IP address>:<16 bit number> Registered IP address

MPLS VPN Connection Model MP-BGP Update - Extended community
The Extended Community is used to: Identify one or more routers where the route has been originated (site) Site of Origin (SOO) Selects sites which should receive the route Route-Target

The Label can be assigned only by the router which address is the Next-Hop attribute PE routers re-write the Next-Hop with their own address (loopback interface address) “Next-Hop-Self” BGP command towards iBGP neighbors Loopback addresses are advertised into the backbone IGP PE addresses used as BGP Next-Hop must be uniquely known in the backbone IGP No summarisation of loopback addresses in the core

VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2 P P PE-1 PE-2 Site-2 CE-2 VPN Backbone IGP BGP,RIPv2 update for Net1,Next-Hop=CE-1 P P Site-1 CE-1 VPN-IPv4 update: RD:Net1, Next-hop=PE-1 SOO=Site1, RT=Green, Label=(intCE1) PE routers receive IPv4 updates (EBGP, RIPv2, Static) PE routers translate into VPN-IPv4 Assign a SOO and RT based on configuration Re-write Next-Hop attribute Assign a label based on VRF and/or interface Send MP-iBGP update to all PE neighbors

VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2 P P PE-1 PE-2 Site-2 CE-2 BGP,OSPF, RIPv2 update for Net1 Next-Hop=CE-1 VPN Backbone IGP P P Site-1 CE-1 VPN-IPv4 update: RD:Net1, Next-hop=PE-1 SOO=Site1, RT=Green, Label=(intCE1) Receiving PEs translate to IPv4 Insert the route into the VRF identified by the RT attribute (based on PE configuration) The label associated to the VPN-IPv4 address will be set on packet forwarded towards the destination

Route distribution to sites is driven by the Site of Origin (SOO) and Route-target attributes BGP Extended Community attribute A route is installed in the site VRF corresponding to the Route-target attribute Driven by PE configuration A PE which connects sites belonging to multiple VPNs will install the route into the site VRF if the Route-target attribute contains one or more VPNs to which the site is associated

MPLS Forwarding Packet forwarding
PE and P routers have BGP next-hop reachability through the backbone IGP Labels are distributed through LDP (hop-by-hop) corresponding to BGP Next-Hops Label Stack is used for packet forwarding Top label indicates BGP Next-Hop (interior label) Second level label indicates outgoing interface or VRF (exterior label)

MPLS Forwarding Packet forwarding
MPLS nodes forward packets based on the top label P routers do not have BGP (nor VPN) knowledge No VPN routing information No Internet routing information

MPLS Forwarding Penultimate Hop Popping
The upstream LDP peer of the BGP next-hop (PE router) will pop the first level label The penultimate hop will pop the label Requested through LDP The egress PE router will forward the packet based on the second level label which gives the outgoing interface (and VPN)

MPLS Forwarding MPLS Forwarding - Penultimate Hop Popping
IGP Label(PE2) VPN Label IP packet P routers switch the packets based on the IGP label (label on top of the stack) IP packet PE2 receives the packets with the label corresponding to the outgoing interface (VRF) One single lookup Label is popped and packet sent to IP neighbor CE1 VPN Label IP packet Penultimate Hop Popping P2 is the penultimate hop for the BGP next-hop P2 remove the top label This has been requested through LDP by PE2 IP packet PE1 CE2 IGP Label(PE2) VPN Label IP packet PE1 receives IP packet Lookup is done on site VRF BGP route with Next-Hop and Label is found BGP next-hop (PE2) is reachable through IGP route with associated label P1 P2 PE2 CE3

MPLS VPN Forwarding VPN_A VPN_B CE PE1 PE2 VPN_A CE VPN_A P P PE CE T8T2 Data P P Data VPN_B CE <RD_B,10.2> , iBGP NH= PE2 , T2 T8 <RD_B,10.1> , iBGP next hop PE1 <RD_B,10.2> , iBGP next hop PE2 <RD_B,10.3> , iBGP next hop PE3 <RD_A,11.6> , iBGP next hop PE1 <RD_A,10.1> , iBGP next hop PE4 <RD_A,10.4> , iBGP next hop PE4 <RD_A,10.2> , iBGP next hop PE2 T1 T7 T2 T8 T3 T9 T4 T7 T5 TB T6 TB T7 T8 Ingress PE receives normal IP Packets from CE router PE router does “IP Longest Match” from VPN_B FIB , find iBGP next hop PE2 and impose a stack of labels: exterior Label T2 + Interior Label T8

MPLS VPN Forwarding in / out
VPN_A VPN_B CE PE1 PE2 VPN_A CE Data T2 Data TB T2 Data VPN_A P P PE CE TAT2 Data T8 T2 Data P P VPN_B CE in / out T7 T8 T9 Ta Tb T8, TA Tu Tw Tx Ty Tz All Subsequent P routers do switch the packet Solely on Interior Label Egress PE router, removes Interior Label Egress PE uses Exterior Label to select which VPN/CE to forward the packet to. Exterior Label is removed and packet routed to CE router

MPLS VPN mechanisms VRF and Multiple Routing Instances
VRF: VPN Routing and Forwarding Instance VRF Routing Protocol Context VRF Routing Tables VRF CEF Forwarding Tables

VPN aware Routing Protocols Select/Install routes in appropriate routing table Per-instance router variables Not necessarily per-instance routing processes eBGP, OSPF, RIPv2, Static

VRF Routing table contains routes which should be available to a particular set of sites Analogous to standard IOS routing table, supports the same set of mechanisms Interfaces (sites) are assigned to VRFs One VRF per interface (sub-interface, tunnel or virtual-template) Possible many interfaces per VRF

Static BGP RIP Routing processes Routing contexts VRF Routing tables VRF Forwarding tables Routing processes run within specific routing contexts Populate specific VPN routing table and FIBs (VRF) Interfaces are assigned to VRFs

Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B Logical view Multihop MP-iBGP P P PE PE Routing view VRF for site-1 Site-1 routes Site-2 routes VRF for site-2 Site-1 routes Site-2 routes Site-3 routes VRF for site-3 Site-2 routes Site-3 routes Site-4 routes VRF for site-4 Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

Mechanisms Topologies Scaling BGP-4 Enhancements Cap. Negotiation, MPLS, Route Refresh, ORF Configuration

MPLS VPN Topologies iBGP sessions VPN_A VPN_B CE PE VPN_A CE VPN_A P P PE CE P P VPN_B PE CE VPN-IPv4 address are propagated together with the associated label in BGP Multiprotocol extension Extended Community attribute (route-target) is associated to each VPN-IPv4 address, to populate the site VRF

MPLS VPN Topologies VPN sites with optimal intra-VPN routing
Each site has full routing knowledge of all other sites (of same VPN) Each CE announces his own address space MP-BGP VPN-IPv4 updates are propagated between PEs Routing is optimal in the backbone Each route has the BGP Next-Hop closest to the destination No site is used as central point for connectivity

MPLS VPN Topologies VPN sites with optimal intra-VPN routing
EBGP/RIP/Static N3 NH=CE3 Routing Table on CE3 N1, PE3 N2, PE3 N3, Local VRF for site-3 N1,NH=PE1 N2,NH=PE2 N3,NH=CE3 IntCE3 PE3 VPN-IPv4 updates exchanged between PEs RD:N1, NH=PE1,Label=IntCE1, RT=Blue RD:N2, NH=PE2,Label=IntCE2, RT=Blue RD:N3, NH=PE3,Label=IntCE3, RT=Blue N2,NH=CE2 EBGP/RIP/Static PE1 IntCE2 VRF for site-1 N1,NH=CE1 N2,NH=PE2 N3,NH=PE3 VRF for site-2 N1,NH=PE1 N2,NH=CE2 N3,NH=PE3 Site-2 PE2 IntCE1 N2 Routing Table on CE2 N1,NH=PE2 N2,Local N3,NH=PE2 Routing Table on CE1 N1, Local N2, PE1 N3, PE1 EBGP/RIP/Static N1 NH=CE1 Site-1 N1

MPLS VPN Topologies VPN sites with Hub & Spoke routing
One central site has full routing knowledge of all other sites (of same VPN) Hub-Site Other sites will send traffic to Hub-Site for any destination Spoke-Sites Hub-Site is the central transit point between Spoke-Sites Use of central services at Hub-Site

VPN-IPv4 update advertised by PE1 RD:N1, NH=PE1,Label=IntCE1, RT=Hub Site-1 N1 IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported BGP/RIPv2 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE3-Spoke PE2 CE2 BGP/RIPv2 VPN-IPv4 updates advertised by PE3 RD:N1, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N2, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N3, NH=PE3,Label=IntCE3-Spoke, RT=Spoke VPN-IPv4 update advertised by PE2 RD:N2, NH=PE2,Label=IntCE2, RT=Hub Routes are imported/exported into VRFs based on RT value of the VPN-IPv4 updates PE3 uses 2 (sub)interfaces with two different VRFs

IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported Site-1 N1 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 BGP/RIPv2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 CE3-Spoke PE2 IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE2 BGP/RIPv2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) Traffic from one spoke to another will travel across the hub site Hub site may host central services Security, NAT, centralised Internet access

If PE and Hub-site use BGP the PE should not check the received AS_PATH The update the Hub-site advertise contains the VPN backbone AS number By configuration the AS_PATH check is disabled Routing loops are detected through the SOO attribute PE and CE routers may use RIPv2 and/or static routing

MPLS VPN Internet Routing
In a VPN, sites may need to have Internet connectivity Connectivity to the Internet means: Being able to reach Internet destinations Being able to be reachable from any Internet source Security mechanism MUST be used as in ANY other kind of Internet connectivity

MPLS VPN Internet Routing
The Internet routing table is treated separately In the VPN backbone the Internet routes are in the Global routing table of PE routers Labels are not assigned to external (BGP) routes P routers need not (and will not) run BGP

MPLS VPN Internet routing VRF specific default route
A default route is installed into the site VRF and pointing to a Internet Gateway The default route is NOT part of any VPN A single label is used for packets forwarded according to the default route The label is the IGP label corresponding to the IP address of the Internet gateway Known in the IGP

PE router originates CE routes for the Internet Customer (site) routes are known in the site VRF Not in the global table The PE/CE interface is NOT known in the global table. However: A static route for customer routes and pointing to the PE/CE interface is installed in the global table This static route is redistributed into BGP-4 global table and advertised to the Internet Gateway The Internet gateway knows customer routes and with the PE address as next-hop

The Internet Gateway specified in the default route (into the VRF) need NOT to be directly connected Different Internet gateways can be used for different VRFs Using default route for Internet routing does NOT allow any other default route for intra-VPN routing As in any other routing scheme

ip vrf VPN-A rd 100:1 route-target both 100:1 ! Interface Serial0 ip address ip vrf forwarding VPN-A Router bgp 100 no bgp default ipv4-unicast network mask neighbor remote 100 neighbor activate neighbor next-hop-self neighbor update-source loopback0 address-family ipv4 vrf VPN-A neighbor remote-as 65502 neighbor activate exit-address-family address-family vpnv4 neighbor activate ip route Serial0 ip route vrf VPN-A global BGP-4 Internet PE-IG MP-BGP PE PE Serial0 Site-1 Network /16 Site-2

IP packet D=cisco.com Internet PE-IG Label = 3 IP packet D=cisco.com Global Table and LFIB /32 Label=3 /32 Label=5 ... PE PE Site-2 VRF / (global) Site-1 routes Site-2 routes Serial0 IP packet D=cisco.com Site-1 Network /16 Site-2

PE routers need not to hold the Internet table PE routers will use BGP-4 sessions to originate customer routes Packet forwarding is done with a single label identifying the Internet Gateway IP address More labels if Traffic Engineering is used

MPLS VPN Internet Routing Separated (sub)interfaces
If CE wishes to receive and announce routes from/to the Internet A dedicated BGP session is used over a separate (sub) interface The PE imports CE routes into the global routing table and advertise them to the Internet The interface is not part of any VPN and does not use any VRF Default route or Internet routes are exported to the CE PE needs to have Internet routing table

The PE uses separate (sub)interfaces with the CE One (sub)interface for VPN routing associated to a VRF Can be a tunnel interface One (sub)interface for Internet routing Associated to the global routing table

ip vrf VPN-A rd 100:1 route-target both 100:1 ! Interface Serial0 no ip address Interface Serial0.1 ip address ip vrf forwarding VPN-A Interface Serial0.2 ip address Router bgp 100 no bgp default ipv4-unicast neighbor remote 100 neighbor activate neighbor next-hop-self neighbor update-source loopback0 neighbor remote 502 address-family ipv4 vrf VPN-A neighbor remote-as 502 neighbor activate exit-address-family address-family vpnv4 neighbor activate BGP-4 Internet PE-IG MP-BGP PE PE Serial0.1 Serial0.2 BGP-4 Site-1 Network /16 Site-2

IP packet D=cisco.com Internet PE-IG Label = 3 IP packet D=cisco.com PE Global Table Internet routes ---> , Label=3 PE PE Serial0.1 Serial0.2 IP packet D=cisco.com Serial0.1 Serial0.2 Site-1 CE routing table Site-2 routes ----> Serial0.1 Internet routes ---> Serial0.2 Network /16 Site-2

Scaling Existing BGP techniques can be used to scale the route distribution: route reflectors Each edge router needs only the information for the VPNs it supports Directly connected VPNs RRs are used to distribute VPN routing information

Scaling Very highly scalable: Easy to add new sites
Initial VPN release: 1000 VPNs x 1000 sites/VPN = 1,000,000 sites Architecture supports 100,000+ VPNs, 10,000,000+ sites BGP “segmentation” through RRs is essential !!!! Easy to add new sites configure the site on the PE connected to it the network automagically does the rest See also platform issues, later on

MPLS-VPN Scaling BGP Route Reflectors may be partitioned
VPN_A VPN_B P PE CE RR Route Reflectors PE1 PE2 Route Reflectors may be partitioned Each RR store routes for a set of VPNs Thus, no BGP router needs to store ALL VPNs information PEs will peer to RRs according to the VPNs they directly connect

MPLS-VPN Scaling BGP updates filtering
iBGP full mesh between PEs results in flooding all VPNs routes to all PEs Scaling problems when large amount of routes. In addition PEs need only routes for attached VRFs Therefore each PE will discard any VPN-IPv4 route that hasn’t a route-target configured to be imported in any of the attached VRFs This reduces significantly the amount of information each PE has to store Volume of BGP table is equivalent of volume of attached VRFs (nothing more)

MPLS-VPN Scaling BGP updates filtering
Import RT=yellow VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ VRFs for VPNs yellow green PE MP-iBGP sessions VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green Each VRF has an import and export policy configured Policies use route-target attribute (extended community) PE receives MP-iBGP updates for VPN-IPv4 routes If route-target is equal to any of the import values configured in the PE, the update is accepted Otherwise it is silently discarded

MPLS-VPN Scaling Route Refresh
VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ Import RT=yellow 2. PE issue a Route-Refresh to all neighbors in order to ask for re-transmission PE VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green 1. PE doesn’t have red routes (previously filtered out) Import RT=red 3. Neighbors re-send updates and “red” route-target is now accepted Policy may change in the PE if VRF modifications are done New VRFs, removal of VRFs However, the PE may not have stored routing information which become useful after a change PE request a re-transmission of updates to neighbors Route-Refresh

MPLS-VPN Scaling Outbound Route Filters - ORF
VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ Import RT=yellow 2. PE issue a ORF message to all neighbors in order not to receive red routes PE VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green 1. PE doesn’t need red routes 3. Neighbors dynamically configure the outbound filter and send updates accordingly PE router will discard update with unused route-target Optimisation requires these updates NOT to be sent Outbound Route Filter (ORF) allows a router to tell its neighbors which filter to use prior to propagate BGP updates

Mechanisms Topologies Scaling BGP-4 Enhancements Cap. Negotiation, MPLS, Route Refresh, ORF Configuration

MPLS VPN - Configuration
VPN knowledge is on PE routers PE router have to be configured for VRF and Route Distinguisher VRF import/export policies (based on Route-target) Routing protocol used with CEs MP-BGP between PE routers BGP for Internet routers With other PE routers With CE routers

MPLS VPN - Configuration VRF and Route Distinguisher
RD is configured on PE routers (for each VRF) VRFs are associated to RDs in each PE Common (good) practice is to use the same RD for the same VPN in all PEs But not mandatory VRF configuration command ip vrf <vrf-symbolic-name> rd <route-distinguisher-value> route-target import <community> route-target export <community>

CLI - VRF configuration
ip vrf site1 rd 100:1 route-target export 100:1 route-target import 100:1 ip vrf site2 rd 100:2 route-target export 100:2 route-target import 100:2 Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B ip vrf site3 rd 100:3 route-target export 100:2 route-target import 100:2 route-target import 100:3 route-target export 100:3 ip vrf site-4 rd 100:4 route-target export 100:3 Multihop MP-iBGP P P PE1 PE2 VRF for site-1 (100:1) Site-1 routes Site-2 routes VRF for site-2 (100:2) Site-1 routes Site-2 routes Site-3 routes VRF for site-3 (100:3) Site-2 routes Site-3 routes Site-4 routes VRF for site-4 (100:4) Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

MPLS VPN - Configuration PE/CE routing protocols
PE/CE may use BGP, RIPv2 or Static routes A routing context is used for each VRF Routing contexts are defined within the routing protocol instance Address-family router sub-command Router rip version 2 address-family ipv4 vrf <vrf-symbolic-name> … any common router sub-command …

BGP uses same “address-family” command Router BGP <asn> ... address-family ipv4 vrf <vrf-symbolic-name> … any common router BGP sub-command … Static routes are configured per VRF ip route vrf <vrf-symbolic-name> …

MPLS VPN - Configuration PE router commands
All show commands are VRF based Show ip route vrf <vrf-symbolic-name> ... Show ip protocol vrf <vrf-symbolic-name> Show ip cef <vrf-symbolic-name> … … PING and Telnet commands are VRF based telnet /vrf <vrf-symbolic-name> ping vrf <vrf-symbolic-name>

ip vrf site1 rd 100:1 route-target export 100:12 route-target import 100:12 ip vrf site2 rd 100:2 route-target import 100:23 route-target export 100:23 ! interface Serial3/6 ip vrf forwarding site1 ip address encapsulation ppp interface Serial3/7 ip vrf forwarding site2 ip address Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B ip vrf site3 rd 100:3 route-target export 100:23 route-target import 100:23 route-target import 100:34 route-target export 100:34 ip vrf site-4 rd 100:4 ! interface Serial4/6 ip vrf forwarding site3 ip address encapsulation ppp interface Serial4/7 ip vrf forwarding site4 ip address Multihop MP-iBGP P P PE1 PE2 VRF for site-2 (100:2) Site-1 routes Site-2 routes Site-3 routes VRF for site-3 (100:3) Site-2 routes Site-3 routes Site-4 routes VRF for site-1 (100:1) Site-1 routes Site-2 routes VRF for site-4 (100:4) Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

router bgp 100 no bgp default ipv4-unicast neighbor remote-as 100 neighbor update-source Loop0 ! address-family ipv4 vrf site4 neighbor remote-as 65504 neighbor activate exit-address-family address-family ipv4 vrf site3 neighbor remote-as 65503 neighbor activate address-family vpnv4 neighbor activate neighbor next-hop-self router bgp 100 no bgp default ipv4-unicast neighbor remote-as 100 neighbor update-source Loop0 ! address-family ipv4 vrf site2 neighbor remote-as 65502 neighbor activate exit-address-family address-family ipv4 vrf site1 neighbor remote-as 65501 neighbor activate address-family vpnv4 neighbor activate neighbor next-hop-self Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B Multihop MP-iBGP P P PE1 PE2 VRF for site-2 (100:2) Site-1 routes Site-2 routes Site-3 routes VRF for site-3 (100:2) Site-2 routes Site-3 routes Site-4 routes VRF for site-1 (100:1) Site-1 routes Site-2 routes VRF for site-4 (100:3) Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

Summary Supports large scale VPN services
Increases value add by the VPN Service Provider Decreases Service Provider’s cost of providing VPN services Mechanisms are general enough to enable VPN Service Provider to support a wide range of VPN customers See RFC2547

Route Target VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2 P P PE-1 PE-2 Site-2 CE-2 VPN Backbone IGP P BGP,RIPv2 update for Net1,Next-Hop=CE-1 P Site-1 CE-1 ip vrf odd rd 100:1 route-target export “Green” route-target import “Green” VPN-IPv4 update: RD:Net1, Next-hop=PE-1 SOO=Site1, RT=Green, Label=(intCE1) Receiving PE is inserting the route into the VRF identified by the RT attribute (based on PE configuration) In this example RT = Green.

VRFs for VPNs yellow green
Inbound Filtering Proprietary feature PE creates a union of all configured RTs and automatically compares all incoming RTs for non null intersection Import RT=yellow VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ VRFs for VPNs yellow green PE MP-iBGP sessions VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green VPN-IPv4 update is silently rejected when it reaches PE since there isn’t any VRF configured with import RT = Red. Automatic (always on) rejection of all prefixes where at least one route target extended community attribute does not match any of route targets configured at the PE. Any VRF configuration change triggers “Route Refresh”

Route Refresh Based on: draft-chen-bgp-route-refresh-01.txt
When the inbound policy has been modified, the BGP speaker sends a Route-Refresh message to its neighbors With AFI, Sub-AFI attributes Neighbors will re-transmit all routes for that particular AFI and Sub-AFI Routers not refresh capable will reset BGP session Used for vpnv4 sessions, for ipv4 sessions manual soft refresh trigger: clear ip bgp neighbour x.x.x.x soft-in

Route Refresh and filtering
VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ Import RT=yellow 2. PE issue a Route-Refresh to all neighbors in order to ask for re-transmission PE VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green 1. PE doesn’t have red routes (previously filtered out) Import RT=red 3. Neighbors re-send updates and “red” route-target is now accepted Policy may change in the PE if VRF modifications are done New VRFs, removal of VRFs, RT addition or deletion However, the PE may not have stored routing information which become useful after a change PE request a re-transmission of updates to neighbors via Route-Refresh

Allow AS One central site has full routing knowledge of all other sites (of same VPN) Hub-Site Other sites will send traffic to Hub-Site for any destination Spoke-Sites Hub-Site is the central transit point between Spoke-Sites Use of central services at Hub-Site

Allow AS VPN-IPv4 update advertised by PE1 RD:N1, NH=PE1,Label=IntCE1, RT=Hub Site-1 N1 IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported BGP/RIPv2 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE3-Spoke PE2 CE2 BGP/RIPv2 VPN-IPv4 updates advertised by PE3 RD:N1, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N2, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N3, NH=PE3,Label=IntCE3-Spoke, RT=Spoke VPN-IPv4 update advertised by PE2 RD:N2, NH=PE2,Label=IntCE2, RT=Hub Routes are imported/exported into VRFs based on RT value of the VPN-IPv4 updates PE3 uses 2 (sub)interfaces with two different VRFs

Allow AS IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported Site-1 N1 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 BGP/RIPv2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 CE3-Spoke PE2 IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE2 BGP/RIPv2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) Traffic from one spoke to another will travel across the hub site Hub site may host central services Security, NAT, centralised Internet access

Allow AS If PE and Hub-site use BGP the PE should not check the received AS_PATH The update the Hub-site advertise contains the VPN backbone AS number By configuration the AS_PATH check is disabled Allow AS Routing loops are suppressed by the limit of occurrence of provider ASN in the AS_PATH Therefore, PE will REJECT the update if its ASN appears more than 3 times in the AS_PATH 3 is the default and can be overwritten with <opt>

Allow AS PE1 PE3 PE2 Site-1 CE1 Site-3 CE3-Hub Site-2 CE3-Spoke CE2
ASN: 251 Site-1 ASN: 250 ASN: 100 CE1 eBGP4 update: /32 AS_PATH: PE1 Site-3 /32 CE3-Hub ASN: 252 PE3 N3 Site-2 N2 CE3-Spoke eBGP4 update: /32 AS_PATH: PE2 CE2 ! address-family ipv4 vrf Hub neighbor remote-as 250 neighbor activate neighbor remote-as 250 neighbor activate neighbor allowas-in <opt> no auto-summary no synchronization exit-address-family

Allow AS with ASN override
eBGP4 update: /32 AS_PATH: 250 ASN: 250 VPN-IPv4 RD: /32, AS_PATH: 250 Site-1 ASN: 250 ASN: 100 eBGP4 update: /32 AS_PATH: CE1 /32 eBGP4 update: /32 AS_PATH: PE1 Site-3 CE3-Hub ASN: 250 VPN-IPv4 RD: /32, AS_PATH: PE3 N3 Site-2 N2 eBGP4 update: /32 AS_PATH: CE3-Spoke PE2 CE2 Now the AS_PATH contains four occurrences of the provider ASN. This update will not be accepted anymore if the CE re-advertise it back to any PE ! address-family ipv4 vrf Hub neighbor remote-as 250 neighbor activate neighbor remote-as 250 neighbor activate neighbor allowas-in <opt> neighbor as-override no auto-summary no synchronization exit-address-family

ASN Override When BGP is used between PE and CE routers, the customer VPN may want to re-use ASN in different sites Private ASN procedures already exist in order to strip the private ASN from the AS_PATH However, these procedures have following constraints: Private ASN is stripped if only private ASN are present in the AS_PATH Private ASN is stripped if NOT equal to the neighbouring ASN Private ASN procedures do NOT allow the re-use of same ASN in a MPLS-VPN environment

ASN Override New procedures have been implemented in order to re-use the same ASN on all VPN sites The procedures allows the use of private as well as public ASN Same ASN may be used for all sites, whatever is their VPN

ASN Override With ASN override configured the PE does following
If the last ASN in the AS_PATH is equal to the neighbouring one, it is replaced by the provider ASN If last ASN has multiple occurrences (due to AS_PATH prepend) all the occurrences are replaced with provider-ASN value After this operation, normal eBGP operation occur: Provider ASN is added to the AS_PATH

SOO is not needed for stub sites
ASN Override ASN override feature is used in conjunction with SOO in order to prevent routing loops In case of multihomed sites SOO is not needed for stub sites Sites connected to a single PE Multi-homed sites need to use SOO

ASN Override ASN: 100 PE-1 PE-2 CE-1 CE-2 ASN: 250 ASN: 250
ip vrf odd rd 100:1 route-target export 100:3 route-target import 100:3 ! interface Serial1 ip vrf forwarding odd ip address router bgp 100 no synchronization no bgp default ipv4-unicast neighbor remote-as 100 neighbor update-source Loop0 neighbor activate neighbor next-hop-self no auto-summary address-family ipv4 vrf odd neighbor remote-as 250 neighbor activate neighbor as-override exit-address-family address-family vpnv4 neighbor send-community extended ASN: 100 PE-1 PE-2 CE-1 CE-2 /32 /32 ASN: 250 ASN: 250

ASN Override PE-1 PE-2 ASN: 100 CE-1 CE-2 ASN: 250 ASN: 250
7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 (default for vrf odd) *>i / i *> / i PE-2 performs following actions: 1- Replace last ASN with its own ASN 2- Update AS_PATH with its own ASN 3- Forward the update to CE-2 VPN-IPv4 update: RD: /32 AS_PATH: 250 PE-1 PE-2 ASN: 100 eBGP4 update: /32 AS_PATH: 3640-5#sh ip b Network Next Hop Metric LocPrf Weight Path *> / i *> / i eBGP4 update: /32 AS_PATH: 250 CE-1 CE-2 /32 /32 ASN: 250 ASN: 250

ASN Override with AS_PATH prepend
7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 (default for vrf odd) *>i / i *> / i PE-2 performs following actions: 1- Replace all occurrences of last ASN with its own ASN 2- Update AS_PATH with its own ASN 3- Forward the update to CE-2 VPN-IPv4 update: RD: /32 AS_PATH: PE-1 PE-2 ASN: 100 eBGP4 update: /32 AS_PATH: 3640-5#sh ip b Network Next Hop Metric LocPrf Weight Path *> / i *> / i eBGP4 update: /32 AS_PATH: CE-1 CE-2 /32 /32 ASN: 250 ASN: 250

Site of Origin Used to identify the site Extended Community type
Used to prevent loops when AS_PATH cannot be used When BGP is used between PE and multihomed sites A BGP route is NOT advertised back to the same site Even through different PE/CE connections

Site of Origin SOO for eBGP learned routes
SOO is configured through a route-map command SOO can be applied to routes learned through a particular VRF interface (without the use of BGP between PE and CE) SOO is then configured on the interface SOO is propagated into BGP during redistribution

Site of Origin PE CE Site-1 192.168.0.5/32 ip vrf odd rd 100:1
route-target export 100:3 route-target import 100:3 ! interface Serial1 ip vrf forwarding odd ip address router bgp 100 no synchronization no bgp default ipv4-unicast neighbor remote-as 100 neighbor update-source Loop0 neighbor activate neighbor next-hop-self no auto-summary address-family ipv4 vrf odd neighbor remote-as 250 neighbor activate neighbor route-map setsoo in exit-address-family address-family vpnv4 neighbor send-community extended route-map setsoo permit 10 set extcommunity soo 100:65 PE /32 CE 7200-1#sh ip route vrf odd C /24 is directly connected, Serial2 B [20/0] via , 00:08:44, Serial2 7200-1# 7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 (default for vrf odd) *> / i 7200-1#sh ip bgp vpn all BGP routing table entry for 100:1: /32, version 17 Paths: (1 available, best #1) Advertised to non peer-group peers: 250 from ( ) Origin IGP, metric 0, localpref 100, valid, external, best Extended Community: SoO:100:65 RT:100:3

Site of Origin PE-1 PE-2 CE-1 CE-2 Site-1 SOO=100:65 192.168.0.5/32
VPN-IPv4 update: RD: /32, Next-hop=PE-1 SOO=100:65, RT=100:3, Label=(intCE1) PE-1 PE-2 intCE1 eBGP4 update: /32 PE-2 will not propagate the route since the update SOO is equal to the one configured for the site eBGP4 update: /32 CE-1 Site-1 SOO=100:65 CE-2 /32

Selective Export PE may have to export VRF routes with different route-targets Example: export management routes with particular RT Export command accept route-map Route-map configured into VRF Route-map match or deny statements with extended community list

Selective Export PE CE Site-1 192.168.0.5/32 192.168.50/24
VPN-IPv4 update: RD: /32 RT=100:3 VPN-IPv4 update: RD: /24 RT=100:4 CE /24 ip vrf odd rd 100:1 export map RTMAP route-target import 100:3 ! … access-list 10 permit access-list 11 permit any route-map RTMAP permit 10 match ip address 10 set extcommunity rt 100:3 route-map RTMAP permit 20 match ip address 11 set extcommunity rt 100:4

Selective Import PE may have to import routes based on other criteria than only Route-Target Import command accept route-map Route-map configured into VRF Route-map match or deny statements

Selective Import PE CE Site-1 192.168.0.5/32 192.168.50/24
VPN-IPv4 update: RD: /32 RT=100:3 VPN-IPv4 update: RD: /24 RT=100:4 CE /24 B [200/0] via , 02:17:48 ip vrf odd rd 100:1 import map RTMAP route-target export 100:3 ! … access-list 10 permit route-map RTMAP permit 10 match ip address 10

Added support for extended communities in route-maps
Extended route-maps Added support for extended communities in route-maps Route-Map match/set statements: route-map <Name> permit 10 [no] match extcommunity <1-99> [no] set extcommunity [rt|soo] <ASN:nn | IP-address:nn> Defining Extended Community access list: [no] ip extcommunity-list 1 [permit|deny] [rt|soo] <ASN:nn | IP-address:nn>

Internet routing - VRF specific default route
The PE installs a default route into the site VRF PE router originates CE routes for the Internet The default route points to the Internet router of the VPN backbone Possibility to use different Internet gateways per VRF No VPN default route allowed

MPLS VPN Topologies Internet routing - VRF specific default route
Global routing table with Internet routes Internet PE-IG IP packet D=cisco.com Label=PE-IG Global routing table with Internet routes Destination cisco.com is covered by the default route to PE-IG Global routing table with Internet routes PE Site-1 VRF Site-1 routes Site-2 routes PE Site-2 VRF Site-1 routes Site-2 routes /0 PE-IG IP packet D=cisco.com Site-1 Site-2 Ip route vrf <Name> PE-IG global

Direct Import (RT intersection)
EBGP received prefixes are now added to the vrf table in the router thread itself. Requirement to have a non null intersection between RTs for every VRF has been removed.

CE to CE convergence New BGP mechanism to be used in order to improve convergence time between sites BGP update origination, validation and advertisement Other mechanisms in order to improve import and export processes BGP update next-hop validation (done at scanner on PE) - scan-time adjustment. BGP validates updates by verifying next-hop reachability (first rule on PATH selection) By default the next-hop validation is done once every 60 seconds New command that allows to configure the timer bgp scan-time <5-60>

CE to CE convergence BGP update advertisement interval (default):
EBGP updates are propagated once every 30 seconds iBGP updates are propagated once every 5 seconds Default can be changed on a per neighbor basis neighbor <ip_address> advertisement-interval <0-600> BGP import/export process (IBGP learned into vrf on remote PE) By default import/export actions are performed once every 60 seconds Command to modify the timer: bgp scan-time import <5-60> Timer is configurable ONLY under address-family vpnv4

VRF Size Limit/Warning
New VRF level configuration command: (config-vrf)# maximum routes <number> { <warn percent> | warn-only } When <warn-percent> of <number> is reached then a SYSLOG error message is issued If the number of routes in the VRF routing table reaches <number> then no more routes will be added, a SYSLOG error message will be issued when an attempt is made to add a route which is rejected, throttled to one message per-VRF in 10 minutes.

What Code Is MPLS VPN In? Introduced in 12.0(5)T and 12.0(9)ST
Also in 12.1M and derivatves 12.0(15)SL, 12.0(17)ST for ESR

Things That Make Up MPLS-VPN
MPLS Forwrding – ENG-59293 TAG VPN Functional Spec – ENG-17513 MPLS VPN on GSR E2 cards – ENG-59451 …as a reference to a HW implementation

Software-based platforms
If you are developing a new software-based platform (like 2600, 3600, 4500, etc), should be pretty simple Concentrate on testing different packet paths and interface types

Hardware-based platforms
Label Imposition:could be 0, 1, or 2 labels Label Exposition: need to deal with aggregate label, very likely 2 lookups on the same packet

Label Imposition (Push)
PE1 CE1 PE3 CE2 CE3 PE2 P1 CE4 CE3<->CE4: PE3 imposes 0 labels, does regular FIB lookup in VRF table CE3->CE1: PE3 imposes 1 label (VPN label), IGP label is effectively PHP’d CE3->CE2: PE3 imposes 2 labels: (IGP label to PE2, VPN label) Explicit-null mitigates PHP

Label Exposition (Pop)
VPN advertises “aggregate label” for scalability Aggregate label leads to 2 lookups on egress PE (1 LIB, 1 FIB) Label lookup turns aggregate label into IP address within a VRF, IP lookup necessary to figure out correct L2 encap

Aggregate Label Label VRF 42 Red IP Address Port 3.3.3.0/24 POS1/0 PE1
CE1 PE3 CE3 CE4 VPN label = 42 IP packet Dst = Label VRF 42 Red PE3 does MPLS lookup on VPN label, finds outgoing VRF PE3 does IP lookup in VRF routing table, finds L2 encap, sends packet IP Address Port /24 POS1/0

Sizing Provider Edge (PE) Routers
Platform Specific Considerations QOS Considerations CPU Considerations PE Memory Considerations

Sizing Provider Edge (PE) CPU Considerations
Amount of provisioned QOS # of provisioned VRFs BGP-4 # of backbone BGP peers PE to CE Connectivity Type OSPF PE to CE connectivity type – Overhead associated with RIP V2, BGP-4 and OSPF. Number of neighbors ? Timer adjustment consumes more CPU # of provisioned VRFs – the more VRFs, the more FIB, RIB structures to deal with – IP Background process # of VPN clients/routes – more routes, more scanning (for example BGP scanner) # of backbone peers – more peers, more updates/keepalives etc, more sessions to maintain Packet forwarding - process switching takes more CPU due to lookup etc. CEF builds cache based on topology STATIC Packet forwarding CEF vs. process # of VPN clients/routes Several factors determine CPU Usage

Platform Processor Types
Internal Clock Speed NPE 225 RM5271 262 MHz NPE 300 R7000 NPE 400 350 MHz RSP 4 R5000 200 MHz RSP 8 250 MHz GRP Should mention here that the ESR (10000) and the OSR (7690) will also support PE functionality but are not included at this point. Should also note the GRP-2 (1Ghz processor)

Baseline (No Traffic) CPU Comparison Small VPN: 500 VRFs (11 routes per-VRF)
NPE225 – 262 MHz NPE300 – 262 MHz NPE400 – 350 MHz RSP8 – 250 MHz Should mention here that the ESR (10000) and the OSR (7690) will also support PE functionality but are not included at this point. Should also note the GRP-2 (1Ghz processor)

Sizing Provider Edge (PE) Memory Considerations
# of local VPN routes # of provisioned VRFs BGP-4 # of backbone BGP peers (paths) # of remote VPN routes # of neighbors and type of connectivity OSPF STATIC Unique or non-unique RD allocation ? Spread of IP addressing structure Several factors determine Memory Usage

Sizing Provider Edge (PE) Memory Considerations
BGP Memory Routing Table CEF MPLS IDB Several Areas of Memory Usage

Sizing Provider Edge (PE) BGP Memory
ndc-brighton# show ip bgp v a s BGP router identifier , local AS number 2 BGP table version is 21, main routing table version 21 1 network entries and 2 paths using 189 bytes of memory 2 BGP path attribute entries using 108 bytes of memory 2 BGP AS-PATH entries using 48 bytes of memory 1 BGP extended community entries using 24 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP activity 8/58 prefixes, 8/6 paths, scan interval 15 secs BGP Memory Mp = (N*128) + (P*60) + (Pa * 24) + (Ec * 24) Mp = Total memory used by PE in Bytes N = Number of BGP network entries P = Number of path entries Pa = Number of AS_PATH entries Ec = Number of Extended Community entries

Sizing Provider Edge (PE) Routing Table Memory
ndc-brighton# show memory summary | include IP: Control Block 0x60567BB IP: Control Block Routing Table Memory ndc-brighton# show ip route vrf testing summary IP routing table name is testing(1) Source Networks Subnets Overhead Memory (bytes) connected External: 0 Internal: 0 Local: 0 internal Total Each VRF consumes : 1 IP control block -> 33,184 bytes 1 Network Descriptor Block (NDB) per route (64 bytes) 1 Routing Descriptor Block (RDB) per path (144 bytes)

Sizing Provider Edge (PE) MPLS Memory
ndc-brighton# show memory allocating-process total | include TFIB tag_ 0x60DC5D TFIB tag_rewrite chunk 0x60DC5DB TFIB tag_info chunk 0x60DC5DA TFIB tag_info chunk 0x60DC5D TFIB tag_rewrite chunk ndc-brighton# show memory allocating-process total | include TIB 0x60FC7E TIB entry MPLS Memory MPLS forwarding memory (TFIB) consumes one 'taginfo‘ (64 bytes) per route, plus one forwarding entry (104 bytes) for each path

Sizing Provider Edge (PE) IDB Memory
ndc-brighton# show memory summary | include IDB 0x602F88E *Hardware IDB* 0x602F *Software IDB* Software IDB Hardware IDB Interface Description Block Hardware IDB: bytes (One per physical interface) Software IDB: bytes (One per interface and per sub-interface) Note: The amount of memory required will differ from platform to platform

PE VRF Memory Sizing NO VPN routes
Used Memory 8,187,968 MB Used Memory 56,243,216 MB Used Memory 69,631,904 MB Just the base VRFs (no interfaces defined) ================================================= gig >show mem all tot Head Total(b) Used(b) Free(b) Lowest(b) Largest(b) Processor 61D419A I/O I/O-2 F Allocator PC Summary for: Processor PC Total Count Name 0x605FAA IP: Control Block 0x60E0EEB IP mtrie node 0x6060F5E Exec 0x RMON packet pool 0x60DFF2CC FIB: Control Block 0x604F017C Interrupt Stack 0x6048A *Packet Header* 0x60E0EDA FIB one path ch 0x60793ABC BGP (0) update 0x604AAB Normal 0x60631E9C VPN ecomm list 0x C CCSWVOFR 0x60632B VPN record 0x IPv4 Unicast path-chunk 0x607AE IPv4 Unicast net-chunk(8) 0x604B IDB List Element Chunks 0x604D8A Watched Boolean 0x60DFF Init 0x VPN Target VPN hash ent 0x BGP (0) attr 0x604508D TTY data 0x60EE13F XTagATM VC chunk 0x604AABA Normal With 1000 sub ifs. ==================== gig #show mem all tot Processor 61D419A I/O 0x60EF9B Exec 0x60E0EEB IP mtrie node 0x6060F5E IP Background 0x60494F4C *Software IDB* 0x605FE IP subnet NDB e 0x605F9FDC IDB: IP Routing 0x6048A *Packet Header* 0x60631E9C VPN ecomm list 0x60632B VPN record 0x60B SWIDB_SB: LLC2 Info 0x60987DBC SWIDB_SB: NETBIOS Info 0x VPN Target VPN hash ent 0x606B DSS-SB 0x ARP Input 0x CDP sw subblock

VPN Memory Comparison

PE Memory Sizing Design Rules
~ 60-70K per VRF 33K for base VRF control block, other memory such as CEF, TFIB overhead, IDBs and so on ~ bytes per route (includes CEF, TFIB and RIB Memory in BGP) Remember IOS uses memory! Remember Internet Routes! Remember to leave transient memory Recommended to leave ~ 20MB free

PE Memory Sizing Design Observations
128 MB platforms are very limited (NPE 225, 3640 *NOT* suitable for full Internet table and VPNs!!!) 256 MB Minimum recommended on PE devices Limit the number of RDs per VRF in the same VPN unless you require iBGP load balancing with RRs

VRF and Route Limits Summary
VRF Limits Constrained mainly by CPU Between 500 & 1000 VRFs for static routing (depending on platform – 10 routes per VRF) Between 250 & 500 VRFs if using EBGP or RIPv2 (depending on platform routes per VRF) VPN & Global Route Limits Constrained mainly by available memory With 256 MB, 200,000 routes total (IPv4 and VPNv4) If Internet table is present, this reduces the memory available for VPNs (Current calculations are near 65 Meg for 100K Internet routes – with tightly packed attributes) we should caution that these results are from the lab and in the real world the BGP memory requirement will be higher due to an increase in the number of attributes NPE400 -> 1500 small VPNs with statics. 400 medium VPNs with statics. 500 eBGP medium. RSP4/8 -> 1000 small VPNs with statics. 200 medium VPNs with statics. 250 eBGP medium.

Core Topology

VPN toplogy NOTES: VXR15,16,12,11 are PEs VXR14,13,10,9 are CEs
all CEs have x as their RID GSR6 is VPNv4 RR

MPLS VPN TOI eosborne@cisco.com.

Similar presentations

Presentation on theme: "MPLS VPN TOI eosborne@cisco.com."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

MPLS VPN TOI eosborne@cisco.com.

Similar presentations

Presentation on theme: "MPLS VPN TOI eosborne@cisco.com."— Presentation transcript:

Similar presentations

About project

Feedback