MPLS VPN TOI eosborne@cisco.com

Agenda How MPLS VPN works What Code Is MPLS VPN In? Platform Issues in Implementation Lab Demo - config

How MPLS-VPN Works Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling Configuration

MPLS-VPN What is a VPN ? An IP network infrastructure delivering private network services over a public infrastructure Use a layer 3 backbone Scalability, easy provisioning Global as well as non-unique private address space QoS Controlled access Easy configuration for customers

VPN Models - The Overlay model Private trunks over a TELCO/SP shared infrastructure Leased/Dialup lines FR/ATM circuits IP (GRE) tunnelling Transparency between provider and customer networks Optimal routing requires full mesh over over backbone

VPN Models - The Peer model Both provider and customer network use same network protocol CE and PE routers have a routing adjacency at each site All provider routers hold the full routing information about all customer networks Private addresses are not allowed May use the virtual router capability Multiple routing and forwarding tables based on Customer Networks

VPN Models - MPLS-VPN: The True Peer model Same as Peer model BUT !!! Provider Edge routers receive and hold routing information only about VPNs directly connected Reduces the amount of routing information a PE router will store Routing information is proportional to the number of VPNs a router is attached to MPLS is used within the backbone to switch packets (no need of full routing)

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling Configuration

MPLS-VPN Terminology Provider Network (P-Network) The backbone under control of a Service Provider Customer Network (C-Network) Network under customer control CE router Customer Edge router. Part of the C-network and interfaces to a PE router

MPLS-VPN Terminology Site PE router P router Set of (sub)networks part of the C-network and co-located A site is connected to the VPN backbone through one or more PE/CE links PE router Provider Edge router. Part of the P-Network and interfaces to CE routers P router Provider (core) router, without knowledge of VPN

MPLS-VPN Terminology Border router Extended Community Provider Edge router interfacing to other provider networks Extended Community BGP attribute used to identify a Route-origin, Route-target Site of Origin Identifier (SOO) 64 bits identifying routers where the route has been originated

MPLS-VPN Terminology Route-Target Route Distinguisher 64 bits identifying routers that should receive the route Route Distinguisher Attributes of each route used to uniquely identify prefixes among VPNs (64 bits) VRF based (not VPN based) VPN-IPv4 addresses Address including the 64 bits Route Distinguisher and the 32 bits IP address

MPLS-VPN Terminology VRF VPN-Aware network VPN Routing and Forwarding Instance Routing table and FIB table Populated by routing protocol contexts VPN-Aware network A provider backbone where MPLS-VPN is deployed

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling Configuration

MPLS VPN Connection Model A VPN is a collection of sites sharing a common routing information (routing table) A site can be part of different VPNs A VPN has to be seen as a community of interest (or Closed User Group) Multiple Routing/Forwarding instances (VRF) on PE routers

MPLS VPN Connection Model Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B A site belonging to different VPNs may or MAY NOT be used as a transit point between VPNs If two or more VPNs have a common site, address space must be unique among these VPNs

MPLS VPN Connection Model The VPN backbone is composed by MPLS LSRs PE routers (edge LSRs) P routers (core LSRs) PE routers are faced to CE routers and distribute VPN information through MP-BGP to other PE routers VPN-IPv4 addresses, Extended Community, Label P routers do not run BGP and do not have any VPN knowledge

MPLS VPN Connection Model VPN_A VPN_B 10.1.0.0 10.2.0.0 11.6.0.0 CE PE iBGP sessions VPN_A 11.5.0.0 CE VPN_A P P 10.1.0.0 PE CE P P VPN_B PE CE 10.3.0.0 P routers (LSRs) are in the core of the MPLS cloud PE routers use MPLS with the core and plain IP with CE routers P and PE routers share a common IGP PE router are MP-iBGP fully meshed

MPLS VPN Connection Model PE CE Site-2 Site-1 EBGP,OSPF, RIPv2,Static PE and CE routers exchange routing information through: EBGP, OSPF, RIPv2, Static routing CE router run standard routing software

MPLS VPN Connection Model CE Site-1 PE EBGP,OSPF, RIPv2,Static CE VPN Backbone IGP (OSPF, ISIS) Site-2 PE routers maintain separate routing tables The global routing table With all PE and P routes Populated by the VPN backbone IGP (ISIS or OSPF) VRF (VPN Routing and Forwarding) Routing and Forwarding table associated with one or more directly connected sites (CEs) VRF are associated to (sub/virtual/tunnel)interfaces Interfaces may share the same VRF if the connected sites may share the same routing information

MPLS VPN Connection Model CE Site-1 PE CE Site-2 Different site sharing the same routing information, may share the same VRF Interfaces connecting these sites will use the same VRF Sites belonging to the same VPN may share same VRF

MPLS VPN Connection Model CE Site-1 PE EBGP,OSPF, RIPv2,Static VPN Backbone IGP CE Site-2 The routes the PE receives from CE routers are installed in the appropriate VRF The routes the PE receives through the backbone IGP are installed in the global routing table By using separate VRFs, addresses need NOT to be unique among VPNs

MPLS VPN Connection Model The Global Routing Table is populated by IGP protocols. In PE routers it may contain the BGP Internet routes (standard BGP-4 routes) BGP-4 (IPv4) routes go into global routing table MP-BGP (VPN-IPv4) routes go into VRFs

MPLS VPN Connection Model PE PE VPN Backbone IGP P P iBGP session PE and P routers share a common IGP (ISIS or OSPF) PEs establish MP-iBGP sessions between them PEs use MP-BGP to exchange routing information related to the connected sites and VPNs VPN-IPv4 addresses, Extended Community, Label

MPLS VPN Connection Model MP-BGP Update VPN-IPV4 address Route Distinguisher 64 bits Makes the IPv4 route globally unique RD is configured in the PE for each VRF RD may or may not be related to a site or a VPN IPv4 address (32bits) Extended Community attribute (64 bits) Site of Origin (SOO): identifies the originating site Route-target (RT): identifies the set of sites the route has to be advertised to

MPLS VPN Connection Model MP-BGP Update Any other standard BGP attribute Local Preference MED Next-hop AS_PATH Standard Community ... A Label identifying: The outgoing interface The VRF where a lookup has to be done (aggregate label) The BGP label will be the second label in the label stack of packets travelling in the core

MPLS VPN Connection Model MP-BGP Update - Extended community BGP extended community attribute Structured, to support multiple applications 64 bits for increased range General form <16bits type>:<ASN>:<32 bit number> Registered AS number <16bits type>:<IP address>:<16 bit number> Registered IP address

MPLS VPN Connection Model MP-BGP Update - Extended community The Extended Community is used to: Identify one or more routers where the route has been originated (site) Site of Origin (SOO) Selects sites which should receive the route Route-Target

MPLS VPN Connection Model MP-BGP Update The Label can be assigned only by the router which address is the Next-Hop attribute PE routers re-write the Next-Hop with their own address (loopback interface address) “Next-Hop-Self” BGP command towards iBGP neighbors Loopback addresses are advertised into the backbone IGP PE addresses used as BGP Next-Hop must be uniquely known in the backbone IGP No summarisation of loopback addresses in the core

MPLS VPN Connection Model VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2 P P PE-1 PE-2 Site-2 CE-2 VPN Backbone IGP BGP,RIPv2 update for Net1,Next-Hop=CE-1 P P Site-1 CE-1 VPN-IPv4 update: RD:Net1, Next-hop=PE-1 SOO=Site1, RT=Green, Label=(intCE1) PE routers receive IPv4 updates (EBGP, RIPv2, Static) PE routers translate into VPN-IPv4 Assign a SOO and RT based on configuration Re-write Next-Hop attribute Assign a label based on VRF and/or interface Send MP-iBGP update to all PE neighbors

MPLS VPN Connection Model VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2 P P PE-1 PE-2 Site-2 CE-2 BGP,OSPF, RIPv2 update for Net1 Next-Hop=CE-1 VPN Backbone IGP P P Site-1 CE-1 VPN-IPv4 update: RD:Net1, Next-hop=PE-1 SOO=Site1, RT=Green, Label=(intCE1) Receiving PEs translate to IPv4 Insert the route into the VRF identified by the RT attribute (based on PE configuration) The label associated to the VPN-IPv4 address will be set on packet forwarded towards the destination

MPLS VPN Connection Model Route distribution to sites is driven by the Site of Origin (SOO) and Route-target attributes BGP Extended Community attribute A route is installed in the site VRF corresponding to the Route-target attribute Driven by PE configuration A PE which connects sites belonging to multiple VPNs will install the route into the site VRF if the Route-target attribute contains one or more VPNs to which the site is associated

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling Configuration

MPLS Forwarding Packet forwarding PE and P routers have BGP next-hop reachability through the backbone IGP Labels are distributed through LDP (hop-by-hop) corresponding to BGP Next-Hops Label Stack is used for packet forwarding Top label indicates BGP Next-Hop (interior label) Second level label indicates outgoing interface or VRF (exterior label)

MPLS Forwarding Packet forwarding MPLS nodes forward packets based on the top label P routers do not have BGP (nor VPN) knowledge No VPN routing information No Internet routing information

MPLS Forwarding Penultimate Hop Popping The upstream LDP peer of the BGP next-hop (PE router) will pop the first level label The penultimate hop will pop the label Requested through LDP The egress PE router will forward the packet based on the second level label which gives the outgoing interface (and VPN)

MPLS Forwarding MPLS Forwarding - Penultimate Hop Popping IGP Label(PE2) VPN Label IP packet P routers switch the packets based on the IGP label (label on top of the stack) IP packet PE2 receives the packets with the label corresponding to the outgoing interface (VRF) One single lookup Label is popped and packet sent to IP neighbor CE1 VPN Label IP packet Penultimate Hop Popping P2 is the penultimate hop for the BGP next-hop P2 remove the top label This has been requested through LDP by PE2 IP packet PE1 CE2 IGP Label(PE2) VPN Label IP packet PE1 receives IP packet Lookup is done on site VRF BGP route with Next-Hop and Label is found BGP next-hop (PE2) is reachable through IGP route with associated label P1 P2 PE2 CE3

MPLS VPN Forwarding VPN_A VPN_B 10.1.0.0 10.2.0.0 11.6.0.0 CE PE1 PE2 VPN_A 11.5.0.0 CE VPN_A P P 10.1.0.0 PE CE T8T2 Data P P Data VPN_B CE 10.3.0.0 <RD_B,10.2> , iBGP NH= PE2 , T2 T8 <RD_B,10.1> , iBGP next hop PE1 <RD_B,10.2> , iBGP next hop PE2 <RD_B,10.3> , iBGP next hop PE3 <RD_A,11.6> , iBGP next hop PE1 <RD_A,10.1> , iBGP next hop PE4 <RD_A,10.4> , iBGP next hop PE4 <RD_A,10.2> , iBGP next hop PE2 T1 T7 T2 T8 T3 T9 T4 T7 T5 TB T6 TB T7 T8 Ingress PE receives normal IP Packets from CE router PE router does “IP Longest Match” from VPN_B FIB , find iBGP next hop PE2 and impose a stack of labels: exterior Label T2 + Interior Label T8

MPLS VPN Forwarding in / out VPN_A VPN_B 10.1.0.0 10.2.0.0 11.6.0.0 CE PE1 PE2 VPN_A 11.5.0.0 CE Data T2 Data TB T2 Data VPN_A P P 10.1.0.0 PE CE TAT2 Data T8 T2 Data P P VPN_B CE 10.3.0.0 in / out T7 T8 T9 Ta Tb T8, TA Tu Tw Tx Ty Tz All Subsequent P routers do switch the packet Solely on Interior Label Egress PE router, removes Interior Label Egress PE uses Exterior Label to select which VPN/CE to forward the packet to. Exterior Label is removed and packet routed to CE router

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling Configuration

MPLS VPN mechanisms VRF and Multiple Routing Instances VRF: VPN Routing and Forwarding Instance VRF Routing Protocol Context VRF Routing Tables VRF CEF Forwarding Tables

MPLS VPN mechanisms VRF and Multiple Routing Instances VPN aware Routing Protocols Select/Install routes in appropriate routing table Per-instance router variables Not necessarily per-instance routing processes eBGP, OSPF, RIPv2, Static

MPLS VPN mechanisms VRF and Multiple Routing Instances VRF Routing table contains routes which should be available to a particular set of sites Analogous to standard IOS routing table, supports the same set of mechanisms Interfaces (sites) are assigned to VRFs One VRF per interface (sub-interface, tunnel or virtual-template) Possible many interfaces per VRF

MPLS VPN mechanisms VRF and Multiple Routing Instances Static BGP RIP Routing processes Routing contexts VRF Routing tables VRF Forwarding tables Routing processes run within specific routing contexts Populate specific VPN routing table and FIBs (VRF) Interfaces are assigned to VRFs

MPLS VPN mechanisms VRF and Multiple Routing Instances Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B Logical view Multihop MP-iBGP P P PE PE Routing view VRF for site-1 Site-1 routes Site-2 routes VRF for site-2 Site-1 routes Site-2 routes Site-3 routes VRF for site-3 Site-2 routes Site-3 routes Site-4 routes VRF for site-4 Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling BGP-4 Enhancements Cap. Negotiation, MPLS, Route Refresh, ORF Configuration

MPLS VPN Topologies iBGP sessions VPN_A VPN_B 10.1.0.0 10.2.0.0 11.6.0.0 CE PE VPN_A 11.5.0.0 CE VPN_A P P 10.1.0.0 PE CE P P VPN_B PE CE 10.3.0.0 VPN-IPv4 address are propagated together with the associated label in BGP Multiprotocol extension Extended Community attribute (route-target) is associated to each VPN-IPv4 address, to populate the site VRF

MPLS VPN Topologies VPN sites with optimal intra-VPN routing Each site has full routing knowledge of all other sites (of same VPN) Each CE announces his own address space MP-BGP VPN-IPv4 updates are propagated between PEs Routing is optimal in the backbone Each route has the BGP Next-Hop closest to the destination No site is used as central point for connectivity

MPLS VPN Topologies VPN sites with optimal intra-VPN routing EBGP/RIP/Static N3 NH=CE3 Routing Table on CE3 N1, PE3 N2, PE3 N3, Local VRF for site-3 N1,NH=PE1 N2,NH=PE2 N3,NH=CE3 IntCE3 PE3 VPN-IPv4 updates exchanged between PEs RD:N1, NH=PE1,Label=IntCE1, RT=Blue RD:N2, NH=PE2,Label=IntCE2, RT=Blue RD:N3, NH=PE3,Label=IntCE3, RT=Blue N2,NH=CE2 EBGP/RIP/Static PE1 IntCE2 VRF for site-1 N1,NH=CE1 N2,NH=PE2 N3,NH=PE3 VRF for site-2 N1,NH=PE1 N2,NH=CE2 N3,NH=PE3 Site-2 PE2 IntCE1 N2 Routing Table on CE2 N1,NH=PE2 N2,Local N3,NH=PE2 Routing Table on CE1 N1, Local N2, PE1 N3, PE1 EBGP/RIP/Static N1 NH=CE1 Site-1 N1

MPLS VPN Topologies VPN sites with Hub & Spoke routing One central site has full routing knowledge of all other sites (of same VPN) Hub-Site Other sites will send traffic to Hub-Site for any destination Spoke-Sites Hub-Site is the central transit point between Spoke-Sites Use of central services at Hub-Site

MPLS VPN Topologies VPN sites with Hub & Spoke routing VPN-IPv4 update advertised by PE1 RD:N1, NH=PE1,Label=IntCE1, RT=Hub Site-1 N1 IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported BGP/RIPv2 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE3-Spoke PE2 CE2 BGP/RIPv2 VPN-IPv4 updates advertised by PE3 RD:N1, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N2, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N3, NH=PE3,Label=IntCE3-Spoke, RT=Spoke VPN-IPv4 update advertised by PE2 RD:N2, NH=PE2,Label=IntCE2, RT=Hub Routes are imported/exported into VRFs based on RT value of the VPN-IPv4 updates PE3 uses 2 (sub)interfaces with two different VRFs

MPLS VPN Topologies VPN sites with Hub & Spoke routing IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported Site-1 N1 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 BGP/RIPv2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 CE3-Spoke PE2 IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE2 BGP/RIPv2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) Traffic from one spoke to another will travel across the hub site Hub site may host central services Security, NAT, centralised Internet access

MPLS VPN Topologies VPN sites with Hub & Spoke routing If PE and Hub-site use BGP the PE should not check the received AS_PATH The update the Hub-site advertise contains the VPN backbone AS number By configuration the AS_PATH check is disabled Routing loops are detected through the SOO attribute PE and CE routers may use RIPv2 and/or static routing

MPLS VPN Internet Routing In a VPN, sites may need to have Internet connectivity Connectivity to the Internet means: Being able to reach Internet destinations Being able to be reachable from any Internet source Security mechanism MUST be used as in ANY other kind of Internet connectivity

MPLS VPN Internet Routing The Internet routing table is treated separately In the VPN backbone the Internet routes are in the Global routing table of PE routers Labels are not assigned to external (BGP) routes P routers need not (and will not) run BGP

MPLS VPN Internet routing VRF specific default route A default route is installed into the site VRF and pointing to a Internet Gateway The default route is NOT part of any VPN A single label is used for packets forwarded according to the default route The label is the IGP label corresponding to the IP address of the Internet gateway Known in the IGP

MPLS VPN Internet routing VRF specific default route PE router originates CE routes for the Internet Customer (site) routes are known in the site VRF Not in the global table The PE/CE interface is NOT known in the global table. However: A static route for customer routes and pointing to the PE/CE interface is installed in the global table This static route is redistributed into BGP-4 global table and advertised to the Internet Gateway The Internet gateway knows customer routes and with the PE address as next-hop

MPLS VPN Internet routing VRF specific default route The Internet Gateway specified in the default route (into the VRF) need NOT to be directly connected Different Internet gateways can be used for different VRFs Using default route for Internet routing does NOT allow any other default route for intra-VPN routing As in any other routing scheme

MPLS VPN Internet routing VRF specific default route 192.168.1.1 ip vrf VPN-A rd 100:1 route-target both 100:1 ! Interface Serial0 ip address 192.168.10.1 255.255.255.0 ip vrf forwarding VPN-A Router bgp 100 no bgp default ipv4-unicast network 171.68.0.0 mask 255.255.0.0 neighbor 192.168.1.1 remote 100 neighbor 192.168.1.1 activate neighbor 192.168.1.1 next-hop-self neighbor 192.168.1.1 update-source loopback0 address-family ipv4 vrf VPN-A neighbor 192.168.10.2 remote-as 65502 neighbor 192.168.10.2 activate exit-address-family address-family vpnv4 neighbor 192.168.1.2 activate ip route 171.68.0.0 255.255.0.0 Serial0 ip route vrf VPN-A 0.0.0.0 0.0.0.0 192.168.1.1 global BGP-4 Internet PE-IG MP-BGP 192.168.1.2 PE PE Serial0 Site-1 Network 171.68.0.0/16 Site-2

MPLS VPN Internet routing VRF specific default route 192.168.1.1 IP packet D=cisco.com Internet PE-IG Label = 3 IP packet D=cisco.com Global Table and LFIB 192.168.1.1/32 Label=3 192.168.1.2/32 Label=5 ... 192.168.1.2 PE PE Site-2 VRF 0.0.0.0/0 192.168.1.1 (global) Site-1 routes Site-2 routes Serial0 IP packet D=cisco.com Site-1 Network 171.68.0.0/16 Site-2

MPLS VPN Internet routing VRF specific default route PE routers need not to hold the Internet table PE routers will use BGP-4 sessions to originate customer routes Packet forwarding is done with a single label identifying the Internet Gateway IP address More labels if Traffic Engineering is used

MPLS VPN Internet Routing Separated (sub)interfaces If CE wishes to receive and announce routes from/to the Internet A dedicated BGP session is used over a separate (sub) interface The PE imports CE routes into the global routing table and advertise them to the Internet The interface is not part of any VPN and does not use any VRF Default route or Internet routes are exported to the CE PE needs to have Internet routing table

MPLS VPN Internet Routing Separated (sub)interfaces The PE uses separate (sub)interfaces with the CE One (sub)interface for VPN routing associated to a VRF Can be a tunnel interface One (sub)interface for Internet routing Associated to the global routing table

MPLS VPN Internet Routing Separated (sub)interfaces ip vrf VPN-A rd 100:1 route-target both 100:1 ! Interface Serial0 no ip address Interface Serial0.1 ip address 192.168.10.1 255.255.255.0 ip vrf forwarding VPN-A Interface Serial0.2 ip address 171.68.10.1 255.255.255.0 Router bgp 100 no bgp default ipv4-unicast neighbor 192.168.1.1 remote 100 neighbor 192.168.1.1 activate neighbor 192.168.1.1 next-hop-self neighbor 192.168.1.1 update-source loopback0 neighbor 171.68.10.2 remote 502 address-family ipv4 vrf VPN-A neighbor 192.168.10.2 remote-as 502 neighbor 192.168.10.2 activate exit-address-family address-family vpnv4 neighbor 192.168.1.2 activate 192.168.1.1 BGP-4 Internet PE-IG MP-BGP PE 192.168.1.2 PE Serial0.1 Serial0.2 BGP-4 Site-1 Network 171.68.0.0/16 Site-2

MPLS VPN Internet Routing Separated (sub)interfaces 192.168.1.1 IP packet D=cisco.com Internet PE-IG Label = 3 IP packet D=cisco.com PE Global Table Internet routes ---> 192.168.1.1 192.168.1.1, Label=3 192.168.1.2 PE PE Serial0.1 Serial0.2 IP packet D=cisco.com Serial0.1 Serial0.2 Site-1 CE routing table Site-2 routes ----> Serial0.1 Internet routes ---> Serial0.2 Network 171.68.0.0/16 Site-2

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling Configuration

Scaling Existing BGP techniques can be used to scale the route distribution: route reflectors Each edge router needs only the information for the VPNs it supports Directly connected VPNs RRs are used to distribute VPN routing information

Scaling Very highly scalable: Easy to add new sites Initial VPN release: 1000 VPNs x 1000 sites/VPN = 1,000,000 sites Architecture supports 100,000+ VPNs, 10,000,000+ sites BGP “segmentation” through RRs is essential !!!! Easy to add new sites configure the site on the PE connected to it the network automagically does the rest See also platform issues, later on

MPLS-VPN Scaling BGP Route Reflectors may be partitioned VPN_A VPN_B 10.3.0.0 10.1.0.0 11.5.0.0 P PE CE RR Route Reflectors 10.2.0.0 11.6.0.0 PE1 PE2 Route Reflectors may be partitioned Each RR store routes for a set of VPNs Thus, no BGP router needs to store ALL VPNs information PEs will peer to RRs according to the VPNs they directly connect

MPLS-VPN Scaling BGP updates filtering iBGP full mesh between PEs results in flooding all VPNs routes to all PEs Scaling problems when large amount of routes. In addition PEs need only routes for attached VRFs Therefore each PE will discard any VPN-IPv4 route that hasn’t a route-target configured to be imported in any of the attached VRFs This reduces significantly the amount of information each PE has to store Volume of BGP table is equivalent of volume of attached VRFs (nothing more)

MPLS-VPN Scaling BGP updates filtering Import RT=yellow VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ VRFs for VPNs yellow green PE MP-iBGP sessions VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green Each VRF has an import and export policy configured Policies use route-target attribute (extended community) PE receives MP-iBGP updates for VPN-IPv4 routes If route-target is equal to any of the import values configured in the PE, the update is accepted Otherwise it is silently discarded

MPLS-VPN Scaling Route Refresh VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ Import RT=yellow 2. PE issue a Route-Refresh to all neighbors in order to ask for re-transmission PE VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green 1. PE doesn’t have red routes (previously filtered out) Import RT=red 3. Neighbors re-send updates and “red” route-target is now accepted Policy may change in the PE if VRF modifications are done New VRFs, removal of VRFs However, the PE may not have stored routing information which become useful after a change PE request a re-transmission of updates to neighbors Route-Refresh

MPLS-VPN Scaling Outbound Route Filters - ORF VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ Import RT=yellow 2. PE issue a ORF message to all neighbors in order not to receive red routes PE VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green 1. PE doesn’t need red routes 3. Neighbors dynamically configure the outbound filter and send updates accordingly PE router will discard update with unused route-target Optimisation requires these updates NOT to be sent Outbound Route Filter (ORF) allows a router to tell its neighbors which filter to use prior to propagate BGP updates

Agenda Concepts and goals Terminology Connection model Forwarding Mechanisms Topologies Scaling BGP-4 Enhancements Cap. Negotiation, MPLS, Route Refresh, ORF Configuration

MPLS VPN - Configuration VPN knowledge is on PE routers PE router have to be configured for VRF and Route Distinguisher VRF import/export policies (based on Route-target) Routing protocol used with CEs MP-BGP between PE routers BGP for Internet routers With other PE routers With CE routers

MPLS VPN - Configuration VRF and Route Distinguisher RD is configured on PE routers (for each VRF) VRFs are associated to RDs in each PE Common (good) practice is to use the same RD for the same VPN in all PEs But not mandatory VRF configuration command ip vrf <vrf-symbolic-name> rd <route-distinguisher-value> route-target import <community> route-target export <community>

CLI - VRF configuration ip vrf site1 rd 100:1 route-target export 100:1 route-target import 100:1 ip vrf site2 rd 100:2 route-target export 100:2 route-target import 100:2 Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B ip vrf site3 rd 100:3 route-target export 100:2 route-target import 100:2 route-target import 100:3 route-target export 100:3 ip vrf site-4 rd 100:4 route-target export 100:3 Multihop MP-iBGP P P PE1 PE2 VRF for site-1 (100:1) Site-1 routes Site-2 routes VRF for site-2 (100:2) Site-1 routes Site-2 routes Site-3 routes VRF for site-3 (100:3) Site-2 routes Site-3 routes Site-4 routes VRF for site-4 (100:4) Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

MPLS VPN - Configuration PE/CE routing protocols PE/CE may use BGP, RIPv2 or Static routes A routing context is used for each VRF Routing contexts are defined within the routing protocol instance Address-family router sub-command Router rip version 2 address-family ipv4 vrf <vrf-symbolic-name> … any common router sub-command …

MPLS VPN - Configuration PE/CE routing protocols BGP uses same “address-family” command Router BGP <asn> ... address-family ipv4 vrf <vrf-symbolic-name> … any common router BGP sub-command … Static routes are configured per VRF ip route vrf <vrf-symbolic-name> …

MPLS VPN - Configuration PE router commands All show commands are VRF based Show ip route vrf <vrf-symbolic-name> ... Show ip protocol vrf <vrf-symbolic-name> Show ip cef <vrf-symbolic-name> … … PING and Telnet commands are VRF based telnet /vrf <vrf-symbolic-name> ping vrf <vrf-symbolic-name>

MPLS VPN - Configuration PE/CE routing protocols ip vrf site1 rd 100:1 route-target export 100:12 route-target import 100:12 ip vrf site2 rd 100:2 route-target import 100:23 route-target export 100:23 ! interface Serial3/6 ip vrf forwarding site1 ip address 192.168.61.6 255.255.255.0 encapsulation ppp interface Serial3/7 ip vrf forwarding site2 ip address 192.168.62.6 255.255.255.0 Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B ip vrf site3 rd 100:3 route-target export 100:23 route-target import 100:23 route-target import 100:34 route-target export 100:34 ip vrf site-4 rd 100:4 ! interface Serial4/6 ip vrf forwarding site3 ip address 192.168.73.7 255.255.255.0 encapsulation ppp interface Serial4/7 ip vrf forwarding site4 ip address 192.168.74.7 255.255.255.0 Multihop MP-iBGP P P PE1 PE2 VRF for site-2 (100:2) Site-1 routes Site-2 routes Site-3 routes VRF for site-3 (100:3) Site-2 routes Site-3 routes Site-4 routes VRF for site-1 (100:1) Site-1 routes Site-2 routes VRF for site-4 (100:4) Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

MPLS VPN - Configuration PE/CE routing protocols router bgp 100 no bgp default ipv4-unicast neighbor 6.6.6.6 remote-as 100 neighbor 6.6.6.6 update-source Loop0 ! address-family ipv4 vrf site4 neighbor 192.168.74.4 remote-as 65504 neighbor 192.168.74.4 activate exit-address-family address-family ipv4 vrf site3 neighbor 192.168.73.3 remote-as 65503 neighbor 192.168.73.3 activate address-family vpnv4 neighbor 6.6.6.6 activate neighbor 6.6.6.6 next-hop-self router bgp 100 no bgp default ipv4-unicast neighbor 7.7.7.7 remote-as 100 neighbor 7.7.7.7 update-source Loop0 ! address-family ipv4 vrf site2 neighbor 192.168.62.2 remote-as 65502 neighbor 192.168.62.2 activate exit-address-family address-family ipv4 vrf site1 neighbor 192.168.61.1 remote-as 65501 neighbor 192.168.61.1 activate address-family vpnv4 neighbor 7.7.7.7 activate neighbor 7.7.7.7 next-hop-self Site-1 Site-3 Site-4 Site-2 VPN-A VPN-C VPN-B Multihop MP-iBGP P P PE1 PE2 VRF for site-2 (100:2) Site-1 routes Site-2 routes Site-3 routes VRF for site-3 (100:2) Site-2 routes Site-3 routes Site-4 routes VRF for site-1 (100:1) Site-1 routes Site-2 routes VRF for site-4 (100:3) Site-3 routes Site-4 routes Site-1 Site-2 Site-3 Site-4

Summary Supports large scale VPN services Increases value add by the VPN Service Provider Decreases Service Provider’s cost of providing VPN services Mechanisms are general enough to enable VPN Service Provider to support a wide range of VPN customers See RFC2547

Route Target VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2 P P PE-1 PE-2 Site-2 CE-2 VPN Backbone IGP P BGP,RIPv2 update for Net1,Next-Hop=CE-1 P Site-1 CE-1 ip vrf odd rd 100:1 route-target export “Green” route-target import “Green” VPN-IPv4 update: RD:Net1, Next-hop=PE-1 SOO=Site1, RT=Green, Label=(intCE1) Receiving PE is inserting the route into the VRF identified by the RT attribute (based on PE configuration) In this example RT = Green.

VRFs for VPNs yellow green Inbound Filtering Proprietary feature PE creates a union of all configured RTs and automatically compares all incoming RTs for non null intersection Import RT=yellow VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ VRFs for VPNs yellow green PE MP-iBGP sessions VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green VPN-IPv4 update is silently rejected when it reaches PE since there isn’t any VRF configured with import RT = Red. Automatic (always on) rejection of all prefixes where at least one route target extended community attribute does not match any of route targets configured at the PE. Any VRF configuration change triggers “Route Refresh”

Route Refresh Based on: draft-chen-bgp-route-refresh-01.txt When the inbound policy has been modified, the BGP speaker sends a Route-Refresh message to its neighbors With AFI, Sub-AFI attributes Neighbors will re-transmit all routes for that particular AFI and Sub-AFI Routers not refresh capable will reset BGP session Used for vpnv4 sessions, for ipv4 sessions manual soft refresh trigger: clear ip bgp neighbour x.x.x.x soft-in

Route Refresh and filtering VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Green, Label=XYZ Import RT=yellow 2. PE issue a Route-Refresh to all neighbors in order to ask for re-transmission PE VPN-IPv4 update: RD:Net1, Next-hop=PE-X SOO=Site1, RT=Red, Label=XYZ Import RT=green 1. PE doesn’t have red routes (previously filtered out) Import RT=red 3. Neighbors re-send updates and “red” route-target is now accepted Policy may change in the PE if VRF modifications are done New VRFs, removal of VRFs, RT addition or deletion However, the PE may not have stored routing information which become useful after a change PE request a re-transmission of updates to neighbors via Route-Refresh

Allow AS One central site has full routing knowledge of all other sites (of same VPN) Hub-Site Other sites will send traffic to Hub-Site for any destination Spoke-Sites Hub-Site is the central transit point between Spoke-Sites Use of central services at Hub-Site

Allow AS VPN-IPv4 update advertised by PE1 RD:N1, NH=PE1,Label=IntCE1, RT=Hub Site-1 N1 IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported BGP/RIPv2 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE3-Spoke PE2 CE2 BGP/RIPv2 VPN-IPv4 updates advertised by PE3 RD:N1, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N2, NH=PE3,Label=IntCE3-Spoke, RT=Spoke RD:N3, NH=PE3,Label=IntCE3-Spoke, RT=Spoke VPN-IPv4 update advertised by PE2 RD:N2, NH=PE2,Label=IntCE2, RT=Hub Routes are imported/exported into VRFs based on RT value of the VPN-IPv4 updates PE3 uses 2 (sub)interfaces with two different VRFs

Allow AS IntCE1 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=CE1 (exported) N2,NH=PE3 (imported) N3,NH=PE3 (imported Site-1 N1 IntCE3-Hub VRF (Import RT=Hub) N1,NH=PE1 N2,NH=PE2 BGP/RIPv2 CE1 PE1 Site-3 CE3-Hub PE3 N3 Site-2 N2 CE3-Spoke PE2 IntCE3-Spoke VRF (Export RT=Spoke) N1,NH=CE3-Spoke N2,NH=CE3-Spoke N3,NH=CE3-Spoke CE2 BGP/RIPv2 IntCE2 VRF (Import RT=Spoke) (Export RT=Hub) N1,NH=PE3 (imported) N2,NH=CE2 (exported) N3,NH=PE3 (imported) Traffic from one spoke to another will travel across the hub site Hub site may host central services Security, NAT, centralised Internet access

Allow AS If PE and Hub-site use BGP the PE should not check the received AS_PATH The update the Hub-site advertise contains the VPN backbone AS number By configuration the AS_PATH check is disabled Allow AS Routing loops are suppressed by the limit of occurrence of provider ASN in the AS_PATH Therefore, PE will REJECT the update if its ASN appears more than 3 times in the AS_PATH 3 is the default and can be overwritten with <opt>

Allow AS PE1 PE3 PE2 Site-1 CE1 Site-3 CE3-Hub Site-2 CE3-Spoke CE2 ASN: 251 Site-1 ASN: 250 ASN: 100 CE1 eBGP4 update: 192.168.0.5/32 AS_PATH: 100 251 PE1 Site-3 192.168.0.5/32 CE3-Hub ASN: 252 PE3 N3 Site-2 N2 CE3-Spoke eBGP4 update: 192.168.0.5/32 AS_PATH: 250 100 251 PE2 CE2 ! address-family ipv4 vrf Hub neighbor 192.168.73.3 remote-as 250 neighbor 192.168.73.3 activate neighbor 192.168.74.4 remote-as 250 neighbor 192.168.74.4 activate neighbor 192.168.74.4 allowas-in <opt> no auto-summary no synchronization exit-address-family

Allow AS with ASN override eBGP4 update: 192.168.0.5/32 AS_PATH: 250 ASN: 250 VPN-IPv4 RD:192.168.0.5/32, AS_PATH: 250 Site-1 ASN: 250 ASN: 100 eBGP4 update: 192.168.0.5/32 AS_PATH: 100 100 CE1 192.168.0.5/32 eBGP4 update: 192.168.0.5/32 AS_PATH: 100 100 100 100 PE1 Site-3 CE3-Hub ASN: 250 VPN-IPv4 RD:192.168.0.5/32, AS_PATH: 250 100 100 PE3 N3 Site-2 N2 eBGP4 update: 192.168.0.5/32 AS_PATH: 250 100 100 CE3-Spoke PE2 CE2 Now the AS_PATH contains four occurrences of the provider ASN. This update will not be accepted anymore if the CE re-advertise it back to any PE ! address-family ipv4 vrf Hub neighbor 192.168.73.3 remote-as 250 neighbor 192.168.73.3 activate neighbor 192.168.74.4 remote-as 250 neighbor 192.168.74.4 activate neighbor 192.168.74.4 allowas-in <opt> neighbor 192.168.74.4 as-override no auto-summary no synchronization exit-address-family

ASN Override When BGP is used between PE and CE routers, the customer VPN may want to re-use ASN in different sites Private ASN procedures already exist in order to strip the private ASN from the AS_PATH However, these procedures have following constraints: Private ASN is stripped if only private ASN are present in the AS_PATH Private ASN is stripped if NOT equal to the neighbouring ASN Private ASN procedures do NOT allow the re-use of same ASN in a MPLS-VPN environment

ASN Override New procedures have been implemented in order to re-use the same ASN on all VPN sites The procedures allows the use of private as well as public ASN Same ASN may be used for all sites, whatever is their VPN

ASN Override With ASN override configured the PE does following If the last ASN in the AS_PATH is equal to the neighbouring one, it is replaced by the provider ASN If last ASN has multiple occurrences (due to AS_PATH prepend) all the occurrences are replaced with provider-ASN value After this operation, normal eBGP operation occur: Provider ASN is added to the AS_PATH

SOO is not needed for stub sites ASN Override ASN override feature is used in conjunction with SOO in order to prevent routing loops In case of multihomed sites SOO is not needed for stub sites Sites connected to a single PE Multi-homed sites need to use SOO

ASN Override ASN: 100 PE-1 PE-2 CE-1 CE-2 ASN: 250 ASN: 250 ip vrf odd rd 100:1 route-target export 100:3 route-target import 100:3 ! interface Serial1 ip vrf forwarding odd ip address 192.168.73.7 255.255.255.0 router bgp 100 no synchronization no bgp default ipv4-unicast neighbor 192.168.0.6 remote-as 100 neighbor 192.168.0.6 update-source Loop0 neighbor 192.168.0.6 activate neighbor 192.168.0.6 next-hop-self no auto-summary address-family ipv4 vrf odd neighbor 192.168.73.3 remote-as 250 neighbor 192.168.73.3 activate neighbor 192.168.73.3 as-override exit-address-family address-family vpnv4 neighbor 192.168.0.6 send-community extended ASN: 100 PE-1 PE-2 CE-1 CE-2 192.168.0.5/32 192.168.0.3/32 ASN: 250 ASN: 250

ASN Override PE-1 PE-2 ASN: 100 CE-1 CE-2 ASN: 250 ASN: 250 7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 (default for vrf odd) *>i192.168.0.3/32 192.168.0.7 0 0 250 i *> 192.168.0.5/32 192.168.65.5 0 0 250 i PE-2 performs following actions: 1- Replace last ASN with its own ASN 2- Update AS_PATH with its own ASN 3- Forward the update to CE-2 VPN-IPv4 update: RD:192.168.0.5/32 AS_PATH: 250 PE-1 PE-2 ASN: 100 eBGP4 update: 192.168.0.5/32 AS_PATH:100 100 3640-5#sh ip b Network Next Hop Metric LocPrf Weight Path *> 192.168.0.5/32 192.168.73.7 0 100 100 i *> 192.168.0.3/32 0.0.0.0 0 i eBGP4 update: 192.168.0.5/32 AS_PATH: 250 CE-1 CE-2 192.168.0.5/32 192.168.0.3/32 ASN: 250 ASN: 250

ASN Override with AS_PATH prepend 7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 (default for vrf odd) *>i192.168.0.3/32 192.168.0.7 0 0 250 i *> 192.168.0.5/32 192.168.65.5 0 0 250 250 250 i PE-2 performs following actions: 1- Replace all occurrences of last ASN with its own ASN 2- Update AS_PATH with its own ASN 3- Forward the update to CE-2 VPN-IPv4 update: RD:192.168.0.5/32 AS_PATH: 250 250 250 PE-1 PE-2 ASN: 100 eBGP4 update: 192.168.0.5/32 AS_PATH:100 100 100 100 3640-5#sh ip b Network Next Hop Metric LocPrf Weight Path *> 192.168.0.5/32 192.168.73.7 0 100 100 100 100 i *> 192.168.0.3/32 0.0.0.0 0 i eBGP4 update: 192.168.0.5/32 AS_PATH: 250 250 250 CE-1 CE-2 192.168.0.5/32 192.168.0.3/32 ASN: 250 ASN: 250

Site of Origin Used to identify the site Extended Community type Used to prevent loops when AS_PATH cannot be used When BGP is used between PE and multihomed sites A BGP route is NOT advertised back to the same site Even through different PE/CE connections

Site of Origin SOO for eBGP learned routes SOO is configured through a route-map command SOO can be applied to routes learned through a particular VRF interface (without the use of BGP between PE and CE) SOO is then configured on the interface SOO is propagated into BGP during redistribution

Site of Origin PE CE Site-1 192.168.0.5/32 ip vrf odd rd 100:1 route-target export 100:3 route-target import 100:3 ! interface Serial1 ip vrf forwarding odd ip address 192.168.65.6 255.255.255.0 router bgp 100 no synchronization no bgp default ipv4-unicast neighbor 192.168.0.7 remote-as 100 neighbor 192.168.0.7 update-source Loop0 neighbor 192.168.0.7 activate neighbor 192.168.0.7 next-hop-self no auto-summary address-family ipv4 vrf odd neighbor 192.168.65.5 remote-as 250 neighbor 192.168.65.5 activate neighbor 192.168.65.5 route-map setsoo in exit-address-family address-family vpnv4 neighbor 192.168.0.7 send-community extended route-map setsoo permit 10 set extcommunity soo 100:65 PE 192.168.0.5/32 CE 7200-1#sh ip route vrf odd C 192.168.65.0/24 is directly connected, Serial2 B 192.168.0.5 [20/0] via 192.168.65.5, 00:08:44, Serial2 7200-1# 7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 (default for vrf odd) *> 192.168.0.5/32 192.168.65.5 0 0 250 i 7200-1#sh ip bgp vpn all 192.168.0.5 BGP routing table entry for 100:1:192.168.0.5/32, version 17 Paths: (1 available, best #1) Advertised to non peer-group peers: 192.168.0.7 250 192.168.65.5 from 192.168.65.5 (192.168.0.5) Origin IGP, metric 0, localpref 100, valid, external, best Extended Community: SoO:100:65 RT:100:3

Site of Origin PE-1 PE-2 CE-1 CE-2 Site-1 SOO=100:65 192.168.0.5/32 VPN-IPv4 update: RD:192.168.0.5/32, Next-hop=PE-1 SOO=100:65, RT=100:3, Label=(intCE1) PE-1 PE-2 intCE1 eBGP4 update: 192.168.0.5/32 PE-2 will not propagate the route since the update SOO is equal to the one configured for the site eBGP4 update: 192.168.0.5/32 CE-1 Site-1 SOO=100:65 CE-2 192.168.0.5/32

Selective Export PE may have to export VRF routes with different route-targets Example: export management routes with particular RT Export command accept route-map Route-map configured into VRF Route-map match or deny statements with extended community list

Selective Export PE CE Site-1 192.168.0.5/32 192.168.50/24 VPN-IPv4 update: RD:192.168.0.5/32 RT=100:3 VPN-IPv4 update: RD:192.168.50.0/24 RT=100:4 CE 192.168.50/24 ip vrf odd rd 100:1 export map RTMAP route-target import 100:3 ! … access-list 10 permit 192.168.0.5 0.0.0.0 access-list 11 permit any route-map RTMAP permit 10 match ip address 10 set extcommunity rt 100:3 route-map RTMAP permit 20 match ip address 11 set extcommunity rt 100:4

Selective Import PE may have to import routes based on other criteria than only Route-Target Import command accept route-map Route-map configured into VRF Route-map match or deny statements

Selective Import PE CE Site-1 192.168.0.5/32 192.168.50/24 VPN-IPv4 update: RD:192.168.30.3/32 RT=100:3 VPN-IPv4 update: RD:192.168.30.0/24 RT=100:4 CE 192.168.50/24 B 192.168.30.0 [200/0] via 192.168.0.7, 02:17:48 ip vrf odd rd 100:1 import map RTMAP route-target export 100:3 ! … access-list 10 permit 192.168.30.0 0.0.0.0 route-map RTMAP permit 10 match ip address 10

Added support for extended communities in route-maps Extended route-maps Added support for extended communities in route-maps Route-Map match/set statements: route-map <Name> permit 10 [no] match extcommunity <1-99> [no] set extcommunity [rt|soo] <ASN:nn | IP-address:nn> Defining Extended Community access list: [no] ip extcommunity-list 1 [permit|deny] [rt|soo] <ASN:nn | IP-address:nn>

Internet routing - VRF specific default route The PE installs a default route into the site VRF PE router originates CE routes for the Internet The default route points to the Internet router of the VPN backbone Possibility to use different Internet gateways per VRF No VPN default route allowed

MPLS VPN Topologies Internet routing - VRF specific default route Global routing table with Internet routes Internet PE-IG IP packet D=cisco.com Label=PE-IG Global routing table with Internet routes Destination cisco.com is covered by the default route to PE-IG Global routing table with Internet routes PE Site-1 VRF Site-1 routes Site-2 routes PE Site-2 VRF Site-1 routes Site-2 routes 0.0.0.0/0 PE-IG IP packet D=cisco.com Site-1 Site-2 Ip route vrf <Name> 0.0.0.0 0.0.0.0 PE-IG global

Direct Import (RT intersection) EBGP received prefixes are now added to the vrf table in the router thread itself. Requirement to have a non null intersection between RTs for every VRF has been removed.

CE to CE convergence New BGP mechanism to be used in order to improve convergence time between sites BGP update origination, validation and advertisement Other mechanisms in order to improve import and export processes BGP update next-hop validation (done at scanner on PE) - scan-time adjustment. BGP validates updates by verifying next-hop reachability (first rule on PATH selection) By default the next-hop validation is done once every 60 seconds New command that allows to configure the timer bgp scan-time <5-60>

CE to CE convergence BGP update advertisement interval (default): EBGP updates are propagated once every 30 seconds iBGP updates are propagated once every 5 seconds Default can be changed on a per neighbor basis neighbor <ip_address> advertisement-interval <0-600> BGP import/export process (IBGP learned into vrf on remote PE) By default import/export actions are performed once every 60 seconds Command to modify the timer: bgp scan-time import <5-60> Timer is configurable ONLY under address-family vpnv4

VRF Size Limit/Warning New VRF level configuration command: (config-vrf)# maximum routes <number> { <warn percent> | warn-only } When <warn-percent> of <number> is reached then a SYSLOG error message is issued If the number of routes in the VRF routing table reaches <number> then no more routes will be added, a SYSLOG error message will be issued when an attempt is made to add a route which is rejected, throttled to one message per-VRF in 10 minutes.

Agenda How MPLS VPN works What Code Is MPLS VPN In? Platform Issues in Implementation Lab Demo - config

What Code Is MPLS VPN In? Introduced in 12.0(5)T and 12.0(9)ST Also in 12.1M and derivatves 12.0(15)SL, 12.0(17)ST for ESR

Agenda How MPLS VPN works What Code Is MPLS VPN In? Platform Issues in Implementation Lab Demo - config

Things That Make Up MPLS-VPN MPLS Forwrding – ENG-59293 TAG VPN Functional Spec – ENG-17513 MPLS VPN on GSR E2 cards – ENG-59451 …as a reference to a HW implementation

Software-based platforms If you are developing a new software-based platform (like 2600, 3600, 4500, etc), should be pretty simple Concentrate on testing different packet paths and interface types

Hardware-based platforms Label Imposition:could be 0, 1, or 2 labels Label Exposition: need to deal with aggregate label, very likely 2 lookups on the same packet

Label Imposition (Push) PE1 CE1 PE3 CE2 CE3 PE2 P1 CE4 CE3<->CE4: PE3 imposes 0 labels, does regular FIB lookup in VRF table CE3->CE1: PE3 imposes 1 label (VPN label), IGP label is effectively PHP’d CE3->CE2: PE3 imposes 2 labels: (IGP label to PE2, VPN label) Explicit-null mitigates PHP

Label Exposition (Pop) VPN advertises “aggregate label” for scalability Aggregate label leads to 2 lookups on egress PE (1 LIB, 1 FIB) Label lookup turns aggregate label into IP address within a VRF, IP lookup necessary to figure out correct L2 encap

Aggregate Label Label VRF 42 Red IP Address Port 3.3.3.0/24 POS1/0 PE1 CE1 PE3 CE3 CE4 VPN label = 42 IP packet Dst = 3.3.3.3 Label VRF 42 Red PE3 does MPLS lookup on VPN label, finds outgoing VRF PE3 does IP lookup in VRF routing table, finds L2 encap, sends packet IP Address Port 3.3.3.0/24 POS1/0

Sizing Provider Edge (PE) Routers Platform Specific Considerations QOS Considerations CPU Considerations PE Memory Considerations

Sizing Provider Edge (PE) CPU Considerations Amount of provisioned QOS # of provisioned VRFs BGP-4 # of backbone BGP peers PE to CE Connectivity Type OSPF PE to CE connectivity type – Overhead associated with RIP V2, BGP-4 and OSPF. Number of neighbors ? Timer adjustment consumes more CPU # of provisioned VRFs – the more VRFs, the more FIB, RIB structures to deal with – IP Background process # of VPN clients/routes – more routes, more scanning (for example BGP scanner) # of backbone peers – more peers, more updates/keepalives etc, more sessions to maintain Packet forwarding - process switching takes more CPU due to lookup etc. CEF builds cache based on topology STATIC Packet forwarding CEF vs. process # of VPN clients/routes Several factors determine CPU Usage

Platform Processor Types Internal Clock Speed NPE 225 RM5271 262 MHz NPE 300 R7000 NPE 400 350 MHz RSP 4 R5000 200 MHz RSP 8 250 MHz GRP Should mention here that the ESR (10000) and the OSR (7690) will also support PE functionality but are not included at this point. Should also note the GRP-2 (1Ghz processor)

Baseline (No Traffic) CPU Comparison Small VPN: 500 VRFs (11 routes per-VRF) NPE225 – 262 MHz NPE300 – 262 MHz NPE400 – 350 MHz RSP8 – 250 MHz Should mention here that the ESR (10000) and the OSR (7690) will also support PE functionality but are not included at this point. Should also note the GRP-2 (1Ghz processor)

Sizing Provider Edge (PE) Memory Considerations # of local VPN routes # of provisioned VRFs BGP-4 # of backbone BGP peers (paths) # of remote VPN routes # of neighbors and type of connectivity OSPF STATIC Unique or non-unique RD allocation ? Spread of IP addressing structure Several factors determine Memory Usage

Sizing Provider Edge (PE) Memory Considerations BGP Memory Routing Table CEF MPLS IDB Several Areas of Memory Usage

Sizing Provider Edge (PE) BGP Memory ndc-brighton# show ip bgp v a s BGP router identifier 10.3.1.9, local AS number 2 BGP table version is 21, main routing table version 21 1 network entries and 2 paths using 189 bytes of memory 2 BGP path attribute entries using 108 bytes of memory 2 BGP AS-PATH entries using 48 bytes of memory 1 BGP extended community entries using 24 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP activity 8/58 prefixes, 8/6 paths, scan interval 15 secs BGP Memory Mp = (N*128) + (P*60) + (Pa * 24) + (Ec * 24) Mp = Total memory used by PE in Bytes N = Number of BGP network entries P = Number of path entries Pa = Number of AS_PATH entries Ec = Number of Extended Community entries

Sizing Provider Edge (PE) Routing Table Memory ndc-brighton# show memory summary | include IP: Control Block  0x60567BB0 33184 101 3351584 IP: Control Block Routing Table Memory ndc-brighton# show ip route vrf testing summary  IP routing table name is testing(1) Source Networks Subnets Overhead Memory (bytes) connected 0 1 64 144 External: 0 Internal: 0 Local: 0 internal 1 1164 Total 1 1 64 1308 Each VRF consumes : 1 IP control block -> 33,184 bytes 1 Network Descriptor Block (NDB) per route (64 bytes) 1 Routing Descriptor Block (RDB) per path (144 bytes)

Sizing Provider Edge (PE) MPLS Memory ndc-brighton# show memory allocating-process total | include TFIB tag_   0x60DC5D54 8101672 125 TFIB tag_rewrite chunk 0x60DC5DB4 4141564 64 TFIB tag_info chunk 0x60DC5DA4 65540 1 TFIB tag_info chunk 0x60DC5D44 65540 1 TFIB tag_rewrite chunk ndc-brighton# show memory allocating-process total | include TIB 0x60FC7E10 24228 134 TIB entry MPLS Memory MPLS forwarding memory (TFIB) consumes one 'taginfo‘ (64 bytes) per route, plus one forwarding entry (104 bytes) for each path

Sizing Provider Edge (PE) IDB Memory ndc-brighton# show memory summary | include IDB   0x602F88E8 4692 9 42228 *Hardware IDB* 0x602F8904 2576 9 23184 *Software IDB* Software IDB Hardware IDB Interface Description Block Hardware IDB: 4692 bytes (One per physical interface) Software IDB: 2576 bytes (One per interface and per sub-interface) Note: The amount of memory required will differ from platform to platform

PE VRF Memory Sizing NO VPN routes Used Memory 8,187,968 MB Used Memory 56,243,216 MB Used Memory 69,631,904 MB Just the base VRFs (no interfaces defined) ================================================= gig-7200-7>show mem all tot Head Total(b) Used(b) Free(b) Lowest(b) Largest(b) Processor 61D419A0 229369440 56215976 173153464 172002680 172346860 I/O 20000000 33554448 264432 33290016 33290016 33289532 I/O-2 F800000 8388624 5110000 3278624 3278624 3278268 Allocator PC Summary for: Processor PC Total Count Name 0x605FAA18 33261228 1001 IP: Control Block 0x60E0EEB4 8503196 258 IP mtrie node 0x6060F5E8 3900000 1000 Exec 0x60273238 1937580 2 RMON packet pool 0x60DFF2CC 1117116 1001 FIB: Control Block 0x604F017C 829752 108 Interrupt Stack 0x6048A350 466440 762 *Packet Header* 0x60E0EDA0 404904 12 FIB one path ch 0x60793ABC 393240 6 BGP (0) update 0x604AAB10 360860 15 Normal 0x60631E9C 352080 2000 VPN ecomm list 0x6133754C 284716 1 CCSWVOFR 0x60632B88 208060 1000 VPN record 0x60793210 196620 3 IPv4 Unicast path-chunk 0x607AE058 196620 3 IPv4 Unicast net-chunk(8) 0x604B0478 168128 1 IDB List Element Chunks 0x604D8A98 167472 1255 Watched Boolean 0x60DFF948 131116 1 Init 0x60632134 124048 1000 VPN Target VPN hash ent 0x60793924 118320 4 BGP (0) attr 0x604508D4 74520 18 TTY data 0x60EE13F4 72216 2 XTagATM VC chunk 0x604AABA4 70720 26 Normal With 1000 sub ifs. ==================== gig-7200-7#show mem all tot Processor 61D419A0 229369440 77907700 151461740 150266496 150388188 I/O 20000000 33554432 493564 33060868 32796144 32583228 0x60EF9B64 12152000 1000 Exec 0x60E0EEB4 8503164 258 IP mtrie node 0x6060F5E8 7800000 2000 IP Background 0x60494F4C 2548000 1000 *Software IDB* 0x605FE778 1376340 21 IP subnet NDB e 0x605F9FDC 591420 1004 IDB: IP Routing 0x6048A350 579564 947 *Packet Header* 0x60631E9C 352084 2000 VPN ecomm list 0x60632B88 208116 1000 VPN record 0x60B23518 149344 1004 SWIDB_SB: LLC2 Info 0x60987DBC 133140 1004 SWIDB_SB: NETBIOS Info 0x60632134 124116 1000 VPN Target VPN hash ent 0x606B9960 108692 1003 DSS-SB 0x60567888 84984 1005 ARP Input 0x60649460 84844 1003 CDP sw subblock

VPN Memory Comparison

PE Memory Sizing Design Rules ~ 60-70K per VRF 33K for base VRF control block, other memory such as CEF, TFIB overhead, IDBs and so on ~800-900 bytes per route (includes CEF, TFIB and RIB Memory in BGP) Remember IOS uses memory! Remember Internet Routes! Remember to leave transient memory Recommended to leave ~ 20MB free

PE Memory Sizing Design Observations 128 MB platforms are very limited (NPE 225, 3640 *NOT* suitable for full Internet table and VPNs!!!) 256 MB Minimum recommended on PE devices Limit the number of RDs per VRF in the same VPN unless you require iBGP load balancing with RRs

VRF and Route Limits Summary VRF Limits Constrained mainly by CPU Between 500 & 1000 VRFs for static routing (depending on platform – 10 routes per VRF) Between 250 & 500 VRFs if using EBGP or RIPv2 (depending on platform - 500 routes per VRF) VPN & Global Route Limits Constrained mainly by available memory With 256 MB, 200,000 routes total (IPv4 and VPNv4) If Internet table is present, this reduces the memory available for VPNs (Current calculations are near 65 Meg for 100K Internet routes – with tightly packed attributes) we should caution that these results are from the lab and in the real world the BGP memory requirement will be higher due to an increase in the number of attributes NPE400 -> 1500 small VPNs with statics. 400 medium VPNs with statics. 500 eBGP medium. RSP4/8 -> 1000 small VPNs with statics. 200 medium VPNs with statics. 250 eBGP medium.

Agenda How MPLS VPN works What Code Is MPLS VPN In? Platform Issues in Implementation Lab Demo - config

Core Topology

VPN toplogy NOTES: VXR15,16,12,11 are PEs VXR14,13,10,9 are CEs all CEs have 192.168.1.x as their RID GSR6 is VPNv4 RR