Presentation is loading. Please wait.

Presentation is loading. Please wait.

Use of BGP and MPLS VPNs: A Case Study

Similar presentations

Presentation on theme: "Use of BGP and MPLS VPNs: A Case Study"— Presentation transcript:

1 Use of BGP and MPLS VPNs: A Case Study
Fred P. Baker CCIE#3555

2 Contents Current Network The MPLS VPN project Routing Objectives
What we did How we tested

3 Current Network

4 Current Environment Hub and spoke to 4 data centers
Sites do not in general connect to 2 data centers due to cost and OSPF issues Generally place servers by geography You servers are in the data center your links are in Mostly Frame Relay to ATM interworking with some private lines 70 of some 350 remote sites have 2 links ATM PVC dual mesh between the data centers 12000 agent location network done by MCI with combination of DSL and Fractional T1

5 Address Space /8 Mostly inside Some BP /16 Used all over /12 Extranet /16 Public address space Used mostly by extranet Some legacy inside

6 Core ATM PVCs 2 10meg between each pair of data centers
2 routers on the core So 2 meshes

7 Allstate Core

8 address allocation /11 for core 1 per data center

9 Allstate Data Center

10 Routing Protocol Single OSPF AS Cisco and OS/390 based routers only
Firewalls now static routed Peer authentication soon

11 Remote sites AT&T frame relay at the site ATM into the data center
Some ISDN backup A remote site is connected to a single data center (for now) Servers and applications tend to have geographic affinity

12 Remote Site

13 Remote Site Switch Layer

14 Agent Broadband 10,000 locations Connected via IPSEC VPN
WorldCom managed routers NO split tunneling IPSec Transport with GRE tunnel to Dallas and Hudson Agent PCs are 10.*.*.* Agent access is via Allstate Internet Proxy

15 Overview

16 Agent Broadband in Data Center

17 Agent office

18 Internet/Extranet We do not use the default route
There are 3 data center with ISP connections We code static routes to the firewalls (we don’t trust firewalls running dynamic routing protocols) and redist to OSPF

19 The project

20 The project We use a single data network provider
This is a single point of failure of that providers ATM/Frame networks Add a second data provider Initially to use for the dual attached sites Then convert 1 of the core ATM meshes to the second provider

21 Layer 2 vs Layer 3 provider
Frame Relay is layer 2 connectivity The routers have a direct peering relationship Many providers are offering Layer 3 Costs are the same or even less MPLS VPN is the data transport Many providers are using MPLS to move even layer 2 networks You have a routing relationships with the provider not with yourself So More complex to configure and fix Not a simple OSPF network anymore

22 Which one we picked Layer 3…
DR becomes free do not need to run more PVCs to a DR data center The data center placement of servers assumption is changing Apps are being put to 1 DC Also there is more site to site traffic than we expect So we can reduce traffic on the ATM core And increase response time Do dual homed sites first convert 1 link to L3 Single homed late

23 MPLS VPN VPN A/Site 1 VPN A/Site 2 VPN A/Site 3 VPN B/Site 2
CEA1 CEB3 CEA3 CEB2 CEA2 CE1B1 CE2B1 PE1 PE2 PE3 P1 P2 P3 10.1/16 10.2/16 10.3/16 10.4/16

24 Route types CE customer Edge PE provider Edge P providers your router
run BGP to provider Knows nothing about other customers or provider routes PE provider Edge Knows about all local customer VPNS Has multiple routing tables P providers Transport only No customer routes

25 Routing objectives Support load share from the home DC
Remote site goes direct to non home DC over L3 Remote site directly to remote site Reduce transit of the core Support a L3 provider in the core replacing 1 ATM mesh Do not use remote sites to transit traffic

26 Technical Objectives Limit the number of bgp attributes used
Keep the remote site configuration simple Do not inject the default route unless you must How to inject the Internet routes

27 Routing protocol design

28 Don’t forget the 3 rules of routing
Longest subnet mask Lowest distance Best metric

29 BGP features we used As path Path length filters No export Backdoor
If AS Paths are equal then router uses eBGP route

30 How to route Must look at the routes going BOTH ways
Routes to Routes from The routes you advertise drags traffic to you The routes you take in is how you route back We load share by having each router use a different path, then send equal cost into IGP

31 Result Use MPLS VPN based L3 provider Remote sites 2nd link to L3
Each data center connects to L3 Will not use L3 to route between DCs due to QoS concerns

32 Routing Use BGP at remote sites EACH data center is 1 AS
Can use OSPF with SOME providers but not all BGP works much better Each site is 1 AS EACH data center is 1 AS This allows us to put an L3 provider in later BGP routes BETWEEN ASes Address ASes from private space This is ok because provider is a VPN

33 Route injection to/from BGP
Allstate Data Center Explicit network statements to BGP Redist BGP to OSPF Remote site routes Redist from OSPF Decided that using network statements to complex BGP routers send just default route to any switches We will accept the extra LAN transit Internet routes Redist static

34 Internet routes There will be non BGP L3 switches between Inet and allstate core Redist static into OSPF already So just redist into BGP also Put internet router in same AS as datacenter (have to as no direct path) Use sync Send to L3 provider and to sites over L3

35 BGP to L3 provider (and then remote sites
Data center side Send data center /11s Send internet routes Take routes from L3 provider Do not forward other eBGP learned routes Remote site side Send all local routes do not forward other learned eBGP routes Remember the no export to kill transit Receive all routes Want to take L3 when I can

36 DC to Remote site FR Send all bgp derived routes
Do as prepend of the data center AS This makes AS path =2 for DC on FR and L3 paths This makes AS Path=3 for DC to DC via ATM core so site to remote DC traffic over L3

37 Remote site to DC on FR Do as prepend of 1 AS at remote end
Need this so FR and L3 paths have AS Path=2 so we load share Filter routes with AS Path >1 I only want to send the local site routes up the FR link Do not want DC to send transit traffic to site

38 IBGP in the remote site Set next hop self
Routers must have a shared Enet No redist of BGP to OSPF So cant use sync so cant transit a L3 switch Do not forward routes I learn via FR Do not want a transit from L3 up the FR link Do not want a transit to L3 from FR link Set no export attribute on routes from DC over the FR link This prevents site from passing them to L3 Cannot AS path filter on IBGP because I want to pass the DC route via iBGP Why I use no export

39 Results

40 DC to DC Each site learns over ATM network with AS Path = 1
Cannot route over L3 provider

41 Remote site to non home dc
Non home DC sent via L3 AS Path = 2 Home data sends via FR AS Path = 3 due to prepend Use if L3 down

42 non home dc to remote site
Non Home DC learns remote site routes from L3 Home data center sends only the /11 summary so longest match says L3

43 home dc to remote site Load share Routes from L3 have AS Path = 2
Routes from FR have AS Path = 2 due to prepend So each router uses eBGP route

44 remote site to home dc Don’t care as much about load share
Routes from L3 have AS Path = 2 Routes from FR have AS Path = 2 due to prepend So each router uses eBGP route

45 remote site to remote site
Use L3 network Learn site specific routes directly from site Learn /11 summaries from DCs

46 Agent routes Only dual DC connected things that don’t use BGP
Many routes summarized as /19s I get these from MCI as OSPF externals Have not decided how to inject them They go to two data centers for redundancy So I need to send them via BGP So a router will get an OSPF external from the local MCI connection and the other data center via BGP eBGP < OSPF so BOOM Use backdoor on core routers to set distance on the agent routes to > than OSPF So if local MCI connection up use it, else transit core

47 Testing

48 Local Testing Use 7 routers 1 remote site OSPF route not shown Paths
iBGP at remote L3 FR to home DC Inter DC

49 CPOC Cisco Proof Of Concept In Raleigh and San Jose
Lab use is free (if you are big enough) Send in specific test plan Your SE goes in a week ahead of time Lab is all setup when you arrive

50 Testing Test migrations Test routing Measure convergence
based on our policies failovers Measure convergence Test a migration of a core ATM mesh to L3 Get some data and experience on the MPLS side Try multicast over MPLS/VPN

51 CPOC Network Diagram

52 CPOC Learnings Inject all links both ATM core and L3 into BGP as they will source pings Turn sync off due to code defect You must explicitly code send community in iBGP If you reference a non-existent as-path statement NO ROUTES OSPF LSAs stay in the data base up to 90 minutes due to timer jitter This is a migration issue Do lots of clear routes/clear ip bgp in the migration Need to change the BGP timers as default convergence is 3 minutes iBGP only sends the best route

53 Going forward Already run BGP to some remote sites
Migrate the core to bgp first Do a dress rehearsal Will be a big scary change so plan well Examine tools May not be able to assume we will get traps May have to watch the BGP tables for changes Get a test connection in place

Download ppt "Use of BGP and MPLS VPNs: A Case Study"

Similar presentations

Ads by Google