1. Evolution Towards Global Routing Scalability
RRG March 09
Evolution Towards Global Routing Scalability
Varun Khare, University of Arizona
Beichuan Zhang, University of Arizona
Lixia Zhang, UCLA

2. Internet Routing Scalability: a problem?
DFZ routing table is growing uncontrollably
Different parts of the Internet view the scalability problem differently.
Most Stub ASes don’t carry full table internally
Certain ISPs can afford to upgrade routers
Within AS some routers experience problems more severely
No unanimous agreement about the problem

3. Routing Scalability Problem
Dan Jen conducted a survey of people with operational experience on the priority of scalability problems. The resultant ordering:
FIB size growth
RIB size growth
Churn due to Edge Dynamics
Survey look through it

4. Evolution towards Scalable Routing
Propose solution to individual problems rather than a comprehensive solution
Let ISPs decide when to take actions for which problem
The problem/solution can be related to each other
As ISPs move towards scaling the above measures, we steer the routing architecture to converge in the direction of “separation”
This talk: to illustrate feasibility of convergence
Not about virtual-aggregation, but take it as first step in the convergence process

5. Evolution –vs– Incremental Deployment
New architectural solutions (like LISP, APT) see big benefits when deployed by lots of ISPs
Incremental deployment of a new design allow an ISP running new architecture to inter-operate with legacy ISPs, but
Deploying a new design means relatively high cost
Immediate gain can be low
In evolutionary path, individual ISPs take actions on their own to solve an immediate problem
Cumulative changes over time can evolve the system to converge towards desired direction

6. Talk Outline Introduction Evolutionary Solution scenario
FIB size reduction
RIB size reduction
Edge Churn Problem
Evolution Path: Definition
Conclusion

7. Controlling FIB Size on Old Routers
Solution: Virtual Aggregation deployable by single ISP [ViAggre by Ballani et. al., HotNets’ 08]
Don’t need coordination with anyone else
Only need to make change in router configurations
ViAggre brings immediate FIB reduction
Divide FIB table amongst APRs. Routers suppress FIB.
Routers only maintain routes to APRs in their FIB
APR maintains routes to fraction of address space
Talk Overview
Specific evolutionary example; not fixed in design; feasibility of the evolutionary path being taken and the system converging to the idea of separation
AS1
PE1
Prefix Next Hop
1.1/16 …
2.1/16 …
BEFORE

8. Controlling FIB Size on Old Routers
Solution: Virtual Aggregation deployable by single ISP [ViAggre by Ballani et. al., HotNets’ 08]
Don’t need coordination with anyone else
Only need to make change in router configurations
ViAggre brings immediate FIB reduction
Divide FIB table amongst APRs. Routers suppress FIB.
Routers only maintain routes to APRs in their FIB
APR maintains routes to fraction of address space
Talk Overview
Specific evolutionary example; not fixed in design; feasibility of the evolutionary path being taken and the system converging to the idea of separation
Slide 6-7 combine text; let example flow
AS1
APR1
1/8
1.1/16 …
APR2
2/8
2.1/16 …
PE1
1/8 APR1
2/8 APR2
AFTER

9. Side-effects of deploying ViAggre
Stretch faced by packet
Within individual AS the stretch can be small
APR is a concentration point of data traffic
Bottleneck
1.1/16 …
1.2/16 PE-EXT
AS1
APR1
1/8
PE2
PE-EXT
PE1
Stretch
1/8 APR1 2/8 APR2
AFTER

10. Global deployment of ViAggre
Initially packet experiences stretch and encap-decap cost only in few ISPs deploying ViAggre
With global deployment, packet experiences stretch and encap-decap cost in every AS
Every ISP is mapping to its local external router for the destination prefix
1.2/16 PE3
Slide 9=10 again combine text; let example flow through
encap
PE4
APR1
decap
AS1
AS2
PE2
CE1
DST
1.2/16
PE1
PE3
stretch
1.2/16 PE4
1/ APR1

11. Global deployment of ViAggre
Initially packet experiences stretch and encap-decap cost only in few ISPs deploying ViAggre
With global deployment, packet experiences stretch and encap-decap cost in every AS
Every ISP is mapping to its local external router for the destination prefix
1.2/16 PE3
encap
PE4
encap
1.2/16 CE1
APR1
APR2
decap
AS1
AS2
decap
PE2
PE1
PE3
CE1
DST
1.2/16
stretch
stretch
1/ APR1
1/ APR2

12. Solve stretch and encap-decap cost
If propagate mapping info across ISPs: only need to Map-encap once Inter-AS ViAggre [Xiaohu et. al. Internet Draft]
Piggyback mapping info on BGP updates
Either on provider prefix or customer prefix
Need to upgrade APR’s BGP implementation
APR2
encap
1.2/ PE4
PE PE2
1.2/ PE4
AS2
AS1
PE4
APR1
PE3
PE2
PE1
decap
CE1
DST
1.2/16
1/ APR2
1/ APR1

13. RIB size problem
Problem: As more ISPs share mapping info, the mapping table will become very big.
It can cost all routers lots of memory since BGP stores multiple copies of mapping info in RIBs-IN.
FIB
1.2/16 PE-EXT MAP PE2
RIB
1.2/16 AS1, AS2 MAP PE2
1.2/16 AS3, AS2 MAP PE2
APR1
AS1
How big routing table  share estimation only when asked
Discuss new problems at the end of animation for the Stage
PE1
FIB
1/8 APR1
RIB
1.2/16 AS1, AS2 MAP PE2
1.2/16 AS3, AS2 MAP PE2

14. RIB size issue at different routers
RIB Size Problem
Proposed Solution
At Internal Routers
No need to stores routes to virtualized prefixes BRs can filter announcement of virtualized prefixes
At Border Routers (BR)
No need to store full routing table
APRs exchange BGP updates with map info on separate BGP session with APRs/DMs in neighboring ISPs via multi-hop BGP session
At APRs
No need to store multiple copies of map info
Upgrade APRs BGP implementation

15. Mapping Info stored by everyone
BRs and Internal Routers store full Routing Table with Mapping Info in RIB
APRs store routes with Mapping Info to fraction of address space in RIB
AS2
AS1
APR1
PE3
PE1
PE2
APR2
BGP Announcement
PE1 AS X
BGP + MAP Announcement
P | AS X, AS Y | MAP PE1
Customer AS Y, Prefix P

16. Mapping Info stored only by APRs
BRs and Internal Routers store routes only to APRs and PE routers in RIB
APRs can avoid storing multiple copies of Mapping info in RIB
MAP Announcements
P MAP PE1
AS1
APR1
APR2
AS2
PE1
PE2
PE3
Customer AS Y, Prefix P
BGP Announcements
PE1 AS X

17. A side note: whose prefixes get aggregated out?
Each ISP will keep its internal routes
ISPs will exchange reach ability of their networks
So that an ISP can encapsulate packets from ingress routers to (possibly another ISP’s) egress routers that connect to destination user sites
An ISP aggregates out the prefixes that belong to remote (non-direct) users

18. Excessive Churn due to edge dynamics
Need to insulate core from edge dynamics
When border (between provider and customer) failure happens
Mapping updates propagated out to ensure best path taken by data packets.
Mapping updates not propagated out and as data packet reaches failed link, provider network handles it by APT’s data-driven redirection.
Need to upgrade APRs/DMs/routers.
APRs share the edge churn load and shield other non-APR routers from it
Talk about APR split load and specific impact at various places as seen necessary

19. Pushing Mapping Updates
Internet
MAP Withdraw
P Map PE1
APR1 X
Customer AS Y, Prefix P
Customer AS Z
APR2
APR 90 CE
MAP Announcement
P Map PE2 (change)
MAP Announcement
P Map PE2
Data Packet

20. Suppress Mapping Updates
APT data-driven redirection
Internet
APR1 X
Customer AS Y, Prefix P
Customer AS Z
APR2
APR 90 CE
MAP Announcement
P Map PE1 (no change)
Data Packet

21. What is an evolution path?
The Internet routing architecture may go through several stages of change.
Each stage focuses on an immediate problem with enough economic impact that warrants a change.
Each stage offers a solution with reasonable deployment cost considering the problem.
As solution gains popularity, its downside may become prevalent to turn into the next problem
It is possible that extra work done in a certain stage may subsequently become redundant
Desire is to take incremental steps towards a healthy routing system
Hopefully we want to take these incremental steps to converge to healthy routing system
It could be possible the extra work where things done in previous stage maybe become redundant

22. Conclusion
We show the existence of an evolutionary path where each stage is motivated by immediate benefits
Gradually as all ISPs evolve through the stages the Internet routing would converge towards a routing architecture that separates edge prefixes from core routing
“Separation” itself is not the starting point, but the direction the system converges towards in the end

23. Thank You
Questions? Comments?