SuperDREAM Testbed Pete Siemsen August 2007

Why SuperDREAM? FRGP problems: Single Points of Failure –Routers –Level 3 switch –Member circuits –Commodity Internet Members can’t connect at multiple sites

Existing Single Points of Failure 1200 Larimer –Router, switch, ATM switch Level 3 –switch/router – failure disconnects BiSON from Denver BPoP –switch/router Movaz boxes not addressed here

NLRLevel 3QwestAbilene BiSON DREAM FRGP logical view Level 3 PoP 1200 Larimer Larimer switch Larimer router Larimer ATM switch BPoP BPoP switch/router Level 3 switch/router M20 router For ICG only ICG

SuperDREAM goals Redundancy Allow members to connect to two sites Provide multiple paths to Commodity providers Optimize Service offerings Routing

FRGP services Default service: access to other FRGP members and to FRGP peers (Comcast, RMIX) Commodity Internet + TransitRail Abilene (I2) NLR PacketNet

Proposed Solution: VRFs VRFs - Virtual Routing and Forwarding. Separate routing tables allows a router to partition services in a natural way. 2 possible models: per-service VRFs or per- permutation VRFs

Per-service VRF solution Every FRGP router will have just these VRFs: –Members & peers VRF –A Commodity/TransitRail VRF –An NLR PacketNet VRF –An I2 VRF

Per-service VRFs… Each member will have a VLAN and a BGP session for each service that they buy.

Per-permutation VRF solution Every router will have a VRF for each permutation of services bought by members –Members & peers VRF –A Commodity/TransitRail VRF –An NLR VRF –An I2 VRF –An NLR/I2 VRF –A Commodity/Transitrail/NLR VRF –A Commodity/Transitrail/I2 VRF –A Commodity/Transitrail/NLR/I2 VRF

Solution trade-offs Per servicePer permutation Number of connections HigherLower Memory usageLowerHigher Rate limitsEasierHarder StatisticsEasierHarder Member complexityHigherLower

Commodity redundancy Commodity ISPs will be spread across FRGP sites. Commodity access will no longer depend on the Larimer router. Commodity traffic will flow to the globally “best” ISP, not necessarily the ISP nearest to each member.

Rate limits (per-service) Member traffic will rate limited at the member’s primary connection point. This greatly simplifies engineering.

Rate limits (per-permutation) Member traffic will be rate-limited at multiple points.

We can’t apply a single rate limit to traffic on two separate routers. Multihomed members must have a primary and an idle backup per service. Load-sharing Level 3 Qwest Level 3Larimer member

Constraints Maintain charging algorithm Have one engineering solution for everyone All members will do BGP Solution must fit in router memory

Open issues Implementation difficulty Unknown memory limits Engineering troubleshooting/debug MPLS vs. VLANs Cisco/Juniper interoperability Long-term FRGP 10G routing solution

TestBed