CANARIE – CA*net 3 “The Customer Empowered Networking Revolution” http://www.canarie.ca http://www.canet3.net Background Papers on Gigabit to The Home and Optical Internet Architecture Design Available Optical Internet News list: Send e-mail to Bill@Canarie.ca Bill.St.Arnaud@canarie.ca http://www.canarie.ca/~bstarn Tel: +1.613.785.0426

The Message In mid 1990s the prevailing wisdom was that commercial sector would drive design of Internet infrastructure R&E networks would focus on applications or specialized services As a result in North America R&E networks were commercialized or discontinued e.g NSFnet & CA*net However new network technologies and most importantly dark fiber is allowing R&E networks to once again redefine telecommunications not only for themselves but also for businesses and most importantly the last mile to the home R&E networks may become the cornerstone of municipal fiber to the home networks LAN architectures, technologies and most importantly LAN economics are invading the WAN Control and management of the optics and wavelengths will increasingly be under the domain of the LAN customer at the edge, as opposed to the traditional carrier in the center Over time the current hierarchical “connection oriented” telecom environment will look more like the Internet which is made up of autonomous peering networks. These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community.

Customer Empowered Networks Universities in Quebec are building their own 2000km fiber network Universities in Alberta are deploying their own 400 km 4xGbe dark fiber network School boards and municipalities throughout North America are deploying their own open access, dark fiber networks Carrier are selling “dim wavelengths” managed by customer to interconnect dark fiber networks Williams, Level 3, Hermes Typical cost is one time $20K US per school for a 20 year IRU In Ottawa we are deploying a 60km- 144 strand network connecting 26 institutions – cost $1m US

Dark Fiber Builds in Quebec

Ottawa Fiber Build Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes 26 sites Point-to-point topology 144 fibre pairs Route diversity requirement for one member 85 km run $11k - $50K per site Total project cost $CDN 1.25 million Cost per strand less than $.50 per strand per meter 80% aerial Due to overwhelming response to first build – planning for second build under way

Why Customer Owned Dark Fiber First - low cost Up to 1000% reduction over current telecom prices. 6-12 month payback Second - LAN invades the WAN – no complex SONET or ATM required in network Network Restoral & Protection can be done by customer using a variety of techniques such as wireless backup, or relocating servers to a multi-homed site, etc Third - Enables new applications and services not possible with traditional telecom service providers Relocation of servers and extending LAN to central site Out sourcing LAN and web servers to a 3rd party because no performance impact IP telephony in the wide area (Spokane) HDTV video Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points Much easier to out source servers, e-commerce etc to a 3rd party at a carrier neutral collocation facility Customers will start with dark fiber but will eventually extend further outwards with customer owned wavelengths Extending the Internet model of autonomous peering networks to the telecom world

CA*net 4?? Concept: Three models: Industry/Government partnership to build network with scalable growth in number of wavelengths Many wavelengths using Canadian WDM gear designed for carrying IP only Network infrastructure to be funded up to 20 years Where possible, use RAN fiber infrastructure to connect between provinces Three models: International wavelengths and national network: $50 - $150m National network only: $20-50m Minimal network (not coast to coast): $5 - 10m Connect together existing dark fiber projects in provinces RANs without dark fiber may not be able to participate

Optical BGP - OBGP Proposed new protocol where RANs ( and eventually universities) control routing of wavelengths across the network Marriage of CA*net 2 and CA*net 3 concepts RANs would have direct peering with each other and international peers CANARIE would offer optional aggregation and international peering where applicable May significantly reduce cost of commodity Internet by allowing direct peerings with many other networks (commercial and non-commercial)

O-BGP (Optical BGP) Control of optical routing and switches across an optical cloud is by the customer – not the carrier A radical new approach to the challenge of scaling of large networks Use establishment of BGP neighbors or peers at network configuration stage for process to establish light path cross connects Edge routers have large number of direct adjacencies to other routers Customers control of portions of OXC which becomes part of their AS Optical cross connects look like BGP speaking peers BGP peering sessions are setup with separate TCP channel outside of optical path or with a Lightpath Route Arbiter All customer requires from carrier is dark fiber, dim wavelengths, dark spaces and dumb switches Traditional BGP gives no indication of route congestion or QoS, but with DWDM wave lengths edge router will have a simple QoS path of guaranteed bandwidth Wavelengths will become new instrument for settlement and exchange eventually leading to futures market in wavelengths May allow smaller ISPs and R&E networks to route around large ISPs that dominate the Internet by massive direct peerings with like minded networks

Current View of Optical Internets ISP AS 4 AS 1 Customers buy managed service at the edge Optical VLAN AS 1 Customer AS 3 BGP Peering is done at the edge Big Carrier Optical Cloud using MPLS and IGP for management of wavelengths for provisioning, restoral and protection AS 2

OBGP Optical Internets Customer is now responsible for wavelength configuration, restoral and protection ISP Customer BGP Peering is done inside the optical switch BGP Big Carrier Optical Cloud disappears other than provisioning of electrical power to switches

BGP Routing + OXC = OBGP AS 200 180.10.10.0 BGP Neighbor BGP Neighbor 1.1.1.1 3.3.3.1 BGP Neighbor Router B Metric 100 Metric 100 1.1.1.2 2.2.2.1 4.4.4.1 3.3.3.2 Router A Metric 200 Metric 200 Router C 2.2.2.2 4.4.4.2 AS 100 170.10.10.0 AS 300 190.10.10.0 Figure 2.0

Virtual BGP Router L0 172.16.2.254 BGP Neighbor 255.255.255.255 2.2.2.1 L0 172.16.90.1 255.255.255.255 Router B L0 172.16.40.1 255.255.255.255 1.1.1.1 1.1.1.2 2.2.2.2 180.10.10.0 Router A 4.4.4.1 Router C 3.3.3.2 4.4.4.2 3.3.3.1 170.10.10.0 190.10.10.0 L0 172.16.1.254 255.255.255.255 AS 100 AS 300 AS 200 BGP Neighbor BGP Neighbor Figure 4.0

CA*net 4 – Distributed OIX AS 549 ONet AS 271 BCnet AS 376 RISQ OBGP OBGP OBGP New York Seattle Chicago Figure 12.0 10 10 9 9 10 9 10 10 10 10 10 10 10

ITN first step to global matrix of OBGP wavelengths? With ITN to support international transit? North American transit through Abilene & CA*net 3 European transit through DANTE, Nordunet, SURFnet, etc? Asian transit through APAN? Eventually layer 3 transit to be replaced by wavelength transit? But use OBGP for wavelength management Discussions are underway with international carriers to acquire initial wavelengths