Optical Networking CS 294-3 2/5/2002 John Strand AT&T Optical Networks Research Dept. jls@research.att.com I hope that you will ask questions freely at any time during these lectures. This will help me to adjust my presentations to best help you. It also will make the lectures more interesting for me. If you have additional questions, please feel free to contact me after November 5 at AT&T: My Email address is: jls@research.att.com My postal address is: John Strand AT&T, Room 4-212 100 Schulz Drive Red Bank, N.J. 07701 U. S. A. U. of California - Berkeley - EECS Dept. jls@photonics.eecs.berkeley.edu The Views Expressed In This Talk Are The Author’s. They Do Not Necessarily Represent The Views Of AT&T Or Any Other Corporation Or Individual.

Outline Transport - Traditional TDM Networks Optical Networking Optical Networking & IP There will be 5 lectures, as shown on this Viewgraph. Each lecture will be about 3 hours long, with a 15 minute break in the middle. These lectures are based primarily on my experience as architect of AT&T’s domestic transport network and designer of the algorithms which AT&T uses to engineer this network. I will try to highlight the issues which AT&T and our U.S. competitors are wrestling with today. The lectures were initially developed to train AT&T network planners and researchers. The version you will be seeing has been modified to make it more appropriate for university seminars and industry meetings. I hope you will forgive the focus on the U.S. telecommunications industry. This is primarily due to my limited understanding of the situation in China. Through your questions I may be able to help you relate our experience to the problems your country will be facing. Concentrate On Intercity Networks Time Constraint Metro, Access Optical Networks More Complex, Less Mature

Layering IP Layers Transport Layers Prototype Apps Context Awareness Services Adaptation Wide-Area PM & Monitoring Possible Service Architecture Layers Transport Layers DS1 DS3 SONET- Path Line Section Conduit Wavelength Fiber Cable ROW IP Layers "Service" Application Tranport Network Link Physical

Basic DS-1 Signal Format 193 Bits Time Slot 2 Time Slot 3 Time Slot 1 Time Slot 23 Time Slot 24 F Bit o o o o o o o o o o o 1 bit 8 bits 8 bits 8 bits 8 bits 8 bits Designed To Carry 24 Full Duplex 64 Kilobit/sec Voice Circuits (“DS-0’s”) Transmission Rate = 1.55 Megabits/Second (8000 frames/sec * 193 bits/frame) DS-1 Is A Protocol; T-1 Is A Specific AT&T Implementation Of This Protocol For Local Networks This Is The Traditional Building Block For Transmission Networks In U.S. The Slowest Inter-Office Signal Is The DS-1. It was defined in the 1960’s to carry 24 64 kb/sec full duplex voice channels. Many variations have since been defined. It is built on 193-bit frames, with 8000 frames/second. Frequently A DS-1 formatted signal is mis-named a “T-1”. The DS-1 is the protocol, T-1 is a specific physical implementation of the protocol. The same confusion exists at higher rates - T-3 and DS-3, for example.

What is SONET? Synchronous Optical Network standard Defines a digital hierarchy of synchronous signals Maps asynchronous signals (DS1, DS3) to synchronous format Defines electrical and optical connections between equipment Allows for interconnection of different vendors’ equipment Provides overhead channels for interoffice Operations, Administration, Maintenance, & Provisioning (OAM&P) SONET Interface SONET Network Element SONET Network Element Digital Tributaries Digital Tributaries SONET defines all aspects of the interface between SONET-compatible network elements. SONET is the North American standard. In most other parts of the world, the standard is "Synchronous Digital Hierarchy" (SDH). SONET and SDH are "almost" identical. If you understand one you should have no problem learning about the other. Because all of VG's were prepared for SONET-oriented audiences I will describe SONET.

SONET Structure Byte-Interleaved Multiplexing PRS: Primary Source Network Of Synchronized Clocks PRS: Primary Source ST2: Stratum 2 ST3: Stratum 3 Must Be Synchronized Byte-Interleaved Multiplexing

Digital Signal Hierarchies Most Common Rates DS-1 (1.544 Mb/s) Asynchronous ("Plesiochronous") [Non-Standardized] DS-3 (45 Mb/s) VT1.5 (1.7 Mb/s) SONET STS-3 (156 Mb/s) STS-12 (622 Mb/s) STS-48 (2500 Mb/s) STS-192 (10000 Mb/s) STS-1 (52 Mb/s) Capacity (DS-1 Equiv) 1 28 84 336 1344 5376 VC-11 SDH STM-4 STM-16 VC-3 STM-1 STM-64 For high capacity transport, lower bit-rate signals like a DS-1 are combined together into successively larger signals. The older, asynchronous hierarchy combined up to 28 DS-1’s together in a single 45 Megabit/sec (Mb/sec) signal. A number of non-standardized higher rates were also used. (AT&T combined 36 DS-3’s together into a 1700 Mb/sec signal, for example. During the last decade, a new set of synchronous standards have emerged. In the U.S., this is the “Synchronous Optical NETwork” (SONET) standard. Outside of the U.S. a very similar “Synchronous Digital Network” (SDH) standard has been adopted. SONET is based on a 52 Mb/sec format called the “STS-1”. (STS = “Synchronous Transfer Signal”). These can be combined together using a byte-interleaving technique. An STS-n signal consists of n STS-1 signals combined together in this fashion. In most established networks, such as AT&T’s, it would be too expensive to replace all the existing DS-1 and DS-3 based equipment. Instead each DS-3 is encapsulated in an STS-1 for SONET transport. SDH is based on the 155 Mb/sec STM-1. This is “almost” identical to the SONET STS-3. DS: Digital Signal SONET: Synchronous Optical NETwork (US) SDH: Synchronous Digital Hierarchy (ITU) STS: Synchronous Transport Signal STM: Synchronous Transfer Mode VC: Virtual Container VT: Virtual Tributary

SONET Rates Optical Designation Bit Rate (Mb/s) Level STS-1 OC-1 51.840 STS-3 OC-3 155.520 STS-12 OC-12 622.080 STS-48 OC-48 2,488.320 STS-192 OC-192 9,953.280 STS-768 OC-768 39,813.120 A Summary of the SONET rates. "STS" refers to the protocol - the frame structure. When an STS-N is transported optically it is frequently called an "OC-N" (for "Optical Carrier"); when transported electrically it can be called an "EC-N". STS = SYNCHRONOUS TRANSPORT SIGNAL OC = OPTICAL CARRIER (“..result of a direct optical converions of the STS after synchronous scrambling” - ANSI) EC (Not Shown) = ELECTRICAL CARRIER

SONET STS-1 Frame Structure 87 Bytes 3 t T O H 87 Columns Ptr P O H P O H SPE F I x e d S t u f F I x e d S t u f Synchronous Payload Envelope (SPE) t 9 Rows The STS-1 has an 810 byte frame, usually shown as 90 columns and 9 rows. This frame is transmitted 8000 times a second - the same rate as a DS1 or DS3, for a total of 51.84 Mbit/sec. There are 3 overhead columns (described later). These contain a pointer showing where the "Synchronous Payload Envelope" (SPE) containing the information being transmitted starts. There is a significant amount of overhead, which is detailed in the "Efficiencies" table in the lower right. For example, if an STS-1 is used to transport a DS3, only 86.3% of the 51.84 Mb/sec STS-1 signal is carrying content. 87 Bytes 3

STS-N And STS-Nc (N = 3, 12, 48, 192) STS-N Formed By Byte-Interleaving N STS-1 Signals 3N Columns of Transport Overhead Frame Aligned Redundant Fields Not Used - eg APS, Datacomm N Distinct Payloads (87N Bytes) NOT Frame Aligned N Columns Of Path Overhead - All Used 2N Columns Of Fixed Stuff Bytes 84N Columns Of Information STS-Nc Single Payload 1 Column Of Path Overhead 3N - 1 Columns Of Fixed Stuff Bytes 87N - N/3 Columns Of Information Higher rate signals are created by byte-interleaving STS-1. This puts 3N columns of OH in an STS-N, even though only 3 of these columns are normally used. STS-N and STS-Nc differ in the structure of their payloads: STS-N has N separate 87-column SPE's, each with a Path OH column. STS-Nc has one large N * 87 column SPE, and has only 1 Path OH column. The STS-Nc is of particular value for data communications. It allows the use of a single queue at the transmit end of the circuit.

Virtual Container (VC) Multiplex Section (MS) SONET/SDH Layering Multiplexer Or Other PTE Cross- Connect Regenerator (derived clock) SONET Path Line Section SDH Virtual Container (VC) Multiplex Section (MS) Regenerator Section (RS) Key Feature: Basis Of Fault Management & Restoration Maintenance

Service Survivability Objectives Typical Commercial Networks 1000 100 10 Restoration Time Objectives (secs) 1 0.1 0.01 Standard Voice Leased Lines Frame Relay IP Services New trends in IP services: supporting real-time application, e.g. voice and video, & mission critical data => Require much faster restoration than traditional IP rerouting

Network Outage Analysis Voice Services Equipment Failure Route Failure Backhoe, Flood, Train Wreck ~1/1000 km/year

Survivability 101 Survivability Requires: Fault Detection Switch Fabrics To Put Failed Facility On This Capacity Spare Inter-Office Capacity X A B Y Protection: Pre-Allocated Restoration: Dynamically Allocated Survivability Requires: Each line in this example represents a fiber route. Suppose there is a facility routed X-A-B-Y. If there is a failure between A and B, this circuit could be rerouted onto X-A-C-D-B-Y. This requires spare capacity on A-C, C-D, and D-A. It also requires a switching capacity at A, C, D, and B to reroute the failed circuits. Fault Detection Control Logic To Identify Fault & Reroute Failed Circuits

Effect Of Restoration Topology Restoration Overbuild (Protection Capacity/Service Capacity) Degree 2 Nodes "Ring" "Mesh" 100% 50% Degree 3 Nodes The topology of a network puts a lower bound on the amount of additional capacity required for complete restoration. Suppose the load on each link is constant. If there are N diverse paths between each pair of nodes, then the extra capacity required to restore a single path failure can be spread over the remaining N-1 paths. When rings are used, in essence the network is decomposed into a set of "Degree 2" subnetworks. Thus 100% "Overbuild" is required. If a restoration method allowing more sharing of restoration capacity is used, it is normally possible to come close to needing only 1/(D - 1) additional capacity, where D is the average nodal degree. However, it is difficult to restore as rapidly as a ring if a more complex rerouting algorithm is used, because of the time required to determine the state of the network after a failure and to determine which reroutes to apply. 1/(N-1) Degree N Nodes Degree = # Of Physically Diverse Routes

Ring Example SONET: Bi-Directional Line-Switched Ring (BLSR) SDH: Multiplex Section Shared Protection Ring (MS-SPRING) S S: Service B C P P: Protection Different Fibers But Same Cable A D Original Circuit Now suppose that the Service OC-48 between E and F fails. The switch fabrics in the 2 ADM's reroute the traffic over the protection OC48. F E Protection Switch

X Ring Example S B C P A D F E SONET: Bi-Directional Line-Switched Ring (BLSR) SDH: Multiplex Section Shared Protection Ring (MS-SPRING) S S: Service Standardization: Physical Layer & Signaling Standardized Client State Information Not Standardized OAM Not Standardized B C P P: Protection Different Fibers But Same Cable A D Original Circuit Now suppose that the Service OC-48 between E and F fails. The switch fabrics in the 2 ADM's reroute the traffic over the protection OC48. X F E Protection Switch

Public Switched Telephone Network (PSTN) Central Office Toll Network Toll Connect Trunks Inter-Toll Trunks Switches: Terminate Trunks Switch Individual Calls 64 Kb/sec FDX Circuits Central Office CO Customer Premises Equipment (CPE)

Basic Service Types CPE Central Office PSTN Transport Network Switch ~ 100 Intercity Switches* Switch "POTS" PBX Transport Network Shared By Many Services ~10x As Many Offices* Private Line (PL) POTS: Plain Old Telephone Service * ATT Network

Entering The Transport Network 64 kb/s 1.5 Mb/s 45 - 622 Mb/s 2.5 - 10 Gb/s POTS 1 1 o 28 O 1.5 Mb/s PL 45 - 2500 Mb/s PL 1Gb Ethernet 1 - 10 Gb/s PL, 10 Gb WAN Ethernet & VG PL O D S 1 W D M Backbone Fiber Network 24 D S 3 O C 192 10 Gb WAN Ethernet SONET Framed - 9.953 Gb/s Asynchronous * POTS: "Plain Old Telephone Service" VG: Voice Grade PL: Private Line

Service Routing Transport Layer Service Layer (e.g., POTS or PL)

Outline Transport - Traditional TDM Networks Optical Networks Optical Networking & IP There will be 5 lectures, as shown on this Viewgraph. Each lecture will be about 3 hours long, with a 15 minute break in the middle. These lectures are based primarily on my experience as architect of AT&T’s domestic transport network and designer of the algorithms which AT&T uses to engineer this network. I will try to highlight the issues which AT&T and our U.S. competitors are wrestling with today. The lectures were initially developed to train AT&T network planners and researchers. The version you will be seeing has been modified to make it more appropriate for university seminars and industry meetings. I hope you will forgive the focus on the U.S. telecommunications industry. This is primarily due to my limited understanding of the situation in China. Through your questions I may be able to help you relate our experience to the problems your country will be facing.

Fiber Structure Lightpack Cable Design Typical Loss: 0.2 – 0.25 dB/km Protection Layers Pure Glass Core 8.3 micron* Glass Cladding Protects “core” Serves as a “Light guide” 125 micron Inner Polymer Coating Outer Polymer Coating 250 micron Typical Loss: 0.2 – 0.25 dB/km Plus Connector Loss Single Fiber * Single Mode Fiber; Multi-Mode Has A 50 Micron Core

Intercity Fiber Network About 50,00 Route Miles Of Fiber Cable

All-Optical Amplification Of Multi-Wavelength Signal!!! Optical Amplifier/WDM Revolution Frequency-registered transmitters Receivers R l1 All-Optical Amplification Of Multi-Wavelength Signal!!! R l2 OA OA WDM Mux WDM DeMux l3 R 40 - 120 km (80 km typically) lN R Up to 10,000 km (600 km in 2001 basic commercial products) WDM: Wavelength Division Multiplex OA: Optical Amplifier

A. Willner

Optical Amplifier/WDM Revolution Conventional Transmission - 20 Gb/s 1310 RPTR LTE 40km DS3 120 km OA OC-48 DS3 OC3/12 12 fibers 1 fiber; 36 regenerators 1 optical amplifier In Each Direction: 12 Fibers 36 Regenerators Economic Advantage Is Distance Dependent Intercity: Compelling Metro: Depends On Dark Fiber Availability WDM: Wavelength Division Multiplex OA: Optical Amplifier

* Single Fiber Capacity Bandwidth (Bits / l) Capacity = (Bandwidth/ l) Moore's Law Bandwidth (Bandwidth/ l) (Bits / l) * Capacity = Source: K. Coffman & A. Odlyzko, “Internet Growth: Is There A Moore’s Law For Data Traffic?” (research.att.com/~amo)

Single Fiber Capacity Bandwidth Fiber loss 1300 1525 1565 1600 1400

Single Fiber Capacity Bandwidth/l - C Band x2 4 THz ~ 125 GHz/nm 199.0 196.0 195.0 194.0 193.0 192.0 ¦(THz) l (nm) 1505 1510 1530 1535 1540 1545 1550 1555 1560 1565 ~ 125 GHz/nm C-Band 50 GHz Spacing (For OC48) (80 l) x (2.5 GHz/l) = 200 GHz (40 l) x (10 GHz/l) = 400 GHz x2 100 GHz Spacing (For OC192)

Transport Layer Model LA CHCG LA CHCG PHNX SONET ADM Layer Hard- Wired “Packet” “Packet” 1/0 DCS 4E Service Layers LA CHCG “Packet” “Packet” LA CHCG DS1 (1.5 Mb/s) 3/1 DCS Layer ATM/IP DS3 (45 Mb/s) 3/1 DCS Core ATM/IP Layers DACS III 3/3 DCS Layer (DACS III) DS3 (45 Mb/s) PHNX SONET ADM Layer ADM ADM ADM ADM ADM ADM ADM OC48+ (2.5+ Gb/s) Hard- Wired Wavelength Path Crossconnect Wavelength Mux Section Proprietary (20-400 Gb/s) OTS (OTS: Optical Transport System) Media Layer Fiber Conduit/ Sheath

Optical Cross-Connect (OXC) Alternatives Line Rate (Proprietary) Line Rate (Proprietary) OC48 OC192 OC48 OC192 Wavelength Path Cross- Connect ADM’s SDCS Other Service Equipment D W D M Wavelength Multiplex Section (WMS) Cross- Connect D W D M D W D M Fewer Bits Per Port Compatible With Opaque Architecture Better For Restoration More Bits Per Port Multivendor & Restoration Issues WIXC Would Go Here Could Have Either Optical Or Electrical Fabric

Office Architecture Impact Optical Cross-Connect (Wavelength Path Cross-Connect) Service Layers Service Layers OXC Adds An Additional Cost Operationally Essential In Larger Offices & Offices With High Churn Allows Software Controlled Provisioning

Opaque Wavelength Path Crossconnect Standard cross-office optics (1.3 mm) Optical transport system (1.55 mm) Optical transport system (1.55 mm) l-Mux ... ... Wavelength Path Crossconnect (Optical or Electronic Interior) ... ... Fibers In Fibers Out ... ... Transparency = node-bypass ... ... ... ... Add ports Drop ports

Opaque Wavelength Path Crossconnect (Electrical Fabric) 640 Gbps, To 48 Tbps Power, Cooling Optical Modules Processor Module Line Modules Timing Module Transparent Switch Modules (STS-1 Granularity) Processor Module

Fixed l Lasers in Optical Switches MUX DWDM Today’s switches are surrounded with OEO –> 70% of system cost Fixed Wavelength T x R LR SR LR SR SR LR T x R M U X T x R D E M U X DWDM DWDM Fixed wavelength transponders are required for each input and output fiber T x R SR LR T x R MUX DWDM

An Early MEMS Device Free-Space Micromachined Optical Switch (FS-MOS) Switch Time < 1 ms 8x8 is 1 cm x 1 cm Opportunities To Extend To Significantly Larger Arrays On A Single Substrate Measured Switching Times Under 1 ms (500ms) 3-D MEMS (2 degrees of Freedom) seems to be the Currently preferred architecture Output fibers Micro lens Free-rotating switch-mirror array Silicon substrate Si substrate Input fibers Switch reconfigured by actuating selected micromirrors

An 8 x 8 Switch Chip size: 1 cm x 1 cm Source: L-Y. Lin

Contained Domain of Transparency TDRs TDRs Optical Domain Circuit Switch TDRs Issues: Transmission Engineering Concerns Especially For Non-Tree Topologies Fault Detection & Localization Without Wavelength Conversion Becomes Separate Single-l Networks

Optical Transport System (OTS) OTS System Length Optical Transport System (OTS) ADM o D W D M ADM o D W D M 80 km 80 km 80 km 80 km 80 km 80 km 80 km OA OA OA OA OA OA 560 Km “7 x 25 dB Spacing” Key Variables: Distance Between OA's Number of Spans

Domains Of Transparency Transponder Costs In Traditional Systems

Domains Of Transparency Ultra Long-Haul (ULH) Economics 1 3 5 7 9 No. Of Standard OTS Systems (5 span) In Series 1.25 1.5 1.75 2 Utilization (%) 100 ULH/Standard Cost Ratio ULH Wins Standard Transponder OA/OADM DWDM ~500 km OTS Type Standard . . A B C D Typical ULH Technology Enhancements: Strong Forward Error Correction Raman Amplification Dynamic Power Management More Expensive Terminals & OA’s Fewer Transponders At Intermediate Locations ~5000 km Ultra-Long Haul (ULH)

Contained Domain of Transparency TDRs TDRs Optical Domain Circuit Switch TDRs Issues: Transmission Engineering Concerns Especially For Non-Tree Topologies Fault Detection & Localization Without Wavelength Conversion Becomes Separate Single-l Networks Single-Vendor For The Forseeable Future

Distance Before OEO Regen Limiting Factors PMD Constraint ~ (B * DPMD)-2 Nonlinearities Launch Power (PL) Operating Region ASE Constraint ~ PL/SNRmin In Large All-Optical Domains Each Vendor Trades Off The Design Parameters Differently This Makes Routing In Multi-Vendor Networks Difficult To Standardize Other System Parameters Length Of All-Optical Path PL Launch Power SNRmin Min SNR PMD Polarization Mode Dispersion B Bandwidth of l DPMD PMD Parameter (fiber dependent) ASE Amplified Spontaneous Emission Refs: A. Chiu, J. Strand, R. Tkach, "Issues for Routing In The Optical Layer", IEEE Communications (2001) J. Strand (ed.), IETF I-D "Impairments And Other Constraints On Optical Layer Routing", draft-ietf-ipo-impairments-01.txt

Outline Transport - Traditional TDM Networks Optical Networking Optical Networking & IP There will be 5 lectures, as shown on this Viewgraph. Each lecture will be about 3 hours long, with a 15 minute break in the middle. These lectures are based primarily on my experience as architect of AT&T’s domestic transport network and designer of the algorithms which AT&T uses to engineer this network. I will try to highlight the issues which AT&T and our U.S. competitors are wrestling with today. The lectures were initially developed to train AT&T network planners and researchers. The version you will be seeing has been modified to make it more appropriate for university seminars and industry meetings. I hope you will forgive the focus on the U.S. telecommunications industry. This is primarily due to my limited understanding of the situation in China. Through your questions I may be able to help you relate our experience to the problems your country will be facing. Concentrate On Intercity Networks Time Constraint Metro, Access Optical Networks More Complex, Less Mature

IP Transport Transport For IP - IP For Transport - Data Services (Mostly IP-Based) Voice & Other TDM-Based Services DS1 (1.5 Mb/Sec) Transport For IP - Defining Functionality Of These Interfaces Wideband & Broadband DCS Layers DS3 (45 Mb/Sec) - STM-4 (622 Mb/Sec) Digital Transmission Layer IP For Transport - Introducing IP Functionality Into The Optical Layer STM-16c (2.5 Gb/Sec) - STM-64c (10 Gb/Sec) Optical Layer Proprietary (20 Gb/Sec - 400+ Gb/Sec) Media Layer

IP Transport Transport For IP - IP For Transport - Data Services (Mostly IP-Based) Voice & Other TDM-Based Services DS1 (1.5 Mb/Sec) Transport For IP - Defining Functionality Of These Interfaces Wideband & Broadband DCS Layers DS3 (45 Mb/Sec) - STM-4 (622 Mb/Sec) Digital Transmission Layer IP For Transport - Introducing IP Functionality Into The Optical Layer STM-16c (2.5 Gb/Sec) - STM-64c (10 Gb/Sec) Optical Layer Proprietary (20 Gb/Sec - 400+ Gb/Sec) Media Layer

Replacing The OLXC With A Router IP For Transport Replacing The OLXC With A Router IP Services Non-IP Services Non-IP Services IP Router OXC OLXC Office Architecture “Big Fat Router” Office Architecture

Comparing The Architectures IP For Transport Comparing The Architectures Ports & Assumed Costs OLXC $x Per OC48 IP Router $y Per OC48 IP Router Through (a) Terminating (1 – a) IP Router “Big Fat Router” Office Architecture OLXC OLXC Office Architecture OLXC Architecture Less Expensive If: OLXC Cost x Typical Values: a = 0.8 x/y << 0.2 Router Cost < a y

MPLS Transport Hierarchy IP X s LSP s LSP X Physical Transmission System SONET (STS-N) OCh Etc. LSP t LSP u LSP Y LSP a LSP s LSP t Label Switched Path's (LSP's) Are LOGICAL, NOT PHYSICAL Need Not Occupy Bandwidth Specific LSP’s Change At Each MPLS Node: z End-to-end connection defined at set-up

MPLS Tunneling "Virtual" Muxing - No Utilization Penalty POP 3 PUSH 7 SWAP 7=>11 PUSH 42 11 42 SWAP 42 => 88 11 88 POP 88 SWAP 11=>3 3 LSP 3 LSP 7 LSP 11 LSP 42 LSP 88 "Virtual" Muxing - No Utilization Penalty This Is A Key Driver For Replacing TDM

TDM Multiplexing DS1 DS3 STS-48 STS-48 DS3 DS3 Tunneling Using MPLS LSP's Is Analogous To TDM Multiplexing DS1 DS3 STS-48 STS-48 DS3 DS3

From MPLS To GMPLS GMPLS: Generalized MPLS Implicit Label (1) (2) POP 3 PUSH 7 SWAP 7=>11 PUSH 42 11 42 SWAP 42 => 88 11 88 POP 88 SWAP 11=>3 3 LSP 3 LSP 7 STS-192 (1) STS-192 (2) LSP 11 LSP 42 LSP 88 GMPLS: Generalized MPLS

GMPLS In An OXC Network Vision: 1. Select source, destination, and service 2. OSPF determines optimal route 3. RSVP-TE/CR-LDP establishes circuit Label Request Message Label Mapping Message Vision: Provisioning Time: Weeks To Milliseconds Greatly Simplify Process ISSUE: Standards Lagging Need - Proprietary Control Planes Are Being Deployed Rapidly Source: Sycamore OFC2000

Many Technologies - One Network GMPLS Vision Many Technologies - One Network FS: Fiber Switched LS: Lambda Switched PS: Packet Switched FA: Forwarding Adjacency

GMPLS Overlay Network Model ~ Router Connection Requests, etc. Optical Network UNI Overlay Network Optical Network (OXC) computes the path Network Level Abstraction For IP Control Plane

GMPLS Peer Network Model Router ~ Topology & Capacity Information Optical Network Network Signalling Peer Network Router computes the path (Routers have enough information about the characteristics of the optical devices/network) Link-level abstraction For IP Layer Control Plane

Canarie OBGP Current View of Optical Internets ISP AS 4 AS 1 Customers buy managed service at the edge Optical VLAN AS 1 Customer AS 3 BGP Peering is done at the edge Big Carrier Optical Cloud using MPLS and IGP for management of wavelengths for provisioning, restoral and protection AS 2 B. St. Arnaud

BGP Routing + OXC = OBGP Canarie OBGP BGP Routing + OXC = OBGP Virtual BGP Routing process runs on Router B CPU which controls optical switch AS 200 180.10.10.0 BGP Neighbor 1.1.1.1 3.3.3.2 BGP Neighbor Router B Metric 100 Metric 100 1.1.1.2 2.2.2.3 4.4.4.3 3.3.3.1 Router A Metric 200 Metric 200 Router C 2.2.2.4 4.4.4.4 AS 100 170.10.10.0 AS 300 190.10.10.0 B. St. Arnaud

Dark Fiber Mapped to Dim Wavelength Canarie OBGP Vision School Aggregating Router ISP Controlled Optical Switch ISP A Dark Fiber OiBGP IGP BGP Multi Home Router IGP Dark Fiber Mapped to Dim Wavelength Customer Controlled Optical Switch ISP Controlled Optical Switch BGP neighbors University X IGP IGP OBGP OBGP Customer Owned Dark Fiber ISP B OBGP University Y B. St. Arnaud

Optical Interworking Forum Services Concept ISP AS 4 AS 1 Customers buy managed service at the edge Optical VLAN AS 1 Customer AS 3 BGP Peering is done at the edge Bandwidth On Demand - Connection Request Over UNI Specifying QoS Desired - Overlay Model OVPN - Dedicated Subnet Configured By Customer - Peer Model AS 2

Examples of network views View from domain A via a distance-vector or path-vector protocol Domain 2 Reachable Address list Domain 1 Reachable Address list Domain 4 Reachable Address list Domain 3 Reachable Address list Domain 5 Reachable Address list

Examples of Network views View from any domain of the rest of the network via a link state protocol Protection 1:N, N=3 Available BW = … SRLG = … Domain 2 Reachable Address list Domain 1 Reachable Address list Protection 1+1 Available BW = … SRLG = … Protection 1+1 Available BW = … SRLG = … Protection 1+1 Available BW = … SRLG = … Domain 4 Reachable Address list Protection 1:N, N=7 Available BW = … SRLG = … Protection 1+1 Available BW = … SRLG = … Domain 3 Reachable Address list Domain 5 Reachable Address list Protection 1:N, N=10 Available BW = … SRLG = …

Initial OIF NNI Target Single carrier’s network User control Domain Control Domain A C UNI NNI B firewall L2/L3 Load Balancer Single carrier’s network NNI Why Single Carrier Multi-Domain First? Standards Lag Deployment - Vendor Proprietary Control Planes Rapid & Unpredictable Technological Change Makes It Unlikely That Standards Will Keep Up Uncertain Business Model Initial Multi-Carrier NNI Likely To Be LEC/IXC (JLS Opinion)

Metro/Core Characteristics oif2001.639 - Application-Driven Assumptions And Requirements Metro/Core Characteristics 1. Significant Differences In Technology, Economic Trade-Offs, & Services Supported 2. Likely To Be Multi-Vendor 3. Proprietary Or Customized IGP's Are Likely 4. Significant Operational Autonomy Information Trust, Not Always Policy Trust Domains Likely To Require Control Of The Use Of Their Resources 5. Routing Carrier-Specific NMS May Be Involved High Unit Costs, Long Connection Times Make Economics An Important Consideration 6. Conduit & Fiber Cable Sharing Make SRG Information Across Domains Complex - Will Frequently Not Be Available Metro Metro A X Metro Y J K

Metro/Core Characteristics oif2001.639 - Application-Driven Assumptions And Requirements Metro/Core Characteristics 1. Significant Differences In Technology, Economic Trade-Offs, & Services Supported 2. Likely To Be Multi-Vendor 3. Proprietary Or Customized IGP's Are Likely 4. Significant Operational Autonomy Information Trust, Not Always Policy Trust Domains Likely To Require Control Of The Use Of Their Resources 5. Routing Carrier-Specific NMS May Be Involved High Unit Costs, Long Connection Times Make Economics An Important Consideration 6. Conduit & Fiber Cable Sharing Make SRG Information Across Domains Complex - Will Frequently Not Be Available Y Metro Metro Metro A Y J K

Multi-Vendors In Backbone - Characteristics oif2001.639 - Application-Driven Assumptions And Requirements Multi-Vendors In Backbone - Characteristics 1. Proprietary Or Customized IGP's Are Likely 2. Information & Policy Trust Not Likely To Be An Issue 3. Vendor-Specific Technologies & Constraints Not Captured In Standards Are Likely (E.g., All-Optical, Tunable Lasers, Adaptive Wavebands) J K L A Z N P Q S T U M R B Y Routing Costs: A(nodes) + B(distance) Large A Small B Small A Large B Opaque Network (Vendor A) Express Domain of Transparency (Vendor B) J K L N P Q S T U M R Z A J K L N P Q S T U M R B Y

IP Transport Transport For IP - IP For Transport - Data Services (Mostly IP-Based) Voice & Other TDM-Based Services DS1 (1.5 Mb/Sec) Transport For IP - Defining Functionality Of These Interfaces Wideband & Broadband DCS Layers DS3 (45 Mb/Sec) - STM-4 (622 Mb/Sec) Digital Transmission Layer IP For Transport - Introducing IP Functionality Into The Optical Layer STM-16c (2.5 Gb/Sec) - STM-64c (10 Gb/Sec) Optical Layer Proprietary (20 Gb/Sec - 400+ Gb/Sec) Media Layer

Traffic On U.S. Long Distance Network 1997 – 1999 Source: K. Coffman & A. Odlyzko

Entering The Transport Network 64 kb/s 1.5 Mb/s 45 - 622 Mb/s 2.5 - 10 Gb/s POTS & VG PL 1 24 O 1 o 28 BW Growth Rates 1.5 Mb/s PL 45 - 622 Mb/s PL Backbone Fiber Network 1 - 10 Gb/s PL POTS: "Plain Old Telephone Service" VG: Voice Grade PL: Private Line

US Domestic Backbone (Mid-’99) 268,794 OC-12 Miles

X X Transport Layering Functionality & Value Added Data Services (Mostly IP-Based) Voice & Other TDM-Based Services X X Functionality & Value Added DS1 (1.5 Mb/Sec) Wideband & Broadband DCS Layers DS3 (45 Mb/Sec) - OC-12 (622 Mb/Sec) Digital Transmission Layer ADM's Rings OC-48c (2.5 Gb/Sec) - OC-192c (10 Gb/Sec) Optical Layer Optical Transport Systems (DWDM, OA,OADM) "Optical Cross-Connects" Proprietary (20 Gb/Sec - 400+ Gb/Sec) Media Layer Fiber Conduit

Possible Solution Elements Transport For IP Customer Drivers Possible Solution Elements Price - $/OC48/month Availability How Quickly Where Displacement Of Internal ISP Costs Interfaces Cost Of Reliability Buffer Capacity Peak Loads Traffic Shifts Traffic Growth Network Management Differentiators QoS Rapid Provisioning Optical Network Interworking Heterogeneous Technologies Metro/Core Other Backbone Providers Flexible Bandwidth Asymmetric Circuits Concatenated Links Virtual Concatenation Inverse Multiplexing Additional Customer Restoration Options Re-Provisioning Customer Control Speed Options Sub-OC48 Functionality Layer 1 Interface Enhancements

Restoration Refresher Key Trade-Off Restoration Granularity Unit Capacity Cost Optical Layer Services Layers Digital Transmission DCS Layers Connection STS-1 => STS-12 STS-48+ l or Fiber Services Layer (IP) Can Restore Exactly The Right Connections Optical Layer More Economical If Large Bundles Of Connections Need To Be Restored

ISP Peering Relationships Customer Provider EXPENSIVE Peer (Frequently) No $$

Reducing The BGP Hop Count Transport For IP Reducing The BGP Hop Count A Z R C X B Y Toll Switching Hierarchy Internet ISP Hierarchy Local ISP Regional ISP Tier 1 ISP Hi-Usage Trunks Optical Direct Connects Typical Transit Cost (Telia): $1K - 10K / Mbps /Year

References T. E. Stern & K. Bala, Multiwavelength Optical Networks, Addison-Wesley, 1999 J. L. Strand, “Optical Network Architecture Evolution”, chapter in I. Kaminow and T. Li (eds.), Optical Fiber Telecommunications IV, Academic Press, to appear March 2002 R. Ramaswami and K. N. Sivarajan, Optical Networks: A Practical Perspective, San Francisco: Morgan Kaufmann, 1998. B. Mukherjee, Optical Communications Networks, New York: McGraw Hill, 1997. R. H. Cardwell, O. J. Wasem, H. Kobrinski, “WDM Architectures and Economics in Metropolitan Areas", Optical Networks, vol. 1 no.3, pp. 41-50 O. Gerstel and R. Ramaswami, "Optical Layer Survivability: A Services Perspective", IEEE Communications Magazine, vol. 38 no. 3, March 2000, pp. 104-113.    R. D. Doverspike, S. Phillips, and Jeffery R. Westbrook, "Future Transport Network Architectures", IEEE Communications Magazine, vol. 37 no. 8, August 1999, pp. 96-101. R. Doverspike and J. Yates, "Challenges for MPLS in Optical Network Restoration", IEEE Communications Magazine, vol. 39 no. 2, Feb. 2001, pp. 89-96. M. W. Maeda, "Management and Control of Transparent Optical Networks", IEEE J. on Selected Areas In Communications, vol. 16, no. 7, Sept. 1998, pp. 1008-1023. J. L. Strand, J.; A. L. Chiu, , R. Tkach,. “Issues For Routing In The Optical Layer”, IEEE Communications Magazine, 2/2001, vol. 39, no. 2, pp. 81 –87 John Strand, Robert Doverspike, Guangzhi Li, “Importance of Wavelength Conversion In An Optical Network”, Optical Networks Magazine, vol. 2 No. 3 (May/June 2001), pp. 33-44 R. W. Tkach, E. L. Goldstein, J. A. Nagel, J. L. Strand, “Fundamental limits of optical transparency”, OFC '98, pp. 161 -162

Some Relevant U.S. Web Sites “Tier 1” Inter City Service Providers AT&T http://www.att.com MCI Worldcom http://www.wcom.com Sprint http://www.sprint.com New Entrants Qwest http://www.qwest.com Level3 http://www.Level3.com Frontier http://www.frontiercorp.com Williams http://www.williams.com Major Equipment Providers Lucent http://www.lucent.com Alcatel http://www.alcatel.com Nortel http://www.nortel.com Cisco http://www.cisco.com NEC http://www.nec.com New Equipment Vendors Ciena & Lightera http://www.ciena.com Cisco & Monterey http://www.montereynets.com Avici http://www.avici.com Juniper http://www.juniper.net Sycamore http://www.sycamore.com Standards Organizations ITU http://www.itu.int T1 http://www.t1.org OIF http://www.oiforum.com IETF http://www.ietf.org ATM Forum http://www.atmforum.com New Business Models Band-X http://www.band-x.com Arbinet http://www.arbinet.com Government Sites: FCC http://www.fcc.gov NTIA http://www.ntia.doc.gov