1. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 1 SONET: Broadband Convergence at Layer 1 Shivkumar Kalyanaraman Rensselaer Polytechnic Institute shivkuma@ecse.rpi.edu http://www.ecse.rpi.edu/Homepages/shivkuma Based in part on slides of Nick McKeown (Stanford)

2. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 2 Telephony: Multiplexing  Telephone Trunks between central offices carry hundreds of conversations: Can’t run thick bundles!  Send many calls on the same wire: multiplexing  Analog multiplexing  bandlimit call to 3.4 KHz and frequency shift onto higher bandwidth trunk  Digital multiplexing: convert voice to samples  8000 samples/sec => call = 64 Kbps

3. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 3 Telephony: Multiplexing Hierarchy  Pre-SONET:  Telephone call: 64 kbps  T1 line: 1.544 Mbps = 24 calls (aka DS1)  T3 line: 45 Mbps = 28 T1 lines (aka DS3)  Multiplexing and de-multiplexing based upon strict timing (synchronous)  At higher rates, jitter is a problem  Have to resort to bit-stuffing and complex extraction => costly “plesiochronous” hierarchy  SONET developed for higher multiplexing aggregates  Use of “pointers” like C to avoid bit-stuffing

4. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 4 Digital Telephony in 1984 Key System Aspects: M13 Building Blocks M13 Building Blocks Asynchronous Operation Asynchronous Operation Electrical DS3 Signals Electrical DS3 Signals Proprietary Fiber Systems Proprietary Fiber Systems Brute Force Cross Connect Brute Force Cross Connect AT&T Network/Western Electric Equipment AT&T Network/Western Electric Equipment CentralOffice CentralOffice CentralOffice Fiber Fiber Optic TransmissionSystems Switches Switches Leased Line Leased Line M13M13 DS3 DS1 DS3 DS1 Cross Connect M13 DS1 DS1 No Guaranteed TimingSynchronization

5. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 5 Digital Carrier Hierarchy (contd) q Multiplexing trunk networks: called “carrier” systems (eg: T-carrier): q allowed fast addition of digital trunk capacity without expensive layout of new cables q Time frames (125 us) and a per-frame bit in the T-carrier for synchronization => TDM q Each phone call (DS0) occupies same position in the frame q Overhead bits: error control q “robbed” bits in voice call for OAM information q Too many 0s => synch loss (max number = 15) q “yellow alarm”. 1s density etc => usable b/w = 7bits/frame => 56 kbps q Europe: E1; more streamlined framing & 2.048 Mbps q Variants: Concatenated T1, Un-channelized (raw) T1

6. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 6 Digital Hierarchy (Contd) q 1980s: demand for bandwidth. But > T3s not available except in proprietary form q Fiber-optic interface for T3 was proprietary q Primitive online OAM&P capabilities (eg: robbed bits…) q Fewer operators: interoperability/mid-span meet not critical q Changed dramatically after 1984 deregulation! q Public vs Private Networks: q Private: Customer operates n/w (eg: w/ private leased lines): developed from PBX & SNA q Public: Provider operates n/w for subscribers q More public networks (eg: X.25) outside US q Drivers of SONET: q IBM SNA/mainframes => hub-and-spoke networking q Increase of PCs => client-server & p2p computing => more demands on long-distance trunks q T-carrier evolution rate much slower than computing trends

7. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 7 Digital Hierarchy (Contd) q Digital streams organized as bytes (eg: voice samples, data) q Byte interleaving: (eg: 24 DS0 -> DS1) q service one byte from each input port into a transmission frame q Simple device: T1 mux a.k.a channel bank q Very convenient for processing, add-drop multiplexor (ADM) or Digital Cross-connect System (DCS) functions (fig 3.8/3.10) q ADM/DCS does both mux (“add”) and demux (“drop”) functions => need to do this with minimal buffering, fast/scalable processing q Bit-interleaving (eg: DS1 -> DS2 etc) q Cant use buffers to mask jitter! => bit stuffing q Partly because high speed memory was costly then! q “Plesiochronous hierarchy” => harder to ADM/DCS because full de-stuffing/de-multiplexing necessary before these functions q DS3s used to be muxed using proprietary optical methods (eg: M13 mux): SONET solves all these problems

8. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 8 US Telephone Network Structure (after 1984 divestiture)

9. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 9 Post-AT&T Divestiture Dilemmas Needs: Support Faster Fiber Support Faster Fiber Support New Services Support New Services Allow Other Topologies Allow Other Topologies Standardize Redundancy Standardize Redundancy Common OAM&P Common OAM&P Scalable Cross Connect Scalable Cross Connect DifferentCarriers,Vendors Switches Switches Leased Line Leased Line LAN Services LAN Services Data Services Data Services M13 SupportOtherTopologies, Protect Fibers DS1 Internal DS3 Cross Connect

10. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 10 The SONET Standards Process 19841985198619871988 SONET/SDHStandardsApproved ANSI Approves SYNTRAN Divestiture Exchange Carriers Standards Associate (ECSA) T1 Committee Formed ANSI T1X1 ApprovesProject Bellcore Proposed SONET Principles To ANSI T1X1 CCITT Expresses Interest in SONET British and Japanese Participation in T1X1 CCITT XVIII Begins Study Group CEPT Proposes Merged ANSI/CCITT Standard US T1X1 Accepts Modifications >400 Technical Proposals Rate Discussions AT&T vs. Bellcore Rate Discussions AT&T vs. Bellcore (resolved w/ virtual tributary concept) International Changes For Byte/Bit Interleaving, Frames, Data Rates International Changes For Byte/Bit Interleaving, Frames, Data Rates Phase I, II, III Separate APS, etc. Phase I, II, III Separate APS, etc. ITU’s SDH initiative… ITU’s SDH initiative… SONET Concept Developed By Bellcore

11. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 11 SONET Standards Story q SYNTRAN: pre-divestiture effort, no pointer concept. q SONET: primarily US (divestiture) driven q AT&T vs Bellcore debate: 146.432 Mbps vs 50.688 Mbps: compromise at 49.94 Mbps q Virtual tributary concept to transport DS-1 services q 1986: CCITT (ITU) starts own effort (SDH) q June 1987: change SONET from bit-interleaved to byte- interleaved; and rate from 49.92 to 51.84 Mbps q Phased rollouts: q 1988 = Phase 1: signal level interoperability q Phase II: OAM&P functions: embedded channel & electrical I/f specification, APS work initiated q Phase III: OSI network management adopted q Seamless worldwide connectivity (allowed Europe to merge its E-hierarchy into SDH)

12. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 12 SONET: Achievements q 1. Standard multiplexing using multiples of 51.84 Mbps (STS-1 and STS-N) as building blocks q 2. Optical signal standard for interconnecting multiple vendor equipment q 3. Extensive OAM&P capabilities q 4. Multiplexing formats for existing digital signals (DS1, DS2 etc) q 5. Supports ITU hierarchy (E1 etc) q 6. Accomodates other applications: B-ISDN etc

13. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 13 SONET Lingo q OC-N: Optical carrier Nx51.84 Mbps q Approximate heuristic: bit rate = N/20 Gbps (eg: OC-48 => 48/20 = 2.4 Gbps) q Overhead percentage = 3.45% for all N (unlike PDH!) q OC signal is sent after scrambling to avoid long string of zeros and ones to enable clock recovery q STS-N: Synchronous Transport Signal (electronic equivalent of OC) q Envelope: Payload + end-system overhead q Synchronous payload envelope (SPE): 9 rows, 87 columns in STS-1 q Overhead: management OAM&P portion q Concatenation: “un-channelized” (envelope can carry “super-rate” data payloads: eg: ATM): Eg: OC-3c q Method of concatenation different from that of T-carrier hierarchy…

14. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 14 SONET Multiplexing Possibilities Asynchronous DS-3 Virtual Tributaries for DS1 etc STS-3c for CEPT-4 and B- ISDN STS-1s are mutually synchronized irrespective of inputs

15. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 15 STS-1 Frame Format 90 Bytes Or “Columns” 9Rows Small Rectangle =1 Byte Two-dimensional frame representation (90 bytes x 9 bytes)… Frame Transmission: Top Row First, Sent Left To Right Time-frame: 125  s/Frame Frame Size & Rate: 810 Bytes/Frame * 8000 Frames/s * 8 b/byte= 51.84 Mbps For STS-3, only the number of columns changes (90x3 = 270) STS = Synchronous Transport Signal

16. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 16 STS-1 Headers 90 Bytes Or “Columns” 9Rows Section Overhead (SOH) Line Overhead (LOH) Path Overhead (POH): Floating => can begin anywhere Line + Section overhead = Transport Overhead (TOH)

17. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 17 SONET Equipment Types PTE SONET End Device - I.e. Telephony Switch, Router Section Termination (STE) Section Termination (STE)SectionsRepeaters PTE Line Line Termination (LTE) Line Termination (LTE) Path Path Termination (PTE) Path Termination (PTE)

18. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 18 SONET Overhead Processing

19. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 19 Headers: Section Overhead (SOH) A1 =0xF6A2=0x28J0/Z0STS-ID B1BIP-8E1OrderwireF1User D1 Data Com D2 D3 Section Overhead 9 Bytes Total Originated And Terminated By All Section Devices (Regenerators, Multiplexers, CPE) Other Fields Pass Unaffected RcvSOH Selected Fields: A1,A2 - Framing Bytes BIP-8 - Bit Interleaved Parity F1 User - Proprietary OAM Management XmtSOH

20. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 20 Headers: Line Overhead (LOH) H1PointerH2PointerH3 Pointer Act B2BIP-8K1APSK2APS D4 Data Com D5 D6 Line Overhead 18 Bytes Total Originated And Terminated By All Line Devices (Multiplexers, CPE) LOH+SOH=TOH (Transport OH) Selected Fields: H1-3 - Payload Pointers K1, K2 - Automatic Protection Switching D4-D12 - 576 kbps OSI/CMIP D7 Data Com D8 D9 D10 D11 D12 S1SyncM0REIE1Orderwire RcvSOHXmtSOH XmtSOH XmtLOH RcvSOH RcvLOH

21. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 21 Floating Payload: SONET LOH Pointers SPE is not frame-aligned: overlaps multiple frames! Avoids buffer management complexity & artificial delays Allows direct access to byte-synchronous lower-level signals (eg: DS-1) with just one frame recovery procedure

22. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 22 SPE: Synchronous Payload Envelope Synchronous Payload Envelope Contains POH + Data First Byte Follows First Byte Of POH Wraps In Subsequent Columns May Span Frames Up To 49.536 Mbps for Data: Enough for DS3 Defined Payloads Virtual Tributaries (For DS1, DS2) DS3 SMDS ATM PPP …

23. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 23 Headers: Path Overhead (POH) J1Trace B3BIP-8 C2 Sig Label Path Overhead H1,H2 fields of LOH points to Beginning of POH Selected fields: BIP-8 - Parity C2 - Payload Type Indicator G1 - End End Path Status G1 Path Stat F2User H4Indicator PTE Z3Growth Z4Growth Z5Tandem PTESTE Frame N Frame N+1 Frame N Frame N+1 POH Beginning Floats Within Frame 9 Bytes (1 Column) Spans Frames Originated And Terminated By All Path Devices (I.e. CPE, Switches) End-to-end OAM support

24. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 24 STS-1 Headers: Putting it Together

25. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 25 Accommodating Jitter Positive Stuff Negative Stuff To Shorten/Lengthen Frame: Byte After H3 Ignored; Or H3 Holds Extra Byte H1, H2 Values Indicate Changes - Maximum Every 4 Frames Requires Close (Not Exact) Clock Synch Among Elements

26. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 26 Clock Synchronization Building Integrated Timing System Hierarchical Clocking Distribution Hierarchical Clocking Distribution Normally All Synch’d To Stratum 1 (Can Be Cesium/Rubidium Clock) Normally All Synch’d To Stratum 1 (Can Be Cesium/Rubidium Clock) Dedicated Link Or Recovered Dedicated Link Or Recovered Fallback To Higher Stratum In Failure (Temperature Controlled Crystal) Fallback To Higher Stratum In Failure (Temperature Controlled Crystal) PTE PTE BITSBITS BITS PrimaryReference BackupReference Level 1: 10 -11Level 1: 10 -11 Level 2: 1.6x10 -8Level 2: 1.6x10 -8 Level 3: 4.6x10 -6Level 3: 4.6x10 -6 Level 4: 32x10 -6Level 4: 32x10 -6

27. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 27 STS-N Frame Format Composite Frames: Byte Interleaved STS-1’s Byte Interleaved STS-1’s Clock Rate = Nx51.84 Mbps Clock Rate = Nx51.84 Mbps 9 colns overhead 9 colns overhead 90xN Bytes Or “Columns” N Individual STS-1 Frames Examples Examples STS-1 51.84 Mbps STS-3 155.520 Mbps STS-12 622.080 Mbps STS-48 2.48832 Gbps STS-192 9.95323 Gbps Multiple frame streams, w/ independent payload pointers Note: header columns also interleaved

28. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 28 STS-N: Generic Frame Format STS-1 STS-N

29. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 29 Example: STS-3 Frame Format

30. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 30 STS-Nc Frame Format Concatenated mode: Same TOH Structure And Data Rates As STS-N Same TOH Structure And Data Rates As STS-N Not All TOH Bytes Used Not All TOH Bytes Used First H1, H2 Point To POH First H1, H2 Point To POH Single Payload In Rest Of SPE Single Payload In Rest Of SPE Accommodates FDDI, E4, data Accommodates FDDI, E4, data 90xN Bytes Or “Columns” Transport Overhead: SOH+LOH Current IP over SONET technologies use concatenated mode: OC-3c (155 Mbps) to OC-192c (10 Gbps) rates a.k.a “super-rate” payloads

31. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 31 Virtual Tributaries (Containers) q Opposite of STS-N: sub-multiplexing q STS-1 is divided into 7 virtual tributary groups (12 columns ea), which can be subdivided further q VT groups are byte-interleaved to create a basic SONET SPE q VT1.5: most popular quickly access T1 lines within the STS-1 frame q SDH uses the word “virtual containers” (VCs)

32. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 32 Virtual Tributaries: Pointers q VT payload (a.k.a VT SPE) floats inside the VT q One more level of pointer used to access it. q Can access a T1 with just two pointer operations q Very complex to do the same function in DS-3 q Eg: accessing DS0 within DS-3 requires FULL de- multiplexing: a.k.a stacked multiplexing or mux- mountains!

33. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 33 SONET Transmission Encoding OC-N Is Optical Carrier STS-N Long Reach: 40 km 1310 or 1550 nm SM Intermediate Reach: 15 km 1310 or 1550 nm SM Short Reach:Long Reach 2 km 1310 nm MM E O Electrical Transmission Standard STS-1: B3ZS (BPV), 450’ STS-1: B3ZS (BPV), 450’ STS-3: Coded Mark Inversion, 225’ STS-3: Coded Mark Inversion, 225’ Useful Intra-Office Connection Useful Intra-Office Connection 1+x 6 +x 7 Scrambling Ensures Ones Density Ensures Ones Density Does Not Include A1, A2, C1 Bytes Does Not Include A1, A2, C1 Bytes Output Is NRZ Encoded Output Is NRZ Encoded

34. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 34 SONET Scrambling

35. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 35 Packet Over SONET (POS) PPPByteStuffFCSScramblingSONETFraming Standard PPP Encapsulation Magic Number Recommended No Address and Control Compression No Protocol Field Compression Standard CRC Computation OC3 May Use CRC-16 Other Speeds Use CRC-32 Special Data Scrambler 1+ x43 Polynomial Protects Against Transmitted Frames Containing Synch Bytes Or Insufficient Ones Density SONET Framing OC3, OC12, OC48, OC192 Defined C2 Byte = 0x16 With Scrambling C2 Byte = oxCF Without (OC-3)

36. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 36 Practical SONET Architectures Today: multiple “stacked” rings over DWDM (different s)

37. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 37 SONET Network Elements Nonstandard, Functional Names TM: Terminal Mux: (aka LTE: ends of pt-pt links) ADM: Add-Drop Mux DCC: Digital Cross Connect (Wideband and Broadband) MN: Matched Node D+R: Drop and Repeat ADM TM DS1s DS1s MNMN MN DCC D+R D+R D+R MN

38. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 38 Digital Cross Connects (DCS) q Cross-connects thousands of streams under software control (replaces patch panel) q Handles perf monitoring, PDH/SONET streams, and also provides ADM functions q Grooming: q Grouping traffic with similar destinations, QoS etc q Muxing/extracting streams also q Narrow-/wide-/broad-band and optical crossconnects

39. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 39 Topology Building Blocks DCC ADM ADM ADM DCC ADM ADM ADM 2 Fiber Ring Each Line Is Full Duplex DCC ADM ADM ADM 4 Fiber Ring Each Line Is Full Duplex DCC ADM ADM ADM Uni- vs. Bi- Directional All Traffic Runs Clockwise, vs Either Way

40. Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 40 APS ADM Line Protection Switching Uses TOH Trunk Application Backup Capacity Is Idle Supports 1:n, N=1-14 Automatic Protection Switching Line Or Path Based Line Or Path Based Revertive vs. Non-Revertive Revertive vs. Non-Revertive Mechanism For Intentional Cutover Mechanism For Intentional Cutover Restoration Times ~ 50 ms Restoration Times ~ 50 ms K1, K2 Bytes Signal Change K1, K2 Bytes Signal Change Common Uses: 2 Fiber UPSR or ULSR, 4 Fiber BPSR Common Uses: 2 Fiber UPSR or ULSR, 4 Fiber BPSR ADMADMADM Path Protection Switching Uses POH Access Line Applications Duplicate Traffic Sent On Protect 1+1 ADMADM