TransPAC-Pacific Wave 100G Jennifer M. Schopf, PhD Director, International Networks Indiana University; PI NSF TransPAC IRNC award jmschopf@iu.edu Professor Ron Johnson CEO, Pacific Northwest Gigapop, Professor, University of Washington ronj@uw.edu Professor Jun Murai Dean and Professor Keio University, Founder, WIDE Supported by the National Science Foundation

Note – talk 10 mins is to include: TransPAC-PacificWave 100Gbps Tokyo-Seattle link (Schopf/Johnson/Murai) schedule, endpoints, users, purposes, connected networks, technologies]

Use this slide to set the context of the link Pacific Wave Backplane Use this slide to set the context of the link

TransPAC-Pacific Wave 100G Dedicated 100Gb/s wavelength between Pacific Wave in Seattle and Tokyo, Japan Network fabric provided by Pacific Northwest Gigapop Points of Presence in Tokyo at WIDE/T-REX and at Tata Pacific Wave Points of Presence in USA are in Seattle, Los Angeles & Chicago (@ StarLight) 5 year agreement

Endpoint: Seattle TransPAC/Pacific Wave is connected at 100g to Pacific Wave >100g distributed peering fabric which has major peering points in Seattle (Westin) and Los Angeles (‘One Wilshire’ complex), and has other peering capabilities in Sunnyvale & Chicago (@ StarLight) Pacific Wave’s production 100g distributed peering fabric includes: 100g connectivity to USA DoE ESnet & Internet2 at multiple locations, as well as to major cloud providers and all 100g Pacific Rim R&E nets. 100g and/or 10g to nearly all international Pacific Rim R&E Networks the full range of major .com peerings and ‘Tier 1’ ISP connectivity Pacific Wave also includes: a dedicated 100g Science DMZ backplane between Seattle, Sunnyvale and Los Angeles with extensions to Denver, Albuquerque and Chicago. the backbone for the NSF funded ‘Pacific Research Platform’ (PRP) DMZ Dedicated SDN fabric spanning Seattle, Sunnyvale & LA Dedicated SDX fabric spanning Seattle, Sunnyvale & LA

T-REX Endpoint: Tokyo L3SW NE40E WIDE L3SW MLX38 PWave OTN WIDE OTN As of Dec 2015 Endpoint: Tokyo NTT OTEMACHI 7F TY2 4F UNIV TOKYO 100GE 3/0/0 L3SW NE40E WIDE 100GE 3/1/0 L3SW MLX38 PWave 3/2 OTN OSN9800 WIDE OTN OSN9800 WIDE 3/1 100 TATA TGN-Pacific 100 100 100 100 100G 2/0/0 T-REX NAO (H)OSN1800 WIDE Westin Building Seattle WA 10 T-LEX L3 SW MLXe8 WIDE 10 JGN-X 100 100 L3SW (B)MLX38 PWave 10 SINET Router MX2020 NII/SINET 10 100 3F NTT OTEMACHI ANNEX 9F 100GE Japan US Cable Network (JUS) Router T640 APAN-JP LA ONE-WILSHIRE Operational Apr 2016 KDDI OTEMACHI 5F EAST

Pacific Wave/Transpac4 100G Endpoint: Tokyo to CANARIE Pacific Wave/Transpac4 100G to UK, EU to Internet2 Seattle PW/RC 100G (?) 100G (April 2016) New York SINET 100G (April 2016) to Internet2 to South Amrica Tokyo Los Angels to APAN CN/RC 100G (?) TransPAC3 10G Hawaii CENIC TREX/WIDE SINET SINET TransPAC APAN Note: All dotted lines are PLAN or Uncertain Internet2

Schedule Used experimentally at SC’15 Tested clean Dec 12, 2015 Now fully operational as a production network WIDE Tokyo access & peering point now operational Interconnects to/at the Pacific Wave USA distributed peering facility now fully operational Tata-Tokyo facility router access and peering point planned to be in production by May 1 2016 XX say something about timing on peerings? XxX Anything else?

Users Usage Policy – all R&E traffic allowed TransPAC/Pacific Wave is technically AUP-free Pacific Wave is AUP-Free Peering/routing to shift gradually, starting at 2015 years end, discussion at APAN in January TransPAC is supporting end-user engagement to increase efficient use of the networks Example application from SC’15:

University of Tokyo TCP Experiment at SC’15 250-300ms Pacific Wave BackPlane 500-600ms Example from SC’15 University of Tokyo TCP Experiment at SC’15

University of Tokyo Test over 100G ultra long distance links Investigating what happens to TCP performance over ultra long, ultra large circuits Cubic-TCP Jumbo Frames CPU Load Limitations of the TCP Protocol 1GB Max buffer (Linux kernel limitation) 32 bit sequence numbers (RFC 793) Example from SC’15

Achieved 100G several times during conference – limitation was seattle-austin link, not this one!

Connected Networks/Topologies Via Pacific Wave, Internet2 & ESnet etc, in USA All major R&E networks including multiple 100g connections to Internet2 and ESnet Nearly all Pacific Rim R&E Networks StarLight and all European Networks attached Ampath and all South American Networks attached Peering with most major .com’s Connectivity to most major Tier 1 ISP’s Peering (often at 1 or more 100g) to major Clouds Peering with other major peering sites

Connected Networks/Topologies Via Tokyo All WIDE peerings & interconnects Any other connections/peering via WIDE/T-REX Any other connections/peering via TransPAC-Pacific Wave router at Tata Plans to peer with APAN, TEIN, and prior collaborators on TransPAC

Pacific Wave’s: Peering + Transit; Research DMZ (which provides the NSF funded “Pacific Research Platform” (PRP) backbone); plus SDX & SDN capabilities Peerings (International & USA) Research DMZ core node 100g Internet2 AL2S & AL3S 100g ESnet Tier 1 ISP ports .com & cloud peerings Peerings (International & USA) Research DMZ core node 100g Internet2 AL2S & AL3S Tier 1 ISP ports .com & cloud peerings Tokyo @ WIDE & Tata CANARIE 100g TransPAC Pacific Wave 100g Seattle PNWGP 100g Chicago @ StarLight 100g Testbed 10g SDN 100g Shared wave including Transit and DMZ, SDN, SDX & Peering access Dedicated 100g waves for: Peering/Transit; Research DMZ (incl. Pacific Research Platform (PRP)); and vLANS for SDX & SDN Dedicated 10g waves for SDN and SDX Testbed 10g SDX Peering DMZ Core Denver FRGP/NCAR PIREN / AARNET 40g Sunnyvale CENIC 100g Testbed 10g SDX Testbed 10g SDN Albuquerque NMGP Peering DMZ Core Los Angeles CENIC 100g 100g PIREN/AARNET 40g El Paso International Access PoP Peerings (International & USA) Research DMZ core node 100g Internet2 AL2S & AL3S 100g ESnet Tier 1 ISP ports .com & cloud peerings Hawaii Jan 1 2016

TransPAC plans for 2016 additional circuit & peerings Had planned to partner on the Singaren-I2 100g Collaboration but won’t own part of it Interconnect in Tokyo as well as LA-Pacific Wave?! Exploring alternative options away from our traditional paths Guam to ? Peer in Tokyo and/or Seattle with all other willing R&E nets or peering points

Acknowledgements IN@IU is funded by US NSF award #0962968 for TransPAC3 US NSF award #1450904 for TransPAC4 The TransPAC PacificWave 100gb/s network fabric is provided by Pacific Northwest GigaPop Pacific Wave is partially funded by a NSF IRNC award to CENIC & Pacific NorthWest Gigapop XX please add others as needed