1. 1 Masaki Hirabaru Network Architecture Group PL Meeting New Generation Network Research Center July 26, 2006 A Role of Network Architecture in e-VLBI

2. 2 Network Architecture B A C D (1) Full Mesh Needs Architecture (Optimization) Seeds B A C D (2) Blocking SW

3. 3 Contents Past: Packet Switching Approach Stored On-going: Circuit Switching Approach Real-Time Single Large Data Flow

4. 4 Motivations MIT Haystack – NICT Kashima e-VLBI Experiment on August 27, 2003 to measure UT1-UTC in 24 hours –41.54 GB NICT → MIT 107 Mbps (~50 mins) 41.54 GB MIT → NICT 44.6 Mbps (~120 mins) –RTT ~220 ms, UDP throughput 300-400 Mbps However TCP ~6-8 Mbps (per session, tuned) –BBFTP with 5 x 10 TCP sessions to gain performance HUT – NICT Kashima Gigabit VLBI Experiment -RTT ~325 ms, UDP throughput ~70 Mbps However TCP ~2 Mbps (as is), ~10 Mbps (tuned) -Netants (5 TCP sessions with ftp stream restart extension) There was bandwidth available but we could not utilize.

5. 5 e-VLBI (Very Long Baseline Interferometry) e-VLBI geographically distributed observation, interconnecting radio antennas over the world ASTRONOMY GEODESY Delay(d) radio signal from a star correlator A/D clock A/D Internet clock ~Gbps A B A B d Large Bandwidth-Delay Product Network issue NICT Kashima Space Center 34m Partners: MIT Haystack, NASA, Onsala (Sweden), Shanghai Observatory, JIVE, HUT, CSIRO, etc. Gigabit / real-time VLBI multi-gigabit rate sampling NGC4261

6. 6 Observing Bandwidth  Data rate  (Precision of Time Delay) -1  (SNR) 1/2 Wave Length / Baseline Length  Angular Resolution Baseline Length  (EOP Precision) -1 VLBI - Characteristics Faster Data Rate = Higher Sensitivity Longer Distance = Better Resolution

7. 7 Long Distant Rover Control (at least) 7 minutes one way delay Image Command Earth Mars When operator saw collision, it was too late.

8. 8 Example How much speed can we get? Receiver Sender High- Speed Backbone L2/L3 SW 1G 100M Delay at light speed: 100ms 1G A higher-speed device spoils the performance ??? 100M Average TCP Throughput less than 20Mbps Q=50

9. 9 Example: From Tokyo to Boston TCP on a fast long path with a bottleneck bottleneck overflow queue loss Tokyo sender rate control Boston receiver loss detection feedback It takes 150ms to know the loss (buffer overflow). It keeps overflowing during the period… 150ms is very long for the high-speed network. 150ms at 1Gbps generates ~19MByte on the wire. Los Angeles 50ms 100ms bw 1G bw 0.8G Buffer 25MB

10. 10 TCP’s Way of Rate Control (slow-start) RTT (200ms) 20ms40ms80ms160ms t 1Gbps rate average rate 150 Mbps average rate overflows with a 1000-packet queue 100Mbps

11. 11 (a) HighSpeed (b) Scalable (c) BIC (d) FAST Bottleneck bandwidth and queue size TCP Burstyness

12. 12 TCP Performance with Different Queue Sizes

13. 13 Measuring Bottleneck Queue Sizes ReceiverSender Capacit y C packet train lost packet measured packet Queue Size = C x (Delay max – Delay min ) RouterSwitch 1Gbps (10G) 100Mbps (1G) b) Typical Bottleneck Cases RouterSwitch a) Queue ~100 Queue ~1000 VLANs

14. 14 Bottleneck Detection Black Box – Traditional by Hirabaru Gray Box – ECN (similarity: Optical Grid by Harai) White Box – SIRENS by Kobayashi and Nakauchi

15. 15 Difference in Approaches - Stored (Packet Switching): Variable Bandwidth, Available - Real-Time (Circuit Switching): Constant, Guaranteed

16. 16 NICT Koganei and DRAGON e-VLBI Experiment 2007 - Interoperability - NICT Koganei PC (.30) 9816T 10G NICT Kashima 9816GB U-Node 9816GB 10G PC (.29) FES12G Tokyo 9816GB SL F10 GS4000 Chicago 10G HOPI DRAGON MIT Haystack Washington DC PC SW Correlator SW JGN II VLAN ID 1196, 98XXX Allied Telesis CenterCOM L2 Switch, U-node NEC ADM (MTU 1500B limitation), GS4000 Hitachi L2 Switch TLD: Tunable Laser Diode (Tunable Wavelength Transmitter/Receiver) Internet DRAGON CTL? NICT(Koganei)-GMPLS GMPLS RSVP-TE JGN II GS4000 NICT-Optical Grid JGN II L1(Optical Fiber) TLD CTL TLD CTL OXC CTL GS4000 SW 10G 1G Circuit speed ENNI SW ENNI SW NICT CTL NICT CTL ENNI SW - Path/Packet API - Bottleneck shift

17. 17 e-VLBI Contributions to Network Architecture Future Requirements from e-VLBI Combination and Control Network devices and elements Additional requirements: Loss-tolerant, scientific data streaming Field Trials Problem Finding

18. 18 A Case Study: VLBI System Transitions and Architecture K5 Data Acquisition Terminal 1st Generation 2nd Generation 1983~ Open-Reel Tape Hardware Correlator 1990~ Cassette Tape Hardware Correlator e-VLBI over ATM 3rd Generation 2002~ PC-based System Hard-disk Storage Software Correlator e-VLBI over Internet K3 Correlator (Center) K3 Recorder (Right) K4 Terminal K4 Correlator 64Mbps 256Mbps 1 ～ 2Gbps Optimized Platform = Architecture

19. 19 Kwangju Busan 2.5G Fukuoka Korea 2.5G SONET KOREN Taegu Daejon 10G 1G (10G) 1G Seoul XP Genkai XP Kitakyushu Kashima 1G (10G) Fukuoka Japan 250km 1,000km 2.5G JGN II 9,000km 4,000km Los Angeles Chicago Washington DC MIT Haystack 10G 2.4G APII/JGNII Abilene Koganei 1G(10G) Indianapolis 100km bwctl server Experiment for High-Performance Scientific Data Transfer 10G Tokyo XP / JGN II I-NOC *Performance Measurement Point Directory http://e2epi.internet2.edu/pipes/pmp/pmp-dir.html perf server e-vlbi server JGNII 10G GEANT SWITCH 7,000km TransPAC Pittsburgh U of Tokyo A BWCTL account available for CMM including Korean researchers International collaboration to support for science applications

20. 20 e-VLBI Show-case for Network Architecture 2004 年 6 月 30 日 : JGN II を活用して、 VLBI 観測デー タを日米間で高速伝送することにより、地球の自 転速度の変動を表す UT1 を約 4.5 時間という極めて 短時間のうちに推定することに成功しました。 ( 報 道発表 ) IDEA Award Winner 2006 by Internet2 Very High Speed Electronic Very Long Baseline Interferometry (e-VLBI) Alan Whitney, MIT Haystack Observatory Yasuhiro Koyama, NICT Arpad Szomoru, Joint Institute for VLBI in Europe (JIVE) Hisao Uose, NTT Laboratories GEMnet2/GALAXY Project