1. Large File Transfer on 20,000 km - Between Korea and Switzerland - 2004.8.26 Yusung Kim, Daewon Kim, Joonbok Lee, Kilnam Chon yskim@cosmos.kaist.ac.kr The 3 rd International HEP DataGrid Workshop Korea Advanced Institute of Science and Technology

2. Content 1. Objective 2. Factors Affecting Performance 3. Experiment Overview 4. Experiment on 20,000 km between Korea and Switzerland 5. Experiment on 10,000 km between Korea and USA 6. Experiment Result Summary 7. Issues 8. Further Work 9. Conclusion Appendix

3. 1. Objective To obtain maximum throughput on 20,000 km between Korea and Switzerland

4. 2. Factors Affecting Performance 1) Distance 2) Number of Streams 3) Transport Protocol 4) Routing 5) Server –CPU, RAM, OS (Linux,…)

5. 3. Experiment Overview Distance 20,000 km Korea – Switzerland 10,000 km Korea – USA OrganizationKAIST – CERNKAIST – Caltech RTT (hop count)360 ms (5 hops)203 ms (7 hops) Effective Bandwidth622 Mbps Transport ProtocolsTCP Reno, FAST TCP, BIC TCP Number of Streamsfrom 1 to 100 streams

6. 4. Experiment on 20,000 km between Korea and Switzerland Organization KAIST – CERN RTT (hop count) 360 ms (5 hops) Experiment Period 2004.7.13 ~ 2004.7.25 - 5 trials, each trial for 1 hour 2004.8.20 ~ 2004.8.25 - 5 trials, each trial for 30 min Effective Bandwidth 622 Mbps Transport Protocols TCP Reno, FAST TCP, BIC TCP Number of Streams from 1 to 100 streams

7. 4.1 Network Diagram : 20,000 km between Korea and Switzerland 1 G 10 G 622M Edoardo Martelli / CERN Switzerland USA/Canada CA*net 4 host 3Com 4924 host KREONET Korea 1GbE OC 192 KAIST StarLight 1 GbE DataTag / CERN 2 x 1GbE Cisco 7609 GSR 12406 ONS 15454 Daejeon ONS 15454 Seattle 2 x OC 12 ONS 15454 Chicago 1GbE Starlight Force10 Juniper T320 10 GbE Cisco 7609

8. 4.2 Experiment results on 20,000 km between Korea and Switzerland BIC TCP FAST TCP TCP Reno number of streams throughput (Mbps) RTT : 360 ms (5hops) Effective Bandwidth : 622 Mbps

9. 5. Experiment on 10,000 km between Korea and USA OrganizationKAIST – Caltech RTT (hop count)203 ms (7 hops) Experiment Period 2004.8.20 ~ 2004.8.25 5 trials, each trial for 30 min Effective Bandwidth622 Mbps Transport ProtocolsTCP Reno, FAST TCP, BIC TCP Number of Streamsfrom 1 to 100 streams

10. 5.1 Network Diagram : 10,000 km between Korea and USA Yang Xia / Caltech host Daejeon, Korea KREONET Seattle, US 3com 4924 KAIST Caltech 1 G 10 G 622 Mbps 2 x OC 12 2 x 1GbE 1GbE PNG OC 192 Cisco7606 host GSR 12406 Cisco 7609 Abilene LA CENIC OC 192 ONS 15454 Foundry BigIron 400 ONS 15454

11. 5.2 Experiment results on 10,000 km between Korea and USA BIC TCP FAST TCP TCP Reno number of streams throughput (Mbps) RTT : 203 ms (7hops) Effective Bandwidth: 622 Mbps TCP Reno by SLAC* 10,000 km between Stanford Linear Accelerator Center (SLAC) and CERN, RTT 170 ms, 622 Mbps (see Appendix A)

12. 5.3 Result comparison : 20,000 km, 10,000 km, and 200km 20,000 km : Korea – Switzerland 10,000 km : Korea – USA 200 km : Korea – Korea number of streams Link utilization (%) DistanceOrganizationRTT (hop count)BandwidthTCP 20,000 kmKAIST – CERN360 ms (5hops)622 Mbps BIC TCP 10,000 kmKAIST – Caltech170 ms (7hops)622 Mbps 200 kmKAIST – Kyungpook Univ.2.7 ms (2hops)1 Gbps

13. 6. Experiment Result Summary Maximum Throughput TCP RenoFAST TCPBIC TCP 20,000km between Korea and Switzerland ( 360ms, 5 hpos, 622 Mbps) 385 Mbps (62%) at 100 streams 380 Mbps (61%) at 100 streams 532 Mbps (86%) at 100 streams BIC TCP performed better than others.

14. 7. Issues 1) Lambda Networking can support higher throughput on high speed and long latency network? 2) We need to analyze TCP behavior including window size management on TCP Reno, FAST TCP, and BIC TCP. 3) Some multiple streams showed lower throughput than those with smaller number of streams. (specially on 10,000 km)

15. 8. Further Work Large File Transfer experiment with 10 Gbps on 20,000 km in late 2004 or early 2005

16. 9. Conclusion We experimented large file transfer on 20,000km with 3 kinds of transport protocols. BIC TCP performed better than others. We need a further investigation of the multiple stream performance.

17. Appendix A. 10,000 km experiment between SLAC and CERN on 2003.10.23 RTT : 170 ms Effective Bandwidth : 622 Mbps Each transfer for 20 min TCP Reno TCP Reno 16 streams FAST TCPBIC TCP Throughput 278 Mbps (45%) 433 Mbps (70%) 317 Mbps (51%) 338 Mbps (54%) [http://www-iepm.slac.stanford.edu/bw/tcp-eval/cern/index.html] * SLAC : Stanford Linear Accelerator Center, http://www.slac.stanford.edu/ * CERN : Conseil Européen pour la Recherche Nucléaire, http://www.cern.ch/

18. B. 10Gbps experiment on 10,000 km between Caltech and CERN on 2004.5.6 [http://ultralight.caltech.edu/lsr-winhec/] 10959 km (172 ms) 7Gbps: transferred 860 GB in 16 min Using 8 streams

19. C. End host configuration KAIST Kyungpook Univ. CaltechCERN CPU Intel Xeon 2.8 GHz AMD dual 2.1 GHz Intel Xeon dual 2.8 GHz Intel Xeon dual 2.4 GHz RAM2 G Network Interface card Gigabit Ethernet OS Linux Redhat 9.0 (kernel 2.4.20) Transport Protocol TCP Reno, FAST TCP, BIC TCP ApplicationIperf 1.7.0

20. D. Experiment on 200km between KAIST and Kyungpook Univ. Period 2004.8.21 ~ 2004.8.23 5 trials, each trial for 30 min RTT (hop count)2.7 ms (2 hops) Bandwidth1 Gbps Transport ProtocolsTCP Reno, FAST TCP, BIC TCP Number of Streamsfrom 1 to 100 streams

21. D.1. Network Diagram: KAIST - Kyungpook Univ. 3Com 4924 KAIST 2.5 G 1GbE KREONET 1GbE Kyungpook Univ. host Cisco 7609 Cisco 7507 DaejeonDaegu

22. D.2 Experiment results on 200km between KAIST and Kyungpook Univ. BIC TCP FAST TCP TCP Reno number of streams throughput (Mbps) RTT : 2.7 ms (2 hops) 1 Gbps link