1. Atlas Canada Lightpath Data Transfer Trial Corrie Kost, Steve McDonald (TRIUMF) Bryan Caron (Alberta), Wade Hong (Carleton)

2. The iGrid2002 Network

3. TRIUMF Local Loop Only one SM fiber pair available between TRIUMF and the CA*net 4 ONS at the BCnet POP CWDM gear arrived at TRIUMF on Sept 4th Extreme 10GbE LR blades arrived on the same day

4. CERN Local Loop Extreme BD with LR Blade arrives on Sept 17 within 2 days of shipping ONS finally installed and operational on Sept 20th

5. We are live continent to continent! e2e lightpath up and running 21:45 CET traceroute to cern-10g (192.168.2.2), 30 hops max, 38 byte packets 1 cern-10g (192.168.2.2) 161.780 ms 161.760 ms 161.754 ms

6. The Network Hardware At CERN and TRIUMF –Intel 10GbE Server NICs (alpha) –Extreme Networks Black Diamond 6808 At TRIUMF –MRV Nbase Xyplex Fiber Driver WDM44s to multiplex 2GbEs to CA*net 4 ONS at BCnet POP

7. Intel 10GbE Cards Intel kindly loaned us 2 of their Pro/10GbE LR server adapters cards despite the end of their Alpha program –based on Intel® 82597EX 10 Gigabit Ethernet Controller

8. Extreme Networks Extreme Networks generously loaned and shipped to us 2 Black Diamond 6808s with their new10GbE LRi blades

9. Extreme Networks TRIUMF CERN

10. The Endpoint Hosts Linux-based PC Servers –SuperMicro P4DL6 (Dual Xeon 2GHz) –ServerWorks GC-LE Chipset –400 MHz front side bus –1 GB DDR2100 RAM –Dual Channel Ultra 160 onboard SCSI –SysKonnect 9843 SX GbE –3 independent PCI buses –PCI-X 64 bit/133 Mhz capable

11. IDE Disk Arrays CERN Receive HostTRIUMF Send Host

12. Operating System Redhat 7.3 based Linux kernel 2.4.18-3 –Needed to support filesystems > 1TB Upgrades and patches –Patched to 2.4.18-10 –Intel Pro 10GbE Linux driver (early stable) –SysKonnect 9843 SX Linux driver (latest) –Ported Sylvain Ravot’s tcp tune patches

13. BBFTP Transfer Vancouver ONS ons-van01(enet_15/1) Vancouver ONS ons-van01(enet_15/2)

14. BBFTP Transfer Chicago ONS GigE Port 1 Chicago ONS GigE Port 2

15. Tsunami Transfer Vancouver ONS ons-van01(enet_15/1) Vancouver ONS ons-van01(enet_15/2)

16. Tsunami Transfer Chicago ONS GigE Port 1 Chicago ONS GigE Port 2

17. Sunday Nite Summaries

18. Tuesday Nite Summaries Vancouver ONS ons-van01(enet_15/1) Vancouver ONS ons-van01(enet_15/2)

19. Comparative Results Tool Transferred Average Max Avg wuftp 100 MbE600 MB 3.4 Mbps wuftp 10 GbE6442 MB 71 Mbps iperf275 MB940 Mbps1136 Mbps pftp600MB532 Mbps bbftp (10 streams)1.4 TB666 Mbps710 Mbps Tsunami - disk to disk0.5 TB700 Mbps825 Mbps Tusnami - disk to memory12 GB> 1GBps

20. Lessons Learned Linux software RAID faster than most conventional SCSI based systems More disk spindles the better Channel bonding of two GbEs seems to work very well (on an unshared link) The larger files the better for thruput

21. Acknowledgements Canarie –Bill St. Arnaud, Rene Hatem, Damir Pobric, Thomas Tam, Jun Jian Atlas Canada –Mike Vetterli, Randall Sobie, Jim Pinfold, Pekka Sinervo, Gerald Oakham, Bob Orr, Michel Lefebrve, Richard Keeler HEPnet Canada –Dean Karlen TRIUMF –Peter Gumplinger, Fred Jones, Mike Losty, Jack Chakhalian, Renee Poutissou BCNET –Mike Hrybyk, Marilyn Hay, Dennis O’Reilly, Don McWilliams

22. Acknowledgements Extreme Networks –Amyn Pirmohamed, Steven Flowers, John Casselman, Darrell Clarke, Rob Bazinet, Damaris Soellner Intel Corporation –Hugues Morin, Caroline Larson, Peter Molnar, Harrison Li, Layne Flake, Jesse Brandeburg

23. Acknowledgements Indiana University –Mark Meiss, Stephen Wallace Caltech –Sylvain Ravot, Harvey Neuman CERN –Olivier Martin, Paolo Moroni, Martin Fluckiger, Stanley Cannon, J.P Martin-Flatin SURFnet/Universiteit van Amsterdam –Pieter de Boer, Dennis Paus, Erik.Radius, Erik-Jan.Bos, Leon Gommans, Bert Andree, Cees de Laat

24. Acknowledgements Yotta –Geoff Hayward, Reg Joseph, Ying Xie, E. Siu BCIT –Bill Rutherford Jalaam –Loki Jorgensen Netera –Gary Finley