CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 1 EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester.

Slides:

Advertisements

Similar presentations

TERENA Networking Conference, Lyngby, May 2007, R. Hughes-Jones Manchester 1 The Performance of High Throughput Data Flows for e-VLBI in Europe Multiple.

Advertisements

TCP transfers over high latency/bandwidth network & Grid TCP Sylvain Ravot

ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Network Connectivity and Projects – a Perspective from the UK Richard Hughes-Jones PPNCG SuperJANET4.

IGRID2002 Radio Astronomy VLBI Demo. uWeb based demonstration sending VLBI data A controlled stream of UDP packets Mbit/s on the production network.

DataTAG CERN Oct 2002 R. Hughes-Jones Manchester Initial Performance Measurements With DataTAG PCs Gigabit Ethernet NICs (Work in progress Oct 02)

Project Partners Project Collaborators The Council for the Central Laboratory of the Research Councils Funded by EPSRC GR/T04465/01

CALICE, Mar 2007, R. Hughes-Jones Manchester 1 Protocols Working with 10 Gigabit Ethernet Richard Hughes-Jones The University of Manchester

ER2002 Radio Astronomy VLBI Demo. uWeb based demonstration sending VLBI data A controlled stream of UDP packets Mbit/s on the production network.

JIVE VLBI Network Meeting 15 Jan 2003 R. Hughes-Jones Manchester The EVN-NREN Project Richard Hughes-Jones The University of Manchester.

GNEW2004 CERN March 2004 R. Hughes-Jones Manchester 1 End-2-End Network Monitoring What do we do ? What do we use it for? Richard Hughes-Jones Many people.

20th-21st June 2005 NeSC Edinburgh End-user systems: NICs, MotherBoards, Disks, TCP Stacks & Applications Richard Hughes-Jones.

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Networking for ATLAS Remote Farms Richard Hughes-Jones The University.

Slide: 1 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 1 Update on Remote Real-Time Computing Farms For ATLAS Trigger DAQ. Richard Hughes-Jones.

GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.

CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester 1 EU DataGrid - Network Monitoring Richard Hughes-Jones, University of.

Astronomy Sysman Meeting 29/30 April 02 R. Hughes-Jones Manchester 1 Introduction to the PPNCG Networking for the PPARC Community  Introduction to the.

Slide: 1 Richard Hughes-Jones PFLDnet2005 Lyon Feb 05 R. Hughes-Jones Manchester 1 Investigating the interaction between high-performance network and disk.

DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester Some Measurements on the SuperJANET 4 Production Network (UK Work in progress)

JIVE VLBI Network Meeting 28 Jan 2004 R. Hughes-Jones Manchester Brief Report on Tests Related to the e-VLBI Project Richard Hughes-Jones The University.

T2UK RAL 15 Mar 2006, R. Hughes-Jones Manchester 1 ATLAS Networking & T2UK Richard Hughes-Jones The University of Manchester then.

CALICE UCL, 20 Feb 2006, R. Hughes-Jones Manchester 1 10 Gigabit Ethernet Test Lab PCI-X Motherboards Related work & Initial tests Richard Hughes-Jones.

DataTAG Meeting CERN 7-8 May 03 R. Hughes-Jones Manchester 1 High Throughput: Progress and Current Results Lots of people helped: MB-NG team at UCL MB-NG.

EDG WP7 Networking Demonstration uDemonstration sending HEP data CERN to SARA Multiple streams of TCP packets Tuned TCP parameters: ifconfig eth0 txqueuelen.

PFLDNet Argonne Feb 2004 R. Hughes-Jones Manchester 1 UDP Performance and PCI-X Activity of the Intel 10 Gigabit Ethernet Adapter on: HP rx2600 Dual Itanium.

© 2006 Open Grid Forum Interactions Between Networks, Protocols & Applications HPCN-RG Richard Hughes-Jones OGF20, Manchester, May 2007,

Slide: 1 Richard Hughes-Jones CHEP2004 Interlaken Sep 04 R. Hughes-Jones Manchester 1 Bringing High-Performance Networking to HEP users Richard Hughes-Jones.

ESLEA Bedfont Lakes Dec 04 Richard Hughes-Jones Network Measurement & Characterisation and the Challenge of SuperComputing SC200x.

CdL was here DataTAG CERN Sep 2002 R. Hughes-Jones Manchester 1 European Topology: NRNs & Geant SuperJANET4 CERN UvA Manc SURFnet RAL.

GridPP Collaboration Meeting May 2002 R. Hughes-Jones Manchester 1 Networking in Under 30 Minutes ! Richard Hughes-Jones, University of Manchester.

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 1 Discussion of Methodology for MPLS QoS & High Performance High throughput Investigations.

GGF4 Toronto Feb 2002 R. Hughes-Jones Manchester Initial Performance Measurements Gigabit Ethernet NICs 64 bit PCI Motherboards (Work in progress Mar 02)

13th-14th July 2004 University College London End-user systems: NICs, MotherBoards, TCP Stacks & Applications Richard Hughes-Jones.

KEK Network Qi Fazhi KEK SW L2/L3 Switch for outside connections Central L2/L3 Switch A Netscreen Firewall Super Sinet Router 10GbE 2 x GbE IDS.

R. Hughes-Jones ESnet Meeting Kyoto July 2000 Networking for HEP in the UK uIntroduction to the PPNCG uNetwork Topologies uMonitoring Activities uPeering.

Slide: 1 Richard Hughes-Jones e-VLBI Network Meeting 28 Jan 2005 R. Hughes-Jones Manchester 1 TCP/IP Overview & Performance Richard Hughes-Jones The University.

ESLEA PMB, Manchester, 23 Nov 2006, R. Hughes-Jones Manchester 1 Protocols Progress with Current Work. Richard Hughes-Jones The University of Manchester.

High Rate Internet Data Transfer for eVLBI Ralph Spencer, Richard Hughes-Jones and Simon Casey The University of Manchester All Hands Sept 2005.

High TCP performance over wide area networks Arlington, VA May 8, 2002 Sylvain Ravot CalTech HENP Working Group.

1 Network Performance Optimisation and Load Balancing Wulf Thannhaeuser.

ESLEA VLBI Bits&Bytes Workshop, 4-5 May 2006, R. Hughes-Jones Manchester 1 VLBI Data Transfer Tests Recent and Current Work. Richard Hughes-Jones The University.

Connect. Communicate. Collaborate 4 Gigabit Onsala - Jodrell Lightpath for e-VLBI The iNetTest Unit Development of Real Time eVLBI at Jodrell Bank Observatory.

Internet data transfer record between CERN and California Sylvain Ravot (Caltech) Paolo Moroni (CERN)

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester MB – NG SuperJANET4 Development Network SuperJANET4 Production Network Leeds RAL / UKERNA RAL.

ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester 1 Protocols DCCP and dccpmon. Richard Hughes-Jones The University of Manchester.

CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester UDPmon and TCPstream Tools to understand Network Performance Richard.

PFLDNet Workshop February 2003 R. Hughes-Jones Manchester Some Performance Measurements Gigabit Ethernet NICs & Server Quality Motherboards Richard Hughes-Jones.

Collaboration Meeting, 4 Jul 2006, R. Hughes-Jones Manchester 1 Collaborations in Networking and Protocols HEP and Radio Astronomy Richard Hughes-Jones.

High bit rate tests between Manchester and JIVE Looking at data rates attainable with UDP along with packet loss and reordering statistics Simon Casey,

DataGrid WP7 Meeting Amsterdam Nov 01 R. Hughes-Jones Manchester 1 UDPmon Measuring Throughput with UDP  Send a burst of UDP frames spaced at regular.

GNEW2004 CERN March 2004 R. Hughes-Jones Manchester 1 Lessons Learned in Grid Networking or How do we get end-2-end performance to Real Users ? Richard.

Networks ∙ Services ∙ People Richard-Hughes Jones eduPERT Training Session, Porto A Hands-On Session udpmon for Network Troubleshooting 18/06/2015.

Final EU Review - 24/03/2004 DataTAG is a project funded by the European Commission under contract IST Richard Hughes-Jones The University of.

1 eVLBI Developments at Jodrell Bank Observatory Ralph Spencer, Richard Hughes- Jones, Simon Casey, Paul Burgess, The University of Manchester.

ESLEA VLBI Bits&Bytes Workshop, 31 Aug 2006, R. Hughes-Jones Manchester 1 vlbi_udp Throughput Performance and Stability. Richard Hughes-Jones The University.

The EU DataTAG Project Richard Hughes-Jones Based on Olivier H. Martin GGF3 Frascati, Italy Oct 2001.

L1/HLT trigger farm Bologna setup 0 By Gianluca Peco INFN Bologna Genève,

Recent experience with PCI-X 2.0 and PCI-E network interfaces and emerging server systems Yang Xia Caltech US LHC Network Working Group October 23, 2006.

Connect. Communicate. Collaborate 4 Gigabit Onsala - Jodrell Lightpath for e-VLBI Richard Hughes-Jones.

DataGrid WP7 Meeting Jan 2002 R. Hughes-Jones Manchester Initial Performance Measurements Gigabit Ethernet NICs 64 bit PCI Motherboards (Work in progress)

MB MPLS MPLS Technical Meeting Sep 2001 R. Hughes-Jones Manchester SuperJANET Development Network Testbed – Cisco GSR SuperJANET4 C-PoP – Cisco GSR.

iperf a gnu tool for IP networks

CALICE TDAQ Application Network Protocols 10 Gigabit Lab

R. Hughes-Jones Manchester

Networking between China and Europe

Networking for grid Network capacity Network throughput

Data Transfer Node Performance GÉANT & AENEAS

Mar 2001 ATLAS T2UK Meeting R. Hughes-Jones

MB-NG Review High Performance Network Demonstration 21 April 2004

MB – NG SuperJANET4 Development Network

Presentation transcript:

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 1 EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester Thanks to a wireless connected Titanium in the

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 2 European Topology: NRNs & Geant SuperJANET4 CERN UvA Manc SURFnet

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 3 Gigabit Throughput on the Production WAN Manc - RAL 570 Mbit/s 91% of the 622 Mbit access link between SuperJANET4 and RAL 1472 bytes propagation ~21  s Manc-UvA (SARA) 750 Mbit/s SJANET4 + Geant + SURFnet Manc – CERN 460 Mbit/s CERN PC had a 32 bit PCI bus

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 4 Gigabit TCP on the Production WAN (01) Throughput vs TCP buffer size TCP window sizes in Mbytes calculated from RTT*bandwidth Window for BW 1 Gbit/s Window Mbytes for UDP BW 750 Mbit/s 460 Mbit/s

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 5 Gigabit TCP on the Production WAN Man-CERN Throughput vs n-streams Default buffer size slope = ~25 Mbit/s/stream up to 9 streams then 15 Mbit/s/stream With larger buffers rate of increase per stream is larger Plateaus at about 7 streams giving a total throughput of ~400 Mbit/s

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 6 UDP Throughput: SLAC - Man SLAC – Manc 470 Mbit/s 75% of the 622 Mbit access link SuperJANET4 peers with ESnet at 622Mbit in NY

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 7 Gigabit TCP on throughput Man-SLAC Throughput vs n-streams Much less than for European links Buffer required: rtt*BW (622Mbit) = ~14 Mbytes With larger buffers > default, rate of increase per stream is ~ 5.4 Mbit/s/stream No Plateau

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 8 RiPE 1 way UvA  RAL ms RiPE 1 way RAL  UvA ms UDPmon Loss Throughput Mbit/s MAN – UvA ~750Mbit/s UDPmon & Ripe MAN-SARA from 25 May 02

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 9 UDPmon Loss Throughput ~140 Mbit/s Man  Lyon Ping Man  Lyon 40 ms rtt Iperf TCP Throughput ~ 47 Mbit/s Man  Lyon byte buffer iperf Pinger & UDPmon UK-Lyon From 25 May 02 – 04 Jun 02

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 10 Work on End Systems: PCI: SysKonnect SK-9843  Motherboard: SuperMicro 370DLE Chipset: ServerWorks III LE Chipset  CPU: PIII 800 MHz PCI:64 bit 66 MHz  RedHat 7.1 Kernel SK bytes sent Wait 20 us Sk bytes sent Wait 10 us Frames are back-to-back Can drive at line speed Cannot go any faster ! Gig Eth frames back to back

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 11 PCI: Intel Pro/1000  Motherboard: SuperMicro 370DLE Chipset:: ServerWorks III LE Chipset  CPU: PIII 800 MHz PCI:64 bit 66 MHz  RedHat 7.1 Kernel IT66M bytes sent Wait 11 us ~4.7us on send PCI bus PCI bus ~45% occupancy ~ 3.25 us on PCI for data recv IT66M bytes sent Wait 11 us Packets lost Action of pause packet?

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 12 UDP Throughput: Intel Pro/1000 on B2B P4DP6 Max throughput 950Mbit/s Some throughput drop for packets >1000 bytes Loss NIC dependent Loss not due to user  Kernel moves Traced to discards in the receiving IP layer ??? Motherboard: SuperMicro P4DP6 Chipset: Intel E7500 (Plumas) CPU: Dual Xeon Prestonia (2cpu/die) 2.2 GHz Slot 4: PCI, 64 bit, 66 MHz RedHat 7.2 Kernel

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 13 Packet Loss: Where? Intel Pro 1000 on 370DLE 1472 byte packets Expected loss in transmitter ! UDPmon UDP IP Eth drv UDPmon UDP IP Eth drv HW N Gen N Transmit N Lost IpDiscards N Received Gig Switch No loss at switch But Pause packet seen to sender

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 14 Interrupt Coalescence: Latency Intel Pro 1000 on 370DLE 800 MHz CPU

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 15 Interrupt Coalescence: Latency Intel Pro 1000 on 370DLE 800 MHz CPU

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 16 Interrupt Coalescence: Throughput Intel Pro 1000 on 370DLE