Presentation is loading. Please wait.

Presentation is loading. Please wait.

MB-NG Review – 24 April 2004 Richard Hughes-Jones The University of Manchester, UK MB-NG Review High Performance Network Demonstration 21 April 2004.

Published bySkylar Viel Modified over 9 years ago

Similar presentations

Presentation on theme: "MB-NG Review – 24 April 2004 Richard Hughes-Jones The University of Manchester, UK MB-NG Review High Performance Network Demonstration 21 April 2004."— Presentation transcript:

1 MB-NG Review – 24 April 2004 Richard Hughes-Jones The University of Manchester, UK MB-NG Review High Performance Network Demonstration 21 April 2004

2 MB-NG Review, April 20042 It works ? So what’s the Problem with TCP TCP has 2 phases: Slowstart & Congestion Avoidance AIMD and High Bandwidth – Long Distance networks Poor performance of TCP in high bandwidth wide area networks is due in part to the TCP congestion control algorithm - cwnd congestion window For each ack in a RTT without loss: cwnd -> cwnd + a / cwnd - Additive Increase, a=1 For each window experiencing loss: cwnd -> cwnd – b (cwnd) - Multiplicative Decrease, b= ½ Time to recover from 1 packet loss ~100 ms rtt:

3 MB-NG Review, April 20043 Investigation of new TCP Stacks High Speed TCP a and b vary depending on current cwnd using a table a increases more rapidly with larger cwnd – returns to the ‘optimal’ cwnd size sooner for the network path b decreases less aggressively and, as a consequence, so does the cwnd. The effect is that there is not such a decrease in throughput. Scalable TCP a and b are fixed adjustments for the increase and decrease of cwnd a = 1/100 – the increase is greater than TCP Reno b = 1/8 – the decrease on loss is less than TCP Reno Scalable over any link speed. Fast TCP Uses round trip time as well as packet loss to indicate congestion with rapid convergence to fair equilibrium for throughput. HSTCP-LP High Speed (Low Priority) – backs off if rtt increases BiC-TCP – Additive increase large cwnd; binary search small cwnd H-TCP – after congestion standard then switch to high performance ●●●

4 MB-NG Review, April 20044 Comparison of TCP Stacks TCP Response Function Throughput vs Loss Rate – steeper: faster recovery Drop packets in kernel MB-NG rtt 6ms DataTAG rtt 120 ms

5 MB-NG Review, April 20045 Multi-Gigabit flows at SC2003 BW Challenge Three Server systems with 10 GigEthernet NICs Used the DataTAG altAIMD stack 9000 byte MTU Send mem-mem iperf TCP streams From SLAC/FNAL booth in Phoenix to: Chicago Starlight rtt 65 ms window 60 MB Phoenix CPU 2.2 GHz 3.1 Gbit hstcp I=1.6% Amsterdam SARA rtt 175 ms window 200 MB Phoenix CPU 2.2 GHz 4.35 Gbit hstcp I=6.9% New TCP stacks are very Stable Both used Abilene to Chicago

6 MB-NG Review, April 20046 Transfer Applications – Throughput [1] 2Gbyte file transferred RAID0 disks Manc – UCL GridFTP See alternate 600/800 Mbit and zero Apache web server + curl-based client See steady 720 Mbit

7 MB-NG Review, April 20047 Transfer Applications – Throughput [2] 2Gbyte file transferred RAID5 - 4disks Manc – RAL bbcp Mean 710 Mbit/s GridFTP See many zeros Mean ~710 Mean ~620

8 MB-NG Review, April 20048 Topology of the MB – NG Network Key Gigabit Ethernet 2.5 Gbit POS Access MPLS Admin. Domains UCL Domain Edge Router Cisco 7609 man01 man03 Boundary Router Cisco 7609 RAL Domain Manchester Domain lon02 man02 ral01 UKERNA Development Network Boundary Router Cisco 7609 ral02 lon03 lon01

9 MB-NG Review, April 20049 High Throughput Demo Manchester man03lon01 2.5 Gbit SDH MB-NG Core 1 GEth Cisco GSR Cisco 7609 Cisco 7609 London Dual Zeon 2.2 GHz Send data with TCP Drop Packets Monitor TCP with Web100

10 MB-NG Review, April 200410 Standard to HS-TCP No loss, but output queue filled by sender

11 MB-NG Review, April 200411 HS-TCP to Scalable No loss, but output queue filled by sender

12 MB-NG Review, April 200412 Standard, HS-TCP, Scalable Drop 1 in 25,000

13 MB-NG Review, April 200413 Standard Reno TCP Drop 1 in 10 6

14 MB-NG Review, April 200414 Focus on Helping Real Users: Throughput CERN -SARA Using the GÉANT Backup Link 1 GByte disk-disk transfers Blue is the Data Red is the TCP ACKs Standard TCP Average Throughput 167 Mbit/s Users see 5 - 50 Mbit/s! High-Speed TCP Average Throughput 345 Mbit/s Scalable TCP Average Throughput 340 Mbit/s Technology link to EU Projects: DataGrid DataTAG & GÉANT

15 MB-NG Review, April 200415 BaBar Case Study: Host, PCI & RAID Controller Performance RAID0 (striped) & RAID5 (stripped with redundancy) 3Ware 7506 Parallel 66 MHz 3Ware 7505 Parallel 33 MHz 3Ware 8506 Serial ATA 66 MHz ICP Serial ATA 33/66 MHz Tested on Dual 2.2 GHz Xeon Supermicro P4DP8-G2 motherboard Disk: Maxtor 160GB 7200rpm 8MB Cache Read ahead kernel tuning: /proc/sys/vm/max-readahead Disk – Memory Read Speeds Memory - Disk Write Speeds

16 MB-NG Review, April 200416 Topology of the MB – NG Network Key Gigabit Ethernet 2.5 Gbit POS Access MPLS Admin. Domains UCL Domain Edge Router Cisco 7609 man01 man03 Boundary Router Cisco 7609 RAL Domain Manchester Domain lon02 man02 ral01 UKERNA Development Network Boundary Router Cisco 7609 ral02 lon03 lon01 HW RAID

17 MB-NG Review, April 200417 BaBar Data: Throughput on MB–NG kit RAID5 - 4disks RAL - Manc Includes small files ~Kbytes bbftp 1 stream with compression bbftp 6 streams bbftp 1 stream no compression 10 * 2 G byte files – each peak is a 20 G byte transfer bbftp 1 stream Files ≥ 1 Mbyte With bb diag

18 MB-NG Review, April 200418 Helping Real Users Radio Astronomy VLBI PoC with NRNs & GEANT 1024 Mbit/s 24 on 7 NOW

19 MB-NG Review, April 200419 1472 byte Packets man -> JIVE FWHM 22 µs (B2B 3 µs ) VLBI Project: Throughput Jitter & 1-way Delay 1-way Delay – note the packet loss (points with zero 1 –way delay) 1472 byte Packets Manchester -> Dwingeloo JIVE

20 MB-NG Review, April 200420 Case Study: ATLAS LHC Tests streaming built Events from Level3 Trigger to remote compute farm in real time 500 Mbit to 1 Gbit CERN – Man Investigation of use of new high performance TCPs Testing concepts in the ATLAS Offline Computing model More Mesh than Star: CERN Tier0 to Tier 1s Tier 2s to all Tier 1s Tests planned over production networks: Lancaster-Manchester NNW SuperJANET4 Lancaster-Manchester to CERN

21 MB-NG Review, April 200421

22 MB-NG Review, April 200422 Scalable TCP DataTAG Drop 1 in 10 6

23 MB-NG Review, April 200423 HS-TCP DataTAG Drop 1 in 10 6

24 MB-NG Review, April 200424 Standard Reno TCP DataTAG Drop 1 in 10 6 Transition highspeed to Standard TCP @ 520s

25 MB-NG Review, April 200425 Summary Multi-Gigabit transfers are possible and stable Demonstrated that new TCP stacks help performance DataTAG has made major contributions to understanding of high-speed networking There has been significant technology transfer between DataTAG and other projects Now reaching out to real users. But still much research to do: Achieve performance – Protocol vs implementation issues Stability / Sharing issues Optical transports & hybrid networks

26 MB-NG Review, April 200426 10 Gigabit: Tuning PCI-X mmrbc 1024 bytes mmrbc 2048 bytes mmrbc 4096 bytes 5.7Gbit/s mmrbc 512 bytes CSR Access PCI-X Sequence Data Transfer Interrupt & CSR Update 16080 byte packets every 200 µs Intel PRO/10GbE LR Adapter PCI-X bus occupancy vs mmrbc Measured times Times based on PCI-X times from the logic analyser Expected throughput ~7 Gbit/s

27 MB-NG Review, April 200427 DataTAG Testbed

28 MB-NG Review, April 200428 BaBar Case Study: Disk Performance BaBar Disk Server Tyan Tiger S2466N motherboard 1 64bit 66 MHz PCI bus Athlon MP2000+ CPU AMD-760 MPX chipset 3Ware 7500-8 RAID5 8 * 200Gb Maxtor IDE 7200rpm disks Note the VM parameter readahead max Disk to memory (read) Max throughput 1.2 Gbit/s 150 MBytes/s) Memory to disk (write) Max throughput 400 Mbit/s 50 MBytes/s) [not as fast as Raid0]

29 MB-NG Review, April 200429 RAID Controller Performance RAID 0 RAID 5 Read Speed Write Speed

30 MB-NG Review, April 200430 BaBar: Serial ATA Raid Controllers RAID5 3Ware 66 MHz PCI ICP 66 MHz PCI

31 MB-NG Review, April 200431 Measure the time between lost packets in the time series of packets sent. Lost 1410 in 0.6s Is it a Poisson process? Assume Poisson is stationary λ(t) = λ Use Prob. Density Function: P(t) = λ e -λt Mean λ = 2360 / s [426 µs] Plot log: slope -0.0028 expect -0.0024 Could be additional process involved VLBI Project: Packet Loss Distribution

32 MB-NG Review, April 200432 The performance of the end host / disks BaBar Case Study: RAID BW & PCI Activity 3Ware 7500-8 RAID5 parallel EIDE 3Ware forces PCI bus to 33 MHz BaBar Tyan to MB-NG SuperMicro Network mem-mem 619 Mbit/s Disk – disk throughput bbcp 40-45 Mbytes/s (320 – 360 Mbit/s) PCI bus effectively full! User throughput ~ 250 Mbit/s Read from RAID5 Disks Write to RAID5 Disks

Download ppt "MB-NG Review – 24 April 2004 Richard Hughes-Jones The University of Manchester, UK MB-NG Review High Performance Network Demonstration 21 April 2004."

Similar presentations

Helping TCP Work at Gbps Cheng Jin the FAST project at Caltech

Helping TCP Work at Gbps Cheng Jin the FAST project at Caltech
2.008 -Spring 20041 2.008 Process Control. 2.008 -Spring 20042 Outline 1.Optimization 2.Statistical Process Control 3.In-Process Control.

Spring Process Control Spring Outline 1.Optimization 2.Statistical Process Control 3.In-Process Control.
ESLEA and HEPs Work on UKLight Network. ESLEA Exploitation of Switched Lightpaths in E- sciences Applications Exploitation of Switched Lightpaths in E-

ESLEA and HEPs Work on UKLight Network. ESLEA Exploitation of Switched Lightpaths in E- sciences Applications Exploitation of Switched Lightpaths in E-
Tony Doyle - University of Glasgow GridPP EDG - UK Contributions Architecture Testbed-1 Network Monitoring Certificates & Security Storage Element R-GMA.

Tony Doyle - University of Glasgow GridPP EDG - UK Contributions Architecture Testbed-1 Network Monitoring Certificates & Security Storage Element R-GMA.
GridPP Meeting Edinburgh 4-5 Feb 04 R. Hughes-Jones Manchester 1 High Performance Networking for ALL Members of GridPP are in many Network collaborations.

GridPP Meeting Edinburgh 4-5 Feb 04 R. Hughes-Jones Manchester 1 High Performance Networking for ALL Members of GridPP are in many Network collaborations.
E-Science All Hands Meeting 1-4 Sep 03 R. Hughes-Jones Manchester 1 High Bandwidth High Throughput in the MB-NG & DataTAG Projects Richard Hughes-Jones,

E-Science All Hands Meeting 1-4 Sep 03 R. Hughes-Jones Manchester 1 High Bandwidth High Throughput in the MB-NG & DataTAG Projects Richard Hughes-Jones,
Storage System Integration with High Performance Networks Jon Bakken and Don Petravick FNAL.

Storage System Integration with High Performance Networks Jon Bakken and Don Petravick FNAL.
D. Elia, R. SantoroITS week / SPD meeting - May 12, 20041 Test beam data analysis D. Elia, R. Santoro – Bari SPD Group Alignments, plane rotation for setup.

D. Elia, R. SantoroITS week / SPD meeting - May 12, Test beam data analysis D. Elia, R. Santoro – Bari SPD Group Alignments, plane rotation for setup.
17 May 20041 Multiple Sites. 17 May 20042 Multiple Sites This presentation assumes you are already familiar with Doors and all its standard commands It.

17 May Multiple Sites. 17 May Multiple Sites This presentation assumes you are already familiar with Doors and all its standard commands It.
TCP transfers over high latency/bandwidth network & Grid TCP Sylvain Ravot

TCP transfers over high latency/bandwidth network & Grid TCP Sylvain Ravot
IGRID2002 Radio Astronomy VLBI Demo. uWeb based demonstration sending VLBI data A controlled stream of UDP packets 256-500 Mbit/s on the production network.

IGRID2002 Radio Astronomy VLBI Demo. uWeb based demonstration sending VLBI data A controlled stream of UDP packets Mbit/s on the production network.
MB - NG MB-NG Technical Meeting 03 May 02 R. Hughes-Jones Manchester 1 Task2 Traffic Generation and Measurement Definitions Pass-1.

MB - NG MB-NG Technical Meeting 03 May 02 R. Hughes-Jones Manchester 1 Task2 Traffic Generation and Measurement Definitions Pass-1.
DataTAG CERN Oct 2002 R. Hughes-Jones Manchester Initial Performance Measurements With DataTAG PCs Gigabit Ethernet NICs (Work in progress Oct 02)

DataTAG CERN Oct 2002 R. Hughes-Jones Manchester Initial Performance Measurements With DataTAG PCs Gigabit Ethernet NICs (Work in progress Oct 02)
JIVE VLBI Network Meeting 15 Jan 2003 R. Hughes-Jones Manchester The EVN-NREN Project Richard Hughes-Jones The University of Manchester.

JIVE VLBI Network Meeting 15 Jan 2003 R. Hughes-Jones Manchester The EVN-NREN Project Richard Hughes-Jones The University of Manchester.
GNEW2004 CERN March 2004 R. Hughes-Jones Manchester 1 End-2-End Network Monitoring What do we do ? What do we use it for? Richard Hughes-Jones Many people.

GNEW2004 CERN March 2004 R. Hughes-Jones Manchester 1 End-2-End Network Monitoring What do we do ? What do we use it for? Richard Hughes-Jones Many people.
Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Networking for ATLAS Remote Farms Richard Hughes-Jones The University.

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Networking for ATLAS Remote Farms Richard Hughes-Jones The University.
GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.

GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.
CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 1 EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester.

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 1 EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester.
Slide: 1 Richard Hughes-Jones PFLDnet2005 Lyon Feb 05 R. Hughes-Jones Manchester 1 Investigating the interaction between high-performance network and disk.

Slide: 1 Richard Hughes-Jones PFLDnet2005 Lyon Feb 05 R. Hughes-Jones Manchester 1 Investigating the interaction between high-performance network and disk.
Optical Networking Status of Discussion in the UK Richard Hughes-Jones The University of Manchester Particle Physics Network Coordination Group TERENA.

Optical Networking Status of Discussion in the UK Richard Hughes-Jones The University of Manchester Particle Physics Network Coordination Group TERENA.

Similar presentations

Ads by Google