1. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Network Connectivity and Projects – a Perspective from the UK Richard Hughes-Jones PPNCG SuperJANET4 Status Connectivity to Europe Connectivity to the US Grid Networking Projects

2. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester SuperJANET4: Backbone and Access links Worldcom supplied the transmission UKERNA layer the IP service Core PoP IP router at Worldcom Backbone Access Router at MANs Access Links: Large MAN 2.5 Gbit -> 10-20 Gbit Medium MAN 622 Mbit -> 2.5 Gbit 4 node DWDM development net Deployment Status: Backbone Oct 00 Routers Nov 00 All sites Mar 01 Proved to be Stable Constant growth of traffic

3. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester SuperJANET4: ping rtt Core routers Jun 01

4. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester SuperJANET4: ping rtt Core routers Lancaster Glasgow MAN / LAN Issues Bristol Cambridge

5. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester London MAN Upgrade UDP Throughput Mbit/s UDP Packet loss % UDPmon Tests Manchester – London MAN was 155 Mbit ATM 1 st Oct Time interval in Weeks Richard HJ

6. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester ICFAMON Plot from RAL to CERN for 19 th Oct to 1 st Nov 2001 Connectivity to Europe – Access links (1) UK Access link 155 Mbit ATM Sustained rate 130 Mbit Contract to end of Nov 01 Bad news for users !

7. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Traceping Oxford to CERN for 31 st October 2001 Connectivity to Europe – Access links (2) loss around ten155-gw.ja.net router J. Macallister

8. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Connectivity to Europe : Geant Start mid November 2001 UKERNA switched off TEN-155 3 Dec 2001 Monitoring – see later

9. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Connectivity to US 6 * 155 Mbit links US link about to go to 2.5 G bit IP commodity attached in London Research traffic over 2.5G bit Peer in Hudson St. 622 Mbit to Esnet 622 Mbit to Abilene.

10. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Monitoring: US Traffic UKERNA Traffic data Kbit/s. Blue Traffic from US; Maroon Traffic to US 17 Jul 2000 (300kbit/s)7 Dec 2001 (900kbit/s) peak is 88% of total BW 930 Mbit 10 minute averages Last 7 days 1 hour averages

11. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Monitoring: US Traffic - Trends Jan 99 – Nov 02 UKERNA – US Traffic Daily totals Mbytes Weekend Xmas 99 Weekday Xmas 01 Peak load ~88 % J. Macallister

12. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Grid Network Projects DataGrid GridPP Extending DataGrid to BaBar, D0, CDF,… in UK DataTAG Foci: Network resarch, Grid inter-operability MB-NG UK funded 2.5Gbit Development Network Optical Switching

13. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Network Monitoring Several tools in test – plugged into a coherent structure: PingER, RIPE one way times, iperf, UDPmon, rTPL, GridFTP, and NWS prediction engine continuous tests for last few months to selected sites: DL Man RL UCL CERN Lyon Bologna SARA NBI SLAC … The aims of monitoring for the Grid: to inform Grid applications, via the middleware, of the current status of the network – input for resource broker and scheduling to identify fault conditions in the operation of the Grid to understand the instantaneous, day-to-day, and month-by-month behaviour of the network – provide advice on configuration etc. Report written on LDAP scheme for publishing the network information Will be used by UK e-science sites and non HEP WPs

14. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester UDPmon Measuring Throughput with UDP Send a burst of UDP frames spaced at regular intervals Vary the frame size and the frame transmit spacing Record The time to send and the time to receive the frames The number received, the number lost, number out of order The received inter-packet spacing Zero stats OK done Get remote statistics Send statistics Send data frames at regular intervals Time to send Time to receive Inter-packet time

15. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Local Network Monitoring Store & Analysis of Data (Access) Access to current and historic data and metrics via the Web, i.e. WP7 NM Pages, access to metric forecasts Backend LDAP script to fetch metrics Monitor process to push metrics local LDAP Server Grid Application access via LDAP Schema to - monitoring metrics; - location of monitoring data. PingER (RIPE TTB) IperfER UDPmon rTPL NWS etc LDAP Schema Grid Apps GridFTP Network Monitoring Architecture Robin Tasker

16. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Network Monitoring Components PingNetmonUDPmoniPerfRipe Cron script plot Table LDAP raw control Cron script control Cron script plot Table LDAP rawplot Table LDAP raw WEB DisplayAnalysisGrid BrokerPredictions Web I/f Scheduler Tool Clients LDAP raw LDAP raw

17. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Ping & UDP throughput MAN-RAL From 20 Oct 01 PingER rtt (ms) dl – RAL 1000 byte packet Forecast UDPmon Zero packet loss! UDPmon throughput Mbit/s man – RAL 300 * 1400 byte frames

18. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Ping & UDP throughput MAN-CERN From 20 Oct 01 PingER rtt (ms) dl – cern 1000 byte packet Forecast UDPmon throughput Mbit/s man – cern 300 * 1400 byte frames

19. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester iperf & UDP throughput MAN-CERN From 20 Oct 01 Iperf throughput ucl – cern 262144 byte buffer Forecast UDPmon throughput Mbit/s man – cern 300 * 1400 byte frames

20. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester iperf & UDP throughput MAN-SARA From 20 Oct 01 Iperf throughput ucl – sara 262144 byte buffer Forecast UDPmon throughput Mbit/s man – sara 300 * 1400 byte frames

21. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester iperf & Pinger UK-Bologna From 20 Oct 01 Iperf throughput ucl – Bologna 262144 byte buffer Forecast in green PingER rtt (ms) dl – Bologna 1000 byte packet Forecast

22. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester iperf throughput UCL-SARA From 1 Nov 01 – Geant Operational Iperf throughput ucl – sara 262144 byte buffer Geant Enabled Routing Stable

23. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Brief outline of work plan High Performance High Throughput: Understanding the end system HW, best way to monitor traffic and protocol packets, Type and effect of background traffic, Throughput vs rtt, throughput vs window-size, Throughput using multiple TCP streams, and effect on the Network GridFTP and BBftp at Gigabit, Effect of different TCP stacks, Use of non-TCP protocols, and effect on the Network Show and tell demos Bandwidth Reservation and QoS Identification of traffic classes and middleware components requiring QoS Study of traffic differentiation and traffic engineering techniques demonstration of QoS & Bandwith Reservation

24. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester The EU DataTAG project EU Transatlantic Gigabit project. Status: approved for 4 M ECU start on 1 Dec 2001 Partners: CERN/PPARC/INFN/UvA. IN2P3 sub- contractor The main foci are: Grid Network Research including: Provisioning (CERN) Investigations of high performance data transport (PPARC) End-to-end inter-domain QoS + BW / network resource reservation Bulk data transfer and monitoring (UvA) Interoperability between Grids in Europe and the US PPDG, GriPhyN, DTF, iVDGL (USA) PPDG, GriPhyN, DTF, iVDGL (USA)

25. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester DataTAG project Abilene ESNET MREN STAR-TAP UK SuperJANET4 IT GARR- B IT GARR- B GEANT NewYork Olivier Martin NL SURFne t NL SURFne t 2.5 Gbit lambda between CERN and Starlight POS 2 nd half 2002 – WDM later STAR-LIGHT CERN

26. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester 2.5 Gbps SDH Lambda 100/1000 Mbps Ethernet Amsterdam GSR λ Broker Application CERN StarLight Italy UK Optical Switching - DataTAG 7600 Iwire Abilene ESnet

27. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester MB – NG SuperJANET Testbed MB - NG SuperJANET 4 Production Network Leeds RAL / UKERN A RAL / UKERNA SJ4 Dev C-PoP Reading SJ4 Dev C-PoP Warringto n Manc MCC ULCC UCL SJ4 Dev C-PoP London Gigabit Ethernet 2.5 Gbit POS 10 Gbit POS

28. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester E-science core project MB - NG u Project to investigate and pilot: end-to-end traffic engineering and management over multiple administrative domains – MPLS in core diffserv at the edges. Managed bandwidth and Quality-of-Service provision. (Robin Tasker) High performance high bandwidth data transfers. (Richard HJ) Demonstrate end-to-end network services to CERN using Dante EU- DataGrid and the US DataTAG. u Status: approved with requested funds start on 1 Dec 2001 u Partners:CISCO, CLRC, Manchester, UCL, UKERNA plus Lancaster and Southampton (IPv6) u A technical meeting held – draft spec. of the HW required u Would like to use real Grid traffic – as demonstrator: CDF UCL-RAL BaBar Man-RAL

29. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester And Finally in case it helps: GridPP and DataGrid have been working on packaging Globus v2 as a.rpm kit. Makes installation simpler. Available from DataGrid WP6 web: marianne.in2p3.fr/datagrid/testbed1/globus

30. ICFA SCIC Meeting CERN Dec 01 R. Hughes-Jones Manchester Conclusion & Status SuperJANET4 stable working well Access links to Campus becoming very inportant Network Links and peering to US good overall US-UK bandwidth upgrade overdue Geant access to Europe OK (at the moment !) Considerable UK interest in the Internet2 tests Grid: Lots of interesting work to do …