2. Measurement tools and the adoption of Abilene Observatory Jin Tanaka & Takatoshi Ikeda APAN-JP NOC APAN Cairns meeting, July 2004

3. Background Measurement Tools in APAN-JP Selection and Provision of Measurement data Deployment plan of Observatory Outline

4. Background Various measurement tools are installed independently by research projects in APAN-JP. Operators can t grasp the whole measurement tools. Some demonstrations are being done without any notifications. Multiple measurement tools sometimes deal with the same data simultaneously. We can see the cases that the same data are measured with multiple machines. Many researchers have paid attention to throughput data and routing information as the research material. Operators are investigating the network condition in spots. Operators are required to provide the researchers not only with connectivity but with more advanced services as needed. Operators are required to collaborate with the researchers, just before the events or the demonstrations.

5. Members of APAN-JP NOC Network Researchers Network Engineers 24 x 7 Operators Experiments & Advanced Services Basic Services including Performance Trouble Tickets

6. Measurement Tools in APAN-JP Throughput -1- Machine nameInstalled tools FunctionProjectAdministra tor Note Perf3Iperf Bandwidth measurement Internet End-to- end Performance Monitoring - Bandwidth to the World (IEPM- BW) project KDDI Labs http://www.slac.stanfo rd.edu/comp/net/band width- tests/hercules/html/sc atplots/node1.jp.apan. net.html Perf4&7Iperf Bandwidth measurement Operation tool of NOC KDDI Labs - perf5& perf6& genkai-perf1& genkai-perf2 Web100 *¹ FastTCP Iperf BWCTL Original tools Bandwidth measurement High- performance network for eVLBI M.Hirabaru (NICT) http://mrtg.koganei.itr c.net/ * ¹ http://www.web100.org/

7. Measurement Tools in APAN-JP Throughput -2- Machine nameInstalled tools FunctionProjectAdministra tor Note fs01&fs02FastTCP Bandwidth measurement FastTCP test across TransPAC R.Jayaram S.Low (caltech) - ?GNET-1 *² Bandwidth measurement Bandwidth challenge O.Tatebe (AIST) https://datafarm.apgri d.org/software/ bwctlIperf BWCTL Scheduling and resource allocation wrapper for Iperf piPEsY.Kitamura (NICT) - * ² http://www.gtrc.aist.go.jp/gnet/

8. Machine nameInstalled tools FunctionProjectAdministra tor Note rvmrtd Routing information (BGP) APII BGP collector for APAN Measurement WG M.Hirabaru (NICT) http://mrtg.koganei.itrc.net/ maru1 & maru2 Zebra * Tcpdump Original tools Routing information (BGP) Analysis of the relation between traffic and routing M.Maruyama (U-Tokyo) http://maru2.jp.apa n.net ope1net-snmp Tcpdump Original tools Routing information (OSPF,IS-IS) Research for IGP stability Z. Sheu (NICT) http://ope1.jp.apan.net/cgi-bin/lsa- stat?net=apan&area =backbone ns2Original tools Routing information(BGP) Operation tool of NOC KDDI Labs http://ns2.jp.apan. net/nr/ http://ns2.jp.apan. net/bgproute/ Measurement Tools in APAN-JP Routing * http://www.zebra.org/

9. Machine name Installed tools FunctionProjectAdministra tor Note ?GNET-1 One-way Delay measurement Bandwidth challenge O.Tatebe (AIST) https://datafarm.a pgrid.org/software/ pe2(koganei. wide.ad.jp) qpe OWAMP One-way Delay measurement Research for One- Way Delay Z. Sheu (NICT) http://pe2.koganei. wide.ad.jp/cgi- bin/owd-stat ns2Smoke ping * Round Trip Time measurement Operation tool of NOC KDDI Labs http://www.jp.apan.net/measurement/ rtt/ Measurement Tools in APAN-JP Latency * http://people.ee.ethz.ch/~oetiker/webtools/smokeping/

10. Machine name Installed tools FunctionProjectAdministra tor Note nsCVS Original tools Config backup and history Operation tool of NOC NOC Engineer- Measurement Tools in APAN-JP Router

11. Measurement Tools in APAN-JP Netflow Machine nameInstalled tools FunctionProjectAdministra tor Note nms3Flow tools NetflowObservatoryKDDI Labs & NOC Engineer -

12. Machine name Installed tools FunctionProjectAdministra tor Note nssyslogd Original tools Reporting router information Operation tool of NOC NOC Engineer - Measurement Tools in APAN-JP Syslog

13. Measurement Tools in APAN-JP Usage Machine nameInstalled tools FunctionProjectAdministra tor Note nsMRTG RRDTool Traffic measurement Operation tool of NOC NOC Engineerhttp://www.jp.apan. net/NOC/tm/ mrtgCricket RRDTool Original tools Traffic measurement Operation tool of NOC NOC Engineer http://mrtg.jp.apan.ne t/cricket/router- interfaces/ mrtg.koganei. itrc.net MRTG Original tools Traffic measurement M.Hirabaru (NICT) http://mrtg.koganei.itr c.net/

14. Selection and Provision of Measurement Data Sort out data required for researchers Throughput Iperf, BWCTL Latency OWAMP,RRD Netflow cflow,flowtools GNET-1 IGP Routing information Tcpdump … etc. to be provided by the Observatory system that NOC members will coordinate. to be adopted into the Observatory system as new common tools. Evaluation Selection Advanced backbone services Collocated systems assisted by NOC Engineers to be left as the private tools.

15. Selection and Provision of Measurement Data Potential New Common Tool ~Example~ GNET-1 Developed by AIST (The National Institute of Advanced Industrial Science and Technology, Japan) GNET-1 is a gigabit network testbed, which consists of four Gigabit Ethernet interfaces, four fast and large amount of SRAMs and a large-scale FPGA connected to the them. Function Wide area network emulation Network instrumentation Traffic shaping Traffic generation Powerful tool for developing network-aware grid software like GridMPI and Gfarm

16. Selection and Provision of Measurement Data TransPAC/APAN Observatory Systems We are planning to provide the services of two categories: Advanced backbone data Backbone data collected by NOC using equipment located in TransPAC/APAN nodes. Data item and data format will become the same as those of Abilene observatory. Collocation Systems assisted by NOC engineers Data collected by respective research projects using their own equipments collocated in XP. Accounts should be provided for NOC operators and engineers. Permission of exporting data should be authorized by the projects. Just started by the Joint TransPAC and APAN-JP NOC members.

17. Selection and Provision of Measurement Data Backbone Data Collection Necessity for backbone measurement Advanced Backbone measurement is required for improving NOC services. Advanced data enables to allocate the network resources effectively. It helps operators to find where is a bottle-neck? It activates network researches with advanced backbone data. E2E Measurement Backbone Measurement Observatory NOC operators cant coordinate the E2E measurement. Congestion of measuring activities will be relieved.

18. Selection and Provision of Measurement Data Proposal of TransPAC/APAN Observatory Measurement should be done in the best conditions Observatory systems should be installed in the best environments of TransPAC/APAN network. i.e. To be set as close to the router interfaces as possible, using the best interface cards of Jumbo Frame. NOC should maintain the systems and export data to authorized researchers. A good common platform for performance measurement should be installed at each XP. Common measurement method should be developed Common tools are installed, providing the common data. Data should be formatted in the global standards. i.e. GGF, piPEs, IPPM(IETF), IRPM … etc.

19. Selection and Provision of Measurement Data Formation made by Observatory Observatory system will grow up R&E networks! Observatory Developed by NOC researchers & maintained by NOC engineers/operators Common Tools with High Priority Output data in standard format Data with authentication Software, Papers, etc. Researchers can get measurement data of the global networks and work together with foreign researchers. NOC Advanced Service NOC Basic Service Helpful in operation

20. Deployment plan of Observatory Observatory system could be deployed across the TransPAC circuits. This deployment is to : get data over the long latency link cooperate with the Abilene Observatory

21. Deployment plan of Observatory Location Japan Tokyo XP U.S. To be decided by Indiana University folks Average RTT 190ms TokyoXP Chicago/Indianapolis Los Angeles

22. Deployment plan of Observatory Machines There are machines in each side for observatory. (NMS1-4) The configuration is the same as that of Abilene Observatory : optimized for observatory based on their experience made easy to cooperate with the Abilene Observatory managed by NOC Hardware Specification Chassis : SuperMicro SuperServer 6013P CPU : Dual 3.06GHz Intel Xeon Memory : 2 x 1GB DDR266 Disk : 2 x SCSI 73GB (RAID1)(NMS2~4 3 x SCSI 73GB (RAID5)(NMS1 NIC : Dual 10/100/1000 on-board Intel 1000base-SX NMS1)

23. Deployment plan of Observatory Roles of NMS hostOSMeasurementtoolNetwork NMS1Linux 2.4.26 web100 2.3.8 Throughput (regularly) iperfGbE to router & switch MTU 9000 NMS2Linux 2.4.26 web100 2.3.8 Throughput (on-demand) iperfGbE to switch MTU 9000 NMS3FreeBSD 5.2.1-REL- p8 Netflow Usage Flow-tools Net-snmp +RRD GbE to switch MTU 9000 NMS4FreeBSD 5.2.1-REL- p8 LatecyOWAMPGbE to switch MTU 9000 Routing, Router, Syslog and Usage data will be evaluated for the deployment. NICs on all servers are for Gigabit Ethernet.

24. Deployment plan of Observatory Network Layout (TokyoXP)

25. Deployment plan of Observatory Rack Front View

26. Deployment plan of Observatory Rack Rear View

27. Deployment plan of Observatory Network Layout (U.S.) Indiana University folks will talk about the network layout. (For reference) Current Situation of StarLight site No L3 device Create VLAN for the observatory over the TransPAC circuit. It is L2 connectivity between TokyoXP and StarLight site In the future, L3 device might be installed, and reconfiguration of network layout will be studied. Power supply AC power unprotected at the cost of circuit reliability --

28. Current Status Current Status (TokyoXP) The machines has installed the OS and connected the network Ready to collect data Netflow Ready to collect a Netflow data with Juniper router Problem with the Procket routers: The Procket routers create the UDP data of MTU size for netflow, IP packet containing UDP header and IP header exceed the MTU size. Hence fragmentation is occurred and flow-tools cannot receive packets correctly. ->will be fixed in the next software version Throughput and Latency Ready to collect throughput and latency data with the peering machine located in Chicago, managed by John-san. Need to tune some parameter for good throughput data (buffer, flow-control ) Our L2 switch didn t perform wire-rate switching. Some packets was discarded at input port while testing a high bit rate. we will investigate in detail.

29. Roadmap under discussion with Indiana University, but the following schedule is envisaged: 2004.7 Netflow & piPEs (temporal) Website was formed: http://www.jp.apan.net/NOC/Observatory/ 2004.8 start to develop the real system 2004.10 Connection with Abilene Observatory 2005.1 Presentation at APAN Bangkok meeting

30. Deployment into APAN We expect that Observatory system will be widely deployed in APAN. TransPAC Observatory APAN Observatory

31. References Abilene Observatory http://abilene.internet2.edu/observatory/ http://loadrunner.uits.iu.edu/~neteng/nms/ Tools web100 http://www.web100.org/http://www.web100.org/ GNET-1 http://www.gtrc.aist.go.jp/gnet/http://www.gtrc.aist.go.jp/gnet/ zebra http://www.zebra.org/http://www.zebra.org/ Smokeping http://people.ee.ethz.ch/~oetiker/webtools/smokeping/ http://people.ee.ethz.ch/~oetiker/webtools/smokeping/ Flow-tools http://www.splintered.net/sw/flow-tools/http://www.splintered.net/sw/flow-tools/ OWAMP http://e2epi.internet2.edu/owamp/http://e2epi.internet2.edu/owamp/ Iperf http://dast.nlanr.net/Projects/Iperf/http://dast.nlanr.net/Projects/Iperf/

32. Thank you Please feel free to contact us if you have any questions and comments Jin Tanaka – tanaka@kddnet.ad.jptanaka@kddnet.ad.jp Takatoshi Ikeda – tk-ikeda@kddlabs.jptk-ikeda@kddlabs.jp