1. 1 Network Measurement Summary ESCC, Feb. 2006 Joe Metzger ESnet Engineering Group Lawrence Berkeley National Laboratory

2. 2 General Trends 1 There has been a community shift in the focus of network measurement efforts o From supporting the people running the networks o To supporting the people using the network o How we get there -Backbone providers agree on one set of tools to archive and publish measurement data –ESnet, Internet2, GEANT, most European NRENs and others are all collaborating on PerfSONAR –Standardization on collection tools may also happen -The user community develops analysis and visualization tools that leverage the backbone provider data to meet their needs –EGEE network analysis tool –PerfsonarUI –MonALISA?

3. 3 General Trends 2 10 Gigabit networking will present new challenges o The high costs of 10GE routers is pushing people to deploy different architectures using lower cost switches. o Many of these lower cost devices provide very limited diagnostic and debugging support. o Over subscription of circuits and bottlenecks within equipment may cause un-expected packet loss. o The differences between OC192, 10GE LANPHY, 10GE WANPHY and almost 10GE line-rate equipment may cause buffering problems. o These issues will lead to problems that will be challenging to identify and resolve.

4. 4 ESnet Deployed Tools & Capabilities  ESnet Performance Centers  SNMP & Flow Stats  Owamp  PerfSONAR  Utilization Measurement Archive  Traceroute Visualization

5. 5 ESnet Performance Centers A web interface that lets ESnet users perform tests to points in the ESnet core Ping Traceroute Iperf Pathload Pathrate Pipechar

6. 6 2 SNMP Collection Systems One system based on MRTG o Collects interface utilization, error and discard data every minute from ~1383 physical and logical interfaces o Used for day to day operations. o Portions accessible via http://netinfo.es.net Second collection system based on netSNMP o Collects interface utilization o Monthly reporting o Details back to 1990 -http://www1.es.net/pub/esnet-stats/ The two systems back each other up

7. 7 Netflow Collect at ESnet ingress points Trending o Identify large traffic flows o Observation: About 100 host pairs generate 30% of all ESnet traffic Peering Analysis o ESnet has around 100 peers -Most of them at multiple locations -Look at “what if” scenarios for traffic engineering Diagnosing problems o Identifying endpoints of a flow that saturated a site access link

8. 8 OWAMP Initially deployed to measure & compare o DOE Lab to DOE Lab performance o DOE Lab to University performance o Mesh between -FERMI, LBNL, BNL -OSU, NCS, SDSC Very small network changes can generate measurable differences in latency. OWAMP monitors deployed in most ESnet hubs

9. 9 PerfSONAR Utilization MA Provide ESnet users and network researchers access to ESnet link utilization data o Deployed a PerfSONAR Measurement Archive (MA) -Uses the “EGEE” tagged release of PerfSonar Java MA -Provides a web services interface using NMWG V2 schema -Data source is RRD files populated by SNMP system –Copied to MA every 5 minutes o Provides link capacity and link utilization information for over 800 ESnet interfaces. Software that uses the ESnet MA o EGEE network diagnostic tool o CNM (DFN Customer Network Management tool) o PerfSONARUI o Traceroute Visualizer

10. 10 PerfSONAR Traceroute Visualizer Displays a web form User pastes a traceroute output into the form Parses the form data to find IP addresses Displays the traceroute with interface utilization graphs for all the interfaces it can find. Developed a quick demo to show the value of a standardized measurement infrastructure Show demo if time permits

11. 11 Traceroute ESnet PerfSONAR Traceroute Visualizer Trace Submitted Tracing route to cache3.bnl.gov 130.199.3.21 over a maximum of 30 hops 1 1 ms 1 ms 1 ms joem-fe-stub.es.net 198.124.224.5 2 21 ms 21 ms 21 ms chi-ameslab.es.net 134.55.208.38 3 21 ms 21 ms 21 ms chicr1-ge0-chirt1.es.net 134.55.209.189 4 41 ms 41 ms 41 ms aoacr1-oc192-chicr1.es.net 134.55.209.58 5 43 ms 43 ms 43 ms bnl-oc48-aoacr1.es.net 134.55.209.130 6 43 ms 43 ms 43 ms bnl-esbnl.es.net 198.124.216.114 Analyzing Trace

12. 12 CNM from DFN 1

13. 13 CNM from DFN 2

14. 14 PerfsonarUI 1

15. 15 PerfsonarUI 2

16. 16 PerfsonarUI 3

17. 17 ESnet Future Plans 1 General o Continue the PerfSONAR collaboration with Internet2, GEANT, and the European National Research & Education Networks o Deploy additional components of the PerfSONAR architecture as it is developed -Lookup service -Topology service -Bandwidth measurement points -Latency measurement points -Authentication service o Enhance & extend our end to end monitoring efforts -Include more LHC sites -Interoperate with other tools used in LHC community

18. 18 ESnet Future Plans 2 SNMP o Use PerfSONAR to continue to publish ESnet interface data Latency o Integrate existing ESnet latency MPs with PerfSONAR o Deploy new latency MP’s in new hubs Bandwidth o Integrate ESnet Performance Centers with PerfSONAR -GE capable bandwidth MPs o Deploy new PerfSONAR bandwidth MPs -10 GE capable

19. 19 Suggestions for Sites Deploy test servers at the edge of your network. o OWAMP for measuring latency o IPERF/BWCTL for measuring bandwidth o Internet2 is measurement workshops Collect interface utilization data o Capture data with MRTG, Cricket, SNAPP or other tools. Join the PerfSONAR development team if you are actively developing measurement tools Think about data export policies Plan to deploy PerfSONAR in the summer o After we generate a release…

20. 20 Summary ESnet will deploy tools: o To run the backbone o To provide the data necessary for end users to develop and tune applications Sites should deploy tools that will allow them to: o Determine if their applications are working correctly o Determine if the local network is working correctly o Generate useful information for diagnosing problems Use these tools to: o Continuously document performance so you know when it changes o Share your network measurements results

21. 21 Pointers o http://www.perfsonar.net http://www.perfsonar.net o http://netinfo.es.net http://netinfo.es.net o https://performance.es.net https://performance.es.net o http://e2epi.internet2.edu http://e2epi.internet2.edu