1. 1 Deploying Measurement Systems in ESnet Joint Techs, Feb. 2006 Joseph Metzger ESnet Engineering Group Lawrence Berkeley National Laboratory

2. 2 Outline Deployed Tools & Capabilities  ESnet Performance Centers  SNMP & Flow Stats  Owamp  PerfSONAR  Utilization Measurement Archive  Traceroute Visualization Future plans

3. 3 ESnet Performance Centers A web interface that lets ESnet users perform tests to points in the ESnet core Ping Traceroute Iperf Pathload Pathrate Pipechar

4. 4 2 SNMP Collection Systems One system based on MRTG o Collects interface utilization, error and discard data every minute from ~1383 physical and logical interfaces o Used for day to day operations. o Portions accessible via http://netinfo.es.net Second collection system based on netSNMP o Collects interface utilization o Monthly reporting o Details back to 1990 -http://www1.es.net/pub/esnet-stats/ The two systems back each other up

5. 5 Netflow Collect at ESnet ingress points Trending o Identify large traffic flows o Observation: About 100 host pairs generate 30% of all ESnet traffic Peering Analysis o ESnet has around 100 peers -Most of them at multiple locations -Look at “what if” scenarios for traffic engineering Diagnosing problems o Identifying endpoints of a flow that saturated a site access link

6. 6 OWAMP Initially deployed to measure & compare o DOE Lab to DOE Lab performance o DOE Lab to University performance o Mesh between -FERMI, LBNL, BNL -OSU, NCS, SDSC Very small network changes can generate measurable differences in latency. OWAMP monitors deployed in most ESnet hubs

7. 7 Metro DWDM Reroute Added about 0.35 ms Latency Fiber Re-Route

8. 8 Asymmetric Routing between FERMI and SDSC (LBL to CENIC link Maintenance)

9. 9 PerfSONAR Utilization MA Provide ESnet users and network researchers access to ESnet link utilization data o Deployed a PerfSONAR Measurement Archive (MA) -Uses the “EGEE” tagged release of PerfSonar Java MA -Provides a web services interface using NMWG V2 schema -Data source is RRD files populated by SNMP system –Copied to MA every 5 minutes o Provides link capacity and link utilization information for over 800 ESnet interfaces. Software that uses the ESnet MA o EGEE network diagnostic tool o CNM (DFN Customer Network Management tool) o PerfSONARUI o Traceroute Visualizer

10. 10 PerfSONAR Traceroute Visualizer Displays a web form User pastes a traceroute output into the form Parses the form data to find IP addresses Displays the traceroute with interface utilization graphs for all the interfaces it can find. A quick demo developed to show the value of a standardized measurement infrastructure Show demo if time permits

11. 11 Future Plans 1 General o Continue the PerfSONAR collaboration with Internet2, GEANT, and the European National Research & Education Networks o Deploy additional components of the PerfSONAR architecture as it is developed -Lookup service -Topology service -Bandwidth measurement points -Latency measurement points -Authentication service o Enhance & extend our end to end monitoring efforts -Include more LHC sites -Interoperate with other tools used in LHC community

12. 12 Future Plans 2 SNMP o Use PerfSONAR to continue to publish ESnet interface data Latency o Integrate existing ESnet latency MPs with PerfSONAR o Deploy new latency MP’s in new hubs Bandwidth o Integrate ESnet Performance Centers with PerfSONAR -GE capable bandwidth MPs o Deploy new PerfSONAR bandwidth MPs -10 GE capable