1. OnTimeMeasure Integration with Gush Prasad Calyam, Ph.D. (PI) pcalyam@osc.edu Paul Schopis, (Co-PI) Tony Zhu (Software Programmer) Alex Berryman (REU Student) Prototype Design Last Updated: January 21 st 2011

2. OnTimeMeasure Overview Goal: –Provide GENI community with a shared measurement service for provisioning on-going and on-demand measurement requests Expected Outcomes: –OnTimeMeasure Software to perform centralized and distributed measurement orchestration and provisioning of active measurements Centralized orchestration for continuous monitoring, persistent measurements storage and processed network measurement feeds Distributed orchestration for on-demand (real-time) measurement requests without need for persistent measurements storage –Measurement service that uses OnTimeMeasure software in GENI experiments to enable: Network paths monitoring Network weather forecasting Network performance anomaly detection Network-bottleneck fault-location diagnosis GENI Project Wiki – http://groups.geni.net/geni/wiki/OnTimeMeasurehttp://groups.geni.net/geni/wiki/OnTimeMeasure Researcher Web-portal – http://ontime.oar.nethttp://ontime.oar.net 2

3. Gush Overview Gush runs on a user desktop and executes commands on Client nodes in a GENI Slice The application description is written in an XML format with following specs: –Web addresses of software packages –List of target clients –Commands to be executed Two configuration files are required in Gush (filled in using a helper script): 1.Directory.xml: defines where to locate resources during execution port map slice Gush queries PlanetLab database about user slice and auto populates the Directory.xml file 2.Gush.prefs: define working dir and client prefix Source: Gush project website - http://gush.cs.williams.edu 3

4. Advantages of OnTimeMeasure-Gush Integration Improve Experiment Measurements in Classroom: –Students can instrument active and passive measurements within GENI; query/analyze measurement time series by using OnTimeMeasure-Gush –Get on-demand plots of measurement time series results of their experiments with simple commands in Gush Dont need to teach students GNUPlot Installation Ease of OnTimeMeasure: –Configure once and re-use in new slices –Login and install on a number of nodes from user desktop Gush takes care of ProtoGENI/PlanetLab authentication Management Automation, Scalability and Security of OnTimeMeasure : –Identical configuration process on Node Beacons can be done in parallel across large number of nodes –GENI experimenter can securely script the request/management/query of OnTimeMeasure measurements through Gush to address a network science and engineering research issue 4

5. OnTimeMeasure Software Modules Customizable software [*] developed at OSC/OARnet Two main modules installed within a GENI experiment slice –Node Beacon Installs tools that measure network health metrics such as: route changes, delay, jitter, loss, bandwidth –TCP/UDP Iperf, Traceroute, Ping, Pathload, OWAMP, etc. Runs measurements based on a schedule and outputs results –Root Beacon Installs Apache, MySQL and other packages Creates database tables and configuration files Generates measurement schedules for node beacons Collects data and provides dashboard visualization, statistical analysis (i.e., anomaly detection and weather forecasting) with alarm generation NOTE: Typically, two or more Node Beacons and one Root Beacon need to be installed on separate dedicated servers in a slice; it is however possible to install Root Beacon on one of the Node Beacon servers [*] P. Calyam, C.-G.Lee, E. Ekici, M. Haffner, N. Howes, Orchestrating Network-wide Active Measurements for Supporting Distributed Computing Applications, IEEE Transactions on Computers Journal (TC), 2006. Download Node/Root Beacon Software – http://ontime.oar.net/download/OnTimeMeasure_latest.phphttp://ontime.oar.net/download/OnTimeMeasure_latest.php 5

6. OnTimeMeasure Software Modules (2) Two modules that are external to the GENI experiment slice –OnTime Beacon User Features: Web-interface client for users to login, manage slice information and interactively control Node/Root Beacons within their slices Admin Features: User management (approve new users, maintain user information), Meta-data information repository NOTE: Typically, users do NOT need to install OnTime Beacon software and should use the instance hosted at - http://ontime.oar.net; GENI aggregate providers who wish to have a dedicated OnTime Beacon for their users can install their own instance of OnTime Beacon on a dedicated serverhttp://ontime.oar.net –OnTime Control Command-line interface client for users to login, manage slice information and control Node/Root Beacons within their slices in an automated manner NOTE: Typically, users need to install OnTime Control software in stand-alone mode on their desktops; if using experiment workflow tools such as GUSH, OnTime Control software will need to be installed on a Node/Root Beacon Download OnTime Beacon Software ( Beta! ) – http://ontime.oar.net/download/OnTimeBeacon_latest.phphttp://ontime.oar.net/download/OnTimeBeacon_latest.php Download OnTime Control Software ( Beta! ) – http://ontime.oar.net/download/OnTimeControl_latest.phphttp://ontime.oar.net/download/OnTimeControl_latest.php 6

7. OnTimeMeasure Architecture 7

8. OnTimeMeasure Centralized Orchestration Centralized scheduling for continuous monitoring, persistent measurements storage and processed network measurement feeds –Useful for network weathermaps and long-standing experiments with advanced measurement analysis capabilities 8

9. OnTimeMeasure User Workflow (ProtoGENI or PlanetLab Aggregates) 1.User creates an experiment slice using ProtoGENI or PlanetLab control framework tools –For ProtoGENI slice creation, see - http://groups.geni.net/geni/wiki/OnTime-Install http://groups.geni.net/geni/wiki/OnTime-Install –For PlanetLab slice creation, see - http://groups.geni.net/geni/wiki/OTM-PlanetLabInstall http://groups.geni.net/geni/wiki/OTM-PlanetLabInstall 2.User registers at the Researcher Web-Portal (http://ontime.oar.net), provides Slice Rspec information and requests installation of measurement instancehttp://ontime.oar.net –Slice RSpec should include reservation of any required measurement resources 3.Each experiment slice needing a measurement service gets its own OnTimeMeasure software instance –Node/Root Beacons need to be installed as slivers based on the instructions provided in Step-1 for the specific aggregate 4.Valid login to the Researcher Web-Portal allows user to interact with the measurement service in his/her experiment slice. Specifically, the user can: i.Submit measurement requests ii.Control the measurement service iii.Query measurement data 9

10. Deployment Steps for OnTimeMeasure-Gush Slice must be created using an RSpec file or using graphical interfaces XML file of the application description must be created with at least two parts: –Commands to be executed only on the Root Beacon –Commands to be executed on all of the Node Beacons Description of the slice resources must be written in directory.xml Gush.prefs must be configured correctly as per Gush setup instructions Gush Controller Otm- deploy.xml Root Beacon Node Beacon Root Beacon directory.xml gush.prefs Node Beacon GENI Slice 10

11. OnTime Control (Stand-alone Mode) User installs OnTime Control on his/her desktop –Requires Python install –Manually configure slice topology in.yaml files User executes python scripts that query a SQL database on the Root Beacon –Requires root db user information Measurement result time-series files/plots are automatically downloaded into current directory on user desktop User Desktop OnTime Control Python Scripts Slice Topolgy.yaml files Root Beacon Root Beacon DB Node Beacon Node Beacon Node Beacon Node Beacon Node Beacon Node Beacon GENI Slice 11

12. OnTime Control (GUSH Integration mode) OnTime Control package resides on Root/Node Beacon; Gush is installed on user desktop Write a set of XML experiment files in the Gush format that interacts with the OnTime Control python scripts Gush topology description (directory.xml) is passed to the OnTime Control Output of the OnTime Control is a URL of a text/plot file containing the results of the measurement –Is it possible to auto-download URL information to current directory of user desktop within Gush? User Desktop OnTime Control Python Scripts Slice Topology Root Beacon Root Beacon DB Node Beacon Node Beacon Node Beacon Node Beacon Node Beacon Node Beacon GUSH Controller otm- control.xml directory.xml GENI Slice 12

13. Integration Timeline Prototype –Software bundle and Wiki page (Milestone: End of January 2011) OnTimeMeasure-Gush Demo –GENI experiment use case (Milestone: March GEC10) GENI User Case Studies –Class for CDN at Williams College? (Milestone: ?) 13

14. Reference List OnTime Control Wiki –http://groups.geni.net/geni/wiki/OTM-Controlhttp://groups.geni.net/geni/wiki/OTM-Control OnTimeMeasure-PlanetLab Wiki –http://groups.geni.net/geni/wiki/OTM-PlanetLabInstallhttp://groups.geni.net/geni/wiki/OTM-PlanetLabInstall OnTimeMeasure-Gush Wiki –To be created 14