Performance Analysis and Monitoring Facilities in CPN Tools Tutorial CPN’05 October 25, 2005 Lisa Wells

Oct. 25, 2005 Lisa Wells, CPN'05 2 Plan Observing and controlling simulations Introduction to monitors Example: queue system Demo 1 Performance facilities Demo 2

Oct. 25, 2005 Lisa Wells, CPN'05 3 Simulations Simulations of CP-nets are run for many reasons:  Debugging  Analysis of system behavior (performance or functional)  Presentation of a model to project team  Communication with external programs The usefulness of the simulations is heavily dependent on the flexibility and functionality of the simulator.  Access to simulation information  Stopping simulations

Oct. 25, 2005 Lisa Wells, CPN'05 4 Accessing simulation information It is often useful to be able to exchange information between the CPN simulator and external processes or files. Code segments can be added to transitions for:  Reading and writing in files  Calculating some performance measures  Sending and receiving information from external processes

Oct. 25, 2005 Lisa Wells, CPN'05 5 Controlling a simulation Simulation stop criteria  Number of steps executed  Dependent on model time  Dependent on markings or transitions

Oct. 25, 2005 Lisa Wells, CPN'05 6 Common functionality These controlling and accessing activities share a common pattern: If certain conditions are fulfilled, then do something.  If transition T occurs, then save information in a file.  If there are no tokens on place P, then stop the simulation.  If model time is greater than C, then calculate the average number of tokens on place P.  If transition T occurs, send the values of variables x, y, and z to the animation facilities.

Oct. 25, 2005 Lisa Wells, CPN'05 7 Current problems in CPN Tools Problems controlling and accessing simulation information:  Cannot access marking information.  Net structure may have to be changed to obtain desired functionality.  If multiple transitions need to be inspected, then code segments must be duplicated.  Code segments cannot be disabled.  Only low-level support, which is difficult to use.

Oct. 25, 2005 Lisa Wells, CPN'05 8 Monitors in CPN Tools monitor (verb) to watch, keep track of, or check, usually for a special purpose Merriam-Webster’s Collegiate Dictionary A monitor is a mechanism that is used to observe, inspect, control or modify a simulation of a CP-net. Important characteristics of monitors:  They can inspect the states and events of a simulation, and take appropriate actions based on the observations.  There is an explicit separation between monitoring the behavior of a net, and modeling the behavior of the system.

Oct. 25, 2005 Lisa Wells, CPN'05 9 Levels of monitors Standard monitors  Very easy to define  Do not require users to write any code Parameterized monitors  Similar to standard monitors, but slightly more flexible, and require some programming User-defined monitors  Very flexible, but require more programming

Oct. 25, 2005 Lisa Wells, CPN'05 10 Kinds of monitors Simulation breakpoint monitors Write-in-file monitors User-defined monitors Data collector monitors

Oct. 25, 2005 Lisa Wells, CPN'05 11 Monitoring subnets A monitor can inspect markings and occurring transitions, with variable bindings, during a simulation  Zero or more places  Zero or more transitions Monitors are activated after simulation steps  After every simulation step, if no transitions are monitored  After every relevant transition has occurred

Oct. 25, 2005 Lisa Wells, CPN'05 12 Monitoring functions A monitor typically has several functions:  Initialization function Called once before a simulation  Predicate function Called after simulation steps  Observation function Called when predicate function returns true Extracts relevant data from the model  Action function Does something appropriate with the observed value  Stop function Called once after a simulation

Oct. 25, 2005 Lisa Wells, CPN'05 13 Example: simple queue system A single server processes two kinds of jobs. If the server is busy when a job arrives, the job is added to a queue. Job inter-arrival times and server processing times are exponentially distributed. Queuing strategy is FIFO. Server Queue Job arrival

Oct. 25, 2005 Lisa Wells, CPN'05 14 Template code: Write-in-File Monitor one place, and two transitions fun pred (bindelem, System'Queue_1_mark : Jobs ms) = let fun predBindElem (Arrivals'Arrive (1, {jobs, job})) = true | predBindElem (Server'Start (1, {jobs, job})) = true | predBindElem _ = false in predBindElem bindelem end fun init (System'Queue_1_mark : Jobs ms) = ""

Oct. 25, 2005 Lisa Wells, CPN'05 15 Demo 1 Monitoring tools, monitor index entries, marking menus, syntax checking, …

Oct. 25, 2005 Lisa Wells, CPN'05 16 Performance analysis using CPN CP-nets have the potential to be used to model and analyze the performance, i.e. quantitative aspects, of systems. In practice, there are very few studies using CPN for performance analysis. This is due, in part, to lack of tool support.

Oct. 25, 2005 Lisa Wells, CPN'05 17 Performance facilities for CPN Tools High-level support for simulation-based performance analysis is currently being implemented for CPN Tools.  Monitors Data collection during simulations Net-specific simulation breakpoints  Simulation output  Managing data files  Running multiple simulations Generating a status file for multiple simulations Collecting data from independent, terminating simulations Calculating confidence intervals for performance measures

Oct. 25, 2005 Lisa Wells, CPN'05 18 Performance analysis of the queue system Data collection monitors for calculating performance measures, such as:  Expected average delay in queue  Expected average queue length  Expected utilization of the server Breakpoint monitor:  Stop a simulation when 100 jobs have passed through the queue The system can be modeled in CPN Tools, but until now it has been very difficult to collect and process data, and to create net-specific breakpoints.

Oct. 25, 2005 Lisa Wells, CPN'05 19 Data collector monitors The data that is collected is numerical data, i.e. integers, infinite integers, reals. Statistics are calculated. Data can be saved in log files. Data can be post- processed, e.g. plotted in graphs, after a simulation has completed. #data counter step time

Oct. 25, 2005 Lisa Wells, CPN'05 20 Estimating average queue delay Wish to calculate average queue delay for the first n jobs in the queue. When a job arrives in the system, the time of its arrival must be noted and saved. When a job leaves the queue, its queue delay is (current time – arrival time).

Oct. 25, 2005 Lisa Wells, CPN'05 21 Estimating average queue length Wish to calculate the time- average queue length until n jobs have left the queue. When calculating statistics, the length of the queue is weighted with a time interval. Measure the length of the queue when the length changes. The time-average queue length is equal to the area under an appropriate curve divided by the appropriate interval of time.

Oct. 25, 2005 Lisa Wells, CPN'05 22 Estimating server utilization Wish to calculate the utilization of the server until n jobs have left the queue. Use an indicator variable: server is busy => 1, server is idle => 0. Weight the values with appropriate intervals of time. The utilization of the server is then equal to the area under an appropriate curve divided by the appropriate interval of time.

Oct. 25, 2005 Lisa Wells, CPN'05 23 Demo Performance analysis of the queue system…