1. Output Analysis and Experimentation for Systems Simulation

2. Performance Measure Simulation output analysis concerns with using simulation to estimate the quantities of interest in a simulation model Quantities are referred as Performance Measure Examples of performance measures include –The average system time of the first n customers –The long-run average system time per customer –The first time the system breaks down –The long-run average fraction of time the system is down.

3. Types of simulation There are two types of simulation: 1)Transient Simulation (Short-term simulation) 2)Steady-State Simulation (Long-term simulation) Transient simulation is easier to study than the steady-state simulation, because we cannot simulate the system until infinity.

4. Transient Simulation We are interested in studying the performance measure of the system for a finite time horizon. This time could be deterministic such as: –The performance of a system for one day, one month etc. Or Stochastic (the time a certain event occurs) such as: –The performance of the system until the first time it breaks down.

5. Transient Simulation Suppose we want to estimate  = E{f(X(t)): 0 < t < T} Where –f is a deterministic function, –{X(t)} is a stochastic process (a sequence of random variables), –T is a stopping time either deterministic or stochastic. Example. X(t) =N(t): the number of customers in an M/M/1 queuing system at time t.

6. Estimating Performance Measure in the Transient Simulation Suppose we want to estimate the average number of customers in system; N, before it breaks down We simulate the system n times until it breaks down. Each run, we get an estimate of the number of customers as N i (Note that N i is a random variable). We estimate the average number of customers in the system by the estimator

7. Properties of estimators 1)Unbiased: E{Z n } =  2)Strongly consistent: almost surely (i.e. with probability = 1) 3)The sequence {Z n } is asymptotically normal: The convergence here in distribution.

8. Confidence Interval for the mean The 100 (1-  ) % confidence interval I  is an interval that satisfies: P{   I  } = 1 –  Use the asymptotic normal property to build the confidence interval as Where Z  can be found in the standard normal distribution table that represent the z value where the area under the normal distribution is 1- .

9.  is the standard deviation of the mean. It can be estimated by s where: Some values for Z  :are as follows For 90% C.I, use Z  =1.645 For 95% C.I, use Z  =1.96

10. Steady-State Simulation We are interested in studying the performance measure of the system for a long run period of time. We assume that the system will eventually settle down (becomes stable).

11. Steady-State Simulation Two methods: –Multiple Replications with Initial Deletion. –Batch Means Method.

12. Multiple Replications with Initial Deletion Run the simulation for a long simulation run Delete initial observations (Initial Bias Contamination) Evaluate the average of the remaining data as a one sample of the mean. Repeat for several times to get estimate for the mean an confidence interval. Disadvantage of Multiple Replications Each replication we delete some observations, so we do loose some expenses of simulation

13. Batch Means Method Do only one very long simulation run, say on the interval [0,T]. Delete few contaminated observations at the beginning [0,T 0 ] Divide the remaining [T 0, T] into k subintervals [T 0, T 1 ), [T 0, T 2 ), …, [T k-1, T], and Find a sample of the performance measure Z j in each subinterval j Construct the 100(1-  ) % confidence interval for the data Z 1, Z 2, …, Z k