Simulation Techniques Overview Simulation environments emulation/ exec- driven event- driven sim trace- driven sim stochastic sim Workload parameters System Config parameters Factor levels Result Data

© 2003, Carla Ellis Why Simulate System to be tested is not (yet) available Large design space to be explored Need more detail than analytical modeling Dynamic

© 2003, Carla Ellis Common Mistakes Inappropriate level of detail –Details have to be known and “right” –Start with less detail and get preliminary results – then refine Improper language/package –Bad software engineering practices Unverified – bugs –Mysterious results not explored Not validated against real system – incorrect assumptions

© 2003, Carla Ellis Common Mistakes Improperly handled initial conditions Too short simulations Poor random number generators Improper selection of seeds –Share on stream for multiple uses –Use the same seed for multiple streams (correlated behavior)

© 2003, Carla Ellis Terminology State variables – define the state of the system Event – change in system state Time models: –Continuous – system state always defined –Discreet – system state only at discreet instants in time State models: –Continuous – state variables take continuous values –Discreet – state variables discreet values

© 2003, Carla Ellis Terminology Time State ContinuousDiscreet Continuousy(t) = sin (wt) y(t+1) = y(t) +  Discreety(t) = t mod n y(t+1) = y(t) + i

© 2003, Carla Ellis Time State ContinuousDiscreet Continuousy(t) = sin (wt) y(t) = tide(t) y(t+1) = y(t) +  y(date) = max(tide(t)) Discreety(t) = t mod n y(t) = population y(t+1) = y(t) + I y(date) = max attendence of cps296.6(date) Terminology t=0 to 23:59

© 2003, Carla Ellis Terminology Deterministic vs. probablistic –Probabilistic gives different result on replications Static vs. dynamic –Dynamic – state changes with time Open vs. closed –Open – input comes in from outside –Closed – no external input, circulating in model Stable vs. unstable – does it ever settle into a steady state?

© 2003, Carla Ellis Types Emulation – directly execute on one system, mimicking another –Modelnet –Accounting issues may resemble simulation Monte Carlo simulation – static systems Trace-driven –As discussed in workload discussion, traces should be independent of system under study (no feedback effects) Discrete-event –Time-driven – time advances in unit steps –Event-driven – time jumps to next event time

© 2003, Carla Ellis Structure of Discrete Event Simulation eventQ scheduler Event handlers State var results

© 2003, Carla Ellis Structure of Discrete Event Simulation eventQ scheduler Event handlers State var results

Simulation Techniques Overview Simulation environments emulation exec- driven sim trace- driven sim stochastic sim Workload parameters System Config parameters Factor levels Result Data Discrete events

Simulation Techniques Overview Simulation environments emulation exec- driven sim trace- driven sim stochastic sim Workload parameters System Config parameters Factor levels Result Data Discrete events open open or closed Events – directly exec.

© 2003, Carla Ellis Validation Are the assumptions of the model reasonable and represent the real system – producing results that are meaningful to the real system. –Assumptions –Input parameters –Output values and conclusions Compare to another representation –Real system measurements –Theoretical results –Expert intuition

© 2003, Carla Ellis Verification Debugging for correct implementation of the model – general software debugging strategies. Testing techniques specific to simulation –Understanding the effect of input parameters allows continuity test small D in input should yield small D in results –Consistency – produces similar results for inputs that should have similar effects –Degeneracy tests – works at the extremes

© 2003, Carla Ellis Transient Removal Assuming steady-state performance Long runs Initialization at a approx. a steady state Truncation – ignoring the 1 st l observations based on range Initial data deletion – relative change in overall mean decreases after l Moving average of independent replications

© 2003, Carla Ellis Initial Data Deletion x l -x x l