1. T.C ATILIM UNIVERSITY MODES ADVANCED SYSTEM SIMULATION MODES 650

2. WORK SMARTER, NOT HARDER: GUIDELINES FOR DESIGNING SIMULATION EXPERIMENTS S USAN M. S ANCHEZ O PERATIONS R ESEARCH D EPARTMENT AND G RADUATE S CHOOL OF B USINESS & P UBLIC P OLICY N AVAL P OSTGRADUATE S CHOOL M ONTEREY, CA 93943-5219, U.S.A. REPRESENTED BY : A DEL A GILA

3. D ESIGN O F E XPERIMENTS (DOE) Experimental Designs indicate how to vary the setting of factors (qualitative or quantitative) to see whether and how they affect the response.

4. D ESIGN OF E XPERIMENTS (DOE) A statistics-based approach to designed experiments A methodology to achieve a predictive knowledge of a complex, multi-variable process with the fewest trials possible (information and time) An optimization of the experimental process

5. C ONCEPTS The relation between the response and the factors can be represented. Mathematically, graphical methods, In general, a design is a matrix. Cols (factors) Rows (combination of factor levels) or ( design point) Natural and Coding levels. Replication.

6. F ALLS TO A VOID Firstly, when several people each suggest an interesting combination of factor setting. Secondly, when the people starts with a baseline scenario and vary one factor at a time. Ex. The children’s game of capture the flag

7. W HY P ROJECTS ARE A LWAYS L ATE task priority days=27days C. path The probabilities of completing or failing to complete can be estimated. A real world projects generally is more complicated 1

8. DESIGN S We can do Treating factors Explore the system Know which factor or interactions have the greatest influence Seek solutions

9. DESIGNS 3 3 FACTORIAL EXPERIMENT Geometric Presentation X 3 X 2 X 1

10. 2 K F ACTORIAL D ESIGNS (C OURSE G RIDS ) ( When) Most efficient if we can assume that the simulation response is well fit by a model with only linear main effects or interactions. Constraction 2k2k Factors (# columns) Level ( - + ). In each column with 2 n-1 n= col number (T.C.) Treatment Combinations (# rows ) ( design points) Graphically

11. 2 K F ACTORIAL D ESIGNS (C OURSE G RIDS ) 2_ways of Interactions stronger than 3_way…….. 3 ways of Interactions For K factors and r ways of Interactions # r ways of interactions = If K =3, then 3 way of interaction and 2 k -1 terms + the intercept. We may ignore one or more interactions to get one or more Degree Of Freedom Main effects

12. M K F ACTORIAL D ESIGNS (F INER G RIDS ) 3 k means 3 levels ( -, 0, and +) for coded levels. The larger the value of m for an m k factorial design, the better its space-filling properties.

13. Minimal level of stealth then speed is more important Good results Poor results Middle results

14. M K F ACTORIAL D ESIGNS (F INER G RIDS ) Advantages Provide greater detail easy to interpret the results Disadvantages Massive data required for more than 10 factors The number of high order interaction is just waste of effort. So, we need more efficient types of EOD in exploring many factors

15. L ATIN H YPERCUBE D ESIGNS K=2 # Factors N=11 #Design points

16. L ATIN H YPERCUBE D ESIGNS Comparing with the 2 2 factorial design, the LH design provides some information about what happens in the center of the experimental region. Comparing with 11 2, No detailed information about the boundaries between regions of poor, fair, and good performance, but we do find that success occurs when both speed and stealth. More benefits for large K (Saving the time )

17. L ATIN H YPERCUBE D ESIGNS Random LH designs have good orthogonal properties if N is much larger than k, but for smaller designs some factors might have high pair wise correlations. The solutions One approach is to generate many random LH designs and then choose a good one. Data Requirements for Nearly Orthogonal Latin Hypercube Designs.

18. L ATIN H YPERCUBE D ESIGNS

19. F OR A RANDOM LH DESIGN, Each column is randomly permuted. In one replication, each of the k factors will be sampled exactly once at each of its N levels.

20. 2 K - P R ESOLUTION 5 F RACTIONAL F ACTORIAL D ESIGNS LH designs work best when most factors have many levels. Sometimes many factors take on only a few levels. We can cut down the number of runs required HOW? MkMk Level ( - + ). In each column with 2 n-1 n= col number Factors (# columns)

21. Consider Assume no interactions, instead put 4 factors. The resulting design is called a 2 7-4 fractional factorial, because the base design varies seven factors in only 2 7-4 = 8 runs instead of 2 7 = 128 runs! Terms for a 2 7-4 Fractional Factorial Design If we take b=2 replications, we can examine seven factors in only 16 runs

22. 2 K - P R ESOLUTION 5 F RACTIONAL F ACTORIAL D ESIGNS Disadvantages if there truly are strong interactions but we ignore them when setting up the experiment, these design will not be good. A compromise is to use R5 fractional factorials. These lead to use (a half fraction ) 2 k-1 Except for k<4.

23. C ENTRAL C OMPOSITE D ESIGNS (CCD) 2 k-p factorials sample each factor at only 2 levels No idea about what happens in between To estimate all full second –order models ( main effect, 2-way interaction, and quadratic effects), we need CCD

24. C ONSTRUCTION OF CCD Start with a 2 k factorial or R5 2 k-p fractional factorial design. Then add a center point and two star points The center point(0,0,0) (c,0,0) if c=1 the star points will be on the face of the cube (0,-c,0) 2323

25. C ROSSED AND C OMBINED D ESIGN What are robust decisions or policies? A robust design approach means that the factors are classified into two groups: decision factors, which represent factors that are controllable in the real world setting the simulation models, noise factors, which are uncontrollable or controllable only at great cost in the real world, but potentially affect the system’s performance. Specific factors, such as run lengths.

26. C ROSSED AND C OMBINED D ESIGN