1. STEPS IN THE DEVELOPMENT OF MODEL INPUT DATA Evaluate chosen distribution and associated parameters for goodness-of-fit Goodness-of-fit test provide helpful guidance for evaluating the suitability of a potential input model. However since there are no single correct distribution in real application, verdict of such test need necessarily be followed. But it will help in making sure at some % that the distribution chosen is very likely suitable.

2. Chi-Square Test One procedure for testing the hypothesis that a random sample of size n of random variable X follows a specific distributional form is the Chi-square goodness-of-fit test. It is to test whether to accept or reject a hypothesis It is valid for large sample size

3. SandalsSneakers Leather shoes BootsOtherTotal Male617139550 Female135716950 Total1922202514100 Table 1.d. Male and Female Undergraduate Footwear Preferences: Observed Frequencies with Row and Column Totals

4. SandalsSneakers Leather shoes BootsOtherTotal Male observed617139550 Male expected9.5111012.57 Female observed135716950 Female expected9.5111012.57 Total1922202514100 Male/Sandals:((19 X 50)/100) = 9. 5 Male/Sneakers:((22 X 50)/100) = 11 Male/Leather Shoes:((20 X 50)/100) = 10 Male/Boots:((25 X 50)/100) = 12. 5 Male/Other:((14 X 50)/100) = 7 Female/Sandals:((19 X 50)/100) = 9. 5 Female/Sneakers:((22 X 50)/100) = 11 Female/Leather Shoes:((20 X 50)/100) = 10 Female/Boots:((25 X 50)/100) = 12. 5 Female/Other:((14 X 50)/100) = 7 Table 1.e. Male and Female Undergraduate Footwear Preferences: Observed and Expected Frequencies

5. SandalsSneakers Leather shoes BootsOtherTotal Male observed617139550 Male expected9.5111012.57 Female observed135716950 Female expected9.5111012.57 Total1922202514100 Male/Sandals:((6 - 9. 5) 2 /9. 5) =1. 289 Male/Sneakers:((17 - 11) 2 /11) =3. 273 Male/Leather Shoes:((13 - 10) 2 /10) =0. 900 Male/Boots:((9 – 12. 5) 2 /12. 5) =0. 980 Male/Other:((5 - 7) 2 /7) =0. 571 Female/Sandals:((13 - 9. 5) 2 /9. 5) =1.289 Female/Sneakers:((5 - 11) 2 /11) =3. 273 Female/Leather Shoes:((7 - 10) 2 /10) =0. 900 Female/Boots:((16 - 12. 5) 2 /12. 5) =0. 980 Female/Other:((9 - 7) 2 /7) =0. 571 Table 1.f. Male and Female Undergraduate Footwear Preferences: Observed and Expected Frequencies Plus Chi Square Table 1's chi square value of 14.026

6. Reading Playing computer games Swimming Collecting stamps OtherTotal Male15351056 Female20517239 Total Derive the chi-square value from the above table Exercise

7. Kolmogorov -Smirnov Test Formalizes the idea behind examining a q-q plot. Is particularly useful when sample sizes are small and no parameter has been estimated from the data. Does not take any special tables when an exponential distribution is assumed.

8. ANALYZING THE OUTPUT Verification Validation Single model Output analysis Verification Concern with building the model right. It is utilized in the comparison of the conceptual model to the computer representation that implements that conception.

9. Verification It will see whether the model is implemented correctly in the computer. Are the input parameters and logical structure of the model correctly represented Concern on the model building.

10. Verification Achieve through the calibration of the model, an interactive process of comparing the model to actual system behavior and using discrepancies between the two, and the insight gained, to improve the model. Process repeated until model accuracy is judged to be acceptable

11. Verification Steps suggested for verification process Have the computerized representation be checked by other parties other than its developer. Closely examine the model output for reasonableness under a variety of settings of input parameters. Have the computerized representation print out a wide variety of output statistic

12. Verification Make computerized representation as self-documenting as possible. Give a precise definition of every variable used and a general description of the purpose of each major section code. If the computerized representation animated, verify what is seen in animation imitates the actual system. Examples of error is entities that disappear unintentionally during simulation or automated guided vehicles (AGVs) that pass through one another at intersection

13. Verification Use an interactive run controller (IRC) or debugger which is an essential component that checks for logical errors. Graphical interfaces are recommended for accomplishing verification and validation because it simplifies the task of model understanding.

14. Example Definitions of variables: Clock = simulation clock EVTYP=Event type(start,arrival,departure, or stop) NCUST= Number of customer in system at time ‘CLOCK’ STATUS= Status of server (1-busy,0-idle) State of system just after the named event Occurs CLOCK =0EVTYP=‘start’NCUST=0STATUS=0 CLOCK=3 EVTYP=‘arrival’NCUST=1STATUS=0 CLOCK=5 EVTYP=‘depart’NCUST=0STATUS=0 CLOCK=11 EVTYP=‘arrival’NCUST=1STATUS=0 CLOCK=12 EVTYP=‘arrival’NCUST=2STATUS=1 CLOCK=16 EVTYP=‘depart’NCUST=1STATUS=1

15. Verification example When verifying the computer implementation in general purposed languages such as FORTRAN,Pascal,C or C++ or in most simulation languages of a single server queue, an analyst made a run over 16 units of time and observe that the time-average length of the waiting line was L=0.4375 which was a reasonable answer for such a short run.

16. The trace you’ve seen before is a hypothetical printout of the simulation. Here you could see on line 2 clock time 3 the NCUST is 1 but the STATUS is 0. This could be due to a logical error whereby the STATUS was not set to one when customer enter the System. Although through equation the value for the average waiting line is somewhat acceptable but without further verification on the simulation hypothetical printout the logical error we found would likely be identified. Verification example

17. Verification Most simulation software have these built-in capabilities to conduct trace without the programmer having to do extensive programming. It can be imagined that tracing a large span of simulation time can produce an extremely large amount of printout and make it more difficult to trace out error. That is why some software have selective trace that can be set on particular entity or condition. This will help simplify things.

18. Validation Although is conceptually distinct to verification usually are conducted simultaneously by the modeler. Validation is the overall process of comparing the model and its behavior to the real system and its behavior. The comparison of the model to reality is carried out by a variety tests, some subjective and others objective.

19. Validation Subjective test usually involve people, who are knowledgeable about one or more aspect of the system, making judgments about the model and its output. Objective test always require data on the system’s behavior plus the corresponding data produced by the model. Then one or more statistical test are preformed to compare some aspect of the system data set to the same aspect of the model data set.

20. Validation One possible criticism at this stage is that the model has been validated using only a set of data and one way to alleviate this is to collect another set of data or to reserved a portion of the original system data to be used at the final stage of validation. That is after the model has been calibrated using the original data set. The final validation is conducted using the second data set.

21. Validation If discrepancies occurs during the second validation process the modeler must modify the model until it become acceptable. Validation is not an either/or proposition- no model is totally the representation of the real system thus the process of modifying and conducting validation test is time and cost consuming. Due to that fact the modeler and the model user should lay the maximum discrepancy between model predictions and system behavior that is acceptable.

22. Face Validity To make sure high degree of realism is built into the model through reasonable assumptions regarding system structure and reliable data. Make used of insight by user or knowledgeable person to increase the model reasonableness and check for initial model deficiencies. This will involve the user in the calibrating process as the model is iteratively improved.

23. Face Validity Sensitivity analyst can also be used to check the model face validity. The model user is asked if the model behaves in the expected way when one or more input variables is changed. In cases of large-scale simulation models, there are many input variable that can be test on their sensitivity thus modeler must choose only the most critical input variables for the testing.

24. Validation of Model Assumptions Two classes : Structural assumptions and Data assumptions Structural assumptions involve questions of how the system operates and usually simplifications and abstractions of reality. Example : Consider customer in a bank. There might be just one line of customer or individual line per teller. If there are many lines customer may be treated as first come first serve, customer changed line if one line is faster or customer is treated according to numbering status.

25. Validation of Model Assumptions Data assumptions should be based on the collection of reliable data and correct statistical analysis of the data. Example : In the bank study data were collected on : 1.Inter arrival times of customers during several 2 –hour periods of peak loading (“rush-hour” traffic) 2.Inter arrival times during slack period 3.Service times for commercial accounts 4.Service times for personal accounts

26. Validation of Model Assumptions The reliability of the data was verified by consultation with bank managers, who identified typical rush hours and slack times. When combining two or more data sets collected at different times, data reliability can be further enhanced by objective statistical test for homogeneity of data. Example : Do two data set {x} and {y} on service times for personal account, collected at two different times, comes from the same parent population? If so both can be combined. Analyzed the input distribution as what being discussed in class recently.

27. Validating input output Transformation The only objective test of the model as a whole is its ability to predict the future behavior of the real system when the model input data match the real inputs and when a policy implemented in the model is implemented at some point in the system. In other words the structure of the model should be accurate enough for the model to make good predictions not just for one input data set, but for the range of input data sets which are of interest or importance.

28. Validating input output Transformation In this phase of validation process, the model is viewed as an input-output transformation. That is the model accepts values of the input parameters and transforms these inputs into output measures of performance. It is this correspondence that is being validated. Instead of validating the model by predicting the future, the modeler may use past historical data which has been reserve for special purposes. Thus accurate “prediction of the past” may replace prediction of the future for purposed of validation.

29. Validating input output Transformation A model is usually develop with primary interest in a specific set of system responses to be measured under some range of input conditions. If the model is later used for purpose different from its original purpose the model should be revalidated in terms of the new responses of interest and under the possibly new input conditions.

30. Validating input output Transformation A necessary condition for the validation of input-output transformations is that some version of the system under study exist, so that system data under at least one set of input conditions can be collected to compare to model predictions. If the system is still in the planning stage and no system operating data can be collected, complete input-output validation is not possible. Other type of validation should be conducted to the extent possible.

31. Validating input output Transformation There are situation whereby sub systems of planned system exist and a partial input-output validation can be conducted. Similar tested model can be used as medium of comparison towards alternative systems designs with confidence that the model of the existing system can be transferred to the model of the new model system. This is exceptional accepted in situation whereby the new model is relatively minor modification of the old model. It may result a major change to the actual system.

32. Validating input output Transformation Changes to the computerized representation of the system are as follow : 1.Minor changes of single numerical parameters such as speed, arrival of customer etc. 2.Minor changes of the form of statistical distribution such as service time or time failure of a machine. 3.Major changes in logical structure of sub system, such as change in queue discipline or change in the scheduling rule of a shop model 4.Major changes involving different design for new system, such as computerized replacing non-computerized system.