1. 7 - 1 Chapter 15: Monte Carlo Simulation PowerPoint Slides Prepared By: Alan Olinsky Bryant University Management Science: The Art of Modeling with Spreadsheets, 2e S.G. Powell K.R. Baker © John Wiley and Sons, Inc.

2. Introduction  Monte Carlo simulation is an important and flexible tool for modeling situations in which uncertainty is a key factor.  In order to perform Monte Carlo simulation on spreadsheet models, we use the Excel add-in Crystal Ball.  Simulation can describe not only what the outcomes of a given decision could be, but also the probabilities with which these outcomes will occur.  In fact, the result of a simulation is the entire probability distribution of outcomes.  In a sense, simulation is an advanced form of sensitivity analysis in which we attach a probability to each possible outcome.

3. Introduction  We often wish to determine the probability of a particular set of outcomes.  Such “tail probabilities” are often suitable measures of the risk associated with a decision.  While decision trees provide a simple means for analyzing decisions with uncertainty and risk, simulation is the tool of choice when there are a large number of uncertainties, especially when these are represented by continuous distributions.  Simulation is also a practical method when the underlying model is complex.  However, it is important to realize that, just as with decision trees, the result of a simulation is a probability distribution for each outcome.  Analyzing these distributions and extracting managerial insights is an important part of the art of simulation.

4. Essential Features of a Simulation  Start with a base case model and determine which of the input parameters to represent as uncertain.  Develop probability distributions for those inputs.  Take random samples from those inputs and calculate the resulting output, repeating the process until a clear picture of the output distribution emerges.  Create a histogram of the outcomes and interpret it.  Simulation provides two essential pieces of information: mean values (also called expected values) and tail probabilities (in this case, the probability of a positive profit).

5. Modeling Tip: Creating Simulation Models  Beginners to simulation modeling often find it difficult to build an initial spreadsheet model. This may be because a simulation model must correctly evaluate a large or even infinite number of different random inputs.  One useful trick is to fix the random inputs at some arbitrary value and build a spreadsheet model to evaluate those inputs.  This step allows us to build and debug a spreadsheet with no uncertainty, which is a simpler task than debugging a simulation model.  Only after we have debugged this model do we introduce uncertainty in sales.

6. Sensitivity Analysis  The base-case model should be thoroughly explored, using Data Sensitivity, Tornado Chart, or other methods, before undertaking a simulation analysis.  We can think of simulation as a sophisticated approach to sensitivity analysis.  Whereas sensitivity analysis is a necessary first step, and can often reveal unexpected relationships in the model, a simulation analysis is required to analyze the combined effects of changes in many inputs.

7. Selecting Probability Distributions – Creating Assumption Cells  Our first task is to convert our base-case model into a simulation model by replacing our fixed (deterministic) assumptions with probability distributions.  In Crystal Ball, cells containing probability distributions are called Assumption cells.  Select Define►Define Assumption from the Crystal Ball menu.  This step displays the Distribution Gallery, which is an input window containing all the probability distributions that are available in Crystal Ball.  Scroll until the appropriate distribution appears, select it, and press OK.  Enter the appropriate parameters.  When copying probability distributions in Crystal Ball, we must remember to use the Crystal Ball commands Define►Copy Data and Define►Paste Data.

8. Example of Simulation Model

9. Selecting outputs – creating Forecast cells  The second step in setting up a simulation model is to define the model outputs so that Crystal Ball can save these values during a simulation run.  In Crystal Ball, all output cells are called Forecast cells.  Highlight the cell containing the output value.  Select Define►Define Forecast from the Crystal Ball menu.  In the Name field in the Define Forecast window, Crystal Ball enters a name for the cell if there is a label nearby.  Click on OK to save the Forecast cell.

10. Setting Simulation Parameters  Crystal Ball allows the user to configure each simulation run by choosing values for a number of parameters.  These options are located under Run►Run Preferences on the Crystal Ball menu.  Most of these options can safely be left at their default settings.  The number of trials in Crystal Ball is the number of times model outputs (Forecasts) are calculated for different random values of the inputs.  Select Run►Run Preferences from the Crystal Ball menu.  Select the Trials tab.

11. The Run Preferences Window in Crystal Ball

12. Analyzing Simulation Outputs  To run the simulation, follow these steps: Select Run from the Crystal Ball menu. If the Reset Simulation option is available, select it. (This option allows Crystal Ball to erase previous simulation results.) Select Run►Start Simulation from the Crystal Ball menu.  Three windows open while Crystal Ball carries out the simulation run: One is a Control Panel that keeps track of the number of trials. The other two are Forecast windows that display histograms of the Forecast cells.

13. Example of Output: Forecast Window

14. Summary of the Simulation Process  Selecting uncertain parameters  Selecting probability distributions  Selecting output(s)  Running a simulation  Analyzing outputs

15. Crystal Ball Tip: Entering Assumption Cells  Highlight the target cell.  Select Define►Define Assumption.  Click on All in the Distribution Gallery window to display all 21 available families of distributions.  Click on a distribution to highlight it.  Click OK to select it.  Enter the required parameters either by entering numbers or cell addresses (starting with “=”) in the parameter field(s).  Click on Enter to display the distribution in the Distribution Gallery window.  Click on OK to enter it into the target cell.  Select Define►Cell Preferences to change the background color and other features of the display for Assumption cells.

16. Crystal Ball Tip: Entering Forecast Cells  Highlight the target cell.  Select Define►Define Forecast.  Enter a name for the Forecast in the Name field.  Enter units in the Units field (optional).  Click on OK to enter it into the target cell.  Select Define►Cell Preferences to change the background color and other features of the display for Forecast cells.

17. Crystal Ball Tip: Setting the Number of Trials  Select Run►Run Preferences.  Click on the Trials tab.  Enter the number of trials in the Number of trials to run field.  Click on OK to return.

18. Crystal Ball Tip: Analyzing Crystal Ball Outputs  Select Analyze►Forecast Charts.  Click on the Forecasts of interest and select Open.  In the Forecast window, there are five options under View for displaying the data: Frequency Cumulative Frequency Reverse Cumulative Frequency Statistics Percentiles  To display the mean value on the Forecast chart, select Preferences►Chart in the Forecast window. Select the Chart Type tab and select Mean under Marker lines.  To display all values of the Forecast cell, select Preferences►Chart in the Forecast window. Select the Axis tab and select the Fixed type scale (from – Infinity to +Infinity).

19. Simulation Sensitivity  Our Sensitivity Toolkit includes an option for this purpose called CB Sensitivity.  CB Sensitivity runs Crystal Ball over a range of input parameters and records the mean (or other statistics) for as many Forecast cells as we choose.  This is analogous to running Solver Sensitivity to evaluate the sensitivity of the optimal solution to an underlying parameter.

20. Example of First Input Window for CB Sensitivity

21. Example of Second Input Window for CB Sensitivity

22. Crystal Ball Tip: Running CB Sensitivity  Create and run a simulation model with at least one Forecast cell.  Open CB Sensitivity by choosing Sensitivity Toolkit►CB Sensitivity.  In the first input window, select from among the Forecast cells listed (all Forecast cells in the selected sheet will be listed automatically).  Select the statistics to record from these options: Number of Trials Mean Standard Deviation Minimum Maximum Mean Standard Error

23. Crystal Ball Tip: Running CB Sensitivity  Select One-Way table or Two-Way table. In a one-way table, a single parameter is varied; in a two-way table, two parameters are varied independently.  Click on Next.  In the second input window, input the Cell to Vary; this is the cell that will take on a range of successive values.  Select Input Type: Begin, End, Increment Begin, End, Num Obs  In the first option, the simulation is repeated for values of the parameter from the First Value to the Last Value in steps given by the Increment. In the second option, the range from First Option to Last Option is divided by the number of observations (Num Obs), and the simulation is repeated for that number of evenly spaced alternatives.  Input the First value, Last Value, and Increment/N.  Click on Finish.

24. Selecting Uncertain Parameters  Uncertain parameters should be selected only after a thorough sensitivity analysis.  The purpose of this sensitivity testing should be to discover which parameters have a significant impact on the results, and what the likely range of uncertainty is for each parameter.  The process begins with simple what-if testing of high and low values.  The Data Sensitivity tool can also be used to test a range of inputs and to determine whether the model is linear in the given parameter.  The Tornado Chart tool can then be used to test the impact of entire sets of parameters.

25. Selecting Uncertain Parameters  The easiest approach, although not the most revealing, is to vary each parameter by the same percentage (the Constant Percentage option).  More information is required to assign a separate range of variation to each parameter (the Variable Percentage option), but the results are more meaningful.  Finally, we can assess the extreme values of the distribution of each parameter (with the Percentiles option), to show how carefully selected extreme (and possibly asymmetric) inputs affect the outputs.  While none of these methods require us to develop a complete probability distribution, the Variable Percentage and Percentiles options do require us to think carefully about uncertainty.

26. Selecting Probability Distributions  Once we have selected a set of uncertain parameters, the next step is to choose a probability distribution for each one.  But which type of distribution should we choose: discrete, uniform, normal, triangular, or perhaps something else?  And once we have chosen a type of distribution, how do we choose its specific parameters (such as the mean and standard deviation for the normal distribution)?  While Crystal Ball provides a menu of twenty-one types of distributions, most business analysts use only a small handful of them.

27. Empirical Data and Judgmental Data  Empirical data consists of numerical observations from experience, such as monthly sales for the past four years or daily stock returns over the previous year.  Judgmental data are estimates made by experts in the field or by the decision makers most closely involved in the analysis.  We can learn to ask decision makers for probability estimates such as the mean, the minimum, or the 10th and 90th percentiles needed for tornado-chart analysis.

28. Empirical data Alone are Seldom Sufficient  In most cases, unless we are doing scientific research, no empirical data at all will be available. (Judgmental data, on the other hand, are usually available.)  Even if empirical data are available, the information may be biased or otherwise inappropriate for the purposes at hand.  Even if appropriate empirical data are available, it requires judgment to determine whether the distribution that provides the best fit to the given empirical data is appropriate in the model.  In many cases, the results of interest depend on the mean and variance of an uncertain parameter, but not on the specific form of the probability distribution.

29. Six Essential Distributions  The Yes/No distribution is used in situations where an uncertain parameter can take on one of only two possible values.  Discrete uniform distribution: We often wish to model a random outcome that takes on a small number of discrete values.  The binomial distribution is used for the number of outcomes on repeated trials.  The continuous uniform distribution describes an outcome that is equally likely to fall anywhere between a minimum and a maximum value.  The triangular distribution is a more flexible family of continuous distributions: these distributions are specified by three parameters: the minimum, maximum, and most likely values.  The normal distribution is defined in terms of its mean and standard deviation, but Crystal Ball provides the option to use alternative input parameters.

30. A Yes/No Distribution

31. A Discrete Uniform Distribution

32. A Binomial Distribution

33. A Continuous Uniform Distribution

34. A Triangular Distribution

35. A Normal Distribution

36. Crystal Ball Tip: Entering Distributions  Crystal Ball provides two distinct ways to enter probability distributions:  Through the Distribution Gallery (Define►Define Assumption).  Use Crystal Ball functions for distributions, such as CB.NORMAL or CB.TRIANGULAR.  Crystal Ball provides a function for every distribution available in the Distribution Gallery.  These functions act just like any other Excel functions, so they can be embedded in other functions, are self-documenting, and can take inputs from other cells.  They can also be copied using the standard Excel Copy and Paste functions.  Many users of Crystal Ball eventually gravitate toward using these functions instead of using the Distribution Gallery.

37. Using Data to Choose a Distribution: Example

38. Fitting Distributions to Data  Crystal Ball provides several formal methods for fitting a distribution to a set of empirical data.  One alternative, which is useful for fitting one distribution at a time, is to use the Fit option in the Distribution Gallery.  If we need to fit a number of distributions, the Batch Fit tool (on the Run menu: Run►Tools►Batch Fit) is the preferred approach.

39. Ensuring Precision in Outputs: Simulation Error  Every time we run a simulation, we are performing an experiment.  With any simulation result, there is some difference, or error, between our estimate and the true value we are after: this is called simulation error.  As with any good experiment, a well planned simulation study requires effort to measure this error.  More specifically, we must ensure that whatever conclusions we draw from the simulation study are not seriously compromised by simulation error.

40. Ensuring Precision in Outputs: Model Error  Simulation error is not the only source of error in our modeling efforts.  It may not even be the most important source of error. All models are abstractions of the real situation they mimic, and for that reason, they differ in their behavior from the real thing.  We can use the term model error to refer to this divergence between the behavior of the model and the behavior of the real thing.  In most practical situations, model error is a much larger problem than simulation error.  Nonetheless, simulation error itself can cause problems in interpreting model results, and for that reason, it should be measured and controlled.

41. Precision versus Accuracy  An estimate based on a larger sample is more precise.  While it is important to ensure an appropriate level of precision in our results, there is a trade-off between the precision of the results and the time it takes to get them.  Thus, an effective modeler does not waste time on long simulation runs unless the additional precision has more value than the next best use of that time.

42. An Experimental Method  The simplest approach to determining the precision of a simulation estimate is to experiment with multiple independent runs.  We must ensure that different random numbers are used each time we run a simulation.  In Crystal Ball, we select Run►Run Preferences, select the Sampling tab, and make sure the Use same sequence of random numbers box is not checked.

43. An Experimental Method (Continued)  We perform five to ten simulations using this run length and compare the estimates of the outcome measure.  If they are too far apart for our purposes, we have not achieved sufficient precision in our estimates, and so we cannot rely on any single run at the current run length.  We therefore increase the run length, possibly to 500 or 1,000.  We make another five or ten runs at this new run length and again compare the results. When the set of results has a sufficiently small range, we have an appropriate sample size.

44. Crystal Ball Tip: Setting Run Length  Use independent random seeds.  Select a low run length.  Perform a small number of independent samples and compare results.  If the range of results is too wide, increase the run length and repeat the process from the start.  Stop when the range of results among independent samples is small enough, given the precision needed in the problem.

45. Simulation Error in a Decision Context  Analysts devote excessive effort to ensuring that individual simulation runs are highly precise.  But the ultimate goal of a simulation study is usually not to estimate a single number.  The broader goal is to provide help in making a decision.  The ultimate test of our efforts is whether we have made a good decision, not whether our simulation results are highly precise.

46. Forecast Charts  The Forecast Preferences window has four tabs whose functions are summarized below.  Forecast Window: select the view of the data, determine whether it will be displayed automatically while running or when stopped, and fit a distribution to the results  Precision: set the parameters for Precision Control  Filter: include or exclude specified values  Auto Extract: automatically display the mean or other statistics on the spreadsheet.

47. The Forecast Preferences Window

48. Chart Preferences  The Chart Preferences window has three tabs whose functions are summarized below.  General: set chart title, number of bins for histogram, and special effects  Chart Type: set chart type and marker lines, such as the mean  Axis: set axis label, scale, and format

49. The Chart Preferences Window

50. Certainty Sliders  Certainty sliders are the small black triangles at the extreme right and left of the histogram.  These triangles can be moved to select a range of outcomes.  Crystal Ball displays the percentage of trials that lie in the highlighted range in the Certainty window at the bottom of the chart.  As an alternative, we can enter numbers directly in the boxes at the bottom of the window; when we press Enter, Crystal Ball uses these limits to calculate the Certainty value.

51. Probability  Because the histogram is only a summary of the underlying data, the required estimate can be quite rough.  A better alternative is to define a Forecast cell in the model that takes on the value 1 when the event in question occurs (such as NPV > 0) and 0 otherwise.  When the Forecast window for this cell is displayed, it shows the probability estimate without any error due to summarizing the data.

52. The Statistics View  More detailed information on the outcome distribution can be found by choosing View►Statistics in the Forecast window.  This table shows the number of Trials, the Mean, and a host of other summary measures.  The most useful of these are the Minimum, Maximum, and the Mean Std. Error.

53. The Statistics View: An Example

54. Percentiles View  This table reports the percentiles of the outcome distribution.  The default is to show every 10th percentile.  For other percentiles, we can use the Crystal Ball function CB.GETFOREPERCENTFN, which returns the given percentile value of a Forecast cell.

55. Percentiles View: An Example

56. Extract Data and Create Report  In some circumstances, it is useful to work directly with the simulation data itself and not with the various Crystal Ball Forecast windows.  The option Analyze►Extract Data allows the user to create a spreadsheet containing the raw results for each Forecast cell.  This is particularly useful if custom graphs are desired.  A related option is Analyze►Create Report, which allows the user to create a spreadsheet with both the input assumptions and the outputs.  This is a handy means of documenting multiple runs of the same model.

57. Crystal Ball Tip: Entering Distributions  There are four ways to view and record the outputs of Forecast cells in Crystal Ball.  Forecast display: When defining a Forecast cell, the Forecast Window tab offers the option to show the Forecast window during the simulation run or at the end. To run simulations at the fastest possible speed do not check Show automatically.  Analyze Forecast Charts: If one or more Forecast windows are not displayed after a simulation run, they can be opened by choosing Analyze►Forecast Charts.  Auto extract: The Auto Extract tab on the Define Forecast window offers the option to record simulation results directly on the spreadsheet. One can choose to record the forecast name, the number of trials, and any of 12 statistics (mean, MSE, and so on).  GETFORESTATFN: The function CB.GETFORESTATFN can also be used to record any statistic of a Forecast cell directly in the spreadsheet. This approach is somewhat more transparent than the Auto Extract option and is therefore preferred.

58. *When not to simulate  Occasionally we may go to the trouble of conducting a simulation only to discover that the effort could have been avoided.  However, in complex models with many uncertain parameters, it is often difficult to determine whether simulation can be avoided.  Moreover, we do not often know in advance exactly which outputs we want to analyze.  Thus, unless our model is particularly simple and we suspect that linearity holds, simulation remains our general purpose tool for analyzing uncertain situations.

59. Summary  Simulation shows us how uncertainty in the inputs influences the outputs of our analysis.  Like optimization, simulation can be seen as a sophisticated form of sensitivity analysis.  In Excel, simulation can be carried out conveniently using Crystal Ball, which provides all the probability models needed to express the uncertainties in our assumptions, automates the repetitive process of sampling from these distributions, and provides extensive methods for displaying and analyzing the results.  Simulation is a powerful tool when used appropriately, but it should never be used before an appropriate sensitivity analysis is carried out on a deterministic version of the model.  What-if analysis, involving use of Data Sensitivity and Tornado Charts, uncovers those input parameters that have the biggest impact on the outcomes.  These should be the focus of any uncertainty analysis.

60. Summary  Every simulation analysis involves four major activities: selecting uncertain parameters selecting probability distributions ensuring precision in the outcomes interpreting outcome distributions  While simulation is more sophisticated than simple spreadsheet modeling, it is one of the most widely used of the advanced management science tools.  Often, the biggest challenge with simulation is translating the results into a form that managers can understand and act upon.

61. Copyright 2008 John Wiley & Sons, Inc. All rights reserved. Reproduction or translation of this work beyond that permitted in section 117 of the 1976 United States Copyright Act without express permission of the copyright owner is unlawful. Request for further information should be addressed to the Permissions Department, John Wiley & Sons, Inc. The purchaser may make back-up copies for his/her own use only and not for distribution or resale. The Publisher assumes no responsibility for errors, omissions, or damages caused by the use of these programs or from the use of the information herein.