Presentation on theme: "Experimental Design, Response Surface Analysis, and Optimization"— Presentation transcript:
1 Experimental Design, Response Surface Analysis, and Optimization
2 Outline Motivation and Terminology Difficulties in Solving the Basic ProblemExamples of Factors and ResponsesTypes/Examples of Experimental DesignFull Factorial DesignsRandomness of EffectsExample: Full Factorial DesignSituations with Many FactorsResponse Surfaces and MetamodelsRegression AnalysisResponse Surface Methodology
3 Motivation and Terminology Useful when there are many alternatives to consider (e.g., numerous capacity levels of various types, numerous parameters for a proposed inventory system)Two basic types of variables: factors and responsesFactors: input parameters:-controllable or uncontrollable-quantitative or qualitativeResponses: outputs from the simulation model:-uncertain in natureBasic problem: find the best levels (or values of the parameters) in terms of the responsesExperimental Design can tell you which alternatives to simulate so that you obtain the desired information with the least amount of simulation
4 Difficulties in Solving the Basic Problem Multiple ResponsesUncertain Responses
5 Examples of Factors1. Mean interarrival time (uncontrollable, quantitative)2. Mean service time (controllable or uncontrollable, quantitative)3. Number of servers (controllable, quantitative)4. Queuing discipline (controllable, qualitative)5. Reorder point (controllable, quantitative)6. Mean interdemand time (uncontrollable, quantitative)7. Distribution of interdemand time (uncontrollable, qualitative)
6 Examples of Responses 1. Mean daily production rate 2. Mean time in the system for patients3. Mean inventory level4. Number of customers who wait for more than 5 minutes
7 Types/Examples of Experimental Designs Completely Randomized DesignsRandomized Complete Block DesignsNested Factorial DesignsSplit Plot Type DesignsLatin Square Type DesignsFull Factorial DesignsFractional Factorial Designs
8 2k Factorial DesignsSuppose that we have k (k > 2) factors. A 2k factorial design would require that two levels be chosen for each factor, and that n simulation runs (replications) be made at each of the 2k possible factor-level combinations (design points). For 3 factors, this yields a Design Matrix:
9 Design Matrix for 3 Factors 2k Factorial DesignsDesign Matrix for 3 FactorsFactor 1 Factor 2 Factor 3Design Point Level Level Level ResponseO1O2O3O4O5O6O7O8“+” refers to one level of a factor and “-” refers to the other level. Normally, for quantitative factors, the smallest and largest levels for each factor are chosen.
10 2k Factorial Designs: Estimating Main Effects The main effect of factor 1 is the change in the response variable as a result of the change in the level of the factor, averaged over all levels of all of the other factorsIf the effect of some factor depends on the level of another factor, these factors are said to interact.The degree of interaction (two-factor interaction effect) between two factors i and j is defined as half the difference between the average effect of factor i when factor j is at its “+” level ( and all factors other than i and j are held constant) and the average effect of i when j is at its “-” level; for example,
11 Example: Full Factorial (2k) Design Consider a simulation model of reorder point, reorder quantity inventory system. The two decision variables, or factors, to consider are the reorder point (P) and the reorder quantity (Q) for the inventory system. The maximum and minimum allowable values for each are given below:Suppose that the response variable output by the model is the long-run average monthly cost (composed of three components: holding cost, shortage costs, and ordering costs) in thousands of dollars.
12 Example: Full Factorial (2k) Design A 22 factorial design matrix with simulation results (for 10 replications at each design point) might be given by:where a factor level of “-” indicates the minimum possible value for that factor, and a factor level of “+” indicates the maximum possible level; for example, design point 2 has P=40 and Q=15.
13 Example: Full Factorial (2k) Design The response given is the average cost over the 10 replications. Now, the main effects are given by:The interaction effect (ePQ) is given by:Therefore, the average effect of increasing P from 20 to 40 is to increase monthly cost by 1.2, and the average effect of increasing Q from 15 to 50 is to decrease monthly cost by Hence, it would be advisable to set P as low as possible and set Q as high as possible.
14 Example: Full Factorial (2k) Design Also since the interaction effect is positive, it would seem advisable to set P and Q at opposite levels. (Of course, all of the above could be inferred from a cursory analysis of the responses for the various design points.) Note also that the literal interpretation of main effects assumes no interaction effects (pages 669 and 670 of Law and Kelton, 1991).
15 Randomness of EffectsNote that the main and interaction effects computed in the previous examples are just random variables. To determine if the effects are “significant” or real, and not due to random fluctuations, one could compute values for the main and interaction effects 10 times (once for each replication) and form confidence intervals for each of the main effects, and the interactions effect. If the confidence interval contains 0, then the effect is not statistically significant. (Note that statistical significance does not necessarily imply practical significance).
16 Situations with Many Factors When there are many factors to consider, full factorial, or even fractional factorial designs may not be feasible from a computational standpoint.Other types of design (e.g., Plackett-Burman designs or “supersaturated designs”) may be appropriate (Mauro, 1986).Another approach is to reduce the number of factors to consider via “factor screening” techniques, involving, for example, group screening in which a whole group is treated as a factor.
17 Response Surfaces and Metamodels A response surface is a graph of a response variable as a function of the various factors.A metamodel (literally, model of the simulation model), is an algebraic representation of the simulation model, with the factors as independent variables and a response as the dependent variable. It represents an approximation of the response surface.The typical metamodel used in a simulation application is a regression model.
18 Response Surfaces and Metamodels A metamodel through the use of response surface methodology can be used to find optimal values for a set of factors. It can also be used to answer “what if” questions. (Experimentation with a metamodel is typically much less expensive than using a simulation model directly).An experimental design process assumes a particular metamodel, e.g.,
19 Basic Concepts of Regression Analysis Regression is used to determine the “best” functional relation among variables.Suppose that the functional relationship is represented as:E(Y) = f (X1, ..., Xp / B1, ..., BE)where E(Y) is the expected value of the response variable Y; the X1, ..., Xp are factors; and the B1, ..., BE are function parameters; e.g.,E(Y) = B1 + B2 X1 + B3 X2 + B4 X1 X2
20 Basic Concepts of Regression Analysis The observed value for Y, for a given set of X ’s, is assumed to be a random variable, given by:Y = f (X1, ..., Xp/B1, ..., BE) +Where is a random variable with mean equal to 0 and variance The values for B1,...,BE are obtained by minimizing the sum of squares of the deviations.
21 Response Surface Methodology Source: (Fu, 1994)Response surface methodology (RSM) involves a combination of metamodeling (i.e., regression) and sequential procedures (iterative optimization).
22 Response Surface Methodology RSM involves two phases:Fit a linear regression model to some initial data points in the search space (through replications of the simulation model). Estimate a steepest descent direction from the linear regressions model, and a step size to find a new (and better) solution in the search space. Repeat this process until the linear regression model becomes inadequate (indicated by when the slope of the linear response surface is approximately 0; i.e., when the interaction effects become larger than than the main effects).Fit a nonlinear quadratic regression equation to this new area of the search space. Then find the optimum of this equation.
23 Terminology in Experimental Design Source:(Ostle, 1963)Replication - the repetition of the basic experimentTreatment - a specific combination of several factor levelsExperimental Unit - the unit to which a single treatment is applied to one replication of the basic experimentExperimental Error - the failure of two identically treated experimental units to yield identical resultsConfounding - the “mixing up” of two or more factors so that it’s impossible to separate the effects
24 Terminology in Experimental Design Randomization - randomly assigning treatments to experimental units (assures independent distribution of errors)Main Effect (of a factor) - a measure of the change in a response variable to changes in the level of the factor averaged over all levels of all the other factorsInteraction is an additional effect (on the response) due to the combined influence of two or more factors