Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Design of Experiments

Similar presentations


Presentation on theme: "Introduction to Design of Experiments"— Presentation transcript:

1 Introduction to Design of Experiments
Dr. Lotfi K. Gaafar The American University in Cairo Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller This presentation uses information from Paul A. Keller of QA Publishing, LLC. Lotfi K. Gaafar 2004

2 Overview Controllable factors Input Output Process
Uncontrollable factors Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

3 Designed Experiment Terminology
Response: Mfg: Yield of a Process Service: Customer Satisfaction Controlled Factors: set to predefined levels for DOE Mfg: Furnace Temp., Fill Pressure, Material Moisture Service: Process Design, Follow-up Uncontrollable Factors: factors that cannot be controlled in actual operations, but may be controlled during experimentation. Mfg: Humidity, air pollution Service: Arrival rate, efficiency The response in a designed experiment is the parameter we are observing as the outcome of the experiment. For example, in a manufacturing process, we may be concerned about the density of an injection molded part. We will change a variety of conditions within the process and measure the resulting part density. In a service process, we may seek to measure the impact of process changes on customer satisfaction. The parameters that we vary in the process to achieve changes in the response are known as factors. Generally, we will control these factors by setting them at specific levels for each run, or trial, of the experiment. We will run the experiment at various conditions of each of the factors, so that the effect of each factor on the response can be calculated. In an injection molding process, we may want to investigate the effect of changing furnace temperature, fill pressure and the moisture of the raw material. Even though we cannot generally set the moisture level of the material in normal operations, we can sample and segregate the material into two or more distinct levels for the experiment. Likewise in a service process, we may choose, for the experiment, to test the effect of two different process designs, with and without customer follow-up. The factors that are not generally controlled in your operations are sometimes called subsidiary factors. Taguchi referred to these as noise factors, or the outer array. Examples include ambient temperature, humidity, and vibration. As mentioned, it is preferred to control these factors for the experiment. Normally in a designed experiment, we will randomize the order of the trials to prevent any bias in the estimates. In some case, however, we cannot fully randomize the experimental trials, and instead run the experiments in blocks. Examples of blocking factors include the day of the week, a batch of material, a run of the furnace, an airline flight, and so on. In each of these cases, we may have multiple runs that can be randomized within the block, but these blocking factors cannot be randomized within the entire experiment. There are other factors, sometimes called Casual Factors, that may have an impact on our experimental response, such as temperature, humidity, time of day, and so on. If we think these factors are truly important, we should make them controllable factors for the experiment. If we can’t, or choose not to since it would increase the size of the experiment, we should at least measure them. We can then estimate if they are correlated with the response, which would suggest the need for additional experimental runs to analyze their effect. © 2003 QA Publishing, LLC By Paul A. Keller

4 Designed vs. Traditional Experiments
Traditional: vary one factor at a time Factor Response is deviation from “base” How do you maximize the result? What is Effect of each Factor? How does a Designed Experiment differ from a traditional experiment? In the traditional experiments we learned in grade school, we vary one factor at a time. In the table shown, we start out with each of the factors (cycle time, personalized response and response type) set at the low level of each factor. We measure the response, in this case Customer Satisfaction, and consider that a baseline. We then run a second trial to see the effect of changing Cycle Time. The difference between the response from the baseline (Trial 1) and the observed response for Trial 2 is assumed to be the effect of the factor we varied. In this case, we would assume that raising the Cycle Time from Low to High results in a decrease in Customer Satisfaction of 14 units. Likewise, we can estimate the effect of a Personalized Response by comparing Trials 3 and 1 and the effects of Response Type by comparing Trials 4 and 1. In this way, we estimate the effect of the Personalized Response as a decrease in Customer Satisfaction of 7 units, and the effect of Phone vs. as a decrease of 8 units in Customer Satisfaction Score. Based on these observations, we can maximize Customer Satisfaction by setting the factors as follows: Cycle Time: Low; Personalized Response: No; Response Type: Phone. © 2003 QA Publishing, LLC By Paul A. Keller

5 One factor at a time Ignores effect of Interaction Trial 3 Trial 2
The problem with the traditional one-factor-at-a-time experiment is that it ignores the effect of interactions. The graph shown is from a designed experiment of the same process. You can see that at the high Cycle Time setting (shown by the line labeled ), we observed a Satisfaction Score of 21.0 at the No Personalized Response condition. This is the Trial 2 from the previous slide, and is circled on the graph above. Trial 3 is shown on the line labeled at the Yes Personalized Response condition. When we look at the Cycle Time line, we see that the as we move from No Personalized Response to Yes Personalized Response (left to right along the line), there is very little change in Customer Satisfaction Score. In other words, Personalized Response doesn’t make much difference. However, if we look at the line, we see that the as we move from No Personalized Response to Yes Personalized Response (left to right along the line), there is very LARGE change in Customer Satisfaction Score. This implies that Personalized Response is a significant contributor to the change in Customer Satisfaction Score. The implication is that our estimate of the effect of Personalized Response changes, depending on whether we measure the effects at low Cycle Time or high Cycle Time. This means there is an interaction between Personalized Response and Cycle Time. Trial 3 Trial 2 © 2003 QA Publishing, LLC By Paul A. Keller

6 Implications of Interaction
We may think a factor is unimportant if we don’t vary other factors at the same time. We may improve the process, but it only works if other factors remain constant. We may be able to reduce the effect of a factor by minimizing variation of another. When interactions are ignored, we may see haphazard results from our improvement efforts: We may think a factor is unimportant if we don’t vary other factors at the same time, as shown in the example. We may improve the process, but it only works if other factors remain constant. If we don’t vary the right factors, we may think we’ve made an improvement, but then it ‘goes away’. This can happen when there is another factor present that we failed to consider. We may be able to reduce the effect of a factor by minimizing variation of another. This is the Taguchi approach to Robust Design, and can be seen by looking at the prior example in a slightly different way. Since the effect of Pressure changes were negligible at low Temperature, if we could keep Temperature near its low setting then changes in Pressure wouldn’t make much difference. © 2003 QA Publishing, LLC By Paul A. Keller

7 Designed Experiments Vs. Historical Data
Designed to detect specific factors and interactions (orthogonal) Relatively short period of time Casual Factors observed and/or controlled Recorded anomalies Historical May be incapable of detecting interactions May lack range to detect factor significance Unrecognized biases Changing environment We are often tempted to use historical data with Multiple Regression analysis to look for patterns or significance of process factors. This so called data mining has some usefulness, but also lacks many of the attributes that allow designed experiments to more efficiently estimate parameter effects with less data. Designed Experiments estimate parameter effects with less data using an orthogonal array of data, designed to detect specific factors and their interactions. The data is collected over relatively small period of time, allowing the experimenters to control the conditions under which the data is collected. Casual factors such as environmental conditions, personnel, and so on are observed or controlled, with anomalies recorded. Historical, or happenstance, data often is incapable of detecting interactions. The effects of interactions can only be estimated when the data includes the necessary combinations of factors, randomized to remove bias from main effects. The data may not include sufficient variation in each factor or interaction to statistically estimate a significance of the factor or interaction. The uncontrolled nature of the data collection may allow other factors (often unrecorded) to contribute to noise in the data that may cloud the effects of each factor. Since the data is not run in random order, it is possible for unrecognized factors that vary over time to bias the results. © 2003 QA Publishing, LLC By Paul A. Keller

8 DOE: Objectives Determine influential variables (factors)
Determine where to set influential factors to optimize response Determine where to set influential factors to minimize response variability Determine where to set influential factors to minimize the effect of the uncontrollable factors Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

9 DOE: Applications in Process Development
Improve process yield Reduce variability Reduce development time Reduce overall costs Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

10 DOE: Applications in Design
Evaluate and compare alternatives Evaluate material alternatives Product robustness Determine key design parameters Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

11 DOE: Basic Principles Replication Blocking Randomization
Error estimation Accuracy Blocking Unimportant significant factor Precision Randomization Independence Even out uncontrollable factors Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

12 DOE Steps Problem statement Choice of factors, levels, and ranges
Choice of response variable(s) Choice of experimental design Performing the experiment Statistical analysis Conclusions and recommendations Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

13 Resource Allocation Don’t commit all resources to one design
Start with Screening design Only 25% of resources on any one experiment Learn from each design What did you do wrong? Excluded factors, wrong conditions, etc. What to do next? Sometimes next stage of improvement isn’t worth the cost of another experiment When planning resources for experimental design, don’t commit all your resources to one design. A good rule of thumb is that no more than 20% of your total resources should be expended on any one experiment. The reason is that each successive experiment will provide information that will be confirmed or expanded on in subsequent experiments. The results of each experiment will provide information on what went wrong, and what to do next. We may have missed some critical factors, or perhaps did not vary factors sufficiently, so need to run a subsequent experiment to collect more information. The best, and most common approach, is to begin with an effective screening design, which will provide information on key factors and the two-factor interactions between these factors. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

14 Selecting Factors For each response, brainstorm likely factors
For screening, if more than 5-7 factors: Reduce factor list through ranking Nominal Group Technique, Prioritization Matrix Hold some factors constant ex: raw material type/supplier Factors are selected for the designed experiment by brainstorming among the team members. This will typically result in a long list of potential factors. For an effective yet small screening design, we’d like to limit the design to five or seven key factors. Even though this may seem “impossible” for your process, it is often the best practice. If cost and time are of little concern, then add more factors as necessary. But when cost or time is limited, we can reduce the number of factors using the Nominal Group Technique or Prioritization Matrix. Alternatively, we could decide to hold some factors constant, effectively excluding them from the analysis. Factors that are neither held constant nor included are potential lurking factors. © 2003 QA Publishing, LLC By Paul A. Keller

15 Selecting Factor Level Values
Spanning entire region likely to yield the most understanding. If factor's levels are close, measured effect may be statistically insignificant Moving off current operating points presents a risk. Probing techniques: Response Surface Analysis Evolutionary Operation (EVOP): converge on best solution When we define the levels for each factor, we want to span the region of interest. It’s helpful to think of the expected variation we are likely to see for the factor during normal operations, but sometimes this results in factor levels being too close to measure an effect. For example, if we think temperature typically only varies from 70 to 80 degrees, we may not see much of an effect due to temperature over that ten degree difference. It’s generally better to think of “worse case scenarios”, where the factor may vary considerably more. Generally, the wider the difference between the factor levels, the easier the effect will be to measure. When we start moving far away from normal operating conditions, we can enter unknown terrain that can even be hazardous for some processes. In this case, we might be better to keep the factor levels at reasonable values, and if the factor is significant, perform additional experiments using the Response Surface or Evolutionary Operation techniques to find optimal factor levels. © 2003 QA Publishing, LLC By Paul A. Keller

16 Effects of Aliasing: Confounding
Aliased parameters are CONFOUNDED Cannot be estimated independently of one another Estimates are linear combination of confounded parameters Aliasing creates other confounded pairs If ABC = D, then A = BCD; B = ACD; C = ABD; AB = CD; AC = BD; AD = BC; The effect of aliasing is that the aliased parameters are CONFOUNDED with one another. This implies that the parameters cannot be estimated independently of one another. For example, if Factor D is aliased with the ABC Interaction, then when we estimate the effect of Factor D, we cannot be sure whether the effect is due to Factor D, the ABC Interaction, or a linear combination of D and ABC. The intended aliasing also creates some unintended confounding between all the other possible combinations of the aliased pair, as shown in the above slide. These can be verified in the prior slide by noticing, for example, that the results of multiplying the Factor A and Factor B columns provide the same result for all rows as multiplying the Factor C and Factor D columns. This provides evidence that the AB Interaction is confounded with the CD Interaction. © 2003 QA Publishing, LLC By Paul A. Keller

17 Desirable Designs (ref: Box, G. E. P. and N. R. Draper. Robust Designs
Desirable Designs (ref: Box, G.E.P. and N.R. Draper. Robust Designs. Biometrika 62 (1975): ) Provide sufficient distribution of information throughout region of interest Provide model that predicts the response, as close as possible to true response, at all points w/in region of interest Provide ability to detect model lack of fit There are several desirable characteristics of experimental designs: A good design provide a sufficient distribution of information throughout the region of interest. This requires we define the particular segment of the response region we are trying to understand. We may begin the experimental process looking at a wide region, then narrow our focus to a particular region looks interesting. A good design should provide the necessary conditions to develop a model that predicts the response, as close as possible to the true response, at all points within the stated region of interest. This may require three or more levels of particular factors. A desirable design is one that allows the analyst to detect a lack of fit in the model. (continued on next slide) © 2003 QA Publishing, LLC By Paul A. Keller

18 Desirable Designs (cont. ) (ref: Box, G. E. P. and N. R. Draper
Desirable Designs (cont.) (ref: Box, G.E.P. and N.R. Draper. Robust Designs. Biometrika 62 (1975): ) Allow blocking Allow sequential buildup of design Provides internal estimate of error variance Provide simple means of calculating estimates of coefficients Blocking is often required to meet the limitations of data collection, or when we wish to add runs to designs (such as folding). Allow sequential buildup of design, such as in folding, or added axial points. Provides internal estimate of error variance. Provide simple means of calculating estimates of coefficients © 2003 QA Publishing, LLC By Paul A. Keller

19 Design Performance Considerations
Number of Runs minimal best Design Resolution indicates which, if any, interactions can be independently estimated Minimum Detectable Effect Orthogonality & Balance Other: D-Optimal, A-Optimal & G-Optimal There are a number of ways to measure the performance of a design. Many times, we consider the best designs those that can estimate the necessary parameters with the minimal number of runs. Unfortunately, designs of equal runs may not really provide similar performance. Design Resolution provides an indication of the interaction types that can be independently estimated with the design. Recall that when factors and interactions are confounded, they cannot be independently estimated. Other design performance considerations include Minimum Detectable Effect, Orthogonality, Balance, and classes of designs known as D-Optimal, A-Optimal and G-Optimal. © 2003 QA Publishing, LLC By Paul A. Keller

20 Design Resolution Resolution III Resolution IV
Estimates of Main factor effects only; all interactions may be confounded with one another and MF may be confounded with interactions. Resolution IV Estimates of MF are not confounded with 2-factor interactions but may be confounded with higher order interactions. Two factor interactions may be confounded with one another and with higher order interactions. In Resolution III (three) designs, the main effects and the interactions may be confounded with interactions. These are typically useful only as a preliminary screening design. In Resolution IV (four) designs, main factors are not confounded with two-factor interactions, but may be confounded with higher order interactions. Two factor interactions may be confounded with one another and with higher order interactions. © 2003 QA Publishing, LLC By Paul A. Keller

21 Design Resolution (continued)
Resolution V Estimates of MF and 2-factor effects are not confounded with one another but may be confounded with higher-order interactions. Three-factor and higher interactions may be confounded. Resolution VI Estimates of MF and 2-factor effects are not confounded with each other or with 3-factor interactions. Three-factor and higher interactions may be confounded with one another. In Resolution V (five) designs, the estimates of main factor and two-factor interactions are not confounded with one another but may be confounded with higher-order interactions. Three-factor and higher order interactions may be confounded. In Resolution VI (six) designs, the estimates of main factor and two-factor interactions are not confounded with each other or with 3-factor interactions. Three-factor and higher order interactions may be confounded with one another. Resolution V or VI designs provide the most detail needed for first order models. © 2003 QA Publishing, LLC By Paul A. Keller

22 Design Resolution (continued)
Resolution VII Estimates of MF, 2-factor and 3-factor effects are not confounded with one another but may be confounded with higher order interactions. Four-factor and higher interactions may be confounded. Resolution vs. Number of Trials In Resolution VII (seven) designs, estimates of main factor, two-factor and three-factor interactions are not confounded with one another but may be confounded with higher order interactions. Four-factor and higher interactions order may be confounded. As the Resolution increase, the Number of Trials also increases, quite dramatically with the number of factors. © 2003 QA Publishing, LLC By Paul A. Keller

23 Orthogonality Orthogonality refers to the property of a design that assures that all specified parameters may be estimated independently of any other If sum of factors’ columns in standard format equal 0, then design is orthogonal Some writers lump balance as part of orthogonality. Orthogonality refers to the property of a design that assures that all specified parameters may be estimated independently of any other. It’s easy to check for orthogonality: if the sum of the factors' columns in standard format equal 0, then the design is orthogonal. Some writers lump orthogonality with balance, which is different. © 2003 QA Publishing, LLC By Paul A. Keller

24 Balance Balance implies data is properly distributed over design space. uniform physical distribution an equal number of levels of each factor. Some designs sacrifice balance to achieve better distribution of variance or predicted error Ex: Central Composite. Balance may be sacrificed by avoiding extreme combinations of factors Ex: Box-Behnken Instead, Balance implies that the data is properly distributed over the design space. It implies a uniform physical distribution of the data, and an equal number of levels of each factor. Designs do not necessarily need balance to be good designs. Rather, some designs (such as Central Composite designs to be discussed shortly) sacrifice balance to achieve better distribution of the variance or predicted error. Balance may also be sacrificed by avoiding extreme combinations of factors, such as in the example Box-Behnken design that follows. © 2003 QA Publishing, LLC By Paul A. Keller

25 Sample Designs Box Behnken Plackett Burman
2k designs (fractional, confounding, fold over, projection) 3k designs Mixed level designs Latin Squares Central Composite (with axial points) John’s ¾ If the analysis indicates significance of a Noise factor, then it is possible that Main & Noise Factors interact. If interaction is suspected, it is best to run additional experimental trials treating the significant Noise factors as Main Factors and look for interaction between the noise and (original) main factors. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

26 Sample Designs Nested Designs Split Plots Simplex lattice design
Simplex centroid design D- Optimal A- Optimal If the analysis indicates significance of a Noise factor, then it is possible that Main & Noise Factors interact. If interaction is suspected, it is best to run additional experimental trials treating the significant Noise factors as Main Factors and look for interaction between the noise and (original) main factors. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller

27 General Guidelines 1. Good understanding of the problem
Research has shown that one of the key reasons for an industrial experiment to be unsuccessful is due to lack of understanding of the problem itself. The success of any industrially designed experiment will heavily rely on the nature of the problem at hand. The success of the experiment also requires team effort. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm

28 General Guidelines 2. Conduct a thorough and in-depth Brainstorming Session The successful application of DOE requires a mixture of statistical, planning, engineering, communication and teamwork skills. Brainstorming must be treated as an integral part in the design of effective experiments. It is advised to consider the following key issues while conducting brainstorming session: Identification of the process variables, the number of levels of each process variable and other relevant information about the experiment Development of team spirit and positive attitude in order to assure greater participation of the team members. How well does the experiment simulate users’ environment? Who will do what and how? How quickly does the experimenter need to provide the results to management? Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm

29 General Guidelines 3. Select the appropriate response or quality characteristic A response is the performance characteristic of a product which is most critical to customers and often reflects the product quality. It is important to choose and measure an appropriate response for the experiment. The following tips may be useful to engineers in selecting the quality characteristics for industrial experiments. Use responses that can be measured accurately. Use responses which are directly related to the energy transfer associated with the fundamental mechanism of the product or the process. Use responses which are complete, i.e., they should cover the input-output relationship for the product or the process. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm

30 General Guidelines 4. Choose a suitable design for the experiment
The choice of an experimental design will be dependent upon the following factors: Number of factors and interactions (if any) to be studied Complexity of using each design Statistical validity and effectiveness of each design Ease of understanding and implementation Nature of the problem Cost and time constraints Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm

31 General Guidelines 5. Perform a screening experiment
A screening experiment is useful to reduce the number of process variables to a manageable number and thereby reduce the number of experimental runs and costs associated with the entire experimentation process. For example, one may be able to study seven factors using just eight experimental trials. It is advisable not to invest more than 25% of the experimental budget in the first phase of any experimentation such as screening. Having identified the key factors, the interactions among them can be studied using full or fractional factorial experiments (Box et al., 1978). Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm

32 General Guidelines 6. Use Blocking Strategy to increase the efficiency of experimentation Blocking can be used to minimize experimental results being influenced by variations from shift-to-shift, day-to-day or machine-to-machine. The blocks can be batches of different shifts, different machines, raw materials and so on. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm

33 General Guidelines 7. Perform Confirmatory trials/experiments
It is necessary to perform a confirmatory experiment/trial to verify the results from the statistical analysis. Some of the possible causes for not achieving the objective of the experiment are: wrong choice of design for the experiment inappropriate choice of response for the experiment failure to identify the key process variables which affect the response inadequate measurement system for making measurements lack of statistical skills, and so on. Lotfi K. Gaafar 2004 © 2003 QA Publishing, LLC By Paul A. Keller From:http://www.qualityamerica.com/knowledgecente/articles/ANTONYdoe1.htm


Download ppt "Introduction to Design of Experiments"

Similar presentations


Ads by Google