Six Sigma Greenbelt Training Introduction to DoE (Design of Experiments) Dave Merritt 12/7/16

Learning Objectives Understand what a designed experiment is Determine the basic types of experiments Identify the advantages and disadvantages of each type of experiment. Perform a design of experiment using Minitab Learn the basics to analyzing and interpreting the results from a design of experiments

What is Design of Experiments? An experiment used to identify or screen important factors affecting a process A method to develop empirical models of those processes A process that allows you to gain the maximum amount of information with the minimum number of trials.

Methods of Experimentation Some of the basic types of experiment are: Trial and error One Factor at a time (OFAT) Full factorial Fractional Factorial Others (including EVOP, Response Surface, etc.) “Others” not covered in Green Belt program: EVOP = evolutionary operation. Involves the introduction of small, planned changes in operating systems to determine the direction of improvement. Response surface methodology – used to optimize a process when you suspect there are non-linear responses. Most mechanical processes have linear responses; chemical processes have non-linear responses.

Trial and Error Perhaps the most well known and used method Objective is to provide a quick fix to the problem The “quick fix” method involves randomly and non-randomly making changes, often 2 or more parameters The fix is usually a “band aid” where the symptoms are removed but the cause of the problem goes undetected Some time the process returns to an acceptable state unrelated to our random efforts. Wrong conclusions are made from this. In trial and error, experimentation knowledge is not expanded but hindered Touch on each bullet on this slide. Inject some real life examples of how this is used within FNGP. Example: The case treat process goes out of control and produces non-bond on every part number on the production floor. Our panicked reaction is to change the post cure temperature, the cleaner concentration, the time in the phosphate tank, the number of adhesive dips and even the rubber prep storage conditions. The problem is corrected, but which change actually fixed the problem and what was the root cause??? Since we don’t know, the problem reoccurs. At FNGP, we do a good job of identifying potential root causes (inputs). The problem is that we typically start trying changes to all the inputs without weeding out the critical inputs (hypothesis testing) and without understand how to measure the results. After many trials and errors, we may find something that seems to correct the problem, but we don’t necessarily understand what it was or how great the impact.

Trial and Error Example Problem: Gas mileage for car is 20 mpg. Would like to get > 30 mpg. Solution: Change Brand of gas Change octane rating Drive slower Tune-up car Wash and wax car New tires Remove hood ornament and external radio antenna What if it works? If there was a change, what caused it? What if it doesn’t work? If it works, we are unable to determine what the root cause was due to changing so many variables at once. Very likely that the problem will reoccur.

One Factor at a Time (OFAT) The old dogma in running an experiment is to hold everything constant and vary only factor at a time Is this necessarily true? Can we hold all other variables constant? Imagine if there are a large number of possible factors affecting the response variable How long would OFAT take to identify the critical factors? OFAT is very inefficient, it does not take in consideration the relationship of two or more factors.

One Factor at a Time (OFAT) Example Problem: Gas mileage is 20 mpg How many more combinations would you need to figure out the best configuration of variables? How can you explain the above results? If there were more variables, how long would it take to get a good solution? This assumes that line 1 is the current set-up and then varies each factor, leaving the remaining factors at the current level. All factors have been tested, but the optimum level has not been identified. (see slide 11).

OFAT Misses Interactions Red = MPG 23 65 27 Speed Change only one factor at a time on each trial Step 1: Start at TP-30, O-85, S-55____________Result 25 Step 2: Keep TP-30, O-85, Increase S-65______________Result 23 (worse than 25) Step 3: Keep TP-30, Return S-55, Increase O-91________Result 27 (better than 23, Is that best? We haven’t tried changing TP) Step 4: Increase TP-35, Keep S-55, Return O-85________Result 27 (same as last step right? We’ve tried changing each factor so there’s no reason to continue) What we missed is the interaction. If we changed both Octane and Tire Pressure we get the best result. OFAT misses this relationship. 27 25 32 55 30 91 Tire Pressure Octane 35 85

One Factor at a Time Although OFAT may simplify the analysis of results, the experiment efficiency given up is significant Don’t know the effects of changing one factor while other factors are changing (a reality) Unnecessary experiments may be run Time to find causal factors (factors that affect the response) is significant Perhaps the most critical pitfall of using OFAT experiments is the inability to detect or learn about interactions! Bullet 4 is critical.

DoE - Full Factorial Examine every possible combination of factors at the tested levels. Experimental strategy that allows us to answer most questions completely Determines main effects of the factors manipulated on response variables Also measures the effect of interactions on the response variables Estimate levels to set factors for best results Full factorial’s may represent an extensive amount of time and resources to conduct. Be aware and do not conduct DOE’s just to do them. DOE’s tend to be be costly.

Create a Full Factorial Design – Step 1

Create a Full Factorial Design – Step 2

Create a Full Factorial Design – Step 3

Create a Full Factorial Design – Step 4

DoE Matrix

Analyze Factorial Design Run the different combinations Enter results

Analyze Factorial Design - Continued

Pareto Chart of Effects

Factorial Plots

Main Effects The steeper the slope, the larger the effect. Horizontal line indicates no effect, “flat lined”

Interaction Plots The degree of non-parallelism is key. The greater the intersections, the stronger the effect of the interaction.

Conclusions? This shows a major flaw in OFAT. Depending on what variables you try first and in what order will determine if you find the best combination using OFAT. (All combinations were not tested in OFAT.) 3 factors at 2 levels each 2^3 = 8 trials (2*2*2*)= 8 Typically, we can identify many more factors. 10 factors at 2 levels each – 2^10 = 1024

Full Factorial Designs The limitations of a full factorial experiment are not in theory but in practicality. The resources to run a full factorial design can be significant Full factorial designs are good for investigating a few variables (2-4) Also good for optimizing a process Biggest problem is that the number of runs needed increases exponentially with the number of factors! Bullet #4 is the biggest problem with Full Factorial Designs; definitely discuss this with the class. Number of trials = # of levels raised to the # of factors. This is not suitable for screening many variables. Use to optimize a process after the critical variables have been reduced by hypothesis testing or through a screening DOE.

Full Factorial Design Slide is proving the point made in the previous slide. The number of experiments needed becomes an issue!

Fractional Factorial A fractional factorial looks only at a fraction of all the possible combinations contained in a full factorial If there are many factors being investigated, information can be obtained with less investment It is the fractional factorial that has significantly increased the use of experimentation throughout the world The resources necessary to complete a fractional factorial are manageable, and not much information is given up Interactions can be lost on some fractional factorial designs Most efficient and practical (and economical) way of experimenting.

Fractional Factorials Potential information available from a designed experiment is proportional to the number of trial runs Reducing runs cost us info, usually interactions If knowledge of process is limited, using a fractional factorial design is worthwhile. If knowledge is extensive, consider using a full factorial experiment Limitations: Give up some interactions Fear of statistics Benefits: Economy Speed Less Runs The Benefits section is huge in the reason for why Fractional factorials should be used.

Summary This section introduced the concepts of experimentation OFAT and “trial and error” are not efficient ways of experimenting Can lead to the wrong conclusions Design of experiments is good tool to use for gaining the most knowledge of a process by performing the fewest runs Very efficient method of experimentation Full and fractional factorial designs have trade-offs Important to know when to use what experiment