1. Six Sigma Greenbelt Training
Introduction to DoE (Design of Experiments)
Dave Merritt
12/7/16

2. Learning Objectives Understand what a designed experiment is
Determine the basic types of experiments
Identify the advantages and disadvantages of each type of experiment.
Perform a design of experiment using Minitab
Learn the basics to analyzing and interpreting the results from a design of experiments

3. What is Design of Experiments?
An experiment used to identify or screen important factors affecting a process
A method to develop empirical models of those processes
A process that allows you to gain the maximum amount of information with the minimum number of trials.

4. Methods of Experimentation
Some of the basic types of experiment are:
Trial and error
One Factor at a time (OFAT)
Full factorial
Fractional Factorial
Others (including EVOP, Response Surface, etc.)
“Others” not covered in Green Belt program:
EVOP = evolutionary operation. Involves the introduction of small, planned changes in operating systems to determine the direction of improvement.
Response surface methodology – used to optimize a process when you suspect there are non-linear responses.
Most mechanical processes have linear responses; chemical processes have non-linear responses.

5. Trial and Error Perhaps the most well known and used method
Objective is to provide a quick fix to the problem
The “quick fix” method involves randomly and non-randomly making changes, often 2 or more parameters
The fix is usually a “band aid” where the symptoms are removed but the cause of the problem goes undetected
Some time the process returns to an acceptable state unrelated to our random efforts.
Wrong conclusions are made from this.
In trial and error, experimentation knowledge is not expanded but hindered
Touch on each bullet on this slide. Inject some real life examples of how this is used within FNGP.
Example:
The case treat process goes out of control and produces non-bond on every part number on the production floor. Our panicked reaction is to change the post cure temperature, the cleaner concentration, the time in the phosphate tank, the number of adhesive dips and even the rubber prep storage conditions. The problem is corrected, but which change actually fixed the problem and what was the root cause??? Since we don’t know, the problem reoccurs.
At FNGP, we do a good job of identifying potential root causes (inputs). The problem is that we typically start trying changes to all the inputs without weeding out the critical inputs (hypothesis testing) and without understand how to measure the results. After many trials and errors, we may find something that seems to correct the problem, but we don’t necessarily understand what it was or how great the impact.

6. Trial and Error Example
Problem: Gas mileage for car is 20 mpg. Would like to get > 30 mpg.
Solution:
Change Brand of gas
Change octane rating
Drive slower
Tune-up car
Wash and wax car
New tires
Remove hood ornament and external radio antenna
What if it works? If there was a change, what caused it?
What if it doesn’t work?
If it works, we are unable to determine what the root cause was due to changing so many variables at once. Very likely that the problem will reoccur.

7. One Factor at a Time (OFAT)
The old dogma in running an experiment is to hold everything constant and vary only factor at a time
Is this necessarily true?
Can we hold all other variables constant?
Imagine if there are a large number of possible factors affecting the response variable
How long would OFAT take to identify the critical factors?
OFAT is very inefficient, it does not take in consideration the relationship of two or more factors.

8. One Factor at a Time (OFAT) Example
Problem: Gas mileage is 20 mpg How many more combinations would you need to figure out the best configuration of variables? How can you explain the above results? If there were more variables, how long would it take to get a good solution?
This assumes that line 1 is the current set-up and then varies each factor, leaving the remaining factors at the current level.
All factors have been tested, but the optimum level has not been identified. (see slide 11).

9. OFAT Misses Interactions
Red = MPG 23 65 27
Speed
Change only one factor at a time on each trial
Step 1: Start at TP-30, O-85, S-55____________Result 25
Step 2: Keep TP-30, O-85, Increase S-65______________Result 23 (worse than 25)
Step 3: Keep TP-30, Return S-55, Increase O-91________Result 27 (better than 23, Is that best? We haven’t tried changing TP)
Step 4: Increase TP-35, Keep S-55, Return O-85________Result 27 (same as last step right? We’ve tried changing each factor so there’s no reason to continue)
What we missed is the interaction. If we changed both Octane and Tire Pressure we get the best result. OFAT misses this relationship. 27 25 32 55 30 91
Tire Pressure
Octane 35 85

10. One Factor at a Time
Although OFAT may simplify the analysis of results, the experiment efficiency given up is significant
Don’t know the effects of changing one factor while other factors are changing (a reality)
Unnecessary experiments may be run
Time to find causal factors (factors that affect the response) is significant
Perhaps the most critical pitfall of using OFAT experiments is the inability to detect or learn about interactions!
Bullet 4 is critical.

11. DoE - Full Factorial
Examine every possible combination of factors at the tested levels.
Experimental strategy that allows us to answer most questions completely
Determines main effects of the factors manipulated on response variables
Also measures the effect of interactions on the response variables
Estimate levels to set factors for best results
Full factorial’s may represent an extensive amount of time and resources to conduct. Be aware and do not conduct DOE’s just to do them. DOE’s tend to be be costly.

12. Create a Full Factorial Design – Step 1

13. Create a Full Factorial Design – Step 2

14. Create a Full Factorial Design – Step 3

15. Create a Full Factorial Design – Step 4

16. DoE Matrix

17. Analyze Factorial Design
Run the different combinations
Enter results

18. Analyze Factorial Design - Continued

19. Pareto Chart of Effects

20. Factorial Plots

21. Main Effects The steeper the slope, the larger the effect.
Horizontal line indicates no effect, “flat lined”

22. Interaction Plots
The degree of non-parallelism is key. The greater the intersections, the stronger the effect of the interaction.

23. Conclusions?
This shows a major flaw in OFAT. Depending on what variables you try first and in what order will determine if you find the best combination using OFAT. (All combinations were not tested in OFAT.)
3 factors at 2 levels each
2^3 = 8 trials (2*2*2*)= 8
Typically, we can identify many more factors.
10 factors at 2 levels each – 2^10 = 1024

24. Full Factorial Designs
The limitations of a full factorial experiment are not in theory but in practicality.
The resources to run a full factorial design can be significant
Full factorial designs are good for investigating a few variables (2-4)
Also good for optimizing a process
Biggest problem is that the number of runs needed increases exponentially with the number of factors!
Bullet #4 is the biggest problem with Full Factorial Designs; definitely discuss this with the class.
Number of trials = # of levels raised to the # of factors.
This is not suitable for screening many variables. Use to optimize a process after the critical variables have been reduced by hypothesis testing or through a screening DOE.

25. Full Factorial Design
Slide is proving the point made in the previous slide.
The number of experiments needed becomes an issue!

26. Fractional Factorial
A fractional factorial looks only at a fraction of all the possible combinations contained in a full factorial
If there are many factors being investigated, information can be obtained with less investment
It is the fractional factorial that has significantly increased the use of experimentation throughout the world
The resources necessary to complete a fractional factorial are manageable, and not much information is given up
Interactions can be lost on some fractional factorial designs
Most efficient and practical (and economical) way of experimenting.

27. Fractional Factorials
Potential information available from a designed experiment is proportional to the number of trial runs
Reducing runs cost us info, usually interactions
If knowledge of process is limited, using a fractional factorial design is worthwhile.
If knowledge is extensive, consider using a full factorial experiment
Limitations:
Give up some interactions
Fear of statistics
Benefits:
Economy
Speed
Less Runs
The Benefits section is huge in the reason for why Fractional factorials should be used.

28. Summary This section introduced the concepts of experimentation
OFAT and “trial and error” are not efficient ways of experimenting
Can lead to the wrong conclusions
Design of experiments is good tool to use for gaining the most knowledge of a process by performing the fewest runs
Very efficient method of experimentation
Full and fractional factorial designs have trade-offs
Important to know when to use what experiment