2 We will use statistics to make good design decisions! We will categorize populations by the mean, standard deviation, and use control charts to determine if a process is in control.We may be forced to run experiments to characterize our system. We will use valid statistical tools such as Linear Regression, DOE, and Robust Design methods to help us make those characterizations.
3 What is DOESometimes we would like to model or optimize a design or process, but the “first principles” analytic approach (Conservation laws, etc.) is too challenging. Empirical (experimental) models are the best next choice. The organized approach using DOE provides a good modeling tool for optimization of designs and processes
4 ExampleSuppose you are trying to get faster cycle time on an injection molder and are looking for the best settings for several temperatures and pressures. The transient heat transfer and transient fluid mechanics (with changing viscosity) problem is challenging. Instead we change the inputs in an organized way, measure the outputs, and develop an experimental model that can be optimized.
5 Here is an email that made my day! I'm working on a project that is nearing data collection. The study is focused on Thumb-Tip force resulting from muscle/tendon force. We're working with cadaveric specimens so this is awesome lab work. Some of the relationships are expected to be nonlinear so we're looking 10 levels of loading for each tendon. We also wish to document first order and possible second order interactions between tendons if they are significant. Last year with the human powered vehicle team we used minitab to create a test procedure for testing power output resulting from chain ring shape, crank length, and rider. There were 3 chain ring shapes, 3 different crank lengths. If possible, in this current study I would like to run a preliminary factorial experiments to determine which interactions are significant before exhaustively testing every combination at every level of loading. If such a method is appropriate it could save us a lot of time. Could you suggest a reference that I might be able to find at the library or on amazon?Hi Dave, I got this on Monday from Sean Hannon. Isn’t this great?
6 DOE ObjectivesAt the end of this module, the participant will be able to :Understand the basic concepts and advantages of designed experimentsUnderstand key terminology used in experimental designUse different techniques to deal with noise in an experimentMake good design decisions!!!
7 Statistical Design of Experiments Chapter 13 of your text. Your text only looks at DOE in the context of Robust Design. This is one of several important uses of DOE:Determining Significant FactorsConstructing a Model of System PerformanceDesign and Analysis of Experiments, Montgomery and Hines is an excellent reference.Look at the Frisbee Thrower from CatapultI will send you the movie of the Frisbee Thrower
8 What Experiments Can Do Characterize a Process/Productdetermines which X’s most affect the Y’sincludes controllable and uncontrollable X’sidentifies critical X’s and noise variablesidentifies those variables that need to be carefully controlledprovides direction for controlling X’s rather than control charting the Y’sOptimize a Process/Productdetermines where the critical X’s should be setdetermines “real” specification limitsprovides direction for “robust” designsFor the Frisbee thrower, we were interested in maximizing the distance.Factors considered were tire speed, angle, and tire pressure
9 Definition of TermsFactor - A controllable experimental variable thought to influence response (example air flow rate, or in the case of the Frisbee thrower: angle, tire speed, tire pressure)Response - The outcome or result; what you are measuring (cycle time to produce one bottle, distance Frisbee goes)Levels - Specific value of the factor (fast flow vs. slow flow, 15 degrees vs. 30 degrees)Interaction - Factors may not be independent, therefore combinations of factors may be important. Note that these interactions can easily be missed in a straight “hold all other variables constant” scientific approach. If you have interaction effects you can NOT find the global optimum using the “OFAT” (one factor at a time) approach!Replicate – performance of the basic experiment
10 How Can DOE Help?run a relatively small number of tests to isolate the most important factors (screening test).determine if any of the factors interact (combined effects are as important as individual effects) and the level of interaction.predict response for any combination of factors using only empirical resultsoptimize using only empirical resultsdetermine the design space for simulation models
11 Trial and ErrorProblem : Current gas mileage is 23 mpg. Want 30 mpg. But what is the best we can get?We might do the following :Change brand of gasChange octane ratingDrive slowerTune-up carWash and wax carBuy new tiresChange tire pressureTerminology :response variable / dependent variable - what is being measured/optimized ex. gas mileagefactor / main effect / independent variable - a controlled variable being studied at 2 or more levels during the experiment ex. brand of gas
12 From the previous example, here were some potential factors to investigate (factors can be either continuous or discrete) :brand of gas (discrete)octane rating (continuous)driving speed (continuous)tires (discrete)tire pressure (continuous)Terminology :factor levels / main effect levels - the values or settings of the variable being manipulatedex. levels for brand of gas are Shell & Texacoex. levels for speed are 55 & 60ex. levels for octane are 85 & 90
13 Problem: Gas mileage is 23 mpg One Factor at a TimeProblem: Gas mileage is 23 mpgBaseline =>Can you explain the results?How many more runs would you need to figure out the best combination of variables?If there were more variables, how many runs would it take to get an optimized solution?What if there is a specific combination of two or more variables that leads to the best mpg?
14 Problem: Gas Mileage is 23 mpg What conclusion do you make now? Full Factorial ExperimentProblem: Gas Mileage is 23 mpgOFAT RunsWhat conclusion do you make now?
15 A treatment combination Full Factorial ExperimentFull Factorial experiment consists of all possible combinations of the levels of the factorsDesign Matrix is the complete specification of the experimental test runs, as seen in the example belowTreatment Combination is a specific test run set-up, consisting of a specific combination of the factor levelsDesign MatrixA treatment combination
16 What makes up an experiment? Response Variable(s)FactorsRandomizationRepetition and Replication
17 Response VariableThe variable that is measured and the object of the characterization or optimization (the Y)Defining the response variable can be difficultOften selected due to ease of measurementSome questions to ask :How will the results be quantified/analyzed?How good is the measurement system?What are the baseline mean and standard deviation?How big of a change do we care about?Are there several response variables of interest?
18 FactorA variable which is controlled or varied in a systematic way during the experiment (the X)Tested at 2 or more levels to observe its effect on the response variable(s) (Ys)Some questions to ask :what are reasonable ranges to ensure a change in Y?knowledge of relationship, i.e. linear or quadratic, etc?Examplesmaterial, supplier, EGR rate, injection timingcan you think of others?
19 Randomization Randomization can be done in several ways : run the treatment combinations in random orderassign experimental units to treatment combinations randomlyan experimental unit is the entity to which a specific treatment combination is appliedAdvantage of randomization is to “average out” the effects of extraneous factors (called noise) that may be present but were not controlled or measured during the experimentspread the effect of the noise across all runsthese extraneous factors (noise) cause unexplained variation in the response variable(s)
20 Repetition and Replication Repetition : Running several samples during one experimental setup (short-term variability)Replication : Repeating the entire experiment (long-term variability)You can use both in the same experimentRepetition and Replication provide an estimate of the experimental errorthis estimate will be used to determine whether observed differences are statistically significant
21 Repetition Pressure : HHHH LLLL HHHH LLLL HHHH LLLL Temp: HHLL HHLL HHLL HHLL HHLL HHLLTest SequenceRepetitionThe first run is a high pressure and a high temp. The second run is also high pressure and high temp.
23 Steps in DOE 1. Statement of the Problem 2. Selection of Response Variable3. Choice of Factors and LevelsFactors are the potential design parameters, such as angle or tire pressureLevels are the range of values for the factors, 15 degrees or 30 degrees4. Choice of Designscreening testsresponse prediction factor interaction5. Perform Experiment6. Data Analysis
24 23 Factorial Design Example Problem Statement: A soft drink bottler is interested in obtaining more uniform heights in the bottles produced by his manufacturing process. The filling machine theoretically fills each bottle to the correct target height, but in practice, there is variation around this target, and the bottler would like to understand better the sources of this variability and eventually reduce it.Choice of Factors: The process engineer can control three variables during the filling process:(A) Percent Carbonation(B) Operating Pressure(C) Line SpeedPressure and speed are easy to control, but the percent carbonation is more difficult to control during actual manufacturing because it varies with product temperature.
25 23 Factorial Design Example Choice of Levels – Each test will be performed for both high and low levelsSelection of Response Variable – Variation of height of liquid from targetChoice of Design – Interaction effectsPerform ExperimentDetermine what tests are required using tabular dataDetermine the order in which the tests should be performed
26 Treatment combinations Determine which experiments should be performedRunA(% C)B(Pressure)C(Speed)Treatment combinations1-(1)2+a3b4ab5c6ac7bc8abc
27 Determine Order of Experiments Decided to run two replicatesRequires 16 testsPut 16 numbers in a hat and draw out the numbers in a random orderAssume that the number 7 is pulled out first, then run test 7 first. (% C low, Pressure high, line speed high)What happens when you draw a 10?Minitab can do this for you automatically!!
28 Stat>DOE>Factorial>Create Factorial Design Full FactorialNumber ofFactorsNumber ofReplicates
29 Operating Pressure (B) Data for the Fill Height Problem (Average deviation from target in tenths of an inch)Operating Pressure (B)25 psiLine speed (C)30 psiLine Speed (C)(A) PercentCarbonation200(bpm)25010-3-11122365
30 Enter Information Ask for random runs Have them type this in in Minitab
31 Be sure to indicate to students that they won’t necessarily get the same values for % carbonation, pressure and line speed because the values are randomized.
32 Data Gathered on Each Run Low, Low, LowDOE RunABCData1 (1)--3,-12 (a)+0,13 (b)-1,04 (ab)2,35 (c)6 (ac)2,17 (bc)1,18 (abc)6,5For this experiment, the data that is gathered is the deviation from the set point in 10ths of an inch
33 Now, instead of having them type the data in, have them open the supplied Minitab worksheet
34 I now know how this one is done! Algebraic Signs for Calculating EffectsI now know how this one is done!TreatmentCombinationFactorial Effect(1)ABABCACBCABC+-ababcacbcabcABC is obtained from just multiplying the value of A*B*C. A is where A is high and B and C are low. Combinations are obtained by multiplying the individual values together. For example for a, AB is negative and AC is negative, but ABC is positive
35 Calculate Contrast for A ProcedureAdd all runs where A is positive.Subtract all runs where A is negative.The difference is called the contrast.
36 Definition of terms in formulas A,B,C are factors.a, b, c are the levels of factors A, B, C. In our example we are only testing high and low, so a = b = c = 2.n = the number of replicates which happens to be 2 in our example
37 The effect is significant if Fo is greater than the value from the table. ANOVA TABLESource of variationSum of squaresDegrees of freedomMeanSquareFoWho cares?A(a-1)Num = DOFADen = DOFMSEB(b-1)Num = DOFBC(c-1)Num = DOFCAB(a-1)(b-1)Num = DOFABAC(a-1)(c-1)Num = DOFACBC(b-1)(c-1)Num = DOFBCABC(a-1)(b-1)(c-1)Num = DOFABCErrorabc(n-1)Totalabcn-1
38 In English, y… is the sum of all data points In English, y… is the sum of all data points. So SST is the (sum of the square of each data point) - (sum of all data points)2/(abcn)
45 A,B,C are all significant, but none of the interaction effects are significant. The Pareto Chart shows the significant effects. Anything to the right of the red line is significant at a (1-a) level. In our case a =0.05, so we are looking for significant effects at the 0.95 or 95% confidence level. So what is significant here?
47 Term Effect Coef SE Coef T P Constant 1.0000 0.1976 5.06 0.001 Estimated Effects and Coefficients for Deviation from Target (coded units)Term Effect Coef SE Coef T PConstant%CarbonationPressureLine Speed%Carbonation*Pressure%Carbonation*Line SpeedPressure*Line Speed%Carb*Press*Line SpeedS = PRESS = 20R-Sq = 93.59% R-Sq(pred) = 74.36% R-Sq(adj) = 87.98%1-P tells us the level at which the effect is significant. For example, the constant is significant at a level or at 99.9%. % Carbonation is significant at an even higher level. Explain that this is a predictive model for deviation from target. From my class at Cummins, they suggested that you remove, one at a time, any item with a P value > than This means that we would start by removing the 3 way effect.We could construct an equation from this to predict Deviation from Target.Deviation = *(%Carbonation) *(Pressure) *(Line Speed) *(%Carbonation*Pressure) *(%Carbonation*Line Speed) *(Pressure*Line Speed) *(%Carbonation*Pressure*Line Speed)We can actually get a better model, which we will discuss in a few slides.
48 >Stat>DOE>Factorial>Factorial Plots Go to set up
50 Practical Application Carbonation has a large effect, so try to control the temperature more preciselyThere is less deviation at low pressure, so use the low pressureAlthough the slower line speed yields slightly less deviation, the process engineers decided to go ahead with the higher line speed - WHY???The engineer accepts the slightly higher deviation, because the higher line speed allows for greater productivity. Do we really look at the coke level to a tenth of an inch? Now let’s see if we can get even better.
51 We can also use Minitab to construct a predictive model!! Estimated Effects and Coefficients for Deviation from Target (coded units)Term Effect Coef SE Coef T PConstant%CarbonationPressureLine Speed%Carbonation*Pressure%Carbonation*Line SpeedPressure*Line Speed%Carb*Press*Line SpeedS = PRESS = 20R-Sq = 93.59% R-Sq(pred) = 74.36% R-Sq(adj) = 87.98%This is a repeat of the model from an earlier slide. We want to start with deleting the 3 way interaction.It is recommended to delete items with P > 0.200
52 >Stat>DOE>Factorial>Analyze Factorial Design Select this arrow to remove the 3-way interaction term.
53 Estimated Effects and Coefficients for Deviation from Target (coded units) Term Effect Coef SE Coef T PConstant%CarbonationPressureLine Speed%Carbonation*Pressure%Carbonation*Line SpeedPressure*Line SpeedS = PRESS =R-Sq = 92.31% R-Sq(pred) = 75.69% R-Sq(adj) = 87.18%Next term to remove
54 Here is the final model from Minitab with the appropriate terms. Estimated Effects and Coefficients for Deviation from Target (coded units)Term Effect Coef SE Coef T PConstant%CarbonationPressureLine Speed%Carbon*PressS = PRESS =R-Sq = 90.71% R-Sq(pred) = 80.33% R-Sq(adj) = 87.33%Notice that this equation is given in coded units. This means that they are using -1 for the low value and +1 for the high value. I will show this on the next page.Deviation from Target = *(%Carbonation) *(Pressure) *(Line Speed) *(%Carbonation*Pressure)
55 Deviation from Target = -0.625 tenths of an inch Estimated Effects and Coefficients for Deviation from Target (coded units).The term coded units means that the equation uses a -1 for the low value and a +1 for the high value of the data.Deviation from Target = *(%Carbonation) *(Pressure) *(Line Speed) *(%Carbonation*Pressure)Let’s check this for %Carbonation = 10, Pressure = 30 psi, and Line Speed = 200 BPM%Carbonation is at its low value, so it gets a -1. Pressure is at its high value, so it gets +1, Line Speed is at its low value, so it gets a -1.Deviation from Target = *(-1) *(1) *(-1) *(-1*-1)Deviation from Target = tenths of an inchHow does this compare with the actual runs at those settings?The runs conducted at those settings give a 0 and -1. So our model isn’t that far off. We can actually get predicted data points. If they need to do this for ME lab, have them come and talk to me and I will happily show them how to do this.
57 The engineer wants the higher line speed and decides to put the target slightly negative. Why?? I chose this model because the slightly negative deviation saves product and the higher line speed maximizes productivity. I don’t think anyone can see a deviation of -.07 tenths of an inch.NEVER GIVE THIS SETTING TO PRODUCTION UNTIL YOU HAVE VERIFIED THE MODEL!
58 Lawn Mower Example VOC Not too Noisy System Spec Noise Level < 75 dbEngine NoiseBlade Assy NoiseCombustion NoiseBlade SpeedMuffler NoiseBlade AreaBlade WidthMuffler VolumeBlade LengthHole AreaWhy do I show this? Suppose that the VOC says that they don’t want a lawn mower to be too noisy. The engineer then determines that means that the noise level<75 db. How can we evaluate whether or not we will meet that. Well, we determine what causes the noise. We can then look at the components. We can probably get a fairly good idea of engine noise from theoretical considerations. We will also probably need to do a DOE to get Blade assembly noise. We will probably have blade speed, blade width, blade length, grass height, and blade to hsg clearance as factors. We will then do a DOE to construct the appropriate model.Grass HeightDiameterBlade to Hsg Clearance
59 Individual Homework (30 pts): Due Monday, Tuesday 19, 20-Mar-2012 Use the Frisbee Thrower file that is supplied to you.Do a DOE analysis of the data to determine the significant factors.Include a Pareto chart, and a 4 way Residual Plots Chart.Remove the appropriate terms in order to get the best model that you can.Show the equation with the appropriate terms.Include a main effects plot and an interaction plot.Calculate the predicted value (distance) at one setting and compare to the experimental values.Discuss the significance of your results. What do these results mean?