1. New Results about Randomization and Split-Plotting by James M. Lucas 2003 Quality & Productivity Research Conference Yorktown Heights, New York May 21-23, 2003

2. J. M. Lucas and Associates2 Contact Information James M. Lucas J. M. Lucas and Associates 5120 New Kent Road Wilmington, DE 19808 (302) 368-1214 JamesM.Lucas@worldnet.att.net

3. J. M. Lucas and Associates3 Research Team Huey Ju Jeetu Ganju Frank Anbari Peter Goos Malcolm Hazel Derek Webb John Borkowski

4. PRELIMINARIES How do you run Experiments?

5. J. M. Lucas and Associates5 QUESTIONS How many of you are involved with running experiments? How many of you randomize to guard against trends or other unexpected events? If the same level of a factor such as temperature is required on successive runs, how many of you set that factor to a neutral level and reset it?

6. J. M. Lucas and Associates6 ADDITIONAL QUESTIONS How many of you have conducted experiments on the same process on which you have implemented a Quality Control Procedure? What did you find?

7. J. M. Lucas and Associates7 COMPARING RESIDUAL STANDARD DEVIATION FROM AN EXPERIMENT WITH RESIDUAL STANDARD DEVIATION FROM AN IN-CONTROL PROCES MY OBSERVATIONS EXPERIMENTAL STANDARD DEVIATION IS LARGER. 1.5X TO 3X IS COMMON.

8. J. M. Lucas and Associates8 HOW SHOULD EXPERIMENTS BE CONDUCTED? COMPLETE RANDOMIZATION (and the completely randomized design) RANDOMIZED NOT RESET (Also Called Random Run Order (RRO) Experiments) (Often Achieved When Complete Randomization is Assumed) SPLIT PLOT BLOCKING (Especially When There are Hard-to-Change Factors)

9. J. M. Lucas and Associates9 Randomized Not Reset (RNR) Experiments A large fraction (perhaps a large majority) of industrial experiments are Randomized not Reset (RNR) experiments Properties of RNR experiments and a discussion of how experiments should be conducted: L k Factorial Experiments with Hard-to-Change and Easy-to-Change Factors Ju and Lucas, 2002, JQT 34, 411-421[studies one H-T-C factor and uses Random Run Order (RRO) rather than RNR] Factorial Experiments when Factor Levels Are Not Necessarily Reset Webb, Lucas and Borkowski, 2003, JQT, to appear [studies >1 HTC Factor]

10. J. M. Lucas and Associates10 RNR EXPERIMENTS ( Random Run Order Without Resetting Factors) OFTEN USED BY EXPERIMENTERS NEVER EXPLICITLY RECOMMENDED ADVANTAGES Often achieves successful results Can be cost-effective DISADVANTAGES Often can not be detected after experiment is conducted (Ganju and Lucas 99) Biased tests of hypothesis (Ganju and Lucas 97, 02) Can often be improved upon Can miss significant control factors

11. J. M. Lucas and Associates11 Results for Experiments with Hard- to-Change and Easy-to-Change Factors One H-T-C or E-T-C Factor: use split- plot blocking Two H-T-C Factors: may split-plot Three or more H-T-C Factors: consider RNR or Low Cost Options Consider Diccons Rule: Design for the H-T-C Factor

12. J. M. Lucas and Associates12 New Results Joint work with Peter Goos Builds on the Kiefer-Wolfowitz Equivalence Theorem Implications about Computer generated designs (especially when there are Hard-to-Change Factors)

13. J. M. Lucas and Associates13 Kiefer-Wolfowitz Equivalence Theorem is the design probability measure M( ) = XX/n (kxk matrix for a n point design) d(x, ) = x(M( )) -1 x (normalized variance) So called Approximate Theory The following are equivalent: maximizes det M( ) minimizes d(x, ) Max (d(x, ) = k

14. J. M. Lucas and Associates14 Very Important Theorem Helps find Optimum Designs Basis for much computer aided design work Justifies using |XX| Criterion Shows Classical Designs are great Which Response Surface Design is Best Technometrics (1976) 16, 411-417 Computer generated designs not needed for standard situations

15. J. M. Lucas and Associates15 Optimality Criteria Determinant (D-optimality) Maximize |XX| D-efficiency = {|XX/n|/ |X*X*/n*|} 1/k where X* is an optimum n* point design Global (G-optimality) Minimize the maximum variance G-efficiency = k/Max d(x, ) G-efficiency < D-efficiency No bad designs with high G-efficiency

16. J. M. Lucas and Associates16 Computer Generated Design Arrays Different criteria give different n point designs Do not pick a single n Some n values may achieve an excellent design Check other criteria (especially G-) Lucas (1978) Discussion of: D-Optimal Fractions of Three Level Factorial Designs Borkowski (2003) Using A Genetic Algorithm to Generate Small Exact Response Surface Designs

17. J. M. Lucas and Associates17 Equivalence Theorem does not hold for Split-Plot Experiments D- and G- criteria converge to different designs Example: r reps of a 2 3 Factorial (linear terms model) Optimum design depends on d = w 2 / 2 where w is the whole-plot and is the split-plot error For large values of d: D-optimal design has 4 r blocks with I = A = BC G-optimal design has 8r – 2 blocks (Number of observations minus number of split-plot terms)

18. J. M. Lucas and Associates18 Computer Generated Split-Plot Experiments Useful Research Recent publications: Trinca and Gilmour (2001) Multi-stratum Response Surface Designs Technometrics 43: 25- 33 Goos and Vandebroek (2001) Optimal Split-Plot Designs JQT 33: 436-450 Goos and Vandebroek (2003) Outperforming Completely Randomized Designs JQT to appear All use |XX| Criterion

19. J. M. Lucas and Associates19 RELATED SPLIT-PLOT FINDINGS SUPER EFFICIENT EXPERIMENTS (With One or Two Hard-to-Change Factor) SPLIT PLOT BLOCKING GIVES HIGHER PRECISION AND LOWER COSTS THAN COMPLETELY RANDOMIZED EXPERIMENTS

20. J. M. Lucas and Associates20 Design Precision: Calculating Maximum Variance Simplifications for 2 k factorials Sum Variances of individual terms Whole plot terms: w 2 / number blocks + 2 / 2 k Split plot terms: 2 /2 k Completely randomized design has variance: k( w 2 + 2) / 2 k Blocking Observation to achieve Super Efficiency

21. J. M. Lucas and Associates21 2 6-1 with one or two Hard-to-Change Factors Main Effects plus interaction Model 22 Terms = (1 + 6 + 15) Use Resolution V, not VI with I=ABCDE Use four blocks I=A=BCF=ABCF=BCDE=ADEF=DEF Nest Factor B within each A block giving a split-split-plot with 8 Blocks =B 2 =AB 2 =CF 2 =ACF 2 =CDE 2 =ABDEF 2 =BDEF 2 I and A have variance 0 2 /32 + 1 2 /4 + 2 2 /8 B, AB and CF have 0 2 /32 + 2 2 /8 Other terms have variance 0 2 /32 G-efficiency = 22( 0 2 + 1 2 + 2 2 )/(22 0 2 +16 1 2 +20 2 2 ) >1.0 Drop 2 2 terms for one h-t-c factor results

22. J. M. Lucas and Associates22 Observations Does not use Maximum Resolution or Minimum Abberation Similar results for most 2 k factorials

23. Super Efficient Experiments are not always Optimal 2 6-1 Main effects plus 2FI model G-optimum design has 12 blocks when d gets large

24. J. M. Lucas and Associates24 Conclusions Showed K-W Equivalence theorem does not hold for Split-Plot Experiments Discussed Implications Exciting research area Much more to do