Lecture 5.11 © 2015 Michael Stuart Design and Analysis of Experiments Lecture Unit structure –Case study: pigment paste –Components of Variance –Implications for design and analysis of comparative experiments 2.Application to Split Units Designs –Unit and Treatment Structure –Expected Mean Squares and F tests 3.Split Units Designs in Blocks –Illustrations Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.12 © 2015 Michael Stuart Case Study Pigment paste is made in batches of 80 drums. Moisture content is a critical factor. To monitor this, each batch is sampled and the moisture content of the sample is measured. Problem: variation in moisture content seems excessive. Requirement: identify source of excessive variation. 3 possible sources of variation: process (batch to batch) variation, sampling variation, measurement error. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.13 © 2015 Michael Stuart ePeP PP Model for variation in moisture content Process variation  Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.14 © 2015 Michael Stuart ePeP eSeS SS PP Model for variation in moisture content Process variation Sampling variation  Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.15 © 2015 Michael Stuart  SS TT ePeP eSeS PP eTeT e = e P + e S + e T Model for variation in moisture content Process variation Sampling variation Testing variation y Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.16 © 2015 Michael Stuart Model for variation in moisture content Basic model: Y =  + e P + e S + e T Components of variance: Design required for study to estimate variance components. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.17 © 2015 Michael Stuart Hierarchical Design for Variance Component Estimation A batch of pigment paste consists of 80 drums of material. –15 batches were available for testing –2 drums were selected at random from each batch and a sample was taken from each drum. –2 tests for moisture content were run on sub samples of each sample. The results follow Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.18 © 2015 Michael Stuart Aside on Nesting and Crossing Hierarchical / Nested Design Samples are nested in Batches Subsamples are nested in Samples Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 5.19 © 2015 Michael Stuart Aside on Nesting and Crossing Two factors are crossed when each level of one factor occurs with every level of the other factor. –usually concerns treatment factors A factor is nested in another factor (the nesting factor) if each level of the nested factor occurs with exactly one level of the nesting factor. –usually concerns unit / plot factors. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Aside on Nesting and Crossing Crossed Design 3 measurement methods being tested by 4 operators Method 1Method 2Method 3 Operator 1O1M1O1M1 O1M2O1M2 O1M3O1M3 Operator 2O2M1O2M1 O2M2O2M2 O2M3O2M3 Operator 3O3M1O3M1 O3M2O3M2 O3M3O3M3 Operator 4O4M1O4M1 O4M2O4M2 O4M3O4M3 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Aside on Nesting and Crossing Crossed Design 3 measurement methods being tested by 4 operators Method 1Method 2Method 3 Operator 1O1M1O1M1 O1M2O1M2 O1M3O1M3 Operator 2O2M1O2M1 O2M2O2M2 O2M3O2M3 Operator 3O3M1O3M1 O3M2O3M2 O3M3O3M3 Operator 4O4M1O4M1 O4M2O4M2 O4M3O4M3 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Aside on Nesting and Crossing Crossed Design 3 measurement methods being tested by 4 operators Method 1Method 2Method 3 Operator 1O1M1O1M1 O1M2O1M2 O1M3O1M3 Operator 2O2M1O2M1 O2M2O2M2 O2M3O2M3 Operator 3O3M1O3M1 O3M2O3M2 O3M3O3M3 Operator 4O4M1O4M1 O4M2O4M2 O4M3O4M3 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Hierachical / Nested Structure Diploma in Statistics Design and Analysis of Experiments Batches Samples Subsamples

Lecture © 2015 Michael Stuart Hierarchical Design for Variance Component Estimation Samples are nested in Batches Subsamples are nested in Samples Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Hierarchical design Results Ref: PigmentPaste.xls Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Pigment Paste ANOVA Minitab model: Batch Sample(Batch) Test(Sample) Source DF SS MS F P Batch Sample(Batch) Test(Batch Sample) ** Error 0 * * Total Minitab model: Batch Sample(Batch) Source DF SS MS F P Batch Sample(Batch) Error Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Expected Mean Squares Variance Components Source Expected Mean Square Batch Sample(Batch) + 2 Error Basis for F tests One way ANOVA, Ref: Lecture Notes 1.2, p.11 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Aside on Expected Mean Squares Each batch mean involves contributions from 4 subsamples, 2 samples, 1 batch Variance = MS(Batch) estimates batch variance that is, 4 x variance of batch means 4 x Variance = Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Estimating Variance Components MS(Batch)= MS(Sample)= MS(Test) = MS(Sample) – MS(Test) = MS(Batch) – MS(Sample)= = MS(Test) =[MS(Sample) – MS(Test)] / 2 =[MS(Batch) – MS(Sample)] / 4 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Estimating Variance Components Classwork: Calculate the variance components estimates; refer to Slide 18 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Estimating Variance Components Variance Components % of Source Var Comp Total StDev Batch Sample Error Total Sampling variation dominates, testing variation is relatively small. Investigate sampling procedure. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Estimating Variance Components Variance Components % of Source Var Comp Total StDev Batch Sample Error Total Sampling variation dominates, testing variation is relatively small. Investigate sampling procedure. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Sampling procedure Standard procedure: select 5 drums from batch at random, sample all levels of each drum using a specially constructed sampling tube thoroughly mix all samples take a sample from the mixture for analysis Actual practice take a sample from a drum for analysis Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Lecture Unit structure –Case study: pigment paste –Components of Variance –Implications for design and analysis 2.Application to Split Unit Designs –Unit and Treatment Structure –Expected Mean Squares and F tests 3.Split Unit Designs in Blocks –Illustrations Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Units Batches Samples Tests Implications for Design and Analysis Factor Process Sampling method Test method ANOVA MS(Process) MS(Batch variation) MS(Sampling method) MS(P x S) MS(Sampling variation) MS(Test method) MS(Interactions) MS(Testing error) Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Another Example Testing drug treatments for pregnant women 22 women, 10 treatment A, 7 treatment B, 5 "control". Placentas examined for "irregularities": 5 locations, 10 slices, on microscope slides, 5 measurements (counts of "irregularities") per slide, 5,500 measurements in all. No significant treatment effect (10 vs 7) Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Drug treatments for pregnant women Experimental Units Placentas Locations Slices Measurements Treatment Factors Treatment ANOVA MS(Treatment) MS(Placentas) MS(Locations) MS(Slices) MS(Measurements DF 2 19 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Yet Another Example Comparing schools on student performance Schools Classes within schools Students within classes Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Lecture Unit structure –Case study: pigment paste –Components of Variance –Implications for design and analysis 2.Application to Split Unit Designs –Unit and Treatment Structure –Expected Mean Squares and F tests 3.Split Unit Designs with Block Structure –Illustrations –Case study randomised blocks analysis split units analysis Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Split units design: Case study Testing water resistance of four wood stains Stains applied to four panels cut from a board Boards are pretreated with one of two treatments. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Whole units, Subunits, Treatments Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Assessing variation Variation between boards due to –chance –Pretreatments? Variation between panels due to –chance –stains? –pretreatment by stain interaction? Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Units Boards Panels Unit / Treatment Structure Diagram Treatments Pretreatment Stain ANOVA MS(Pretreatment) MS(Boards Variation) MS(Stain) MS(Interaction) MS(Panels Variation) Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Results Pretreatment 1Pretreatment 2 Board Panels Stain Stain Stain Stain Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Minitab analysis Minitab model:Pretreat Board(Pretreat) Stain Pretreat * Stain Source DF SS MS F P Pretreat Board(Pretreat) Stain Pretreat*Stain Error Total Classwork: Verify the F-ratios Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Minitab analysis Minitab model:Pretreat Board(Pretreat) Stain Pretreat * Stain Source DF SS MS F P Pretreat Board(Pretreat) Stain Pretreat*Stain Error Total Classwork: Verify the F-ratios Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart The wrong analysis Assuming a fully randomized experiment (as distinct from separate random allocations of treatments to Boards and to Panels) would result in the following: Analysis of Variance for Resistance Source DF SS MS F P Pretreat Stain Pretreat*Stain Error Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Right and Wrong Right: Source DF SS MS F P Pretreat Board(Pretreat) Stain Pretreat*Stain Error Total Wrong: Source DF SS MS F P Pretreat Stain Pretreat*Stain Error Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Right and Wrong Right: Source DF SS MS F P Pretreat Board(Pretreat) Stain Pretreat*Stain Error Total Wrong: Source DF SS MS F P Pretreat Stain Pretreat*Stain Error Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart The wrong analysis SS(Boards)+ SS(Error) (right)= SS(Error) (wrong) = DF(Boards)+DF(Error) (right)=DF(Error) (wrong) 4+12=16 F(Pretreat) has smaller denominator, MS(Error); F(Pretreat) is now highly significant. MS(Error) is increased, F(Stain) is reduced; F(Stain) is now not statistically significant Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Lecture Unit structure –Case study: pigment paste –Components of Variance –Implications for design and analysis 2.Application to Split Unit Designs –Unit and Treatment Structure –Expected Mean Squares and F tests 3.Split Unit Designs with Block Structure –Illustrations –Case study randomised blocks analysis split units analysis Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Justifying the ANOVA Source Expected Mean Square 1 Pretreat Pretreatment effect 2 Board(Pretreat) Stain + Stain effect 4 Pretreat*Stain + interaction effect 5 Error Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Classwork Use the expected mean squares to confirm the relevant actual mean squares (ref. Slide 35) for forming the F ratios for testing relevant hypotheses. "Most industrial experiments are... split plot in their design.“ C. Daniel (1976) p. 175 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Lecture Review of fractional factorials, confounding 2.Unit structure –Case study: pigment paste –Components of Variance –Implications for design and analysis 3.Application to Split Unit Designs –Unit and Treatment Structure –Expected Mean Squares and F tests 4.Split Unit Designs in Blocks –Illustrations Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Example 1Cambridge Grassland Experiment (1931) Original plan: Investigate two new grassland cultivation treatments: grassland “Rejuvenator”R conventional HarrowH by comparison with no treatment (Control)C in 6 independently randomised blocks of 3 adjacent plots each. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Recall typical agricultural experimental layouts Broadbalk, Rothamsted  Rothamsted Research Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment layout Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment Subsequent addition: investigate 3 fertilisers Farmyard manure F Straw S Artificial fertiliser A by comparison with no fertiliser (Control)C allocated at random to 4 sub plots within each plot. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Cambridge Grassland Experiment Plot structure 72 subplots nested in 18 whole plots nested in 6 blocks Treatment structure 3 grassland treatments randomly allocated to whole plots within blocks 4 fertilisers randomly allocated to subplots within whole plots least variation in between variation most variation Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Units Blocks Plots Subplots Unit / Treatment Structure Diagram Treatments Cultivation Fertiliser ANOVA MS(Blocks) MS(Cultivation) MS(Plots Error / BxC) MS(Fertiliser) MS(CxF) MS(Subplots Error) Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Results Yield was recorded in pounds (lbs) of green produce from a single cut of each subplot made on June 31, 1931 and are shown in the table below. Analysis: Laboratory 2 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Example 2 Electronic components baked in an oven. Two factors thought to influence life times: oven temperature bake time. Trial settings: Oven Temperature (T), °F, 580, 600, 620, 640, Baking time (B), min,5, 10, 15. To save on costly runs, three components baked together at each temperature, one withdrawn at each of the set times. This plan was replicated 3 times (in 3 blocks). Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Example 2 Results of accelerated life time tests for electronic components Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Classwork What are the whole units? What are the whole unit treatments? What are the sub units? What are the sub unit treatments? What is the unit structure? What is the treatment structure? Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Example 2 What are the whole units? What are the whole unit treatments? What are the sub units? What are the sub unit treatments? bake oven temperatures single components baking times Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Unit and Treatment Structures Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Unit structure 36 components nested in 12 bakes nested in 3 replicates Treatment structure 4 temperatures randomly allocated to bakes within replicates 3 baking times randomly allocated to sub components within bakes Classwork:Produce a Unit/Treatment Structure Diagram least variation in between variation most variation Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Experimental Units Replicates Bakes Components Unit / Treatment Structure Diagram Treatment Factors Temperature Baking time ANOVA MS(Replicates) MS(Temperature) MS(Bakes Error) MS(Baking time) MS(TxB) MS(Components Error) Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Replication (Blocking) as a Random Effects Factor Day to day changes in set-up conditions depend on environmental factors, staff factors, timing of critical tasks, –and more, varying more or less unpredictably Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Experimental factors as fixed effects factors Effects of setting controllable levels of controllable factors such as Temperature, Baking time, regarded as fixed, and separate from experimental unit variation and measurement variation. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Block by Treatment Interaction as Random Error Ref: Laboratory 1, Lecture 4.1 Postgraduate Certificate in Statistics Design and Analysis of Experiments Soybean seed germination rates

Lecture © 2015 Michael Stuart Soybean seed germination rates Graphical analysis Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Test for interaction? Analysis of Variance for Rate, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P Block ** Treatment ** Block*Treatment ** Error 0 * * * Total Compare with: Source DF Seq SS Adj SS Adj MS F P Treatment Block Error Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Model including interaction Failuresequalsoverall mean plus Treatment effect plus Block effect plus Treatment by Block interaction effect plus chance variation No replication implies no measure of chance variation, UNLESS no interaction effect. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart A Randomized Blocks Analysis for Temperature effects Results of accelerated life time tests for electronic components, with oven run means Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart A Randomized Blocks Analysis for Temperature effects Results of accelerated life time tests for electronic components, with oven run means Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart A Randomized Blocks Analysis for Temperature effects Mean results of accelerated life time tests for electronic components for each oven run Minitab model: R T Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Minitab results Analysis of Variance for Lifetime Source DF SS MS F P R T Error Total Conclusions: the effect of changing Temperature is highly statistically significant, blocking appears to have been effective. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Minitab results Minitab model: R T Source DF SS MS F P R T Error Total Minitab model: R T R*T Source DF SS MS F P R * * T * * R*T * * Error * ** ** Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Block by Treatment Interaction Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Diagnostic analysis Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Diagnostic analysis Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Split units model in Minitab R T R*T B T*B R*B R is designated as a Random effect factor R*T serves as error term at Oven/Bake level (R*T*B could serve as error term at Component level) Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Analysis of Variance for Lifetime Source DF SS MS F P R x T R*T B T*B R*B Error Total x Not an exact F-test. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Analysis of Variance for Lifetime Source DF SS MS F P R x T R*T B T*B R*B Error Total x Not an exact F-test. Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Diagnostics Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Diagnostics Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Graphical and numerical summaries Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Graphical and numerical summaries Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Expected Mean Squares and their role in constructing F ratios Classwork: Confirm the values of the F ratios in the Analysis of Variance table (other than F(R)) See Slide 79 Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Consequences of incorporating Replication by Baking time interaction in sub plots error Analysis of Variance for Lifetime Source DF SS MS F P R T R*T B T*B Error Total Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Consequences of incorporating Replication by Baking time interaction in sub plots error Exercise: Confirm the values of the F ratios in the Analysis of Variance table Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Reasons for using split units Adding another factor after the experiment started Some factors require better precision than others Changing one factor is –more difficult –more expensive –more time consuming than changing others Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Minute test –How much did you get out of today's class? –How did you find the pace of today's class? –What single point caused you the most difficulty? –What single change by the lecturer would have most improved this class? Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture © 2015 Michael Stuart Reading Review Lecture Notes 1.2, Analysis of Variance Explained, pp Extra Notes:Introduction to Split Units Design and Analysis EM §7.5 for variance components DCM(§§ for variance components), §14.4 for split units (BHH, §§ ) Postgraduate Certificate in Statistics Design and Analysis of Experiments