Design and Analysis of Experiments

Design and Analysis of Experiments
Dr. Tai-Yue Wang Department of Industrial and Information Management National Cheng Kung University Tainan, TAIWAN, ROC This is a basic course blah, blah, blah…

Analysis of Variance Dr. Tai-Yue Wang
Department of Industrial and Information Management National Cheng Kung University Tainan, TAIWAN, ROC This is a basic course blah, blah, blah…

Outline(1/2) Example The ANOVA Analysis of Fixed effects Model
Model adequacy Checking Practical Interpretation of results Determining Sample Size

Outline (2/2) Discovering Dispersion Effects
Nonparametric Methods in the ANOVA

What If There Are More Than Two Factor Levels?
The t-test does not directly apply There are lots of practical situations where there are either more than two levels of interest, or there are several factors of simultaneous interest The ANalysis Of VAriance (ANOVA) is the appropriate analysis “engine” for these types of experiments The ANOVA was developed by Fisher in the early 1920s, and initially applied to agricultural experiments Used extensively today for industrial experiments

An Example(1/6)

An Example(2/6) An engineer is interested in investigating the relationship between the RF power setting and the etch rate for this tool. The objective of an experiment like this is to model the relationship between etch rate and RF power, and to specify the power setting that will give a desired target etch rate. The response variable is etch rate.

An Example(3/6) She is interested in a particular gas (C2F6) and gap (0.80 cm), and wants to test four levels of RF power: 160W, 180W, 200W, and 220W. She decided to test five wafers at each level of RF power. The experimenter chooses 4 levels of RF power 160W, 180W, 200W, and 220W The experiment is replicated 5 times – runs made in random order

An Example --Data

An Example – Data Plot Data: Etch-Rate.mtw
Graph -> Boxplot, Scatterplot

An Example--Questions
Does changing the power change the mean etch rate? Is there an optimum level for power? We would like to have an objective way to answer these questions The t-test really doesn’t apply here – more than two factor levels

The Analysis of Variance
In general, there will be a levels of the factor, or a treatments, and n replicates of the experiment, run in random order…a completely randomized design (CRD) N = an total runs We consider the fixed effects case…the random effects case will be discussed later Objective is to test hypotheses about the equality of the a treatment means

The name “analysis of variance” stems from a partitioning of the total variability in the response variable into components that are consistent with a model for the experiment

The basic single-factor ANOVA model is

Models for the Data There are several ways to write a model for the data Mean model Also known as one-way or single-factor ANOVA

Models for the Data Fixed or random factor?
The a treatments could have been specifically chosen by the experimenter. In this case, the results may apply only to the levels considered in the analysis. fixed effect models

Models for the Data The a treatments could be a random sample from a larger population of treatments. In this case, we should be able to extend the conclusion to all treatments in the population.  random effect models

Analysis of the Fixed Effects Model
Recall the single-factor ANOVA for the fixed effect model Define

Hypothesis 

Thus, the equivalent Hypothesis

Analysis of the Fixed Effects Model-Decomposition
Total variability is measured by the total sum of squares: The basic ANOVA partitioning is:

In detail (=0)

Thus SST SSTreatments SSE

A large value of SSTreatments reflects large differences in treatment means A small value of SSTreatments likely indicates no differences in treatment means Formal statistical hypotheses are:

For SSE Recall

Combine a sample variances The above formula is a pooled estimate of the common variance with each a treatment.

Analysis of the Fixed Effects Model-Mean Squares
Define and df df

Analysis of the Fixed Effects Model-Mean Squares
By mathematics, That is, MSE estimates σ2. If there are no differences in treatments means, MSTreatments also estimates σ2.

Analysis of the Fixed Effects Model-Statistical Analysis
Cochran’s Theorem Let Zi be NID(0,1) for i=1,2,…,v and then Q1, q2,…,Qs are independent chi-square random variables withv1, v2, …,vs degrees of freedom, respectively, If and only if

Cochran’s Theorem implies that are independently distributed chi-square random variables Thus, if the null hypothesis is true, the ratio is distributed as F with a-1 and N-a degrees of freedom.

Cochran’s Theorem implies that Are independently distributed chi-square random variables Thus, if the null hypothesis is true, the ratio is distributed as F with a-1 and N-a degrees of freedom.

Analysis of the Fixed Effects Models-- Summary Table
The reference distribution for F0 is the Fa-1, a(n-1) distribution Reject the null hypothesis (equal treatment means) if

Analysis of the Fixed Effects Models-- Example
Recall the example of etch rate (Etch-Rate.mtw), Hypothesis

ANOVA table

Rejection region

P-value P-value

Minitab Data: Etch-Rate.mtw Stat -> ANOVA -> One-Way Graphs: Residual plots -> Check four-in-one

Minitab One-way ANOVA: Etch Rate versus Power Method Null hypothesis All means are equal Alternative hypothesis At least one mean is different Significance level α = 0.05 Equal variances were assumed for the analysis. Factor Information Factor Levels Values Power , 180, 200, 220 Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Power Error Total Model Summary S R-sq R-sq(adj) R-sq(pred) % % % Means Power N Mean StDev % CI (533.88, ) (570.08, ) (608.08, ) (689.68, ) Pooled StDev =

Coding the observations will not change the results Without the assumption of randomization, the ANOVA F test can be viewed as approximation to the randomization test.

Analysis of the Fixed Effects Model- Estimation of the model parameters
Reasonable estimates of the overall mean and the treatment effects for the single-factor model are given by

Analysis of the Fixed Effects Model- Estimation of the model parameters
Confidence interval for μi Confidence interval for μi - μj

Analysis of the Fixed Effects Model- Unbalanced data
For the unbalanced data

A little (very little) humor…

Model Adequacy Checking
Assumptions on the model Errors are normally distributed and independently distributed with mean zero and constant but unknown variances σ2 Define residual where is an estimate of yij

Model Adequacy Checking --Normality
Normal probability plot

Model Adequacy Checking --Normality
Four-in-one

Model Adequacy Checking --Plot of residuals in time sequence
Residuals vs run order

Model Adequacy Checking --Residuals vs fitted

Defects Horn shape Moon type Test for equal variances Bartlett’s test

Test for equal variances Bartlett’s test Stat ->ANOVA -> Test for Equal Variances

Test for equal variances Bartlett’s test Test for Equal Variances: Etch Rate versus Power Method Null hypothesis All variances are equal Alternative hypothesis At least one variance is different Significance level α = 0.05 95% Bonferroni Confidence Intervals for Standard Deviations Power N StDev CI ( , ) ( , ) ( , ) ( , ) Individual confidence level = 98.75% Tests Test Method Statistic P-Value Multiple comparisons — Levene

Test for equal variances Bartlett’s test

Variance-stabilizing transformation Deal with non-constant variance If observations follows Poisson distribution  square root transformation If observations follows Lognormal distribution  Logarithmic transformation

Variance-stabilizing transformation If observations are binominal data Arcsine transformation Other transformation  check the relationship among observations and mean.

Practical Interpretation of Results – Regression Model
The one-way ANOVA model is a regression model and is similar to Stat -> Regression -> regression -> fit regression model

Practical Interpretation of Results – Regression Model
Computer Results Regression Analysis: Etch Rate versus Power Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Regression Power Error Lack-of-Fit Pure Error Total Model Summary S R-sq R-sq(adj) R-sq(pred) % % % Coefficients Term Coef SE Coef T-Value P-Value VIF Constant Power Regression Equation Etch Rate = Power Fits and Diagnostics for Unusual Observations Obs Etch Rate Fit Resid Std Resid R R Large residual

The Regression Model

Practical Interpretation of Results – Comparison of Means
The analysis of variance tests the hypothesis of equal treatment means Assume that residual analysis is satisfactory If that hypothesis is rejected, we don’t know which specific means are different Determining which specific means differ following an ANOVA is called the multiple comparisons problem

There are lots of ways to do this We will use pairwised t-tests on means…sometimes called Fisher’s Least Significant Difference (or Fisher’s LSD) Method

Fisher Pairwise Comparisons Grouping Information Using the Fisher LSD Method and 95% Confidence Power N Mean Grouping A B C D Means that do not share a letter are significantly different.

Practical Interpretation of Results – Graphical Comparison of Means

Practical Interpretation of Results – Contrasts
A linear combination of parameters So the hypothesis becomes

Examples

Testing t-test Contrast average Contrast Variance Test statistic

Testing F-test Test statistic

Testing F-test Reject hypothesis if

Confidence interval

Standardized contrast Standardize the contrasts when more than one contrast is of interest

Unequal sample size Contrast t-statistic

Unequal sample size Contrast sum of squares

Practical Interpretation of Results – Orthogonal Contrasts
Define two contrasts with coefficients {ci} and {di} are orthogonal contrasts if Unbalanced design

Why use orthogonal contrasts ?  For a treatments, the set of a-1 orthogonal contrasts partition the sum of squares due to treatments into a-1 independent single-degree-of freedom tests performed on orthogonal contrasts are independent.

example

Example for contrast

Practical Interpretation of Results – Scheffe’s method for comparing all contrasts
Comparing any and all possible contrasts between treatment means Suppose that a set of m contrasts in the treatment means of interest have been determined. The corresponding contrast in the treatment averages is

The standard error of this contrast is The critical value against which should be compared is If |Cu|>Sα,u , the hypothesis that contrast Γu equals zero is rejected.

The simultaneous confidence intervals with type I error α

Practical Interpretation of Results – example for Scheffe’s method
Contrast of interest The numerical values of these contrasts are

Practical Interpretation of Results – example for Scheffe’s method
Standard error One percent critical values are |C1|>S0.01,1 and |C1|>S0.01,1 , both contrast hypotheses should be rejected.

Practical Interpretation of Results –comparing pairs of treatment means
Tukey’s test Fisher’s Least significant Difference (LSD) method Hsu’s Methods

Practical Interpretation of Results –comparing pairs of treatment means—Computer output
One-way ANOVA: Etch Rate versus Power Source DF SS MS F P Power Error Total S = R-Sq = 92.61% R-Sq(adj) = 91.22% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev (--*---) (--*---) (--*---) (--*---) Pooled StDev = 18.27

Grouping Information Using Tukey Method Power N Mean Grouping A B C D Means that do not share a letter are significantly different. Tukey 95% Simultaneous Confidence Intervals All Pairwise Comparisons among Levels of Power Individual confidence level = 98.87% Power = 160 subtracted from: Power Lower Center Upper (---*--) (--*---) (---*--)

Power = 180 subtracted from: Power Lower Center Upper (---*--) (--*--) Power = 200 subtracted from: Power Lower Center Upper (--*--)

Hsu's MCB (Multiple Comparisons with the Best) Family error rate = 0.05 Critical value = 2.23 Intervals for level mean minus largest of other level means Level Lower Center Upper (---* ) (--* ) (--* ) ( *--)

Grouping Information Using Fisher Method Power N Mean Grouping A B C D Means that do not share a letter are significantly different. Fisher 95% Individual Confidence Intervals All Pairwise Comparisons among Levels of Power Simultaneous confidence level = 81.11% Power = 160 subtracted from: Power Lower Center Upper (--*-) (-*--) (--*-)

Power = 180 subtracted from: Power Lower Center Upper (--*-) (-*-) Power = 200 subtracted from: Power Lower Center Upper (-*--)

Practical Interpretation of Results –comparing treatment means with a control
Donntt’s method Control– the one to be compared Totally a-1 comparisons Dunnett's comparisons with a control Family error rate = 0.05 Individual error rate = Critical value = 2.59 Control = level (220) of Power Intervals for treatment mean minus control mean Level Lower Center Upper ( * ) ( * ) ( * )

Determining Sample size -- Minitab
Stat  Power and sample size One-way ANOVA Number of level ->4 Sample size -> Max. difference  75 Power value  0.9 SD  25 One-way ANOVA Alpha = Assumed standard deviation = 25 Factors: 1 Number of levels: 4 Maximum Sample Target Difference Size Power Actual Power The sample size is for each level.

Dispersion Effects ANOVA for location effects
Different factor level affects variability dispersion effects Example Average and standard deviation are measured for a response variable.

Dispersion Effects ANOVA found no location effects
Transform the standard deviation to Ratio Obser. Control Algorithm 1 2 3 4 5 6

Dispersion Effects ANOVA found dispersion effects
One-way ANOVA: y=ln(s) versus Algorithm Source DF SS MS F P Algorithm Error Total S = R-Sq = 76.71% R-Sq(adj) = 73.22%

Regression and ANOVA Regression Analysis: Etch Rate versus Power
The regression equation is Etch Rate = Power Predictor Coef SE Coef T P Constant Power S = R-Sq = 88.4% R-Sq(adj) = 87.8% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Unusual Observations Obs Power Etch Rate Fit SE Fit Residual St Resid R R denotes an observation with a large standardized residual.

Nonparametric methods in the ANOVA
When normality is invalid Use Kruskal-Wallis test Rank observation yij in ascending order Replace each observation by its rank, Rij In case tie, assign average rank to them test statistic

Kruskal-Wallis test test statistic where

Kruskal-Wallis test If the null hypothesis is rejected.

Kruskal-Wallis Test: Etch Rate versus Power Kruskal-Wallis Test on Etch Rate Power N Median Ave Rank Z Overall H = DF = 3 P = 0.001 H = DF = 3 P = (adjusted for ties)

Design and Analysis of Experiments

Similar presentations

Presentation on theme: "Design and Analysis of Experiments"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Design and Analysis of Experiments

Similar presentations

Presentation on theme: "Design and Analysis of Experiments"— Presentation transcript:

Similar presentations

About project

Feedback