1 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

2 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

Objectives Understand basic statistical concepts such as samples and populations, statistics and parameters, and power and sample size. Understand important concepts of hypothesis testing such as the normal distribution and the central limit theorem. Identify and analyze a completely randomized design. 3

Populations and Samples A population is defined by the researcher and is usually the set of all possible outcomes of an experiment or process. A sample is a subset of a population. The researcher would like the sample to be representative of the population. Normally, researchers deal with samples rather than populations. With designed experiments, you will actually create two or more populations. Each test, or run, in the experiment is a sample from one or the other population. 4

Parameters and Statistics Parameters are measurements or attributes of a population. Parameters are usually not known. Statistics are measurements or attributes of a sample. Sample statistics are used to estimate population parameters. For example, you might say that the variance of the sample values is 25. With a designed experiment, you will determine if changing a variable manifests itself in your estimates of the parameters. 5

Types of Estimators You calculate statistics from your sample and use these estimators to characterize the population. There are two types of estimators: point estimators estimate a population characteristic with a single value interval estimators estimate a population characteristic with an interval, or range, of values. 6

Normal Distribution 7

Confidence Intervals If the sample mean falls in the shaded region in the distribution of sample means, the 95% confidence interval constructed will contain the population mean. Notice that μ is captured in this interval. 8

Normal Distribution and Central Limit Theorem The normal distribution is without question the most important of all of the probability distributions. It is the basis for the majority of statistical inferences you make as a researcher. The central limit theorem (CLT) states that the sum of independent and identically distributed random variables is approximately normally distributed. Thus, means calculated from samples of size n from a population can be modeled with a normal distribution and the approximation will improve as the sample size, n, increases. 9

Central Limit Theorem 10

Central Limit Theorem This demonstration illustrates the concepts discussed previously. 11

12

Hypothesis or Significance Testing There are four steps to conduct a hypothesis test. 1.State the null (H 0 ) and alternative (H 1 ) hypotheses. 2.State the significance level (alpha). 3.Collect data, compute sample statistics, and compute the p-value under H 0. 4.Make a decision: If p-value< α, there is sufficient evidence to reject H 0. If p-value≥ α, there is not sufficient evidence to reject H 0. 13

14

1.01 Multiple Choice Poll If a p-value is less than your stated alpha, then which of the following is true? a.There is sufficient evidence to reject the null hypothesis. b.There is not sufficient evidence to reject the null hypothesis. 15

1.01 Multiple Choice Poll – Correct Answer If a p-value is less than your stated alpha, then which of the following in true? a.There is sufficient evidence to reject the null hypothesis. b.There is not sufficient evidence to reject the null hypothesis. 16

Types of Errors and Power You perform a hypothesis test and make a decision, but was the decision correct? The probability of Type I error is denoted by . 17

Types of Errors and Power You perform a hypothesis test and make a decision, but was the decision correct? The probability of Type II error is denoted by . The power of the statistical test is 1 - . 18

How Many Observations? 19 Level of Significance Effect Size Power Variability Required Sample Size

20

1.02 Quiz Given that you hold all other terms constant, how does the required sample size change? Match the number with the appropriate letter: 1. The effect size (the difference that is practically important to you) is increased. 2. The significance level (alpha) is decreased. 3. The desired power is increased. 4. The variability in the process decreases. A: a higher sample size is necessary B: a lower sample size is sufficient 21

1.02 Quiz – Correct Answer Given that you hold all other terms constant, how does the required sample size change? Match the number with the appropriate letter: 1. The effect size (the difference that is practically important to you) is increased. 2. The significance level (alpha) is decreased. 3. The desired power is increased. 4. The variability in the process decreases. A: a higher sample size is necessary B: a lower sample size is sufficient 1-B, 2-A, 3-A, 4-B 22

Hardness Measurement Procedure 23

Data Model and Assumptions 24 Where Y i is the i th sample value of the response variable.  is the overall population mean of the response.  i is the error term, or the i th residual. All variability is included here.

Demonstration Information Preliminary sample information indicates that the hardness difference you need to detect is about 0.45 and the standard deviation is about 0.5. The company would like the power of the test to be at least 0.80 and to test at a level of significance of This information will be used to determine the required sample size. 25

Determining Power and Sample Size This demonstration illustrates the concepts discussed previously. 26

Completely Randomized Design Twelve pieces of metal are randomly selected from a randomly selected lot, and measurements of hardness are recorded. You will conduct a significance test to determine if the average Hardness is not equal to 9.5. H 0 :  = 9.5 H 1 :  ≠ 9.5  =

Analyzing Data from a Completely Randomized Design This demonstration illustrates the concepts discussed previously. 28

Conclusions Based on the analysis performed and additional information about the measurement process, a decision was made to perform additional experiments. The manufacturer would like to know if different drill tips affect the measurement of Hardness. 29

30

31 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

Objectives Understand experimental design and its importance. Outline the experimental process. 32

Why Do You Experiment? To discover the sources of variation in a measured response To collect evidence to support or rebut a theory To determine a consistent result of a system, product design, or process To find conditions that yield a maximum or minimum response in a specified range To compare values of the response at different settings of the controllable variables To build a predictive model 33

What Is Design? Experimental design is the planning phase of data collection. It defines the structure of the experiment to ensure the efficient use of collected data. 34

Observational Studies versus Experimental Design Observational StudiesExperimental Design Process outcomes are measured Analyze data collected under identical, or near identical, circumstances Discern how different circumstances influence a response Ranges of input variables are limited to those that occur during data collection Ranges of input variables are manipulated in a controlled manner The values of the input variables might be correlated There is zero, or near zero, correlation between input variables 35

Experiment: Design and Analysis Design and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple

Experiment: Design and Analysis Design and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple. 37 You begin with design!

The Process of Experimenting 1.Define the purpose of the experiment. 2.Document the specific questions to be answered. 3.Define the population of interest. 4.Determine the need for sampling. 5.Define the data collection protocol. 6.Collect the data. 7.Analyze the data. 38

Define the Data Collection Protocol 1.Describe the process. 2.Identify sources of variability in the process. 3.Determine the “best” design for the experiment. 4.Delineate the experimental procedure. 39

Basic Terms Some basic terms used in experimental design are factor factor level treatment or design point response effect experimental unit run replication 40

41

1.03 Quiz Suppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg. Match the term on the left with the appropriate experimental component on the right. 1. Factor 2. Factor Levels 3. Response 4. Run 5. Experimental Unit 42 A. 3 mg, 5 mg, and 7 mg B. Blood pressure reading C. Drug D. Dosage and corresponding blood pressure reading E. A woman

1.03 Quiz – Correct Answer Suppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg. Match the term on the left with the appropriate experimental component on the right. 1. Factor 2. Factor Levels 3. Response 4. Run 5. Experimental Unit 1-C, 2-A, 3-B, 4-D, 5-E 43 A. 3 mg, 5 mg, and 7 mg B. Blood pressure reading C. Drug D. Dosage and corresponding blood pressure reading E. A woman

Three Basic Principles of Designs Randomization of runs prevents systematic biases from being introduced into the experiment. Randomization refers not only to performing the runs in a random order, but also to resetting the conditions after each run. Blocking is a design technique used to reduce or control variability from nuisance factors. Replication enables the experimenter to obtain an estimate of experimental error. 44

45

Exercise This exercise reinforces the concepts discussed previously. 46

47

1.04 Quiz The output from the exercise is given below. Given  = 0.05, is there sufficient evidence that the average shelf life is greater than 120 days? 48

1.04 Quiz – Correct Answer The output from the exercise is given below. Given  = 0.05, is there sufficient evidence that the average shelf life is greater than 120 days? No. The p-value is , which is greater than Therefore, there is not sufficient evidence to reject H 0 ( μ ≤ 120) in favor of H 1 ( μ > 120). 49

50 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

Objectives Understand basic terminology of experimental design. Follow the experimental design process to set up an experiment and determine the appropriate design. Generate and analyze a completely randomized design. 51

Drill Tip Example 52

Define the Purpose of the Experiment The company wants to determine whether Hardness readings from four types of drill tips are different. 53

Document the Specific Questions Are the average Hardness readings from the four types of drill tips significantly different from each other? 54

Define the Populations of Interest The company is interested in all drill tips of these four types produced by its supplier, the XYZ Corporation

Determine the Need for Sampling It is physically impossible to collect information on all of the drill tips made by the XYZ Corporation; therefore, you need to use a sample of each of the populations of drill tips. A sample of the Purple tips is shown below. Each of the other three populations would be sampled the same way

Define the Data Collection Protocol – Describe the process 57...

Define the Data Collection Protocol – Describe the process 58...

Define the Data Collection Protocol – Describe the process 59...

Define the Data Collection Protocol – Describe the process 60

Define the Data Collection Protocol – Identify Sources of Variability in the Process Identify variability caused by the factor(s) of interest other factors, known as nuisance factors. 61

Define the Data Collection Protocol – Identify Sources of Variability in the Process 62 People TrainingShift Hardness Drill Tips Within Lot Between Lot Materials Drill Press Equipment

Define the Data Collection Protocol – Determine the “Best” Design for the Experiment The experiment has one factor, Tip Type, with four levels, Purple, Green, Orange, and Blue. These factor levels are easy to change from run to run. The experimental unit is a quadrant of a metal sheet. The experimental units are believed to be homogeneous. The completely randomized design is appropriate. 63

Define the Data Collection Protocol – Delineate the Experimental Procedure 64

Experimental Units and Replication 65 versus

Determining Power and Sample Size In preparation for this experiment, you have consulted with industry experts and reviewed previous experiments on drill tips. The following information is determined: The expected standard deviation for each treatment group is approximately 0.2. Alpha=0.05. Power needs to be at least 85%. The estimates for the means for each Tip Type are given by 9.0, 9.1, 9.4, and

Determining Power and Sample Size This demonstration illustrates the concepts discussed previously. 67

68 This demonstration illustrates the concepts discussed previously. Creating a Cause-and-Effect Diagram (Optional)

Generating a Completely Randomized Design This demonstration illustrates the concepts discussed previously. 69

The Design 70 P G P O G B G P B P B O G O

The ANOVA Model 71 Where Y ij is the j th value of the response variable for the i th treatment.  the overall population mean of the response.  i the difference between the population mean of thei th treatment and the overall mean, . This is referred to as the effect of treatment i.  ij the error term, or residual. All variability other than that caused by the treatments is included here.

The ANOVA Hypothesis 72 H 0 : All means equal H 1 : Not all means equal H 0 : µ Purple = µ Green = µ Orange = µ Blue H 1 : not ( µ Purple = µ Green = µ Orange = µ Blue )

Partitioning Variability in ANOVA If the between-group variability is larger than the within-group variability, reject H Between Within Total The variability estimated under H 0. The variability estimated under H 1.

Assumptions of ANOVA 74  independent observations  normally distributed residuals  equal variances for each group Comparing Populations AB C D 4

75

1.05 Multiple Choice Poll What is the test statistic for an ANOVA? a.t b.z c.F d.W 76

1.05 Multiple Choice Poll – Correct Answer What is the test statistic for an ANOVA? a.t b.z c.F d.W 77

Analyzing Data from a Completely Randomized Design This demonstration illustrates the concepts discussed previously. 78

Results of the Completely Randomized Design 79 AnticipatedActual Alpha ( α )0.05 Error Standard Deviation Mean for Purple Mean for Green Mean for Orange Mean for Blue Sample Size (n)16 The p-value of the test is You fail to reject H 0.

80

Exercise This exercise reinforces the concepts discussed previously. 81

82

1.06 Poll Is it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?  Yes  No 83

1.06 Poll – Correct Answer Is it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?  Yes  No 84