1 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

2 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

Objectives Understand basic statistical concepts such as samples and populations, statistics and parameters, normal distribution, central limit theorem, and hypothesis testing. Recognize a completely randomized design. 3

Populations and Samples A population is defined by the researcher and is usually the set of all possible outcomes of an experiment or process. A sample is a subset of a population. The researcher would like the sample to be representative of the population. Normally, you as researchers deal with samples rather than populations. With designed experiments, you will actually create two or more populations. Each test, or run, in the experiment is a sample. 4

Statistics and Parameters Statistics are measurements or attributes about a sample. For example, you might say that the variance of the sample values is 25. Parameters are measurements or attributes about a population. Parameters are usually not known and are estimated by sample statistics. With a designed experiment, you will determine if changing a variable manifests itself in the parameters you estimate. 5

Normal Distribution 6

Normal Distribution and Central Limit Theorem The normal distribution is without question the most important of all of the probability distributions. It is the basis for the majority of statistical inferences you make as a researcher. The central limit theorem (CLT) states that the sum of independent and identically distributed random variables is approximately normally distributed. Thus, means calculated from samples of size n from a population can be modeled with a normal distribution and the approximation will improve as the sample size, n, increases. 7

Central Limit Theorem 8

This demonstration illustrates the concepts discussed previously. 9

10

Hypothesis or Significance Testing A hypothesis test is used to assess the evidence provided by the data in favor of some claim about the population. There are four steps to conduct a hypothesis test. 1. State the null (H 0 ) and alternative (H a ) hypotheses. 2. State alpha (significance level). 3. Collect data, compute sample statistics, and compute the p-value under H Make decision: If p-value<α, there is sufficient evidence to reject H 0. If p-value≥α, there is not sufficient evidence to reject H 0. You can never prove the null hypothesis. You can only state whether or not you have evidence to reject that hypothesis. 11

12

1.01 Multiple Choice Poll If a p-value is less than your stated alpha, then which of the following is true? a.There is sufficient evidence to reject the null hypothesis. b.There is not sufficient evidence to reject the null hypothesis. 13

1.01 Multiple Choice Poll – Correct Answer If a p-value is less than your stated alpha, then which of the following in true? a.There is sufficient evidence to reject the null hypothesis. b.There is not sufficient evidence to reject the null hypothesis. 14

Types of Errors and Power You perform a hypothesis test and make a decision, but was the decision correct? The probability of Type I error is denoted by . 15

Types of Errors and Power You perform a hypothesis test and make a decision, but was the decision correct? The probability of Type II error is denoted by . The power of the statistical test is 1 - . 16

How Many Observations? 17 Level of Significance Effect Size Power Variability Required Sample Size

18

1.02 Quiz Given that you hold all other terms constant, how does the required sample size change? Match the number with the appropriate letter: 1. The effect size (the difference that is practically important to you) is increased. 2. The significance level (alpha) is decreased. 3. The desired power is increased. 4. The variability in the process decreases. A: a higher sample size is necessary B: a lower sample size is sufficient 19

1.02 Quiz – Correct Answer Given that you hold all other terms constant, how does the required sample size change? Match the number with the appropriate letter: 1. The effect size (the difference that is practically important to you) is increased. 2. The significance level (alpha) is decreased. 3. The desired power is increased. 4. The variability in the process decreases. A: a higher sample size is necessary B: a lower sample size is sufficient 1-B, 2-A, 3-A, 4-B 20

Hardness Measurement Procedure 21

Data Model and Assumptions 22 This model represents what you think is true about your population. Specifically, you assume that there is a single population with values that are normally distributed. This normal distribution has a mean µ and standard deviation σ that affects the observed errors, ε i.

Demonstration Information Preliminary sample information indicates that the hardness difference you need to detect is about 0.45 and the standard deviation is about 0.5. The company would like the power of the test to be at least 0.80 and to test at a level of significance of This information will be used to determine the required sample size. 23

Determining Power and Sample Size This demonstration illustrates the concepts discussed previously. 24

Completely Randomized Design Twelve sheets are randomly selected from a randomly selected lot, and measurements of hardness are recorded and stored in Hardness.jmp. Conduct a significance test to determine if the average Hardness is not equal to

Analyzing Data from a Completely Randomized Design This demonstration illustrates the concepts discussed previously. 26

Conclusions Based on the analysis performed and additional information about the measurement process, a decision was made to perform additional experiments. The manufacturer would like to know if different drill tips affect the measurement of Hardness. 27

28

29 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

Objectives Understand experimental design and its importance. Outline the experimental process. 30

Why Do You Experiment? To discover the sources of variation in a measured response To collect evidence to support or rebut a theory To determine a consistent result of a system, product design, or process To find conditions that yield a maximum or minimum response in a specified range To compare values of the response at different settings of the controllable variables To build a predictive model 31

What Is Design? Experimental design is the planning phase of data collection. It defines the structure of the experiment to ensure the efficient use of collected data. 32

Ad-Hoc Analysis versus Experimental Design Ad-Hoc RegressionExperimental Design Process outcomes are measured Objective of the process is to make all units identical Objective of the experimental design is to discern which input variables influence a response The ranges of input values are limited The ranges of the input variables are purposefully manipulated The values of the input variables might be correlated There is zero, or near zero, correlation between input variables 33

Experiment: Design and Analysis Design and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple

Experiment: Design and Analysis Design and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple. 35 You begin with design!

The Process of Experimenting 1.Define the purpose of the experiment. 2.Document the specific questions to be answered. 3.Define the population of interest. 4.Determine the need for sampling. 5.Define the data collection protocol. 6.Collect the data. 7.Analyze the data. 36

Define the Data Collection Protocol 1.Describe the process. 2.Identify sources of variability in the process. 3.Determine the “best” design for the experiment. 4.Delineate the experimental procedure. 37

Identifying Sources of Variability 38 People TrainingShift Hardness Drill Tips Within Lot Between Lot Materials Drill Press Equipment

Basic Terms Some basic terms used in experimental design are factor factor level treatment or design point response effect experimental unit run replication. 39

40

1.03 Quiz Suppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg. Match the term on the left with the appropriate experimental component on the right. 1. Factor 2. Factor Levels 3. Response 4. Run 5. Experimental Unit 41 A. 3 mg, 5 mg, and 7 mg B. Blood pressure reading C. Drug D. Dosage and corresponding blood pressure reading E. A woman

1.03 Quiz – Correct Answer Suppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg. Match the term on the left with the appropriate experimental component on the right. 1. Factor 2. Factor Levels 3. Response 4. Run 5. Experimental Unit 1-C, 2-A, 3-B, 4-D, 5-E 42 A. 3 mg, 5 mg, and 7 mg B. Blood pressure reading C. Drug D. Dosage and corresponding blood pressure reading E. A woman

Three Basic Principles of Designs Randomization of runs prevents systematic biases from being introduced into the experiment. Randomization refers not only to performing the runs in a random order, but also to resetting the conditions after each run. Blocking is a design technique used to reduce or control variability from nuisance factors. Replication enables the experimenter to obtain an estimate of experimental error. 43

44

Exercise This exercise reinforces the concepts discussed previously. 45

46

1.04 Quiz The output from the exercise is given below. Given α=.05, is there sufficient evidence that the average shelf life is greater than 120 days? 47

1.04 Quiz – Correct Answer The output from the exercise is given below. Given α=.05, is there sufficient evidence that the average shelf life is greater than 120 days? No. The p-value is.0544, which is greater than.05. Therefore, there is not sufficient evidence to reject Ho (μ=120) in favor of Ha (μ>120). 48

49 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely Randomized Design

Objectives Understand basic terminology of experimental design. Follow the experimental design process to set up an experiment and determine the appropriate design. Generate and analyze a completely randomized design. 50

Drill Tip Example 51

Define the Purpose of the Experiment The company wants to determine whether Hardness readings from four types of drill tips are different. 52

Document the Specific Questions Are the average Hardness readings from the four types of drill tips significantly different from each other? 53

Define the Populations of Interest The company is interested in all drill tips of these four types produced by its supplier, the XYZ Corporation

Determine the Need for Sampling It is physically impossible to collect information on all of the drill tips made by the XYZ Corporation; therefore, you need to use a sample of each of the populations of drill tips. A sample of the Purple tips is shown below. Each of the other three populations would be sampled the same way

Define the Data Collection Protocol – Describe the Process 56...

Define the Data Collection Protocol – Describe the Process 57...

Define the Data Collection Protocol – Describe the Process 58...

Define the Data Collection Protocol – Describe the Process 59

Define the Data Collection Protocol – Identify the Sources of Variability in the Process Identify variability caused by the factor(s) of interest other factors, known as nuisance factors. 60

Define the Data Collection Protocol – Determine the “Best” Design for the Experiment The experiment has one factor, Tip Type, with four levels, Purple, Green, Orange, and Blue. These factor levels are easy to change from run to run. The experimental unit is a quadrant of a metal sheet. The experimental units are believed to be homogeneous. The completely randomized design is appropriate. 61

Define the Data Collection Protocol – Delineate the Experimental Procedure 62

Experimental Units and Replication 63 VERSUS

Determining Power and Sample Size In preparation for this experiment, you have consulted with industry experts and reviewed previous experiments on drill tips. The following information is determined: The expected standard deviation for each treatment group is approximately 0.2. Alpha=0.05. Power needs to be at least 85%. The estimates for the means for each Tip Type are given by 9.0, 9.1, 9.4, and

Determining Power and Sample Size This demonstration illustrates the concepts discussed previously. 65

66 This demonstration illustrates the concepts discussed previously. Creating a Cause-and-Effect Diagram (Optional)

Generating a Completely Randomized Design This demonstration illustrates the concepts discussed previously. 67

The Design 68 P G P O G B G P B P B O G O

The ANOVA Model 69 Now there is a belief that each population (each value of Tip Type, i) has a unique mean, given by µ + α i. Each population is normally distributed, but there is a shift in the mean for each population.

The ANOVA Hypothesis 70 H o : All means equal H 1 : Not all means are equal H 0 : µ Purple = µ Green = µ Orange = µ Blue

Partitioning Variability in ANOVA If the between-group variability is larger than the within-group variability, reject H Between Within Total The variability estimated under H 0. The variability estimated under H a.

Assumptions of ANOVA 72  independent observations  normally distributed residuals  equal variances for each group Comparing Populations AB C D 4

73

1.05 Multiple Choice Poll What is the test statistic for an ANOVA? a.t b.z c.F d.W 74

1.05 Multiple Choice Poll – Correct Answer What is the test statistic for an ANOVA? a.t b.z c.F d.W 75

Analyzing Data from a Completely Randomized Design This demonstration illustrates the concepts discussed previously. 76

Prospective versus Retrospective Power 77 ProspectiveRetrospective Alpha (α).05 Error Standard Deviation Mean for Orange Mean for Purple Mean for Green Mean for Blue Sample Size (n)16 Power (1-β)

78

Exercise This exercise reinforces the concepts discussed previously. 79

80

1.06 Poll Is it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?  Yes  No 81

1.06 Poll – Correct Answer Is it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?  Yes  No 82