1. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 1 MER301: Engineering Reliability LECTURE 12: Chapter 6: Linear Regression Analysis

2. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 2 Summary of Topics  Linear Regression Analysis Simple Regression Model  Least Squares Estimate of the Coefficients  Standard Error of the Coefficients Precision and Significance of a Regression Model  Precision Standard Error of the Coefficients R 2 - Correlation Coefficient Confidence Limits  Significance T-test on Coefficients Analysis of Variance

3. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 3 Mechanistic and Empirical Models  Mechanistic Models are based on physical, chemical, or engineering science knowledge of the phenomenon Ohm’s Law Gas Law Conservation of Energy  Empirical Models relate Variables based on Observed Data two or more variables are related but the mechanistic model relating the variables is unknown or difficult to apply directly

4. L Berkley Davis Copyright 2009 Examples of Mechanistic Physics Based Models MER301: Engineering Reliability Lecture 12 4

5. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 5 Material Properties are Often Described using Empirical Models  Material Properties Y’s Tensile Properties HCF/LCF Fatigue Crack Growth Rate Fracture Toughness Creep Rupture/Relaxation Hot Corrosion Oxidation Erosion TBC Spallation Thresh-hold Stress Intensity  Material Property x’s Temperature Stress/strain Hold time Specimen type  Section size Specimen orientation Processing variables  Grain size/type Loading method Environment

6. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 6 Regression Analysis  For those cases where there is not a Mechanistic Model of an engineering process, data are used to generate an Empirical Model. A powerful technique for creating such a model doing is called Regression Analysis  In Simple Linear Regression, the Dependent Variable Y is a function of one Independent Variable X  Multiple Linear Regression is used when Y is a function of more than one X  The form of regression models is based on the underlying physics as much as possible

7. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 7 Example 12.1( Table 6.1 from text )  Salt concentration in the runoff from a watershed x is the percentage of a particular watershed that is covered by paved roads y is the salt concentration(mg/l) in surface streams in a particular watershed Excel File (Example 12.1)  Make a scatter plot of the data what is the form of the equation that best describes the relationship between y and x?

8. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 8 Example 12.1(Page 1) Scatter Plot of salt y vs roadway area x…

9. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 9 Simple Linear Regression Model

10. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 10 Least Squares Estimation of the Parameters  Given a set of n observations (x 1,y 1 ),(x 2,y 2 ), …, (x n,y n )  A Simple Linear Regression Model is  Method of Least Squares minimizes the sum of squares of the deviations between data points and regression line

11. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 11 Least Squares  The line fitted by LEAST SQUARES is the one that makes the SUM OF SQUARES of the vertical deviations as small as possible  The RESIDUAL SUM OF SQUARES OF DEVIATIONS from the fitted line is given by

12. L Berkley Davis Copyright 2009 Mean of Y a function of the value of x… MER301: Engineering Reliability Lecture 12 12 Mean and Model Variance in a Least Squares Equation Model Variance same for all x…

13. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 13 Residual Sum of Squares…  Least Squares analysis minimizes the sum of the residuals, called which is used to estimate the Model Variance  The Model Variance is used to estimate the variances of the regression coefficients

14. L Berkley Davis Copyright 2009 the slope of regression line is estimated as MER301: Engineering Reliability Lecture 12 14 Sums of Squares: and - a measure of the spread or variation in x - a measure of combined variations of y and x

15. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 15 Sums of Squares: Definitions

16. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 16 Estimates of Regression Line  Estimate of  Estimate of Regression Line

17. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 17 Equations 6-13 and 6-14 in the text give these formulas

18. L Berkley Davis Copyright 2009 Example 12.1 (page 2)  Excel File (Example 12.1) with data  Plot the data on a Scatter Plot  Compute the Least Squares Regression Coefficients and Least Squares Fit  Compute the Model Variance MER301: Engineering Reliability Lecture 12 18

19. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 19 Example 12.1(Page 3) Scatter Plot of salt y vs roadway area x…

20. L Berkley Davis Copyright 2009 Example 12.1 (pa MER301: Engineering Reliability Lecture 12 20

21. L Berkley Davis Copyright 2009 Example 12.1 (page 5) MER301: Engineering Reliability Lecture 12 21 Standard Error=S

22. L Berkley Davis Copyright 2009 Example 12.1 (page 6) 22

23. L Berkley Davis Copyright 2009 23

24. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 24 Precision and Significance of the Regression…  How “good” is the regression prediction(Precision)? One measure focuses on the uncertainty in the coefficients. The “Standard Errors of the Estimates” quantify this. A second measure is “How much of the total variance does the regression account for?” and “how large are the Residuals?” The Coefficient of Determination” is a measure of this.. A third is the “Confidence Limits on the Mean Response” for the predicted value  Both Hypothesis Tests and Analysis of Variance(ANOVA) can be used to test the Significance of a Regression Analysis

25. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 25 Precision and Significance of the Regression…  Dealing with the Precision first….

26. L Berkley Davis Copyright 2009 Standard Errors of the Regression Coefficients MER301: Engineering Reliability Lecture 12 26

27. L Berkley Davis Copyright 2009 Sum of Squares and the Coefficient of Determination  Sum of Squares  -fraction of total variation in y accounted for by regression equation is total variation in y data set is variation regression accounts for.. is variation regression doesn’t account for… MER301: Engineering Reliability Lecture 12 27

28. L Berkley Davis Copyright 2009 Coefficient of Determination and Correlation Coefficient MER301: Engineering Reliability Lecture 12 28

29. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 29 Correlation Coefficient(Chapter 2,3)  Measure of strength of apparent linear relationship -r near +- 1 shows strong linearity -r near 0 shows lack of linearity It is the square root of the Coefficient of Determination

30. L Berkley Davis Copyright 2009 CI’s on Coefficients MER301: Engineering Reliability Lecture 12 30

31. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 31 Confidence Limits of a Regression

32. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 32 Confidence Limits of a Future Observation

33. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 33 Class Exercise-Example 12.2  Conduct a Least Squares analysis of the data in the table Find the Regression Line Calculate the Residual Sum of Squares and the Coefficient of Determination Plot the Regression Line and the data

34. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 34 Class Exercise-Example 12.2

35. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 35 Precision and Significance of the Regression…  How “good” is the regression prediction(Precision)? One measure focuses on the uncertainty in the coefficients? The “Standard Errors of the Estimates” quantify this. A second measure is “How much of the total variance does the regression account for?” and “how large are the Residuals?” The Coefficient of Determination” is a measure of this.. A third is the “Confidence Limits on the Mean Response” for the predicted value  Both Hypothesis Tests and Analysis of Variance(ANOVA) can be used to test the Significance of a Regression Analysis

36. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 36 Precision and Significance of the Regression…  And now the Significance….

37. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 37 Hypothesis Test/Linear Regression  Hypothesis testing is used to evaluate whether one or both of the regression coefficients are equal to particular values  The test is based on the t-distribution  Reject Ho if

38. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 38 Hypothesis Test/Linear Regression  For the significance of the regression,let  Failure to reject Ho means there is not a linear relationship between Y and X

39. L Berkley Davis Copyright 2009 Significance of the Regression Coefficients MER301: Engineering Reliability Lecture 12 39

40. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 40 Analysis of Variance(ANOVA)

41. L Berkley Davis Copyright 2009 Analysis of Variance(ANOVA) MER301: Engineering Reliability Lecture 12 41

42. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 42 Adequacy of the Model-Residual Analysis  The residuals should be normally distributed with a mean of zero and a run chart of the residuals should exhibit random behavior  If the residuals are truly drawn from a normal population, there will not typically be any outliers  If the residuals do not appear normally distributed this could indicate the linear model is not appropriate. Random behavior could indicate a biased experimental process

43. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 43 Adequacy of the Model-Residual Analysis

44. L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 44 Summary of Topics  Linear Regression Analysis Simple Regression Model  Least Squares Estimate of the Coefficients  Variance of the Regression Precision and Significance of a Regression Model  Precision Standard Error of the Coefficients R 2 - Correlation Coefficient Confidence Limits  Significance T-test on Coefficients Analysis of Variance Adequacy of the Model