L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 1 MER301: Engineering Reliability LECTURE 12: Chapter 6: Linear Regression Analysis

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 2 Summary of Topics  Linear Regression Analysis Simple Regression Model  Least Squares Estimate of the Coefficients  Standard Error of the Coefficients Precision and Significance of a Regression Model  Precision Standard Error of the Coefficients R 2 - Correlation Coefficient Confidence Limits  Significance T-test on Coefficients Analysis of Variance

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 3 Mechanistic and Empirical Models  Mechanistic Models are based on physical, chemical, or engineering science knowledge of the phenomenon Ohm’s Law Gas Law Conservation of Energy  Empirical Models relate Variables based on Observed Data two or more variables are related but the mechanistic model relating the variables is unknown or difficult to apply directly

L Berkley Davis Copyright 2009 Examples of Mechanistic Physics Based Models MER301: Engineering Reliability Lecture 12 4

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 5 Material Properties are Often Described using Empirical Models  Material Properties Y’s Tensile Properties HCF/LCF Fatigue Crack Growth Rate Fracture Toughness Creep Rupture/Relaxation Hot Corrosion Oxidation Erosion TBC Spallation Thresh-hold Stress Intensity  Material Property x’s Temperature Stress/strain Hold time Specimen type  Section size Specimen orientation Processing variables  Grain size/type Loading method Environment

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 6 Regression Analysis  For those cases where there is not a Mechanistic Model of an engineering process, data are used to generate an Empirical Model. A powerful technique for creating such a model doing is called Regression Analysis  In Simple Linear Regression, the Dependent Variable Y is a function of one Independent Variable X  Multiple Linear Regression is used when Y is a function of more than one X  The form of regression models is based on the underlying physics as much as possible

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 7 Example 12.1( Table 6.1 from text )  Salt concentration in the runoff from a watershed x is the percentage of a particular watershed that is covered by paved roads y is the salt concentration(mg/l) in surface streams in a particular watershed Excel File (Example 12.1)  Make a scatter plot of the data what is the form of the equation that best describes the relationship between y and x?

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 8 Example 12.1(Page 1) Scatter Plot of salt y vs roadway area x…

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture 12 9 Simple Linear Regression Model

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Least Squares Estimation of the Parameters  Given a set of n observations (x 1,y 1 ),(x 2,y 2 ), …, (x n,y n )  A Simple Linear Regression Model is  Method of Least Squares minimizes the sum of squares of the deviations between data points and regression line

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Least Squares  The line fitted by LEAST SQUARES is the one that makes the SUM OF SQUARES of the vertical deviations as small as possible  The RESIDUAL SUM OF SQUARES OF DEVIATIONS from the fitted line is given by

L Berkley Davis Copyright 2009 Mean of Y a function of the value of x… MER301: Engineering Reliability Lecture Mean and Model Variance in a Least Squares Equation Model Variance same for all x…

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Residual Sum of Squares…  Least Squares analysis minimizes the sum of the residuals, called which is used to estimate the Model Variance  The Model Variance is used to estimate the variances of the regression coefficients

L Berkley Davis Copyright 2009 the slope of regression line is estimated as MER301: Engineering Reliability Lecture Sums of Squares: and - a measure of the spread or variation in x - a measure of combined variations of y and x

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Sums of Squares: Definitions

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Estimates of Regression Line  Estimate of  Estimate of Regression Line

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Equations 6-13 and 6-14 in the text give these formulas

L Berkley Davis Copyright 2009 Example 12.1 (page 2)  Excel File (Example 12.1) with data  Plot the data on a Scatter Plot  Compute the Least Squares Regression Coefficients and Least Squares Fit  Compute the Model Variance MER301: Engineering Reliability Lecture 12 18

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Example 12.1(Page 3) Scatter Plot of salt y vs roadway area x…

L Berkley Davis Copyright 2009 Example 12.1 (pa MER301: Engineering Reliability Lecture 12 20

L Berkley Davis Copyright 2009 Example 12.1 (page 5) MER301: Engineering Reliability Lecture Standard Error=S

L Berkley Davis Copyright 2009 Example 12.1 (page 6) 22

L Berkley Davis Copyright

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Precision and Significance of the Regression…  How “good” is the regression prediction(Precision)? One measure focuses on the uncertainty in the coefficients. The “Standard Errors of the Estimates” quantify this. A second measure is “How much of the total variance does the regression account for?” and “how large are the Residuals?” The Coefficient of Determination” is a measure of this.. A third is the “Confidence Limits on the Mean Response” for the predicted value  Both Hypothesis Tests and Analysis of Variance(ANOVA) can be used to test the Significance of a Regression Analysis

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Precision and Significance of the Regression…  Dealing with the Precision first….

L Berkley Davis Copyright 2009 Standard Errors of the Regression Coefficients MER301: Engineering Reliability Lecture 12 26

L Berkley Davis Copyright 2009 Sum of Squares and the Coefficient of Determination  Sum of Squares  -fraction of total variation in y accounted for by regression equation is total variation in y data set is variation regression accounts for.. is variation regression doesn’t account for… MER301: Engineering Reliability Lecture 12 27

L Berkley Davis Copyright 2009 Coefficient of Determination and Correlation Coefficient MER301: Engineering Reliability Lecture 12 28

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Correlation Coefficient(Chapter 2,3)  Measure of strength of apparent linear relationship -r near +- 1 shows strong linearity -r near 0 shows lack of linearity It is the square root of the Coefficient of Determination

L Berkley Davis Copyright 2009 CI’s on Coefficients MER301: Engineering Reliability Lecture 12 30

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Confidence Limits of a Regression

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Confidence Limits of a Future Observation

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Class Exercise-Example 12.2  Conduct a Least Squares analysis of the data in the table Find the Regression Line Calculate the Residual Sum of Squares and the Coefficient of Determination Plot the Regression Line and the data

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Class Exercise-Example 12.2

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Precision and Significance of the Regression…  How “good” is the regression prediction(Precision)? One measure focuses on the uncertainty in the coefficients? The “Standard Errors of the Estimates” quantify this. A second measure is “How much of the total variance does the regression account for?” and “how large are the Residuals?” The Coefficient of Determination” is a measure of this.. A third is the “Confidence Limits on the Mean Response” for the predicted value  Both Hypothesis Tests and Analysis of Variance(ANOVA) can be used to test the Significance of a Regression Analysis

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Precision and Significance of the Regression…  And now the Significance….

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Hypothesis Test/Linear Regression  Hypothesis testing is used to evaluate whether one or both of the regression coefficients are equal to particular values  The test is based on the t-distribution  Reject Ho if

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Hypothesis Test/Linear Regression  For the significance of the regression,let  Failure to reject Ho means there is not a linear relationship between Y and X

L Berkley Davis Copyright 2009 Significance of the Regression Coefficients MER301: Engineering Reliability Lecture 12 39

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Analysis of Variance(ANOVA)

L Berkley Davis Copyright 2009 Analysis of Variance(ANOVA) MER301: Engineering Reliability Lecture 12 41

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Adequacy of the Model-Residual Analysis  The residuals should be normally distributed with a mean of zero and a run chart of the residuals should exhibit random behavior  If the residuals are truly drawn from a normal population, there will not typically be any outliers  If the residuals do not appear normally distributed this could indicate the linear model is not appropriate. Random behavior could indicate a biased experimental process

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Adequacy of the Model-Residual Analysis

L Berkley Davis Copyright 2009 MER301: Engineering Reliability Lecture Summary of Topics  Linear Regression Analysis Simple Regression Model  Least Squares Estimate of the Coefficients  Variance of the Regression Precision and Significance of a Regression Model  Precision Standard Error of the Coefficients R 2 - Correlation Coefficient Confidence Limits  Significance T-test on Coefficients Analysis of Variance Adequacy of the Model