1. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 171 Least Squares Regression Chapter 17 Linear Regression Fitting a straight line to a set of paired observations: (x 1, y 1 ), (x 2, y 2 ),…,(x n, y n ). y=a 0 +a 1 x+e a 1 - slope a 0 - intercept e- error, or residual, between the model and the observations

2. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 172 Criteria for a “Best” Fit/ Minimize the sum of the residual errors for all available data: n = total number of points However, this is an inadequate criterion, so is the sum of the absolute values

3. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 173 Figure 17.2

4. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 174 Best strategy is to minimize the sum of the squares of the residuals between the measured y and the y calculated with the linear model: Yields a unique line for a given set of data.

5. List-Squares Fit of a Straight Line/ Normal equations, can be solved simultaneously Mean values

6. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 176 Figure 17.3

7. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 177 Figure 17.4

8. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 178 Figure 17.5

9. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University 9 “Goodness” of our fit/ If Total sum of the squares around the mean for the dependent variable, y, is S t Sum of the squares of residuals around the regression line is S r S t -S r quantifies the improvement or error reduction due to describing data in terms of a straight line rather than as an average value. r 2 -coefficient of determination Sqrt(r 2 ) – correlation coefficient

10. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1710 For a perfect fit S r =0 and r=r 2 =1, signifying that the line explains 100 percent of the variability of the data. For r=r 2 =0, S r =S t, the fit represents no improvement.

11. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1711 Polynomial Regression Some engineering data is poorly represented by a straight line. For these cases a curve is better suited to fit the data. The least squares method can readily be extended to fit the data to higher order polynomials (Sec. 17.2).

12. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1712 General Linear Least Squares Minimized by taking its partial derivative w.r.t. each of the coefficients and setting the resulting equation equal to zero

13. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1713 Interpolation Chapter 18 Estimation of intermediate values between precise data points. The most common method is: Although there is one and only one nth-order polynomial that fits n+1 points, there are a variety of mathematical formats in which this polynomial can be expressed: –The Newton polynomial –The Lagrange polynomial

14. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1714 Figure 18.1

15. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1715 Newton’s Divided-Difference Interpolating Polynomials Linear Interpolation/ Is the simplest form of interpolation, connecting two data points with a straight line. f 1 (x) designates that this is a first-order interpolating polynomial. Linear-interpolation formula Slope and a finite divided difference approximation to 1 st derivative

16. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1716 Figure 18.2

17. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 17 Quadratic Interpolation/ If three data points are available, the estimate is improved by introducing some curvature into the line connecting the points. A simple procedure can be used to determine the values of the coefficients.

18. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Chapter 1818 General Form of Newton’s Interpolating Polynomials/ Bracketed function evaluations are finite divided differences

19. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1719 Errors of Newton’s Interpolating Polynomials/ Structure of interpolating polynomials is similar to the Taylor series expansion in the sense that finite divided differences are added sequentially to capture the higher order derivatives. For an n th -order interpolating polynomial, an analogous relationship for the error is: For non differentiable functions, if an additional point f(x n+1 ) is available, an alternative formula can be used that does not require prior knowledge of the function:  Is somewhere containing the unknown and he data

20. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1720 Lagrange Interpolating Polynomials The Lagrange interpolating polynomial is simply a reformulation of the Newton’s polynomial that avoids the computation of divided differences:

21. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1721 As with Newton’s method, the Lagrange version has an estimated error of:

22. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1722 Figure 18.10

23. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1723 Coefficients of an Interpolating Polynomial Although both the Newton and Lagrange polynomials are well suited for determining intermediate values between points, they do not provide a polynomial in conventional form: Since n+1 data points are required to determine n+1 coefficients, simultaneous linear systems of equations can be used to calculate “a”s.

24. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1724 Where “x”s are the knowns and “a”s are the unknowns.

25. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1725 Figure 18.13

26. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1726 Spline Interpolation There are cases where polynomials can lead to erroneous results because of round off error and overshoot. Alternative approach is to apply lower-order polynomials to subsets of data points. Such connecting polynomials are called spline functions.

27. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1727 Figure 18.14

28. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1728 Figure 18.15

29. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1729 Figure 18.16

30. Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 1730 Figure 18.17