Download presentation
Presentation is loading. Please wait.
Published byStephen Green Modified over 9 years ago
1
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Part 4 Chapter 15 General Least Squares and Non- Linear Regression
2
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Part Organization Chapter 14 –Brief Review of Statistics –Linear Regression (How to determine the best fit) –Linearization of Nonlinear Equations Chapter 15 –Polynomial Regression –Multiple Linear Regression Chapter 16 – Skip Chapter 17 – Interpolating Chapter 18 – Spline Interpolation 2
3
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 3 Chapter 15 – General Least Squares Some engineering data is poorly represented by a straight line. For these cases a curve is better suited to fit the data. –In chapter 14 we looked at techniques to linearize other models –This approach allowed us to use linear regression –An alternate approach is to use polynomial regression
4
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 4 Taylor’s Theorem Recall that any smooth function can be approximated by a polynomial Polynomial regression fits a polynomial to a set of data points
5
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 5 General Linear Least Squares Minimized by taking its partial derivative w.r.t. each of the coefficients and setting the resulting equation equal to zero
6
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. For Simple Linear Regression 6
7
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 7 Least-Squares Fit of a Straight Line To minimize Sr, we need to find the partial derivatives with respect to a0 and a1 and set them equal to 0
8
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Coefficients for a Simple Linear Regression 8
9
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. For 2 nd Order Polynomial Regression 9
10
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Example 15.1 10 Fit this data to a second order polynomial
11
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. As before we need to minimize the sum of the squares of the residuals 11 Take the derivatives with respect to the coefficients (a) and set them equal to 0
12
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 3 equations and 3 unknowns 12
13
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. This is starting to get cumbersome We could solve for the coefficients using matrix algebra 13
14
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 14
15
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 15
16
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. This is starting to get cumbersome We could solve for the coefficients using matrix algebra…. But… we could also use the built-in MATLAB function polyfit 16
17
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 17
18
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 18 polyfit has the advantage that you can use it for higher order polynomials
19
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Or use left division 19
20
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 20 General Linear Least Squares Minimized by taking its partial derivative w.r.t. each of the coefficients and setting the resulting equation equal to zero
21
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. For 2 nd Order Polynomial Regression 21
22
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 2.1000 7.7000 13.6000 27.2000 40.9000 61.1000 a 0 * 1 + a 1 * 0 + a 2 * 0 a 0 * 1 + a 1 * 1 + a 2 * 1 a 0 * 1 + a 1 * 2 + a 2 * 4 a 0 * 1 + a 1 * 3 + a 2 * 9 a 0 * 1 + a 1 * 4 + a 2 * 16 a 0 * 1 + a 1 * 5 + a 2 * 25 =
23
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 2.1000 7.7000 13.6000 27.2000 40.9000 61.1000 a 0 * 1 + a 1 * 0 + a 2 * 0 a 0 * 1 + a 1 * 1 + a 2 * 1 a 0 * 1 + a 1 * 2 + a 2 * 4 a 0 * 1 + a 1 * 3 + a 2 * 9 a 0 * 1 + a 1 * 4 + a 2 * 16 a 0 * 1 + a 1 * 5 + a 2 * 25 = If you have an equal number of equations and unknowns the \ operator uses a modified Gaussian elimination strategy
24
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 2.1000 7.7000 13.6000 27.2000 40.9000 61.1000 a 0 * 1 + a 1 * 0 + a 2 * 0 a 0 * 1 + a 1 * 1 + a 2 * 1 a 0 * 1 + a 1 * 2 + a 2 * 4 a 0 * 1 + a 1 * 3 + a 2 * 9 a 0 * 1 + a 1 * 4 + a 2 * 16 a 0 * 1 + a 1 * 5 + a 2 * 25 = If you have more equations than unknowns the system is over-specified, and the \ operator utilizes QR factorization to find the best fit
25
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 25 The General Linear Least Squares Technique can be applied to more than just polynomials Minimized by taking its partial derivative w.r.t. each of the coefficients and setting the resulting equation equal to zero
26
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. What if you want to do a multiple linear regression? 26 Use a similar approach as that outlined in Section 15.2
27
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 27 This is pretty complicated, and it’s only good for 2 variables plus a constant!
28
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 28 This does the same thing, generalized for m dimensions
29
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Or… use left division 29
30
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Or… use left division 30 5 10 9 0 3 27 a 0 * 1 + a 1 * 0 + a 2 * 0 a 0 * 1 + a 1 * 2 + a 2 * 1 a 0 * 1 + a 1 * 2.5 + a 2 * 2 a 0 * 1 + a 1 * 1 + a 2 * 3 a 0 * 1 + a 1 * 4 + a 2 * 6 a 0 * 1 + a 1 * 7 + a 2 * 2 = QR Factorization
31
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Using the Interactive Curve Fitting Tools MATLAB 7 includes interactive plotting tools. They include –basic curve fitting, –more complicated curve fitting –statistical tools
32
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Use the curve fitting tools… Create a graph Making sure that the figure window is the active window select –Tools-> Basic Fitting –The basic fitting window will open on top of the plot
33
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
34
Plot generated using the Basic Fitting Window
35
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Residuals are the difference between the actual and calculated data points
36
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Basic Fitting Window
37
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Data Statistics Window You can also access the data statistics window from the figure menu bar. Select Tools->Data Statistics from the figure window. This window allows you to calculate statistical functions interactively, such as mean and standard deviation, based on the data in the figure, and allows you to save the results to the workspace.
38
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 38
39
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Save to your Current Directory 39
40
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Reload a figure by double clicking it’s name in the current directory 40
41
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Or in the command window 41
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.