Curve Fitting Introduction Least-Squares Regression Linear Regression Polynomial Regression Multiple Linear Regression Today’s class Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 1

Curve Fitting Given a set of discrete points how do you fill in the points in between to form a continuum? Approximation or regression Find a simple function that represents the trend of the curve given that the data may have measurement error or “noise” Interpolation The data is exact, so you need to find a function that passes through all the given points Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 2

Curve Fitting Least-Squares Regression Linear Interpolation Curvilinear Interpolation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 3

Review of Basic Statistics Given a set of data, say, y 1, y 2,……….y N. Arithmetic Mean: Standard Deviation: Variance: Coefficient of Variance: Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 4

Basic Statistics Measured data Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 5

Basic Statistics Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 6

Coefficient of Thermal Expansion Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 7

Histogram A histogram used to depict the distribution of data For large data set, the histogram often approaches the normal distribution Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 8

Linear Least-Squares Approximation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 9

Linear Least-Squares Approximation The goal is to come up with a linear function that comes close to fitting the given data points The closeness is determined by the error or residual Criteria for a best fit Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 10

Least-Squares Regression Minimize sum of all residual errors Positive and negative errors can cancel out Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 11

Least-Squares Regression Minimize sum of absolute value of residual errors May not get a unique solution Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 12

Least-Squares Regression Minimize the maximum residual error May be overly influenced by outliers Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 13

Least-Squares Regression The best idea is to minimize the sum of the squares of the differences between the measured value and the value calculated by the linear model Advantages Positive errors do not cancel out negative errors Large errors are magnified Unique solution Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 14

Linear Least-Squares Regression Minimize the sum of squares of errors 2 equations, 2 unknowns Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 15

Linear Least-Squares Regression normal equations solutions Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 16

Linear Least Squares Approximation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 17

The Standard Error of Estimate & Correlation Coefficient Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 18

Standard Deviation for Regression Line Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 19

Example: Linear Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 20

Example: Linear Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 21

Linearization of Nonlinear Relationships Linear regression is based on the fact that the relationship between dependent and independent variables is linear If the function is not linear, you will need polynomial regression techniques or other nonlinear techniques For certain classes of functions, you can linearize the data and still use linear regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 22

Linearization of Nonlinear Relationships Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 23

Linearization of Nonlinear Relationships Exponential equation Power equation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 24

Linearization of Nonlinear Relationships Saturation-growth-rate equation Rational function Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 25

Linearization of Nonlinear Relationships Summary of steps Translate a non-polynomial relation to a linear relation (translate data set accordingly) Linear regression of the translated data set Translate linear relation back to original relation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 26

Nonlinear Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 27

Quadratic Least Square Approximations Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 28

Polynomial Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 29

Quadratic Least Squares Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 30

Example Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 31

Example Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 32

Standard Error of Estimation & Correlation Coefficient In using quadratic square approximation, S y/x is estimated by The correlation coefficient is calculated by, Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 33

Example Standard Error of Estimation Correlation coefficient Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 34

Cubic Least Square Approximations Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 35

Formulation for Cubic Least Square Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 36

Standard Error of Estimation & Correlation Coefficient In using cubic square approximation, S y/x is estimated by The correlation coefficient is calculated by, Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 37

Polynomial Regression An mth order polynomial will require that you solve a system of m+1 linear equations Standard error Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 38

Next class Multiple Linear Regression Linear Interpolation Fourier Approximation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 39