1. Curve Fitting Introduction Least-Squares Regression Linear Regression Polynomial Regression Multiple Linear Regression Today’s class Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 1

2. Curve Fitting Given a set of discrete points how do you fill in the points in between to form a continuum? Approximation or regression Find a simple function that represents the trend of the curve given that the data may have measurement error or “noise” Interpolation The data is exact, so you need to find a function that passes through all the given points Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 2

3. Curve Fitting Least-Squares Regression Linear Interpolation Curvilinear Interpolation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 3

4. Review of Basic Statistics Given a set of data, say, y 1, y 2,……….y N. Arithmetic Mean: Standard Deviation: Variance: Coefficient of Variance: Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 4

5. Basic Statistics Measured data Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 5

6. Basic Statistics Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 6

7. Coefficient of Thermal Expansion Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 7

8. Histogram A histogram used to depict the distribution of data For large data set, the histogram often approaches the normal distribution Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 8

9. Linear Least-Squares Approximation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 9

10. Linear Least-Squares Approximation The goal is to come up with a linear function that comes close to fitting the given data points The closeness is determined by the error or residual Criteria for a best fit Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 10

11. Least-Squares Regression Minimize sum of all residual errors Positive and negative errors can cancel out Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 11

12. Least-Squares Regression Minimize sum of absolute value of residual errors May not get a unique solution Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 12

13. Least-Squares Regression Minimize the maximum residual error May be overly influenced by outliers Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 13

14. Least-Squares Regression The best idea is to minimize the sum of the squares of the differences between the measured value and the value calculated by the linear model Advantages Positive errors do not cancel out negative errors Large errors are magnified Unique solution Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 14

15. Linear Least-Squares Regression Minimize the sum of squares of errors 2 equations, 2 unknowns Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 15

16. Linear Least-Squares Regression normal equations solutions Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 16

17. Linear Least Squares Approximation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 17

18. The Standard Error of Estimate & Correlation Coefficient Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 18

19. Standard Deviation for Regression Line Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 19

20. Example: Linear Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 20

21. Example: Linear Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 21

22. Linearization of Nonlinear Relationships Linear regression is based on the fact that the relationship between dependent and independent variables is linear If the function is not linear, you will need polynomial regression techniques or other nonlinear techniques For certain classes of functions, you can linearize the data and still use linear regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 22

23. Linearization of Nonlinear Relationships Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 23

24. Linearization of Nonlinear Relationships Exponential equation Power equation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 24

25. Linearization of Nonlinear Relationships Saturation-growth-rate equation Rational function Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 25

26. Linearization of Nonlinear Relationships Summary of steps Translate a non-polynomial relation to a linear relation (translate data set accordingly) Linear regression of the translated data set Translate linear relation back to original relation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 26

27. Nonlinear Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 27

28. Quadratic Least Square Approximations Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 28

29. Polynomial Regression Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 29

30. Quadratic Least Squares Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 30

31. Example Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 31

32. Example Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 32

33. Standard Error of Estimation & Correlation Coefficient In using quadratic square approximation, S y/x is estimated by The correlation coefficient is calculated by, Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 33

34. Example Standard Error of Estimation Correlation coefficient Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 34

35. Cubic Least Square Approximations Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 35

36. Formulation for Cubic Least Square Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 36

37. Standard Error of Estimation & Correlation Coefficient In using cubic square approximation, S y/x is estimated by The correlation coefficient is calculated by, Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 37

38. Polynomial Regression An mth order polynomial will require that you solve a system of m+1 linear equations Standard error Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 38

39. Next class Multiple Linear Regression Linear Interpolation Fourier Approximation Numerical Methods Lecture 19 Prof. Jinbo Bi CSE, UConn 39