1. EE3561_Unit 4(c)AL-DHAIFALLAH14351 EE 3561 : Computational Methods Unit 4 : Least Squares Curve Fitting Dr. Mujahed Al-Dhaifallah (Term 342) Reading Assignment : 17.1 - 17.3

2. Elementary Statistics  A statistical sample is a fraction or a portion of the whole (population) that is studied.  For example Consider Table 1 which contains 14 measurements of the concentration of sodium chlorate produced in a chemical reactor operated at a pH of 7.0. EE3561_Unit 4(c)AL-DHAIFALLAH14352

3. Elementary Statistics  One of the measures of the spread of the data is the range of the data. The range is defined as the difference between the maximum and minimum value of the data as R  However, range may not give a good idea of the spread of the data as some data points may be far away from most other data points (such data points are called outliers). EE3561_Unit 4(c)AL-DHAIFALLAH14353

4. Elementary Statistics  That is why the sum of the square of the differences is considered a better measure. The sum of the squares of the differences, also called summed squared error (SSE), S t, is given by  Since the magnitude of the summed squared error is dependent on the number of data points, an average value of the summed squared error is defined as the variance, σ 2 EE3561_Unit 4(c)AL-DHAIFALLAH14354

5. Elementary Statistics  To bring the variation back to the same level of units as the original data, a new term called standard deviation, σ, is defined as  Furthermore, the ratio of the standard deviation to the mean, known as the coefficient of variation is also used to normalize the spread of a sample. EE3561_Unit 4(c)AL-DHAIFALLAH14355

6. Example EE3561_Unit 46

7. Example EE3561_Unit 4(c)AL-DHAIFALLAH14357

8. EE3561_Unit 4(c)AL-DHAIFALLAH14358 Motivation Given a set of experimental data x123 y5.15.96.3 The relationship between x and y may not be clear we want to find an expression for f(x) 1 2 3

9. EE3561_Unit 4(c)AL-DHAIFALLAH14359 Curve Fitting Given a set of tabulated data, find a curve or a function that best represents the data. Given: 1. The tabulated data 2. The form of the function 3. The curve fitting criteria Find the unknown coefficients

10. EE3561_Unit 4(c)AL-DHAIFALLAH143510 Selection of the functions

11. EE3561_Unit 4(c)AL-DHAIFALLAH143511 Decide on the criterion Chapter 17 Chapter 18

12. EE3561_Unit 4(c)AL-DHAIFALLAH143512 Least Squares xixi x1x1 x2x2 ….xnxn yiyi y1y1 y2y2 ynyn Given The form of the function is assumed to be known but the coefficients are unknown The difference is assumed to be the result of experimental error

13. EE3561_Unit 4(c)AL-DHAIFALLAH143513 Determine the Unknowns

14. EE3561_Unit 4(c)AL-DHAIFALLAH143514 Determine the Unknowns

15. EE3561_Unit 4(c)AL-DHAIFALLAH143515 Example 1 x123 y5.15.96.3

16. EE3561_Unit 4(c)AL-DHAIFALLAH143516 Remember

17. EE3561_Unit 4(c)AL-DHAIFALLAH143517 Example 1

18. EE3561_Unit 4(c)AL-DHAIFALLAH143518 Example 1

19. EE3561_Unit 4(c)AL-DHAIFALLAH143519 Example 1 i123sum xixi 1236 yiyi 5.15.96.317.3 xi2xi2 14914 x i y i 5.111.818.935.8

20. EE3561_Unit 4(c)AL-DHAIFALLAH143520 Example 1

21. EE3561_Unit 4(c)AL-DHAIFALLAH143521 Example 2 x 0.240.650.951.241.732.012.232.52 y 0.23-0.23-1.1-0.450.270.1-0.290.24

22. EE3561_Unit 4(c)AL-DHAIFALLAH143522 Example 2

23. EE3561_Unit 4(c)AL-DHAIFALLAH143523 Example 2

24. EE3561_Unit 4(c)AL-DHAIFALLAH1435 24 i12345678sum xixi.240.65.951.241.732.012.232.52 yiyi 0.23-0.23-1.1-0.450.270.1-0.290.24 ln(x i ) -1.4-0.43-0.05 0.220.550.70.80.92 cos(x i )0.970.80.580.32-0.16-0.43-0.61-0.81 e xi 1.271.922.593.465.647.469.312.42 (ln(x i )) 2 2.040.190.0030.046 0.30.490.640.85 4.56 ln(x)cos(x)-1.39-0.34-0.030.07 -0.09-0.3-0.5-0.75 -3.32 ln(x i ) e xi -1.81-0.82-0.130.74 3.095.21 7.4611.49 25.2 ln(x i ) y i -0.320.10.056-0.10.150.07-0.230.22 -0.06 (cos(x i )) 2 0.940.630.340.110.030.180.380.66 3.26 cos(x i )e xi 1.231.521.51.12-0.89-3.17-5.7-10.1 -14.5 cos(x i )y i 0.22-0.18-0.64-0.15-0.04 0.18-0.2 -0.85 (e xi ) 2 1.623.676.6911.9431.8255.786.4 9 154.5 352.4 e xi y i 0.29-0.44-2.84-1.561.520.75-2.72.98 -1.99

25. Example 2 EE3561_Unit 4(c)AL-DHAIFALLAH143525 Solve to get

26. EE3561_Unit 4(c)AL-DHAIFALLAH143526 How do you judge performance?

27. EE3561_Unit 4(c)AL-DHAIFALLAH143527 Multiple Regression Example: Given the following data It is required to determine a function of two variables f(x,t) = a + b x + c t to explain the data that is best in the least square sense. t0123 x0.10.40.2 f(x,t)3212

28. EE3561_Unit 4(c)AL-DHAIFALLAH143528 Solution of Multiple Regression Construct, the sum of the square of the error and derive the necessary conditions by equating the partial derivatives with respect to the unknown parameters to zero then solve the equations. t0123 x0.10.40.2 f(x,t)3212

29. EE3561_Unit 4(c)AL-DHAIFALLAH143529 Solution of Multiple Regression

30. EE3561_Unit 4(c)AL-DHAIFALLAH143530 Nonlinear least squares problems + More  Examples of nonlinear least squares  Solution of inconsistent equations  Continuous least square problems

31. EE3561_Unit 4(c)AL-DHAIFALLAH143531 Nonlinear Problem Given x123 y2.45.9

32. EE3561_Unit 4(c)AL-DHAIFALLAH143532 Alternative Solution (Linearization Method) Given x123 y2.45.9

33. EE3561_Unit 4(c)AL-DHAIFALLAH143533 Inconsistent System of Equations

34. EE3561_Unit 4(c)AL-DHAIFALLAH143534 Inconsistent System of Equations Reasons Inconsistent equations may occur because of presence of noise in input or output data, or modeling error, etc. Solution if all lines intersect at one point

35. EE3561_Unit 4(c)AL-DHAIFALLAH143535 Inconsistent System of Equations Formulation as a least squares problem

36. EE3561_Unit 4(c)AL-DHAIFALLAH143536 Solution

37. EE3561_Unit 4(c)AL-DHAIFALLAH143537 Solution

38. EE3561_Unit 4(c)AL-DHAIFALLAH143538 Examples (Linearization Method) Given x123 y0.23.2.14

39. Exercise EE3561_Unit 4(c)AL-DHAIFALLAH143539