1. Introduction to Numerical Analysis Using MATLABBy
Dr Rizwan Butt

2. CHAPTER ONE Number Systems and Errors
Introduction
It simply provides an introduction of numerical analysis.
Number Representation and Base of Numbers
Here we consider methods for representing numbers on computers.
1. Normalized Floating-point Representation
It describes how the numbers are stored in the computers.

3. CHAPTER 1 NUMBER SYSTEMS AND ERRORS
1. Human Error
It causes when we use inaccurate measurement of data or inaccurate representation of mathematical constants.
2. Truncation Error
It causes when we are forced to use mathematical techniques which give approximate, rather than exact answer.
3. Round-off Error
This type of errors are associated with the limited number of digits numbers in the computers.

4. CHAPTER 1 NUMBER SYSTEMS AND ERRORS
Effect of Round-off Errors in Arithmetic Operation
Here we analysing the different ways to understand the nature of rounding errors.
1. Rounding off Errors in Addition and Subtraction
It describes how addition and subtraction of numbers are performed in a computer.
2. Rounding off Errors in Multiplication
It describes how multiplication of numbers are performed in a computer.

5. CHAPTER 1 NUMBER SYSTEMS AND ERRORS
3. Rounding off Errors in Division
It describes how division of numbers are performed in a computer.
4. Rounding off Errors in Powers and roots
It describes how the powers and roots of numbers are performed in a computer.

6. CHAPTER TWO Solution of Nonlinear Equations
Introduction
Here we discuss the ways of representing the different types of nonlinear equation f(x) = 0 and how to find approximation of its real root .
Simple Root’s Numerical Methods
Here we discuss how to find the approximation of the simple root (non-repeating) of the nonlinear equation f(x) = 0.

7. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
1. Method of Bisection
This is simple and slow convergence method (but convergence is guaranteed) and is based on the Intermediate Value Theorem. Its strategy is to bisect the interval from one endpoint of the interval to the other endpoint and then retain the half interval whose end still bracket the root.
2. False Position Method
This is slow convergence method and may be thought of as an attempt to improve the convergence characteristic of bisection method. Its also known as the method of linear interpolation.

8. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
3. Fixed-Point Method
This is very general method for finding the root of nonlinear equation and provides us with a theoretical framework within which the convergence properties of subsequent methods can be evaluated. The basic idea of this method is convert the equation f(x) = 0 into an equivalent form x = g(x).
4. Newtons Method
This is fast convergence method (but convergence is not guaranteed) and is also known as method of tangent because after estimated the actual root, the zero of the tangent to the function at that point is determined.

9. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
5. Secant Method
This is fast convergence method (but not like Newton’s method) and is recommended as the best general-purpose method. It is very similar to false position method, but it is not necessary for the interval to contain a root and no account is taken of signs of the numbers f(x_n).
Multiple Root’s Numerical Methods
Here we discuss how to find approximation of multiple root (repeating) of nonlinear equation f(x) = 0 and its order of multiplicity m.

10. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
1. First Modified Newtons Method
It can be useful to find the approximation of multiple root if the order of multiplicity m is given.
2. Second Modified Newtons Method
It can be useful to find the approximation of multiple root if the order of multiplicity m is not given.
Convergence of Iterative Methods
Here we discuss order of convergence of all the iterative methods described in the chapter.

11. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
Acceleration of Convergence
Here we discuss a method which can be applied to any linear convergence iterative method and acceleration of convergence can be achieved.
Systems of Nonlinear Equations
When we are given more than one nonlinear equation. Solving systems of nonlinear is a difficult task.
Newtons Method
We discuss this method for system of two nonlinear equations in two variables. For system of nonlinear equations that have analytical partial derivatives, this method can be used, otherwise not.

12. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
Roots of Polynomials
A very common problem in nonlinear equations is to find the roots of polynomial is discussed here.
1. Horner’s Method
It is one of the most efficient way to evaluate polynomials and their derivatives at a given point. It is helpful for finding the initial approximation for solution by Newton’s method. It is also quit stable.

13. CHAPTER 2 SOLUTION OF NONLINEAR EQUATIONS
2. Muller’s Method
It is generalization of secant method and uses quadratic interpolation among three points. It is a fast convergence method for finding the approximation of simple zero of a polynomial equation.
3. Bairstow’s Method
It can be used to find all the zeros of a polynomial. It is one of the most efficient method for determining real and complex roots of polynomials with real coefficients.

14. CHAPTER THREE Systems of Linear Equations
Introduction
We give the brief introduction of linear equations, linear systems, and their importance.
Properties of Matrices and Determinant
To discuss the solution of the linear systems, it is necessary to introduce the basic properties of matrices and the determinant.
Numerical Methods for Linear Systems
To solve the systems of linear equations using the numerical methods, there are two types of methods available, methods of first type are called direct methods and second type are called iterative methods.

15. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
Direct Methods for Linear Systems
The method of this type refers to a procedure for computing a solution from a form that is mathematically exact. These methods are guaranteed to succeed and are recommended for general-purpose.
1. Cramers Rule
This method is use for solving the linear systems by the use of determinants. It is one of the least efficient method for solving a large number of linear equations. But it is useful for explaining some problems inherent in the solution of linear equations. 2

16. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
2. Gaussian Elimination Method
It is most popular and widely used method for solving linear system. The basic of this method is to convert the original system into equivalent upper-triangular system and from which each unknown is determined by backward substitution.
2.1 Without Pivoting
In converting original system to upper-triangular system if a diagonal element becomes zero, then we have to interchange that equation with any below equation having nonzero diagonal element.

17. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
2.2 Partial Pivoting
In using the Gauss elimination by partial pivoting (or row pivoting), the basic approach is to use the largest (in absolute value) element on or below the diagonal in the column of current interest as the pivotal element for elimination in the rest of that column.
2.3 Complete Pivoting
In this case we search for the largest number (in absolute value) in the entire array instead of just in the first column, and this number is the pivot. This means we need to interchange the columns as well as rows.

18. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
3. Gauss-Jordan Method
It is a modification of Gauss elimination method and is although inefficient for practical calculation but is often useful for theoretical purposes. The basic idea of this method is to convert original system into diagonal system form.
4. LU Decomposition Method
It is also a modification of Gauss elimination method and here we decompose or factorize the coefficient matrix into the product of two triangular matrices (lower and upper).

19. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
4.1 Dollittle’s method (l_ii = 1)
Here the upper-triangular matrix is obtained by forward elimination of Gauss elimination method and the lower-triangular matrix containing the multiples used in the Gauss elimination process as the elements below the diagonal with unity elements on the main diagonal.
4.2 Crout’s method (u_ii = 1)
The Crout’s method, in which upper-triangular matrix has unity on the main diagonal, is similar to the Dollittle’s method in all other aspects. The lower-triangular and upper-triangular matrices are obtained by expanding the matrix equation A = LU term by term to determine the elements of the lower-triangular and upper-triangular matrices.

20. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
4.3 Cholesky method (l_ii = u_ii)
This method is of the same form as the Dollittle’s and Crout’s methods except it is limited to equations involving symmetrical coefficient matrices. This method provides a convenient method for investigating the positive definiteness of symmetric matrices.
Norms of Vectors and Matrices
For solving linear systems, we discuss a method for quantitatively measuring the distance between vectors in R^n and a measure of how well one matrix approximates another.

21. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
Iterative Methods for Solving Linear Systems
These methods start with an arbitrary first approximation to the unknown solution of linear system and then improve this estimate in an infinite but convergent sequence of steps. This type of methods are used for large sparse systems and efficient in terms of computer storage and time requirement.
1. Jacobi Iterative Method
It is a slow convergent iterative method for the linear systems. From its formula, it is seen that the new estimates for solution are computed from the old estimates.
2. Gauss-Seidel Iterative Method
It is a faster convergent iterative method than the Jacobi method for the solution of the linear systems as it uses the most recent calculated values for all x_i.

22. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
Convergence Criteria
We discuss the sufficient conditions for the convergence of Jacobi and Gauss-Seidel methods by showing l_∞-norm of their corresponding iteration matrices less than one.
Eigenvalues and Eigenvectors
We briefly discuss the eigenvalues and eigenvectors of a matrix and show how they can be used to describe the solutions of linear systems.
3. Successive Over-Relaxation Method
It is useful modification of the Gauss-Seidel method. It is the best iterative method of choice and needs to determine optimum value of the parameter.

23. CHAPTER 3 SYSTEMS OF LINEAR EQUATIONS
4. Conjugate Gradient Method
It is very useful when employed as an iterative approximation method for solving large sparse linear systems. The need for estimating parameter is removed in this method.
Conditioning of Linear Systems
We discuss ill-conditioning of linear systems by using the condition number of matrix. The best way to deal with ill-conditioning is to avoid it by reformulating the problem.
Iterative Refinement
We discuss residual corrector method which can be used to improve the approximate solution obtained by any means.

24. CHAPTER FOUR Approximating Functions
Introduction
We describe several numerical methods for approximating functions other than elementary functions. The main purpose of these numerical methods is to replace a complicated function by one which is simpler and more manageable.
Polynomial Interpolation for Uneven Intervals
The data points we consider here in a given functional relationship are not equally spaced.

25. CHAPTER 4 APPROXIMATING FUNCTIONS
1. Lagrange Interpolating Polynomials
It is one of the popular and well known interpolation method to approximate the functions at arbitrary point and provides a direct approach for determining interpolated values regardless of data spacing.
2. Newtons General Interpolating Formula
It is generally more efficient than Lagrange polynomial and it can be adjusted easily for additional data.
3. Aitkens Method
It is an iterative interpolation method which is based on the repeated application of a simple interpolation method.

26. CHAPTER 4 APPROXIMATING FUNCTIONS
Polynomial Interpolation for Even Intervals
The data points we consider here in a given functional relationship are equally spaced and polynomials are based on differences which are easy to use.
1. Newton’s Forward-Difference Formula
It can be used for interpolation near the beginning of table values.
2. Newton’s Backward-Difference Formula
It can be use for interpolation near the end of table values.
3. Some Central-Difference Formulas
These can be used for interpolation in the middle of the table values and among them are Stirling, Bessel, and Gauss formulas.

27. CHAPTER 4 APPROXIMATING FUNCTIONS
Interpolation with Spline Functions
It is an alternative approach to divide the interval into a collection of subintervals and construct a different approximating polynomial on each subinterval, called Piecewise Polynomial Approximation.
1. Linear Spline
One of the simplest piecewise polynomial interpolation for approximating functions and basic of it is simply connect consecutive points with straight lines.
2. Cubic Spline
The most widely cubic spline approximations are patched among ordered data that maintain continuity and smoothness and they are more powerful than polynomial interpolation.

28. CHAPTER 4 APPROXIMATING FUNCTIONS
Least Squares Approximation
Least squares approximation which seeks to minimize the sum (over all data) of the squares of the differences between function values and data values, are most useful for large and rough sets of data.
1. Linear Least Squares
It defines the correct straight line as the one that minimizes the sum of the squares of the distance between the data points and the line.
2. Polynomial Least Squares
When data from experimental results are not linear, then we find the least squares parabola and the extension to a polynomial of higher degree is easily made.

29. CHAPTER 4 APPROXIMATING FUNCTIONS
3. Nonlinear Least Squares
In many cases, data from experimental tests are not linear, then we fit to them two popular exponential forms y = ax^b and y = ae^(bx).
4. Least Squares Plane
When the dependent variable is function of two variables, then the least squares plane can be used to find the approximation of the function.
5. Overdetermined Linear Systems
The least squares solution of overdetermined linear system can be obtained by minimizing the l_2-norm of the residual.

30. CHAPTER 4 APPROXIMATING FUNCTIONS
6. Least Squares with QR Decomposition
The least squares solution of the overdetermined linear system can be obtained by using QR (the orthogonal matrix Q and upper-triangular matrix R) decomposition of a given matrix.
7. Least Squares with Singular Value Decomposition
The least squares solution of the overdetermined linear system can be obtained by using singular value (UDV^T, the two orthogonal matrices U, V and a generalized diagonal matrix D) decomposition of a given matrix.

31. CHAPTER FIVE Differentiation and Integration
Introduction
Here, we deal with techniques for approximating numerically the two fundamental operations of the calculus, differentiation and integration.
Numerical Differentiation
A polynomial p(x) is differentiated to obtain p′(x), which is taken as an approximation to f′(x) for any numerical value x.
Numerical Differentiation Formulas
Here we gave many numerical formulas for approximating the first derivative and second derivative of a function.

32. CHAPTER 5 DIFFERENTIATION AND INTEGRATION
1. First Derivatives Formulas
For finding the approximation of the first derivative of a function, we used two-point, three-point, and five-point formulas.
2. Second Derivatives Formulas
For finding the approximation of the second derivative of a function, we used three-point and five-point formulas.
3. Formulas for Computing Derivatives
Here we gave many forward-difference, backward-difference, and central-difference formulas for approximating the first and second derivative of the function.

33. CHAPTER 5 DIFFERENTIATION AND INTEGRATION
Numerical Integration
Here, we pass a polynomial through points of a function and then integrate this polynomial approximation to a function. For approximating the integral of f(x) between a and b we used Newton-Cotes techniques.
1. Closed Newton-Cotes Formulas
For these formulas, the end-points a and b of the given interval [a, b] are in the set of interpolating points and the formulas can be obtained by integrating polynomials fitted to equispaced data points.
1.1 Trapezoidal Rule
This rule is based on integration of the linear interpolation.

34. CHAPTER 5 DIFFERENTIATION AND INTEGRATION
1.2 Simpson’s Rule
This rule approximates the function f(x) with a quadratic interpolating polynomial.
2. Open Newton-Cotes Formulas
These formulas contain all the points used for approximating within the open interval (a, b) and can be obtained by integrating polynomials fitted to equispaced data points.
3. Repeated use of the Trapezoidal Rule
The repeated Trapezoidal rule is derived by repeating the Trapezoidal rule and for a given domain of integration, error of the repeated Trapezoidal rule is proportional to h_2.

35. CHAPTER 5 DIFFERENTIATION AND INTEGRATION
4. Romberg Integration
The Romberg integration is based on the repeated Trapezoidal rule and using the results of repeated Trapezoidal rule with two different data spacings, a more accurate integral is evaluated.
5. Gaussian Quadratures
The Gauss(-Legendre) quadratures are based on integrating a polynomial fitted to the data points at the roots of a Legendre polynomial and the order of accuracy of a Gauss quadrature is approximately twice as high as that of the Newton-Cotes closed formula using the same number of data points.

36. CHAPTER SIX Ordinary Differential Equations
Introduction
We discussed many numerical methods for solving first-order ordinary differential equations and systems of first-order ordinary differential equations.
Numerical Methods for Solving IVP
Here we discuss many single-step numerical methods and multi-step numerical methods for solving the initial-value problem (IVP) and some numerical methods for solving boundary-value problem (BVP).
1. Single-Step Methods for IVP
These types of methods are self-starting, refer to estimate y′(x) from the initial condition and proceed step-wise. All the information used by these methods is consequently obtained within the interval over which the solution is being approximated.

37. CHAPTER 6 ORDINARY DIFFERENTIAL EQUATIONS
1.1 Euler’s Method
One of the simplest and straight forward but not an efficient numerical method for solving initial-value problem (IVP).
1.2 Higher-Order Taylor’s Methods
For getting higher accuracy, the Taylor’s methods are excellent when the higher-order derivative can be found.
1.3 Runge-Kutta Methods
An important group of methods which allow us to obtain great accuracy at each step and at the same time avoid the need of higher derivatives by evaluating the function at selected points on each subintervals.

38. CHAPTER 6 ORDINARY DIFFERENTIAL EQUATIONS
2. Multi-Steps Methods for IVP
This type of methods make use of information about the solution at more than one point.
2.1 Adams Methods
These methods use the information at multiple steps of the solution to obtain the solution at the next x-value.
2.2 Predictor-Corrector Methods
These methods are combination of an explicit method and implicit method and they are consist of predictor step and corrector step in each interval.

39. CHAPTER 6 ORDINARY DIFFERENTIAL EQUATIONS
Systems of Simultaneous ODE
Here, we require the solution of a system of simultaneous first-order differential equations rather than a single equation.
Higher-Order Differential Equations
Here, we deal the higher-order differential equation (nth-order) and solve it by converting to an equivalent system of (n) first-order equations.
Boundary-Value Problems
Here, we solve ordinary differential equation with known conditions at more than one value of the independent variable.

40. CHAPTER 6 ORDINARY DIFFERENTIAL EQUATIONS
1. The Shooting Method
It is based on by forming a linear combination of the solution to two initial-value problems (linear shooting method) and by converting a boundary-value problem to a sequence of initial-value problems (nonlinear shooting method) which can be solved using the single steps method.
2. The Finite Difference Method
It is based on finite differences and it reduces a boundary-value problem to a system a system of linear equations which can be solved by using the methods discussed in the linear system chapter.

41. CHAPTER SEVEN Eigenvalues and Eigenvectors
Introduction
Here we discussed many numerical methods for solving eigenvalue problems which seem to be a very fundamental part of the structure of universe.
Linear Algebra and Eigenvalues Problems
The solution of many physical problems require the calculations of the eigenvalues and corresponding eigenvectors of a matrix associated with linear system of equations.
Basic Properties of Eigenvalue Problems
We discussed many properties concerning with eigenvalue problems which help us a lot in solving different problems.

42. CHAPTER 7 EIGENVALUES AND EIGENVECTORS
Numerical Methods for Eigenvalue Problems
Here we discussed many numerical methods for finding approximation of the eigenvalues and corresponding eigenvectors of the matrices.
Vector Iterative Methods for Eigenvalues
This type of numerical methods are most useful when matrix involved be comes large and also they are easy means to compute eigenvalues and eigenvectors of a matrix.
1. Power Method
It can be used to compute the eigenvalue of largest modules (dominant eigenvalue) and the corresponding eigenvector of a general matrix.

43. CHAPTER 7 EIGENVALUES AND EIGENVECTORS
2. Inverse Power Method
This modification of the power method can be used to compute the smallest (least) eigenvalue and the corresponding eigenvector of a general matrix.
3. Shifted Inverse Power Method
This modification of the power method consists of by replacing the given matrix A by (A−μI) and the eigenvalues of (A−μI) are the same as those of A except that they have all been shifted by an amount μ.

44. CHAPTER 7 EIGENVALUES AND EIGENVECTORS
Location of Eigenvalues
We deal here with the location of eigenvalues of both symmetric and non- symmetric matrices, that is, the location of zeros of the characteristic poly nomial by using the Gerschgorin Circles Theorem and Rayleigh Quotient Theorem.
Intermediate Eigenvalues
Here we discussed the Deflation method to obtain other eigenvalues of a matrix once the dominant eigenvalue is known.
Eigenvalues of Symmetric Matrices
Here, we developed some methods to find all eigenvalues of a symmetric matrix by using a sequence of similarity transformation that transformed the original matrix into a diagonal or tridiagonal matrix.

45. CHAPTER 7 EIGENVALUES AND EIGENVECTORS
1. Jacobi Method
It can be used to find all eigenvalues and corresponding eigenvectors of a symmetric matrix and it permits the transformation of a matrix into a diagonal.
2. Sturm Sequence Iteration
It can be used in the calculation of eigenvalues of any symmetric tridiagonal matrix.
3. Given’s Method
It can be used to find all eigenvalues of a symmetric matrix (corresponding eigenvectors can be obtained by using shifted inverse power method) and it permits the transformation of a matrix into a tridiagonal.
4. Householder’s Method
This method is a variation of the Given’s method and enable us to reduce a symmetric matrix to a symmetric tridiagonal matrix form.

46. CHAPTER 7 EIGENVALUES AND EIGENVECTORS
Matrix Decomposition Methods
Here we used three matrix decomposition methods and find all the eigenvalues of a general matrix.
1. QR Method
In this method we decomposed the given matrix into a product orthogonal matrix and a upper-triangular matrix which find all the eigenvalues of a general matrix.
2. LR Method
This method is based upon the decomposition of the given matrix into a product lower-triangular matrix (with unit diagonal elements) and a upper- triangular matrix.
3. Singular Value Decomposition
Here we decomposed rectangular real matrix into a product of two orthogonal matrices and generalized diagonal matrix.

47. Appendices includes some mathematical preliminaries. 2. Appendix B
1. Appendix A
includes some mathematical preliminaries.
2. Appendix B
includes the basic commands for software package MATLAB.
3. Appendix C
includes the index of MATLAB programs and MATLAB built-in-functions.
4. Appendix D
includes symbolic computation and Symbolic Math Toolbox functions.
5. Appendix E
includes answers to selected odd-number exercises for all chapters.