4 Review of Matrices Row vector: Column vector: Square matrix: 2nd rowElements are indicated by a i jrowcolumnmth columnRow vector: Column vector:Square matrix:- [A]nxm is a square matrix if n=m.- A system of n equations with n unknonws has a square coefficient matrix.
5 Review of Matrices Main (principle) diagonal: [A]nxn consists of elements aii ; i=1,...,nSymmetric matrix:If aij = aji [A]nxn is a symmetric matrixDiagonal matrix:[A]nxn is diagonal if aij = 0 for all i=1,...,n ; j=1,...,n and ijIdentity matrix:[A]nxn is an identity matrix if it is diagonal with aii=1 i=1,...,n . Shown as [I]
6 Review of Matrices Upper triangular matrix: [A]nxn is upper triangular if aij=0 i=1,...,n ; j=1,...,n and i>jLower triangular matrix:[A]nxn is lower triangular if aij=0 i=1,...,n ; j=1,...,n and i<jInverse of a matrix:[A]-1 is the inverse of [A]nxn if [A]-1[A] = [I]Transpose of a matrix:[B] is the transpose of [A]nxn if bij=aji Shown as [A] or [A]T
7 Special Types of Square Matrices SymmetricDiagonalIdentityUpper TriangularLower Triangular
8 Review of MatricesMatrix multiplication:Note: [A][B] [B][A]
9 Review of Matrices For example augmented with the column vector is Augmented matrix: is a special way of showing two matrices together.For example augmented with thecolumn vector isDeterminant of a matrix:A single number. Determinant of [A] is shown as |A|.
11 Solving Small Numbers of Equations There are many ways to solve a system of linear equations:Graphical methodCramer’s ruleMethod of eliminationNumerical methods for solving larger number of linear equations:- Gauss elimination (Chp.9)- LU decompositions and matrix inversion(Chp.10)For n ≤ 3
12 1. Graphical method For two equations (n = 2): Solve both equations for x2: the intersection of the lines presents the solution.For n = 3, each equation will be a plane on a 3D coordinate system. Solution is the point where these planes intersect.For n > 3, graphical solution is not practical.
13 Graphical Method -Example Solve:Plot x2 vs. x1, the intersection of the lines presents the solution.
14 Graphical Method No solution Infinite solution ill condition (sensitive to round-off errors)
15 Graphical MethodMathematicallyCoefficient matrices of (a) & (b) are singular. There is no unique solution for these systems. Determinants of the coefficient matrices are zero and these matrices can not be inverted.Coefficient matrix of (c) is almost singular. Its inverse is difficult to take. This system has a unique solution, which is not easy to determine numerically because of its extreme sensitivity to round-off errors.
16 2.Determinants and Cramer’s Rule Determinant can be illustrated for a set of three equations:Where [A] is the coefficient matrix:
18 Cramer’s RuleFor a singular system D = 0 Solution can not be obtained.For large systems Cramer’s rule is not practical because calculating determinants is costly.
19 3. Method of EliminationThe basic strategy is to successively solve one of the equations of the set for one of the unknowns and to eliminate that variable from the remaining equations by substitution.The elimination of unknowns can be extended to systems with more than two or three equations. However, the method becomes extremely tedious to solve by hand.
20 Elimination of Unknowns Method 2.5x x2 = 3.04.8x x2 = 5.5Given a 2x2 set of equations:Multiply the 1st eqn by 8.6and the 2nd eqn by 21.50x x2=25.829.76x1 – 53.32x2=34.1Add these equations x1 + 0 x2 = 59.9Solve for x1 : x1 = 59.9/ =Using the 1st eqn solve for x2 :x2 =(3.0–2.5* )/6.2 =Check if these satisfy the 2nd eqn:4.8* –8.6* =(Difference is due to the round-off errors).
21 Naive Gauss Elimination Method It is a formalized way of the previous elimination technique to large sets of equations by developing a systematic scheme or algorithm to eliminate unknowns and to back substitute.As in the case of the solution of two equations, the technique for n equations consists of two phases:1. Forward elimination of unknowns.2. Back substitution.
22 Naive Gauss Elimination Method Consider the following system of n equations.a11x1 + a12x a1nxn = b1 (1)a21x1 + a22x a2nxn = b2 (2)...an1x1 + an2x annxn = bn (n)Form the augmented matrix of [A|B].Step 1 : Forward Elimination: Reduce the system to an upper triangular system.1.1- First eliminate x1 from 2nd to nth equations.- Multiply the 1st eqn. by a21/a11 & subtract it from the 2nd equation.This is the new 2nd eqn.- Multiply the 1st eqn. by a31/a11 & subtract it from the 3rd equation.This is the new 3rd eqn.- Multiply the 1st eqn. by an1/a11 & subtract it from the nth equation.This is the new nth eqn.
23 Naive Gauss Elimination Method (cont’d) Note:In these steps the 1st eqn is the pivot equation and a11 is the pivot element.Note that a division by zero may occur if the pivot element is zero. Naive-Gauss Elimination does not check for this.The modified system is indicates that thesystem is modified once.
24 Naive Gauss Elimination Method (cont’d) 1.2- Now eliminate x2 from 3rd to nth equations.The modified system is Repeat steps (1.1) and (1.2) upto (1.n-1).we will get this upper triangular system
25 Naive Gauss Elimination Method (cont’d) Step 2 : Back substitutionFind the unknowns starting from the last equation.1. Last equation involves only xn. Solve for it.2. Use this xn in the (n-1)th equation and solve for xn-1....3. Use all previously calculated x values in the 1st eqn and solve for x1.
31 Naive Gauss Elimination Method Example 2 (Using 6 Significant Figures) 3.0 x x x3 = 7.850.1 x x x3 = R2-(0.1/3)R10.3 x x x3 = 71.4 R3-(0.3/3)R1Step 1: Forward eliminationx x x3 =x x3 =x x3 =x x x3 =x3 =
33 Pitfalls of Gauss Elimination Methods Division by zero2 x2 + 3 x3 = 84 x1 + 6 x2 + 7 x3 = -32 x1 + x2 + 6 x3 = 5It is possible that during both elimination and back-substitution phases a division by zero can occur.Round-off errorsIn the previous example where up to 6 digits were kept during the calculations and still we end up with close to the real solution.x3 = , instead of x3 = 7.0a11 = 0(the pivot element)
34 Pitfalls of Gauss Elimination (cont’d) 3. Ill-conditioned systemsx x2 = 101.1x1 + 2x2 = 10.4x1 + 2x2 = 101.05x1 + 2x2 = 10.4Ill conditioned systems are those where small changes in coefficients result in large change in solution. Alternatively, it happens when two or more equations are nearly identical, resulting a wide ranges of answers to approximately satisfy the equations. Since round off errors can induce small changes in the coefficients, these changes can lead to large solution errors. x1 = 4.0 & x2 = 3.0 x1 = 8.0 & x2 = 1.0
35 Pitfalls of Gauss Elimination (cont’d) Singular systems.When two equations are identical, we would loose one degree of freedom and be dealing with case of n-1 equations for n unknowns.To check for singularity:After getting the forward elimination process and getting the triangle system, then the determinant for such a system is the product of all the diagonal elements. If a zero diagonal element is created, the determinant is Zero then we have a singular system.The determinant of a singular system is zero.
36 Techniques for Improving Solutions Use of more significant figures to solve for the round-off error.Pivoting. If a pivot element is zero, elimination step leads to division by zero. The same problem may arise, when the pivot element is close to zero. This Problem can be avoided by:Partial pivoting. Switching the rows so that the largest element is the pivot element.Complete pivoting. Searching for the largest element in all rows and columns then switching.ScalingSolve problem of ill-conditioned system.Minimize round-off error
37 Partial PivotingBefore each row is normalized, find the largest available coefficient in the column below the pivot element. The rows can then be switched so that the largest element is the pivot element so that the largest coefficient is used as a pivot.
38 Use of more significant figures to solve for the round-off error :Example. Use Gauss Elimination to solve these 2 equations: (keeping only 4 sig. figures)x x2 =x x2 = 1.000x2 =Solve: x2 = & x1 = 0.0The exact solution is x2 = 2/3 & x1 = 1/3
39 Use of more significant figures to solve for the round-off error :Example (cont’d).
40 Pivoting: ExampleNow, solving the pervious example using the partial pivoting technique:x x2 = 1.000x x2 =The pivot is 1.0x2 =x2 = & x1=0.3333Checking the effect of the # of significant digits:# of dig x2 x Ea% in x1
41 Scaling: ExampleSolve the following equations using naïve gauss elimination:(keeping only 3 sig. figures)2 x1+ 100,000 x2 = 100,000x x2 = 2.0Forward elimination:2 x x2 =- 50,0000 x2 = -50,0000Solve x2 = 1.00 & x1 = 0.00The exact solution is x1 = & x2 =
42 Scaling: Example (cont’d) B) Using the scaling algorithm to solve:2 x1+ 100,000 x2 = 100,000x x2 = 2.0Scaling the first equation by dividing by 100,000:x x2 = 1.0x x2 = 2.0Rows are pivoted:x x2 = 2.0Forward elimination yield:x x2 = 2.0x2 = 1.00Solve: x2 = 1.00 & x1 = 1.00The exact solution is x1 = & x2 =
43 Scaling: Example (cont’d) C) The scaled coefficient indicate that pivoting is necessary.We therefore pivot but retain the original coefficient to give:x x2 = 2.02 x1+ 100,000 x2 = 100,000Forward elimination yields:x x2 = 2.0100,000 x2 = 100,000Solve: x2 = 1.00 & x1 = 1.00Thus, scaling was useful in determining whether pivoting was necessary, but the equation themselves did not require scaling to arrive at a correct result.
44 Determinate Evaluation The determinate can be simply evaluated at the end of the forward elimination step, when the program employs partial pivoting:
45 a) Forward Elimination Example: Gauss Eliminationa) Forward Elimination
48 Example: Gauss Elimination (cont’d) b) Back Substitution
49 Gauss-Jordan Elimination It is a variation of Gauss elimination. The major differences are:When an unknown is eliminated, it is eliminated from all other equations rather than just the subsequent ones.All rows are normalized by dividing them by their pivot elements.Elimination step results in an identity matrix.It is not necessary to employ back substitution to obtain solution.
51 Dividing the 2nd row by 1. 6667 and reducing the second column Dividing the 2nd row by and reducing the second column. (operating above the diagonal as well as below) gives:Divide the 3rd row by and make the elements in the 3rd Column zero.
52 Divide the 4th row by 1.5599 and create zero above the diagonal in the fourth column. Note: Gauss-Jordan method requires almost 50 % more operations than Gauss elimination; therefore it is not recommended to use it.