CHAPTER ONE Matrices and System Equations Objective:To provide solvability conditions of a linear equation Ax=b and introduce the Gaussian elimination method, a systematical approach in solving Ax=b, to solve it.

Outline Motivative Example. Elementary row operations and Elementary Matrices. Some Basic Properties of Matrices. Gaussian Elimination for solving Ax=b. Solvability conditions for Ax=b.

Motivative Example (curve fitting) Given three points( )( )( ),find a polynomial of degree 2 passing through the three given points. Solution: Let the polynomial be Where a,b and c are to be determined Ax=b

Question: Why transform to matrix form? To provide a systematic approach and to use computer resource.

Question: How to solve Ax=b systematically? One way is to put Ax=b in triangular form,which can be easily solved by back-substitution. Definition: A system is said to be in triangular form if in the k-th equation the coefficients of thee first (k-1) variables are all zero and the coefficient of xk is nonzero ( k = 1,…,n)

Eg1:

Solution: By elementary row operations as described below. Question: How to put Ax=b in triangular form while leaving the solution set invariant? Solution: By elementary row operations as described below. Definition: Two systems of equations involing the same variables are said to be equivalent if they have the same solution set.

Before introducing elementary operation, we recall some definitions and notations. (§ 1.3) Equality of two matrices. Multiplication of a matrix by a scalar. Matrix addition. Matrix multiplication. Identity matrix. Multiplicative inverse. Nonsingular and singular matrix. Transpose of a matrix.

Definitions Def. If and , then the where . Matrix Multiplication , where . Def. An (n × n) matrix A is said to be nonsingular or invertible if there exists a matrix B such that AB=BA=I. The matrix B is said to be a multiplicative inverse of A. And B is denoted by A-1. Warning: In general, AB≠BA. Matrix multiplication is not commutative.

Definitions (cont.) Def. The transpose of an (m × n) matrix A is the (n × m) matrix B defined by for j=1,…,n and i=1,…,m. The transpose of A is denoted by AT. Def. An (n × n) matrix A is said to be symmetric if AT=A .

Some Matrix Properties Let be scalars,A,B and C be matrices with proper dimensions. (Commutative Law) (Associative Law) (Distributive Law)

Some Matrix Properties (cont.)

Notations , The matrix is called an augmented matrix. In general, or .

Moreover,we define

Def: Let and .Then is said to be a linear combination of . Note that .We have the next result. Theorem1.3.1: Ax=b is consistent b can be written as a linear combination of colum vectors of A.

Calories Burned Per Hour Application 1: Weight Reduction Table 1 Calories Burned Per Hour Weight in lb Exercise Activity 152 161 170 178 Walking 2 mph 213 225 237 249 Running 5.5 mph 651 688 726 764 Bicycling 5.5mph 304 321 338 356 Tennis 420 441 468 492

Hours Per Day For Each Activity Application 1: Weight Reduction (cont.) Table 2 Hours Per Day For Each Activity Exercise schedule walking Running Bicycling Tennis Monday 1.0 0.0 Tuesday 2.0 Wednesday 0.4 0.5 Thursday Friday

Application 1: Weight Reduction (end) Solution:

Production Costs Per Item (dollars) Application 2: Production Costs Table 3 Production Costs Per Item (dollars) Product Expenses A B C Raw materials 0.1 0.3 0.15 Labor 0.4 0.25 Overhead and miscellaneous 0.2

Amount Produced Per Quarter Application 2: Production Costs (cont.) Table 4 Amount Produced Per Quarter Season Product Summer Fall Winter Spring A 4000 4500 B 2000 2600 2400 2200 C 5800 6200 6000

Application 2: Weight Reduction (cont.) Solution:

Application 2: Weight Reduction (cont.) Solution:

Amount Produced Per Quarter Application 2: Production Costs (end) Solution: Table 5 Amount Produced Per Quarter Season Summer Fall Winter Spring Year Raw materials 1,870 2,160 2,070 1,960 8,060 Labor 3,450 3,940 3,810 3,580 14,780 Overhead and miscellaneous 1,670 1,900 1,830 1,740 7,140 Total production cost 6,990 8,000 7,710 7,280 29,980

Application 5: Networks and Graphs (P.57)

Application 5: Networks and Graphs (cont.) DEF.

Application 5: Networks and Graphs (end) Theorem 1.3.3. If A is an n × n adjacency matrix of a graph and represents the ijth entry of Ak, then is equal to the number of walks of length from to Vi to Vj.

Application 6: Information Retrieval (P.59) Suppose that our database, consists of these book titles: B1. Applied Linear Algebra B2. Elementary Linear Algebra B3. Elementary Linear Algebra with Applications B4. Linear Algebra and Its Applications B5. Linear Algebra with Applications B6. Matrix Algebra with Applications B7. Matrix Theory The collection of key words is given by the following alphabetical list： algebra, application, elementary, linear, matrix, theory

Array Representation for Database of Linear Algebra Books Application 6: Information Retrieval (cont.) Table 8 Array Representation for Database of Linear Algebra Books Books Key Words B1 B2 B3 B4 B5 B6 B7 algebra 1 application elementary linear matrix theory

Application 6: Information Retrieval (end) If the words we are searching for are applied, linear, and algebra, then the database matrix and search vector are given by If we set y= ATx, then

Let’s back to solve Ax=b Eg2

Three types of Elementary row operations. (§ 1.2) Three types of Elementary row operations. I. Interchange two row. II. Multiply a row by . III. Replace a row by its sum with a multiple of another row.

Lead variables and free variables(p.15) Eg: , and are lead variables while and are free variables.

Def. A matrix is said to be in row echelon form if (i) The first nonzero entry in each row is 1. (ii) If row k does not consist entirely of zero, the number of leading zero entries in row k+1 is grater then the number of leading zero entries in row k. (iii) If there are rows whose entries are all zero, they are below the rows having nonzero entries. Def. The process of using row operations I, II, and III to transform a linear system into one whose augmented matrix is in row echelon form is called Gaussian elimination.

Overdetermined and Underdetermined Def. A linear system is said to be overdetermined if there are more equations(m) than unknowns (n). (m > n) Warning: Overdetermined systems are usually (but not always) in consistent. Def. A system of m linear equations in n unknowns is said to be underdetermined if there are fewer equations. (m < n)

Reduced Row Echelon Form Def. A matrix is said to be in reduced row echelon form if: (i) The matrix is in row echelon form. (ii) The first nonzero entry in each row is the only nonzero entry in its column. Def. The process of using elementary row operations to transform a matrix into reduced row echelon form is called Gauss-Jordan reduction.

Application 2: Electrical Networks (P.22)

Application 2: Electrical Networks (end) Kirchhoff’s Laws: 1. At every node the sum of the incoming currents equals the sum of the outgoing currents. 2. Around every closed loop the algebraic sum of the voltage must equal the algebraic sum of the voltage drops.

Application 4: Economic Models For Exchange of Goods (P.25) F M C F M C 1/2 1/4 1/3 1/2 1/4

(§ 1.4) Elementary Matrices Type I ( ): Obtained by interchanging rows i and j from identity matrix. Type II ( ): Obtained from identity matrix by multiplying row i with . Type III ( ): Obtained from identity matrix by adding to row j.

Elementary Row / Column Operation means performing type I row operation on A. means performing type II row operation on A. means performing type III row operation on A. means performing type I column operation on A. means performing type II column operation on A. means performing type III column operation on A.

Theorem1.4.2: If E is an elementary matrix, then E is nonsingular and E-1 is an elementary matrix of the same type. With The solution set of a linear equations is invariant under three types row operation.  and have the solution set.

Def. A matrix B is row equivalent to A if there exists a Row Equivalent (P.71) Def. A matrix B is row equivalent to A if there exists a finite sequence of elementary matrices such that Theorem1.4.3 (a) A is nonsingular. (b) Ax=0 has only the trivial solution 0. (c) A is row equivalent to I.

Proof of Theorem 1.4.3 (a) (b) Let be a solution of Ax=0. (b) (c) Let A ~ U, where U is in reduced row echelon form. Suppose U contains a zero row. by Th1.2.1, Ux=0 has a nontrivial solution thus A~I. (c) (a) A~I  A= E1 …… Ek for some E1 … Ek ∵ each Ei is nonsingular. ∴ A is nonsingular. (by Th.1.2.1) row

Corollary1.4.4 Ax=b has a unique solution A is nonsingular. Pf: " “ The unique solution is . " " Suppose is the unique solution and A is singular. is also a solution of Ax=b. A is nonsingular.

BUT in general, and AB=AC B=C. Eg. Moreover,AC=AB while .

Method For Computing = (I | Ek…E1‧I) ( by ) = (I | A-1) If A is nonsingular and row equivalent to I, so there exists elementary matrices such that then, Ek…E1(A | I)= (Ek…E1‧A | Ek…E1‧I) ( by ) = (I | Ek…E1‧I) ( by ) = (I | A-1) 2004 NCTU ECE Linear Algebra

Example 4. (P.73) Sol: Q: Compute A-1 if . 2004 NCTU ECE Linear Algebra

Example 4. (cont.) Sol: Q: Compute A-1 if . 2004 NCTU ECE Linear Algebra

Diagonal and Triangular Matrices Def. An n × n matrix A is said to be upper triangular if aij=0 for i > j and lower triangular if aij=0 for i > j. Def. An n × n matrix B is diagonal if aij=0 whenever i ≠ j. Triangular Factorization If an n × n matrix C can be reduced to upper triangular form using only row operation III, then C has an LU factorization. The matrix L is unit lower triangular, and if i > j, then lij is the multiple of t he jth row subtracted from the ith row during the reduction process. 2004 NCTU ECE Linear Algebra

Example 6. (P.74) row operation III Mark: 2004 NCTU ECE Linear Algebra

Block Multiplication Let A be an m × n matrix and B is an n × r matrix. It is often useful to partition A and B and express the product in terms of the submatrices of A and B. In general, partition B into columns then partition A into rows , then 2004 NCTU ECE Linear Algebra

Block Multiplication (cont.) Case 1. Case 2. Case 3. 2004 NCTU ECE Linear Algebra

Block Multiplication (cont.) Case 4. Let then 2004 NCTU ECE Linear Algebra

Example 2. (P.85) Let A be an n × n matrix of the form , where A11 is a k × k matrix (k < n ) . Show that A is nonsingular if and only if A11 and A22 are nonsingular. Solution: 2004 NCTU ECE Linear Algebra

Scalar / Inner Product Give two vectors , This product is referred to as a scalar product or an inner product. 2004 NCTU ECE Linear Algebra

Outer Product Give two vectors , The product is referred to as the outer product of . 2004 NCTU ECE Linear Algebra

Outer Product Expansion Suppose that , then This representation is referred to as an outer product expansion . 2004 NCTU ECE Linear Algebra