1. Linear Algebra Textbook: J. Hefferon, “Linear Algebra”
Reference: T.M.Apostol, “Linear Algebra”
Website:
Homework submission:

2. Preface Evolution of Linear Algebra:
Linear simultaneous algebraic equations.
Matrix formulation → matrix algebra.
Vector space.
Linear system of differential equations → Inner product space (Hilbert space).
Applications of Linear Algebra:
Minimax (optimization) problems.
Linear operators (eigen-problems).
Linear approximations to non-linear problems.

3. Computerization Procedural programming ( FORTRAN, C, … )
Routine libraries: EISPACK, LINPACK, IMSL, …
Symbolic programming
Mathematica, Maple, Reduce, …
Dedicated packages:
MATLAB, …

4. Being Computer Aided Leave drudgery to the C. Post-C learning process:
Understand the subject:
Reasoning.
Applicability.
Limitations / Error check.
Comparison / relation with other subjects
Apply by hand concept to simple cases. C.
Make knowledge C. searchable.

5. Goals of this Course Concepts: Linear Systems of Equations
Linear (Vector) Spaces
Linear Transformations / Operators
Matrix Algebra & Determinants
Inner Product (Hilbert) Spaces
Techniques:
Mathematical Proofs
Computational Skills

6. Hefferon Linear Systems Vector Spaces Maps Between Spaces Determinants
Similarity

7. Chapter One: Linear Systems
I. Solving Linear Systems
II. Linear Geometry of n-Space
III. Reduced Echelon Form
Topic: Computer Algebra Systems
Topic: Input-Output Analysis
Topic: Accuracy of Computations
Topic: Analyzing Networks

8. I. Solving Linear Systems
I.1. Gauss’ Method
I.2. Describing the Solution Set
I.3. General = Particular + Homogeneous

9. Example of a Linear System: Balances
Find the unknown masses h & c balanced as follows: →
Example of a Linear System: Chemical Reaction
Toluene + Nitric acid → Trinitrotoluene (TNT) Water
甲苯 + 硝酸 → 三硝基甲苯 (黄色炸藥) + 水

10. Definition: An ordered set of n real numbers is called an n-tuple.
Definition: The set of all n-tuples is called Rn.
Definition 1.1a: A linear equation in variables x1, x2, …, xn has the form
(1)
where a1, a2, …, an R are the equation’s coefficients, and dR its constant.
Definition 1.1b: An n-tuple ( s1, s2, …, sn ) R is a solution of, or satisfies, eq(1) if

11. Definition 1.1c: ( s1, s2, …, sn ) is a solution of the system of m linear equations
(2)
if it is a solution of all m equations in eq(2).
Example 1.2: (1, 5 ) is a solution of
since
(5, 1) is not a solution because

12. I.1. Gauss’ Method
Gauss’method  Systematic elimination of variables.
Example 1.3:  
Forward elimination ( upper triangularization ) :
row 1  row 3 :
3 row 1 :
 row 1 + row 2:
Backward substitution (diagonalization) with ρi = row i :
1/3ρ3 ; ρ2 :
2ρ3 + ρ2 :
6ρ2 +ρ1 :

13. Theorem 1.4 (Gauss’ Method) :
A linear system is invariant under the following operations :
ρi  ρj , where i  j.
ρi → c ρi , where c  0.
ρi → ρi + c ρj , where i  j and c  0.
Definition 1.5 (Elementary reduction / row / Gaussian operations):
Swapping.
Multiplying by a scalar.
Rescaling / pivoting.

14. Example 1.6:

15. Leading variable = 1st variable with non-zero coefficient in a row.
Definition 1.9:
Leading variable = 1st variable with non-zero coefficient in a row.
A system is in echelon form if each leading variable is to the right of the one above it.
A linear system needs not have the same number of equations as unknowns. 
Example 1.6:  

16. Example 1.12 ( System with no solution ) :
A linear system can have exactly one, none, or infinitely many solutions.
Example 1.12 ( System with no solution ) :  
Inconsistent.
Example 1.14 ( System with infinitely many solutions ) :  

17. Unique solution: (Triangular)
Unique solution: (Triangular) 
No solution: (Echelon) 
Many solution: (Echelon)

18. Exercise 1:
Four positive integers are given. Select any three of the integers, find their
arithmetic average, and add this result to the fourth integer. Thus the numbers
29, 23, 21, and 17 are obtained. One of the original integers is:
(a) 19 (b) 21 (c) 23 (d) 29 (e) 17
Exercise 2:
Laugh at this: AHAHA + TEHE = TEHAW.
It resulted from substituting a code letter for each digit of a simple example in addition, and it is required to identify the letters and prove the solution unique.

19. I.2. Describing the Solution Set
Example 2.1: System with many solutions.
z = free variable
Solution set =
z = parameter
Definition 2.2: The non-leading variables in an echelon form are free variables.
Variables used to describe a solution set are parameters of the solution set.

20. Example 2.3: System with 2 free variables.
z, w = free variables
Solution set =
z, w = parameters

21. Matrix Definition 2.6 : Matrix
An mn matrix is a rectangular array of numbers with m rows and n columns.
Each number in the matrix is an entry.
Example: A 23 matrix
i = row index j = column index

22. Example 2.7: Linear system
can be abbreviated as 

23. Definition 2.8: Vector
An m-dimensional (column) vector is an m1 matrix.
An n-dimensional row vector is an 1n matrix.
The entries of a vector are its components.
Definition 2.9: Solution
A linear equation
a1x1+ a2x2+…+ an xn = d
of n unknowns x1 , x2 , …, xn is satisfied by a vector s
 a1s1+ a2s2+…+ an sn = d
A vector satisfies a linear system if it satisfies each equation in the system.
Definition 2.10: Vector Addition (Sum)

24. Definition 2.10a: Matrix Addition (Sum)
Definition 2.11: Scalar Multiplication
Example 2.12:

25. Example 2.13:  
Particular solution

26. Possible scenarios:             

27. Exercise:
Solve the following using matrix notations & express the solution sets in vector notations.
Exercise:
Describe all functions f(x) = a x2 + b x + c such that f(1) = 2 & f(1) = 6.
Describe all functions f(x) = a x2 + b x + c such that f(1) = 2
Exercise:
Consider the linear system { a x + y = a2 , x + a y = 1 }.
Find values of a so that it has
1) No solutions. 2) Infinitely many solutions.

28. I.3. General = Particular + Homogeneous
Definition 3.2: Homogeneous Equations
A linear equation is homogeneous if it can be put in the form a1x1 + a2x2 + … + anxn = 0
A linear system of equations is homogeneous if it can be put in the form
Every linear system has an associated homogeneous system obtained by setting all constant terms to zero.
Definition 3.4: Zero vector = 0 with 0i = 0  i.
A homogeneous system is always consistent since 0 is always a solution.
If s is a solution to a homogeneous system, so is c s  cR .

29. From Example 2.13:
satisfy the assoicated homogeneous system:
&
satisfies the inhomogeneous system:

30. Example 3.5: Homogeneous system with unique solution 0.

31. Lemma 3.7:
For any homogeneous linear system, there exists vectors β1 , …, βk s.t.
where k is the number of free variables in an echelon form of the system.
Proof :
Let the system be of m equations in n unknowns with m  n.
By means of the Gauss method, it can be reduced to an echelon form.
Next, we make sure the leading variable of every row is on the diagonal position by moving any offending columns to the rightmost position.
Let there be p equations of the form 0 = 0 at the bottom:
Moving columns means re-ordering the unknowns.

32. By meaning of backward substitution, one gets
With suitable re-labeling of the unknowns, we get
where xj for j = mp+1, …, n are free variables.
By definition:

33. The non-b part is a kk unit matrix.
Proof is completed by setting βj to be the jth column of the nk matrix z
The non-b part is a kk unit matrix.
Particular solution:

34. Lemma 3.8:
Proof:
1. Every solution s can be be written as s = p + h :
Let s be a solution, then sp satisfies the associated homogeneous system since
Setting h = s  p completes the proof.
2. Every s = p + h is a solution :
Theorem 3.1: For any linear system, there exists vectors β1 , …, βk s.t.
where p is a particular solution and k is the number of free variables.

35. Example 3.10: System with empty solution set
Homogeneous solution set =
→ no particular solution
→ no general solution

36. Solution sets of linear systems are either
Corollary 3.11:
Solution sets of linear systems are either
empty (system is inconsistent / example 3.10)
have one element (solution is unique / example 3.5)
have infinitely many elements (system is indeterminate / example 2.13)
Dim( h) = 1
Dim( h) > 1
p exists
Unique
( 1 general solution )
indeterminate
(  general solutions )
p does not exist
inconsistent
( 0 general solutions )
Definition 3.12 : Singular and Nonsingular Matrices
A square matrix is nonsingular if it is the matrix of coefficients of a homogeneous system with a unique solution.
It is singular otherwise, that is, if it is the matrix of coefficients of a homogeneous system with infinitely many solutions.

37. Definition: Rank of a Matrix
The rank of a matrix is equal to the number of non-zero rows in its echelon form.
Let A be an mn matrix, then Rank A  min( m, n ).
Theorem
A square matrix is nonsingular iff its rank is equal to its dimension.

38. Exercises