1. Chapter 5
Orthogonality

2. 1 The scalar product in Rn
The product xTy is called the scalar product of x and y.
In particular, if x=(x1, …, xn)T and y=(y1, …,yn)T, then
xTy=x1y1+x2y2+‥‥+xnyn
The Scalar Product in R2 and R3
Definition
Let x and y be vectors in either R2 or R3. The distance between x and y is defined to be the number ‖x-y‖.

3. Example If x=(3, 4)T and y=(-1, 7)T, then the distance
between x and y is given by
‖y-x‖= 5

4. Theorem 5.1.1 If x and y are two nonzero vectors in either
R2 or R3 and θ is the angle between them, then
(1) xTy=‖x‖‖y‖cosθ
Corollary ( Cauchy-Schwarz Inequality)
If x and y are vectors in either R2 or R3 , then
(2) ︱xTy︱≤‖x‖‖y‖
with equality holding if and only if one of the vectors is 0 or one
vector is a multiple of the other.

5. Definition
The vector x and y in R2 (or R3) are said to be orthogonal if
xTy=0.

6. Example (a) The vector 0 is orthogonal to every vector in R2.
(b) The vectors and are orthogonal in R2.
(c) The vectors and are orthogonal in R3.

7. Scalar and Vector Projectionsx
z=x-p y
p=αu θ u
The scalar is called the scalar projection of x and y, and
the vector p is called the vector projection of x and y.

8. Scalar projection of x onto y:
Vector projection of x onto y:

9. Example The point Q is the point on the line that is
closet to the point (1, 4). Determine the coordinates of Q.
(1, 4) v w Q

10. Orthogonality in Rn
The vectors x and y are said to be orthogonal if xTy=0.

11. 2 Orthogonal Subspaces Definition
Two subspaces X and Y of Rn are said to be orthogonal if xTy=0 for every x∈X and every y∈Y. If X and Y are orthogonal, we write X⊥Y.
Example Let X be the subspace of R3 spanned by e1, and
let Y be the subspace spanned by e2.
Example Let X be the subspace of R3 spanned by e1 and e2,
and let Y be the subspace spanned by e3.

12. Definition
Let Y be a subspace of Rn . The set of all vectors in Rn that are orthogonal to every vector in Y will be denoted Y⊥. Thus
Y⊥={ x∈Rn︱xTy=0 for every y∈Y }
The set Y⊥ is called the orthogonal complement of Y.
Remarks
1. If X and Y are orthogonal subspaces of Rn, then X∩Y={0}.
2. If Y is a subspace of Rn, then Y⊥ is also a subspace of Rn.

13. Fundamental Subspaces
Theorem ( Fundamental Subspaces Theorem)
If A is an m×n matrix, then N(A)=R(AT) ⊥ and N(AT)=R(A) ⊥.
Theorem If S is a subspace of Rn, then
dim S+dim S⊥=n. Furthermore, if {x1, …, xr} is a basis for S and
{xr+1, …, xn} is a basis for S⊥, then {x1, …, xr, xr+1, …, xn}
is a basis for Rn.

14. Theorem 5.2.3 If S is a subspace of Rn, then Rn=S S⊥.
Definition
If U and V are subspaces of a vector space W and each w∈W can be written uniquely as a sum u+v, where u∈U and v∈V, then we say that W is a direct sum of U and V, and we write W=U V.
Theorem If S is a subspace of Rn, then Rn=S S⊥.
Theorem If S is a subspace of Rn, then (S⊥) ⊥=S.

15. Example Let Theorem 5.2.5 If A is an m×n matrix and b∈Rm, then
either there is a vector x∈Rn such that Ax=b or there is a
vector y∈Rm such that ATy=0 and yTb≠0.
Example Let
Find the bases for N(A), R(AT), N(AT), and R(A).

16. 4 Inner Product Spaces Definition
An inner product on a vector space V is an operation on V that assigns to each pair of vectors x and y in V a real number <x, y> satisfying the following conditions:
Ⅰ. <x, x>≥0 with equality if and only if x=0.
Ⅱ. <x, y>=<y, x> for all x and y in V.
Ⅲ. <αx+βy, z>=α<x, z>+β<y, z> for all x, y, z in V and all scalars α and β.

17. The Vector Space Rm×n
Given A and B in Rm×n, we can define an inner product by

18. Basic Properties of Inner product Spaces
If v is a vector in an inner product space V, the length or norm
of v is given by
Theorem ( The Pythagorean Law )
If u and v are orthogonal vectors in an inner product space V,
then

19. Example If
and
then

20. Definition
If u and v are vectors in an inner product space V and v≠0, then the scalar projection of u onto v is given by
and the vector projection of u onto v is given by

21. Theorem 5.4.2 ( The Cauchy- Schwarz Inequality)
If u and v are any two vectors in an inner product space V, then
Equality holds if and only if u and v are linearly dependent.

22. 5 Orthonormal Sets Definition
Let v1, v2, …, vn be nonzero vectors in an inner product space V. If <vi, vj>=0 whenever i≠j, then { v1, v2, …, vn} is said to be an orthogonal set of vectors.
Example The set {(1, 1, 1)T, (2, 1, -3)T, (4, -5, 1)T} is an
orthogonal set in R3.
Theorem If { v1, v2, …, vn} is an orthogonal set of
nonzero vectors in an inner product space V, then v1, v2, …,vn
are linearly independent.

23. Theorem 5.5.2 Let { u1, u2, …, un} be an orthonoemal basis
Definition
An orthonormal set of vectors is an orthogonal set of unit vectors.
The set {u1, u2, …, un} will be orthonormal if and only if
where
Theorem Let { u1, u2, …, un} be an orthonoemal basis
for an inner product space V. If , then ci=<v, ui>.

24. Corollary 5.5.3 Let { u1, u2, …, un} be an orthonoemal basis
for an inner product space V. If and , then
Corollary If { u1, u2, …, un} is an orthonoemal basis
for an inner product space V and , then

25. Orthogonal Matrices Definition
An n×n matrix Q is said to be an orthogonal matrix if the column vectors of Q form an orthonormal set in Rn.
Theorem An n×n matrix Q is orthogonal if and only if
QTQ=I.
Example For any fixed , the matrix
is orthogonal.

26. Properties of Orthogonal Matrices
If Q is an n×n orthogonal matrix, then
(a) The column vectors of Q form an orthonormal basis for Rn.
(b) QTQ=I
(c) QT=Q-1
(d) det(Q)=1 or -1
(e) The thanspose of an orthogonal matrix is an orthogonal
matrix.
(f) The product of two orthogonal matrices is also an orthogonal

27. The Gram-Schmidt Orthogonalization Process
Theorem ( The Gram-Schmidt Process)
Let {x1, x2, …, xn} be a basis for the inner product space V. Let
and define u2, …, un recursively by
for k=1, …, n-1

28. where
pk=<xk+1, u1>u1+<xk+1, u2>+‥‥<xk+1, uk>uk
is the projection of xk+1 onto Span(u1, u2, …, uk). The set
{u1, u2, …, un}
is an orthonormal basis for V.
Example Let
Find an orthonormal basis for the column space of A.