1. Outline Vandermonde Matrix Fourier Matrix Hardmard Matrix
Sparse Matrix

2. Vandermonde Matrix Vandermonde Matrix Why define Vandermonde Matrix?
Each column is a power order
Geometric progression
The first row is all one
Why define Vandermonde Matrix?
Polynomial evaluation of N points

3. Vandermonde Matrix Vandermonde Matrix Determinant
Very nice and important conclusion
Determinant equals to zero if xi = xj for some i ≠ j
Compare the order and coefficient of the LHS and the RHS

4. Fourier Matrix
Fourier Matrix
A Special Case of Vandermonde Matrix

5. Fourier Matrix
Fourier Transform and Inverse Fourier Transform

6. Hardmard Matrix Hardmard Matrix
Recursive Representation of Hardmard Matrix

7. Hardmard Matrix Hardmard Matrix is Orthogonal Matrix:
Prove via Induction
Determinant

8. Hardmard Matrix
For the Matrix with all Elements 1 or -1, the Determinant Bound
Determinant Upper Bound
Upper Bound Achieved if All Columns/Rows are Orthogonal with Each Other

9. Hardmard Matrix Hardmard Transform:
Recursive computation for fast Hardmard transform
O(N*logN) operations

10. Hardmard Matrix Application: CDMA spectrum spreading
Spreading code for each row

11. Sparse Matrix Sparse matrix: vast majority of elements zero
The following matrix is a tiny example

12. Sparse Matrix Example 1: sparse generation matrix/parity check matrix
Information bits x, coded bits y
Linear Encoding:
Linear Check Relation:
Low density parity check: LDPC code
Hy = 0, sparse matrix H
Low density generation matrix: LDGM code
y = Gx, sparse matrix G

13. Sparse Matrix
Example 1: sparse generation matrix/parity check matrix (Continued)
Bipartite Graph Corresponding to the Sparse Matrix
Efficient Encoding/Decoding based on the Sparse Matrix
LDPC/LDGM code: message passing decoding based on the sparse matrix
an edge between node i and node j if and only if Hij = 1
Node i
Node j

14. Sparse Matrix Example 2: compressive sensing
y = Ax, measurement matrix A, sparse vector x
Question: obtain x based on A and y, and the knowledge of sparsity of x
Solution:
min ||x||0, s.t., y = Ax; non-convex opt.
min ||x||1, s.t., y = Ax; convex opt., high complexity
min ||x||2, s.t., y = Ax; convex opt., nonzero elements in the solution vector

15. Sparse Matrix min ||x||0, s.t., y = Ax;
non-convex optimization, not preferred in practice; heuristic approach for a suboptimal solution
min ||x||1, s.t., y = Ax;
convex optimization, high complexity;
piecewise function with a lot of points not differentiable
min ||x||2, s.t., y = Ax;
convex optimization, vast majority of nonzero elements in the solution vector

16. Sparse Matrix Norm-0: min ||x||0, s.t., y = Ax;
The following Simple Example:
Norm-1 solution included by Norm-0 solution
Norm-2 solution deviates from Norm-0 solution