Download presentation
Presentation is loading. Please wait.
Published byBeverly Stone Modified over 7 years ago
1
Rutgers University School of Engineering Spring :440:127 - Introduction to Computers for Engineers Sophocles J. Orfanidis ECE Department week 10
2
Weekly Topics Week Basics – variables, arrays, matrices, plotting (ch. 2 & 3) Week Basics – operators, functions, strings, cells (ch. 2,3,11) Week Matrices (ch. 4) Week Plotting – 2D and 3D plots (ch. 5) Week User-defined functions (ch. 6) Week Input-output formatting – fprintf, fscanf (ch. 7) – Exam 1 Week Relational & logical operators, if, switch statements (ch. 8) Week For-loops, while-loops (ch. 9) Week Review for weeks 5-8 Week 10 - Matrix algebra – solving linear equations (ch. 10) – Exam 2 Week 11 - Numerical methods I (ch. 13) Week 12 - Numerical methods II (ch. 13) Week 13 - Strings, structures, cell arrays, symbolic math (ch. 11,12) Week 14 – Exam 3 References: H. Moore, MATLAB for Engineers, 4/e, Prentice Hall, 2014 G. Recktenwald, Numerical Methods with MATLAB, Prentice Hall, 2000 A. Gilat, MATLAB, An Introduction with Applications, 4/e, Wiley, 2011
3
Matrix Algebra dot product matrix-vector multiplication
matrix-matrix multiplication matrix inverse solving linear systems least-squares solutions determinant, rank, condition number vector & matrix norms iterative solutions of linear systems examples electric circuits temperature distributions
4
Operators and Expressions
operation element-wise matrix-wise addition subtraction multiplication * * division / / left division \ \ exponentiation ^ ^ transpose w/o complex conjugation .' transpose with complex conjugation ' >> help / >> help precedence used in matrix algebra operations
5
>> [A, A.^2; A^2, A*A] % form sub-blocks ans = 1 2 1 4 3 4 9 16
>> [A, A.^2; A^2, A*A] % form sub-blocks ans = % note A^2 = A*A >> B = 10.^A; >> [B, log10(B)]
6
>> size(A) % [N,M] = size(A), NxM matrix
matrix indexing convention >> A = [1 2 3; 2 0 4; 0 8 5] A = >> size(A) % [N,M] = size(A), NxM matrix ans =
7
dot product The dot product is the basic operation in
matrix-vector and matrix-matrix multiplications dot product a, b must have the same dimension math notations MATLAB notation
8
for complex-valued vectors
dot product for complex-valued vectors complex-conjugate transpose, or, hermitian conjugate of a for real-valued vectors, the operations ' and .' are equivalent math notations MATLAB notation
9
>> a = [1; 2; -3]; b = [4; -5; 2];
ans = -12 >> dot(a,b) % built-in function ans = % same as sum(a.*b)
10
matrix-vector multiplication
combine three dot product operations into a single matrix-vector multiplication
11
A x = b matrix-vector multiplication combine three dot product
operations into a single matrix-vector multiplication A x = b
12
matrix-matrix multiplication
combine three matrix-vector multiplications into a single matrix-matrix multiplication
13
>> A = [4 1 2; ; 2 1 1] A = >> B = [ ; ; ] B = >> C = A*B C =
14
C(i,j) is the dot product of i-th row of A with j-th column of B
note: A*B ¹ B*A
15
(NxK)x(KxM) --> NxM
Rule of thumb: (NxK)x(KxM) --> NxM A is NxK B is KxM then, A*B is NxM
16
vector-vector multiplication
(1x3)x(3x1) --> 1x1 = scalar row * column = scalar (3x1)x(1x3) --> 3x3 column * row = matrix
17
vector-vector multiplication
>> [1, 2, 3] * [ ]' ans = -7 >> [1, 2, 3]' * [ ] row x column = scalar column x row = matrix
18
solving linear systems
A x = b solving linear systems Linear equations have a very large number of applications in engineering, science, social sciences, and economics Linear Programming – Management Science Computer Aided Design – aerodynamics of cars, planes Signal Processing, Communications, Control, Radar, Sonar, Electromagnetics, Oil Exploration, Computer Vision, Pattern & Face Recognition Chip Design – millions of transistors on a chip Economic Models, Finance, Statistical Models, Data Mining, Social Models, Financial Engineering Markov Models – Speech, Biology, Google Pagerank Scientific Computing – solving very large problems the only practical way to solve very large systems is iteratively
19
solving linear systems
matrix inverse always use the backslash operator to solve a linear system, instead of inv(A)
20
solving linear systems (using backslash)
>> x = A\b % solution of A*x = b x = % x = A^-1 * b % x = inv(A) * b 2 3 >> norm(A*x-b) % test - should be zero ans = % or, of the order of eps
21
solving linear systems (using inv)
>> b = [4 8 9]'; >> inv(A) % same as A^(-1) ans = >> x = inv(A) * b % but prefer backslash x = % same as x = A^-1 * b 1.0000 2.0000 3.0000 >> norm(A*x-b) 1.8310e-015 >> inv(sym(A)) ans = [ 18/31, -4/31, -1/31] [ -5/31, 8/31, 2/31] [ -7/31, 5/31, 9/31]
22
A of size NxN and invertible X of size NxK B of size NxK
solving linear systems – back-slash and forward-slash A of size NxN and invertible X of size NxK B of size NxK AX = B --> X = A\B = inv(A)*B equivalent A of size NxN and invertible X of size KxN B of size KxN XA = B --> X = B/A = B*inv(A) equivalent A X = B X A = B
23
x may or may not be unique
solving linear systems – least-squares solutions A of size NxM x of size Mx1 column b of size Nx1 column x = A\b will be used in in weeks 11 & 12 x=A\b is a solution of Ax=b in a least-squares sense, i.e., x minimizes the norm squared of the error e = b – A*x: (b-Ax)'*(b-Ax) = min x may or may not be unique depending on whether the linear system Ax=b is over-determined, under-determined, or whether A has full rank or not pseudo-inverse x = pinv(A)*b; >> help \ >> help pinv
24
A = NxM matrix A' = MxN matrix x = Mx1 column A'*A = MxM matrix
Fundamental Theorem of Linear Algebra – what is it? least-squares solutions - summary A = NxM matrix A' = MxN matrix x = Mx1 column A'*A = MxM matrix b = Nx1 column A'*b = Mx1 column Assuming full rank for A, we have the following cases: 1. N>M, overdetermined case, (most common in practice) x = A\b = unique least-squares solution, same as x = pinv(A)*b, and x = (A'*A)^(-1) * (A'*b) 2. N<M, underdetermined case, (there are many solutions) x=A\b, x=pinv(A)*b, are two possible solutions 3. N=M, square invertible case, x is unique x = A\b is equivalent to x = A^(-1)*b x = A\b is numerically the most accurate method
25
least-squares solutions - example
% overdetermined % full-rank example A = [1 2; 3 4; 5 6] b = [4, 3, 8]'; x = A\b % x = pinv(A)*b % x = (A'*A)\(A'*b) x = -1 2
26
least-squares solutions - example
inverse exists because A was assumed to have full rank minimized value of J achieved at x = x_opt strictly positive-definite quadratic form because of full rank of A, vanishes only at x = x_opt
27
least-squares solutions - example
b = [4, 3, 8]'; x_opt = (A'*A)\(A'*b) J_min = b'*b - ... b'*A*inv(A'*A)*A'*b x_opt = -1 2 J_min = 6
28
least-squares solutions - example
29
least-squares solutions - example
% we can also minimize J with fminsearch, % i.e., the multivariable version of fminbnd J 35*(x(1)+1).^2 +... 88*(x(1)+1).*(x(2)-2)+... 56*(x(2)-2).^2 + 6; x0 = [0,0]'; % arbitrary initial search point [xmin,Jmin] = fminsearch(J,x0) % xmin = % Jmin = 6 % %
30
Invertibility, rank, determinants, condition number
The inverse inv(A) of an NxN square matrix A exists if its determinant is non-zero, or, equivalently if it has full rank, i.e., when its rank is equal to the row or column dimension N >> doc inv >> doc det >> doc rank >> doc cond a = [1 2 3]'; b = [4 5 6]'; A = [a, a+b, b] A = >> det(A) ans = >> rank(A) 2 det(A) = 0
31
Invertibility, rank, determinants, condition number
The larger the cond(A) the more ill-conditioned the linear system, and the less reliable the solution. A = [1, 5, 2, e-8, 5 3, 9, ]; >> cond(A) ans = 3.3227e+009 A\[1; 2; 3] ans = 1 A\[1.001; ; ] ans = det(A) = e-008
32
Determinant and inverse of a 2x2 matrix
Example:
33
Matrix Exponential Used widely in solving linear dynamic systems
>> expm(A) % matrix exponential ans = >> exp(A) % element-wise exponential >> doc expm >> doc exp
34
L1, L2, and L¥ norms of a vector
Vector & Matrix Norms >> doc norm L1, L2, and L¥ norms of a vector used as distance measure between two vectors or matrices L1 norm Euclidean, L2 norm L¥ norm
35
useful for comparing two vectors or matrices
x = [1, -4, 5, 3]; p = inf; switch p case 1 N = sum(abs(x)); % N = norm(x,1); case 2 N = sqrt(sum(abs(x).^2)); % N = norm(x,2); case inf N = max(abs(x)); % N = norm(x,inf); otherwise end useful for comparing two vectors or matrices >> norm(a-b) % a,b vectors of same size >> norm(A-B) % A,B matrices of same size equivalent calculation using the built-in function norm :
36
Euclidean distance dot product
37
Electric Circuits Kirchhoff’s Voltage Law
38
Electric Circuits
39
A x = b
40
A = [25, -15, -10; -15, 30, -15; -10, -15, 30] b = [-7.5; 15; -5] A = b = x = A\b x = 0.5000 1.0000
41
inv(A) ans = inv(sym(A)) --> (1/105) * [ ]
42
Iterative solutions of linear systems Ax=b
the only practical way to solve very large linear systems is iteratively Methods: Jacobi method Gauss-Seidel method Relaxation methods Conjugate Gradient method Others G. H. Golub and C. F. Van Loan, Matrix Computations, 3/e, JHU Press, 1996. D. S. Watkins, Fundamentals of Matrix Computations, 2/e, Wiley, 2002. L. N. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, 1997. A. Bjork, Numerical Methods for Least Squares Problems, SIAM, 1996.
43
scalar example illustrating the Jacobi method rearrange rearrange
turn it into a recursion
44
x(1)=0; % arbitrary for k=1:19 x(k+1) = -0.5*x(k) + 6; end
x(1)=0; % arbitrary for k=1:19 x(k+1) = -0.5*x(k) + 6; end k=1:20; plot(k,x,'b.-');
45
if abs(xnew-x)<=tol break; end x = xnew; k = k+1;
tol=1e-10; x0=0; x=x0; k=1; while 1 xnew = -0.5*x + 6; if abs(xnew-x)<=tol break; end x = xnew; k = k+1; k, abs(x-4) k = 37 ans = 5.8208e-011 tol=1e-10; x0=0; x=x0; k=1; xnew = -0.5*x+6; while abs(xnew-x)>tol x = xnew; k = k+1; xnew = -0.5*x + 6; end k, abs(x-4) k = 37 ans = 5.8208e-011 forever while loop conventional while loop
46
More general version of ax=b, where a,x,b are scalars
choose a splitting a = d – r, such that | r/d | < 1 iterative algorithm: it has solution that converges to b/a: key assumption that guarantees convergence this is the scalar version of the Jacobi and Gauss-Seidel methods
47
Jacobi’s method for A x = b
Define the following ‘Jacobi iteration parameters’ Jacobi iteration:
48
Convergence requires the condition that the so-called ‘spectral radius’ of B be strictly less than unity: This is hard to calculate for large matrices, but a sufficient condition for convergence is that a matrix norm of B, such as the L1, L2, L¥, or Frobenius norms, be less than unity because of the inequality: Note: if the method fails for a system A,b, then, try it on the rearranged system A --> flipud(A) b --> flipud(b) which sometimes works so that
49
Jacobi iteration: Exact solution demonstrates the convergence:
50
Jacobi iteration – alternative form:
Jacobi iteration – relaxation form: relaxation parameter
51
MATLAB implementation
% given A = NxN matrix, b = Nx1 vector % construct the Jacobi parameters I = eye(size(A)); % identity matrix D = diag(diag(A)); % diagonal part of A B = I - D\A; % iteration matrix c = D\b; % Nx1 vector rho = max(abs(eig(B))); % check if rho < 1 Next, implement the Jacobi iteration using a forever while-loop with a stopping condition,
52
MATLAB implementation – v.1
tol = 1e-10; % error tolerance, our choice x = x0; k = 1; % x0 = arbitrary Nx1 vector while % forever loop xnew = B*x + c; if norm(xnew-x) < tol break; end x = xnew; k = k+1; k, x, norm(A*x-b) % print & verify solution norm() measures the difference between the vectors xnew and x, could also use L¥ norm max(abs(xnew-x)) Note that it breaks out of the loop just before the tolerance is reached, one more iteration after the loop would meet or exceed the tolerance.
53
MATLAB implementation – v.2
% in order to save individual iterates x(k), % use a matrix X whose columns are x(k) tol = 1e-10; % error tolerance X(:,1)=x0; k=1; % x0 = arbitrary Nx1 vector while 1 X(:,k+1) = B*X(:,k) + c; if norm(X(:,k+1)-X(:,k) ) < tol break; end k = k+1; k, x = X(:,k) % x = converged solution norm(A*x-b) % verify solution
54
MATLAB implementation – v.3
% another version that saves the iterates tol=1e-10; % error tolerance x=x0; k=1; % x0 = arbitrary Nx1 vector X=x; % X columns are the iterates x(k) while 1 xnew = B*x + c; if norm(xnew-x) < tol break; end x = xnew; k = k+1; X = [X,x]; % append new column to X k, x, norm(A*x-b) % print & verify solution
55
Example A = [2 1 0; 1 5 -1; 1 -2 4]; b = [4 8 9]'; xsol = A\b;
I = eye(size(A)); % 3x3 identity matrix D = diag(diag(A)); % diagonal part of A B = I - D\A; % 3x3 iteration matrix c = D\b; % 3x1 vector rho = max(abs(eig(B)));
56
>> D, B >> c, rho, norm(B,inf)
B = c = 2.00 1.60 2.25 rho = 0.5000 ans = 0.7500
57
tol=1e-12; % error tolerance
x=[0 1 2]'; k=1; % x0 = [0 1 2]' = arbitrary X=x; % X = save all iterates while 1 xnew = B*x + c; if norm(xnew-x)<tol % norm() measures break; % the distance end % between x,xnew x = xnew; k = k+1; X = [X,x]; % append new column end k, x, norm(A*x-b), norm(x-xsol), size(X)
58
>> k, x, norm(A*x-b), norm(x-xsol), size(X)
40 x = 1.0000 2.0000 3.0000 ans = 2.6657e-012 1.3634e-012 3 40 >> K=1:k; >> plot(K,X');
59
Temperature Distribution
follows from discretizing the Laplace equation
60
A x = b
61
A x = b >> x = A\b x = >> A = [ 4 -1 -1 0 20.0000
>> A = [ ]; >> b = [30; 60; 40; 70];
62
display solution T in a rectangular pattern
nodes were numbered in column order T = zeros(2,2); % shape of T T(:) = x T =
63
A x = b Rules for constructing A and b: main diagonal is 4
if nodes i,j are connected, then, -1, otherwise, 0 3) b(i) is sum of boundary values connected to node i
64
Iterative Solution convenient for large number of subdivisions
used also to solve 2D electrostatics problems
65
iterate over internal nodes only
N=4; M=4; left=20; right=30; up=10; dn=40; T(1,:) = repmat(up,1,M); T(N,:) = repmat(dn,1,M); T(:,1) = repmat(left,N,1); T(:,M) = repmat(right,N,1); Tnew = T; tol = 1e-4; K = 100; for k=1:K, for i=2:N-1, for j=2:M-1, Tnew(i,j) = (T(i-1,j) + T(i+1,j) +... T(i,j-1) + T(i,j+1))/4; end if norm(Tnew-T) < tol, break; end T = Tnew; T(1,[1,end]) = nan; T(end,[1,end]) = nan; boundary values iterate over internal nodes only
66
T = % start-up NaN NaN NaN NaN % converged after k = 19 iterations % to within the specified tol = 1e-4 T = NaN NaN NaN NaN
67
T = % after k=1 iteration NaN NaN NaN NaN T = % after k=2 iterations T = % after k=3 iterations
68
N=30; M=30; left=0; right=0; up=0; dn=60; tol = 1e-6; K = 5000; % breaks out at k = 2475 [X,Y] = meshgrid(2:M-1, 2:N-1); Z = T(2:M-1, 2:N-1); surf(X,Y,Z);
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.