Presentation is loading. Please wait.

Presentation is loading. Please wait.

Rutgers University School of Engineering Spring 2015 14:440:127 - Introduction to Computers for Engineers Sophocles J. Orfanidis ECE Department orfanidi@ece.rutgers.edu.

Similar presentations


Presentation on theme: "Rutgers University School of Engineering Spring 2015 14:440:127 - Introduction to Computers for Engineers Sophocles J. Orfanidis ECE Department orfanidi@ece.rutgers.edu."— Presentation transcript:

1 Rutgers University School of Engineering Spring :440:127 - Introduction to Computers for Engineers Sophocles J. Orfanidis ECE Department week 10

2 Weekly Topics Week Basics – variables, arrays, matrices, plotting (ch. 2 & 3) Week Basics – operators, functions, strings, cells (ch. 2,3,11) Week Matrices (ch. 4) Week Plotting – 2D and 3D plots (ch. 5) Week User-defined functions (ch. 6) Week Input-output formatting – fprintf, fscanf (ch. 7) – Exam 1 Week Relational & logical operators, if, switch statements (ch. 8) Week For-loops, while-loops (ch. 9) Week Review for weeks 5-8 Week 10 - Matrix algebra – solving linear equations (ch. 10) – Exam 2 Week 11 - Numerical methods I (ch. 13) Week 12 - Numerical methods II (ch. 13) Week 13 - Strings, structures, cell arrays, symbolic math (ch. 11,12) Week 14 – Exam 3 References: H. Moore, MATLAB for Engineers, 4/e, Prentice Hall, 2014 G. Recktenwald, Numerical Methods with MATLAB, Prentice Hall, 2000 A. Gilat, MATLAB, An Introduction with Applications, 4/e, Wiley, 2011

3 Matrix Algebra dot product matrix-vector multiplication
matrix-matrix multiplication matrix inverse solving linear systems least-squares solutions determinant, rank, condition number vector & matrix norms iterative solutions of linear systems examples electric circuits temperature distributions

4 Operators and Expressions
operation element-wise matrix-wise addition subtraction multiplication * * division / / left division \ \ exponentiation ^ ^ transpose w/o complex conjugation .' transpose with complex conjugation ' >> help / >> help precedence used in matrix algebra operations

5 >> [A, A.^2; A^2, A*A] % form sub-blocks ans = 1 2 1 4 3 4 9 16
>> [A, A.^2; A^2, A*A] % form sub-blocks ans = % note A^2 = A*A >> B = 10.^A; >> [B, log10(B)]

6 >> size(A) % [N,M] = size(A), NxM matrix
matrix indexing convention >> A = [1 2 3; 2 0 4; 0 8 5] A = >> size(A) % [N,M] = size(A), NxM matrix ans =

7 dot product The dot product is the basic operation in
matrix-vector and matrix-matrix multiplications dot product a, b must have the same dimension math notations MATLAB notation

8 for complex-valued vectors
dot product for complex-valued vectors complex-conjugate transpose, or, hermitian conjugate of a for real-valued vectors, the operations ' and .' are equivalent math notations MATLAB notation

9 >> a = [1; 2; -3]; b = [4; -5; 2];
ans = -12 >> dot(a,b) % built-in function ans = % same as sum(a.*b)

10 matrix-vector multiplication
combine three dot product operations into a single matrix-vector multiplication

11 A x = b matrix-vector multiplication combine three dot product
operations into a single matrix-vector multiplication A x = b

12 matrix-matrix multiplication
combine three matrix-vector multiplications into a single matrix-matrix multiplication

13 >> A = [4 1 2; ; 2 1 1] A = >> B = [ ; ; ] B = >> C = A*B C =

14 C(i,j) is the dot product of i-th row of A with j-th column of B
note: A*B ¹ B*A

15 (NxK)x(KxM) --> NxM
Rule of thumb: (NxK)x(KxM) --> NxM A is NxK B is KxM then, A*B is NxM

16 vector-vector multiplication
(1x3)x(3x1) --> 1x1 = scalar row * column = scalar (3x1)x(1x3) --> 3x3 column * row = matrix

17 vector-vector multiplication
>> [1, 2, 3] * [ ]' ans = -7 >> [1, 2, 3]' * [ ] row x column = scalar column x row = matrix

18 solving linear systems
A x = b solving linear systems Linear equations have a very large number of applications in engineering, science, social sciences, and economics Linear Programming – Management Science Computer Aided Design – aerodynamics of cars, planes Signal Processing, Communications, Control, Radar, Sonar, Electromagnetics, Oil Exploration, Computer Vision, Pattern & Face Recognition Chip Design – millions of transistors on a chip Economic Models, Finance, Statistical Models, Data Mining, Social Models, Financial Engineering Markov Models – Speech, Biology, Google Pagerank Scientific Computing – solving very large problems the only practical way to solve very large systems is iteratively

19 solving linear systems
matrix inverse always use the backslash operator to solve a linear system, instead of inv(A)

20 solving linear systems (using backslash)
>> x = A\b % solution of A*x = b x = % x = A^-1 * b % x = inv(A) * b 2 3 >> norm(A*x-b) % test - should be zero ans = % or, of the order of eps

21 solving linear systems (using inv)
>> b = [4 8 9]'; >> inv(A) % same as A^(-1) ans = >> x = inv(A) * b % but prefer backslash x = % same as x = A^-1 * b 1.0000 2.0000 3.0000 >> norm(A*x-b) 1.8310e-015 >> inv(sym(A)) ans = [ 18/31, -4/31, -1/31] [ -5/31, 8/31, 2/31] [ -7/31, 5/31, 9/31]

22 A of size NxN and invertible X of size NxK B of size NxK
solving linear systems – back-slash and forward-slash A of size NxN and invertible X of size NxK B of size NxK AX = B --> X = A\B = inv(A)*B equivalent A of size NxN and invertible X of size KxN B of size KxN XA = B --> X = B/A = B*inv(A) equivalent A X = B X A = B

23 x may or may not be unique
solving linear systems – least-squares solutions A of size NxM x of size Mx1 column b of size Nx1 column x = A\b will be used in in weeks 11 & 12 x=A\b is a solution of Ax=b in a least-squares sense, i.e., x minimizes the norm squared of the error e = b – A*x: (b-Ax)'*(b-Ax) = min x may or may not be unique depending on whether the linear system Ax=b is over-determined, under-determined, or whether A has full rank or not pseudo-inverse x = pinv(A)*b; >> help \ >> help pinv

24 A = NxM matrix A' = MxN matrix x = Mx1 column A'*A = MxM matrix
Fundamental Theorem of Linear Algebra – what is it? least-squares solutions - summary A = NxM matrix A' = MxN matrix x = Mx1 column A'*A = MxM matrix b = Nx1 column A'*b = Mx1 column Assuming full rank for A, we have the following cases: 1. N>M, overdetermined case, (most common in practice) x = A\b = unique least-squares solution, same as x = pinv(A)*b, and x = (A'*A)^(-1) * (A'*b) 2. N<M, underdetermined case, (there are many solutions) x=A\b, x=pinv(A)*b, are two possible solutions 3. N=M, square invertible case, x is unique x = A\b is equivalent to x = A^(-1)*b x = A\b is numerically the most accurate method

25 least-squares solutions - example
% overdetermined % full-rank example A = [1 2; 3 4; 5 6] b = [4, 3, 8]'; x = A\b % x = pinv(A)*b % x = (A'*A)\(A'*b) x = -1 2

26 least-squares solutions - example
inverse exists because A was assumed to have full rank minimized value of J achieved at x = x_opt strictly positive-definite quadratic form because of full rank of A, vanishes only at x = x_opt

27 least-squares solutions - example
b = [4, 3, 8]'; x_opt = (A'*A)\(A'*b) J_min = b'*b - ... b'*A*inv(A'*A)*A'*b x_opt = -1 2 J_min = 6

28 least-squares solutions - example

29 least-squares solutions - example
% we can also minimize J with fminsearch, % i.e., the multivariable version of fminbnd J 35*(x(1)+1).^2 +... 88*(x(1)+1).*(x(2)-2)+... 56*(x(2)-2).^2 + 6; x0 = [0,0]'; % arbitrary initial search point [xmin,Jmin] = fminsearch(J,x0) % xmin = % Jmin = 6 % %

30 Invertibility, rank, determinants, condition number
The inverse inv(A) of an NxN square matrix A exists if its determinant is non-zero, or, equivalently if it has full rank, i.e., when its rank is equal to the row or column dimension N >> doc inv >> doc det >> doc rank >> doc cond a = [1 2 3]'; b = [4 5 6]'; A = [a, a+b, b] A = >> det(A) ans = >> rank(A) 2 det(A) = 0

31 Invertibility, rank, determinants, condition number
The larger the cond(A) the more ill-conditioned the linear system, and the less reliable the solution. A = [1, 5, 2, e-8, 5 3, 9, ]; >> cond(A) ans = 3.3227e+009 A\[1; 2; 3] ans = 1 A\[1.001; ; ] ans = det(A) = e-008

32 Determinant and inverse of a 2x2 matrix
Example:

33 Matrix Exponential Used widely in solving linear dynamic systems
>> expm(A) % matrix exponential ans = >> exp(A) % element-wise exponential >> doc expm >> doc exp

34 L1, L2, and L¥ norms of a vector
Vector & Matrix Norms >> doc norm L1, L2, and L¥ norms of a vector used as distance measure between two vectors or matrices L1 norm Euclidean, L2 norm L¥ norm

35 useful for comparing two vectors or matrices
x = [1, -4, 5, 3]; p = inf; switch p case 1 N = sum(abs(x)); % N = norm(x,1); case 2 N = sqrt(sum(abs(x).^2)); % N = norm(x,2); case inf N = max(abs(x)); % N = norm(x,inf); otherwise end useful for comparing two vectors or matrices >> norm(a-b) % a,b vectors of same size >> norm(A-B) % A,B matrices of same size equivalent calculation using the built-in function norm :

36 Euclidean distance dot product

37 Electric Circuits Kirchhoff’s Voltage Law

38 Electric Circuits

39 A x = b

40 A = [25, -15, -10; -15, 30, -15; -10, -15, 30] b = [-7.5; 15; -5] A = b = x = A\b x = 0.5000 1.0000

41 inv(A) ans = inv(sym(A)) --> (1/105) * [ ]

42 Iterative solutions of linear systems Ax=b
the only practical way to solve very large linear systems is iteratively Methods: Jacobi method Gauss-Seidel method Relaxation methods Conjugate Gradient method Others G. H. Golub and C. F. Van Loan, Matrix Computations, 3/e, JHU Press, 1996. D. S. Watkins, Fundamentals of Matrix Computations, 2/e, Wiley, 2002. L. N. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, 1997. A. Bjork, Numerical Methods for Least Squares Problems, SIAM, 1996.

43 scalar example illustrating the Jacobi method rearrange rearrange
turn it into a recursion

44 x(1)=0; % arbitrary for k=1:19 x(k+1) = -0.5*x(k) + 6; end
x(1)=0; % arbitrary for k=1:19 x(k+1) = -0.5*x(k) + 6; end k=1:20; plot(k,x,'b.-');

45 if abs(xnew-x)<=tol break; end x = xnew; k = k+1;
tol=1e-10; x0=0; x=x0; k=1; while 1 xnew = -0.5*x + 6; if abs(xnew-x)<=tol break; end x = xnew; k = k+1; k, abs(x-4) k = 37 ans = 5.8208e-011 tol=1e-10; x0=0; x=x0; k=1; xnew = -0.5*x+6; while abs(xnew-x)>tol x = xnew; k = k+1; xnew = -0.5*x + 6; end k, abs(x-4) k = 37 ans = 5.8208e-011 forever while loop conventional while loop

46 More general version of ax=b, where a,x,b are scalars
choose a splitting a = d – r, such that | r/d | < 1 iterative algorithm: it has solution that converges to b/a: key assumption that guarantees convergence this is the scalar version of the Jacobi and Gauss-Seidel methods

47 Jacobi’s method for A x = b
Define the following ‘Jacobi iteration parameters’ Jacobi iteration:

48 Convergence requires the condition that the so-called ‘spectral radius’ of B be strictly less than unity: This is hard to calculate for large matrices, but a sufficient condition for convergence is that a matrix norm of B, such as the L1, L2, L¥, or Frobenius norms, be less than unity because of the inequality: Note: if the method fails for a system A,b, then, try it on the rearranged system A --> flipud(A) b --> flipud(b) which sometimes works so that

49 Jacobi iteration: Exact solution demonstrates the convergence:

50 Jacobi iteration – alternative form:
Jacobi iteration – relaxation form: relaxation parameter

51 MATLAB implementation
% given A = NxN matrix, b = Nx1 vector % construct the Jacobi parameters I = eye(size(A)); % identity matrix D = diag(diag(A)); % diagonal part of A B = I - D\A; % iteration matrix c = D\b; % Nx1 vector rho = max(abs(eig(B))); % check if rho < 1 Next, implement the Jacobi iteration using a forever while-loop with a stopping condition,

52 MATLAB implementation – v.1
tol = 1e-10; % error tolerance, our choice x = x0; k = 1; % x0 = arbitrary Nx1 vector while % forever loop xnew = B*x + c; if norm(xnew-x) < tol break; end x = xnew; k = k+1; k, x, norm(A*x-b) % print & verify solution norm() measures the difference between the vectors xnew and x, could also use L¥ norm max(abs(xnew-x)) Note that it breaks out of the loop just before the tolerance is reached, one more iteration after the loop would meet or exceed the tolerance.

53 MATLAB implementation – v.2
% in order to save individual iterates x(k), % use a matrix X whose columns are x(k) tol = 1e-10; % error tolerance X(:,1)=x0; k=1; % x0 = arbitrary Nx1 vector while 1 X(:,k+1) = B*X(:,k) + c; if norm(X(:,k+1)-X(:,k) ) < tol break; end k = k+1; k, x = X(:,k) % x = converged solution norm(A*x-b) % verify solution

54 MATLAB implementation – v.3
% another version that saves the iterates tol=1e-10; % error tolerance x=x0; k=1; % x0 = arbitrary Nx1 vector X=x; % X columns are the iterates x(k) while 1 xnew = B*x + c; if norm(xnew-x) < tol break; end x = xnew; k = k+1; X = [X,x]; % append new column to X k, x, norm(A*x-b) % print & verify solution

55 Example A = [2 1 0; 1 5 -1; 1 -2 4]; b = [4 8 9]'; xsol = A\b;
I = eye(size(A)); % 3x3 identity matrix D = diag(diag(A)); % diagonal part of A B = I - D\A; % 3x3 iteration matrix c = D\b; % 3x1 vector rho = max(abs(eig(B)));

56 >> D, B >> c, rho, norm(B,inf)
B = c = 2.00 1.60 2.25 rho = 0.5000 ans = 0.7500

57 tol=1e-12; % error tolerance
x=[0 1 2]'; k=1; % x0 = [0 1 2]' = arbitrary X=x; % X = save all iterates while 1 xnew = B*x + c; if norm(xnew-x)<tol % norm() measures break; % the distance end % between x,xnew x = xnew; k = k+1; X = [X,x]; % append new column end k, x, norm(A*x-b), norm(x-xsol), size(X)

58 >> k, x, norm(A*x-b), norm(x-xsol), size(X)
40 x = 1.0000 2.0000 3.0000 ans = 2.6657e-012 1.3634e-012 3 40 >> K=1:k; >> plot(K,X');

59 Temperature Distribution
follows from discretizing the Laplace equation

60 A x = b

61 A x = b >> x = A\b x = >> A = [ 4 -1 -1 0 20.0000
>> A = [ ]; >> b = [30; 60; 40; 70];

62 display solution T in a rectangular pattern
nodes were numbered in column order T = zeros(2,2); % shape of T T(:) = x T =

63 A x = b Rules for constructing A and b: main diagonal is 4
if nodes i,j are connected, then, -1, otherwise, 0 3) b(i) is sum of boundary values connected to node i

64 Iterative Solution convenient for large number of subdivisions
used also to solve 2D electrostatics problems

65 iterate over internal nodes only
N=4; M=4; left=20; right=30; up=10; dn=40; T(1,:) = repmat(up,1,M); T(N,:) = repmat(dn,1,M); T(:,1) = repmat(left,N,1); T(:,M) = repmat(right,N,1); Tnew = T; tol = 1e-4; K = 100; for k=1:K, for i=2:N-1, for j=2:M-1, Tnew(i,j) = (T(i-1,j) + T(i+1,j) +... T(i,j-1) + T(i,j+1))/4; end if norm(Tnew-T) < tol, break; end T = Tnew; T(1,[1,end]) = nan; T(end,[1,end]) = nan; boundary values iterate over internal nodes only

66 T = % start-up NaN NaN NaN NaN % converged after k = 19 iterations % to within the specified tol = 1e-4 T = NaN NaN NaN NaN

67 T = % after k=1 iteration NaN NaN NaN NaN T = % after k=2 iterations T = % after k=3 iterations

68 N=30; M=30; left=0; right=0; up=0; dn=60; tol = 1e-6; K = 5000; % breaks out at k = 2475 [X,Y] = meshgrid(2:M-1, 2:N-1); Z = T(2:M-1, 2:N-1); surf(X,Y,Z);


Download ppt "Rutgers University School of Engineering Spring 2015 14:440:127 - Introduction to Computers for Engineers Sophocles J. Orfanidis ECE Department orfanidi@ece.rutgers.edu."

Similar presentations


Ads by Google