Boundary Value Problems and Least Squares Minimization Daniel Baur ETH Zurich, Institut für Chemie- und Bioingenieurwissenschaften ETH Hönggerberg / HCI F128 – Zürich E-Mail: daniel.baur@chem.ethz.ch http://www.morbidelli-group.ethz.ch/education/index Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Boundary Value Problems (BVP) for ODEs Problem definition: Find a solution for a system of ODEs Subject to the boundary conditions (BCs): The total number of BCs has to be equal to the number of equations! Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Shooting method A first approach is to transform the BVP into an initial value problem (IVP), by guessing the missing initial conditions and using the BC to refine the guess, until convergence is reached This way, the same algorithms as for IVPs can be used, but the convergence can be very problematic Too high: reduce initial velocity! Target Too low: increase initial velocity! Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Relaxation methods Relaxation methods try to find a solution that «relaxes» from the disturbance that are the initial conditions Example: Dimensionless tubular reactor The solution «relaxes» into the steady state solution Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Visualization Take a look at the discretized internal profile y yin y0 y1 y2 y3 y4 y5 yN yN+1 x L/Δx L Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Relaxation methods (continued) Therefore, we can iterate until convergence: For a first order reaction, we have a linear system With the «boundary conditions» Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Matrix formulation of the linear system If we cast the linear system into matrix form, we get Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Iterative methods Instead of simply iterating, we can use the properties of the matrix to speed up the solution The main idea is to reformulate the problem in the following way The goal is to choose S in a way that is easy to solve (e.g. diagonal, tridiagonal, triangular), but in some sense similar to A to keep ρ[S-1*(S-A)] small The iteration formula therefore reads Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Jacobi’s method The Jacobi method chooses S to be a diagonal matrix The procedure is therefore given by This method is guaranteed to converge if A is diagonally dominant, i.e. Daniel Baur / Numerical Methods for Chemical Engineers / Iterative Methods

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Collocation method A more sophisticated approach is the collocation method; it is based on approximating the unknown function with a sum of polynomials multiplied with unknown coefficients The coefficients are determined by forcing the approximated solution to satisfy the ODE at a number of points equal to the number of coefficients Matlab has a built-in function bvp4c which implements such a method Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Collocation method (continued) We require that at every point Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Transformation into a first order ODE bvp4c solves first order ODEs, so if we remember the «trick» and transform our ODE, we get And the boundary conditions Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Least squares minimization In data fitting, we generally try to solve a problem of the form where f is some model with parameters β, X are the experimental conditions and y are the results x y Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Some nomenclature In prospect of the statistics lecture, please note the differences in nomenclature! p parameters (in linear regression p = m+1) m variables (plus constant term) n measurements / data points Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Linear least squares minimization If the model f(β,X) is linear (this means that the parameters β only appear in linear terms!), we saw that the unique minimizer to the problem reads However, since XTX is much harder to solve (worse conditioned) than X, we can try to find another decomposition where Q is an orthogonal matrix (Q-1 = QT) and R is an upper triangular matrix Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Linear least squares minimization (continued) Substituing our originial problem This yields the same minimizer, but is easier to solve Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Assignment 1 Solve the steady state of the dimensionless tubular reactor using Jacobi’s method. Use Pe = 10; N = 150; Da = 1; ystart = zeros(N,1); n = 1; Define the core of the matrix A; This is easiest done using diag(): After this, change the first and the last row of A to reflect the boundary conditions and define b (see slide 6). Use A = sparse(A); and b = sparse(b); to tell Matlab that the matrices contain mostly zeros. Check with spy(A) if this is true. This will vastly speed up all calculations. Use Jacobi’s iteration formula and iterate until the change in y is smaller than 1e-8 or 1e5 iterations are exceeded. Plot the results using plot(linspace(0,1,N), y) Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Assignment 1 (continued) Solve the same steady state of the dimensionless tubular reactor using bvp4c bvp4c uses a call of the form sol = bvp4c(ode_fun, bc_fun, solinit); ode_fun is a function taking as inputs a scalar t and a vector y, returning as an output dy / dt bc_fun is a function taking as inputs vectors where the boundary conditions are evaluated, returning as output the residual at the boundary solinit is an initial guess for the solution: solinit = bvpinit(range, @initfun) sol is a struct containing the solution and other parameters Plot the solution using plot(sol.x, sol.y(1,:)) and compare with the solution from the first part. More on the usage of bvp4c on the next slide Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Usage of bvp4c In our case use the following Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Exercise 2 Find online the example files from the lecture for the linear least squares problem. We fit a linear model of the form Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Assignment 2 Run example01.m to see the results of the fit. Read through the code and ask questions if you have any. Is the model really linear, even though it contains x2? In the definition of the data matrix X on line 7, there is a vector of ones. Why is that? Check with the definition of a linear model in matrix form Note: no programming will be required for this assignment. Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Exercise 3 Find online the example files from the lecture for the non-linear least squares problem. We fit a model of the form Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE Assignment 3 Run example02.m to see the results of the fitting. Read through the code and ask questions if you have any. The functions provide a Hessian and a Jacobian specific to the problem. What could be done to make a solver more general and easier to use (remember back when we implemented the Newton method!)? In what cases can the Newton method fail? In what cases can the Gauss-Newton method fail? Note: no programming will be required for this assignment. Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE