Nonlinear Regression 1Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Michael Sokolov ETH Zurich, Institut für Chemie- und Bioingenieurwissenschaften ETH Hönggerberg / HCI F135 – Zürich

Example: Puromycin  Puromycin is an antibiotic used in bio-research to select cells modified by genetic engineering  The Michaelis-Menten model for enzyme kinetics relates the initial velocity of an enzymatic reaction to the substrate concentration x by the equation  At high concentrations, the velocity is essentially constant (θ 1 ), while at low concentrations it is a linear function of the concentration 2Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Puromycin Kinetics 3Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression The model:

Model Linearization  The initial model reads  This can be rearranged to  Renaming the variables yields a linear model 4Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Linearized Model: Fit 5Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Regression Line  1 =  2 =

Regression from Linearized Model 6Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Regression from linearized model  1 =  2 =

Nonlinear Regression  The objective function of non-linear regression reads where y is the vector of responses, x is the vector of observations, θ is the vector of parameters and f is the nonlinear model function  In the 2-parameter case, we can plot S(θ) as a function of the parameter values to look for a minimum 7Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Objective Function S(θ) 8Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Minimum Estimated value of  from linearization

Minimization of S(θ)  If we linearize the model with respect to the parameters, we get where J(θ 0 ) is the Jacobian matrix of f with respect to θ, evaluated at θ 0  In this case, the residuals are  If we now define the objective function in terms of the residuals, we can apply the Gauss-Newton method 9Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Analogy:

Gauss-Newton Method applied to S(θ) 10Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Convergence path of Gauss- Newton Method (  1 ) opt = (  2 ) opt =

Nonlinear Regression 11Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression Nonlinear Regression Regression from linearized model

Matlab Nonlinear Regression Routines  There are several options for doing nonlinear regression:  lsqcurvefit  lsqcurvefit : Best for quickly fitting models to data; Not much statistical output; Provides more numerical information and options; Can use bounded parameter ranges  nlinfit  nlinfit : General nonlinear regression routine; Provides some statistical output  NonLinearModel  NonLinearModel : Most useful for nonlinear regression; All statistical and numerical tools in one place; Works analogously to LinearModel  fitnlm  fitnlm : new version of NonLinearModel (since 2013b) 12Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Using the Nonlinear Regression Routines  First we need to create a function that takes as inputs a vector b, the model parameters, and a vector (or matrix) x, the data points of the observed variables, and returns as output a vector (or matrix) y, the model responses  Then we use one of the nonlinear regression routines: 13Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Plotting the Fitted Curves  Plotting is as simple as calling the cost function or the predict routine of the NonLinearModel 14Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Tukey-Anscombe Plots  Plot of residuals vs. fitted values 15Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Normal Probability Plot 16Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Estimation of the Confidence Intervals 17Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Exercise 1  The product of a chemical reaction contains 50% chlorine at the time of production. The amount of chlorine subsequently drops over time. A model is proposed to predict the chlorine concentration as a function of time: where X is the fraction of chlorine in the product, t is the time and α and β are the model parameters. 18Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Assignment 1 1.Find online the data file chlorine.dat which contains the measurements and some code to load the data. 2.Plot the chlorine concentration as a function of time. Would a linear model seem feasible? 3.Determine suitable initial values for the nonlinear regression by linearizing the model and using linear regression to estimate the parameters. regress LinearModel  Use either regress or a LinearModel to perform the linear regression.  Since the dependence is exponential, simply use a log-model to linearize the model: 19Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Assignment 1 (Continued) 4.Write a model function of the form  function XCl = chlorinemodel(par, t)  where par(1) will be α and par(2) will be β 5.Fit a nonlinear model to the data by using one of the functions given on slide 12. Plot the model with the data, and report the parameters. NonLinearModel dataset  If you use NonLinearModel, you can use as input either a dataset or the measurements as X, y pair. 20Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Exercise 2  An experiment was conducted to evaluate the effect of two different methionin nutrition supplements (sourceA and sourceB) on the growth of newborn turkeys. The turkeys were separated into 10 experimental units with 15 birds each. For each supplement, the amount fed was varied between 0.04% and 0.44% of the total food. The target variable, the mean body weight of the units, is expressed as a function of the dosage of the two supplements, according to the following non-linear model where Y is the mean weight, and x A and x B are the dosages of the two supplements, respectively. 21Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Exercise 2 (Continued)  The model assumes that the intercept θ 1 and the horizontal asymptote θ 1 + θ 2 are the same for both supplements, and that only the growth rates θ 3 *θ 4 and θ 3 for supplements A and B, respectively, are different. The question is whether one of the supplements is superior, i.e. whether θ 4 is significantly different from unity. 22Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression

Assignment 2 1.Find online the data file turkeyweight.dat and some code to load the data into Matlab. 2.Find suitable initial values for the parameters by linearizing the model.  If you cannot solve this use θ 1 = 638, θ 2 = 175, θ 3 = 5 and θ 4 = 1. 3.Write a model function of the form  function y = turkeymodel(theta, x) nlinfit NonLinearModel 4.Perform the nonlinear regression, using nlinfit or NonLinearModel. 5.Are the growth rates for the two supplements significantly different, i.e. is θ 4 significantly different from unity? nlparci coefCI  Use either nlparci or coefCI to estimate the confidence intervals. 23Michael Sokolov / Numerical Methods for Chemical Engineers / Nonlinear Regression