# Numerical Analysis of Processes NAP2 Physical models (transport equations, energy and entropy principles). Empirical models (neural networks and regression.

## Presentation on theme: "Numerical Analysis of Processes NAP2 Physical models (transport equations, energy and entropy principles). Empirical models (neural networks and regression."— Presentation transcript:

Numerical Analysis of Processes NAP2 Physical models (transport equations, energy and entropy principles). Empirical models (neural networks and regression models). Models that use analytical solutions (diffusion). Identification of models based on the best agreement with experiment (regression analysis, optimization) Rudolf Žitný, Ústav procesní a zpracovatelské techniky ČVUT FS 2010

Mathematical MODELs NAP2 Rudolf Žitný, Ústav procesní a zpracovatelské techniky ČVUT FS 2010 Mathematical model = mathematical description (program) of system characteristics (distribution of temperature, velocity, pressure, concentration, performance,...), as a function of time t, space and operating parameters (geometry system, material properties, flow,...). Mathematical model of an apparatus or system Identification of the model parameters on the basis of experiment Experiment evaluation for example diagnostics Optimisation of operational parameters

NAP2 Rudolf Žitný, Ústav procesní a zpracovatelské techniky ČVUT FS 2010 General classification of models Black box - a model based solely on a comparison with experiment. Three basic types are -Neural networks (analogous neurons in the brain - the method of artificial intelligence) -Regression models (e.g. in the form of correlation functions of the power law type Nu = cRe n Pr m ) -Identification of the transfer function of a system E(t) from the measured time history of input x(t) and output y (t) of the system. White box - the model is based entirely on sound physical principles: the balance of mass, momentum, energy (Fourier and Fick's equation, Navier Stokes equations) on energy principles, the desired solution minimizes the total energy of the system (elasticity), on simulations of random movements and interactions of fictitious particles (Monte Carlo, lattice Boltzmann...).- Grey box - models, which is located between the above extremes. Examples are compartment models of flow systems, respecting the physical principle of conservation of mass, but other principles (eg balance of momentum), they are replaced by empirical correlations or data. Mathematical MODELs

Physical Principles NAP2 Continuum models (white boxes) Engineering models based on physical principles are often based on three balances (mass, momentum and energy). In differential form: Moss conservation Momentum conservation Energy conservation Cauchy equilibrium equations. Valid for structural as well as fluid flow analysis, compressible and incompressible. The total energy, the sum of internal, kinetic and potential energy heat flux power of inner forces Most of the equations, you know, can be derived from these conservation equations: Bernoulli, Euler, Navier Stokes, Fourier Kirchhoff.

NAP2 Continuum models (white boxes) The principles of energy are preferred especially in the mechanics of solid phase.In the theory of elasticity it is the Lagrange principle of minimum of potential energyminimum of potential energy deformation energy of internal forces (the product of the stress and strain tensor work of external forces (volumetric, surface, concentrated) u WiWi WeWe W(u) strains and stresses can be expressed in term of displacement u at equilibrium the energy (W i +W e ) is minimum Requirement of zero variation of energy functional  W/  u=0 is the basis of many finite elements design. This approach is represented in dynamics by Hamilton principle of zero variation of functional with kinetic energyHamilton principle "Energy" principles are not so general: each of the variational principle can derive an equivalent Euler Lagrange differential equation, but the converse is not true. Especially with flow the variational principles can be applied only in some cases (eg creeping flow). Physical Principles

Physical principles NAP2 Models determine the type of engineers who are engaged in addressing these issues: FUNCTIONALISTS imagine that each system can have infinitely many forms, and a number, functional can be assigned to each of them. Then they plot these numbers and look for some special points (minima, maxima, inflection). A special mystical significance is assigned to these characteristic points, perhaps, the state of balance, loss of stability, etc. Functionalists work especially in mechanics of solids and are easily identified by the fact that they appretiate only the finite element method (through gritted teeth perhaps Boundary Elements). They use words as Cauchy Green tensor, Kirchhoff - Piola stresses, the seventh-invariant, etc. DIFFERENTIALs believe that the law of conservation of mass, momentum and energy expressed in terms of differential equations can describe the state of a system in the infinitesimal neighborhood of any point (and consequently the whole system too). When choosing numerical methods the differentials are not too picky, but generally choose the method of finite volumes. Often use words like material derivative. PARTICULARISTs do not believe in continuum mechanics. Everything can be derived from the mechanics of a particle. There are only discrete quantities, such as in digital computers. The complexity of reality is caused by a large number of simple processes running in parallel. Particularists can be identified by the fact that shuns all others and are discreet. FATALISTs do not believe that it is possible for a human beeing to understand the laws of God providence and therefore are limited to the empirical description of the observed phenomena. They love expert systems, artificial intelligence methods, engineering correlations and statistics. CHAOTs admit existence of some principles, but do not believe in their meaningful solvability. They hate the smooth curves, and prefer to be spoiled by random number generators. They love disasters, catastrophs and attractors. They are attractive especially for women fascinated by foreign words and colorful flowers of fractals.

Example of models Drying NAP2 Rudolf Žitný, Ústav procesní a zpracovatelské techniky ČVUT FS 2010 As an example we will use different mathematical models of drying or moistening of the material, such as grain (starch, corn, rice, coffee...) in the whole course. There will always be one of the basic results time of drying (or relationship between time, temperature and moisture of dried material). For this purpose simple regression models or neural networks are usually sufficient. The distribution of moisture within the grain must be known (calculated) as soon as the microbial activity or healt risc is to be determined. White or gray boxes models based on transport equations and heat diffusion can be used. When the grain has a simple geometry (ball), and when there is no significant effect of moisture on the diffusion coefficient, the solution can be expressed in the form of infinite series (analytical solution of Fick's equation and FK, only few terms in these series are sufficient). When the geometry is simple, but diffusion and heat transfer coefficients are strongly non-linear, the 1D numerical methods are preferred (finite difference method or the spectral method). The complicated shape of grains (rice, beans) are evaluated by 3D numerical finite element or control volume (commercial software ANSYS, COMSOL and FLUENT are usually used). At present, the concern is focused to changing internal structure of a material, such as cracking, which in turn significantly affect the moisture (free water easily penetrates micro grain). This means that in addition to transport equations the deformation field and the distribution of mechanical stress tensor (elastic problem) must be solved. These models are generally based on the finite element method.

Neural networks NAP2 Artificial neural networks are the brutal techniques of artificial intelligence. Its popularity stems from the fact that more and more people know less and less about the real processes and their nature (maybe it could be called the law of exponential growth of ignorance). ANN (Artificial Neural Network) is designed specifically for those who knows nothing about the processes they want to model. They just have a lot of experimental data in which they are unable to navigate. And have a MATLAB (for example).MATLAB Schiele

Neural Networks NAP2 Neuron= module that calculates one output value from several input. The behavior of a particular neural network is given by the values of the synaptic weights w i (coefficients amplification of signals between interconnected neurons) Input layer (2 neurons) X 1,X 2 Hidden layer (here 4 neurons) Outlet layer (1 neuron) Y Neuron’s response y to N-inputs x i Nejčastěji používané aktivační funkce f (tangens hyperbolický, sigmoidální funkce, znaménková funkce) W i =coefficients of synaptic weights evaluate by special algorithm „learning“ network Nejčastěji používané aktivační funkce f (tangens hyperbolický, sigmoidální funkce, znaménková funkce) The most frequently used activation functions f (tangens hyperbolic, sigmoidal function, sign function sgn). All implemented e.g. in MATLABu.

Regression models NAP2 Delvaux

NAP2 The regression model has the form of a relatively simple function of the independent variable x, and the parameters p 1, p 2,…,p M, which is to be determined so that the values of the function best match the experimental values y.. Frankly neural networks are almost the same. The search parameters p 1, p 2,…,p M are the coefficients of synaptic weights linking neurons. The difference is that the type of the model functions is more or less unified in the ANN and the number of weights is greater than the number of parameters commonly used in regression functions. Regression function is chosen on the basis of experience or simplified ideas of modeled process (reasonable and logically explainable behavior at very small or large values of the independent variables is to be required). It is also desirable that the parameters p 1, p 2,…,p M have clearly defined physical meaning. However, it is true that if we do not know a physical nature of the process or if it is too complex (ie, the same situation as in neural networks) the polynomial regression function y= p 1 +p 2 x+…+p M x M-1 or another „neutral“ function is used. Regression models

NAP2 Let us consider a regression model with the unknown vector p of M parameters The parameters p i should be calculated so that the model prediction best fits the N measured points (x i,y i ). The most frequently used criterion of fit is chi- square (sum of squares of deviations) The quantity  i is standard deviation of measured quantity y i (measurement errror). Regression looks for minimum of the function  2 in the parametric space p 1, p 2,…,p M (sometimes more robust criteria of fit, for example the sum of absolute values of deviations, are used). Quality of a selected model is evaluated by the so called correlation index, which should be close to unity for good models Number of data points N should be greater than the number of calculated parameters M (N=M means interpolation) The worst val;ue r=0 corresponds to the case when it would be better to use a constant as a regression model ) Regression models

NAP2 Linear regression model is a linear combination of selected base functions, for example polynomials g m (x)=x m-1 or goniometric functions g m (x)=sin(mx) Model prediction can be expressed in a matrix form [[A]] is design matrix with N rows corresponding to N/points and M columns for M-base functions. N>M, the case of square matrix N=M is not regression but interpolation. As soon as the standard deviation is constant, the  2 value is proportional to the sum of squares s 2 of deviation between prediction [y predikce ] and measured data [y] This is scalar product of two vectors Linear regression Base functions are analogy of activation functions of neurons and the parameters p m correspond to synaptic weights.

NAP2 Base functions HERMITE polynomial TSCHEBYSHEF I. polynomial Example: orthogonal polynomials

NAP2 Zero first derivatives of s 2 with respect of all model parameters exist at minimum which is the system of M linear algebraic equations for M unknown parameters Transposed matrix [[A]] T has dimension MxN and when multiplied by [[A]] gives square matrix with the dimension MxM [[C]] MxM Matrix [[C]] enables estimate of reliability interval of calculated parameters p 1,…,p M Transposed matrix [[A]] T having dimension MxN multiplied by vector [y] Nx1 gives a vector of M-values  pk is standard deviation of calculated parameter p k for k=1,2,…,M  y jis standard deviation of measured data (it is assumed that all data are measured with the same accuracy) Inversion of matrix [[C]] (inverted matrix is called covariance matrix) Linear regression

Linear regression – example noise filtration NAP2 Principle of Savitzky Golay filter of noised data is very simple: each point of input data x i,y i is associated with the window of N w points left and N w points right and these 2N w +1 points is approximated by regression polynomial of degree k, where k<2N w. Value of this regression polynomial in the point x i substitutes original value y i. Number of data N=1024, width of window N w =50, quadratic polynomial. The SG filtration is implemented in MATLAB as function SGOLAYFILT(X,K,F) where X vector of noised data, K degree of regression polynomial and F=2N w +1 is width of window.

Regression models example-soaking NAP2 The application of Peleg's equation to model water absorption during the soaking of red kidney beans (Phaseolus vulgaris L.)The application of Peleg's equation to model water absorption during the soaking of red kidney beans (Phaseolus vulgaris L.) Journal of Food Engineering, Volume 32, Issue 4, June 1997, Pages 391-401 Nissreen Abu-Ghannam, Brian McKenna Many empirical models are used for description of relationship between moisture content in grains and time, for example exponential Page’s model X e is equilibrium, X O initial moisture of grain There are two model parameters p 1 =k, p 2 =n. Independent variable t is time and even more frequently used Peleg’s model (see paper) Page and Peleg models are nonlinear and in practice are transformed to linear form by logarithm (Page) or (Peleg) as tvar This is new dependent variable y Parameter p 2 characterises equilibrium moisture Peleg’s model is used for beans, chickpeas, peas, nuts… …and then linear regression can be applied for identification of p 1 a p 2

Regression models -soaking NAP2 Linearisation of non-linear models (for example linearisation of Page and Peleg models) means that the optimised parameters p 1, p 2 minimise some other criterion than the  2 and also other characteristics, like covariance matrix, correlation index do not correspond to the assumption of normal distribution of errors. However, this effect is usually small and can be neglected.

Analytical solution diffusion and heat NAP2 Vermeer

Analytical solutions NAP2 Analytical solution exist only at linear models Ordinary differential equations: Partial differential equations: there exist two ways any analytical functions that you know, for example sin, cos, exp, tgh, Bessel functions,… are defined in fact as a solution of ordinary differential equations (see handbooks, for example Kamke E: Differential Gleichungen, Abramowitz M., Stegun I.: Handbook of Mathematical functions). General approach consists in expression of solution in form of infinite power series and identification of coefficients by substitution the expansion into the solved differential equation Fourier method of separation of variables. Solution F(t,x,y,z) is searched in the form of product F=T(t)X(x)Y(y)Z(z). Substituting into partial differential equation results ordinary differential equations for T(t), X(x), Y(y) and Z(z). Application of integral transforms: Fourierovu, Laplaceovu, Hankelovu. Result is algebraic equation, which is to be solved and back-transformation must be used.Fourierovu LaplaceovuHankelovu

Distribution of moisture X (kg water/kg solid) is described by Fick’s equation m=0,1,2 for plate, cylinder, sphere. Fourier method of separation of variables Substituting to Fick’s PDE results This term depends only on time t this terms depends only on r in fact both terms must be constants independent on t, r. The constant is called eigenvalue. X e equilibrium, X 0 initial moisture NAP2 Analytical solution diffusion (1/5)

Solution of G i (t) is exponetial function, solution of F i (r) is cos(  r) for plate (m=0), Bessel function J 0 (  r) for cylinder (m=1) and for sphere (m=2, this is our case of spherical grains) Phase equilibrium is assumed at the sphere surface (r * =1, X=X e ). Therefore X * (t,1)=0 and this condition must be satisfied by any function F i. This is the condition for eigenvalues Dimensionless con centration profile is therefore The coefficients c i must be selected so that the initial condition (distribution of concentration in time zero) will be satisfied. For constant initial concentration X=X 0 must hold X * =1 for arbitrary r *, thewrefore NAP2 Analytical solution diffusion (2/5)

NAP2 The coefficients c i are evaluatred from orthogonality of functions F i (r * ). Functions are orthogonal if their scalar product is zero. Scalar product of functions is defined as integral Proof per partes integration. Thbis is zero because both functions F must satisfy boundary conditions Analytical solution diffusion (3/5)

NAP2 Let us apply orthogonality to previous equation (multiplied by r 2 F j and integrated) Concetration profile or by integration across the volume of sphere total moisture content as a function of time Analytical solution diffusion (4/5)

Heat and mass transfer during the coffee drying processHeat and mass transfer during the coffee drying process Journal of Food Engineering, Volume 99, Issue 4, August 2010, Pages 430-436 Katrin Burmester, Rudolf Eggers Modeling and simulation of heat and mass transfer during drying of solids with hemispherical shell geometryModeling and simulation of heat and mass transfer during drying of solids with hemispherical shell geometry Computers & Chemical Engineering, Volume 35, Issue 2, 9 February 2011, Pages 191-199 I.I. Ruiz-López, H. Ruiz-Espinosa, M.L. Luna-Guevara, M.A. García-Alvarado Diffusion coefficient D ef generaly depends upon temperature and moisture and also the equilibrioum moisture X e is a function of temperature, for example Regression model is therefore strictly speaking nonlinear (with parameters p 1 =D wa, p 2 =E a a p 3 =b) and the analytical solution with substituted D ef is only an approximation. This model used Katrin Burmester for coffee grains NAP2 Analytical solution diffusion (5/5)

Optimalisation NAP2 There are two basic optimisation techniques for calculation of parameters p i minimising  2 of nonlinear models :  Without derivatives, when minimum can be identified only by repeated evaluation of s 2 (or another criterion of fit between prediction and experiment) for arbitrary values of parameters p i. These methods are necessary at very complicated regression models, for example models based upon finite element methods.  Derivative methods, making use values of all first (and sometimes second) derivatives of regression model with respect to all parameters p 1,p 2,…,p M.

NAP2 Gauss method of least sum of squares of deviations Weight coefficients (representing for example variable accuracy of measurin method) Zero derivatives with respect all parameters j=1,2,…,M Linearisation of regression function by Taylor expansion  p is increment of parameters in one iteration data mod el Optimisation methods with derivatives

NAP2 Solution of systém of linear equations in each iteration The most frequently used modification of Gauss method is the Marquardt Levenberg method : diagonal of matrix [[C]] is increased by adding a constant in the case that the iteration are not converging. For very high the matrix [[C]] is almost diagonal and the Gauss method reduces to the gradient steepest descent method (right hand side is in fact gradient of the minimised function s 2 ). value is changing during iterations: when process converges decreases and the faster Gauss method is preferred, while if iterations diverge the increases (gradient method is slower but more reliable).Marquardt Levenberg Optimisation methods with derivatives

Optimisation method without derivatives NAP2 The simplest case is optimisation of only one parameter (M=1). First the global minimum is to be localised (for example by random search). Then the exaxt position of minimum is identified by the method of bisection or by the Golden Section. The golden section method reduces uncertainty interval in each step in the ratio 0.618 and not in the ratio 0.5 as in bisection. However only one value of regression function need to be evaluated in the golden section and not two values necessary for bisection. See algorithm Golden section search and the following slide (definition of golden section)Golden SectionGolden section search f1f1 f2f2 f3f3 f4f4 L1L1 L 2 =0.618L 1 L 3 =0.618L 2 Example: Initially two values f 1 f 2 in golden sections of interval L 1 are calculated. Because f 1 >f 2 the minimum cannot be left, and the interval of uncertainty reduces to L 2. We need again two values in this interval, but one value (f 2 ) was calculated in the previous step, so that only ONE new value need to be calculated. And that is just the glamor of the golden section method and the secret of the magic ratio 0.618.

Golden Section NAP2 A B C1 C2 a p q Quadratic equation for the ratio q/p

NAP2 Example: How many steps of golden section method and how many values of regression function must be evaluated if thousand times reduction of uncertainty interval is required? Result, for 1000times increase accuracy it is sufficent 14 steps. An approach to determine diffusivity in hardening concrete based on measured humidity profilesAn approach to determine diffusivity in hardening concrete based on measured humidity profiles Advanced Cement Based Materials, Volume 2, Issue 4, July 1995, Pages 138-144 D. Xin, D. G. Zollinger, G. D. Allen Diffusion of concrete hardening, and identification of diffusion coefficient by golden section method. Optimisation method without derivatives

NAP2 Principles of the one-parametric optimisation can be applied also for M-parametric optimisation of p 1,…,p M just repeating 1D search separately (Rosenbrock).Rosenbrock However, the most frequent is the simplex method Nelder Mead. Principle is quite simple:Nelder Mead Animated gig from wikipedia.org 1. Simplex formed by M+1 vertices is generated (for two parameters p 1 p 2 it is a triangle). 2. The vertex with the worst value of the regression function is substituted by flipping, expansion or contraction with respect to the gravity center of simplex. Step 2 is repeated until the size of simplex is decreased sufficiently Optimisation method without derivatives

NAP2 Zang H., Zhang S., Hapeshi K.: A review of nature inspired algorithms. J.Bionic Engineering 7 (2010), 232-237 Optimisation method without derivatives Ant Colony Optimisation (AS Ant System, ACS Ant colony system) Bees Algorithm Genetic Algorithm a1a1 a2a2 SOMA Self Organizing Memetic Algorithm (algorithms of a pack of volves)

MATLAB NAP2

Optimisation methods MATLAB NAP2 Linear polynomial regression p=polyfit(x,y,m) Minimalisation of function (without „constraints“, method Nelder Mead) p=fminsearch(fun,p0) where fun is a reference (“handle” denoted by the symbol @) to a user defined function, calculating value which is to be minimised for specified values of model parameters (for example sum of squares of deviations). Vector p0 is initial estimate. Nonlinear regression (nonlinear regression models) p = nlinfit(x,y,modelfun,p0) ci = nlparci(p,resid,'covar',sigma) also statistical evaluation of results (covariance matrix, intervals of uncertainty)

NAP2 time X(moisture) 0 0.9406 1 0.7086 2 0.7196 3 0.5229 4 0.4657 5 0.3796 6 0.3023 7 0.1964 8 0.1545 9 0.1466 xdata=[0 1 2 3 4 5 6 7 8 9]’; ydata=[0.9406 0.7086 …]’; Example: Measured drying curve, 10 points (time and moisture) Apostroph [vektor]’ means transposition, instead of row it will be columnwise vector Plot od date by command plot(xdata,ydata,’*’) Optimisation methods MATLAB

NAP2 Drying curve approximated by cubic polynomial p 1 x 3 +…+p 4 p=polyfit(xdata,ydata,3) p = 0.0001 0.0040 -0.1322 0.9103 Plot polynomial with coefficients p(1)=0.0001, p(2)=0.0040 …Y=polyval(p,xdata) hold on plot(xdata,ydata,'*') plot(xdata,Y) hold on enables plotting more curves into one graph All this could have been done by single command plot(xdata,ydata,'*',xdata,polyval(p,xdata)) Vector of calculated coefficients Optimisation methods MATLAB

NAP2 with A „a scale“ coefficient, diffusion coefficient D ef and radius of particle D ef and R are not independent parameters because only the ratio D ef /R 2 appears in mthe model. The radius R can be selected (for example R=0.01 m) and only wo regression parameters A, D ef, should be minimised. How?  Define xmodel(t,p,pa) as a sum of series (with params p 1 =A, p 2 =D ef, pa 1 =R, pa 2 =n)  Define function calculating sum of squares xdev(xmodel,p,pa,xdata,ydata)  Calculate optimum using p= fminsearch(xdev,p0) The data of the drying curve can be better approximated by diffusion model Optimisation methods MATLAB

NAP2 Model definition function xval = xmodel(t,p,pa) A=p(1); D=p(2); R=pa(1); ni=pa(2); xv=0; for i=1:ni pii=(pi*i)^2; xv=xv+6/pii*exp(-pii*D*t/R^2); end xval=A*xv; In this way can be defined arbitrary model of drying, for example previously identified cubic polynomial, Peleg’s or Page’s models, or even models defined by differential integrations. This last case should be discussed later. This text should be saved as M-file with the filename xmodel.m Optimisation methods MATLAB

NAP2 Definition of goal function (sum of squares of deviations) function sums = xdev(model,p,paux,xdat,ydat) sums=0; n=length(xdat); for i=1:n sums=sums+(model(xdat(i),p,paux)-ydat(i))^2; end Define auxilliary parameters and search regression parameters by fminsearch pa=[0.01,10] p = fminsearch(@(p) xdev(@xmodel,p,pa,xdata,ydata),[.5;0.0003]) R=0.01 and number of terms in expansion n=10 The first parameter fminsearch is function with only one parameter, vector p of optimised parameter. Another non optimised parameters must be in MATLAB specified by using anonymous function @(p) expression. Initial estimate of parameters Optimisation methods MATLAB

NAP2 function [estimates, model] = fitcurve(xdata, ydata) start_point = [1 0.00005]; Hledané parametry jsou dva: A,D. Počáteční odhad. model = @expfun; Expfun je predikce modelu a výpočet součtu čtverců odchylek estimates = fminsearch(model, start_point); volání optimalizační Nelder Mead metody function [sse, FittedCurve] = expfun(params) A = params(1); optimalizovaný škálovací parametr D = params(2); optimalizovaný difuzní koeficient R=0.01; poloměr částice (když ho chcete změnit musíte opravit funkci fitcurve.m) ni=10; počet členů řady(správně nekonečno, ale 10 obvykle stačí) ndata=length(xdata); (počet bodů naměřené křivky sušení) sse=0; výsledek expfun – součet čtverců odchylek for idata=1:ndata xv=0; tady je třeba naprogramovat konkrétní model sušení (Y(X,params)) for i =1:ni pii=( pi * i )^2; xv=xv+6/pii*exp(-pii*D*xdata(idata)/R^2); end FittedCurve(idata) = A* xv; ErrorVector(idata) = FittedCurve(idata) - ydata(idata); sse=sse+ErrorVector(idata)^2; end [estimates, model] = fitcurve(xdata,ydata) Exactly the same procedure can be summarized in the single M-file Optimisation methods MATLAB

EXAM NAP 2 Regression models Optimisation

EXAM NAP2 Balance equations Lagrange variational principle (minimum W i +W e ) Goal function (chi-quadrat) Regression (parameters p minimising chi-quadrat) Minimisation method with derivativatives (Marquardt Levenberg) and without derivatives (golden section, simplex method Nelder Mead)

Similar presentations