MODEGAT Chalmers University of Technology Use of Latent Variables in the Parameter Estimation Process Jonas Sjöblom Energy and Environment Chalmers University of Technology

MODEGAT Chalmers University of Technology NO X Reduction catalysis Introduction ~mm ~µm~nm

MODEGAT Chalmers University of Technology Use of Latent Variables (LV) What is LV? How does it work? How can it be applied in the parameter estimation process? –3 case studies Why is it good? Outline

MODEGAT Chalmers University of Technology Latent Variable modelling Reduces a data matrix (using projections) to new, few and independent components (Latent Variables). Latent Variable (LV) Model: –P: loadings (linear combination of original variables) –T: scores (projections on the subspace defined by P) –# components: # linear independent directions Different types of Latent Variable (LV) models: –Principal Components Analysis (PCA) –Partial Least Squares (PLS) x1=dY/d  1 x2=dY/d  2 x3=dY/d  3 p1 p2 What is LV modelling?

MODEGAT Chalmers University of Technology Parameter Estimation Process How can LV models be applied? Define model and model assumptions Define ”experimental space” Fit parameters Satisfactory results? Yes! Evaluate the Design by LV-model (experimental rank) No! Evaluate the Design (perform experiments) Choice of experiments to perform Use of LV models &2

MODEGAT Chalmers University of Technology Application 1: LV models during the fitting process NO X Storage and Reduction (NSR) Mechanism –62 parameters Poor experimental design Jacobian  f/  used in gradient search –ill-conditioned –Local minima Objective: to improve parameter fitting by analysing parameter correlations and make parameters more orthogonal Ref: Sjoblom et al, Comput. Chem. Eng. 31 (2007)

MODEGAT Chalmers University of Technology Parameter assessment Jacobian  f/  –Evaluated for ALL adjustable parameters (not only fitted ones) Latent Variable (LV) method: –Partial Least Squares (PLS) using the Jacobian as "X" and f (Residual: simulated-observed gas phase concentrations) as “Y” Outcomes: 1.Correlation structure ! 2.Number of independent directions (# parameters to fit) ! 3.Which parameters to choose ! (method 1) 4.Parameter fit in LV space (method 2) How can LV models be used? -appl.1

MODEGAT Chalmers University of Technology LV example: "loading" plot X Y How can LV models be used? -appl.1

MODEGAT Chalmers University of Technology Results  Fitting results are comparable, but the fitting is more efficient (faster) due to fewer and more independent parameters, adopted for the data set at hand Method I (9 selected parameters) Method II (fitting of 9 scores) Method “brute force” (all 62 parameters) How can LV models be used? -appl.1

MODEGAT Chalmers University of Technology Application 2: Model-based DoE for precise parameter estimation "Simple" but realistic system: –NO-oxidation on Pt –Model from Olsson et.al. (1999) –Using simulated data (noise added) as experiments Objective: –How to find the experiments that enable precise estimation of the kinetic parameters Ref: Sjoblom et al, Comput. Chem. Eng 32 (2008)

MODEGAT Chalmers University of Technology Experiment assessment Jacobian  f/  –Evaluated for ALL "possible" experiments (3 iterations) Latent Variable (LV) method: –Principal Component Analysis (PCA) of J (unfolded 3 way matrix) –D-optimal design to select experiments Outcomes: –Correlation structure ! –Number of independent directions (# parameters to fit) ! –Which experiments to choose ! How can LV models be applied? -appl.2 Define model and model assumptions Define experimental space D-optimalChoice of experiments to perform using X or T from LV-model fit, analyze Satisfactory results? Yes! Evaluate the Design by LV-model (experimental rank) No! Evaluate the Design by LV- model Choice of experiments to perform

MODEGAT Chalmers University of Technology Results Overcomes dimensional reduction of the Fischer information matrix: by use of PCA (LV model of unfolded 3-way matrix) Almost perfect fit was obtained but parameter values were different (J not full rank) Using X (as is) or an LV approximation of X performs equally well –but becomes more efficient since it requires less experiments The LV model gives additional information of the dimensionality of selected experiments before they are performed. How can LV models be applied? -appl.2

MODEGAT Chalmers University of Technology Application 3: Extended Sensitivity Analysis for targeted Model Improvements H 2 assisted HC-SCR over Ag-Al 2 O 3 –Detailed model (23 reactions, heat balance) –Acceptable fit, but still significant Lack-of-Fit Objectives: –Verify (falsify) model assumptions –Get indications on how to improve model fit Refs: Creaser et al. Appl.Catal.B 90 (2009) 18-28, Sjöblom PhD Thesis (2009) Chalmers Thesis available at: NO 2 NO 3 CH 2 NO 2 O O O C 8 H 18 CO 2,  H N2N2 NO, H 2

MODEGAT Chalmers University of Technology Experimental Sensitivity analysis of 62 model parameters (not only fitted ones, not only kinetic parameters) –46 kinetic parameters –10 mass and heat transport parameters –6 other parameters Scaled local sensitivities –Unfold 3-way matrix to size n x pk, where n=26025 time points, p=62 parameters and k=5 responses Univariate analysis as well as LV modelling How can LV models be applied? -appl.3

MODEGAT Chalmers University of Technology LV-model and univariate measures How can LV models be applied? -appl.3 PCA model –Scores plot –Loadings plot –25 components Univariate table data –Confidence intervals –Sensitivity average, std, max –Correlations

MODEGAT Chalmers University of Technology Sensitivity Analysis results (examples) Mass transfer model needs attention –Include diffusivities in fitting? –Include internal mass transport? –Targeted transients? Heat transfer model needs attention –Improve/extend temperature measurements? –Consider additional sensors (HC, H 2 )? –Modify heat transfer model? –Targeted experiments? (For more details, see poster) How can LV models be applied? -appl.3

MODEGAT Chalmers University of Technology Ability to master different parts of the process –The model (assumptions) –The available data (experiments) –The parameter values (which to fit) Ability to “change focus” in the process as the fit develops Why are LV models good? Factors for successful parameter estimation Model assumptions ”experimental space” Fit parameters Happy? Yes! No! Evaluate the Design Choice of experiments New PhD project: “Improved methods for parameter estimation” Advertisement out now! Application dead line 20 th sept

MODEGAT Chalmers University of Technology LV Components: Few, New & linearly Independent Few: Improved efficiency Linear: Non-linear systems, LV models provide more robust linearisations Independent: Orthogonal sensitivities fulfils statistical requirements Why are LV models good?

MODEGAT Chalmers University of Technology Conclusions The LV concept is a viable way in the Parameter estimation process Widely applicable –during fitting, DoE, evaluation Proven more efficient (due to fewer dimensions) –Superior? Yet to be “proven”...

MODEGAT Chalmers University of Technology End Acknowledgements The Swedish Research council for financial support The Competence Centre for Catalysis (KCK) for good collaboration Derek Creaser & Bengt Andersson for fruitful supervision Thank you for your attention!