1 Introduction to Model Order Reduction Luca Daniel Massachusetts Institute of Technology

2 Course Outline Numerical Simulation Quick intro to PDE Solvers Quick intro to ODE Solvers Model Order reduction Linear systems Common engineering practice Optimal techniques in terms of model accuracy Efficient techniques in terms of time and memory Non-Linear Systems Parameterized Model Order Reduction Linear Systems Non-Linear Systems Today

3 Model Order Reduction of Non-Linear Systems Introduction Reduction of weakly nonlinear systems (Volterra Series) Reduction of strongly nonlinear systems Trajectory PieceWise Linear (TPWL) and Polynomial (PWP)  with moment matching  with Truncated Balance Realization

4 Jet engine inlet example  Inlet density disturbance d :

5 NonLinear devices in Systems-On-Chip

6 Example of a nonlinear transmission line model

7 Example of a microswitch DISCRETIZE

8 Reduction of NonLinear Systems Non-Linear analog components e.g. MEMs, VCO, LNA NonLinear Reduction PDE Field Solvers or Circuit Simulators

9 Change of variables Equation Testing Projection Framework

10 Projection framework for nonlinear dynamic systems Reduced system: Substitute: in A problem: q=10 n=10 4 Using V T F(Ux) is too expensive!

11 Nonlinear Problem is MUCH Harder In what basis should we project?In what basis should we project? –No simple frequency domain insight –No eigenmodes –No Krylov subspace How do you represent the projection?How do you represent the projection? –New problem for nonlinear

12 How to get the change of variable matrix Analysis of linearized models [Ma88]Analysis of linearized models [Ma88] Sampling of time-simulation data + principal components analysis [Sirovich87]Sampling of time-simulation data + principal components analysis [Sirovich87] Nonlinear balancing [Scherpen93],Nonlinear balancing [Scherpen93], Time-derivatives [Gunupudi99], …Time-derivatives [Gunupudi99], …

13 Model Order Reduction for NonLinear Systems Representation of F(x) using a polynomial (e.g. Taylor’s expansions, Volterra Series) [Phillips00]***** Representation of F(x) using a polynomial (e.g. Taylor’s expansions, Volterra Series) [Phillips00]***** Representation of F(x) using several linearizations (Trajectory Piece-Wise Linear TPWL) [Rewienski01] Representation of F(x) using several linearizations (Trajectory Piece-Wise Linear TPWL) [Rewienski01] Representation of F(x) with several polynomials (PWP PieceWise Polynomial) [Dong03] Representation of F(x) with several polynomials (PWP PieceWise Polynomial) [Dong03]

14 Introduction to Model Order Reduction Thanks to Joel Phillips, Cadence Berkeley Labs III.2 – Reduction of Weakly Non Linear Systems Luca Daniel

15 MOR for nonlinear dynamic systems (J. Phillips, Y. Chen, F. Wang): Representation of F(x) based on Taylor’s expansions Representation of F(x) based on Taylor’s expansions Linear, quadratic reduced order models: Linear, quadratic reduced order models: Approximate nonreduced model: Approximate nonreduced model:

Polynomial Approximations Expand nonlinearity in multi-dimensional polynomial series Differential equation becomes To match first few terms in functional series expansion, only need first few polynomial terms etc. where Each term is a -dimensional tensor, represented as an matrix [note : not static models; frequency effects retained]

Projection of polynomial terms Draw from reduced space as Identity for Kronecker products Project tensors Gives reduced model etc.

Reduced polynomial models Projection procedure produces reduced model in same polynomial form  Key tensor components are compressed to lower dimensionality  Kronecker forms provide convenient general notation How to get projection spaces V?

Variational Analysis Introduce variational parameter Consider parametrized system Expand response in powers of Substitute into differential equation Compare and collect terms for each power of

Variational Analysis Coupled set of differential equations  Each set computes response at one order Each set is linear in its own state variable  First set is just linearized model etc.

Model Reduction First order equation set driven by input of low rank  Krylov subspace for projection Second order equation set driven by input Key insight : recall is well-approximated by thus

Model Reduction Second order equation set is driven by Generate second Krylov space using Compute models by projection  each equation order set separately, or  project original polynomial system using UNION of number of matched moments of order-i given by size of appropriate Krylov subspace

Computational Considerations Size of Krylov spaces grows quickly with functional series order  potentially large models  intrinsic in polynomial descriptions (e.g. Volterra series) Practical for weak nonlinearities needing only few terms in functional series  example: RF datapath  strong nonlinearities included in bias conditions Many local optimizations, extensions are possible  two-sided projectors  multiple reduction steps

RF mixer example MHz dB

Summary weakly nonlinear MOR Projection + polynomial series + variational analysis = rigorous nonlinear reduction procedure Compact Kronecker notation is very useful Is there a way to reduce number of terms in higher- order Krylov space?  Smaller models faster  Diagnose when models based on first order system are failing?  Follow-on reduction?

26 Model Order Reduciton for NonLinear Systems Representation of F(x) using a polynomial (e.g. Taylor’s expansions, Volterra Series) [Phillips00]***** Representation of F(x) using a polynomial (e.g. Taylor’s expansions, Volterra Series) [Phillips00]***** Representation of F(x) using several linearizations (Trajectory Piece-Wise Linear TPWL) [Rewienski01] Representation of F(x) using several linearizations (Trajectory Piece-Wise Linear TPWL) [Rewienski01] Representation of F(x) with several polynomials (PWP PieceWise Polynomial) [Dong03] Representation of F(x) with several polynomials (PWP PieceWise Polynomial) [Dong03]

27 III.3 - Reduction of Strongly NonLinear Systems via Trajectory PieceWise Linear Method Luca Daniel Massachusetts Institute of Technology Thanks to Michał Rewieński, Jacob White and Brad Bond

28 Linearizations around x i, i=0,…, s-1Linearizations around x i, i=0,…, s-1 Piecewise-linear representation [M. Rewienski, J. White ICCAD01] Weighted combination of the models:Weighted combination of the models: Project linearized models:Project linearized models:

29 x3x3 x1x1 x7x7 x6x6 x5x5 Use collection of linear models Background – TPWL [Reiwenski01] x1x1 x2x2 x 10 x8x8 x4x4 x2x2 x 14 x 15 xaxa xbxb x0x0 x 11 x9x9 x 13 #linearizations =#samples n n = 10 4 #samples = = LARGE Model i only valid near x i

30 Background – TPWL: Picking Linearization Points x1x1 x2x2 t y(t) Use training trajectories to pick linearization points Linearization at current state x i State Space Time Domain Simulation

31 Quasi-piecewise-linear MOR – computing weights  For i=0,…,(s-1) compute:  Compute  For i=0,…,(s-1) compute:  Normalize w i.

32 Background – TPWL Reduction of the Linearized Systems Model from linearization 1 A1A1 A1A1 K1K1 K1K1 Model from linearization 2 A2A2 A2A2 K2K2 K2K2 Model from linearization k AkAk AkAk KkKk KkKk =

33 Background – TPWL Constructing the projection matrix V Use moments from EACH linear model to construct V =

34 Background – TPWL: Weighting / Simulation [Riewinski01,Tiwary05,Dong05] Use weighting functions to combine linear models during simulation x1x1 x2x2 Current state Linearization 3 Linearization 2 Linearization 1 C – poorly approximated also well approximated Well approximated

35 Test example of a nonlinear circuit:

36 Computational results – circuit example Input:Input: training input testing input

37 Computational results - circuit example, SSS Har- monics Full nonlinear model Reduced order PWL model Error [%] dc (c 0 ) st (c 1 ) i i0.2 2 nd (c 2 ) i i rd (c 3 ) i i13.5

38 Example of a microswitch DISCRETIZE

39 Computational results - microswitch example testing input training input Input:

40 Computational results - microswitch example

41 training input testing input Computational results – microswitch example

MOR Meeting42 Simulation with the piecewise-linear model

43 x3x3 x2x2 x1x1 x0x0 Generate the reduced linear model Fast approximate simulation x1x1 x2x2 A Use the reduced linear system for simulation

44 Computational results - model order reduction * Matlab implementation

45  70 MOSFETs  13 resistors  9 capacitors  51 nodes OpAmp: training input testing input Computational results – OpAmp example

46  70 MOSFETs  13 resistors  9 capacitors  51 nodes OpAmp: Computational results – OpAmp example

47 Har- monics Full nonlinear model Reduced order TPWL model Error [%] Dc f 2 -f i i2.4 2f 1 -f 2 f 1 f 2 2f 2 -f 1 2f i i i i i i i i i i Computational results – OpAmp example, SSS

48 Computational results – shock propagation in jet engine inlet  Density disturbance d :

49 Computational results – shock propagation in 1D Burgers equation:

50 Questions yet to be answered: How should we pick training inputs? How “often” should we compute linearized models along the training trajectory? What are the errors of representing a nonlinear systems as a trajectory piecewise linear system? What are “the best” reduced order bases for the trajectory PWL model?

51 TPWL macromodels can be accurate for a number of strongly nonlinear dynamical systems, outperforming reduced models based on single-state polynomial expansions. TPWL models use simple, cost-efficient representation of system’s nonlinearity, resulting in very short simulation times. TPWL models may be extracted very efficiently by using a fast approximate simulator. Conclusions

52 Model Order Reduction of Non-Linear Systems Introduction Reduction of weakly nonlinear systems (Volterra Series) Reduction of strongly nonlinear systems (TPWL) Trajectory Pieace Wise Linear and Polynomial  with moment matching  with Truncated Balance Realization

III.4 Reduction of Strongly NonLinear Systems via Trajectory-PieceWise Linear (TPWL) + Truncated Balance Realization (TBR) Luca Daniel Massachusetts Institute of Technology Thanks to Dmitry Vasilyev, Michał Rewieński, Jacob White

Obtaining projection basis Krylov-subspace methods Fast Don’t guarantee accuracy Balanced-truncation methods Expensive (~n 3 ) Guarantee accuracy For example, V=W= colspan(A -1 B, A -2 B, …, A -q B) Let’s now use this!

Use TPWL to handle nonlinearity Before we used Krylov-subspace linear reduction (less accurate) Here we use TBR for projection matrices W and V Our Approach: x0x0 x1x1 x2x2 xnxn …

TBR reduction LTI SYSTEM X (state) t u t y Hankel operator Past input Future output P (controllability) Which states are easier to reach? Q (observability) Which states produces more output? TBR algorithm includes into projection basis most controllable and most observable states

Numerical results – RLC transmission line Error in transient || y r – y || 2 Order of the reduced model TBR-based TPWL beat Krylov-based 4-th order TBR TPWL reaches the limit of TPWL representation

Reducing cost of TBR reduction - Combined Krylov-TBR algorithm Initial Model: (A B C), n Intermediate Model: (A i B i C i ), n i Reduced Model: (A r B r C r ), q Krylov reduction ( W i, V i ): A i = W i T AV i B i = W i T B C i = CV i TBR reduction ( W t, V t ): A r = W t T AV t B r = W t T B C r = CV t

Micromachined device example non-symmetric indefinite Jacobian FD model

* Matlab implementation Initial model size N TBR TPWL q=6 Krylov-TBR TPWL, q=6 Krylov TPWL, q = s30.57 s26.34 s s8.57 s7.75 s s2.73 s3.03 s Performance of Krylov- TBR TPWL MOR extraction procedures* Cost of Krylov-TBR almost equals Krylov

Comparing accuracies of Krylov TPWL method and TBR-based TPWL algorithm Accuracy in transient * Order of the reduced model needed to achieve this accuracy for our models Krylov-based TPWL TBR-based TPWL 5%122 1% %>306 *Testing input equal to training input 5x reduction in order – 125x improvement in efficiency

Future work Are TBR-based TPWL models valid for unstable linearizations? What about systems having the following form (i.e. circuits with nonlinear capacitors):

Conclusions TBR-based linear reduction procedure to generate TPWL reduced models Order reduced 5 times while maintaining comparable accuracy with Krylov TPWL method (efficiency improved 125 times!) Combined Krylov-TBR reduction allows to extract TPWL models at low cost

64 Course Outline Numerical Simulation Quick intro to PDE Solvers Quick intro to ODE Solvers Model Order reduction Linear systems Common engineering practice Optimal techniques in terms of model accuracy Efficient techniques in terms of time and memory Non-Linear Systems Parameterized Model Order Reduction Linear Systems Non-Linear Systems Tomorrow Today