1. Model Order Reduction Luca Daniel University of California, Berkeley Massachusetts Institute of Technology with contributions from: Joel Phillips, Cadence Berkeley Labs Jacob White, Massachusetts Instit. of Technology

2. 2 Conventional Design Flow Funct. Spec Logic Synth. Gate-level Net. RTL Layout Floorplanning Place & Route Front-end Back-end Behav. Simul. Gate-Lev. Sim. Stat. Wire Model Parasitic Extrac.

3. 3 Layout parasitics Wires are not ideal. Parasitics:Wires are not ideal. Parasitics: –Resistance –Capacitance –Inductance Why do we care?Why do we care? –Impact on delay –noise –energy consumption –power distribution Picture from “Digital Integrated Circuits”, Rabaey, Chandrakasan, Nikolic

4. 4 Parasitic Extraction Parasitic Extraction thousands of wires e.g. critical path e.g. gnd/vdd grid tens of circuit elements for gate level spice simulation identify some ports produce equivalent circuit that models response of wires at those ports

5. 5 Parasitic Extraction (the two steps) Electromagnetic Analysis (Tuesday) million of elements thin volume filaments with constant current small surface panels with constant charge Model Order Reduction (Today) tens of elements

6. 6 Why build reduced models? Compression for EfficiencyCompression for Efficiency –It is possible to represent the system under study “precisely” with millions of elements –But the simulation is too slow with the complicated representation AbstractionAbstraction –I do not care at all about the precise representation –In fact I would rather those details were not even there. I may not be able to create or manipulate the precise representation at all.

7. 7 Challenges for reduction algorithms AccuracyAccuracy –Must be controllable and predictable EfficiencyEfficiency –Algorithms should be scalable to handle large systems Numerical robustnessNumerical robustness –Algorithms should work reliably for all reasonable inputs & accuracy requests Models must work in simulationModels must work in simulation –Composability : Combination of two good models is a good model

8. 8 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

9. 9 State-Space Models Linear system of ordinary differential equations (ABCD form)Linear system of ordinary differential equations (ABCD form) State Input Output

10. 10 State-Space Model Example: Interconnect Segment Step 1: Identify internal state variablesStep 1: Identify internal state variables –Example : MNA uses node voltages & inductor current

11. 11 State-Space Model Example: Interconnect Segment Step 2: Identify inputs & outputsStep 2: Identify inputs & outputs –Example : For Z-parameter representation, choose port currents inputs and port voltage outputs    

12. 12 State-Space Model Example: Interconnect Segment Step 3: Write state-space & I/O equationsStep 3: Write state-space & I/O equations –Example : KCL + inductor equation    

13. 13 State-Space Model Example: Interconnect Segment Step 4: Identify state variables & matricesStep 4: Identify state variables & matrices

14. 14 A linear circuit can be expressed as a state space model So in general….So in general….LARGE!

15. 15 A canonical form for model order reduction Assuming A is non- singular we can cast the dynamical linear system into one canonical form for model order reduction Note: not necessarily always the best, but the simplest for educational purposes

16. 16 Our goal: smaller model, still accurate 500,000 x 500,000 Given a large linear system model:Given a large linear system model: 10 x 10 Construct a linear system model with:Construct a linear system model with: -smaller complexity -same fidelity -small reduction cost

17. 17 Key Transform Property: Bilateral Laplace Transform: Frequency Domain Representation

18. 18 Express y(s) as a function of u(s) Transfer Function: System Transfer Function

19. 19 Connection Between the Transfer Function Time Domain Impulse Response Frequency domain representation H(s) u(s)y(s) = H(s) u(s) Linear system h(t) u(t) Linear system Time domain representation The transfer function H(s) is the Laplace Transform of the impulse response h(t)

20. 20 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

21. 21 Model Order Reduction via Eigenmode Analysis Pole-Residue Form Pole-Zero Form (SISO) Ideas for reducing order:Ideas for reducing order: –Drop terms with small residues –Drop terms with large (“fast” modes) –Remove pole/zero near-cancellations –Cluster poles that are “together” How to compute poles and residues?How to compute poles and residues?

22. 22 Computing Poles & Residues Poles are eigenvalues of E -1Poles are eigenvalues of E -1 Diagonalize E residues poles

23. 23 Eigenvalue Based Reduction AdvantagesAdvantages –Conceptually familiar –Simple physical interpretation : retains dominant system modes/poles DrawbacksDrawbacks –Relatively expensive : have to find the poles first –Relatively inefficient. For a given model size, many other approaches can provide better accuracy Rule of thumbRule of thumb –Anything that can be done by manipulating pole/eigenvalues/eigenvectors can probably be done better with more sophisticated analysis, at the same or smaller cost.

24. 24 Defining Accuracy Time-domain response should be “close”Time-domain response should be “close” –For which possible inputs? Frequency response should matchFrequency response should match –At what frequencies?

25. 25 Matching Frequency Response Ensure accuracy for only some inputs?Ensure accuracy for only some inputs? Example:Example: –low frequency inputs, –or some band, –or some points in the frequency response Original matching some part of the frequency response

26. 26 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

27. 27 Original System Transfer Function: Model Reduction = Find a low order (q << N) rational function matching rational function matching Model Order Reduction via Rational Transfer Function Fitting rational function reduced order rational function

28. 28 Reduced Model Dynamical System Reduced Model Transfer Function coefficients coefficients Rational Transfer Function Fitting: Degrees of Freedom

29. 29 Reduced Model Transfer Function Apply any invertible change of variables to the state Many Dynamical Systems have the same transfer function!! Rational Transfer Function Fitting: Degrees of Freedom (cont.) I I

30. 30 Rational Transfer Function Fitting: via Point Matching For i = 1 to 2q cross multiplying generates a linear systemcross multiplying generates a linear system Can match 2q pointsCan match 2q points

31. 31 Columns contain progressively higher powers of the test frequencies: problem is numerically ill-conditionedColumns contain progressively higher powers of the test frequencies: problem is numerically ill-conditioned also... missing data can cause severe accuracy problemsalso... missing data can cause severe accuracy problems Rational Transfer Function Fitting: Point Matching matrix can be ill-conditioned

32. 32 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

33. 33 Point matching vs. Moment Matching Point matching: can be very inaccurate in between points Moment (derivatives) matching: accurate around expansion point, but inaccurate on wide frequency band

34. 34 The Taylor coef. = frequency domain moments = = derivatives of the transfer function (up to a constant) Frequency Domain "Moments" (or Taylor coefficients) of the transfer function Taylor Series Expansion of the original transfer function around s=0

35. 35 Time domain moments of the impulse response Definition:

36. 36 Connection to the time-domain moments of the circuit response Time-domain moments Compare: Hence the the Taylor coeff. are, up to a constant, the time-domain moments of the circuit response.

37. 37 Rational function fitting via moment matching: Pade Approximation (AWE)

38. 38 Rational function fitting via moment matching: Pade Approximation (AWE) –Step 1: calculate the first 2q moments of H(s) –Step 2: calculate the 2q coeff. of the Pade’ approx, matching the first 2q moments of H(s)

39. 39 Step 2: Calculation of Pade’ coeff. (AWE) For coeff. a’s solve the following linear system: For coeff. b’s simply calculate:

40. 40 Pade matrix can be very ill-conditioned matrix powers converge to the eigenvector corresponding to the largest eigenvalue.matrix powers converge to the eigenvector corresponding to the largest eigenvalue. Columns become linearly dependent for large q the problem is numerically very ill-conditioned!

41. 41 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

42. 42 Projection Framework: Change of variables Note: q << N reduced state original state

43. 43 Original SystemOriginal System SubstituteSubstitute Projection Framework Note: now few variables (q<<N) in the state, but still thousands of equations (N)Note: now few variables (q<<N) in the state, but still thousands of equations (N)

44. 44 Projection Framework (cont.) Reduction of number of equations: test multiplying by V q TReduction of number of equations: test multiplying by V q T If V and U biorthogonalIf V and U biorthogonal

45. 45 nxn qxq nxq qxn Projection Framework (cont.)

46. 46 Equation Testing Change of variables Projection Framework

47. 47 Use EigenvectorsUse Eigenvectors Use Time Series DataUse Time Series Data –Compute –Use the SVD to pick q < k important vectors Use Frequency Domain DataUse Frequency Domain Data –Compute –Use the SVD to pick q < k important vectors Use Singular Vectors of System Grammians?Use Singular Vectors of System Grammians? Use Krylov Subspace Vectors?Use Krylov Subspace Vectors? Approaches for picking V and U

48. 48 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

49. 49 Taylor series expansion: U U Intuitive view of Krylov subspace choice for change of base projection matrix change base and use only the first few vectors of the Taylor series expansion: equivalent to match first derivatives around expansion pointchange base and use only the first few vectors of the Taylor series expansion: equivalent to match first derivatives around expansion point

50. 50 Combine point and moment matching: multipoint moment matching Multipole expansion points give larger band Multipole expansion points give larger band Moment (derivates) matching gives more Moment (derivates) matching gives more accurate behavior in between expansion points accurate behavior in between expansion points

51. 51 Compare Pade’ Approximations and Krylov Subspace Projection Framework Krylov Subspace Projection Framework: multipoint moment multipoint moment matching matching numerically very numerically very stable!!! stable!!! Pade approximations: moment matching at moment matching at single DC point single DC point numerically very numerically very ill-conditioned!!! ill-conditioned!!!

52. 52 Aside on Krylov Subspaces - Definition The order k Krylov subspace generated from matrix A and vector b is defined as

53. 53 If and Then Projection Framework: Moment Matching Theorem (E. Grimme 97)

54. 54 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

55. 55 If U and V are such that: Then the first q moments (derivatives) of the reduced system match Special simple case #1: expansion at s=0,V=U, orthonormal U T U=I

56. 56 Algebraic proof of case #1: expansion at s=0, V=U, orthonormal U T U=I apply k times lemma in next slide

57. 57 Lemma:. Note in general: BUT... Substitute: IqIqIqIq U is orthonormal

58. 58 Vectors will line up with dominant eigenspace! Need for Orthonormalization of U

59. 59 Need for Orthonormalization of U (cont.) In "change of base matrix" U transforming to the new reduced state space, we can use ANY columns that span the reduced state spaceIn "change of base matrix" U transforming to the new reduced state space, we can use ANY columns that span the reduced state space In particular we can ORTHONORMALIZE the Krylov subspace vectorsIn particular we can ORTHONORMALIZE the Krylov subspace vectors

60. 60 Normalize new vector For i = 1 to k Generates k+1 vectors! Orthogonalize new vector For j = 1 to i Orthonormalization of U: The Arnoldi Algorithm

61. 61 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

62. 62 Then the first 2q moments of reduced system match If U and V are such that: Special case #2: expansion at s=0, biorthogonal V T U=I

63. 63 Proof of special case #2: expansion at s=0, biorthogonal V T U=U T V=I q (cont.) apply k times the lemma in next slide

64. 64 Lemma:. Substitute: Substitute: IqIqIqIq biorthonormality IqIqIqIq biorthonormality

65. 65 PVL: Pade Via Lanczos [P. Feldmann, R. W. Freund TCAD95] PVL is an implementation of the biorthogonal case 2:PVL is an implementation of the biorthogonal case 2: Use Lanczos process to biorthonormalize the columns of U and V: gives very good numerical stability

66. 66 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

67. 67 Case #3: Intuitive view of subspace choice for general expansion points In stead of expanding around only s=0 we can expand around another pointsIn stead of expanding around only s=0 we can expand around another points For each expansion point the problem can then be put again in the standard formFor each expansion point the problem can then be put again in the standard form

68. 68 Case #3: Intuitive view of Krylov subspace choice for general expansion points (cont.) matches first k j of transfer function around each expansion point s j Hence choosing Krylov subspace s 1 =0 s1s1s1s1 s2s2s2s2 s3s3s3s3

69. 69 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

70. 70 Interconnected Systems ROM Can we assure that the simulation of the composite system will be well-behaved? At least preclude non- physical behavior of the reduced model?Can we assure that the simulation of the composite system will be well-behaved? At least preclude non- physical behavior of the reduced model? In reality, reduced models are only useful when connected together with other models and circuit elements in a composite simulationIn reality, reduced models are only useful when connected together with other models and circuit elements in a composite simulation Consider a state-space model connected to external circuitry (possibly with feedback!)Consider a state-space model connected to external circuitry (possibly with feedback!)

71. 71 Passivity Passive systems do not generate energy. We cannot extract out more energy than is stored. A passive system does not provide energy that is not in its storage elements.Passive systems do not generate energy. We cannot extract out more energy than is stored. A passive system does not provide energy that is not in its storage elements. l If the reduced model is not passive it can generate energy from nothingness and the simulation will explode

72. 72 Interconnecting Passive Systems QD C - + + - QD C - + + - QD C - + + - QD C - + + - The interconnection of stable models is not necessarily stableThe interconnection of stable models is not necessarily stable BUT the interconnection of passive models is a passive model:BUT the interconnection of passive models is a passive model:

73. 73 Positive Real Functions A positive real function is a function internally stable with non-negative real partA positive real function is a function internally stable with non-negative real part (no unstable poles) (no negative resistors) (real response) Hermittian=conjugate and transposed It means its real part is a positive semidefinite matrix at all frequencies

74. 74 Positive Realness & Passivity For systems with immittance (impedance or admittance) matrix representation, positive-realness of the transfer function is equivalent to passivityFor systems with immittance (impedance or admittance) matrix representation, positive-realness of the transfer function is equivalent to passivity ROM    

75. 75 Necessary conditions for passivity for Poles/Zeros The positive-real condition on the matrix rational function implies that:The positive-real condition on the matrix rational function implies that: –If H(s) is positive-real also its inverse is positive real –If H(s) is positive-real it has no poles in the RHP, and hence also no zeros there. Occasional misconception : “if the system function has no poles and no zeros in the RHP the system is passive”.Occasional misconception : “if the system function has no poles and no zeros in the RHP the system is passive”. It is necessary that a positive-real function have no poles or zeros in the RHP, but not sufficient.It is necessary that a positive-real function have no poles or zeros in the RHP, but not sufficient.

76. 76 Sufficient conditions for passivity Sufficient conditions for passivity:Sufficient conditions for passivity: Note that these are NOT necessary conditions (common misconception)Note that these are NOT necessary conditions (common misconception)

77. 77 Congruence Transformations Preserve Positive Semidefinitness Def. congruence transformationDef. congruence transformation same matrix Note: case #1 in the projection framework V=U produces congruence transformationsNote: case #1 in the projection framework V=U produces congruence transformations Property: a congruence transformation preserves the positive semidefiniteness of the matrixProperty: a congruence transformation preserves the positive semidefiniteness of the matrix Proof. Just renameProof. Just rename Note:Note:

78. 78 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA

79. 79 PRIMA (for preserving passivity) (Odabasioglu, Celik, Pileggi TCAD98) A different implementation of case #1: V=U, U T U=I, Arnoldi Krylov Projection Framework: Use Arnoldi: Numerically very stable

80. 80 PRIMA preserves passivity The main difference between and case #1 and PRIMA:The main difference between and case #1 and PRIMA: case #1 applies the projection framework tocase #1 applies the projection framework to PRIMA applies the projection framework toPRIMA applies the projection framework to PRIMA preserves passivity becausePRIMA preserves passivity because –uses Arnoldi so that U=V and the projection becomes a congruence transformation –E and A produced by electromagnetic analysis are typically positive semidefinite while may not be. –input matrix must be equal to output matrix

81. 81 Algebraic proof of moment matching for PRIMA expansion at s=0, V=U, orthonormal U T U=I Used Lemma: If U is orthonormal (U T U=I) and b is a vector such that

82. 82 Proof of lemma Proof:

83. 83 Compare methods number of moments matched by model of order q preserving passivity case #1 (Arnoldi, V=U, U T U=I on sA -1 Ex=x+Bu) qno PRIMA (Arnoldi, V=U, U T U=I on sEx=Ax+Bu) qyes necessary when model is used in a time domain simulator case #2 (PVL, Lanczos,V≠U, V T U=I on sA -1 Ex=x+Bu) 2q more efficient no (good only if model is used in frequency domain)

84. 84 Overview Introduction and MotivationsIntroduction and Motivations State-space modelsState-space models Reduction via eigenmode analysisReduction via eigenmode analysis Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) Reduction via moment matching (Pade, AWE)Reduction via moment matching (Pade, AWE) Reduction via moment matching: (Projection Framework)Reduction via moment matching: (Projection Framework) –general Krylov Subspace methods –case 1: Arnoldi –case 2: PVL –case 3: multipoint moment matching Importance of preserving passivityImportance of preserving passivity –PRIMA Summary and ConclusionsSummary and Conclusions

85. 85 Summary: Conventional Design Flow Funct. Spec Logic Synth. Gate-level Net. RTL Layout Floorplanning Place & Route Front-end Back-end Behav. Simul. Gate-Lev. Sim. Stat. Wire Model Parasitic Extrac.

86. 86 Summary: Parasitic Extraction Parasitic Extraction thousands of wires e.g. critical path e.g. gnd/vdd grid tens of circuit elements for gate level spice simulation identify some ports produce equivalent circuit that models response of wires at those ports

87. 87 Summary: Model Order Reduction (the second step of parasitic extraction) Electromagnetic Analysis (Tuesday) Model Order Reduction (Today) tens of elements million of elements thin volume filaments with constant current small surface panels with constant charge

88. 88 Conclusions Reduction via eigenmodesReduction via eigenmodes –expensive and inefficient Reduction via rational function fitting (point matching)Reduction via rational function fitting (point matching) –inaccurate in between points, numerically ill-conditioned Reduction via moment matching: Pade approximationsReduction via moment matching: Pade approximations –better behavior but covers small frequency band –numerically very ill-conditioned Reduction via moment matching: Krylov Subspace Projection FrameworkReduction via moment matching: Krylov Subspace Projection Framework –allows multipoint expansion moment matching (wider frequency band) –numerically very robust –use PVL is mode efficient for model in frequency domain –use PRIMA to preserve passivity if model is for time domain simulator