1. Princeton University Department of Chemical Engineering and PACM Equation-Free Uncertainty Quantification: Applications to Stochastic Chemical Reactions and Biological Oscillators Yu Zou and Ioannis G. Kevrekidis Department of Chemical Engineering and PACM Princeton University Collaborators: Sung-Joon Moon 1, Katherine A. Bold 1, Michael A. Henson 2 1 Dept. of Che. Engr. and PACM, Princeton University 2 Department of Chemical Engineering University of Massachusetts, Amherst United Technologies Research Center, East Hartford, CT August 21, 2006

2. Princeton University Department of Chemical Engineering and PACM Outline 1.Background for Uncertainty Quantification 2.Fundamentals of Polynomial Chaos 3.Stochastic Galerkin Method 4.Equation-Free Uncertainty Quantification (EF UQ) 5.EF UQ in Stochastic Catalytic Reactions 6.EF UQ in Yeast Glycolytic Oscillations 7.EF UQ in Kuramoto Coupled Oscillators 8.Conclusions

3. Princeton University Department of Chemical Engineering and PACM Background for Uncertainty Quantification Uncertain Phenomena in science and engineering * Inherent Uncertainty: Uncertainty Principle of quantum mechanics, Kinetic theory of gas, … * Uncertainty due to lack of knowledge: randomness of BC, IC and parameters in a mathematical model, measurement errors associated with an inaccurate instrument, … Scopes of application * Estimate and predict propagation of probabilities for model variables: chemical reactants, biological oscillators, stock and bond values, structural random vibration,… * Design and decision making in risk management: optimal selection of parameters in a manufacturing process, assessment of an investment to achieve maximum profit,... * Evaluate and update model predictions via experimental data: validate accuracy of a stochastic model based on experiment, data assimilation, … Modeling Techniques * Sampling methods (non-intrusive): Monte Carlo sampling, Quasi Monte Carlo, Latin Hypercube Sampling, Quadrature/Cubature rules * Non-sampling methods (intrusive) : perturbation methods, higher-order moment analysis, stochastic Galerkin method

4. Princeton University Department of Chemical Engineering and PACM Polynomial Chaos: Orthogonal polynomials of independent random variables =0 if i≠j, with Spectral expansion (Ghanem and Spanos, 1991) a j ’s are called PC coefficients. Notes Selection of Ψ j is dependent on the probability measure or distribution of, e.g., (Wiener-Askey Scheme by Xiu and Karniadarkis, 2002) if is a Gaussian measure, then {Ψ j }are Hermite polynomials; if is a Lesbeque measure, then {Ψ j }are Legendre polynomials. Fundamentals of Polynomial Chaos

5. Princeton University Department of Chemical Engineering and PACM Preliminary Formulation * Model: e.g., ODE * Represent the input in terms of expansion of independent r.v.’s (KL, SVD, POD): e.g., time-dependent parameter * Represent the response in terms of the truncated PC expansion * The solution process involves solving for the PC coefficients α j (t), j=1,2,…,P Model Input: random IC, BC, parameters Response: Solution Stochastic Galerkin (PC expansion) Method (Ghanem and Spanos, 1991)

6. Princeton University Department of Chemical Engineering and PACM Solution technique: Galerkin projection resulting in coupled ODE’s for α j (t), where Advantages and weakness * PC expansion has exponential convergence rate * Model reduction * Free of moment closure problems * Correlation between parameters and solutions ? The coupled ODE’s of PC coefficients may not be obtained explicitly Stochastic Galerkin Method

7. Princeton University Department of Chemical Engineering and PACM Coarse time-stepper (Kevrekidis et al., 2003, 2004) * Lifting (MC, quadrature/cubature): * Microsimulation: * Restriction: For Monte Carlo sampling, For quadrature/cubature-points sampling, is the weight associated with each sampling point. Equation-free Uncertainty Quantification Lifting Restriction Microsimulation

8. Princeton University Department of Chemical Engineering and PACM Projective Integration (Kevrekidis et al., 2003, 2004) LiftingRestriction Fixed-point Computation (Kevrekidis et al., 2003, 2004) α=ΦT(α)α=ΦT(α) Newton’s Method Newton-Krylov GMRES (Kelly, 1995) Δt c (adaptive stepsize control) Δt s (≥t relaxation +h opt )

9. Princeton University Department of Chemical Engineering and PACM Stochastic Catalytic Reactions A (CO) +1/2 B 2 (O 2 ) → AB (CO 2 ) COO2O2 CO 2 vacancy : random parameter set F(Θ) ? +

10. Princeton University Department of Chemical Engineering and PACM Stochastic catalytic reactions: A three-level EF UQ Θ(t) θ(ξ,t) lifting θ(ξ,t+Δt) Θ(t+Δt) micro-simulation restriction Equation Free: Quantities estimated on demand (Kevrekidis et al., 2003, 2004) θ(ξ,t): mean coverages of reactants in catalytic reactions A (CO) +1/2 B 2 (O 2 ) → AB (CO 2 ) x N A (t), N B (t), N * (t)N A (t+Δt), N B (t+Δt), N * (t+Δt) lifting N A =int(N tot θ A )+1 with p A1 int(N tot θ A ) with p A0 The same to N B θ A = /N tot θ B = /N tot restriction Time-stepper Gillespie p1p1 reaction time Gillespie Algorithm

11. Princeton University Department of Chemical Engineering and PACM Stochasic catalytic reactions Projective Integration gPC coefficients Mean coverages Number of sites gPC coefficients Mean coverages Number of sites gPC coefficients Mean coverages Number of sites gPC coefficients Mean coverages Number of sites gPC coefficients Mean coverages Number of sites lifting restriction lifting integrate ΔtfΔtf Δt cc (adaptive stepsize control) Random Steady-state Computation gPC coefficients Mean coverages Number of sites gPC coefficients Mean coverages Number of sites lifting restriction T ΦTΦT Θ=ΦT(Θ)Θ=ΦT(Θ) Δt s (≥t relaxation +h opt )

12. Princeton University Department of Chemical Engineering and PACM Stochastic catalytic reactions α=1.6, γ= 0.04, k r =4, β=6.0+0.25ξ, ξ~U[-1,1] gPC coefficients computed by ensemble average Number of gPC coefficients: 12 N e of θ A, θ B and θ * : 40,000 N e of N A, N B and N * : 1,000 N tot of surface sites: 200 2 Projective Integration

13. Princeton University Department of Chemical Engineering and PACM Projective Integration Stochastic catalytic reactions

14. Princeton University Department of Chemical Engineering and PACM Projective Integration α=1.6, γ = 0.04, k r =4, β=6.0+0.25ξ, ξ~U[-1,1] gPC coefficients computed by Gauss-Legendre quadrature Number of gPC coefficients: 12 N e of θ A, θ B and θ * : 200 N e of N A, N B and N * : 1,000 N tot of surface sites: 200 2 Stochastic catalytic reactions

15. Princeton University Department of Chemical Engineering and PACM α=1.6, γ= 0.04, k r =4, β=6.0+0.25ξ, ξ~U[-1,1] gPC coefficients computed by ensemble average Number of gPC coefficients: 12 N e of θ A, θ B and θ * : 200 N e of N A, N B and N * : 1,000 N tot of surface sites: 200 2 Stochastic catalytic reactions

16. Princeton University Department of Chemical Engineering and PACM Random Steady-State Computation α=1.6, γ= 0.04, k r =4 β= +0.25ξ,, ξ~U[-1,1] gPC coefficients computed by ensemble average Number of gPC coefficients: 12 N e of θ A, θ B and θ * : 40,000 N e of N A, N B and N * : 1,000 N tot of surface sites: 200 2 Stochastic catalytic reactions

17. Princeton University Department of Chemical Engineering and PACM Random Steady-State Computation α=1.6, γ = 0.04, k r =4 β= +0.25ξ,, ξ~U[-1,1] gPC coefficients computed by Gauss-Legendre quadrature Number of gPC coefficients: 12 Ne of θ A, θ B and θ * : 200 N e of N A, N B and N * : 1,000 N tot of surface sites: 200 2 Stochastic catalytic reactions

18. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations (Wolf and Heinrich, Biochem. J. (2000) 345, p321-334) glucose J0J0 glyceraldehyde-3-P/ dihydroxyacetone-P NADHNAD + glycerol v1v1 v2v2 NAD + NADH 1,3-bisphospho-glycerate v3v3 ATP ADP ATP v5v5 pyruvate/acetaldehyde pyruvate/acetaldehyde ex J NADHNAD + v6v6 v4v4 ethanol external environment v7v7 cytosol Notation: A 2 - ADP A 3 - ATP, A 2 +A 3 = A(const) N 1 - NAD + N 2 - NADH, N 1 +N 2 = N(const) S 1 - glucose S 2 - glyceraldehyde-3-P/ dihydroxyacetone-P S 3 - 1,3-bisphospho -glycerate S 4 - pyruvate/acetaldehyde S 4 ex - pyruvate/acetaldehyde ex J 0 - influx of glucose J - outflux of pyruvate/ acetaldehyde Reaction rates: v 1 = k 1 S 1 A 3 [1+(A 3 /K I ) q ] -1 v 2 = k 2 S 2 N 1 v 3 = k 3 S 3 A 2 v 4 = k 4 S 4 N 2 v 5 = k 5 A 3 v 6 = k 6 S 2 N 2 v 7 = kS 4 ex Reaction scheme for a single cell

19. Princeton University Department of Chemical Engineering and PACM Coupled ODEs for multicellular species concentrations Yeast Glycolytic Oscillations

20. Princeton University Department of Chemical Engineering and PACM Heterogeneity of the coupled model: Yeast Glycolytic Oscillations Polynomial Chaos expansion of the solution: Lifting: Fine variables: M – number of cells; M = 1000 Coarse variables: α j and S 4 ex (25 variables totally) (6M+1 variables) Restriction: Minimizingto obtain α j Stratified Sampling (1D LH)

21. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Full ensemble simulation

22. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Projective integration of PC coefficients S1S1 S2S2 S3S3 S4S4 N2N2 A3A3 tt N2N2 A3A3 A phase map of zeroth-order PC coef’s through projective integration Time histories of zeroth-order PC coef’s restricted from the full-ensemble simulation

23. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Limit-cycle computation Poincaré section fixed point limit cycle ___ limit cycle in the space of PC coefficients xxx restricted PC coefficients of a limit cycle of the full-ensemble simulation A3A3 N2N2 Phase maps of zeroth-order PC coef’s through limit-cycle computation Poincaré section: In the space of coarse variables, zeroth-order PC coef. of N 2 is constant In the space of fine variables, N 2 of a single cell is constant

24. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Stability of limit cycles Eigenvalues of Jacobians of the flow maps in the coarse and fine variable spaces Flow map T – period of the limit cycle real imaginary x - PC coefficients o - full-ensemble simulation

25. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Different liftings

26. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Different liftings

27. Princeton University Department of Chemical Engineering and PACM Yeast Glycolytic Oscillations Free oscillator Zeroth-order PC coefficient of N 2 (CPI) N 2 of the free cell (CPI)

28. Princeton University Department of Chemical Engineering and PACM Kuramoto coupled oscillators Lifting: Restriction:

29. Princeton University Department of Chemical Engineering and PACM Kuramoto coupled oscillators

30. Princeton University Department of Chemical Engineering and PACM Conclusions and remarks EF UQ is applied to stochastic dynamical systems with a single random parameter. The EF UQ may be extended to include three levels (scales) for stochastic catalytic reactions. An example with a simple model is shown. Possible extensions to models with multiple random parameters or driven by stochastic processes. References Zou, Y., and Kevrekidis, I.G., Equation-Free Uncertainty Quantification on heterogeneous catalytic reactions, in preparation, available at http://arnold.princeton.edu/~yzou/ Bold, K.A., Zou, Y., Kevrekidis, I.G., and Henson, M.A., Efficient simulation of coupled biological oscillators through Equation-Free Uncertainty Quantification, in preparation, available at http://arnold.princeton.edu/~yzou

31. Princeton University Department of Chemical Engineering and PACM Some other works

32. Princeton University Department of Chemical Engineering and PACM Hybrid QMOM-MC Computation on Particle Coagulation and Sintering Processes Yu Zou, Ioannis G. Kevrekidis Department of Chemical Engineering and PACM Princeton University Rodney O. Fox Department of Chemical and Biological Engineering Iowa State University

33. Princeton University Department of Chemical Engineering and PACM Outline 1.Introduction to particle coagulation and sintering processes 2.Simulation Techniques Monte-Carlo Simulations Population Balance Equations Quadrature Methods of Moments 3. A problem with the Constant-Number MC Scheme 4.Equation-Free Projective Integration 5. Hybrid Univariate QMOM-MC Simulations 6. Hybrid Bivariate QMOM-MC Simulations

34. Princeton University Department of Chemical Engineering and PACM Introduction to particle coagulation and sintering processes particle coagulation http://www.bath.ac.uk/~chsscp/group/ S i O 2 nanoparticle http://www.nano.fraunhofer.de/de/ institute/kompetenz_izm_cit.html polymer www.msm.cam.ac.uk dust cells www.bloodlines.stemcells.com particle sintering

35. Princeton University Department of Chemical Engineering and PACM Monte-Carlo Simulation Schemes Constant-Volume approach (Shah et al. 1977, Liffman 1992) Constant-Number approach (Matsoukas et al. 1991, Smith and Matsoukas 1998) Δt1Δt1 Δt2Δt2 Mean time between successive events: …….. p ~ U(0,1) Δt1Δt1 Δt2Δt2 1,2,….,i,…,j,…,N Yes No coagulation Inter-event time: Self-preserving volume moments:

36. Princeton University Department of Chemical Engineering and PACM Population Balance Equations Univariate number density equation (Smoluchowski, 1916) Bivariate number density equation

37. Princeton University Department of Chemical Engineering and PACM Quadrature Methods of Moments (McGraw 1997) Univariate equation of moments Bivariate equation of mixed moments Product difference algorithm (Gordon 1968) or quadrature-finding subroutines orthog & zrhqr (Numerical Recipe in C) Minimizing (Wright et al. 2001)

38. Princeton University Department of Chemical Engineering and PACM A problem with the Constant-Number MC Scheme Equations in QMOM are for constant-volume processes only. time moments What’s the equation for n(v,t) or n(v,a,t) in the C-N scheme?

39. Princeton University Department of Chemical Engineering and PACM Equation-Free Projective Integration (Kevrekidis et al. 2003, 2004) Lifting Restriction ΔtdΔtd ΔtrΔtr ΔteΔte

40. Princeton University Department of Chemical Engineering and PACM Hybrid Univariate QMOM-CNMC Simulations M 0, M 1/3, M 2/3, M 1, M 4/3, M 5/3 Product difference algorithm or quadrature-finding subroutines orthog & zrhqr v 1, w 1 ; v 2, w 2 ; v 3, w 3 p ~ U(0,1) w 1 w 2 w 3 M 0, M 1/3, M 2/3, M 1, M 4/3, M 5/3 ΔtκΔtκ ΔtκΔtκ ΔtκΔtκ ΔtκΔtκ

41. Princeton University Department of Chemical Engineering and PACM Off-line test (moment derivatives and differentiation step size) h dt

42. Princeton University Department of Chemical Engineering and PACM Off-line test (6 moments and their derivatives – 3 Q-points) ‘true’ lifting relative error ‘true’ lifting relative error h=0.02 Closure of 6 moments

43. Princeton University Department of Chemical Engineering and PACM Off-line test (higher-order moments – 3 Q-points) ‘true’ lifting relative error

44. Princeton University Department of Chemical Engineering and PACM Off-line test (6 moments and their derivatives – 2 Q-points) ‘true’ lifting relative error ‘true’ lifting relative error h=0.02 Closure of 4 moments

45. Princeton University Department of Chemical Engineering and PACM Off-line test (higher-order moments – 2 Q-points) ‘true’ lifting relative error

46. Princeton University Department of Chemical Engineering and PACM Off-line test (6 moments and their derivatives – 1 Q-point) ‘true’ lifting relative error ‘true’ lifting relative error h=0.02

47. Princeton University Department of Chemical Engineering and PACM Off-line test (higher-order moments – 1 Q-point) ‘true’ lifting relative error

48. Princeton University Department of Chemical Engineering and PACM Illustration of Projective Integration

49. Princeton University Department of Chemical Engineering and PACM Numerical Results time moments MC Hybrid QMOM-MC 0.05-0.05-0.4 Δ t d - Δt r - Δt e ΔteΔte CPU time (sec)

50. Princeton University Department of Chemical Engineering and PACM Numerical Results moments 0.05-0.05-0.2 Δ t d - Δt r - Δt e MC Hybrid QMOM-MC time ΔteΔte CPU time (sec)

51. Princeton University Department of Chemical Engineering and PACM Numerical Results moments 0.05-0.05-0.1 Δ t d - Δt r - Δt e MC Hybrid QMOM-MC time ΔteΔte CPU time (sec) MC Hybrid QMOM-MC

52. Princeton University Department of Chemical Engineering and PACM Numerical Results time self-preserving moments 0.05-0.05-0.4 Δ t d - Δt r - Δt e QMOM Hybrid QMOM-MC

53. Princeton University Department of Chemical Engineering and PACM Numerical Results time self-preserving moments 0.05-0.05-0.2 Δ t d - Δt r - Δt e QMOM Hybrid QMOM-MC

54. Princeton University Department of Chemical Engineering and PACM Numerical Results time self-preserving moments 0.05-0.05-0.1 Δ t d - Δt r - Δt e QMOM Hybrid QMOM-MC

55. Princeton University Department of Chemical Engineering and PACM Hybrid Bivariate QMOM-CNMC Simulations M i/3,j/3 i,j=0,1,…,5 v k, a k, w k k=1,2, …, 12 p ~ U(0,1) w 1 … w 12 ΔtκΔtκ M i/3,j/3 i,j=0,1,…,5 Minimizing through the Conjugate Gradient method surface area restructuring ΔtκΔtκ

56. Princeton University Department of Chemical Engineering and PACM Numerical results 0.05-0.05-0.2Δ t d - Δt r - Δt e Hybrid QMOM-MCMC

57. Princeton University Department of Chemical Engineering and PACM Numerical results Hybrid QMOM-MCQMOM 0.05-0.05-0.2Δ t d - Δt r - Δt e

58. Princeton University Department of Chemical Engineering and PACM Numerical results 0.05-0.05-0.1Δ t d - Δt r - Δt e Hybrid QMOM-MCMC

59. Princeton University Department of Chemical Engineering and PACM Numerical results Hybrid QMOM-MCQMOM 0.05-0.05-0.1Δ t d - Δt r - Δt e

60. Princeton University Department of Chemical Engineering and PACM Remarks A hybrid computation is implemented to circumvent the difficulty of no moment equations for the CNMC. Computational load of the hybrid computation is between that of the CNMC and of the QMOM. Lower-order moments can be closed by fewer quadrature points. However, if high-order moments are expected to be well represented, a large number of points have to be used. Reference Y. Zou, I.G. Kevrekidis and R. Fox, Hybrid QMOM-MC computation for particle coagulation and sintering processes, in preparation, 2006, available at http://arnold.princeton.edu/~yzou

61. Princeton University Department of Chemical Engineering and PACM Multiscale Analysis on Re-entrant Production Lines: An Equation-Free Approach Institute of Industrial Engineers Annual Conference and Exposition Orlando, Florida May 20-24, 2006 Yu Zou, Ioannis G. Kevrekidis Department of Chemical Engineering and PACM Princeton University Dieter Armbruster Department of Mathematics, Arizona State University

62. Princeton University Department of Chemical Engineering and PACM Outline 1.Re-entrant production lines 2.Phase model and density equations 3.Time-scale separation and Equation-Free analysis 4.The Lifting step in the EF approach 5.Results of Coarse Projective Integration 6.Results of Coarse Steady-State Computation 7.Remarks

63. Princeton University Department of Chemical Engineering and PACM A Normal Production Line Distribution of processing time (TPT) is determined by the total number of items in process (WIP) at arrival: e n = a n + τ n τ n ~ p(r, WIP(a n )) A Re-entrant Production Line Processing time τ n is affected by later arrivals: τ n = τ n (t) = τ n (WIP(t)) A phase model to determine e n : (Armbruster and Ringhofer, 2005) dφ/dt = 1/ τ n (t), φ(a n )=0, e n is such that φ(e n )=1

64. Princeton University Department of Chemical Engineering and PACM A discretized form of the phase model φ(t+1/ω) = φ(t) + 1/(ωτ n (t)), φ(a n )=0 τ n (t)~ p(r, WIP(t)) ω: update frequency An algorithm for the discretized phase model (a) φ(t+Δt) = φ(t) + Δt/τ(t), Δt < 1/ω τ(t+Δt) = κ(t)η(t)+(1- κ(t))τ(t), t ≥ a n (b) p(κ(t) =1) = ω(τ(t), t)Δt, p(κ(t) =0) =1- ω(τ(t), t) Δt η(t)~ p(r, WIP(t)) (c) φ(a n )=0, τ(a n ) ~ p(r, WIP(a n )) ω(r,t) is chosen as in Armbruster and Ringhofer, 2005

65. Princeton University Department of Chemical Engineering and PACM λ0λ0 σp0σp0 Influx, distribution of throughput time and their characteristic scales

66. Princeton University Department of Chemical Engineering and PACM Density equations ( Armbruster and Ringhofer, 2005 ) joint density of phase and throughput time density of phase In the limit that,

67. Princeton University Department of Chemical Engineering and PACM Time-scale separation τ n (t) t ρ(x,t) t Equation-free analysis (Kevrekidis et al. 2003, 2004) LiftingRestriction Discrete Simulation Phase φ n, TPT τ n Time stepper time scale: O(1/ω)time scale: O(TPT) TPT Phase φ n, TPT τ n ρ(x) k ρ(x) k+1

68. Princeton University Department of Chemical Engineering and PACM Coarse Projective Integration Coarse steady-state computation LiftingRestriction Discrete Simulation Phase φ n, TPT τ n ρ(x)=Φ T (ρ(x)) ρ(x) k ρ(x) k+1 Newton Newton-Krylov GMRES

69. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach Q1: Under what condition does there exist a closed equation for ρ(x,t) or is f(x,r,t) a functional of ρ(x,t)? Q2: How to generate phase and TPT for items from ρ(x,t) under this condition? The only dimensionless parameter: λ 0 σ p 0 λ0λ0 σp0σp0

70. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) Off-line tests t λ(t) TPT p(r,t) 0.1T λ 0 λ=0.5, T=2λ=10, T=2λ=20, T=2 t=16s conditional PDF f(r|x,t) = f(x,r,t)/ρ(x,t)

71. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) λ=0.5, T=4λ=10, T=4λ=20, T=4 λ=0.5, T=8λ=10, T=8λ=20, T=8

72. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) λ=10, T=4λ=20, T=8 cross sections of conditional PDF f(r|x,t) at t=16s Conclusions: as λ 0 σ p 0 » 1, f(r|x,t) = rp(r,t)/C; C=∫rp(r,t)dr i.e., f(x,r,t)= ρ(x,t)rp(r,t)/∫rp(r,t)dr

73. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) The algorithm for lifting 1.Calculate WIP(t): WIP(t) = ∫ρ(x,t)dx 2.Normalized number density f ρ (x,t) = ρ(x,t)/ WIP(t) 3.Let a = int(WIP)+1-WIP; p ~ U[0,1]; if p<a, set the total number of items to int(WIP); else, int(WIP)+1 4.Use ICDF of f ρ (x,t) to generate phases of items 5.Use ICDF of f(r|x,t) to generate TPTs of items 01ξ phase or TPT

74. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) Reinitialization tests – particular realizations of lifting don’t matter! t λ(t) λ 0 t=10s t ρ(x,t) 0 t=10s Restricted densities after lifting True densities

75. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) Relative error of number density after lifting – test a time phase coordinate

76. Princeton University Department of Chemical Engineering and PACM The lifting step in the EF approach (cont’d) time phase coordinate Relative error of number density after lifting – test b

77. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration

78. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time WIP Δt c =0.1sΔt h =0.01s Δt f =0.01s

79. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time WIP

80. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time WIP

81. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time WIP

82. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time WIP

83. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) ΔtcΔtc ΔtcΔtc CPU time (sec) DES PDE

84. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time outflux

85. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time outflux

86. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time outflux

87. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) time outflux

88. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) DES CPI

89. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) POD (Proper Orthogonal Decomposition)-assisted analysis pλpλp normalized λ p 125.677700.68645 25.441510.14547 33.194400.08540 41.170880.03130 50.653900.01748 60.456070.01219 70.440630.01178 80.371350.00993 94.84013e-141.29393e-15

90. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) POD-assisted analysis

91. Princeton University Department of Chemical Engineering and PACM Results of Coarse Projective Integration (cont’d) POD-assisted analysis

92. Princeton University Department of Chemical Engineering and PACM Results of Coarse Steady-State Computation ρ(x)=Φ T (ρ(x)), 0<x<1

93. Princeton University Department of Chemical Engineering and PACM Results of Coarse Steady-State Computation (cont’d)

94. Princeton University Department of Chemical Engineering and PACM Results of Coarse Steady-State Computation (cont’d) A=  (ρ(x))=  Φ T (ρ(x))=  Φ T (  (A))=Ψ T (A), A=(WIP,a 1,a 2,a 3 ) T POD-assisted analysis

95. Princeton University Department of Chemical Engineering and PACM Remarks 1.The EF was applied to computation of re-entrant production lines; a key step is the lifting. 2.Off-line tests were performed to find the ‘constitutive’ equation between f(x,r,t) and ρ(x,t). For more general problems, we should validate if it is still effective to do so. 3. POD can be used to make the analysis more efficient. Reference Y. Zou, I.G. Kevrekidis, D. Armbruster, Multiscale analysis of re-entrant production lines: An equation-free approach, Physica A 363 (2006), pp1-13

96. Princeton University Department of Chemical Engineering and PACM Equation-Free Analysis on IEM Model Yu Zou, Ioannis G. Kevrekidis Department of Chemical Engineering and PACM Princeton University

97. Department of Chemical Engineering and PACM Outline 1.The IEM Model 2.A sample reaction system 3.The simulation scheme 4.Coarse-level PDE description ? 5.Coarse Projective Integration 6.Numerical results

98. Princeton University Department of Chemical Engineering and PACM IEM (interaction-by-exchange-with-the-mean) Model Age α Stream 1 γ 1 Stream 2 γ 2 Mean concentration ageentrance time Internal age distribution

99. Princeton University Department of Chemical Engineering and PACM A sample reaction system Only one stream γ 1 = 1 A dimensionless form

100. Princeton University Department of Chemical Engineering and PACM The simulation scheme ω = α/τ θ = t/τ Δω

101. Princeton University Department of Chemical Engineering and PACM age ω concentrations

102. Princeton University Department of Chemical Engineering and PACM Coarse-level PDE description ? Coarse Projective Integration

103. Princeton University Department of Chemical Engineering and PACM ω = α/τ θ = t/τ restriction lifting Coarse Projective Integration (cont’d)

104. Princeton University Department of Chemical Engineering and PACM Numerical Results 20-20-40 age ω concentrations

105. Princeton University Department of Chemical Engineering and PACM Numerical Results 20-20-40 age ω concentrations

106. Princeton University Department of Chemical Engineering and PACM Numerical Results 20-20-20 age ω concentrations

107. Princeton University Department of Chemical Engineering and PACM Numerical Results

108. Equation-Free Particle-Based Computations in Multiple Dimensions: Coarse Projective Integration and Coarse Renormalization Yu Zou Roger Ghanem Department of Civil Engineering Johns Hopkins University Ioannis Kevrekidis Department of Chemical Engineering Princeton University SIAM Conference on Computational Science and Engineering Orlando, Florida February 14, 2005

109. Introduction Coarse time-stepper for multidimensional particle systems Coarse projective integration for multidimensional particle systems Coarse dynamic renormalization for multidimensional particle systems Self-similar and asymptotic self-similar Brownian particle systems in a Couette flow Numerical examples Conclusions and remarks Outline Equation-Free

110. 1. Equation-free computations for multiscale one-dimensional particle systems Coarse-scale description Fine-scale description Coarse time-stepper (Theodoropoulos, Qian and Kevrekidis, 2000) Introduction Equation-Free y: PDF, CDF or Inverse CDF of particle positions X: particle positions LiftingRestriction Micro Evolution

111. Introduction Coarse projective integration (Gear and Kevrekidis et al, 2002) Equation-Free

112. Coarse dynamic renormalization (Aronson, Betelu and Kevrekidis, 2001; Chen, Debenedetti, Gear and Kevrekidis, 2004) Introduction Equation-Free Template condition

113. Introduction 2. Difficulties The equation-free computations for particle systems were restricted to macroscopically one-dimensional cases. They need to be extended to multiple- dimension situations. Multidimensional CDFs do not have inverses. Direct representation of CDFs may cause CDFs to be out of bound [0,1]. How to choose appropriate coarse observables? How to implement the dynamic renormalization for multidimensional self-similar systems? How to implement the coarse renormalization for self-similar particle systems whose explicit macro PDEs are not known? How to determine the template condition(s)? Do we need multiple templates? Equation-Free

114. Microscale observable: particle positions Macroscale observable: Marginal and conditional inverse CDFs (ICDF) for particle positions Coarse time stepper for multidimensional particle systems Marginal and conditional ICDFs Particle positions Marginal and conditional ICDFs liftingrestriction microscale evolution Equation-Free

115. Components for the coarse time stepper Lifting 1. Generating y-direction particle positions in terms of the marginal ICDF; 2. Generating corresponding x-direction particle positions in terms of the conditional ICDFs; Microscale evolution Evolving particle positions via microscale simulators Restriction 1. Sorting y-direction particle positions and generating the numerical marginal ICDF 2. Numerically computing the CDF values in a 2-dimensional lattice grid 3. Numerically computing conditional CDFs via the CDF grid values 4. Numerically computing conditional ICDFs via the conditional CDFs by interpolation Equation-Free

116. Coarse projective integration for multidimensional particle systems Coarse observables: IF Y (f) and M IF X|Y (f,y k c ) Steps: 1.Lifting 2.Fine-scale evolution 3.Successive restrictions 4.Project ICDFs onto some basis 5. Estimate temporal derivatives; Extrapolate or integrate coefficients of dominant modes 6. Reconstruct the ICDFs Basis: Analytical polynomials (e.g., Shifted Legendre polynomials) Synthesized via Singular Value Decomposition (KL expansion) Equation-Free e.g., Euler or Adams-Bashforth

117. Equation We focus on D xy that has the following symmetric property: There exist an a and p such that for any positive real number A and any function f(x,y) Note: 1. This property is the property of D xy itself 2. It is the necessary condition for the equation to have a self-similar solution. Dynamic Renormalization of Multidimensional PDEs Equation-Free

118. For the symmetric operator D xy the PDE may have a one- parameter family of self-similar solution: where c is a constant, s=t-t 0, and t 0 is the blow-up time. Then If D xy is not explicitly available, the last equation cannot be solved. The renormalized self-similar CDF should be determined in an alternative way. Consider the general scaling then the PDE becomes Dynamic Renormalization of Multidimensional PDEs Equation-Free

119. The template condition to solve for A(t): The x-coordinates corresponding to the rescaled marginal CDFω X =m have the same value e for all t. Impose this template to the PDE for ω, Dynamic renormalization: Dynamic Renormalization of Multidimensional PDEs Equation-Free

120. Dynamical determination of the similarity exponent α : Dynamic Renormalization of Multidimensional PDEs Equation-Free

121. The implication of equations for dynamic renormalization Steady-state ω k Steady-state ω k+1 F k (= ω k ) F k+1 Particle positions at k Particle positions at k+1 From dynamic renormalization to coarse renormalization In the co-expanding frame In the original frame Impose the template to get A. Rescaled by A and A p in two directions In the micro scale LiftingRestriction Equation-Free

122. Coarse renormalization for multidimensional particle systems Initial CDF Marginal and conditional ICDFs lifting Coefficients α Particle positions expansionprojection microscale evolution time interval T’ template condition restriction renormalization Φ T’ rescaling variable A Rescaled marginal and conditional ICDFs Marginal and conditional ICDFs Report rescaled particle positions and CDF Equation-Free

123. Let f(x,y) be an exponential function. If there exist such an a and p, then the transcendental equation should have a solution for p, which is independent of choice of f(x,y), A, (x 1,y 1 ) and (x 2,y 2 ). The previous equation can be rewritten as Let A criterion to check the symmetric property of D xy Equation-Free

124. In case that D xy is not explicitly available, A criterion to judge the symmetric property of D xy M(p) p p 0 p 0 +Δpp1p1 p2p2 Newton’s Method liftingrestriction Equation-Free

125. A self-similar Brownian particle system in a Couette flow Microscale simulator for evolution of particle positions (passive Brownian particles with 1d diffusion in a Couette flow) Corresponding macroscale PDE for evolution of CDF of particle positions (Majda and Kramer, 1999) Self-similar solution to the macroscale PDE due to the δ-function initial condition (inspired by Okubo and Karweit, 1969) Renormalized self-similar solution to the macroscale PDE Equation-Free

126. An asymptotic self-similar Brownian particle system in a Couette flow Microscale simulator for evolution of particle positions (passive Brownian particles with 2d diffusion in a Couette flow) Corresponding macroscale PDE for evolution of CDF of particle positions (Majda and Kramer, 1999) Asymptotic self-similar solution to the macroscale PDE due to the δ- function initial condition (inspired by Okubo and Karweit, 1969) Renormalized asymptotic self-similar solution to the macroscale PDE Equation-Free

127. Numerical examples: True evolution of particle positions and CDFs ΔtΔt10 Δt20 Δt80 Δt90 Δt macroscale microscale lifting restriction CDFParticle positions microscale evolution Initial macroscale condition: Uniform distribution over the square region (-10cm,10cm)×(-10cm,10cm) Number of particles: 2000 Evolution time: 9 seconds Microscale evolution time interval Δt: 0.01 sec Reporting time: 10kΔt sec, k=0,1,2,…,90 The self-similar particle system Equation-Free

128. macroscale lifting restriction T ext CDFParticle positionsMarginal and conditional ICDFs extrapolation microscale T rel T res Initial macroscale condition: Uniform distribution over the square region (-10cm,10cm)× (-10cm,10cm) Number of particles: 2000 Macroscale observable: 1 marginal ICDF and 20 conditional ICDFs Bases for macroscale projection: Shifted Legendre polynomials up to the 5 th order Extrapolation technique: Least square fitting Evolution time: 9 seconds Microscale evolution time interval Δt: 0.01 sec Relaxation time interval T rel : 10Δt; Restriction time interval T res : 10Δt; Extrapolation time steps T ext : 10Δt Reporting time: 10kΔt sec, k=0,1,2,…,90 Numerical examples: Coarse projective integration of particle positions and CDFs microscale evolution The self-similar particle system Equation-Free

129. True evolution vs. coarse projective integration (ICDFs and mode coefficients) The marginal and 1 st conditional ICDFs and their 1 st -order mode coefficients True evolution CPI evolution Equation-Free

130. True evolution vs. coarse projection integration (particle positions) Evolution of true particle positionsEvolution of particle positions via CPI Equation-Free

131. True evolution vs. coarse projective integration (CDF) Evolution of the true CDFEvolution of the CPI CDF Equation-Free

132. True evolution vs. coarse projective integration (cross section of CDF) Comparison for the 45° cross sections of CDFs shows small discrepancy between the true CDF and CPI CDF for a short time evolution. Equation-Free

133. Self-similar Brownian particle system: Check the symmetric property of the implicit macroscale operator D xy f(x,y)=1/16 N(x/4) N(y/4), A=2, f(x,y)=1/25 N(x/5) N(y/5), A=2.5 (u 1,v 1 )=(-2,-2), and (u 2,v 2 )=(3,3). (u 1,v 1 )=(-3,-3), and (u 2,v 2 )=(4,4). Numerical examples: Coarse renormalization of the self-similar particle system No. of iterationspa 05.0-3.34959 12.80093-1.88311 22.99246-2.03967 33.00106-2.04212 43.00370-2.05468 52.99592-2.03927 62.99659-2.03914 72.99753-2.04017 82.99831-2.04189 p=3.0, a≈-2.0, α≈0.5 No. of iterationsPa 05.0-3.38239 13.19621-2.25167 22.97334-2.06146 32.99987-2.08557 42.99574-2.08444 52.99347-2.08425 62.99747-2.08345 72.99247-2.08073 82.99833-2.08601 Equation-Free

134. Fixed-point problem: α =Φ T’ ( α ) Template condition: The x-coordinates corresponding to the renormalized marginal CDF F X =0.4 have the same value -2.832. (c=0.2sec -1 ) Macroscale observable: 1 marginal ICDF and 20 conditional ICDFs Bases for macroscale projection: Shifted Legendre polynomials up to the 5 th order Microscale evolution time step : Δt =0.01 sec T’ =100Δt. Coarse renormalization of the self-similar particle system Equation-Free

135. Evolution of renormalized particle positions (self-similar system) Evolution of renormalized particle positionsEvolution of true particle positions Equation-Free

136. Evolution of renormalized CDF (self-similar system) Evolution of the renormalized CDFEvolution of the true CDF Equation-Free

137. Evolution of a cross section of the renormalized CDF (self-similar system) The real CDF varies with the time while the renormalized CDF remains almost unchanged after a short time of evolution. Equation-Free

138. Comparison between theoretical and simulated renormalized CDFs (self-similar system) Equation-Free

139. Dynamical Determination of Similarity Exponent (self-similar system) tA(t)AtAt 01.00000---- t 1 =1sec1.102680.10268 t 2 =3sec1.277930.08763 ω ω ω ω t1t1 t2t2 Equation-Free

140. Effect of template conditions and evolution time T’ (self-similar system) Equation-Free

141. Asymptotic self-similar Brownian particle system: Check the symmetric property of the implicit macroscale operator D xy f(x,y)=1/16 N(x/4) N(y/4), A=2, f(x,y)=1/25 N(x/5) N(y/5), A=2.5 (u 1,v 1 )=(-2,-2), and (u 2,v 2 )=(3,3). (u 1,v 1 )=(-3,-3), and (u 2,v 2 )=(4,4). Numerical examples: Coarse renormalization of the asymptotic self-similar particle system No. of iterationspa 05.0-3.87764 14.14589-3.32940 23.81151-3.09345 33.78553-3.08460 43.79408-3.08433 53.78576-3.08486 63.79758-3.08472 73.78701-3.09319 83.79517-3.08471 No. of iterationspa 05.0-4.23757 14.47814-3.85506 22.58903-1.95833 34.01019-3.38693 43.64906-2.96049 53.35744-2.69786 63.39829-2.74037 73.40776-2.75770 83.41325-2.74170 Equation-Free

142. Effect of template conditions and evolution time T’ (asymptotic self-similar system) Equation-Free

143. Conclusions and remarks Marginal and conditional ICDFs of particle positions are ideal macroscale observables to characterize the macroscale evolution of multidimensional particle systems. Coarse Projective Integration employing marginal and conditional ICDFs as macroscale observables result in the CDF evolution that agrees well with true predictions. Macroscale PDEs with differential operators possessing the symmetric property may have self-similar solutions. The Coarse renormalization method can effectively obtain the self-similar shapes for these PDEs. Equation-Free

144. Conclusions and remarks For coarse renormalization of multidimensional self-similar particle systems, only a single template condition is needed. The templates in two directions are dependent. The dependency is determined by the macroscale operator D xy itself. Coarse renormalization can also be used to analyze asymptotic self-similar particle systems. A long-time steady state of asymptotic self-similar solution can be obtained via the fixed-point operator as long as the evolution time in the operator is sufficiently large. Equation-Free