1. UCI ICS IGB SISL NKS Washington DC 06/15/06 Towards a Searchable Space of Dynamical System Models Eric Mjolsness Scientific Inference Systems Laboratory (SISL) University of California, Irvine www.ics.uci.edu/~emj In collaboration with: Guy Yosiphon NKS June 2006

2. UCI ICS IGB SISL NKS Washington DC 06/15/06 Motivations shared with NKS Objective exploration of properties of simple computational systems Relation of such to the sciences Example: bit string lexical ordering of cellular automata rules; reducibility relationships; applications to fluid flow

3. UCI ICS IGB SISL NKS Washington DC 06/15/06 Criteria for a space of simple formal systems C1: Demonstrated expressive power in scientific modeling C2: Representation as discrete labeled graph structure –that can be searched and explored computationally –E.g. Bayes nets, Markov Random Fields roughly in order of increasing size - with index nodes (DDs) C3: Self-applicability –useful transformations and searches of such dynamical systems should be expressible … as discrete-time dynamical systems that compute So major changes of representation during learning are not excluded.

4. UCI ICS IGB SISL NKS Washington DC 06/15/06 C1: Demonstration of expressive power in scientific modeling

5. UCI ICS IGB SISL NKS Washington DC 06/15/06 Elementary Processes A(x) B(y) + C(z) with f (x, y, z) B(y) + C(z) A(x) with r (y, z, x) Examples –Chemical reaction networks w/o params –. –XXX from paper Effective conservation laws –E.g. N A (x) dx + N B (y) dy, N A (x) dx + N C (z) dz

6. UCI ICS IGB SISL NKS Washington DC 06/15/06 Amino Acid Syntheses Kmech: Yang, et al. Bioinformatics 21: 774-780, 2005 Amino acid synthesis : Yang et al., J. Biological Chemistry, 280(12):11224-32,, Mar 25 2005. GMWC modeling: Najdi et al., J. Bioinformatics and Comp. Biol., to appear 2006.

7. UCI ICS IGB SISL NKS Washington DC 06/15/06 Example: Anabaena Prusinkiewicz et al. model G. Yosiphon, SISL, UCI

8. UCI ICS IGB SISL NKS Washington DC 06/15/06 Example: Galaxy Morphology G. Yosiphon, SISL, UCI

9. UCI ICS IGB SISL NKS Washington DC 06/15/06 Example: Arabidopsis Shoot Apical Meristem (SAM)

10. UCI ICS IGB SISL NKS Washington DC 06/15/06 Co-visualization of raw and extracted nuclei data Quantification of growth

11. UCI ICS IGB SISL NKS Washington DC 06/15/06 PIN1-GFP expression Time- lapse imaging over 40 hrs (Marcus Heisler, Caltech)

12. UCI ICS IGB SISL NKS Washington DC 06/15/06 Dynamic Phyllotactic Model H. Jönnson, M. Heisler, B. Shapiro, E. Meyerowitz, E. Mjolsness - Proc. Natl Acad. Sci. 1/06 Emergence of new extended, interacting objects: floral meristem primordia. DGs at 3 scales: - molecular; - cellular; - multicellular.

13. UCI ICS IGB SISL NKS Washington DC 06/15/06 Model simulation on growing template

14. UCI ICS IGB SISL NKS Washington DC 06/15/06 Spatial Dynamics in Biological Development Reimplemented weak spring model in 1 page Applying to 1D stem cell niches with diffusion, in plant and animal tissues

15. UCI ICS IGB SISL NKS Washington DC 06/15/06 Ecology: predator-prey models with Elaine Wong, UCI

16. UCI ICS IGB SISL NKS Washington DC 06/15/06 Example: Hierarchical Clustering

17. UCI ICS IGB SISL NKS Washington DC 06/15/06 ML example: Hierarchical Clustering

18. UCI ICS IGB SISL NKS Washington DC 06/15/06 Logic Programming E.g. Horn clauses Rules Operators Project to fixed-point semantics

19. UCI ICS IGB SISL NKS Washington DC 06/15/06 An Operator Algebra for Processes Composition is by independent parallelism Create elementary processes from yet more elementary Basis operators –Term creation/annihilation operators: for each parm value, –Obeying Heisenberg algebra [a i, c j ] = i j or –Yet classical, not quantum, probabilities

20. UCI ICS IGB SISL NKS Washington DC 06/15/06 Basic Operator Algebra Composition Operations: +, * Operator algebra H 1 + H 2 H 1 * H 2 (noncommutative) Informal meaning independent, parallel occurrence instantaneous, serial co-occurrence Syntax parallel rules Multiple terms on LHS, RHS

21. UCI ICS IGB SISL NKS Washington DC 06/15/06 Time Evolution Operators Master equation: d p(t) / dt = H p(t) where 1·H = 0, e.g. H = (H) = H - 1· diag(1·H ) H = time evolution operator –can be infinite-dimensional Formal solution: p(t) = exp(t H) p(0)

22. UCI ICS IGB SISL NKS Washington DC 06/15/06 Discrete-Time Semantics of Stochastic Parameterized Grammars This formulation can also be used as a programming language, expressing algorithms.

23. UCI ICS IGB SISL NKS Washington DC 06/15/06 Algorithm Derivation: Conceptual Map DG rules stochastic program (H, e t H ) (H´, H´ n /(1· H´ n ·p)) Eulers formula Heisenberg Picture Time Ordered Product Expansion CBH (c) (d) Operator Space (high dim) Functional Operator Space Trotter Product Formula

24. UCI ICS IGB SISL NKS Washington DC 06/15/06 C2: Representation as discrete labeled graph structure that can be searched and explored computationally

25. UCI ICS IGB SISL NKS Washington DC 06/15/06 Basic Syntax for a Modeling Language: Stochastic Parameterized Grammars (SPGs) = set of rules Each rule has: –LHS RHS {keyword expression} * –Parameterized term instances within LHS and/or RHS –LHS, RHS: sets (of such terms) with Variables LHS matches subsets of parameterized term instances in the Pool –Keyword clauses specify probability rate, as a product Keyword: with –Algebraic sublanguage for probability rate functions rates are independent of # of other matches; oblivious. Rule/object : verb/noun : reaction/reactant bipartite graphs –… with complex labels

26. UCI ICS IGB SISL NKS Washington DC 06/15/06 Graph Meta-Grammar = 1 = 2 = 3 = 1 = 2

27. UCI ICS IGB SISL NKS Washington DC 06/15/06 Plenum SPG/DG implementation builds on Cellerator experience [Shapiro et al., Bioinformatics 19(5):677-678 2003] computer algebra embedding provides –probability rate language –Symbolic transformations to executability includes mixed stochastic/continuous sims

28. UCI ICS IGB SISL NKS Washington DC 06/15/06 SPG/DG Expressiveness Subsumes … Logic programming (w. Horn clauses) –LHS RHS; all probability rates equal –Hence, any simulation or inference algorithms can in principle be expressed as discrete-time SPGs Chemical reaction networks –No parameters; stoichiometry = weighted labeled bipartite graph Context-free (stochastic) grammars –No parameters; 1 input term/rule –Formally solvable with generating functions Stochastic (finite) Markov processes –No parameters; 1 input/rule, 1 output/rule –Solvable with matrices (or queuing theory?)

29. UCI ICS IGB SISL NKS Washington DC 06/15/06 SPG/DG Expressiveness Subsumes … Bayes Nets –Each variable x gets one rule: Unevaluated-term, {evaluated predecessors(y)} evaluated-term(x) MCMC dynamics –Inverse rule pairs satisfying detailed balance –Each rule can itself have the power of a Boltzmann distribution Probabilistic Object Models –Frameville, PRM, … Petri Nets Graph grammars –Hence, meta-grammars and grammar transformations DGs subsume: ODEs, SDEs, PDEs, SPDEs –Unification with SPGs too

30. UCI ICS IGB SISL NKS Washington DC 06/15/06 C3: Self-applicability -Arrow reversal -Arrow reversal graph grammar exercise -Machine learning by statistical inference -e.g. hierarchical clustering (reported) -? Equilibrium reaction networks for MRFs -Further possible applications …

31. UCI ICS IGB SISL NKS Washington DC 06/15/06 Template: A-Life Concisely expressed in SPGs Steady state condition: total influx into g = total outflow from g

32. UCI ICS IGB SISL NKS Washington DC 06/15/06 Applications to Dynamic Grammar Optimization and a Grammar Soup Map genones to grammars Map hazards to functionality tests Map reproduction to crossover or simulation

33. UCI ICS IGB SISL NKS Washington DC 06/15/06 Conclusions Stochastic process operators as the semantics for a language –A fundamental departure –Specializes to all other dynamics Deterministic, discrete-time, DE, computational, … Graph grammars allow meta-processing Operator algebra leads to novel algorithms Wide variety of examples at multiple scales –Sciences Cell, developmental biology; astronomy; geology multiscale integrated models –AI Pattern Recognition Machine learning Searchable space of simple dynamical system models including computations

34. UCI ICS IGB SISL NKS Washington DC 06/15/06 For More Information www.ics.uci.edu/~emj modeling frameworks