1. Automated Reverse Engineering of Nonlinear Dynamical Systems
July 9, 2007
Automated Reverse Engineering of
Nonlinear Dynamical Systems
Josh Bongard† & Hod Lipson
Computational Synthesis Laboratory
Cornell University
†Current Address: Department of Computer Science
University of Vermont

2. Model:
dθ/dt = ?
dω/dt = ?

3. +f(θ,ω) [f = friction term]
Best evolved model:
dθ/dt = 1.008ω
dω/dt = sin(1.0009θ+0/-1.575/-2.673)
+f(θ,ω) [f = friction term]
Best human-created model:
dθ/dt = ω
dω/dt = -9.8Lsin(θ) +f(θ,ω) [L = length of arm]
First reported algebraic model describing pendula published
by H. Kamerlingh Onnes in 1879.
Form of these equations still presented to this day in engineering textbooks as the mathematical description of a damped pendulum.
Meets criteria (B):
Our model matches the best-known model form for a
pendulum.
Our models contain explicit terms for the friction term f;
friction terms are unknown for physical, damped pendula—
dependent on mechanical coupling, shape of arm,
wind resistance, etc.
Our model automatically captures in symbolic form,
changes to the system (rotation of the main box).
Box is flat
Rotated -1.57rad
Rotated -2.67rad

4. Best evolved model:
dG/dt = ?
dA/dt = ?
dL/dt = ?

5. dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1)
Best evolved model:
dG/dt = 0.96A2/(0.96A2+1)
dA/dt = G( L/(L+1) – A/(A+1) )
dL/dt = -GL/(L+1)
dG/dt = A2/(A2+1) – 0.01G
dA/dt = G( L/(L+1) – A/(A+1) )
dL/dt = -GL/(L+1)

6. dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1)
Best evolved model:
dG/dt = 0.96A2/(0.96A2+1)
dA/dt = G( L/(L+1) – A/(A+1) )
dL/dt = -GL/(L+1)
Lac operon discovered in 1961:
Jacob F, Monod J (1961). "Genetic regulatory mechanisms
in the synthesis of proteins". J Mol Biol. 3:
As of 1998, the best human-created model was:
from
Keener J, Sneyd J (1998) Mathematical Physiology (Springer).
Criteria (B): The result is equal to or better than a result that was
accepted as a new scientific result at the time when it was
published in a peer-reviewed scientific journal.
Took humans 37 years to create this model from observations;
Took our algorithm 5 minutes running on a desktop PC to create
a very similar model from time series data.
dG/dt = A2/(A2+1) – 0.01G
dA/dt = G( L/(L+1) – A/(A+1) )
dL/dt = -GL/(L+1)

7. Best evolved model:
dH/dt = ?
dL/dt = ?

8. Best evolved model:
dH/dt = 3.42x H L
dL/dt = 3.10x H L

9. Best human-created model: dH/dt = αH – βHL dL/dt = -γL + δHL
Best evolved model:
dH/dt = 3.42x H L
dL/dt = 3.10x H L
Best human-created model:
dH/dt = αH – βHL
dL/dt = -γL + δHL
Proposed independently by American biophysicist Alfred Lotka and Italian mathematician Vito Volterra in 1925.
(Volterra, V. (1926) Jour. du conseil intl. pour l’exp. de la mer)
Still forms the basis of most population dynamics models today.
Meets criteria (B): Our model is similar, but simpler:
No nonlinear terms, yet still conveys:
which species in the prey, and which the predator;
prey decreases proportionally to amount of predator;
predator increases proportionally in response to prey;
when integrated, produces coupled oscillations;
predator peaks follow prey peaks.

10. Algorithm correctly inferred the underlying form in 5 minutes.
Criteria (G): The result solves a problem of indisputable difficulty in its field.
Algorithm presented with time series data from nonlinear, coupled systems of increasing size.
Algorithm correctly inferred the underlying form in 5 minutes.
System 1
System 2
System 3
2 variables
dx1/dt= -3x1x1 -3x1x2 +2x2x2
dx2/dt= -x1x1 -3x1x2 -2x2x2
dx1/dt=-3x1x1 +3x1x2 +3x2x2
dx2/dt=-3x1x1-2x1x2 +2x2x2
dx1/dt=3x1x1 -x1x2 -x2x2
dx2/dt=x1x1 +3x1y2 -x2x2
Success rate
20% (no partitioning)
100% (partitioning)
0% (no partitioning)
96.7% (partitioning)
3 variables
dx1/dt=-3x1x3 -2x2x3 -3x3x3
dx2/dt=-3x1x2 +x1x3 -3x2x3
dx3/dt=3x1x2 +3x1x3 –x2x3
dx1/dt=-x1x2 +x1x3 -x2x3
dx2/dt=x1x1 +2x1x2 +2x2x3
dx3/dt=-2x1x1 +x1x2 -3x2x3
dx1/dt=-3x1x2 +x1x3 -x3x3
dx2/dt=-2x1x3 +3x2x3 +3x3x3
dx3/dt=2x1x2 -2x1x3 -2x2x3
63.3% (partitioning)
4 variables
dx1/dt=-x1x1+2x2x3+2x3x3
dx2/dt=x1x2 -3x1x3 -3x2x3
dx3/dt=-x1x1 -x2x4 +3x4x4
dx4/dt=-3x1x2 -3x1x4 –3x3x4
dx1/dt=x1x4+x2x4+x4x4
dx2/dt=-3x1x2-2x2x3-3x3x4
dx3/dt=2x1x2-x1x3+2x2x2
dx4/dt=x1x3+3x2x3-x3x4
dx1/dt=-3x1x1+3x1x2+3x2x4
dx2/dt=-x1x1-2x1x3-3x4x4
dx3/dt=-2x1x4+x2x2-3x3x4
dx4/dt=-x1x2+2x1x4-3x3x4
90% (partitioning)
83.3% (partitioning)
5 variables
dx1/dt=-3x1x5 +3x2x3 -3x2x5
dx2/dt=-3x1x3 -2x3x4 -x4x5
dx3/dt=x1x1 -3x1x4 +x2x4
dx4/dt=3x1x3 -3x1x4 +2x2x2
dx5/dt=3x1x4 +3x3x3 +3x3x4
dx1/dt=-2x2x2+3x3x5+2x4x5
dx2/dt=3x1x2 +x1x5 -2x2x5
dx3/dt=x1x2 +2x2x5 +2x4x5
dx4/dt=2x1x2 +3x1x5 -x4x5
dx5/dt=2x1x5 -x2x5 -2x5x5
dx1/dt=2x1x4 +2x2x3 -x2x4
dx2/dt=x1x3 +3x1x4 +x2x4
dx3/dt=-2x1x1 +2x1x2 -3x1x3
dx4/dt=-3x2x5 +3x3x4 -x3x5
dx5/dt=x1x1 +x1x5 +x2x3
76.7% (partitioning)
6 variables
dx1/dt=-2x1x6+x2x4-2x2x6
dx2/dt=x1x4-x1x5-2x4x4
dx3/dt=2x2x5-x3x4+x5x5
dx4/dt=-3x4x5-2x4x6+2x5x5
dx5/dt=x3x6-2x4x4-3x4x5
dx6/dt=x3x4-x3x6+2x4x6
dx1/dt=-2x1x3-3x2x4+2x3x6
dx2/dt=-3x2x4+x3x4-x3x6
dx3/dt=-x1x2-x1x3+x4x6
dx4/dt=-x1x4+x3x5-2x4x6
dx5/dt=3x1x2-3x1x6-x5x5
dx6/dt=-3x1x3-2x1x6-3x4x6
dx1/dt=x1x5+x1x6+x4x5
dx2/dt=-2x2x5-2x2x6+2x3x6
dx3/dt=-x1x5-2x3x4+x4x4
dx4/dt=3x1x2+3x2x3-2x4x5
dx5/dt=-3x1x5+x2x2+3x2x6
dx6/dt=-x2x5-2x3x5-3x5x6
80% (partitioning)
70% (partitioning)
93% (partitioning)
2 variables
Best-known
predator-prey model;
single pendulum model
3 variables
Best-known
lac operon model of E. coli
as of 1998
5 variables
Best-known
lac operon model of E. coli
as of 2003
(Yildirim & Mackey, 2003)

11. Why should this entry be considered “best”?
Work appeared in one of the world’s most prestigious scientific journals,
and not in an evolutionary computation journal:
Was favorably reviewed by four renowned scientists outside of the EC community.
One reviewer remarked that this approach
“…has a chance of changing the way some disciplines conduct science”.
Citation—Bongard J. and Lipson H.(2007). Automated reverse engineering of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 104(24):