1. 3. Optimization Methods for Molecular Modeling by Barak Raveh

2. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

3. Prerequisites for Tracing the Minimal Energy Conformation I. The energy function: The in-silico energy function should correlate with the (intractable) physical free energy. In particular, they should share the same global energy minimum. II. The sampling strategy: Our sampling strategy should efficiently scan the (enormous) space of protein conformations

4. The Problem: Find Global Minimum on a Rough One Dimensional Surface rough = has multitude of local minima in a multitude of scales. *Adapted from slides by Chen Kaeasar, Ben-Gurion University

5. The landscape is rough because both small pits and the Sea of Galilee are local minima. The Problem: Find Global Minimum on a Rough Two Dimensional Surface *Adapted from slides by Chen Kaeasar, Ben-Gurion University

6. The Problem: Find Global Minimum on a Rough Multi-Dimensional Surface A protein conformation is defined by the set of Cartesian atom coordinates (x,y,z) or by Internal coordinates (φ /ψ/χ torsion angles ; bond angles ; bond lengths) The conformation space of a protein with 100 residues has ≈ 3000 dimensions The X-ray structure of a protein is a point in this space. A 3000-dimensional space cannot be systematically sampled, visualized or comprehended. *Adapted from slides by Chen Kaeasar, Ben-Gurion University

7. Characteristics of the Protein Energetic Landscape smooth?rugged? Images by Ken Dill space of conformations energy

8. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

9. Example: removing clashes from X-ray models Local Minimization Allows the Correction of Minor Local Errors in Structural Models *Adapted from slides by Chen Kaeasar, Ben-Gurion University

10. The path to the closest local minimum = local minimization What kind of minima do we want?

11. *Adapted from slides by Chen Kaeasar, Ben-Gurion University The path to the closest local minimum = local minimization What kind of minima do we want?

12. *Adapted from slides by Chen Kaeasar, Ben-Gurion University The path to the closest local minimum = local minimization What kind of minima do we want?

13. *Adapted from slides by Chen Kaeasar, Ben-Gurion University The path to the closest local minimum = local minimization What kind of minima do we want?

14. *Adapted from slides by Chen Kaeasar, Ben-Gurion University The path to the closest local minimum = local minimization What kind of minima do we want?

15. *Adapted from slides by Chen Kaeasar, Ben-Gurion University The path to the closest local minimum = local minimization What kind of minima do we want?

16. *Adapted from slides by Chen Kaeasar, Ben-Gurion University The path to the closest local minimum = local minimization What kind of minima do we want?

17. A Little Math – Gradients and Hessians Gradients and Hessians generalize the first and second derivatives (respectively) of multi-variate scalar functions ( = functions from vectors to scalars) GradientHessian Energy = f(x 1, y 1, z 1, …, x n, y n, z n )

18. Analytical Energy Gradient (i) Cartesian Coordinates Energy, work and force: recall that  Energy ( = work) is defined as force integrated over distance  Energy gradient in Cartesian coordinates = vector of forces that act upon atoms (but this is not exactly so for statistical energy functions, that aim at the free energy ΔG) E = f(x 1, y 1,z 1, …, x n, y n, z n ) Example: Van der-Waals energy between pairs of atoms – O(n 2 ) pairs:

19. Enrichment: Transforming a gradient between Cartesian and Internal coordinates (see Abe, Braun, Nogoti and Gö, 1984 ; Wedemeyer and Baker, 2003) Consider an infinitesimal rotation of a vector r around a unit vector n . From physical mechanics, it can be shown that: Analytical Energy Gradient (ii) Internal Coordinates (torsions, etc.) E = f(  1,  1,  1,  11,  12, …) Note: For simplicity, bond lengths and bond angles are often ignored cross product – right hand rule n  x r r nn  Using the fold-tree (previous lesson), we can recursively propagate changes in internal coordinates to the whole structure (see Wedemeyer and Baker 2003) nn adapted from image by Sunil Singh http://cnx.org/content/m14014/1.9/

20. Gradient Calculations – Cartesian vs. Internal Coordinates For some terms, Gradient computation is simpler and more natural with Cartesian coordinates, but harder for others: Distance / Cartesian dependent: Van der-Waals term ; Electrostatics ; Solvation Internal-coordinates dependent: Bond length and angle ; Ramachandran and Dunbrack terms (in Rosetta) Combination: Hydrogen-bonds (in some force-fields) Reminder: Internal coordinates provide a natural distinction between soft constraints (flexibility of φ/ψ torsion angles) and hard constraints with steep gradient (fixed length of covalent bonds).  Energy landscape of Cartesian coordinates is more rugged.

21. Analytical solutions require a closed-form algebraic formulation of energy score Numerical solution try to approximate the gradient (or Hessian) – Simple example: f’(x) ≈ f(x+1) – f(x) – Another example: the Secant method (soon) Analytical vs. Numerical Gradient Calculations

22. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

23. Gradient Descent Minimization Algorithm Sliding down an energy gradient good ( = global minimum) local minimum Image by Ken Dill

24. 1.Coordinates vector (Cartesian or Internal coordinates): X=(x 1, x 2,…,x n ) 2.Differentiable energy function: E(X) 3.Gradient vector: *Adapted from slides by Chen Kaeasar, Ben-Gurion University Gradient Descent – System Description

25. Gradient Descent Minimization Algorithm: Parameters: λ = step size ;  = convergence threshold x = random starting point While   (x)  >  – Compute  (x) – x new = x + λ  (x) Line search: find the best step size λ in order to minimize E(x new ) (discussion later) Note on convergence condition: in local minima, the gradient must be zero (but not always the other way around)

26. Line Search Methods – Solving argmin λ E[x + λ  (x)]: (1)This is also an optimization problem, but in one-dimension… (2)Inexact solutions are probably sufficient Interval bracketing – (e.g., golden section, parabolic interpolation, Brent’s search) – Bracketing the local minimum by intervals of decreasing length – Always finds a local minimum Backtracking (e.g., with Armijo / Wolfe conditions) : – Multiply step-size λ by c<1, until some condition is met – Variations: λ can also increase 1-D Newton and Secant methods We will talk about this soon…

27. The (very common) problem: a narrow, winding “valley” in the energy landscape  The narrow valley results in miniscule, zigzag steps 2-D Rosenbrock’s Function: a Banana Shaped Valley Pathologically Slow Convergence for Gradient Descent 100 iterations 1000 iterations 0 iterations 10 iterations

28. (One) Solution: Conjugate Gradient Descent Use a (smart) linear combination of gradients from previous iterations to prevent zigzag motion Parameters: λ = step size ;  = convergence threshold x 0 = random starting point Λ 0 =  (x 0 ) While  Λ i  >  – Λ i+1 =  (x i ) + β i ∙Λ i – choice of β i is important – X i+1 = x i + λ ∙ Λ i Line search: adjust step size λ to minimize E(X i+1 ) gradient descent Conjugated gradient descent The new gradient is “A-orthogonal” to all previous search direction, for exact line search Works best when the surface is approximately quadratic near the minimum (convergence in N iterations), otherwise need to reset the search every N steps (N = dimension of space) The new gradient is “A-orthogonal” to all previous search direction, for exact line search Works best when the surface is approximately quadratic near the minimum (convergence in N iterations), otherwise need to reset the search every N steps (N = dimension of space)

29. A-orthogonality We would like to “stretch” the narrow vallies, and then search only in orthogonal directions, to prevent a waste of time Image taken from: An Introduction to the Conjugate Gradient Method without the Agonizing Pain. Shewchuk 1994

30. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

31. Root Finding – when is f(x) = 0?

32. Taylor’s Series First order approximation: Second order approximation: The full Series: Example: = (a=0)

33. Taylor’s Approximation: f(x)=e x

34. Taylor’s Approximation of f(x) = sin(x) 2x at x=1.5

38. From Taylor’s Series to Root Finding (one-dimension) First order approximation: Root finding by Taylor’s approximation:

39. Newton-Raphson Method for Root Finding (one-dimension) 1.Start from a random x 0 2.While not converged, update x with Taylor’s series: 

40. Image from http://www.codecogs.com/d-ox/maths/rootfinding/newton.php Newton-Raphson: Quadratic Convergence Rate THEOREM: Let x root be a “nice” root of f(x). There exists a “neighborhood” of some size Δ around x root, in which Newton method will converge towards x root quadratically ( = the error decreases quadratically in each round)

41. The Secant Method (one-dimension) Just like Newton-Raphson, but approximate the derivative by drawing a secant line between two previous points: Secant algorithm: 1.Start from two random points: x 0, x 1 2.While not converged: Theoretical convergence rate: golden-ratio (~1.62) Often faster in practice: no gradient calculations

42. Newton’s Method: from Root Finding to Minimization Second order approximation of f(x): Minimum is reached when derivative of approximation is zero: take derivative (by X) So… this is just root finding over the derivative (which makes sense since in local minima, the gradient is zero)

43. Newton’s Method for Minimization: 1.Start from a random vector x=x 0 2.While not converged, update x with Taylor’s series:  Notes: if f’’(x)>0, then x is surely a local minimum point We can choose a different step size than one

44. Newton’s Method for Minimization: Higher Dimensions 1.Start from a random vector x=x 0 2.While not converged, update x with Taylor’s series:  Notes: H is the Hessian matrix (generalization of second derivative to high dimensions) We can choose a different step size using Line Search (see previous slides)

45. Generalizing the Secant Method to High Dimensions: Quasi-Newton Methods Calculating the Hessian (2 nd derivative) is expensive  numerical calculation of Hessian Popular methods: – DFP (Davidson – Fletcher – Powell) – BFGS (Broyden – Fletcher – Goldfarb – Shanno) – Combinations Timeline: Newton-Raphson (17 th century)  Secant method  DFP (1959, 1963)  Broyden Method for roots (1965)  BFGS (1970) Timeline: Newton-Raphson (17 th century)  Secant method  DFP (1959, 1963)  Broyden Method for roots (1965)  BFGS (1970)

46. Comparison of Derivative Methods Quasi-Newton methods take less iterations to converge than conjugate gradient descent, but: – The computation in each iteration is more complex – Takes more memory (a limited memory variant called L-BFGS) For Quasi-Newton methods, BFGS is generally considered superior to the older DFP. – They can be also combined together – Rosetta currently uses DFP, with Brent or Armijo Line-Search

47. Some more Resources on Gradient and Newton Methods Conjugate Gradient Descent http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdfhttp://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf Quasi-Newton Methods: http://www.srl.gatech.edu/education/ME6103/Quasi-Newton.ppt http://www.srl.gatech.edu/education/ME6103/Quasi-Newton.ppt HUJI course on non-linear optimization by Benjamin Yakir http://pluto.huji.ac.il/~msby/opt-files/optimization.html http://pluto.huji.ac.il/~msby/opt-files/optimization.html Line search: – http://pluto.huji.ac.il/~msby/opt-files/opt04.pdf http://pluto.huji.ac.il/~msby/opt-files/opt04.pdf – http://www.physiol.ox.ac.uk/Computing/Online_Documentation/Matlab/toolbox/nnet/backpr59.html http://www.physiol.ox.ac.uk/Computing/Online_Documentation/Matlab/toolbox/nnet/backpr59.html Wikipedia…

48. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

49. Arbitrary starting point Example: predict protein structure from its AA sequence. Harder Goal: Move from an Arbitrary Model to a Correct One *Adapted from slides by Chen Kaeasar, Ben-Gurion University

50. 10 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

51. 100 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

52. 200 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

53. 400 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

54. 800 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

55. 1000 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

56. 1200 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

57. 1400 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

58. 1600 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

59. 1800 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

60. 2000 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

61. 4000 iteration *Adapted from slides by Chen Kaeasar, Ben-Gurion University

62. 7000 iteration This time succeeded, in many cases not. *Adapted from slides by Chen Kaeasar, Ben-Gurion University

63. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

64. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

65. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

66. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

67. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

68. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

69. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

70. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

71. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

72. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

73. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

74. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

75. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

76. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

77. What kind of paths do we want? The path to the global minimum *Adapted from slides by Chen Kaeasar, Ben-Gurion University

78. Monte-Carlo Methods (a.k.a. MC simulations, MC sampling or MC search) Monte-Carlo methods (“casino” methods) are a very general term for estimations that are based on a series of random samples – Samples can be dependent or independent – MC physical simulations are most famous for their role in the Manhattan Project (Uncle of Polish mathematician Stanisław Marcin Ulam’s was said to be a heavy gambler)

79. Example: Estimating Π by Independent Monte-Carlo Samples (I) Suppose we throw darts randomly (and uniformly) at the square: Algorithm: For i=[1..ntrials] x = (random# in [0..r]) y = (random# in [0..r]) distance = sqrt (x^2 + y^2) if distance ≤ r hits++ End Output: Adapted from course slides by Craig Douglas http://www.chem.unl.edu/ze ng/joy/mclab/mcintro.html

80. Buffon's original form was to drop a needle of length L at random on grid of parallel lines of spacing D. For L less than or equal D we obtain : P(needle intersects the grid) = 2 L / PI D. If we drop the needle N times and count R intersections we obtain : P = R / N, PI = 2 L N / R D. Slide from course CS521 by Craig Douglas Georges Louis Leclerc Comte de Buffon (7/9/1707-16/4/1788) http://www.mste.uiuc.edu/reese/buffon/bufjava.html

81. Slide from course CS521 by Craig Douglas

86. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

87. Drunk Sailor’s Random Walk http://www.chem.uoa.gr/applets/AppletSailor/Appl_Sailor2.html What is the probability that the sailor will leave through each exit? 0.25

88. Markov-Chain Monte Carlo (MCMC) Markov-Chain: future state depends only on present state Markov-Chain Monte-Carlo on Graphs: we randomly walk from node to node with a certain probability, that depends only on our current location. 0.5 0.25 0.75

89. Analysis of a Two-Nodes Walk AB 0.5 0.25 0.75 After n rounds, what is the probability of being in node A? Assume Pr n+1 A ≈ Pr n A for a large n :  Pr n+1 A = Pr n A x 0.75 + Pr n B x 0.5  0.25 x Pr n A = Pr n B x 0.5  Pr n A = 2 x Pr n B So: Pr ∞ A = ⅔ Pr ∞ B = ⅓ Assume Pr n+1 A ≈ Pr n A for a large n :  Pr n+1 A = Pr n A x 0.75 + Pr n B x 0.5  0.25 x Pr n A = Pr n B x 0.5  Pr n A = 2 x Pr n B So: Pr ∞ A = ⅔ Pr ∞ B = ⅓

90. After a long run, we want to find low- energy conformations, with high probability Sampling Protein Conformations with MCMC Protein image taken from Chemical Biology, 2006 Markov-Chain Monte-Carlo (MCMC) with “proposals”: 1.Perturb Structure to create a “proposal” 2.Accept or reject new conformation with a “certain” probability Markov-Chain Monte-Carlo (MCMC) with “proposals”: 1.Perturb Structure to create a “proposal” 2.Accept or reject new conformation with a “certain” probability But how? A (physically) natural * choice is the Boltzmann distribution, proportional to: E i = energy of state i k B = Boltzmann constant T = temperature Z = “Partition Function” constant A (physically) natural * choice is the Boltzmann distribution, proportional to: E i = energy of state i k B = Boltzmann constant T = temperature Z = “Partition Function” constant * In theory, the Boltzmann distribution is a bit problematic in non-gas phase, but never mind that for now…

91. The Metropolis-Hastings Criterion Boltzmann Distribution: The energy score and temperature are computed (quite) easily The “only” problem is calculating Z (the “partition function”) – this requires summing over all states. Metropolis showed that MCMC will converge to the true Boltzmann distribution, if we accept a new proposal with probability "Equations of State Calculations by Fast Computing Machines“ – Metropolis, N. et al. Journal of Chemical Physics (1953)

92. If we run till infinity, with good perturbations, we will visit every conformation according to the Boltzmann distribution Sampling Protein Conformations with Metropolis-Hastings MCMC Protein image taken from Chemical Biology, 2006 Markov-Chain Monte-Carlo (MCMC) with “proposals”: 1.Perturb Structure to create a “proposal” 2.Accept or reject new conformation by the Metropolis criterion 3.Repeat for many iterations Markov-Chain Monte-Carlo (MCMC) with “proposals”: 1.Perturb Structure to create a “proposal” 2.Accept or reject new conformation by the Metropolis criterion 3.Repeat for many iterations But we just want to find the energy minimum. If we do our perturbations in a smart manner, we can still cover relevant (realistic, low- energy) parts of the search space

93. Outline Introduction Local Minimization Methods (derivative-based) – Gradient (first order) methods – Newton (second order) methods Monte-Carlo Sampling (MC) – Introduction to MC methods – Markov-chain MC methods (MCMC) – Escaping local-minima

94. *Adapted from slides by Chen Kaeasar, Ben-Gurion University Getting stuck in a local minimum

95. *Adapted from slides by Chen Kaeasar, Ben-Gurion University Getting stuck in a local minimum

96. *Adapted from slides by Chen Kaeasar, Ben-Gurion University Getting stuck in a local minimum

97. *Adapted from slides by Chen Kaeasar, Ben-Gurion University Getting stuck in a local minimum

98. *Adapted from slides by Chen Kaeasar, Ben-Gurion University Getting stuck in a local minimum

99. *Adapted from slides by Chen Kaeasar, Ben-Gurion University Getting stuck in a local minimum

100. Trick 1: Simulated Annealing The Boltzmann distribution depends on the in-silico temperature T: In low temperatures, we will get stuck in local minima (we will always get zero if the energy rises even slightly) In high temperatures, we will always get 1 (jump between conformations like nuts). The Boltzmann distribution depends on the in-silico temperature T: In low temperatures, we will get stuck in local minima (we will always get zero if the energy rises even slightly) In high temperatures, we will always get 1 (jump between conformations like nuts). In simulated annealing, we gradually decrease (“cool down”) the virtual temperature factor, until we converge to a minimum point

101. Trick 2: Monte-Carlo with Energy Minimization (MCM) Scheraga et al., 1987 Derivative-based methods (Gradient Descent, Newton’s method, DFP) are excellent at finding near-by local minima In Rosetta, Monte-Carlo is used for bigger jumps between near-by local minima

102. Trick 3: Switching between Low-Resolution (smooth) and High-Resolution (rugged) energy functions In Rosetta, the Centroid energy function is used to quickly sample large perturbations The Full-Atom energy function is used for fine tuning START energy conformations Smooth Low-res Rugged High-res

103. Trick 4: Repulsive Energy Ramping The repulsive VdW energy is the main reason for getting stuck Start simulations with lowered repulsive energy term, and gradually ramp it up during the simulation Similar rational to Simulated Annealing Trick 5: Modulating Perturbation Step Size A too small perturbation size can lead to a very slow simulation  we remain stuck in the local minimum A large perturbation size can lead to clashes and a very high rejection rate  we remain stuck in the same local minimum We can increase or decrease the step size until a fixed rejection rate (for example, 50%) is achieved

104. Monte-Carlo in Rosetta In Rosetta, it is common to use any of the above tricks, MCM in particular In general, a single simulation is pretty short (no more than a few minutes), but is repeated k independent times – getting k sampled “decoys” – We use energy scoring to decide which is the best decoy structure – hopefully this is the near-native solution – Low-resolution sampling is often used to create a very large number of initial decoys, and only the best ones are moved to high-resolution minimization

105. Summary Derivative-based methods can effectively reach near-by energy minima Metropolis-Hastings MCMC can recover the Boltzmann distribution in some applications, but for protein folding, we cannot hope to cover the huge conformational space, or recover the Boltzmann distribution. Still, useful tricks help us find good low-energy near-native conformations (Simulated Annealing, Monte-Carlo with Minimization, Centroid mode, Ramping, Step size modulation, and other smart sampling steps, etc.). We didn’t cover some very popular non-linear optimization methods: – Linear and Convex Programming ; Expectation Maximization algorithm ; Branch and Bound algorithms ; Dead-End Elimination (Lesson 4) ; Mean Field approach ; And more…