1. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University1 Dynamic cavity method and marginals of non-equilibrium stationary states KITPC/ITP-CAS Program Interdisciplinary Applications of Statistical Physics and Complex Networks E.A., H. Mahmoudi (2010) arXiv:1012.3388 I. Neri, D. Bollé, J. Stat Mech. (2009) P08009 Y. Kanoria, A. Montanari (2009) arXiv:0907.0449

2. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University2 Motivation & outline The cavity method is a way to approximately compute marginals of equilibrium probability distributions – compare Belief Propagation (BP), iterative decoding. Do these methods generalize from equilibrium statistical mechanics to non-equilibrium? I will present results that this is (sometimes) possible. Hamed Mahmoudi will tell more in the workshop next week.

3. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University3 BP and the Bethe-Peierls approximation Marginal (one-spin) probabilities Cost or energy function; sum of local terms Configuration space; N discrete variables JS Yedidia, WT Freeman, Y Weiss (2001) M Mézard, A Montanari, Oxford University Press (2009) Belief propagation Bethe-Peierls approximation In the applications to statistical physics, artificial intelligence and information theory, such marginals (magnetizations, correlation functions etc) are (basically) what can be found by these methods

4. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University4..messages, factor graphs... BP output equation BP update equation (one side) BP update equation (other side) The “messages”andare Lagrange multipliers. Exact if the factor graph is a tree. Otherwise no guarantees.

5. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University5 What then about non- equilbrium? Refined outline of the talk: In non-equilibrium there is no Gibbs distribution. But is there nevertheless a message-passing scheme which (approximate) computes marginals of probability distributions? Compare dynamic mean-field theory. What could this be good for? The example of kinetic Ising models How dynamic cavity method works & numerics

6. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University6 What could be the use? Message-passing for non-equilibrium states should be much faster than Monte Carlo, but more accurate than mean field. However, on locally tree-like graphs. Make bombs? Sorry, but lattice graphs have lots of short loops – message-passing is not usually good. Games on graphs, perhaps states of social networks? Maybe, but that would not be entirely new. Y. Kanoria, A. Montanari (2009) arXiv:0907.0449 Majority dynamics on trees and the dynamic cavity method …determining the stationary state of a Markov chain or a Markov process is a rather general problem…

7. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University7 …the application we had in mind initially, but on which we so far have done little… (Potential) applications… Another approach to kinetic inference by better computation of the marginals (magnetizations and correlations) Roudi et al (2009) Hertz et al (2010) Hertz, Roudi (PRL 2011) Zeng et al arXiv:1011.6216 Aree Witoelar in this program

8. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University8 Queuing theory looks for stationary states called “equilibrium states”. Stationary states of networks of queues Everitt (1994) Pallant, Taylor (1995) Abdalla, Boucherie (2002) Vazquez-abad et al (2002) Most of queuing theory is about networks of reversible or quasi- reversible queues, for which the stationary states are (relatively) simple. But there are also other examples e.g. modeling blocking probabilities in mobile cellular communication systems, with handovers between base stations. source: Wikipedia

9. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University9 S Kauffman (1969) J Hopfield (1982) B Derrida (1987) A Crisanti, H Sompolinski (1988) ….and many others Fully asynchronous updates (master equations); not yet done Synchronous updates; which can yet be varied in several ways (here in two ways) - parallel : simultaneously update all spins - sequential : one spin updated at a time x1x1 x2x2 x3x3 X4 xNxN xixi The diluted asymmetric spin glass

10. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University10 Gaussian or binary Starts as directed Erdős-Renyi graphs; where average connectivity would be c Connections i → j and j → i dependent Coolen parametrization Dilution, asymmetry, and interaction strength

11. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University11 MFT for magnetizations and correlations Crisanti, Sompolinski (1988) Kappen, Spanjers (2000) Hertz et al (2010) Hertz, Roudi arXiv:1009.5946 Zeng et al arXiv:1011.6216 Hertz & Roudi (2011)

12. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University12 The cavity method and BP output for spin histories Cavity graph The BP assumption: spins in cavity independent

13. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University13 BP updates for histories

14. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University14 The time-factorized approximation I Neri, D Bolle (2009) Aurell, Mahmoudi (2010) exact for fully asymmetric networks and parallel updates …and for asymmetric networks the summation over spin i at time t-2 can be done.

15. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University15 The random sequential update model When N goes to infinity, it is reasonable to expect that sequential update tends to fully asynchronous updates. But at finite N there will be a difference. In random sequential updates (poor man’s Master equation) one (randomly picked) spin is updated at a time. Consider first fully asynchronous updates (Master equations):

16. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University16 Dynamic cavity equations for sequential updates Aurell, Mahmoudi (2010) The full dynamic cavity equations: The time factorized approximation (here not exact, even for fully asymmetric networks):

17. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University17 Comparing dynamic cavity equations to Monte Carlo …for more, Hamed Mahmoudi’s talk in next week’s workshop…

18. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University18 Aurell, Mahmoudi (2010) Fully asymmetric networks

19. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University19 Aurell, Mahmoudi (2010) Half-asymmetric networks Sequential Connectivity c=3 Size N=1000 MC averaged over 10 000 samples Local fields ≈ 0.01 Couplings ≈ 1⁄√

20. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University20 Aurell, Mahmoudi (2010) Symmetric networks

21. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University21 Comparison to mean-field

22. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University22 Comparison to mean-field Parallel updates “asymmetric ferromagnet” Other parameters as previous slide Dynamic BP and MC overlap also for finite time

23. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University23 Dynamic cavity method is computationally feasible within the time-factorized approximation. It is much faster than Monte Carlo. It gives quite accurate results on the kinetic Ising models, both in the spin glass and in the ferromagnetic phases, and it is substantially more accurate than mean field. It works (for now) for stationary states, not for transients. Generalization to fully asynchronous dynamics lacking. Good applications are still to be developed. Conclusions

24. KTH/CSC March 10, 2010Erik Aurell, KTH & Aalto University24 Thanks to Hamed Mahmoudi John Herz Yasser Roudi Hong-Li Zeng Mikko Alava Izaac Neri Lenka Zdeborová Silvio Franz