Optimization in mean field random models Johan Wästlund Linköping University Sweden

Statistical Mechanics Each particle has a spin Energy = Hamiltonian depends on spins of interacting particles Ising model: Spins ±1, H = # interacting pairs of opposite spin

Statistical Mechanics Spin configuration  has energy H(  ) Gibbs measure depends on temperature T: T→∞ random state T→0 ground state, i.e. minimizing H(  )

Statistical Mechanics Thermodynamic limit N →∞ Average energy? (suitably normalized)

Disordered Systems Spin glasses AuFe random alloy Fe atoms interact

Disordered Systems Random interactions between Fe atoms Sherrington-Kirkpatrick model:

Disordered Systems Quenched random variables g i,j S-K is a mean field model: No correlation betweeen quenched variables NP hard to find ground state given g i,j

Computer Science Test / evaluate heuristics for NP-hard problems Average case analysis Random problem instances

Combinatorial Optimization Minimum Matching / Assignment Minimum Spanning Tree Traveling Salesman Shortest Path … Points with given distances, minimize total length of configuration

Statistical Physics / Computer Science Spin configuration Hamiltonian Ground state energy Temperature Gibbs measure Thermodynamic limit Feasible solution Cost of solution Cost of minimal solution Artificial parameter T Gibbs measure N→∞

Mean field models Replica-cavity method has given good results for mean field models Parisi solution of S-K model The same methods can be applied to combinatorial optimization problems in mean field models

Mean field models of distance N points Abstract geometry Inter-point distances given by i. i. d. random variables Exponential distribution easiest to analyze (pseudodimension 1)

Matching Set of edges giving a pairing of all points

Spanning tree Network connecting all points

Traveling salesman Tour visiting all points

Mean field limits No normalization needed! (pseudodimension 1) Matching:  2 /12≈0.822 (Mézard & Parisi 1985, rigorous proof by Aldous 2000) Spanning tree:  (3) = 1+1/8+1/27+… ≈1.202 (Frieze 1985) Traveling salesman: … (Krauth- Mézard-Parisi 1989), now established rigorously!

Cavity results Non-rigorous method Aldous derived equivalent equations with the Poisson-Weighted Infinite Tree (PWIT)

Cavity results Non-rigorous quantity X = cost of minimal solution – cost of minimal solution with the root removed Define X 1, X 2, X 3,… similarly on sub-trees Leads to the equation X i distributed like X,  i are times of events in rate 1 Poisson process

Cavity results Analytically, this is equivalent to where

Cavity results Explicit solution Ground state energy

Cavity results Note that the integral is equal to the area under the curve when f(u) is plotted against f(-u) In this case, f satisfies the equation

Cavity results

K-L matching

Similarly, the K-L matching problem leads to the equations:  has rate K and  has rate L min[K] stands for K:th smallest

Shown by Parisi (2006) that this system has an essentially unique solution The ground state energy is given by where x and y satisfy an explicit equation For K=L=2, this equation is Unfortunately the cavity method is not rigorous K-L matching

The exponential bipartite assignment problem n

Exact formula conjectured by Parisi (1998) Suggests proof by induction Researchers in discrete math, combinatorics and graph theory became interested Generalizations…

Generalizations by Coppersmith & Sorkin to incomplete matchings Remarkable paper by M. Buck, C. Chan & D. Robbins (2000) Introduces weighted vertices Extremely close to proving Parisi’s conjecture!

Incomplete matchings n m

Weighted assignment problems Weights  1,…,  m,  1,…,  n on vertices Edge cost exponential of rate  i  j Conjectured formula for the expected cost of minimum assignment Formula for the probability that a vertex participates in solution (trivial for less general setting!)

The Buck-Chan-Robbins urn process Balls are drawn with probabilities proportional to weight 11 22 33

Proofs of the conjectures Two independent proofs of the Parisi and Coppersmith-Sorkin conjectures were announced on March 17, 2003 (Nair, Prabhakar, Sharma and Linusson, Wästlund)

Annealing Powerful idea: Let T→0, forcing the system to converge to its ground state Replica-cavity approach Simulated annealing meta-algorithm (optimization by random local moves)

In the mean field model: Underlying rate 1 variables Y i r i plays the same role as T Local temperature Associate weight to vertices rather than edges

Cavity/annealing method Relax by introducing an extra vertex Let the weight of the extra vertex go to zero Example: Assignment problem with  1 =…=  m =1,  1 =…=  n =1, and  m+1 =  p = P(extra vertex participates) p/n = P(edge (m+1,n) participates)

Annealing p/n = P (edge (m+1,n) participates) When  →0, this is Hence By Buck-Chan-Robbins urn theorem,

Annealing Hence Inductively this establishes the Coppersmith- Sorkin formula

Results with annealing Much simpler proofs of Parisi, Coppersmith-Sorkin, Buck-Chan-Robbins formulas Exact results for higher moments Exact results and limits for optimization problems on the complete graph

The 2-dimensional urn process 2-dimensional time until k balls have been drawn

Limit shape as n→∞ Matching: TSP/2-factor:

Mean field TSP If the edge costs are i.i.d and satisfy P(l<t)/t→1 as t→0 (pseudodimension 1), then as n →∞, A. Frieze proved that whp a 2-factor can be patched to a tour at small cost

Further exact formulas

LP-relaxation of matching in the complete graph K n

Future work Explain why the cavity method gives the same equation as the limit shape in the urn process Establish more detailed cavity predictions Use proof method of Nair-Prabhakar- Sharma in more general settings

Thank you!