1. Simulated Annealing Chapter 4. 7 - 4
Simulated Annealing Chapter from Adaptive Cooperative Systems
Jang, HaYoung

2. The Simulated Annealing Algorithm
Probability Distribution
Partition Function
Transition probability

3. The Cooling Schedule and Convergence of the Algorithm
Convergence properties of the simulated annealing algorithm
It will converge to a set of configurations of minimal energy provided that the temperature is lowered no faster than logarithmically, where Tk is temperature of the kth temperature step and Γ is a positive constant.

4. The Cooling Schedule and Convergence of the Algorithm
NP problems must have a slower than power-law relaxation with any physical dynamics
Residual energy has a logarithmic dependence on the cooling schedule.
Residual energy is defined as , where <E> τ is the expected energy per spin upon teaching T=0 in time τ.

5. The Cooling Schedule and Convergence of the Algorithm
Annealing temperature
It must be sufficiently elevated to allow passage in a reasonable number of trials over the barriers associated with the local minima.
This provides a condition on the selection of the constant Γ.
Phase transition
We must anneal slowly in the vicinity of phase transitions.

6. The Cooling Schedule and Convergence of the Algorithm
Stop Time
The entropy and specific heat are expected to become vanishingly small ass the optimal configuration is approahed.
We may determine the entropy at some temperature of interest by integrating the expression
after fixing the second temperature at a high level where the entropy is know.

7. Traveling Salesman Problems
Total length of the tour
where P denotes a permutation and P(N+1)=P(1).
Partition Function
Simulated annealing algorithm for the TSP
Establish an annealing schedule.
Initialize by randomly selecting a tour.
Use 2-opt supplemented by the pair subregion criterion to randomly choose city pairs.
Calculate the energy difference produced by reserving paths between the cities.
If ΔE is zero or negative, accept the new tour.
If ΔE is positive, accept the change with probability given by the metropolis expression.
Select another city pair, and repeat steps 3 to 6 until the requisite number of iterations are completed.
Lower the temperature, and repeat steps 3 to 7.

8. Graph Partitioning
We want to distribute N circuits between a pair of computer chips.
Minimize the number of signals Ncp.
Maintain a balance between the number of circuits on the two chips.
Energy function embodies our two constraints

9. Microcanonical Annealing
Alternative method based on a microcanonical ensemble
‘demon’ travels about the system executing a random walk on the surface of a constant energy sphere in configuration space.
The number of states with the correct total energy

10. Microcanonical Annealing
At equilibrium state, the demon’s energy will become exponentially distributed,
Number of states accessible to the heat bath
Probability of finding the demon in a state with energy ED
Microcanonical annealing algorithm
Initialize the system in a state i, and initialize the demon’s energy at zero or some positive number.
Select a site at a random, in the manner of a raster scan or some other way.
Make a local change in the state at that site, and calculate the corresponding energy difference ΔE.
If ΔE is negative, accept the change and add the energy ΔE to the demon’s energy ED.
If ΔE is positive, accept the change provided that ΔE ≤ ED, and decrement the demon’s energy by ΔE.
Select another site, and repeat steps 3 to 5 until an equilibration criterion is met.
Remove a small amount of energy from the demon, and repeat steps 2 to 6.

11. Continuous Simulated Annealing
It is based on finding a solution to the stochastic differential equation.
y: any real value in a specified range, characterizes the states of the system.
T: time dependent temperature that specifies the cooling schedule and controls the magnitude of a stochastic noise process W
t: time
U: potential to be minimized
Gibbs distribution
Partition Function
The solution to the stochastic differential equation converges to an equilibrium Gibbs distribution as the temperature is lowered.