1. Greg Knowles ECE 556 - Fall 2004 Professor Yu Hu Hen
Adaptive Simulated Annealing Automated Annealing Parameter Optimization for NP-Hard Combinatorial Problems
Greg Knowles
ECE Fall 2004
Professor Yu Hu Hen

2. Simulated Annealing Overview
A Heuristic for hard combinatorial optimization problems or continuous real time optimization
Idea came from condensed matter physics – annealing of metals
Perturbed Solution Generation
Accept all moves that reduce cost
Accept those that increase cost with low probability (Metropolis Criterion)

3. Key Annealing Parameters
T0 – initial temperature
 – Temperature Decrement Factor
Iterations at each temperature
Stopping conditions

4. ASA Research Leader - Lester Ingber
No Proven “Best” parameters
Considered to be somewhat of an art form
Often decided by trial-and-error
Lester Ingber

5. New Techniques Quenching - Dropping the Temperature suddenly
Non-Exponential acceptance calculation P(δ) = 1 - δ/t
The acceptance calculation takes about one third of the computation time in standard problems.
Mimics exponential equation
Use a look-up table of a set of values over the range d/t
Exponential Calculations done once
Speeds up algorithm by one third
Rounding to precalculated values has no significant effect on solution quality
Optimal Temperature Search
spend some time searching for the optimum temperature and than stay at that temperature for the remainder of the algorithm

6. Initial Temperature Considerations
No known method for finding a suitable starting temperature for a whole range of problems
If the temperature starts too high, early stages of annealing wasted in a random searching
Must be hot enough to allow a move to almost any neighborhood state
Start with a very high temperature and cool it rapidly until about 60% of solutions are accepted
Rapidly heat the system until a certain proportion of solutions are accepted and then slow cooling

7. Ending Temperature Considerations
Typically run until Temperature reaches 0
Nothing gained by wasting time near zero
Limit of accepting bad moves as temperature approaches zero,
Lim{ T->0, P(Temp) ~= P(Temp=0)} = 0
Stop when the system is "frozen"
(i.e. no better or worse moves are being accepted).
“Quelch” the system

8. Iterations per Temperature Considerations
Constant number of iterations at each temperature
Allow enough iterations so that the system stabilizes at each temperature
Exponential to the problem size
Dynamically change the number of iterations as the algorithm progresses
At lower temperatures use a large number of iterations to fully explore the local optimum
At higher temperatures, the number of iterations can be less.

9. Cooling Rate Considerations
Large number of iterations at a few temperatures, a small number of iterations at many temperatures or a balance between the two
Geometric decrement where t = tα S.T. α < 1.
Experience has shown that α should be between 0.8 and 0.99
Only do one iteration at each temperature, but decrease the temperature very slowly.
t = t/(1 + βt)
Experience has shown that β should be between 0 and 0.01