Greg Knowles ECE 556 - Fall 2004 Professor Yu Hu Hen Adaptive Simulated Annealing Automated Annealing Parameter Optimization for NP-Hard Combinatorial Problems Greg Knowles ECE 556 - Fall 2004 Professor Yu Hu Hen

Simulated Annealing Overview A Heuristic for hard combinatorial optimization problems or continuous real time optimization Idea came from condensed matter physics – annealing of metals Perturbed Solution Generation Accept all moves that reduce cost Accept those that increase cost with low probability (Metropolis Criterion)

Key Annealing Parameters T0 – initial temperature  – Temperature Decrement Factor Iterations at each temperature Stopping conditions

ASA Research Leader - Lester Ingber No Proven “Best” parameters Considered to be somewhat of an art form Often decided by trial-and-error Lester Ingber

New Techniques Quenching - Dropping the Temperature suddenly Non-Exponential acceptance calculation. P(δ) = 1 - δ/t The acceptance calculation takes about one third of the computation time in standard problems. Mimics exponential equation Use a look-up table of a set of values over the range d/t Exponential Calculations done once Speeds up algorithm by one third Rounding to precalculated values has no significant effect on solution quality Optimal Temperature Search spend some time searching for the optimum temperature and than stay at that temperature for the remainder of the algorithm

Initial Temperature Considerations No known method for finding a suitable starting temperature for a whole range of problems If the temperature starts too high, early stages of annealing wasted in a random searching Must be hot enough to allow a move to almost any neighborhood state Start with a very high temperature and cool it rapidly until about 60% of solutions are accepted Rapidly heat the system until a certain proportion of solutions are accepted and then slow cooling

Ending Temperature Considerations Typically run until Temperature reaches 0 Nothing gained by wasting time near zero Limit of accepting bad moves as temperature approaches zero, Lim{ T->0, P(Temp) ~= P(Temp=0)} = 0 Stop when the system is "frozen" (i.e. no better or worse moves are being accepted). “Quelch” the system

Iterations per Temperature Considerations Constant number of iterations at each temperature Allow enough iterations so that the system stabilizes at each temperature Exponential to the problem size Dynamically change the number of iterations as the algorithm progresses At lower temperatures use a large number of iterations to fully explore the local optimum At higher temperatures, the number of iterations can be less.

Cooling Rate Considerations Large number of iterations at a few temperatures, a small number of iterations at many temperatures or a balance between the two Geometric decrement where t = tα S.T. α < 1. Experience has shown that α should be between 0.8 and 0.99 Only do one iteration at each temperature, but decrease the temperature very slowly. t = t/(1 + βt) Experience has shown that β should be between 0 and 0.01