Simulated Annealing & Boltzmann Machines 虞台文 大同大學資工所 智慧型多媒體研究室

Content Overview Simulated Annealing Deterministic Annealing Boltzmann Machines

Simulated Annealing & Boltzmann Machines Overview 大同大學資工所 智慧型多媒體研究室

Hill Climbing E > 0 E E < 0 E: cost (energy)

The Problem with Hill Climbing Gets stuck at local minima Gradient decent approach Hopfield neural networks Possible solutions Try different initial states Increase the size of the neighborhood (e.g. in TSP try 3-opt rather than 2-opt)

Stochastic Approaches Goal: escape from local-minima. Stochastic Approaches Stochastic optimization refers to the minimization (or maximization) of a function in the presence of randomness in the optimization process. The randomness may be present as either noise in measurements or Monte Carlo randomness in the search procedure, or both.

Two Important Methods Simulated Annealing (SA) Genetic Algorithms (GA) Motivated by the physical annealing process Evolution from a single solution Genetic Algorithms (GA) Motivated by the evolution process of biology Evolution from multiple solutions

Two Important Methods Simulated Annealing (SA) Genetic Algorithms (GA) Motivated by the physical annealing process Evolution from a single solution Genetic Algorithms (GA) Motivated by the evolution process of biology Evolution from multiple solutions Kirkpatrick, S , Gelatt, C.D., Vecchi, M.P. 1983. “Optimization by Simulated Annealing.” Science, vol 220, No. 4598, pp 671-680. J. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press, 1975.

Simulated Annealing & Boltzmann Machines 大同大學資工所 智慧型多媒體研究室

Global Optimization

Statistical Mechanics in a Nutshell +  Statistical mechanics is the study of the behavior of very large systems of interacting components in thermal equilibrium at a temperature, say T.

T Boltzmann Factor kB : Boltzmann constant +  kB : Boltzmann constant Z(T) : Boltzmann partition function

T Boltzmann Factor T1 < T2 < T3 E Raising temperature the system becomes more `active’ the average energy becomes higher Boltzmann Factor T +  T1 < T2 < T3 E

Simulation Metropolis Acceptance Criterion E: cost (energy)

Simulation Metropolis Acceptance Criterion 1 T1 < T2 < T3

Simulated Annealing Algorithm Create initial solution S Initialize temperature T repeat for k = 1 to iteration-length do Generate a random transition from S to S’ Let E = E(S’)  E(S) if E < 0 then S = S’ else if exp[E/T] > rand(0,1) then S = S’ Reduce temperature T until no change in E(S) Return S

Simulated Annealing Algorithm Hill Climbing Simulated Annealing Algorithm Create initial solution S Initialize temperature T repeat for k = 1 to iteration-length do Generate a random transition from S to S’ Let E = E(S’)  E(S) if E < 0 then S = S’ else if exp[E/T] > rand(0,1) then S = S’ Reduce temperature T until no change in E(S) Return S

Main Components of SA Solution representation Appropriate for computing energy (cost) Transition mechanism between solutions Incremental changes of solutions Cooling schedule Initial system temperature Temperature decrement function Number of iterations between temperature change Acceptance criteria Stop criteria

Traveling Salesman Problem Example Given n-city locations specified in a two-dimensional space, find the minimum tour length. The salesman must visit each and every city only once and should return to the starting city forming a closed path. Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Traveling Salesman Problem Example Traveling Salesman Problem

Solution Representation (TSP) Assume cities are fully connected with symmetric distance. 1 2 3 5 8 9 11 10 6 4 7 ∞

Solution Representation (TSP) 1 2 3 4 6 5 7 9 11 8 10 1 2 3 5 8 9 11 10 6 4 7 >

Energy (Cost) Computation (TSP) d10,1 1 2 3 4 6 5 7 9 11 8 10 d23 d12 d12 d23 d34 d46 d65 d57 d79 d9.11 d11,8 1 2 3 5 8 9 11 10 6 4 7 d8,10 > d34 d46 d10,1 d9,11 d65 d11,8 d57 d79 d8,10

State Transition (TSP) d10,1 1 2 3 4 5 6 7 8 9 10 d12 d23 d34 d45 d56 d67 d78 d89 d9,10 1 2 3 4 5 6 7 8 9 10 1. Randomly select two edges

State Transition (TSP) d10,1 1 2 3 4 5 6 7 8 9 10 d34 d89 d12 d23 d45 d56 d67 d78 d9,10 1 2 3 4 5 6 7 8 9 10 1. Randomly select two edges 2. Swap the path

State Transition (TSP) d10,1 1 2 3 4 8 5 7 6 6 7 5 8 4 9 10 d38 d87 d76 d65 d54 d49 d12 d23 d9,10 1 10 2 1. Randomly select two edges 3 9 2. Swap the path 8 4 7 5 6

Cooling Schedules Geometric Schedule Empirical evidence shows that typically 0.8    0.99 yields successful applications (fairly slow cooling schedules).

Simulation 100 cities are randomly chosen from 1010 square. 100-city TSP Simulation

Simulated Annealing (T0 is starting temperature) 100 cities are randomly chosen from 1010 square. 100-city TSP 1000N iterations are made for each test. Simulation Total Length Direct Search Simulated Annealing (T0 is starting temperature) T0=1 T0=2 T0=5 T0=10 T0=25 T0=50 T0=100 Test1 89.211 82.757 82.732 79.113 81.792 82.701 79.405 79.528 Test2 89.755 81.325 81.334 80.532 83.166 80.461 79.25 82.549 Test3 81.44 82.063 84.296 80.629 81.658 80.35 79.21 79.933 Test4 85.038 82.449 82.388 80.996 79.764 82.688 82.131 84.17 Test5 87.256 80.658 80.814 82.261 80.82 81.338 80.406 79.869 Test6 88.989 82.92 81.284 82.607 80.599 83.526 81.43 80.894 Test7 85.895 84.479 80.807 80.402 80.837 79.504 81.052 82.201 Test8 85.654 80.549 81.251 80.195 82.878 80.169 80.604 80.125 Test9 88.246 79.832 79.123 80.617 81.676 80.741 81.306 Test10 84.586 82.446 82.855 81.249 82.885 79.68 81.665 Average 86.607 82.011 81.759 80.711 81.501 81.21 80.476 81.224 Each temperature T is hold for 100N reconfigurations or 10N successful reconfigurations, whichever comes first. T is reduced by 10% each time.

Simulated Annealing (T0 is starting temperature) 100 cities are randomly chosen from 1010 square. 100-city TSP Simulation Total Length Direct Search Simulated Annealing (T0 is starting temperature) T0=1 T0=2 T0=5 T0=10 T0=25 T0=50 T0=100 Test1 89.211 82.757 82.732 79.113 81.792 82.701 79.405 79.528 Test2 89.755 81.325 81.334 80.532 83.166 80.461 79.25 82.549 Test3 81.44 82.063 84.296 80.629 81.658 80.35 79.21 79.933 Test4 85.038 82.449 82.388 80.996 79.764 82.688 82.131 84.17 Test5 87.256 80.658 80.814 82.261 80.82 81.338 80.406 79.869 Test6 88.989 82.92 81.284 82.607 80.599 83.526 81.43 80.894 Test7 85.895 84.479 80.807 80.402 80.837 79.504 81.052 82.201 Test8 85.654 80.549 81.251 80.195 82.878 80.169 80.604 80.125 Test9 88.246 79.832 79.123 80.617 81.676 80.741 81.306 Test10 84.586 82.446 82.855 81.249 82.885 79.68 81.665 Average 86.607 82.011 81.759 80.711 81.501 81.21 80.476 81.224

Simulated Annealing & Boltzmann Machines Deterministic Annealing 大同大學資工所 智慧型多媒體研究室

The Problems of SA SA techniques are inherently slow because of their randomized local search strategy. Converge to global optimum in probability one sense only if the cooling schedule is in the order of

The Problems of SA Geman, S. & Geman, D. (1984) “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,” IEEE Trans. on Pattern Analysis and Machine Intelligence 6, 721-741. SA techniques are inherently slow because of their randomized local search strategy. Converge to global optimum in probability one sense only if the cooling schedule is in the order of Geman and Geman [1984]

Review Simulated Annealing Algorithm Create initial solution S{0, 1}n Initialize temperature T repeat for k = 1 to iteration-length do Generate a random transition from S to S’ by inverting a random bit si Let E = E(S’)  E(S) if E < 0 then S = S’ else if exp[E/T] > rand(0,1) then S = S’ Reduce temperature T until no change in E(S) Return S

Review Simulated Annealing Algorithm Create initial solution S{0, 1}n Initialize temperature T repeat for k = 1 to iteration-length do Generate a random transition from S to S’ by inverting a random bit si Let E = E(S’)  E(S) if E < 0 then S = S’ else if exp[E/T] > rand(0,1) then S = S’ Reduce temperature T until no change in E(S) Return S Stochastic nature

Deterministic Annealing (DA) Also called mean-field annealing. Deterministic Annealing (DA) Create initial solution S[0, 1]n Initialize temperature T repeat for k = 1 to iteration-length do Choose a random bit si Reduce temperature T until convergence criterion met Return S T1 < T2 < T3 1 Deterministic behavior

Simulated Annealing & Boltzmann Machines 大同大學資工所 智慧型多媒體研究室

+ Boltzmann Machines Discrete Hopfield NN Boltzmann Machine Simulated Annealing

Update Rules Discrete Hopfield NN Boltzmann Machine Unipolar neuron

Update Rules Discrete Hopfield NN Boltzmann Machine Unipolar neuron Cooling schedule is required. Update Rules T=0 T=1 T=2 T=3 T= Discrete Hopfield NN Boltzmann Machine Unipolar neuron

Exercises Computer Simulations on the same TSP problem demonstrated previously using Simulated Annealing Deterministic Annealing, and Boltzmann Machine. Perform some analyses on your results.