1. Simulated Annealing
A physical analogy

2. Boltzmann Distribution
Here α=1/(kT)
k is the Boltzmann constant
T is temperature in degrees Kelvin
High energy states are nearly equiprobable
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

3. DRAFT: Copyright GA Tagliarini, PhD
General Optimization
Suppose f:Rn→R
Minimize f(x), subject to xεRn
Maximization problems can be solved by minimizing –f(x)
Minima may be local or global
Let S be a subset of Rn. If there exists a point pεS such that f(p)≤f(x) for all xεS, then p is a local minimum of f over S. If S=Rn, then p is a global minimum of f.
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

4. DRAFT: Copyright GA Tagliarini, PhD
A Sample Problem
Given a set of points P1=(x1, y1),…, Pi=(xi, yi),…,Pn=(xn, yn)
Find a set of parameters a, b, c, and d that create the sine function g(x)=a sin(2πb(x-c))+d that “best fits” the data
Where best fit is the minimum of the least squares sum f(x)
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

5. Sample Problem (Continued)
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

6. Constrained Optimization
Suppose f:Rn→R
Minimize f(x), subject to xεRn
Sometimes there may be additional constraints, such as:
xεZn (integer programming)
xε{0, 1}n (zero/one programming)
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

7. Artificial (Simulated) Annealing
Analogous to Boltzmann distribution but Gaussian
Mimics the temperature and physical constant of the atomic model
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

8. DRAFT: Copyright GA Tagliarini, PhD
The Model
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

9. DRAFT: Copyright GA Tagliarini, PhD
The Model
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

10. DRAFT: Copyright GA Tagliarini, PhD
The Model
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

11. DRAFT: Copyright GA Tagliarini, PhD
The Model
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

12. DRAFT: Copyright GA Tagliarini, PhD
The Model
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

13. Case 2: Specifically When To Keep The New Estimate
Generate a value z from a uniform random distribution over the interval [0, 1/(kT)]
If p(Δf)>z,
Then x1 = x0 + Δx and iterate from x1
Else generate new values of Δx until an acceptable Δx is found
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD

14. DRAFT: Copyright GA Tagliarini, PhD
Annealing Schedule
Set the “temperature” proportional to
The reciprocal of log time (Geman and Geman), T(t)=T0/(1+ln(1+t))
Some statistical distribution (Szu and Hartley)
The reciprocal of time, T(t)=T0/(1+t)
Remember to offset time from zero
11/30/2018
DRAFT: Copyright GA Tagliarini, PhD