A Simple Example The Traveling Salesman Problem: Find a tour of a given set of cities so that each city is visited only once the total distance traveled is minimized

Representation Representation is an ordered list of city numbers known as an order-based GA. 1) London 3) Dunedin 5) Beijing 7) Tokyo 2) Venice 4) Singapore 6) Phoenix 8) Victoria CityList1 ( ) CityList2 ( )

Crossover Crossover combines inversion and recombination: * * Parent1 ( ) Parent2 ( ) Child ( ) This operator is called the Order1 crossover.

Mutation involves reordering of the list: * Before: ( ) After: ( ) Mutation

TSP Example: 30 Cities

Solution i (Distance = 941)

Solution j (Distance = 800)

Solution k (Distance = 652)

Best Solution (Distance = 420)

Overview of Performance

Considering the GA Technology “Almost eight years ago... people at Microsoft wrote a program [that] uses some genetic things for finding short code sequences. Windows 2.0 and 3.2, NT, and almost all Microsoft applications products have shipped with pieces of code created by that system.” - Nathan Myhrvold, Microsoft Advanced Technology Group, Wired, September 1995

Benefits of Genetic Algorithms Concept is easy to understand Modular, separate from application Supports multi-objective optimization Good for “noisy” environments Always an answer; answer gets better with time Inherently parallel; easily distributed

Benefits of Genetic Algorithms (cont.) Many ways to speed up and improve a GA- based application as knowledge about problem domain is gained Easy to exploit previous or alternate solutions Flexible building blocks for hybrid applications Substantial history and range of use

When to Use a GA Alternate solutions are too slow or overly complicated Need an exploratory tool to examine new approaches Problem is similar to one that has already been successfully solved by using a GA Want to hybridize with an existing solution Benefits of the GA technology meet key problem requirements

Some GA Application Types

Applications Decision Making Data Mining Scheduling Computer games Stock Market Trading Medical Information Systems Applications Finance Applications

Simulated Annealing This is an idea from the manufacture of steel.... Atoms move around randomly, sticking together, breaking apart The bonds that are strong have a better chance of lasting... So, heat the steel, then cool it slowly

Simulated Annealing Start temperature, end temperature, and cooling schedule are all problem-dependant, cost function dependant,.... About as scientific as picking lotto numbers At the start of the run, you want to accept most moves; at the end, you'll want to reject most moves that don't improve. What numbers should you use? Try playing around with them, and find out for yourself!

Simulated Annealing for TSP Pick two random (or adjacent) cities; how much do we save (or pay) if we swap their order? If we improve, do it; otherwise check the probability If you want to try this, play around with temperature and cooling schedules to see what works

Simulated Annealing for Restaurant Pick a random pair of tables and swap them (or rotate, shift,....) If it’s better, keep it If it’s worse, maybe keep it

Simulated Annealing While we’re not out of time Pick a random change If things get better, keep it If (Random from 0 to 1) < e^(- delta Cost/temp), keep it Maybe reduce the temperature

We Can’t Get The Optimal Answer for All Problems And simulated annealing doesn’t really get us any guaranteed solution. Maybe it’s good. Maybe it’s bad. If we lower our objective, for some problems we can guarantee that our solution is within some factor of optimal....

Approximation Algorithms Some problems we can solve: MST, shortest path, bitonic TSP, maximum flow, sorting..... But for others, we can use approximation algorithms: Steiner tree heuristics, general TSP, job scheduling, vertex cover, set cover,.... Sometimes this is better than things like simulated annealing. Sometimes not.

Tabu Search Like Hill Climbing but requires less memory. We remember the last N places that we “visited” and avoid them. Iterative procedure for solving discrete combinotorial optimization developed by GLOVER A move from one solution to another results in best available solution. To escape from local optima & to prevent cycling, some moves are classified as Tabu moves.

A local search algorithm starts from some initial solution and moves from neighbor to neighbor as long as possible while decreasing the objective function value. The neighborhood function N The starting solution s N(s) N(s,k) = N(s) - T(s,k) ‘t’ tabu moves Let Mk={D1, D2,…, D t} denote the set of the last t moves; T(s,k) = {D1-1(s),…, Dt-1(s)} Di-1(s) is the solution obtained from s by restoring some of the attributes that changed when move Di was applied

Basic Tabu Search Algorithm k := 1. generate initial solution WHILE the stopping condition is not met DO Identify N(s). (Neighbourhood set) Identify T(s,k). (Tabu set) Identify A(s,k). (Aspirant set) Choose the best s’ N(s,k) = {N(s) - T(s,k)} U A(s,k). Memorize s’ if it improves the previous best known solution s := s’. k := k+1. END WHILE

29 Example: Flexible Job Shop Problem n jobs m machines Alternative machines for each operation one operation -- one machine (no preemption) Assign each operation to a machine (routing problem) Order the operations on the machines (sequencing problem) Goal: minimize the make span

30 The neighborhood function used based on moving operations

31 Example: tabu list size

Conclusions Question:‘If GAs are so smart, why ain’t they rich?’ Answer:‘Genetic algorithms are rich - rich in application across a large and growing number of disciplines.’ - David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning

T HANKYOU