1. B.Ombuki-Berman1 Swarm Intelligence Ant-based algorithms Ref: Various Internet resources, books, journal papers (see assignment 3 references)

2. B.Ombuki-Berman2 Ant-based algorithms

3. B.Ombuki-Berman 3 Swarm Intelligence Originated from the study of colonies, swarms of social organism Studies of the social behavior of organisms in swarms inspired the development of very efficient algorithms  the foraging behaviour of ants resulted in ant colony optimization algorithms  simulation studies of of bird flocks results in Particle swarm optimization A very recent and young field in computer science, with much potential ..lots of room for contributions!

4. B.Ombuki-Berman 4 Ant-based algorithms Ant-based systems are a population-based stochastic search methods.  Sound familiar- it is similar to genetic algorithms There is a population of ants, with each ant finding a solution and then communicating with the other ants, how?

5. B.Ombuki-Berman 5 Ant System Swarm Intelligence Algorithm  Based on real life animal swarms/groups  Exhibit efficient ways to solve problems Ant System  Developed by Marco Dorigo, 1991  Modeled after real life ant colonies, based on results of experiment by Goss

6. B.Ombuki-Berman 6 Ant System Experiment by Goss et al ’89  Ants started at nest  Food placed some distance away  Paths of different length between nest and food  Ants found shortest path!

7. B.Ombuki-Berman 7 Ant Algorithms for shortest path Ants choose paths depending on pheromone After collecting food, paths are marked After some time, the shortest path has the highest probability

8. B.Ombuki-Berman 8 Ant System When ants travel they mark their path with substance called pheromone  Attracts other ants When an ant reaches a fork in its path the direction it follows is based on amount of pheromone it detects  Decision probabilistically made This causes positive feedback situation (i.e. Choosing a path increases the probability it will be chosen)

9. B.Ombuki-Berman 9 Ant algorithms We need to explore the search space, rather than simply mapping a route Ants should be allowed to explore paths and follow the best paths with some probability in proportion to the intensity of the pheromone on a given edge/trail. If the ants simply follow the path with the highest amount of pheromone on it, our search will quickly likely settle on a very sub- optimal solution

10. B.Ombuki-Berman 10 Ant algorithms The probability of an ant following a certain route is a function of both the pheromone intensity, and of what the ant can see. Furthermore, the pheromone trail must not build unbounded, hence evaporation is needed.

11. B.Ombuki-Berman 11 Ant System Group of ants start at home/nest  An initial amount of pheromone already placed on edges Travel on edges  Edges contain pheromone amount Visit nodes  Probability of Selecting next node Based on distance between nodes and pheromone amount

12. B.Ombuki-Berman 12 Ant System Ants travel from node to node until end  decision based on transition probability (called state transition) Once all ants finished  Solutions compared  Pheromone evaporation applied to all edges  Pheromone increased along each edge of best/each ant’s path Original ant system: at each iteration, the pheromone values are updated by all the ants that have build a solution in the iteration itself.  Daemon activities can be run (like local search) Redo until termination criteria met

13. B.Ombuki-Berman 13 Requirements Problem being solved must be in graphical format  Since algorithm is based on path finding behavior  Not always apparent Must be finite (must have a start and end)

14. B.Ombuki-Berman 14 Algorithm While ( termination not satisfied )  create ants Starting point depends on problem constraints Initial pheromone is > 0, but very small  Find solutions  Pheromone update  Daemon activities {optional}

15. B.Ombuki-Berman 15 Algorithm While ( termination not satisfied )  create ants  Find solutions Transition probability:  Pheromone update  Daemon activities {optional} Quantity of pheromone Heuristic distance α,β constants

16. B.Ombuki-Berman 16 Algorithm (AS) While ( termination not satisfied )  create ants  Find solutions  Pheromone update  Daemon activities {optional} Evaporation rate Pheromone laid by each ant that uses edge (i,j)

17. B.Ombuki-Berman 17 NOTE: Ant System (AS) Pheromone update : T is the tour done at time t by ant k, L is the length, Q is a heuristic parameter. Pheromone decay:

18. B.Ombuki-Berman 18 Algorithm While ( termination not satisfied )  create ants  Find solutions  Pheromone evaporation  Daemon activities {optional} Usually, a local search algorithm is employed here May also appear after “Find solutions” stage

19. B.Ombuki-Berman 19 Problems Ant System tends to converge quickly This means that its exploitation of the best solution found is too high, it should be exploring solution space more  Pheromone evaporation/update rule (better rule may exist) what is the evaporation rate? Led to extensions of the ant system  MAX-MIN Ant system  Ant colony system  Foot-Stepping  Others (will not be discussed)

20. B.Ombuki-Berman 20 MAX-MIN Ant System Developed by Stutzle and Hoos 2000 An improvement over the original Ant System to allow for more exploration  Introduced use of tending to global best from iteration best solution over time i.e., only the best ant updates the pheromone trails, and that, the value of the pheromone is bound, upper and lower bound imposed  bounds on pheromone that are dependant on solution quality  Bounds set empirically Max-min has obtained good results for many types of problems

21. B.Ombuki-Berman 21 MAX-MIN Ant System MMAS  High exploration at the beginning  Only best ant can add pheromone  Sometimes uses local search to improve its performance

22. B.Ombuki-Berman 22 Ant Colony System (ACS) Most popular/interesting contribution of ACS is introduction of a local pheromone update in addition to the pheromone update performed at the end of the construction process (known as offline pheromone update) Local pheromone update is performed by all ants after each construction step  Each ant applies it only to the last edge traversed: where is the pheromone decay coefficient

23. B.Ombuki-Berman 23 Ant Colony System The offline pheromone update, similary to MAX-MIN is applied to the end of each iteration, by only one ant, which can either be the iteration-best, or best-so-far. The update rule is slightly different from MAX-MIN though (see paper) Another important difference between ACS and Ant System is in the decision rule used by the ants during the construction process by using the so called pseudo random proportional rule.

24. B.Ombuki-Berman 24 Compared to GAs Compared to genetic algorithms:  retains memory of entire colony instead of previous generation only  less affected by poor initial solutions (due to combination of random path selection and colony memory)

25. B.Ombuki-Berman 25 ACO for TSP Input: set of cities given, and distance between each city is known Goal: Find the shortest tour that allows each city to visited exactly once. ACO algorithm set parameters, initiliaze pheromone trails while termination condition not met do ConstructAntSolution ApplyLocal search (optional) UpdatePheromones EndWhile

26. B.Ombuki-Berman 26 Ant Algorithms and the TSP At the start of the algorithm an ant is placed in each city Variations have been proposed by Dorigo et al.

27. B.Ombuki-Berman 27 Ant Algorithms and the TSP Time, t, is discrete. t(0) marks the start of the algorithm. At t+1 every ant will have moved to a new city Assuming that the TSP is being represented as a fully connected graph, each edge has an intensity of trail on it. This represents the pheromone trail laid by the ants Let T i,j (t) represent the intensity of trail edge (i,j) at time t

28. B.Ombuki-Berman 28 Ant Algorithms and the TSP When an ant decides which city to move to next, it does so probabilistically, based on the distance to that city and the amount of pheromone intensity on the connecting edge The distance to the next city, is known as the visibility, n ij, and is defined as 1/d ij, where, d, is the distance between cities i and j.

29. B.Ombuki-Berman 29 Ant Algorithms and the TSP At each time unit, i.e., constructive step, evaporation takes place The amount of evaporation, p, is a value between 0 and 1

30. B.Ombuki-Berman 30 Ant Algorithms and the TSP In order to prevent ants from visiting the same city in the same tour a data structure, Tabu list, is maintained This prevents ants from visiting cities they have already visited Tabu k is defined as the list for the k th ant and it holds the cities that have already been visited

31. B.Ombuki-Berman 31 Ant Algorithms and the TSP After each ant tour the trail intensity on each edge is updated using the following formula T ij (t + n) = p. T ij (t) + ΔT ij Q is a constant and L k is the tour length of the k th ant

32. B.Ombuki-Berman 32 Ant Algorithms and the TSP Transition Probability where  and  are control parameters that control the relative importance of pheromone trail versus visibility

33. B.Ombuki-Berman 33 Ant Systems Algorithm for TSP Initialize Place each ant in a randomly chosen city Choose NextCity(For Each Ant) more cities to visit For Each Ant Return to the initial cities Update pheromone level using the tour cost for each ant Print Best tour yes No Stopping criteria yes No

34. B.Ombuki-Berman 34 Ant Algorithms - Applications Marco Dorigo, who did the seminal work on ant algorithms, maintains a WWW page devoted to this subject http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html Check this site further information about ant algorithms, tutorial, software, applications and main publications.

35. B.Ombuki-Berman35 Ant Colony Optimization applied to Job Shop Scheduling Mario Ventresca and B.Ombuki-Berman

36. 36 Overview Ant System Max-Min and Foot Stepping Ant System for JSSP Brief overview of results Concluding remarks

37. B.Ombuki-Berman 37 Job Shop Scheduling (JSSP) Difficult NP-Hard Optimization Problem Involves assigning jobs to machines Jobs broken into tasks  Same number of tasks per job as machines  Each task has processing time Task constraints and limitations  Processed according to predefined order  No concurrent processing  No-preemption

38. B.Ombuki-Berman 38 Job Shop Scheduling Example Instance: (machine, duration) Possible Schedule (not optimal) Job 1 Job 2 (2,10)(1,2)(3,5) (1,12)(3,6)(2,2) M1M1 Job 1 Job 2 24 M2M2 Job 1 Job 2 32 M3M3 Job2Job 135

39. B.Ombuki-Berman 39 Recall Problems Ant System tends to converge quickly Extensions  Max-Min  Ant Colony system  Foot-Stepping

40. B.Ombuki-Berman 40 Ant System and JSSP Graphical Format 2 Job 1 1 2 3 Task M1 M3M1M2 M3 Uni-directional links maintain pre-ordering criteria, distance is processing time NOTE!! Once a task has been visited it cannot be returned to

41. B.Ombuki-Berman 41 Foot Stepping An original contribution used to enhance exploration in the Ant System In nature it would be like stepping on an already found ant path (being careful not to step on any)  Ants are forced to find a different way around, maybe (hopefully) even a better path

42. B.Ombuki-Berman 42 Foot-Stepping Developed in 2003 (Ventresca, Ombuki) Allow ants to perform further search as opposed to hybridization (via local search)  More like nature, uses ants to improve solutions by altering environment Idea  Alter pheromone values to force ants to search more of solution space  Apply when ants seem to have converged to a solution

43. B.Ombuki-Berman 43 Foot Stepping Two main types  Non-Propagation based Foot Stepping (NPFS)  Propagation based Foot Stepping (PFS) NPFS works in the same manner as real life, but the foot goes away… NPFS, Basic Algorithm: 1. Select a set of vertices 2. cause highest pheromone leaving each vertex to become very small (not zero!!) 3. Cause other pheromone values to increase 4. Continue running ant algorithm until or near stagnation, reinforcing best solution 5. Repeat desired number of times

44. B.Ombuki-Berman 44 Foot Stepping Propagation  From the original footstep location, decreasing amounts of pheromone are deposited along each proceeding vertex’s non-maximal pheromone edges  Propagation is based on the solution of the best ant thus far PFS Basic Algorithm : 1. Select a set V, of vertices 2. cause highest pheromone leaving each vertex to become very small (again, not zero!!) 3. Cause other pheromone values to increase 4. From each vertex of V propagate pheromone to end of graph or desired distance 5. Continue running ant algorithm until or near stagnation, reinforcing best solution 6. Repeat desired number of times

45. B.Ombuki-Berman 45 Summary of Results Foot-Stepping always improved the Ant System results (unless AS found an optimal)  Most foot-steps caused improvements Foot-Stepping was compared to Max-Min  Both techniques seemed to yield similar results  Nearly all solutions were within 2-5% of each other

46. B.Ombuki-Berman 46 Concluding remarks A pheromone altering strategy (Foot Stepping) was presented that improved solution quality over a basic Ant System for the JSSP  Alters the ants decision policy probabilities, thus forcing them to explore around the current solution  Shows potential for leaving local optima  Seems to yield similar results to Max-Min Future work involves  Statistics to provide confidence in method  Examine different propagation techniques and types problems