Lecture 9: Tabu Search © J. Christopher Beck 2005.

Slides:

Advertisements

Similar presentations

Lecture 6: Job Shop Scheduling Introduction

Advertisements

© J. Christopher Beck Lecture 7: Shifting Bottleneck.

© J. Christopher Beck Lecture 17: Tabu Search.

Lecture 10: Integer Programming & Branch-and-Bound

© J. Christopher Beck Lecture 15: CP Search.

MAE 552 – Heuristic Optimization Lecture 24 March 20, 2002 Topic: Tabu Search.

Introduction to Artificial Intelligence Local Search (updated 4/30/2006) Henry Kautz.

© J. Christopher Beck Lecture 11: Constraint Programming 1.

© J. Christopher Beck Lecture 22: Local Search for Sports Scheduling.

MAE 552 – Heuristic Optimization Lecture 25 March 22, 2002 Topic: Tabu Search.

Metaheuristics The idea: search the solution space directly. No math models, only a set of algorithmic steps, iterative method. Find a feasible solution.

Internet Traffic Engineering by Optimizing OSPF Weights Bernard Fortz (Universit é Libre de Bruxelles) Mikkel Thorup (AT&T Labs-Research) Presented by.

© J. Christopher Beck Lecture 5: Project Planning 2.

© J. Christopher Beck Lecture 13: Modeling in Constraint Programming.

FORS 8450 Advanced Forest Planning Lecture 11 Tabu Search.

© J. Christopher Beck Lecture 6: Time/Cost Trade-off in Project Planning.

Lecture 9: Simplified Shifting Bottleneck

FORS 8450 Advanced Forest Planning Lecture 6 Threshold Accepting.

© J. Christopher Beck Lecture 10: (Full) Shifting Bottleneck.

Heuristic Methods for the Single- Machine Problem Chapter 4 Elements of Sequencing and Scheduling by Kenneth R. Baker Byung-Hyun Ha R2.

1 EE5900 Advanced Embedded System For Smart Infrastructure Static Scheduling.

Local Search. Systematic versus local search u Systematic search  Breadth-first, depth-first, IDDFS, A*, IDA*, etc  Keep one or more paths in memory.

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

© J. Christopher Beck Lecture 16: Local Search.

1 Job Shop Scheduling. 2 Job shop environment: m machines, n jobs objective function Each job follows a predetermined route Routes are not necessarily.

CMPT 463. What will be covered A* search Local search Game tree Constraint satisfaction problems (CSP)

General Purpose Procedures Applied to Scheduling

Tabu Search for Solving Personnel Scheduling Problem

Optimization Problems

The NP class. NP-completeness

School of Computer Science & Engineering

CSCI 4310 Lecture 10: Local Search Algorithms

Search: Advanced Topics Computer Science cpsc322, Lecture 9

BackTracking CS255.

Finding a Path With Largest Smallest Edge

The minimum cost flow problem

Lecture 11: Tree Search © J. Christopher Beck 2008.

Local Search Algorithms

Instructor: Vincent Conitzer

Tabu Search Review: Branch and bound has a “rigid” memory structure (i.e. all branches are completed or fathomed). Simulated Annealing has no memory structure.

B. Jayalakshmi and Alok Singh 2015

Local Search Strategies: From N-Queens to Walksat

Computer Science cpsc322, Lecture 14

Games with Chance Other Search Algorithms

Search: Advanced Topics Computer Science cpsc322, Lecture 9

General Purpose Procedures Applied to Scheduling

Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Nov. 7, 2000

Heuristics Definition – a heuristic is an inexact algorithm that is based on intuitive and plausible arguments which are “likely” to lead to reasonable.

Randomized Hill Climbing

Haim Kaplan and Uri Zwick

Optimization Problems

Heuristics Local Search

Advanced Analysis of Algorithms

School of Computer Science & Engineering

More on Search: A* and Optimization

Stochastic Local Search Variants Computer Science cpsc322, Lecture 16

Heuristics Local Search

EMIS 8373: Integer Programming

Mental Health and Wellness Resources

Lecture 14 Shortest Path (cont’d) Minimum Spanning Tree

Artificial Intelligence

CSE 373: Data Structures and Algorithms

EE5900 Advanced Embedded System For Smart Infrastructure

IOE/MFG 543 Chapter 14: General purpose procedures for scheduling in practice Sections : Dispatching rules and filtered beam search.

Lecture 9 Administration Heuristic search, continued

Local Search Algorithms

Lecture 13 Shortest Path (cont’d) Minimum Spanning Tree

CSC 380: Design and Analysis of Algorithms

Local Search Algorithms

CSCI 465 Data Communications and Networks Lecture 16

Presentation transcript:

Lecture 9: Tabu Search © J. Christopher Beck 2005

Outline Crystal Maze Tabu & 1-machine JSP Tabu & JSP Local Search Tabu Search Tabu & 1-machine JSP Example C.5.4 Tabu & JSP N1 neighborhood © J. Christopher Beck 2005

Crystal Maze You have 5 minutes! Place the numbers 1 through 8 in the nodes such that: Each number appears exactly once No connected nodes have consecutive numbers ? © J. Christopher Beck 2005

Local Search Idea Randomly assign values (even if the constraints are “broken”) Initial state will probably be infeasible Make “moves” to try to move toward a solution © J. Christopher Beck 2005

Random Initial Solution 2 ? 5 ? ? 1 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Random Initial Solution 2 ? 5 ? 1 ? 3 ? ? 6 8 ? ? 4 ? 7 “Broken” constraint Cost = # of broken constraints © J. Christopher Beck 2005

What Should We Do Now? Move: Which two numbers? Swap two numbers Randomly pick a pair The pair that will lead to the biggest decrease in cost Cost: number of broken constraints © J. Christopher Beck 2005

What Should We Do Now? Move: Which two numbers? Swap two numbers Randomly pick a pair The pair that will lead to the biggest decrease in cost Cost: number of broken constraints © J. Christopher Beck 2005

Random Initial Solution 2 ? 5 ? ? 1 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 © J. Christopher Beck 2005

Random Initial Solution 2 ? 5 ? ? 1 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Swap 1 & 2 1 ? 5 ? ? 2 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 © J. Christopher Beck 2005

Random Initial Solution 2 ? 5 ? ? 1 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Swap 1 & 3 2 ? 5 ? ? 3 1 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 © J. Christopher Beck 2005

Random Initial Solution 2 ? 5 ? ? 1 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Swap 1 & 4 2 ? 5 ? ? 4 3 ? 6 ? 8 ? 1 ? 7 ? © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 -1 © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 -1 -2 -3 -1 -2 -3 © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 -1 -2 -3 -1 -2 -3 © J. Christopher Beck 2005

Current State 2 ? 5 ? ? 1 3 ? 6 ? 8 ? 4 ? 7 ? © J. Christopher Beck 2005

Swap 1 & 7: Cost 3 2 ? 5 ? ? 7 3 ? 6 ? 8 ? 4 ? 1 ? © J. Christopher Beck 2005

New Cost Difference Table 1 2 3 4 5 6 7 8 -1 © J. Christopher Beck 2005

Current State 2 ? 5 ? ? 7 3 ? 6 ? 8 ? 4 ? 1 ? © J. Christopher Beck 2005

Swap 3 & 8: Cost 2 2 ? 5 ? ? 7 8 ? 6 ? 3 ? 4 ? 1 ? © J. Christopher Beck 2005

Swap 6 & 7: Cost 1 2 ? 5 ? ? 6 8 ? 7 ? 3 ? 4 ? 1 ? © J. Christopher Beck 2005

Moves Initial State: Cost 6 Swap 1 & 7: Cost 3 Swap 3 & 8: Cost 2 © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 © J. Christopher Beck 2005

Now what? There are no improving moves to make! So far, we have been “hill-climbing” cost moves © J. Christopher Beck 2005

Now what? Options: Restart from a new random state Take the least worse move (increase cost by minimal amount) Try a new style of local search © J. Christopher Beck 2005

Now what? Options: Restart from a new random state Take the least worse move (increase cost by minimal amount) Try a new style of local search © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 © J. Christopher Beck 2005

Swap 1 & 2: Cost 2 1 ? 5 ? ? 6 8 ? 7 ? 3 ? 4 ? 2 ? © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 -1 © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 -1 © J. Christopher Beck 2005

Moves Initial State: Cost 6 Swap 1 & 7: Cost 3 Swap 3 & 8: Cost 2 © J. Christopher Beck 2005

Moves Initial State: Cost 6 Swap 1 & 7: Cost 3 Swap 3 & 8: Cost 2 © J. Christopher Beck 2005

Moves Initial State: Cost 6 Swap 1 & 7: Cost 3 Swap 3 & 8: Cost 2 Swap 1 & 2: Cost 1 … and so on © J. Christopher Beck 2005

Now what? Options: Restart from a new random state Take the least worse move (increase cost by minimal amount) Try a new style of local search © J. Christopher Beck 2005

Now what? Options: Restart from a new random state Take the least worse move (increase cost by minimal amount) Try a new style of local search © J. Christopher Beck 2005

Tabu Search Idea Local search but: Keep a small list of the moves we made so that we don’t revisit the same state Keep a list of 4 pairs: the nodes that the numbers were in before we moved them © J. Christopher Beck 2005

Assign Labels to Nodes B E ? ? C F ? ? ? ? A H D G ? ? © J. Christopher Beck 2005

Return to a State Before We Started Cycling Initial State: Cost 6 Swap 1 & 7: Cost 3 Swap 3 & 8: Cost 2 Swap 6 & 7: Cost 1 © J. Christopher Beck 2005

Just Swapped 6 & 7: Cost 1 B E 2 ? 5 ? C F 6 ? 8 ? ? 7 3 ? A H D G 4 ? © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 © J. Christopher Beck 2005

Just Swapped 6 & 7: Cost 1 B E 2 ? 5 ? C F 6 ? 8 ? ? 7 3 ? A H D G 4 ? © J. Christopher Beck 2005

Swap 1 & 2: Cost 2 B E 1 ? 5 ? C F 6 ? 8 ? 7 ? 3 ? A H D G 4 ? 2 ? Tabu: [(1G,2B)] © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 -1 © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 X -1 © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 X -1 © J. Christopher Beck 2005

Swap 3 & 5: Cost 1 B E 1 ? 3 ? C F 6 ? 8 ? 7 ? 5 ? A H D G 4 ? 2 ? Tabu: [(3H,5E),(1G,2B)] © J. Christopher Beck 2005

Cost Difference Table 1 2 3 4 5 6 7 8 X © J. Christopher Beck 2005

Swap 1 & 3: Cost 1 B E 3 ? 1 ? C F 6 ? 8 ? 7 ? 5 ? A H D G 4 ? 2 ? Tabu: [(1B,3E),(3H,5E),(1G,2B)] © J. Christopher Beck 2005

Swap 5 & 7: Cost 1 B E 3 ? 1 ? C F 6 ? 8 ? 5 ? 7 ? A H D G 4 ? 2 ? Tabu: [(5H,7F),(1B,3E),(3H,5E),(1G,2B)] © J. Christopher Beck 2005

Swap 1 & 5: Cost 1 B E 3 ? 5 ? C F 6 ? 8 ? 1 ? 7 ? A H D G 4 ? 2 ? Tabu: [(1E,5F),(5H,7F),(1B,3E),(3H,5E),(1G,2B)] © J. Christopher Beck 2005

Swap 1 & 5: Cost 1 B E 3 ? 5 ? C F 6 ? 8 ? 1 ? 7 ? A H D G 4 ? 2 ? Tabu: [(1E,5F),(5H,7F),(1B,3E),(3H,5E)] © J. Christopher Beck 2005

Swap 2 & 4: Cost 1 B E 3 ? 5 ? C F 6 ? 8 ? 1 ? 7 ? A H D G 2 ? 4 ? Tabu: [(2G,4D),(1E,5F),(5H,7F),(1B,3E)] © J. Christopher Beck 2005

Swap 2 & 6: Cost 1 B E 3 ? 5 ? C F 2 ? 8 ? 1 ? 7 ? A H D G 6 ? 4 ? Tabu: [(2D,6A),(2G,4D),(1E,5F),(5H,7F)] © J. Christopher Beck 2005

Swap 3 & 5: Cost 0 B E 5 ? 3 ? C F 2 ? 8 ? 1 ? 7 ? A H D G 6 ? 4 ? Tabu: [(3B,5E),(2D,6A),(2G,4D),(1E,5F)] © J. Christopher Beck 2005

Tabu Search A type of local search Start with some (maybe random) initial state Look at the moves in the “neighborhood” and take the best one Remember the last k moves (“tabu list”) so you don’t undo them © J. Christopher Beck 2005

Tabu Search & 1-Machine Example C.5.4, min wjTj Tabu List Size = 2 Jobs 1 2 3 4 pj 10 13 dj 12 wj 14 Jj, pj, dj, wj J1, 10, 4, 14 © J. Christopher Beck 2005

TS Step 1: Find Initial Solution Arbitrarily choose (2, 1, 4, 3) J4, 4, 12, 12 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 © J. Christopher Beck 2005

TS Step 2: Evaluate Neighborhood & Select Move Neighborhood: swap Adjacent pairwise interchange J3, 13, 1, 1 J1, 10, 4, 14 J2, 10, 2, 12 J4, 4, 12, 12 Choose (1,4) J3, 13, 1, 1 J1, 10, 4, 14 J2, 10, 2, 12 J4, 4, 12, 12 wjTj = 480 J4, 4, 12, 12 J3, 13, 1, 1 J1, 10, 4, 14 J2, 10, 2, 12 wjTj = 436 J3, 13, 1, 1 J1, 10, 4, 14 J2, 10, 2, 12 J4, 4, 12, 12 wjTj = 652 © J. Christopher Beck 2005

TS Step 3: Add Move to Tabu List wjTj = 436 J4, 4, 12, 12 Tabu: ((1, 4)) J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 Best so far: (2, 4, 1, 3): 436 © J. Christopher Beck 2005

TS Step 2: Evaluate Neighborhood & Select Move wjTj = 436 J4, 4, 12, 12 Tabu: ((1, 4)) J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 Choose (4,2) J4, 4, 12, 12 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 wjTj = 460 J4, 4, 12, 12 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 wjTj = 500 J4, 4, 12, 12 J2, 10, 2, 12 J3, 13, 1, 1 J1, 10, 4, 14 wjTj = 608 © J. Christopher Beck 2005

TS Step 3: Add Move to Tabu List J4, 4, 12, 12 wjTj = 460 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 Tabu: ((2,4) (1, 4)) Best so far: (2, 4, 1, 3): 436 © J. Christopher Beck 2005

TS Step 2: Evaluate Neighborhood & Select Move J4, 4, 12, 12 wjTj = 460 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 Tabu: ((2,4) (1, 4)) Choose (1,2) J4, 4, 12, 12 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 wjTj = 436 J4, 4, 12, 12 J1, 10, 4, 14 J2, 10, 2, 12 J3, 13, 1, 1 wjTj = 440 J4, 4, 12, 12 J2, 10, 2, 12 J3, 13, 1, 1 J1, 10, 4, 14 wjTj = 632 © J. Christopher Beck 2005

TS Step 3: Add Move to Tabu List J4, 4, 12, 12 wjTj = 440 J1, 10, 4, 14 J2, 10, 2, 12 J3, 13, 1, 1 Tabu: ((1,2), (2,4)) Best so far: (2, 4, 1, 3): 436 © J. Christopher Beck 2005

TS Step 2: Evaluate Neighborhood & Select Move J4, 4, 12, 12 wjTj = 440 J1, 10, 4, 14 J2, 10, 2, 12 J3, 13, 1, 1 Tabu: ((1,2), (2,4)) Choose (1,4) J4, 4, 12, 12 J1, 10, 4, 14 J2, 10, 2, 12 J3, 13, 1, 1 wjTj = 408 J4, 4, 12, 12 J2, 10, 2, 12 J1, 10, 4, 14 J3, 13, 1, 1 wjTj = 460 J4, 4, 12, 12 J1, 10, 4, 14 J3, 13, 1, 1 J2, 10, 2, 12 wjTj = 586 © J. Christopher Beck 2005

TS Step 3: Add Move to Tabu List J4, 4, 12, 12 wjTj = 408 J1, 10, 4, 14 J2, 10, 2, 12 J3, 13, 1, 1 Tabu: ((1,4), (1,2)) Best so far: (1,4,2,3): 408 And so on until a bound on the number of iterations (actually 408 is optimal, but TS has no way of knowing) © J. Christopher Beck 2005

Tabu Search Framework Find initial solution (something quick) Until N moves are done Evaluate neighborhood Choose best non-tabu neighbor Update tabu list Return best solution found © J. Christopher Beck 2005

Tabu Search Framework To completely define a TS algorithm, you must define How the initial solution is found Format and length of tabu list How to generate a neighborhood from a given state © J. Christopher Beck 2005

TS & JSP J0 J1 J2 J0 J2 J1 J0 J1 J2 J1 J2 J0 © J. Christopher Beck 2005

TS & JSP How might you find a first solution? What could your tabu list format be? Can you define a neighborhood? © J. Christopher Beck 2005

The N1 Neighborhood for JSP © J. Christopher Beck 2005

The N1 Neighborhood for JSP Pick a critical path Neighborhood: all swaps on the chosen CP © J. Christopher Beck 2005

TS on Cut Out JSP Run TS on JSP that we have been used in previous lectures Choose initial solution method Choose TS format & length Choose neighborhood Activities Jobs 1 2 3 4 M1, 9 M2, 8 M3, 4 M4, 4 M1, 5 M2, 6 M4, 3 M3, 6 M3, 10 M1, 4 M2, 9 M4, 2 © J. Christopher Beck 2005