Local Search: walksat, ant colonies, and genetic algorithms

Tonight Course evaluations Local search Final exam

Issue How to search really large spaces 10^30,000 + states systematic search hopeless! Local search iterative repair, hill climbing, gradient descent, … guess a start state repeat until satisfied: make small change move to a local neighbor

Examples Learning weights in a neural network Space: set of weights Neighborhood: small delta Learning CPT’s in a Bayesian network EM algorithm These are optimization problems… what about local search for decision problems?

N-Queens N-Queens Demo

GSAT Guess a random truth assignment while (#unsat clauses > 0){ flip a variable that minimizes number of unsatisfied clauses }

Room for improvement Less dramatic performance on structured problems (e.g. planning) Too greedy – can become stuck in local minima Solution: allow some “uphill” moves – many different strategies possible… Simulated annealing Tabu lists Alternate GSAT + random flips

Random Walk Guess a random truth assignment while (#unsat clauses > 0){ pick an unsat clause flip any variable in the clause } Suppose clauses are of length 2; what is probably a flip is “correct”? Solves 2-SAT in O(n^2) time!

Greediness + Randomness If clause length > 2, then pure random walk is very unlikely to converge Suppose we random walk greedily?

Walksat Guess a random truth assignment while (#unsat clauses > 0){ pick an unsat clause with probability p { flip a variable that minimizes number of newly-unsatisfied clauses } otherwise { flip any variable in clause }

Walksat results Best known method for many problems graph coloring (certain) planning problems hard random problems For many (all?) other domains, can mix random walk & backtrack search run backtrack DPLL until time out restart with new random seed for choosing branching variables

Stochastic Backtrack Search Quasigroup Completion Demo

Parallel local search Techniques considered so far only look at one point on the search space at a time… Easy to run many copies in parallel with different random seeds Called portfolio approach Often gives super-linear speedup!

Run time distributions Often there is a great variability in run time depending on random seeds… As parallel processes are added, probability of getting a “lucky” short run can increase quickly!

Informed parallel search Can we further improve performance by exchanging information between the parallel processes? Genetic algorithms Ant colony algorithms

Genetic algorithms Idea: the genetic code of an individual is a point in the search space a gene = a dimension in the search space Reproduction = local moves Asexual reproduction (mutation) = create a child by a small change in the parent Sexual reproduction = create a child by mixing genes of two parents

19 The GA Cycle of Reproduction reproduction population evaluation modification discard deleted members parents children modified children evaluated children

20 A Simple Example The Traveling Salesman Problem: Find a tour of a given set of cities so that  each city is visited only once  the total distance traveled is minimized

21 Representation Representation is an ordered list of city numbers known as an order-based GA. 1) London 3) Dunedin 5) Beijing 7) Tokyo 2) Venice 4) Singapore 6) Phoenix 8) Victoria CityList1 ( ) CityList2 ( )

22 Crossover Crossover combines inversion and recombination: * * Parent1 ( ) Parent2 ( ) Child ( ) This operator is called the Order1 crossover.

23 Mutation involves reordering of the list: * Before: ( ) After: ( ) Mutation

24 TSP GA Demo l mple.html mple.html

25 TSP Example: 30 Cities

26 Solution i (Distance = 941)

27 Solution j (Distance = 800)

28 Solution k (Distance = 652)

29 Best Solution (Distance = 420)

30 Overview of Performance

Practical Applications Design of small sorting circuits Optimization of code fragments Used in Windows kernel??? Training neural networks Can outperform backprop Airport scheduling (Generally) unanswered questions: Would other local search techniques work just as well? Is crossover really key?

Ant colony optimization Idea: Each ant in a colony is local search process Ants (processes) communicate by laying down phenoromes at visited states (“this place looked good to me”) When deciding where to move, ants prefer (with some probability) to follow trail left by other ants Phenorome eventually “evaporates”

Real ants

Why might it work? More “intelligent” noise Ant t10 Ant t3 gradient not gradient, but a good alternative

Beyond phenomerones Combine local gradients to get more global view Ant t10 Ant t10 gradient move by weighted sum of B’s own gradient and A’s gradient

Does it work? Many successful applications in engineering & science Appears to outperform single-thread local search procedures for quadratic programming More work needed to determine exactly why metaphor works when it does!

Summary Local search is an important tool for large, complex optimization and decision problems Ideas: Iterative repair Noise to escape local optima Much work on parallel local search Portfolios Genetic algorithms Ant colony optimization

So…. End of the course. Is that all there is??? Where was the artificial intelligence??!!

In my view… AI is the study of general problem- solving strategies “Meta” algorithms 101 Life: one damned problem after another Strategies evolve at many levels Genetic – “real” evolution Individual human – logical thinking Society – spread of innovation

Unity A relatively small number of concepts various state-space search strategies syntactic structure (both formal logic & natural language) learning strategies – minimizing entropy, error, complexity arise over and over again in biological and artificial systems

Why? This unity of thought biology computation is a cause for wonder … Work in AI is one way of working on the ultimate questions Why? Why here? Why now? Why us?

As the Dali Lama once said… “But if anything I have said is not helpful to you… … then just forget about it!”