Download presentation

Presentation is loading. Please wait.

Published byDevon Beecroft Modified over 3 years ago

1
Frédéric Saubion LERIA Learning and Intelligent OptimizatioN Conference Autonomous Search

2
Based on joint works on this topic with : G. Di Tollo A. Fiahlo Y. Hamadi F. Lardeux J. Maturana E. Monfroy M. Schoenauer M. Sebag Learning and Intelligent OptimizatioN Conference

3
1.Introduction 2.Main Ideas 3.Taxonomy of AS 4.Focus on examples 5.Conclusion and challenges Outline

4
Introduction Generic modeling tools for engineers (Decision) Variables Domains Constraints Mathematical Model Mathematical Model Solver Solving Constraint Optimization and Satifaction Problems

5
Introduction Map coloring problem Satifaction Problems

6
Introduction Map coloring problem Satifaction Problems

7
Introduction Map coloring problem Satifaction Problems

8
Introduction Optimization Problems Travelling Salesman Problem : find a round trip across cities with minimal cost

9
Introduction Optimization Problems Travelling Salesman Problem : find a round trip across cities with minimal cost

10
Introduction Optimization Problems Travelling Salesman Problem : find a round trip across cities with minimal cost

11
Introduction Search landscapes are difficult to explore Many variables Complex constraints Problems are more and more complex

12
Introduction Search landscapes are difficult to explore Exploration vs. Exploitation Balance Problems are more and more complex

13
Introduction An illustrative example : solving SAT SAT CNF instance Devising more and more complex Solving algorithms Litterals Clauses Assignment (1 0 0)

14
Introduction Devising more and more complex Solving algorithms How to explore the binary search space (assignments) to find a solution ?

15
Introduction Devising more and more complex Solving algorithms Use Local Search

16
Introduction Devising more and more complex Solving algorithms Basic Local Search 0 1 0 1 1 Choose a random initial assignment

17
Introduction Devising more and more complex Solving algorithms Basic Local Search Compute the number of true and false clauses

18
Introduction Devising more and more complex Solving algorithms Basic Local Search Try to improve by changing a value (flip) 0 1 0 1 1 0 1 1 1 1 Move to a neighbor

19
Introduction Devising more and more complex Solving algorithms Basic Local Search Until finding a solution

20
Introduction Devising more and more complex Solving algorithms Short overview of the story : a first greedy version GSAT Bart Selman, Hector J. Levesque, David G. Mitchell: A New Method for Solving Hard Satisfiability Problems.AAAI 1992: 440-446 A first boat for binary seas

21
Introduction Devising more and more complex Solving algorithms 1 0 1 0 1 1 0 0 11 11 000 Problem : Many possible moves (many variables)

22
Introduction Devising more and more complex Solving algorithms Restrict neighborhood Select a false clause C abcdefg 0101100

23
Introduction Devising more and more complex Solving algorithms Get stuck in local optima

24
Introduction Devising more and more complex Solving algorithms Add pertubations Select a false clause C With a random probability p Perform a random flip for C With (1-p) Select the variable with best IMP Perform best move If solution then stop Else go on Parameter !

25
Introduction Devising more and more complex Solving algorithms Use restarts False Clauses Iterations Parameter !

26
Introduction Devising more and more complex Solving algorithms WalkSAT : adding a noise and random restart Henry A. Kautz, Bart Selman: Noise Strategies for Improving Local Search..AAAI 1994

27
Introduction Devising more and more complex Solving algorithms How to break ties ? 0 1 0 1 1 0 0 +3

28
Introduction Devising more and more complex Solving algorithms Add more sophisticated heuristics Compute the age of the variable If the best variable is not the most recent then flip Else With a random probability p Perform a random flip the second best With (1-p) Flip the best Parameter !

29
Introduction Devising more and more complex Solving algorithms Novelty : using more strategies to perform improvements (age of the variable) D.A. McAllester, B. Selman and H. Kautz. Evidence for invariant in local search.In Proceedings of AAAI-97, AAAI Press 1997, pages 321-326.

30
Introduction Devising more and more complex Solving algorithms And improvements go on … Novelty +,Novelty ++, …, TNM, Sattime…

31
Introduction Devising more and more complex Solving algorithms Captain Jack : many indicators and thus selection strategies Dave A. D. Tompkins, Adrian Balint, Holger H. Hoos: Captain Jack: New Variable Selection Heuristics in Local Search for SAT. SAT 2011: 302-316

32
Introduction Adding more parameters and heuristics Devising more and more complex Solving algorithms More flexible algorithms Fit to different instances Set parameters/heuristics values Understand the behavior

33
John Rice. The algorithm selection problem. Technical Report CSD-TR 152, Computer science department, Purdue University, 1975. The Algorithm Selection Problem Main ideas

34
John Rice. The algorithm selection problem. Technical Report CSD-TR 152, Computer science department, Purdue University, 1975. The Algorithm Selection Problem Main ideas

35
Related Questions Main ideas Using several algorithms for solving a class of problems

36
Related Questions Main ideas Adjusting the parameters of one algorithm

37
Main Objectives Main ideas Need for more autonomous solving tools Increasing number of works in this trend : LION, Special sessions in EA conferences (GECCO,…) …

38
Ideas for More Autonomous Solvers How to use an algorithm that includes Many parameters Many possible heuristics or components Ideas

39
Ideas for More Autonomous Solvers How to use an algorithm that include Many parameters Many possible heuristics or components How to automate all these choices ? Ideas

40
Off-line Automated Tuning Ideas Run your solver on some problems Experiment automatically parameters values

41
Off-line Automated Tuning Ideas Run your solver on new problems with these parameters values

42
Off-line Automated Tuning Ideas Question : Generality of the parameters ?

43
On-line Parameter Control Ideas Try to react during the resolution by changing the parameter

44
On-line Parameter Control Ideas Example : try to increase some parameter when possible

45
On-line Parameter Control Ideas Question : How to react efficiently ?

46
Hyper Heuristics Ideas Combine basic solving heuristics

47
Hyper Heuristics Ideas Get new solvers

48
Hyper Heuristics Ideas Question : How to learn the suitable solver ?

49
Portfolios Based Solvers Ideas Use different types of solvers

50
Portfolios Based Solvers Ideas Learn how to select the right solver for a given problem

51
Portfolios Based Solvers Ideas Question : Reliability of the learning process ?

52
Why introducing the concept of Autonomous Search ? Taxonomy Taxonomies

53
Classification : Solving Solving Methods Tree-Based Search Metaheuristics SLSEA On-line Off-line Auto Complete/incomplete search, Model representation Other optimization paradigms (e.g., ACO ) Taxonomy

54
Classification : Parameters Solving Methods Tree-Based Search SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Numerical/discrete values Components of the solver Vs. Configuration of the solver Taxonomy

55
Classification : Settings Solving Methods Tree-Based Search EA Parameter setting method On-line Off-line Auto Parameter type Experiment-based Feedback Control Measures and learning techniques (reinforcement learning, statistical learning, case- base reasonning…) Taxonomy

56
Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Parameter Setting in Evolutionary Computation

57
Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Parameter Setting Parameter Tuning Parameter Control DeterministicAdaptiveSelf-adaptive

58
Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Optimization of algorithms (automated tuning)

59
Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Optimization of algorithms (automated tuning) SLS Based (ParamILS) Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, Thomas Stützle: ParamILS: An Automatic Algorithm Configuration Framework. J. Artif. Intell. Res. (JAIR) 36: 267-306 (2009) GA Based (Revac) Volker Nannen, A. E. Eiben: Efficient relevance estimation and value calibration of evolutionary algorithm parameters. IEEE Congress on Evolutionary Computation 2007: 103-110 Racing techniques Mauro Birattari, Thomas Stützle, Luis Paquete, Klaus Varrentrapp: A Racing Algorithm for Configuring Metaheuristics. GECCO 2002: 11- 18 …

60
Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Reactive Search Learning for SLS

61
Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Hyper heuristics

62
Hyper Heuristics Taxonomy Two possible views heuristics to choose heuristics heuristics to generate heuristics Burke EK, Hyde M, Kendall G, Ochoa G, Ozcan E, Woodward J Handbook of Meta-heuristics, A Classification of Hyper-heuristics Approaches

63
Proposing a general view of AS Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Autonomous Search

64
Autonomous Search Genesis Taxonomy Gather works from the CSP solving community Workshop Autonomous Search CP 2007, Providence (RI)

65
Autonomous Search Genesis Taxonomy Identify common concepts, goals and challenges for future works

66
Requirements for an Autonomous Search System Parameters Taxonomy Modify its internal components Parameters Fine grain heuristics Coarse grain solving techniques Model representation React to external forces and opportunities Search landscape analysis (quality, diversity,...) External knowledge (prediction models, rules,...)

67
CS Related Areas Taxonomy Solving Techniques Point of View Constraint Programming Operation Research Evolutionary Computation Adjustment Techniques Point of View Reinforcement Learning Statistical Learning Information Theory

68
CS Related Areas Taxonomy Solving Techniques Point of View Constraint Programming Operation Research Evolutionary Computation Adjustment Techniques Point of View Reinforcement Learning Statistical Learning Information Theory Not limited to…

69
Examples of works Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Examples

70
Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Manual Empirical Tuning Deciding the Size of a Tabu List Mazure B, Sais L, Gregoire E, Tabu search for sat. In : AAAI/IAAI, pp 281285, 1997

71
Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Automated Parameter Tuning (SLS based) Hutter F, Hoos H, Stutzle T Automatic algorithm configuration based on local search. AAAI 2007

72
Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Portfolios Approaches Features based linear regression and classiers Xu L, Hutter F, Hoos HH, Leyton-Brown K Satzilla : portfolio-based algorithm selection for sat. JAIR 2008

73
Examples of works Examples Solving Methods Tree-Based Search Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Learning Combinations of Well- known Heuristics Epstein S, Freuder E, Wallace R Learning to support constraint programmers. Comput Intell 2005

74
Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Discovering heuristics (variable selection in SAT SLS) Alex S. Fukunaga : Automated Discovery of Local Search Heuristics for Satisability Testing. Evolutionary Computation 2008

75
Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Automated Tuning Adjusting the size of a Tabu List R. Battiti, G. Tecchiolli : The Reactive Tabu Search. INFORMS Journal on Computing 6(2): 126-140 (1994)

76
Examples of works Examples Solving Methods Tree-Based Search Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Controlling Variable Orderings and Values Selection in Search Heuristics Boussemart F, Hemery F, Lecoutre C, Sais L Boosting systematic search by weighting constraints. ECAI2004 2004

77
Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Adaptive Genetic Algorithms A.E. Eiben, Z. Michalewicz, M. Schoenauer, J. E. Smith Parameter Control in Evolutionary Algorithms. In Parameter Setting in Evolutionary Algorithms 2007

78
Focus on an example : how to design an AS system ? Generic Evolutionary Algorithm for Constraint Satisfaction Problem Focus Population Apply variation operator New Population Solution ?

79
Focus on an example Generic Search Algorithm for Constraint Satisfaction Problem Focus Configuration Apply variation operator New configuration Solution ? Question : How to choose the suitable operator at each iteration ?

80
Focus on an example Idea Associate a probability of application to each operator (parameter) Select an operator according to this probability scheme Focus

81
Focus on an example Question : how to set the probabilities (parameters of the algorithm) ? Focus

82
Focus on an example Use a principled approach to tune your parameter Search in the parameters space Assess the performance of each setting through runs of the algorithm on selected Instances ParamILS, REVAC, F-Race … Focus

83
Focus on an example Second idea: Control the probability during the run Evaluate the performance of each operator after application Adjust the parameters according to the performances Focus

84
General process for control (Automated Operator Selection) Focus

85
Focus on an example What are the suitable criteria ? -Quality -Fitness diversity -Genotypic diversity -Time -… Focus

86
Focus on an example What are the suitable criteria ? -Quality -Fitness diversity -Genotypic diversity -Time -… Focus

87
Different performnce mesearues Focus Sliding Windows Mean or Max ? How to measure the impact ?

88
Focus on an example What is the performance of the operators ? Fix a search policy Dynamic policy Values against rank … Focus

89
Focus on an example What is the performance of the operators ? Fix a search policy Focus

90
Focus on an example What is the performance of the operators ? No values : Pareto rank of the operators Area under the curve Focus

91
Estimating efficience of operators How to reward the operators ? Proportionally to their performance Focus

92
Estimating efficience of operators Using UCB (Upper Confidence Bound) (reinforcement learning technique) Exploration + Exploitation of the operators Choosing the operator having the best UCB Focus

93
Estimating efficience of operators Warning : UCB converge asymptotically to Gain for the MAB But here we have dynamic changes Use of statistical test to restart learning. Focus

94
Different selection processes Focus

95
How to assess the performances of your system ? Focus

96
Whats next ? Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Focus

97
Conclusion Many different possible approaches Guidelines for designing new autonomous Solvers Off-line/On-line Behavioural parameters/components Control of the efficient heuristics/discovering new heuristics … Conclusion

98
Challenges Comparing performances Autonomous vs. ad-hoc Off-line Tuning vs. On-line control Representative benchmarking Conclusion

99
Challenges Comparing performances Methodologies for comparisons New competitions Chesc ( Cross-domain Heuristic Search Challenge) G. Ochoa and her team Related to No Free-Lunch Theorems More reliable on more problems Conclusion

100
Challenges Parameters induced by the AS system Abstract parameters should be more easy to control (e.g., EvE balance) New parameters should be less sensitive than original ones Fewer paramaters are easier to adjust Conclusion

101
Challenges Learning Interactions solving-learning Improving learning off-line Short term (react) vs. long term (prediction) Continuous search (Arbelaez, Hamadi & Sebag) Conclusion

102
Challenges Distributed and parallel computing Improving algorithms space exploration Sharing information on parameters Sharing information on problems Conclusion

103
Challenges Towards more generic on-line control tools Identify generic control techniques and measures Control various components type (behavioral parameters, objective functions, heuristics…) Conclusion

104
Some books to read Conclusion

105
And ;-) Conclusion

106
So Sorry for missing references and works Conclusion I will not forget important works and references

107
So Questions Conclusion I will not forget important works and references

Similar presentations

OK

Local Search: walksat, ant colonies, and genetic algorithms.

Local Search: walksat, ant colonies, and genetic algorithms.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google