1. Frédéric Saubion LERIA Learning and Intelligent OptimizatioN Conference Autonomous Search

2. Based on joint works on this topic with : G. Di Tollo A. Fiahlo Y. Hamadi F. Lardeux J. Maturana E. Monfroy M. Schoenauer M. Sebag Learning and Intelligent OptimizatioN Conference

3. 1.Introduction 2.Main Ideas 3.Taxonomy of AS 4.Focus on examples 5.Conclusion and challenges Outline

4. Introduction Generic modeling tools for engineers (Decision) Variables Domains Constraints Mathematical Model Mathematical Model Solver Solving Constraint Optimization and Satifaction Problems

5. Introduction Map coloring problem Satifaction Problems

6. Introduction Map coloring problem Satifaction Problems

7. Introduction Map coloring problem Satifaction Problems

8. Introduction Optimization Problems Travelling Salesman Problem : find a round trip across cities with minimal cost

9. Introduction Optimization Problems Travelling Salesman Problem : find a round trip across cities with minimal cost

10. Introduction Optimization Problems Travelling Salesman Problem : find a round trip across cities with minimal cost

11. Introduction Search landscapes are difficult to explore Many variables Complex constraints Problems are more and more complex

12. Introduction Search landscapes are difficult to explore Exploration vs. Exploitation Balance Problems are more and more complex

13. Introduction An illustrative example : solving SAT SAT CNF instance Devising more and more complex Solving algorithms Litterals Clauses Assignment (1 0 0)

14. Introduction Devising more and more complex Solving algorithms How to explore the binary search space (assignments) to find a solution ?

15. Introduction Devising more and more complex Solving algorithms Use Local Search

16. Introduction Devising more and more complex Solving algorithms Basic Local Search 0 1 0 1 1 Choose a random initial assignment

17. Introduction Devising more and more complex Solving algorithms Basic Local Search Compute the number of true and false clauses

18. Introduction Devising more and more complex Solving algorithms Basic Local Search Try to improve by changing a value (flip) 0 1 0 1 1 0 1 1 1 1 Move to a neighbor

19. Introduction Devising more and more complex Solving algorithms Basic Local Search Until finding a solution

20. Introduction Devising more and more complex Solving algorithms Short overview of the story : a first greedy version GSAT Bart Selman, Hector J. Levesque, David G. Mitchell: A New Method for Solving Hard Satisfiability Problems.AAAI 1992: 440-446 A first boat for binary seas

21. Introduction Devising more and more complex Solving algorithms 1 0 1 0 1 1 0 0 11 11 000 Problem : Many possible moves (many variables)

22. Introduction Devising more and more complex Solving algorithms Restrict neighborhood Select a false clause C abcdefg 0101100

23. Introduction Devising more and more complex Solving algorithms Get stuck in local optima

24. Introduction Devising more and more complex Solving algorithms Add pertubations Select a false clause C With a random probability p Perform a random flip for C With (1-p) Select the variable with best IMP Perform best move If solution then stop Else go on Parameter !

25. Introduction Devising more and more complex Solving algorithms Use restarts False Clauses Iterations Parameter !

26. Introduction Devising more and more complex Solving algorithms WalkSAT : adding a noise and random restart Henry A. Kautz, Bart Selman: Noise Strategies for Improving Local Search..AAAI 1994

27. Introduction Devising more and more complex Solving algorithms How to break ties ? 0 1 0 1 1 0 0 +3

28. Introduction Devising more and more complex Solving algorithms Add more sophisticated heuristics Compute the age of the variable If the best variable is not the most recent then flip Else With a random probability p Perform a random flip the second best With (1-p) Flip the best Parameter !

29. Introduction Devising more and more complex Solving algorithms Novelty : using more strategies to perform improvements (age of the variable) D.A. McAllester, B. Selman and H. Kautz. Evidence for invariant in local search.In Proceedings of AAAI-97, AAAI Press 1997, pages 321-326.

30. Introduction Devising more and more complex Solving algorithms And improvements go on … Novelty +,Novelty ++, …, TNM, Sattime…

31. Introduction Devising more and more complex Solving algorithms Captain Jack : many indicators and thus selection strategies Dave A. D. Tompkins, Adrian Balint, Holger H. Hoos: Captain Jack: New Variable Selection Heuristics in Local Search for SAT. SAT 2011: 302-316

32. Introduction Adding more parameters and heuristics Devising more and more complex Solving algorithms More flexible algorithms Fit to different instances Set parameters/heuristics values Understand the behavior

33. John Rice. The algorithm selection problem. Technical Report CSD-TR 152, Computer science department, Purdue University, 1975. The Algorithm Selection Problem Main ideas

34. John Rice. The algorithm selection problem. Technical Report CSD-TR 152, Computer science department, Purdue University, 1975. The Algorithm Selection Problem Main ideas

35. Related Questions Main ideas Using several algorithms for solving a class of problems

36. Related Questions Main ideas Adjusting the parameters of one algorithm

37. Main Objectives Main ideas Need for more autonomous solving tools Increasing number of works in this trend : LION, Special sessions in EA conferences (GECCO,…) …

38. Ideas for More Autonomous Solvers How to use an algorithm that includes Many parameters Many possible heuristics or components Ideas

39. Ideas for More Autonomous Solvers How to use an algorithm that include Many parameters Many possible heuristics or components How to automate all these choices ? Ideas

40. Off-line Automated Tuning Ideas Run your solver on some problems Experiment automatically parameters values

41. Off-line Automated Tuning Ideas Run your solver on new problems with these parameters values

42. Off-line Automated Tuning Ideas Question : Generality of the parameters ?

43. On-line Parameter Control Ideas Try to react during the resolution by changing the parameter

44. On-line Parameter Control Ideas Example : try to increase some parameter when possible

45. On-line Parameter Control Ideas Question : How to react efficiently ?

46. Hyper Heuristics Ideas Combine basic solving heuristics

47. Hyper Heuristics Ideas Get new solvers

48. Hyper Heuristics Ideas Question : How to learn the suitable solver ?

49. Portfolios Based Solvers Ideas Use different types of solvers

50. Portfolios Based Solvers Ideas Learn how to select the right solver for a given problem

51. Portfolios Based Solvers Ideas Question : Reliability of the learning process ?

52. Why introducing the concept of Autonomous Search ? Taxonomy Taxonomies

53. Classification : Solving Solving Methods Tree-Based Search Metaheuristics SLSEA On-line Off-line Auto Complete/incomplete search, Model representation Other optimization paradigms (e.g., ACO ) Taxonomy

54. Classification : Parameters Solving Methods Tree-Based Search SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Numerical/discrete values Components of the solver Vs. Configuration of the solver Taxonomy

55. Classification : Settings Solving Methods Tree-Based Search EA Parameter setting method On-line Off-line Auto Parameter type Experiment-based Feedback Control Measures and learning techniques (reinforcement learning, statistical learning, case- base reasonning…) Taxonomy

56. Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Parameter Setting in Evolutionary Computation

57. Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Parameter Setting Parameter Tuning Parameter Control DeterministicAdaptiveSelf-adaptive

58. Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Optimization of algorithms (automated tuning)

59. Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Optimization of algorithms (automated tuning) SLS Based (ParamILS) Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, Thomas Stützle: ParamILS: An Automatic Algorithm Configuration Framework. J. Artif. Intell. Res. (JAIR) 36: 267-306 (2009) GA Based (Revac) Volker Nannen, A. E. Eiben: Efficient relevance estimation and value calibration of evolutionary algorithm parameters. IEEE Congress on Evolutionary Computation 2007: 103-110 Racing techniques Mauro Birattari, Thomas Stützle, Luis Paquete, Klaus Varrentrapp: A Racing Algorithm for Configuring Metaheuristics. GECCO 2002: 11- 18 …

60. Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Reactive Search Learning for SLS

61. Related Approaches Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Hyper heuristics

62. Hyper Heuristics Taxonomy Two possible views heuristics to choose heuristics heuristics to generate heuristics Burke EK, Hyde M, Kendall G, Ochoa G, Ozcan E, Woodward J Handbook of Meta-heuristics, A Classification of Hyper-heuristics Approaches

63. Proposing a general view of AS Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Taxonomy Autonomous Search

64. Autonomous Search Genesis Taxonomy Gather works from the CSP solving community Workshop Autonomous Search CP 2007, Providence (RI)

65. Autonomous Search Genesis Taxonomy Identify common concepts, goals and challenges for future works

66. Requirements for an Autonomous Search System Parameters Taxonomy Modify its internal components Parameters Fine grain heuristics Coarse grain solving techniques Model representation React to external forces and opportunities Search landscape analysis (quality, diversity,...) External knowledge (prediction models, rules,...)

67. CS Related Areas Taxonomy Solving Techniques Point of View Constraint Programming Operation Research Evolutionary Computation Adjustment Techniques Point of View Reinforcement Learning Statistical Learning Information Theory

68. CS Related Areas Taxonomy Solving Techniques Point of View Constraint Programming Operation Research Evolutionary Computation Adjustment Techniques Point of View Reinforcement Learning Statistical Learning Information Theory Not limited to…

69. Examples of works Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Examples

70. Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Manual Empirical Tuning Deciding the Size of a Tabu List Mazure B, Sais L, Gregoire E, Tabu search for sat. In : AAAI/IAAI, pp 281285, 1997

71. Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Automated Parameter Tuning (SLS based) Hutter F, Hoos H, Stutzle T Automatic algorithm configuration based on local search. AAAI 2007

72. Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Portfolios Approaches Features based linear regression and classiers Xu L, Hutter F, Hoos HH, Leyton-Brown K Satzilla : portfolio-based algorithm selection for sat. JAIR 2008

73. Examples of works Examples Solving Methods Tree-Based Search Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Learning Combinations of Well- known Heuristics Epstein S, Freuder E, Wallace R Learning to support constraint programmers. Comput Intell 2005

74. Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Discovering heuristics (variable selection in SAT SLS) Alex S. Fukunaga : Automated Discovery of Local Search Heuristics for Satisability Testing. Evolutionary Computation 2008

75. Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Automated Tuning Adjusting the size of a Tabu List R. Battiti, G. Tecchiolli : The Reactive Tabu Search. INFORMS Journal on Computing 6(2): 126-140 (1994)

76. Examples of works Examples Solving Methods Tree-Based Search Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Controlling Variable Orderings and Values Selection in Search Heuristics Boussemart F, Hemery F, Lecoutre C, Sais L Boosting systematic search by weighting constraints. ECAI2004 2004

77. Examples of works Examples Solving Methods Tree-Based Search Metaheuristics SLS EA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Adaptive Genetic Algorithms A.E. Eiben, Z. Michalewicz, M. Schoenauer, J. E. Smith Parameter Control in Evolutionary Algorithms. In Parameter Setting in Evolutionary Algorithms 2007

78. Focus on an example : how to design an AS system ? Generic Evolutionary Algorithm for Constraint Satisfaction Problem Focus Population Apply variation operator New Population Solution ?

79. Focus on an example Generic Search Algorithm for Constraint Satisfaction Problem Focus Configuration Apply variation operator New configuration Solution ? Question : How to choose the suitable operator at each iteration ?

80. Focus on an example Idea Associate a probability of application to each operator (parameter) Select an operator according to this probability scheme Focus

81. Focus on an example Question : how to set the probabilities (parameters of the algorithm) ? Focus

82. Focus on an example Use a principled approach to tune your parameter Search in the parameters space Assess the performance of each setting through runs of the algorithm on selected Instances ParamILS, REVAC, F-Race … Focus

83. Focus on an example Second idea: Control the probability during the run Evaluate the performance of each operator after application Adjust the parameters according to the performances Focus

84. General process for control (Automated Operator Selection) Focus

85. Focus on an example What are the suitable criteria ? -Quality -Fitness diversity -Genotypic diversity -Time -… Focus

86. Focus on an example What are the suitable criteria ? -Quality -Fitness diversity -Genotypic diversity -Time -… Focus

87. Different performnce mesearues Focus Sliding Windows Mean or Max ? How to measure the impact ?

88. Focus on an example What is the performance of the operators ? Fix a search policy Dynamic policy Values against rank … Focus

89. Focus on an example What is the performance of the operators ? Fix a search policy Focus

90. Focus on an example What is the performance of the operators ? No values : Pareto rank of the operators Area under the curve Focus

91. Estimating efficience of operators How to reward the operators ? Proportionally to their performance Focus

92. Estimating efficience of operators Using UCB (Upper Confidence Bound) (reinforcement learning technique) Exploration + Exploitation of the operators Choosing the operator having the best UCB Focus

93. Estimating efficience of operators Warning : UCB converge asymptotically to Gain for the MAB But here we have dynamic changes Use of statistical test to restart learning. Focus

94. Different selection processes Focus

95. How to assess the performances of your system ? Focus

96. Whats next ? Solving Methods Tree-Based Search Metaheuristics SLSEA Parameter setting method On-line Off-line Auto Parameter type Structural Behavioral Focus

97. Conclusion Many different possible approaches Guidelines for designing new autonomous Solvers Off-line/On-line Behavioural parameters/components Control of the efficient heuristics/discovering new heuristics … Conclusion

98. Challenges Comparing performances Autonomous vs. ad-hoc Off-line Tuning vs. On-line control Representative benchmarking Conclusion

99. Challenges Comparing performances Methodologies for comparisons New competitions Chesc ( Cross-domain Heuristic Search Challenge) G. Ochoa and her team Related to No Free-Lunch Theorems More reliable on more problems Conclusion

100. Challenges Parameters induced by the AS system Abstract parameters should be more easy to control (e.g., EvE balance) New parameters should be less sensitive than original ones Fewer paramaters are easier to adjust Conclusion

101. Challenges Learning Interactions solving-learning Improving learning off-line Short term (react) vs. long term (prediction) Continuous search (Arbelaez, Hamadi & Sebag) Conclusion

102. Challenges Distributed and parallel computing Improving algorithms space exploration Sharing information on parameters Sharing information on problems Conclusion

103. Challenges Towards more generic on-line control tools Identify generic control techniques and measures Control various components type (behavioral parameters, objective functions, heuristics…) Conclusion

104. Some books to read Conclusion

105. And ;-) Conclusion

106. So Sorry for missing references and works Conclusion I will not forget important works and references

107. So Questions Conclusion I will not forget important works and references