1. Multi-Scale Search for Black-Box Optimization: Theory & Algorithms
Abdullah Al-Dujaili
October 2016

2. Black-Box Optimization
Recurrent topic of interest for centuries
Many applications:
Control/planning
Machine learning
Design/ manufacture
Many sub-fields
Convex
Discrete
Multi-objective
Wikipedia

3. Black-Box Optimization
A search problem through point-wise evaluations.
Objective Function
Zero-order (value)
Closed-form
High-order (gradient)
Smoothness

4. Black-Box Optimization Mathematically:

5. Example in Graphics
Geijtenbeek, Thomas, Michiel van de Panne, and A. Frank van der Stappen. "Flexible muscle-based locomotion for bipedal creatures."  ACM Transactions on Graphics (TOG) 32.6 (2013): 206.
The muscle routing and control parameters are optimized using the Covariance Matrix Adaptation [Hansen, 2006] black-box algorithm.

6. Challenges in Black-box Optimization
Dimensionality
Separability
Modality
Complexity
Ruggedness
Conditioning

7. Approaches in Black-box Optimization
Passive
A grid of n points
Return the best point
Inefficient
Active
Sequential decision-making
Next point depends on the previous points.
Exploration vs. Exploitation
Objective Function
Solver

8. Exploration vs. Exploitation
Initial investigations date back to Thompson in 1933 & Robbins in 1952
Formally know as the multi-armed bandit problem.
In Continuous Black-Box Optimization, divide-and-conquer partitioning trees (hierarchical bandits)

9. Multi-Scale Search for Black-Box Optimization
Employ a divide-and-conquer partitioning tree over the search space.
Assign exploration and exploitation scores for each node.
Iteratively, expand nodes based on their scores

10. Classical Method: Lipschitzian Optimization
At time t=0, interval = [a,b]

11. Graphical Interpretation of B-values
Global Search
Local Search
Function Value
Slope C
Selected Interval
Size

12. MSO Algorithms in Literature
Lipschitzian Optimization (LO)
B. O. Shubert. A sequential method seeking the global maximum of a function. SIAM Journal on Numerical Analysis, 9(3): , 1972.
S. Piyavskii. An algorithm for finding the absolute extremum of a function. USSR Computational Mathematics and Mathematical Physics, 12(4):57{67, 1972.
Branch and Bound (BB)
J. Pinter. Global optimization in action: continuous and Lipschitz optimization: algorithms, implementations and applications, volume 6. Springer Science & Business Media, 1995.
Dividing RECTangles (DIRECT)
Jones, D.R., Perttunen, C.D. and Stuckman, B.E., Lipschitzian optimization without the Lipschitz constant. Journal of Optimization Theory and Applications, 79(1), pp
Multilevel Coordinate Search (MCS)
Huyer, W. and Neumaier, A., Global optimization by multilevel coordinate search. Journal of Global Optimization, 14(4), pp
Simultaneous Optimistic Optimization (S00)
Munos, R., Optimistic optimization of deterministic functions without the knowledge of its smoothness. In Advances in neural information processing systems.
Finite-Time Analysis of the above:
Al-Dujaili, Abdullah, S. Suresh, and N. Sundararajan. "MSO: a framework for bound-constrained black-box global optimization algorithms." Journal of Global Optimization (2016): 1-35.
Naïve Multi-Scale Search for Black-Box Optimization (this talk)
Al-Dujaili, Abdullah, and S. Suresh. "A Naive multi-scale search algorithm for global optimization problems." Information Sciences 372 (2016):

13. Recent Multi-scale Search Optimization (MSO)
MSO has been dominantly exploratory
The DIRECT algorithm may reduce to an exhaustive grid search.
Some incorporates local search (exploitation) as a separate component
The MCS algorithm
Expensive Optimization is becoming more relevant (i.e., limited number of function evaluations)
Incorporate local search (exploitation) in the MSO framework.

14. RECENT ALGORITHM FOR Expensive Black-Box Optimization
Naïve Multi-scale Search Optimization (NMSO)
Function value as exploitation score
Depth as exploration score
Depth-wise expansion until no further improvement is noticed and revisit the root.

15. No further improvement in NMSO
Expand one coordinate at a depth
At depth h, choose the (or one) node with the best function value to expand
For the child nodes of the expanded node (h,i), compute:
Decide to continue or put the child nodes in a basket for exploitation at a later stage
Nodes in the basket are expanded only if they have been selected/visited in V number of sweeps while in the basket.

16. No further improvement in NMSO for 2D problem

17. Theoretical Analysis

18. Empirical Analysis
Performance as a function of computational budget (number of function evaluations).

19. Demo

20. Limitations / Future Work
The algorithm is so naïve and so many things can be explored:
Large Scale (high dimensionality)
Adaptive Encoding
Parameter Tuning/ Update
Surrogate Model