2. Formulation of the problem
The function (cost or object function):
examples:
- light curve flux at the given phase as function of N parameters
- chi2 value of the O-C data
The minimization problem:
or, in short:
or, if it is constrained:

3. Formulation of the problem:
A final touch - multi-dimensional minimization without derivatives
“without derivatives” does not mean that the derivatives
do not exist or that they are not regular, it only means
that the derivatives will not be evaluated.
thus the family of derivative-less methods is most suitable
for the problem where we cannot (or would not) write
the derivatives explicitly.
all minimization methods without derivatives are globally
convergent as soon as a minimum (local or global) is
contained within the corresponding parameter hyperspace.

4. Parameter correlation, degeneracy, and minimum globality
Although globally convergent, these methods cannot escape from
local minima nor can they reveal the globality of the minimum.
1) Heuristic scanning. Initiate the minimizer from many starting
points in parameter hyperspace and locate the parts that
attract most scans.
2) Parameter kicking. Once a mininum is reached, perturb it by
taking a Gaussian step in a random direction.
Be wary of formal error estimates,
since the right combination of the
wrong parameters can give you
severe headache!

5. Bracketing 1-D function using Brent's algorithm
For bracketing a minimum in 1-D, we need a triplet of points: (a, b, c).
If a < b < c and the following relation holds:
then there is a minimum which we may find e.g. by bisection.
Bracketing a minimum of a given multidimensional function is not possible!

6. Using Brent's algorithm for multidimensional minimization?
In multidimensional hyperspace, we can initiate the line
minimization from point p in the direction determined
by the given vector u.
Once the 1-D minimization
along u is complete, a new
direction is chosen and the
whole process is repeated.
The choice of directions
distinguishes between
different methods, which
among themselves differ in
applicability and generality.

7. Using Brent's algorithm for multidimensional minimization?
The simplest approach to choose the direction set would
be to adopt unit vectors e1 through eN and to minimize
along their respective directions.
A way out: Powell's conjugate direction set method:
Establish a set of “non-interfering” directions that are
independent among themselves. Thus, line minimization
along one direction is not “spoiled” by the subsequent
minimization along the other.
Two problems:
1) the parameter-set for eclipsing binary stars is induced
from physical description, thus non-ortogonal and
2) even if the parameter-set was ortogonal, the set of
unit-vector directions would fail miserably in case of
elongated minima valleys.

8. A way out: Powell's conjugate direction set method
Expand the cost function f(x) in Taylor series about p:
Formally evaluate its gradient
and its variation along u:
Note that ∇f must be perpendicular to u at minimum!
After f is minimized along u, the algorithm proposes a
new direction v so that this requirement is fulfiled:
: conjugate directions

9. Powell's algorithm put to test:

11. Comparison between the number of iterations required
for the Nelder & Mead's Downhill Simplex (NMS; left)
and Powell's Direction Set (PDS; right):

12. Applying Powell's direction set method to eclipsing binaries
The only quantitative way to put the method to the test
is to use synthetic data:
Most important pars:
i = 85o
q = 0.83
T1 = 6200K
T2 = 5820K
W1 = 5.25
W2 = 5.6
This test binary submitted to DC and NMS in the past.

13. Applying Powell's direction set method to eclipsing binaries
Since they are known, we may compare thier succesfulness:

14. Applying Powell's direction set method to eclipsing binaries
There is a “down-side” to this method: it deprives the
researcher of its usual run-of-the-mill HS coffee:

15. Applications & benefits
In addition to NMS and DC, Powell's direction set
algorithm proves to be a solid candidate for fully
automatic application on the large survey data such
as OGLE, TASS, ASAS, CoRoT, and also Gaia.
Benefits:
- ortogonalization of the hyperspace
- robustness (globally convergent method)
- stability (no need for numerical derivatives)
- simplicity (machine is a worker, not a brain)
- constrained and unconstrained minimization
- formal error estimates from the Hessian matrix

16. Applications & benefits
In addition to NMS and DC, Powell's direction set
algorithm proves to be a solid candidate for fully
automatic application on the large survey data such
as OGLE, TASS, ASAS, CoRoT, and also Gaia.
Downsides:
- gets stuck in local minima
- still too slow: several hours for a complete HS
- no clear notion of the hyperspace shape
Powell's direction set method proves to be an ideal
companion to DC and NMS for solving the inverse
problem of eclipsing binary stars.