1. Transformation Methods MOM (Method of Multipliers) Study of Engineering Optimization Guanyao Huang RUbiNet - Robust and Ubiquitous Networking Research Group 2010 July 9th

2. Introduction  Motivation Ill-condition of subproblems in penalty approaches.  Method: MOM R is no longer iteratively updated, and are updated.  Benefits Contour shape remains the same.

3. Motivation Inverse penalty

4.  Convergence is associated with ever- increasing distortion of the penalty contours, which increases significantly the possibility of failure of the unconstrained search method. The unconstrained search might not be completed successfully.

5. Solution  Method of centers Which is equivalent to: These parameter-free methods, although attractive on the surface, are exactly equivalent to SUMT with a particular choice of updating rule for R.

6. MOM: augment the lagrangian to form an unconstrained function whose minimum is a Kuhn-Tucker point of the original problem Another solution: MOM

7. Detailed procedure

8. Property: Second order derivative If the constraints are linear

9. Link between Lagrange multiplier and KT (1)  When iteration terminate?  or then: with: The limit points of is KT point

10. Link between Lagrange multiplier and KT (2)  Lagrange multiplier estimates:

11. MOM characteristics We still have the problem of choosing R! (6.3.6)

12. Revisit example 6.1

15. Variable Bounds  Experience: simply set the out-of- bounds variables to their violated bounds simultaneously.  An example of MOM: Example 6.4. Some bounds are not linear. DFP: Davidon – Fletcher – Powell