Download presentation

Presentation is loading. Please wait.

Published byLesly Badger Modified about 1 year ago

1
A Newton Method for Linear Programming Olvi L. Mangasarian University of California at San Diego

2
Outline Fast Newton method for class of linear programs Large number of constraints (millions) Moderate number of variables (hundreds) Method is based on an overlooked fact Dual of asymptotic exterior penalty problem of primal LP Provides an exact least 2-norm solution of the dual LP Use a finite value of the penalty parameter Exact least 2-norm dual solution generates: Highly accurate primal solution when it is unique Globally convergent Newton method established Eleven lines of MATLAB code Method solves LP with 2-million constraints & 100 variables 14-place accuracy in 17.4 minutes on 400Mhz machine CPLEX 6.5 ran out of memory on same problem Dual method for million-variable LP

3
Primal-Dual LPs Primal Exterior Penalty & Dual Least 2-Norm Problem Dual LP: Exact Dual Least 2-Norm: Primal LP: Asymptotic Primal Exterior Penalty:

4
The Plus Function

5
Dual Least 2-Norm: Karush-Kuhn-Tucker Optimality Conditions for Least 2-Norm Dual Lagrangian: KKT Conditions: Equivalently:

6
Karush-Kuhn-Tucker Optimality Conditions for Least 2-Norm Dual (Continued) Thus : Hence: Thus: solves the exterior penalty problem for the primal LP We have:

7
Equivalence of Exact Least 2-Norm Dual Solution to Finite-Parameter Primal Penalty Minimization

8
Primal Exterior Penalty Primal Exterior Penalty( exact least 2-norm dual solution) : Gradient: Generalized Hessian: where:

9
LP Newton Algorithm (LPN)

10
LPN Convergence

11
Remarks on LPN

12
lpgen: Random Solvable LP Generator %lpgen: Generate random solvable lp: min c'x s.t. Ax =< b; A:m-by-n %Input: m,n,d(ensity); Output: A,b,c; (x,u): primal-dual solution pl=inline('(abs(x)+x)/2');%pl(us) function tic;A=sprand(m,n,d);A=100*(A-0.5*spones(A)); u=sparse(10*pl(rand(m,1)-(m-3*n)/m)); x=10*spdiags((sign(pl(rand(n,1)-rand(n,1)))),0,n,n)*(rand(n,1)-rand(n,1)); c=-A'*u;b=A*x+spdiags((ones(m,1)-sign(pl(u))),0,m,m)*10*ones(m,1);toc0=toc; format short e;[m n d toc0] Elements of A uniformly distributed between –50 and +50 Primal random solution x with elements in [-10,10], approximately half of which are zero Dual random solution u in [0,10], approximately 3n of which are positive

13
lpnewt1: MATLAB LPN Algorithm without Armijo %lpnewt1: Solve primal LP: min c'x s.t. Ax= =0 %Input: c,A,b,epsi,delta,tol,itmax;Output:v(l2norm dual sol),z primal sol epsi=1e-3;tol=1e-12;delta=1e-4;itmax=100;%default inputs pl=inline('(abs(x)+x)/2');%pl(us) function tic;i=0;z=0;y=((A(1:n,:))'*A(1:n,:)+epsi*eye(n))\(A(1:n,:))'*b(1:n);%y0 while (i tol & toc<1800) df=A'*pl((A*y-b))+epsi*c; d2f=A'*spdiags(sign(pl(A*y-b)),0,m,m)*A+delta*speye(n); z=y;y=y-d2f\df; i=i+1; end toc1=toc;v=pl(A*y-b)/epsi;t=find(v);z=A(t,:)\b(t);toc2=toc; format short e;[epsi delta tol i-1 toc1 toc2 norm(x-y,inf) norm(x-z,inf)]

14
LPN & CPLEX 6.5 Comparison (LPN without Armijo vs. Dual Simplex) (400 Mhz Pentium II 2Gig) (oom=out of memory)

15
What About the Case of: n>> m ? LPN is not appropriate LPN needs to invert a very large n-by-n matrix Use instead DLPN (Dual LPN): Find least 2-norm primal solution By solving dual exterior penalty problem with finite penalty parameter

16
Equivalence of Exact Least 2-Norm Primal Probelm and Asymptotic Dual Penalty Problem

17
Difficulty: Nonnegativity Constrained Dual Penalty Use an exterior penalty to handle nonnegativity constraint By: Replace:

18
D LPN & CPLEX 7.5 Comparison (DLPN without Armijo vs. Primal Simplex ) (1.8 MHz Pentium 4 1Gig)(oom=out of memory;error 1001) (infs/unbd=presolve determines infeasibility/unboundedness;error 1101 )

19
Conclusion LPN & DLPN: Fast new methods for solving linear programs LPN capable of solving problems with millions of constraints and hundreds of variables DLPN capable of solving problems with hundreds of constraints and millions of variables Eleven lines of MATLAB code for each of LPN & DLPN Competitive with state-of-the-art CPLEX Very suitable for classification problems Armijo stepsize not needed in many applications Typical termination in 3 to 30 steps

20
Future Work Investigate & establish finite termination for LPN Extend to convex quadratic programming Establish convergence/termination under no assumptions for LPN Generate self-contained codes for classification and data- mining problems

21
Paper & MATLAB Code Available

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google