1. On implicit-factorization block preconditioners Sue Dollar 1,2, Nick Gould 3, Wil Schilders 2,4 and Andy Wathen 1 1 Oxford University Computing Laboratory, Oxford, UK 2 CASA, Technische Universiteit Eindhoven, The Netherlands 3 Rutherford Appleton Laboratory, Chilton, UK 4 Philips Research Laboratory, Eindhoven, The Netherlands

2. Summary of the talk Introduction Direct versus Iterative Methods Preconditioned Conjugate Gradient Method Constraint Preconditioners Implicit Constraint Preconditioners Numerical Examples Future work and conclusions

3. Introduction Interested in solving structured linear systems of equations full rank positive semi-definite

4. Example 1: Equality constrained minimization Interior-point sub-problem Θ (small) barrier weights First-order optimality

5. Factory

6. Example 2: Inequality constrained minimization Interior-point sub-problem C (small) barrier weights First-order optimality

7. Direct vs. Iterative Methods Direct methods –Gaussian elimination with a pivoting strategy –Bunch-Parlett factorization Iterative methods –Krylov subspace methods –MINRES & GMRES – find solution of (1) within n+m iterations –CG may fail because of the indefiniteness of system –Often advantageous to use a preconditioner P

8. PCG Possible to use the preconditioned conjugate-gradient method (Gould, Hribar, Nocedal) (2)

9. Projected PCG –Can perform iteration in the original (x, z) space so long as preconditioner chosen carefully … Solve Iterate until convergence –… –Solve –…

10. Constraint Preconditioners Case C=0 – The matrix P -1 H has – an eigenvalue at 1 with multiplicity 2m – n-m eigenvalues which are defined by the generalized eigenvalue problem (Keller, Gould, Wathen)

11. Case rank(C)=l The matrix P -1 H has at least 2m-l eigenvalues at 1 n-m eigenvalues which are defined by the generalized eigenvalue problem m eigenvalues defined by the generalized eigenvalue problem where w=[x y ; x z ], subject to Y 2 BY 1 x y ≠0.

12. CVXQP1_S: m=50, n=100, G=diag(A)

14. Explicit vs. implicit constraint preconditioner Explicit constraint preconditioners: choose G, and then factorize Implicit constraint preconditioners: find easily-invertible factors R and S so that always holds Pick parts of R and S to match * (≡ G) to parts of A

15. Easily invertible B 1 Require that both R and S are easily block invertible - some sub-blocks should be zero

16. Example 1: C=0 can recover any G (Schilders)

17. Example 2 Example 3

18. Numerical Examples CUTEr test set Case C=0 Explicit G=IImplicit G 22 =I namenmfact.iter.totalfact.iter.totalratio CVXQP11000050001.7322.980.066271.470.49 CVXQP21000025000.04520.150.03290.281.80 CVXQP31000075003.1126.090.085161.350.22 POWELL201000050000.1210.190.0403146.5233.0 PRIMALC123990.004720.00920.002310.00460.50 PRIMALC223870.004110.00710.002310.00420.58 QPNBOEI17263510.028590.440.0044130.0600.13 QPNBOEI23051660.0071550.110.0029170.0310.29 UBH1900960000.1190.500.04910.160.33 26/40 problems solved faster (20 if take into account perm time)

19. Case C=I Explicit G=IImplicit G 22 =I namenmfact.iter.totalfact.iter.totalratio CVXQP11000050001.9823.410.062141.020.30 CVXQP21000025000.04820.160.030110.332.10 CVXQP31000075006.73210.50.08381.090.10 POWELL201000050000.08120.170.04020.130.77 PRIMALC123990.004510.00800.002410.00440.55 PRIMALC223870.003710.00690.002110.00380.56 QPNBOEI17263510.029260.270.0040170.0770.28 QPNBOEI23051660.008060.0310.0027140.0321.03 UBH1900960000.1220.260.04320.220.83 29/40 problems solved faster (25 if take into account perm time)

20. Conclusions New method for constructing preconditioners for CG methods for a variety of important problems Preconditioners –implicitly respect crucial structure –easy to apply –flexible and capable of improving eigenvalue clusters Extend the class of problems for which CG is applicable Still under development…but will be available as part of GALAHAD Open questions How to pick basis B 1 efficiently and so as to improve eigenvalue clusters How to approximate blocks of H in G What about Mr Greedy?

22. Conclusions New method for constructing preconditioners for CG methods for a variety of important problems Preconditioners –implicitly respect crucial structure –easy to apply –flexible and capable of improving eigenvalue clusters Extend the class of problems for which CG is applicable Still under development…but will be available as part of GALAHAD Open questions How to pick basis B 1 efficiently and so as to improve eigenvalue clusters How to approximate blocks of H in G