1. INFOMRS 2011 @ Charlotte1 Parallel Computation for SDPs Focusing on the Sparsity of Schur Complements Matrices Makoto Yamashita @ Tokyo Tech Katsuki Fujisawa @ Chuo Univ Mituhiro Fukuda @ Tokyo Tech Kazuhide Nakata @ Tokyo Tech Maho Nakata @ RIKEN INFORMS Annual Meeting @ Charlotte 2011/11/15 (2011/11/13-2011/11/16)

2. INFOMRS 2011 @ Charlotte2 Key phrase  SDPARA: The fastest solver for large SDPs available at http://sdpa.sf.net/ SemiDefinite Programming Algorithm paRAllel veresion

3. INFOMRS 2011 @ Charlotte3 SDPA Online Solver 1.Log-in the online solver 2.Upload your problem 3.Push ’ Execute ’ button 4.Receive the result via Web/Mail http://sdpa.sf.net/ ⇒ Online Solver

4. INFOMRS 2011 @ Charlotte4 Outline 1.SDP applications 2.Standard form and Primal-Dual Interior-Point Methods 3.Inside of SDPARA 4.Numerical Results 5.Conclusion

5. INFOMRS 2011 @ Charlotte5 SDP Applications 1.Control theory  Against swing, we want to keep stability.  Stability Condition ⇒ Lyapnov Condition ⇒ SDP

6. INFOMRS 2011 @ Charlotte6  Ground state energy  Locate electrons  Schrodinger Equation ⇒ Reduced Density Matrix ⇒ SDP SDP Applications 2. Quantum Chemistry

7. INFOMRS 2011 @ Charlotte7 SDP Applications 3. Sensor Network Localization  Distance Information ⇒ Sensor Locations  Protein Structure

8. INFOMRS 2011 @ Charlotte8 Standard form  The variables are  Inner Product is  The size is roughly determined by Our target

9. INFOMRS 2011 @ Charlotte9 Primal-Dual Interior-Point Methods Feasible region Optimal Central Path Target

10. INFOMRS 2011 @ Charlotte10 Schur Complement Matrix where Schur Complement Equation Schur Complement Matrix 1. ELEMENTS (Evaluation of SCM) 2. CHOLESKY (Cholesky factorization of SCM)

11. INFOMRS 2011 @ Charlotte11 Computation time on single processor  SDPARA replaces these bottleneks by parallel computation ControlPOP ELEMENTS22228668 CHOLESKY15931992 Total239862713 Time unit is second, SDPA 7, Xeon 5460 (3.16GHz)

12. INFOMRS 2011 @ Charlotte12 Dense & Sparse SCM SDPARA can select Dense or Sparse automatically. Fully dense SCM (100%) Quantum Chemistry Sparse SCM (9.26%) POP

13. INFOMRS 2011 @ Charlotte13 Different Approaches DenseSparse ELEMENTSRow-wise distribution Formula-cost-based distribution CHOLESKYParallel dense Cholesky (Scalapack) Parallel sparse Cholesky (MUMPS)

14. INFOMRS 2011 @ Charlotte14 Three formulas for ELEMENTS densesparse All rows are independent.

15. INFOMRS 2011 @ Charlotte15 Row-wise distribution  Assign servers in a cyclic manner  Simple idea ⇒ Very EFFICINENT  High scalability Server1 Server2 Server3 Server2 Server3 Server4 Server1 Server4

16. INFOMRS 2011 @ Charlotte16 Numerical Results on Dense SCM  Quantum Chemistry (m=7230, SCM=100%), middle size  SDPARA 7.3.1, Xeon X5460, 3.16GHz x2, 48GB memory ELEMENTS 15x speedup Total 13x speedup Very fast!!

17. INFOMRS 2011 @ Charlotte17 Drawback of Row-wise to Sparse SCM densesparse  Simple row-wise is ineffective for sparse SCM  We estimate cost of each element

18. INFOMRS 2011 @ Charlotte18 Formula-cost-based distribution 150403020 13520 7010 505 30 3 Server1190 Server2185 Server3188 Good load-balance

19. INFOMRS 2011 @ Charlotte19 Numerical Results on Sparse SCM  Control Theory (m=109,246, SCM=4.39%), middle size  SDPARA 7.3.1, Xeon X5460, 3.16GHz x2, 48GB memory ELEMENTS 13x speedup CHOLESKY 4.7xspeedup Total 5x speedup

20. INFOMRS 2011 @ Charlotte20 Comparison with PCSDP by SDP with Dense SCM  developed by Ivanov & de Klerk Servers124816 PCSDP53768278541427379954050 SDPARA598320021680901565 Time unit is second SDP: B.2P Quantum Chemistry (m = 7230, SCM = 100%) Xeon X5460, 3.16GHz x2, 48GB memory SDPARA is 8x faster by MPI & Multi-Threading

21. INFOMRS 2011 @ Charlotte21 Comparison with PCSDP by SDP with Sparse SCM  SDPARA handles SCM as sparse  Only SDPARA can solve this size #sensors 1,000 (m=16450; density=1.23%) #Servers124816 PCSDPO.M.1527887591368 SDPARA28.222.116.713.827.3 #sensors 35,000 (m=527096; density=6.53 × 10−3%) #Servers124816 PCSDPOut of Memory SDPARA1080845614540506

22. INFOMRS 2011 @ Charlotte22 Extremely Large-Scale SDPs  16 Servers [Xeon X5670(2.93GHz), 128GB Memory] mSCMtime Esc32_b(QAP)198,432100%129,186 second (1.5days) Other solvers can handle only The LARGEST solved SDP in the world

23. INFOMRS 2011 @ Charlotte23 Conclusion  Row-wise & Formula-cost-based distribution  parallel Cholesky factorization  SDPARA: The fastest solver for large SDPs  http://sdpa.sf.net/ & Online solver Thank you very much for your attention.