1. Max Cut using Semidefinite Programming
Algorithms in Action
Max Cut using Semidefinite Programming
Haim Kaplan, Uri Zwick
Tel Aviv University
May 2016 Last updated: June 5, 2016

2. Positive Semidefinite (PSD) Matrices
A symmetric 𝑛×𝑛 matrix 𝐴 is PSD iff:
𝒙 𝑇 𝐴𝒙= 𝑖,𝑗 𝑎 𝑖,𝑗 𝑥 𝑖 𝑥 𝑗 ≥0, for every 𝒙∈ ℝ 𝑛 .
𝐴= 𝐵 𝑇 𝐵, for some 𝑚×𝑛 matrix 𝐵.
All the eigenvalues of 𝐴 are non-negative.
There exist real random variables 𝑍 1 ,…, 𝑍 𝑛 such that 𝔼 𝑍 𝑖 𝑍 𝑗 = 𝑎 𝑖,𝑗 .
Notation: 𝐴≽𝟎 iff 𝐴 is PSD.

3. Positive Semidefinite Programming
max 𝐶 𝑋
s.t. 𝐴 𝑗  𝑋≤ 𝑏 𝑗 , 𝑗∈ 𝑚
𝑋≽0
𝑋, 𝐴 1 ,…, 𝐴 𝑚 ∈ ℝ 𝑛×𝑛 , 𝑏 1 ,…, 𝑏 𝑚 ∈ℝ
𝐴 𝐵= 𝑖,𝑗 𝑎 𝑖,𝑗 𝑏 𝑖,𝑗 (matrix inner product)
𝐴≽0 ((symmetric) positive semidefinite)
⟺ 𝒙 𝑇 𝐴𝒙≥0 for every 𝒙∈ ℝ 𝑛
Can be approximated using multiplicative updates.
Interesting application: Approximation algorithm for MAX CUT

4. Linear Programming Semidefinite Programming
max 𝑐⋅𝒙
s.t. 𝒂 𝑗 ⋅𝒙≤ 𝑏 𝑗
𝒙≥0
max 𝐶 𝑋
s.t. 𝐴 𝑗  𝑋≤ 𝑏 𝑗
𝑋≽0
Can be solved exactly in polynomial time
Can be solved almost exactly in polynomial time

5. Algorithms work well in practice, not only in theory!
LP/SDP algorithms
Simplex method (LP only)
Ellipsoid method
Interior point methods
Algorithms work well in practice, not only in theory!
Even faster approximation algorithms using multiplicative weights updates.

6. Semidefinite Programming (Equivalent formulation – Vector Programming)
max 𝑐 𝑖, 𝑗 ( 𝒗 𝑖 ∙ 𝒗 𝑗 )
s.t 𝑎 𝑖, 𝑗 𝑘 ( 𝒗 𝑖 ∙ 𝒗 𝑗 ) ≤ 𝑏 𝑘
𝒗 𝑖 ∈ ℝ 𝑛
The variables are now the vectors 𝒗 1 , 𝒗 2 , …, 𝒗 𝑛 .
𝑋≽𝟎 iff 𝑋= 𝐵 𝑇 𝐵. If 𝐵=[ 𝒗 1 𝒗 2 … 𝒗 𝑛 ] then 𝑥 𝑖,𝑗 = 𝒗 𝑖 ∙ 𝒗 𝑗 .

7. The MAX CUT problem
Edges may be weighted

8. The MAX CUT problem: motivation
Given: 𝑛 activities, 𝑚 persons.
Each activity can be scheduled either in the morning or in the afternoon.
Each person interested in two activities.
Task: schedule the activities to maximize the number of persons that can enjoy both activities.
If exactly 𝒏/𝟐 of the activities have to be held in the morning, we get MAX BISECTION.

9. A quadratic integer programming formulation of MAX CUT
s.t. 𝑥 𝑖 ∈{−1,+1}
Left side of the cut
Right side of the cut

10. This is an SDP, and hence can be solved in polynomial time.
An SDP Relaxation of MAX CUT [Delorme-Poljak (1993)] [Goemans-Williamson (1995)]
max 𝑤 𝑖,𝑗 1− 𝒗 𝑖 ∙ 𝒗 𝑗 2
s.t. 𝒗 𝑖 ∈ ℝ 𝑛 , 𝒗 𝑖 ∙ 𝒗 𝑖 =1
This is an SDP, and hence can be solved in polynomial time.
The optimal value of the SDP gives an upper bound on the weight of the maximum cut.
Can we use an optimal solution of the relaxation to obtain a heavy cut?

11. An SDP Relaxation of MAX CUT – Geometric intuition
Embed the vertices of the graph on the unit sphere in ℝ 𝑛 such that vertices that are joined by edges are far apart.

12. An SDP Relaxation of MAX CUT – Example: 𝐶 3
2𝜋 3
𝑂𝑃𝑇=2
𝒗 𝑖 ∙ 𝒗 𝑗 = 𝒗 𝑖 𝒗 𝑗 cos 𝜃
𝑂𝑃𝑇 𝑆𝐷𝑃 = = 8 9 =0.888…
cos 2𝜋 3 =− 1 2
𝑆𝐷𝑃=3 1− − = 9 4
Integrality ratio

13. An SDP Relaxation of MAX CUT – Another example: 𝐶 5
𝒗 4
2𝜋 5
𝒗 2
𝒗 1
𝒗 5
𝒗 3
𝑂𝑃𝑇=4
Optimal solution lies in ℝ 2 , even though we are allowed to use ℝ 5 .
𝑂𝑃𝑇 𝑆𝐷𝑃 = = …
cos 4𝜋 5 =−
𝑆𝐷𝑃=
Integrality ratio

14. Random hyperplane rounding [Goemans-Williamson (1995)]
Choose a random hyperplane passing through the origin.
Use the cut defined by the hyperplane!

15. To choose a random hyperplane, choose a random normal vector
Choosing a random hyperplane
To choose a random hyperplane, choose a random normal vector 𝒓
If 𝒓 = 𝑟 1 , 𝑟 2 ,…, 𝑟 𝑛 , 𝑟 1 , 𝑟 2 ,…, 𝑟 𝑛 ∼𝑁(0,1), independent of each other, then the direction of 𝒓 is uniformly distributed over the n-dimensional unit sphere.

16. The probability that two vectors are separated by a random hyperplane
𝜃 𝜋
𝒗 𝑖
ℙ 𝒗 𝑖 ∙𝒓 𝒗 𝑗 ∙𝒓 <0 =
If 𝒗 𝑖 ,𝒗 𝑗 are unit vectors,
𝒗 𝑖 ∙ 𝒗 𝑗 =
cos 𝜃 𝜃
𝒗 𝑗
𝜃= cos −1 ( 𝒗 𝑖 ∙ 𝒗 𝑗 )

17. Analysis of the MAX CUT Algorithm [Goemans-Williamson (1995)]
𝐸𝑋𝑃= 𝑤 𝑖,𝑗 cos −1 ( 𝒗 𝑖 ∙ 𝒗 𝑗 ) 𝜋
Expected weight of the cut obtained using a random hyperplane.
Value of the SDP relaxation, which is an upper bound on the weight of an optimal cut.
𝑆𝐷𝑃= 𝑤 𝑖,𝑗 1− 𝒗 𝑖 ∙ 𝒗 𝑗 2
𝐸𝑋𝑃 𝑆𝐷𝑃 ≥ min −1≤𝑥< 𝜋 cos −1 𝑥 1−𝑥 = min 0≤𝜃<𝜋 2 𝜋 𝜃 1− cos 𝜃 = …
Lower bound on the approximation ratio of the algorithm.

18. 2 𝜋 cos −1 𝑥 1−𝑥

19. 2 𝜋 cos −1 𝑥 1−𝑥
𝑥 ∗ ≃−
𝜃 ∗ = cos −1 𝑥 ∗ ≃ ≃ ∘

20. Yes! Is the analysis tight?
The approximation ratio of the algorithm is 𝛼 𝐺𝑊 = min 0≤𝜃<𝜋 2 𝜋 𝜃 1− cos 𝜃 = …
[Karloff (1999)]
The integrality gap of the SDP relaxation is also 𝛼 𝐺𝑊 .
[Feige-Schechtman (1999)]

21. The MAX CUT problem: status
Problem is NP-hard
Problem is APX-hard (no PTAS unless P=NP)
Best approximation ratio known, without SDP, is only 0.5. (Local search, random cut, …)
With SDP, an approximation ratio of can be obtained! [Goemans-Williamson (1995)]
Getting an approximation ratio of is NP-hard! (PCP theorem, … , [Håstad (1997)])
An approximation ration of is optimal assuming the Unique Games Conjecture [Khot-Kindler-Mossel-O’Donnell (2007)]