1. Submodularity Reading Group Submodular Function Minimization via Linear Programming M. Pawan Kumar http://www.robots.ox.ac.uk/~oval/

2. Separation and Optimization Submodular Function Minimization Outline

3. Solving Linear Programs s.t. A x ≤ b max x c T x Optimization Feasibility asks if there exists an x such that c T x ≥ K A x ≤ b Optimization via binary search on K Feasible solution For a given K

4. Feasibility via Ellipsoid Method Feasible region of LP

5. Feasibility via Ellipsoid Method Ellipsoid containing feasible region of LP

6. Feasibility via Ellipsoid Method Centroid of ellipsoid

7. Feasibility via Ellipsoid Method Separating hyperplane for centroid

8. Feasibility via Ellipsoid Method Smallest ellipsoid containing “truncated” ellipsoid

9. Feasibility via Ellipsoid Method Centroid of ellipsoid

10. Feasibility via Ellipsoid Method Separating hyperplane for centroid

11. Feasibility via Ellipsoid Method Smallest ellipsoid containing “truncated” ellipsoid

12. Feasibility via Ellipsoid Method Centroid of ellipsoid

13. Feasibility via Ellipsoid Method Terminate when feasible solution is found

14. Separating hyperplane in polynomial time –Check each of the ‘m’ LP constraints in O(n) time New ellipsoid in polynomial time –Shor (1971), Nemirovsky and Yudin (1972) Polynomial iterations (Khachiyan 1979, 1980) –Volume of ellipsoid reduces exponentially Only requires a separation oracle –Constraint matrix A can be very large Ellipsoid Method

15. Separation implies easy optimization What about the reverse? Polymatroids admit greedy optimization Do they allow easy separation? Why are we even interested in this? Optimization vs. Separation

16. Separation and Optimization –Polar Polyhedron –Separation via Optimization –Poly-time Equivalence Submodular Function Minimization Outline

17. Polar Polyhedron Polyhedron P = {x: Ax ≤ b} Polar Polyhedron P* = {y: for all x ∈ P, x T y ≤ 1} Assume 0 is in the interior of P (P*)* = PProof? b > 0 No “loss of generality” as P can be translated

18. P is a subset of (P*)* If x ∈ P, then for all y ∈ P* we have x T y ≤ 1 (P*)* = {z: for all y ∈ P*, z T y ≤ 1} Therefore, x ∈ (P*)*

19. (P*)* is a subset of P Let there be an x ∉ P There must exist a separating hyperplane c T x > d c T z ≤ d, for all z ∈ P Since 0 ∈ interior of P, d > 0 Without loss of generality, d = 1

20. (P*)* is a subset of P Let there be an x ∉ P There must exist a separating hyperplane c T x > 1 c T z ≤ 1, for all z ∈ P c ∈ P* x ∉ (P*)* Why? Hence proved

21. Separation and Optimization –Polar Polyhedron –Separation via Optimization –Poly-time Equivalence Submodular Function Minimization Outline

22. Optimization Problem over P Polyhedron P = {x: Ax ≤ b} max c T x x ∈ P

23. Separation Problem over P* Polar Polyhedron P* = {y: for all x ∈ P, x T y ≤ 1} Given y, return ‘YES’ if y ∈ P* Otherwise, return separating hyperplane

24. Using Optimization for Separation Set c = y max c T x x ∈ P C* = If C* ≤ 1, then return ‘YES’ If C* > 1, then return x* Optimal solution x*

25. Separation and Optimization –Polar Polyhedron –Separation via Optimization –Poly-time Equivalence Submodular Function Minimization Outline

26. Poly-Time Equivalence Optimization on P Separation on P* Polarity Optimization on P* Ellipsoid method Separation on (P*)* = P Polarity Ellipsoid method

27. Separation and Optimization Submodular Function Minimization Outline

28. Submodular Function Minimization min T ⊆ S f(T) We will assume f(null set) = 0 If not, we can add a constant

29. Submodular Function Minimization min T ⊆ S f(T) Brute force search is exponential in |S| We will prove that SFM is easy First, we need two properties

30. Property 1 f(U) = max {x(U) | x ∈ EP f } Set w = v U Proof? f is a submodular function over S

31. Property 2 f is a submodular function over S Define f’(U) = min T ⊆ U f(T) f’ is also submodularProof?

32. Proof Sketch We have to prove the following f’(T) + f’(U) ≥ f’(T ∪ U) + f’(T ∩ U) for all T, U ⊆ S f’(T) = f(X) for some X ⊆ T f’(U) = f(Y) for some Y ⊆ S

33. Proof Sketch f’(T) + f’(U)= f(X) + f(Y) ≥ f(X ∪ Y) + f(X ∩ Y) ≥ f’(T ∪ U) + f’(T ∩ U)

34. Property 2 Continued f is a submodular function over S Define f’(U) = min T ⊆ U f(T) f’ is also submodular EP f’ = { x ∈ EP f, x ≤ 0} Proof?

35. Proof Sketch If x ∈ P, then x(U) ≤ f(U) for all U ⊆ S x(T\U) + x(U) ≤ f(U), for all U ⊆ T ⊆ S Why? Because x ∈ EP f Why? Because x ≤ 0 We can show that { x ∈ EP f, x ≤ 0} = P ⊆ EP f’

36. Proof Sketch We can show that { x ∈ EP f, x ≤ 0} = P ⊇ EP f’ If x ∈ EP f’ then x ∈ EP f For any s ∈ S, x s ≤ 0 Why? Because x(U) ≤ f(U) for all U ⊆ S Why? Because x s ≤ f(null set) = 0

37. Submodular Function Minimization min T ⊆ S f(T) = f’(S) f’(U) = min T ⊆ U f(T) Optimization over EP f is easy = max{x(S) | x ∈ EP f’ } = max{x(S) | x ∈ EP f, x ≤ 0} Separation over EP f is easy Separation over EP f’ is easy Optimization over EP f’ is easy Hence Proved