Download presentation

Presentation is loading. Please wait.

Published byTerry Wyse Modified over 2 years ago

1
Convex Programming Brookes Vision Reading Group

2
Huh? What is convex ??? What is programming ??? What is convex programming ???

3
Huh? What is convex ??? What is programming ??? What is convex programming ???

4
Convex Function f(t x + (1-t) y) <= t f(x) + (1-t) f(y)

5
Convex Function Is a linear function convex ???

6
Convex Set Region above a convex function is a convex set.

7
Convex Set Is the set of all positive semidefinite matrices convex??

8
Huh? What is convex ??? What is programming ??? What is convex programming ???

9
Programming Objective function to be minimized/maximized. Constraints to be satisfied. Example Objective function Constraints

10
Example Feasible region Vertices Objective function Optimal solution

11
Huh? What is convex ??? What is programming ??? What is convex programming ???

12
Convex Programming Convex optimization function Convex feasible region Why is it so important ??? Global optimum can be found in polynomial time. Many practical problems are convex Non-convex problems can be relaxed to convex ones.

13
Convex Programming Convex optimization function Convex feasible region Examples ??? Linear Programming Refer to Vladimir/Pushmeets reading group Second Order Cone Programming What ??? Semidefinite Programming All this sounds Greek and Latin !!!!

14
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP 2 out of 3 is not bad !!!

15
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

16
Second Order Cone || u || < t u - vector of dimension d-1 t - scalar Cone lies in d dimensions Second Order Cone defines a convex set Example: Second Order Cone in 3D x 2 + y 2 <= z 2

17
Hmmm ICE CREAM !!

18
Second Order Cone Programming Minimize f T x Subject to || A i x+ b i || <= c i T x + d i i = 1, …, L Linear Objective Function Affine mapping of SOC Constraints are SOC of n i dimensions Feasible regions are intersections of conic regions

19
Example

20
Why SOCP ?? A more general convex problem than LP – LP SOCP Fast algorithms for finding global optimum – LP - O(n 3 ) – SOCP - O(L 1/2 ) iterations of O(n 2 n i ) Many standard problems are SOCP-able

21
SOCP-able Problems Convex quadratically constrained quadratic programming Sum of norms Maximum of norms Problems with hyperbolic constraints

22
SOCP-able Problems Convex quadratically constrained quadratic programming Sum of norms Maximum of norms Problems with hyperbolic constraints

23
QCQP Minimize x T P 0 x + 2 q 0 T x + r 0 Subject to x T P i x + 2 q i T x + r i P i >= 0 || P 0 1/2 x + P 0 -1/2 x || 2 + r 0 -q 0 T P 0 -1 p 0

24
QCQP Minimize x T P 0 x + 2 q 0 T x + r 0 Subject to x T P i x + 2 q i T x + r i Minimize t Subject to || P 0 1/2 x + P 0 -1/2 x || < = t || P 0 1/2 x + P 0 -1/2 x || < = (r 0 -q 0 T P 0 -1 p 0 ) 1/2

25
SOCP-able Problems Convex quadratically constrained quadratic programming Sum of norms Maximum of norms Problems with hyperbolic constraints

26
Sum of Norms Minimize || F i x + g i || Minimize t i Subject to || F i x + g i || <= t i Special Case: L-1 norm minimization

27
SOCP-able Problems Convex quadratically constrained quadratic programming Sum of norms Maximum of norms Problems with hyperbolic constraints

28
Maximum of Norms Minimize max || F i x + g i || Minimize t Subject to || F i x + g i || <= t Special Case: L-inf norm minimization

29
You werent expecting a question, were you ??

30
SOCP-able Problems Convex quadratically constrained quadratic programming Sum of norms Maximum of norms Problems with hyperbolic constraints

31
Hyperbolic Constraints w 2 <= xy x >= 0, y >= 0 || [2w; x-y] || <= x+y

32
Lets see if everyone was awake !

33
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

34
Semidefinite Programming Minimize C X Subject to A i X = b i X >= 0 Linear Objective Function Linear Constraints Linear Programming on Semidefinite Matrices

35
Why SDP ?? A more general convex problem than SOCP – LP SOCP SDP Generality comes at a cost though – SOCP - O(L 1/2 ) iterations of O(n 2 n i ) – SDP - O((n i ) 1/2 ) iterations of O(n 2 n i 2 ) Many standard problems are SDP-able

36
SDP-able Problems Minimizing the maximum eigenvalue Class separation with ellipsoids

37
SDP-able Problems Minimizing the maximum eigenvalue Class separation with ellipsoids

38
Minimizing the Maximum Eigenvalue Matrix M(z) To find vector z* such that max is minimized. Let max (M(z)) <= n max (M(z)-nI) <= 0 min (nI - M(z)) >= 0 nI - M(z) >= 0

39
Minimizing the Maximum Eigenvalue Matrix M(z) To find vector z* such that max is minimized. Max -n nI - M(z) >= 0

40
SDP-able Problems Minimizing the maximum eigenvalue Class separation with ellipsoids

41

42

43
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

44
Non-Convex Problems Minimize x T Q 0 x + 2q 0 T x + r 0 Subject to x T Q i x + 2q i T x + r i < = 0 Q i >= 0 => Convex Redefine x in homogenous coordinates. y = (1; x) Non-Convex Quadratic Programming Problem !!!

45
Non-Convex Problems Minimize x T Q 0 x + 2q 0 T x + r 0 Subject to x T Q i x + 2q i T x + r i < = 0 Minimize y T M 0 y Subject to y T M i y < = 0 M i = [ r i q i T ; q i Q i ] Lets solve this now !!!

46
Non-Convex Problems Problem is NP-hard. Lets relax the problem to make it convex. Pray !!!

47
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

48
SDP Relaxation Minimize y T M 0 y Subject to y T M i y < = 0 Minimize M 0 Y Subject to M i Y < = 0 Y = yy T Bad Constraint !!!! No donut for you !!!

49
SDP Relaxation Minimize y T M 0 y Subject to y T M i y < = 0 Minimize M 0 Y Subject to M i Y < = 0 Y >= 0 SDP Problem Nothing left to do …. but Pray Note that we have squared the number of variables.

50
Example - Max Cut Graph: G=(V,E) Maximum-Cut

51
Graph: G=(V,E) Maximum-Cut Example - Max Cut - x i = -1 - x i = +1

52
Graph: G=(V,E) Maximum-Cut Example - Max Cut Alright !!! So its an integer programming problem !!! Doesnt look like quadratic programming to me !!!

53
Max Cut as an IQP Max Cut problem can be written as Naah !! Lets get it into the standard quadratic form.

54

55
Max Cut as an IQP Max Cut problem can be written as Naah !! Lets get it into the standard quadratic form.

56
Solving Max Cut using SDP Relaxations To the white board. (You didnt think Ill prepare slides for this, did you??)

57
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

58
SOCP Relaxation Minimize y T M 0 y Subject to y T M i y < = 0 Minimize M 0 Y Subject to M i Y < = 0 Y >= 0 Remember Y = [1 x T ; x X] X - xx T >= 0

59
SOCP Relaxation Say youre given C = { C 1, C 2, … C n } such that C j >= 0 C j (X - xx T ) >= 0 (Ux) T (Ux) <= C j X Wait.. Isnt this a hyperbolic constraint Therefore, its SOCP-able.

60
SOCP Relaxation Minimize y T M 0 y Subject to y T M i y < = 0 Minimize Q 0 X + 2q 0 T x + r 0 Subject to Q i X + 2q i T x + r i < = 0 C j (X - xx T ) >= 0 C j C

61
SOCP Relaxation If C is the infinite set of all semidefinite matrices SOCP Relaxation = SDP Relaxation If C is finite, SOCP relaxation is looser than SDP relaxation. Then why SOCP relaxation ??? Efficiency - Accuracy Tradeoff

62
Choice of C Remember we had squared the number of variables. Lets try to reduce them with our choice of C. For a general problem - Kim and Kojima Using the structure of a specific problem - e.g. Muramatsu and Suzuki for Max Cut

63
Choice of C Minimize c T x Subject to Q i X + 2q i T x + r i < = 0 Q X + 2q T x + r <= 0 Q = n i u i u i T Let 1 >= 2 >= …. k >= 0 >= k+1 >= n

64
Choice of C C = Q + = k i u i u i T Q X + 2q T x + r <= 0 x T Q + x - Q + X <= 0 x T Q + x + k+1 i u i u i T X + 2q T x + r <= 0 zizi

65
Choice of C C = Q + = k i u i u i T x T Q + x + k+1 i z i + 2q T x + r <= 0 u i u i T i = k+1, k+2, … n x T u i u i T x - u i u i T X <= 0 Q X + 2q T x + r <= 0

66
Choice of C C = Q + = k i u i u i T x T Q + x + k+1 i z i + 2q T x + r <= 0 u i u i T i = k+1, k+2, … n x T u i u i T x - z i <= 0 Q X + 2q T x + r <= 0

67
Specific Problem Example - Max Cut e i = [0 0 …. 1 0 …0] u ij = e i + e j v ij = e i - e j C = e i e i T i = 1, …, |V| u ij u ij T (i,j) E v ij v ij T (i,j) E

68
Specific Problem Example - Max Cut Warning: Scary equations to follow.

69
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

70
Outline Convex Optimization – Second Order Cone Programming (SOCP) – Semidefinite Programming (SDP) Non-convex optimization – SDP relaxations – SOCP relaxations Optimization Algorithms –Interior Point Method for SOCP –Interior Point Method for SDP

71
Back to work now !!!

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google