1. Heuristics for the Hidden Clique Problem Robert Krauthgamer (IBM Almaden) Joint work with Uri Feige (Weizmann)

2. Heuristics for the Hidden Clique Problem 2 Max-Clique Given a graph on n vertices, find a clique (complete subgraph) of maximum size.  Equivalently, find a maximum stable set (induced empty graph) Not approximable within ratio n 1-  for every fixed  >0, unless NP has randomized polynomial time algorithms [Hastad’96].

3. Heuristics for the Hidden Clique Problem 3 Hidden/planted clique model Suggested by [Jerrum’92], [Kucera’95]: 1. Generate a random graph G n,1/2. Every two vertices connected by edge w/ probability ½. 2. Plant a clique of size k. (kÀ2log n) Placed on k randomly chosen vertices. WHP it is a unique maximum clique. Goal: Efficiently find the maximum clique with high probability (WHP) over inputs.

4. Heuristics for the Hidden Clique Problem 4 Why is this problem interesting? A heuristic is an algorithm that need not always output an optimal solution (no worst- case guarantee). Can evaluate heuristics:  Experimental performance – benchmarks  Performance guarantees – families of inputs on which a heuristic performs well This talk – random and semi-random inputs. Similar in spirit to Smooth Analysis [Spielman-Teng] Average-case hardness [Levin]

5. Heuristics for the Hidden Clique Problem 5 Known results (G n,1/2 + k-clique) Motivation: No planted clique – finding maximum clique in a random graph G n,1/2 [Karp’72] k>(n log n) 1/2 Highest degree vertices[Kucera’95] k =  (n 1/2 ) Spectral properties of the graph [Alon-Krivelevich- Sudakov’98] Semidefinite relaxation (semi-random inputs) [Feige-K.’99] Spectral algorithm[McSherry’01] k = n 1/2 -  Open[Jerrum’92] [Feige-K.,’02]

6. Heuristics for the Hidden Clique Problem 6 Semi-random (sandwich) model 1. Generate two graphs G L µ G H  Both contain the same k-clique  (Inclusion is with respect to edges) 2. Adversary chooses arbitrary graph G * sandwiched in between G L µ G * µ G H.  Can have less structure (e.g., highly irregular) In our case: G H = G n,1/2 + planted clique G L = empty graph + k-clique. Sandwich model suggested by [Feige-Kilian’98] motivated by [Santha-Vazirani’95, Blum-Spencer’95] semi-random models.

7. Heuristics for the Hidden Clique Problem 7 Heuristic for sandwich model Plan: Start with an algorithm for G H Then show the same algorithms works for G *  Unlike the eigenvalue technique of [AKS] Additional trick (from [AKS]):  May assume k ¸ cn 1/2 for a large constant c.  For a small fixed c>0, “guess” O(log 1/c) vertices of the clique (by brute force) and work with (subgraph induced on) their neighbors.

8. Heuristics for the Hidden Clique Problem 8 The Lovasz theta function (G) A relaxation for stable set problem  And thus also for max-clique (abusing notation).  (Stable set = induced subgraph with no edges) Computable in polynomial time  Up to small additive error  By semidefinite programming Worst-case integrality ratio is n 1-o(1) [Feige’95] On random graph, (G n,1/2 ) ' n 1/2 [Juhasz’82]

9. Heuristics for the Hidden Clique Problem 9 Using this relaxation for hidden clique Lemma. WHP (G H ) = k. Proof idea:  As a relaxation, clearly (G H ) ¸ k.  Use a characterization of (G) as a minimization problem (SDP duality): ( G) = min M { 1 (M) : constraints on M }  Make a careful choice of the matrix M to conclude that (G H ) · k.

10. Heuristics for the Hidden Clique Problem 10 Locating the clique Let v be a vertex of G H.  G H -v is also a random graph + planted clique.  Hence, WHP ( G H -v)  (G H ) iff v belongs to the planted clique WHP holds for all vertices simultaneously

11. Heuristics for the Hidden Clique Problem 11 The hidden clique heuristic The algorithm:  Compute S = { v2V : ( G H -v)  (G H ) }  Check that S forms a clique (Actually one SDP computation suffices.) Main Theorem. For k=cn 1/2 WHP  The algorithm finds the planted clique  And gives a certificate it is a maximum clique. Similar approach previously used for min- bisection by [Boppana’87].

12. Heuristics for the Hidden Clique Problem 12 Sandwich model Theta function is monotone with respect to addition/removal of edges.  Recal that G L µ G * µ G H. Hence, k · (G L ) · (G * ) · (G H ) · k. Hence, WHP (G * ) = k.  Whenever the proof works for G H it also works for G *, i.e. an adversary cannot affect the algorithm performance.  WHP

13. Heuristics for the Hidden Clique Problem 13 The [Lovasz-Schrijver’91] relaxations A method for generating stronger relaxations  Works for any 0-1 integer programming relaxation P  Two main flavors: polyhedral N(P) and semidefinite N + (P) A Lift-and-project method:  Add n 2 variables y ij (relaxed quadratic constraints x i 2 =x i )  Then project on the n original variables Can be applied iteratively  N n (P) is a tight relaxation (convex hull of integral solutions) Weak optimization oracle for P implies weak optimization for N(P) and N + (P).  Argument can be iterated any fixed number of times.

14. Heuristics for the Hidden Clique Problem 14 Application to stable set Start with naive relaxation for G=(V,E): FR(G): x i +x j · 1for all ij2E x i ¸ 0for all i2V Lemma [LS’91]: N + (FR) is at least as strong as the theta function. N + -rank = least k such that N + k (FR) is tight. Lemma [LS’91]: N + -rank ·  (G). What is the probable value of N + k (FR)?  Does it lead to improved hidden clique heuristics?

15. Heuristics for the Hidden Clique Problem 15 The probable value of the [LS] relaxations Main Theorem. For random graph G n,1/2 WHP  The value of N + k (FR) is roughly (n/2 k ) 1/2.  And thus the N + -rank is  (log n). Relaxations do not offer heuristic for k=o(n 1/2 )  They do not seem to distinguish G H from G n,1/2.  Not better than “guessing” O(1) clique vertices.

16. Heuristics for the Hidden Clique Problem 16 The upper bound The [LS’91] proof that N + -rank ·  (G) shows:  Each application of N + is at least as strong as “guessing” one vertex in the stable set. After k iterations we are left with a random graph G’ on n’ ' n/2 k vertices  Hence, N + (FR) is at least as strong as theta function and its value is at most O((n’) 1/2 ).

17. Heuristics for the Hidden Clique Problem 17 The lower bound Show iteratively that a “uniform” solution (x i =1/(2 k n) 1/2 for all i) is feasible for N + k (FR)  Follows by definition, given that All degrees in G’ are about their expectation n’/2 All eigenvalues of G’ are >-n 1/2 (as expected) The base case N + (FR) is similar to the theta function

18. Heuristics for the Hidden Clique Problem 18 Related Work [Stephen-Tuncel’99]: N + -rank =  (G) for the line graph of the complete graph K t (a stable set is a perfect matching in K t ) [Cook-Dash’01], [Goemans-Tuncel’01]: There exists a polytope P whose N + -rank is n. (not a stable set relaxation) Question: When does N + k perform better than “guessing” k variables?  The first iteration is exceptional (introduces semidefiniteness)

19. Heuristics for the Hidden Clique Problem 19 Open problems Better Heuristics?  Or evidence it is impossible? Use in approximation algorithms?  For vertex cover, N k (FR) has integrality ratio 2-o(1) [Arora- Bollobas-Lovasz’03]  What about other problems?