Complexity ©D.Moshkovits 1 Hardness of Approximation

Complexity ©D.Moshkovits 2 Introduction Objectives: –To show several approximation problems are NP-hard Overview: –Reminder: How to show inapproximability? –Probabilistic Checkable Proofs –Hardness of approximation for clique

Complexity ©D.Moshkovits 3 Promise Problems Sometimes you can promise something about the input It doesn’t matter what you say for inputs that do not satisfy the promise I know my graph has clique of size n/4! Does it have a clique of size n/2?

Complexity ©D.Moshkovits 4 Promise Problems & Approximation We’ll see promise problems of a certain type, called gap problems, can be utilized to prove hardness of approximation.

Complexity ©D.Moshkovits 5 Optimization Problems Consider an optimization problem P: instances: x 1,x 2,x 3,… optimization measure feasible solutions all graphs Example: all cliques in that graph the clique’s size (max)

Complexity ©D.Moshkovits 6 Each Instance Has an Optimal Solution OPT x1x1 x2x2 x3x3 x4x4

Complexity ©D.Moshkovits 7 Approximation (Max Version) OPT xixi

Complexity ©D.Moshkovits 8 How To Show Hardness of Approximation? Hardness of distinguishing far off instances  Hardness of approximation OPT AB gap xixi

Complexity ©D.Moshkovits 9 Gap Problems (Max Version) Instance: … Problem: to distinguish between the following two cases: The maximal solution  B The maximal solution ≤ A

Complexity ©D.Moshkovits 10 Formally: Claim: If the [A,B]-gap version of a problem is NP-hard, then that problem is NP-hard to approximate to within factor B/A.

Complexity ©D.Moshkovits 11 Formally: Proof (for maximization): Suppose there is an approximation algorithm that outputs C≤C* so that C*/C ≤ B/A A proper distinguisher: * If C  A, return ‘YES’ * Otherwise return ‘NO’

Complexity ©D.Moshkovits 12 Proof Since C ≥ C*·A/B, (1) If C* > B (the correct answer is ‘YES’), then necessarily C ≥ C*·A/B > B·A/B = A (we answer ‘YES’) (2) If C*≤A (the correct answer is ‘NO’), then necessarily C≤C*≤A (we answer ‘NO’).

Complexity ©D.Moshkovits 13 Translating To Decision Problems To Prove Hardness Optimization ProblemsApproximation Problems Threshold ProblemsGap Problems Is the size of the max clique > ½n? Is the size of the max clique > ¾n or < ¼n?

Complexity ©D.Moshkovits 14 Idea We’ve shown “standard” problems are NP-hard by reductions from 3SAT. We want to prove gap-problems are NP- hard, Why won’t we prove some canonical gap- problem is NP-hard and reduce from it? If a reduction reduces one gap-problem to another we refer to it as gap- preserving

Complexity ©D.Moshkovits 15 Gap-3SAT[  ] Instance: a set of 3-clauses {c 1,…,c m } over variables v 1,…,v n. Problem: to distinguish between the following two cases: There exists an assignment that satisfies all clauses. No assignment can satisfy more than 7/8+  of the clauses.

Complexity ©D.Moshkovits 16 Gap-3SAT: Example ( x 1  x 2  x 3 ) ( x 1   x 2   x 2 ) (  x 1  x 2  x 3 ) (  x 1   x 2   x 2 ) (  x 1  x 2   x 3 ) (  x 3   x 3   x 3 )  = { x 1  F ; x 2  T ; x 3  F } satisfies 5/6 of the clauses

Complexity ©D.Moshkovits 17 Why 7/8? Claim: For any set of clauses with exactly three independent literals, there always exists an assignment that satisfies at least 7/8 clauses.

Complexity ©D.Moshkovits 18 The Probabilistic Method Proof: Consider a random assignment. x1x1 x2x2 x3x3 xnxn...

Complexity ©D.Moshkovits Find the Expectation Let Y i be the random variable indicating the outcome of the i-th clause. For any 1  i  m,  FFFF FFTT FTFT FTTT TFFT TFTT TTFT TTTT

Complexity ©D.Moshkovits Find the Expectation The number of clauses satisfied is a random variable Y=  Y i. By the linearity of the expectation: E[Y] = E[  Y i ] =  E[Y i ] = 7/8m

Complexity ©D.Moshkovits Conclude Existence Thus, there exists an assignment which satisfies at least the expected number of clauses.

Complexity ©D.Moshkovits 22 PCP (Without Proof) Theorem (PCP): For any  >0, Gap-3SAT[  ] is NP-hard. This is tight! Gap-3SAT[0] is polynomial time decidable

Complexity ©D.Moshkovits 23 Why Is It Called PCP? (Probabilistically Checkable Proofs) 3SAT has a polynomial membership proof checkable in polynomial time. x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 x n-3 x n-2 x n-1 xnxn... My formula is satisfiable! Prove it! This assignment satisfies it!

Complexity ©D.Moshkovits 24 Why Is It Called PCP? (Probabilistically Checkable Proofs) …Now our verifier has to check the assignment satisfies all clauses…

Complexity ©D.Moshkovits 25 Why Is It Called PCP? (Probabilistically Checkable Proofs) But gap-3SAT also has a polynomial membership proof checkable in polynomial time. x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 x n-3 x n-2 x n-1 xnxn... My formula is satisfiable! Prove it! This assignment satisfies it!

Complexity ©D.Moshkovits 26 Why Is It Called PCP? (Probabilistically Checkable Proofs) And for gap-3SAT the verifier would be right with high probability, even if he (1)picks at random a constant number of clauses and (2)checks only them In a NO instance of gap-3SAT, 1/8 of the clauses are not satisfied!

Complexity ©D.Moshkovits 27 Why Is It Called PCP? (Probabilistically Checkable Proofs) Since gap-3SAT is NP-hard, All NP problems have probabilistically checkable proofs.

Complexity ©D.Moshkovits 28 Gap Preserving Reductions P1P1 P2P2 YES don’t care NO YES don’t care NO

Complexity ©D.Moshkovits 29 Hardness of Approximation Do the reductions we’ve seen also work for the gap versions (i.e approximation preserving)? We’ll revisit the CLIQUE example.

Complexity ©D.Moshkovits 30 CLIQUE Construction a part for each clause a vertex for each literal edge indicates consistency: one is not the negation of the other

Complexity ©D.Moshkovits 31 Cliques & Truth-Assignments A Clique C  V corresponds to the assignment A:V  {T,F} s.t  C  A(  )=T. An edge between two vertices implies the corresponding literals can be both assigned T. Thus each clique corresponds to a satisfying truth-assignment

Complexity ©D.Moshkovits 32 Gap Preservation If there is an assignment that satisfies all clauses, there is a clique of size m. If there is a clique of size  m (for some 0<  <1) there is an assignment that satisfies at least  of the clauses.

Complexity ©D.Moshkovits 33 Gap-CLIQUE (Ver1) The following problem is NP-hard for any  >0: Instance: a graph G=(V,E) composed of m independent sets of size 3. Problem: to distinguish between: There’s a clique of size m=|V|/3 Every clique is of size at most (7/8+  )m

Complexity ©D.Moshkovits 34 Corollary Theorem: for any  >0, CLIQUE is hard to approximate within a factor of 1/(7/8+  )

Complexity ©D.Moshkovits 35 Can We Do Better? The bigger the gap is, the better the hardness result. We’ll see an improved result for CLIQUE.

Complexity ©D.Moshkovits Amplification A part for every k clauses vertex for each satisfying assignment to the k clauses edge indicates consistency Given an instance of the Gap-3SAT problem and a constant k (to be determined later):

Complexity ©D.Moshkovits 37 Boolean Assignments Each clause has at most 7 satisfying assignments. Thus k clauses have at most 7 k satisfying assignments.  FFFF FFTT FTFT FTTT TFFT TFTT TTFT TTTT

Complexity ©D.Moshkovits 38 Consistency Two assignments are inconsistent, if they give the same variable different truth-values. x   y   z xyzwxyzw FFT  x  w   y FTT

Complexity ©D.Moshkovits 39 The Graph G  = Given  = {C 1, …, C m } over variables y 1, …, y n denote Y(C 1, …, C k ) the set of variables which appear in C 1, …, C k Vertices Edges between every two consistent assignments

Complexity ©D.Moshkovits 40 Cliques & Assignments Observation: A clique on  of the parts corresponds to an assignment which satisfies all relevant clauses

Complexity ©D.Moshkovits 41 Correctness (1) If there is a satisfying assignment, then picking the corresponding assignment in each of the parts yields a clique of size read: “m choose k” i.e. m!/k!(m-k)!

Complexity ©D.Moshkovits 42 Observation Fix an assignment. If 1/8 of the clauses are false, then only (7/8) k of the sets of k clauses are satisfiable.

Complexity ©D.Moshkovits 43 Correctness (2) For any 0<  <1, set k so (7/8+  ) k <  If there is a clique with representatives in ≥  of the parts There is an assignment satisfying ≥  fraction of the k-tuples of clauses Ruling out the NO case, in which no assignment satisfies more than 1/8-  of the clauses.

Complexity ©D.Moshkovits 44 Gap-CLIQUE (Ver2) The following problem is NP-hard for any 0<  <1: Instance: a graph G=(V,E) composed of m independent sets of size r. Problem: to distinguish between: There’s a clique of size m = |V|/r Every clique is of size at most  m

Complexity ©D.Moshkovits 45 Corollary Theorem: MAX-CLIQUE is NP-hard to approximate to within any constant factor.

Complexity ©D.Moshkovits 46 Chromatic Number Instance: a graph G=(V,E). Problem: To minimize , so that there exists a function f:V  {1,…,  }, for which (u,v)  E  f(u)  f(v)

Complexity ©D.Moshkovits 47 Chromatic Number

Complexity ©D.Moshkovits 48 Chromatic Number Observation: Each color class is an independent set

Complexity ©D.Moshkovits 49 Clique Cover Number (CCN) Instance: a graph G=(V,E). Problem: To minimize , so that there exists a function f:V  {1,…,  }, for which (u,v)  E  f(u)=f(v)

Complexity ©D.Moshkovits 50 Clique Cover Number (CCN)

Complexity ©D.Moshkovits 51 Observation Claim: The CCN problem on graph G is the CHROMATIC-NUMBER problem of the complement graph G c.

Complexity ©D.Moshkovits 52 Reduction Idea CLIQUECCN q same under cyclic shift clique preserving m GG’

Complexity ©D.Moshkovits 53 Correctness Given such transformation: –MAX-CLIQUE(G) = m  CCN(G’) = q –MAX-CLIQUE(G) q/  to be determined later

Complexity ©D.Moshkovits 54 Transformation T:V  [q] for any v 1,v 2,v 3,v 4,v 5,v 6, T(v 1 )+T(v 2 )+T(v 3 )  T(v 4 )+T(v 5 )+T(v 6 ) (mod q) {v 1,v 2,v 3 }={v 4,v 5,v 6 } T is unique for triplets

Complexity ©D.Moshkovits 55 Observations Such T is unique for pairs and for single vertices as well: If T(x)+T(u)=T(v)+T(w) (mod q), then {x,u}={v,w} If T(x)=T(y) (mod q), then x=y T is unique for: triplets, pairs and single vertex

Complexity ©D.Moshkovits 56 Vertices of CCN CLIQUECCN q m GG’... Each vertex v  G clique in layer i is mapped to vertex T(v)  G CCN in layer i.

Complexity ©D.Moshkovits 57 Using the Transformation 01234… (q-1) vivi vjvj T(v i )=1 T(v j )=4 CLIQUE CCN G CCN has m layers each of 0….q-1 vertices.

Complexity ©D.Moshkovits 58 Edges of CCN (1) T(s) T(t) (s,t)  E CLIQUE  (T(s),T(t))  E CCN

Complexity ©D.Moshkovits 59 Edges of CCN (2) Closing edges under cyclic shift For every (x,y)  E, s.t. x  layer_iand y  layer_j if x’-y’=x-y (mod q), and x’  layer_iand y’  layer_j Then (x’,y’)  E

Complexity ©D.Moshkovits 60 Completing the CCN Graph Construction GG’... CLIQUECCN... Y=T(t)=4 y’-x’=y-x (mod q) x’ Y’ x=T(s)=3

Complexity ©D.Moshkovits 61 Edge Origin Unique T(s) T(t) First Observation: This edge comes only from (s,t) (x,y)  E’ iff  (s,t)  E r  {0..q-1} s.t x=T(s)+r1 y=T(b)+r1

Complexity ©D.Moshkovits 62 Triangle Consistency Second Observation: A triangle only comes from a triangle. x z y a c b CLIQUECCN T(a)=x T(b)=y T(c)=z

Complexity ©D.Moshkovits 63 Proof - Triangle Consistency claim: is a triangle in G clique iff is a triangle in G CCN. Proof  ): (  trivial- by def. of T) (x,y)  E’ iff  (a 1,b 1 )  E r 1  [q] s.t x=T(a 1 )+r 1 y=T(b 1 )+r 1 (y,z)  E’ iff  (b 2,c 2 )  E r 2  [q] s.t y=T(b 2 )+r 2 z=T(c 2 )+r 2 (z,x)  E’ iff  (c 3,a 3 )  E r 3  [q] s.t z=T(c 3 )+r 3 x=T(a 3 )+r 3 x+y+z  T(a 1 )+ T(b 2 )+T(c 3 )+r 1 +r 2 +r 3  T(a 3 )+ T(b 1 )+T(c 2 )+r 3 +r 1 +r 2  {a 1, a 2, a 3 } = {b 1, b 2, b 3 } and because {a i }, {b i }, {c i } are from 3 different layers then by the definition of T:  a 1 = b 3, a 2 =b 1, a 3 =b 2 and r 1 = r 2 =r 3  x z y a c b

Complexity ©D.Moshkovits 64 Clique Preservation Corollary: {c 1,…,c k } is a clique in the CLIQUE graph iff {T(c 1 )+r,…,T(c k ) +r} r  {0…q-1} are q different cliques in the CCN graph.

Complexity ©D.Moshkovits 65 What did we get by now? –MAX-CLIQUE(G) = m –  CCN(G’) has q distinct cliques of size m  CCN(G’) = q –MAX-CLIQUE(G) <  m –  CCN(G’) > q/  By the last corollary

Complexity ©D.Moshkovits 66 What Remains? It remains to show how to construct the transformation T in polynomial time.

Complexity ©D.Moshkovits 67 feasible values Greedy Construction v6v6 v2v2 v1v1 v5v5 v3v3 v4v4 vertices we determined forbidden values v2v2 v6v6 v1v1 v2v2 v6v6 v5v5 v6v6 v3v3 v6v6 v4v4 v6v6 v1v1 v6v6

Complexity ©D.Moshkovits 68 Greedy Construction - Analysis At most values are ruled out totally (choose a triple and a couple), so for q=n 5 the greedy construction works. Corollary: There exists a polynomial time algorithm which constructs a triplet unique transformation with q=n 5

Complexity ©D.Moshkovits 69 Corollaries Theorem: CCN is NP-hard to approximate within any constant factor. Theorem: CHROMATIC-NUMBER is NP-hard to approximate within any constant factor.

Complexity ©D.Moshkovits 70 Summary We saw how to show hardness of approximation and explained the concept of gap problems. We presented the PCP theorem, stating that 3SAT is hard to approximate within some constant factor. 

Complexity ©D.Moshkovits 71 Summary We saw that some of the reductions we know were approximation preserving. That was the case for the 3SAT  p CLIQUE reduction. 

Complexity ©D.Moshkovits 72 Summary However, that reduction gave us a weak result for CLIQUE, So we showed how to amplify it. 

Complexity ©D.Moshkovits 73 Summary Then we introduced a new problem, called CHROMATIC-NUMBER. We reduced gap-CLIQUE to its gap version, showing it was in fact NP- hard to approximate. 