1. Message Passing for the Coloring Problem: Gallager Meets Alon and Kahale Sonny Ben-Shimon and Dan Vilenchik Tel Aviv University AofA June, 2007 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A

2. June 20th, 2007Message Passing Algorithm for the Coloring Problem 2 LDPC codes Low Density Parity Check Codes – C is a binary linear code (i.e. a linear subspace of GF [ 2 ] n ) – A is a m £ n binary matrix s.t. Ac = 0 for every codeword c in C –Every row of A has a Hamming-weight of at most k (constant) c c’c’ “ noisy channel ”

3. June 20th, 2007Message Passing Algorithm for the Coloring Problem 3 Gallager’s Decoding Algorithm From the matrix A build an auxiliary bipartite graph B [ A ] –Left side consists of the variables x 1, x 2,…, x n –Right side consists of the constraints C 1, C 2,…, C r Its preferred value from the previous round What is should be to satisfy the constraint

4. June 20th, 2007Message Passing Algorithm for the Coloring Problem 4 Gallager’s Decoding Algorithm x1x1 x2x2 x3x3 x4x4 x5x5 C1C1 C2C2 C3C3 C4C4

5. June 20th, 2007Message Passing Algorithm for the Coloring Problem 5 Decoding via Message Passing [SiSp]: Assume B [ A ] is an expander graph and dist( c,c ’) · n/const then Gallager’s decoding algorithm converges

6. June 20th, 2007Message Passing Algorithm for the Coloring Problem 6 Random 3-colorable graphs The planted model G n, p, 3 [Kučera] –Partition the vertex set, V, into 3 color classes of size n/3 each. Denote by  * : V ! { 1,2,3 } the partition –Include each random edge between two distinct color classed with probability p=p(n) [AK] if p(n) ¸ const/n then  * can be recovered w.h.p. in polynomial time

7. June 20th, 2007Message Passing Algorithm for the Coloring Problem 7 Alon and Kahale’s Coloring Algorithm Obtain an initial 3-coloring of the graph (using spectral methods) For i = 1 to log n do: for all v 2 V greedily color v While 9 v 2 V with < const neighbors colored in some other color uncolor v U ½ V - the set of uncolored vertices If there exists a connected component in G[U] of size at least log n – FAIL Else, exhaustively extend the coloring of V n U to G[U] Not necessarily proper

8. June 20th, 2007Message Passing Algorithm for the Coloring Problem 8 Similarities of Decoding and Coloring Two 3 -colorings ,  are at distance t if –They disagree on the color of at least t vertices in every permutation of the color classes –There exists one permutation obtaining equality Given a sampled graph G from G n, p, 3,  : V ! { 1,2,3 } s.t. dist( ,  * ) · n/const we try to recover  * –  in [AK] is given by the spectral method[AK] Not necessarily proper So, maybe we can use Gallager’s algorithm?!Gallager’s algorithm

9. June 20th, 2007Message Passing Algorithm for the Coloring Problem 9 Gallager for Coloring From the graph G build an auxiliary bipartite graph B [ G ] –Left side consists of the variables (vertices) x 1, x 2,…, x n –Right side consists of the constraints (edges) C 1, C 2,…, C r Its most likely color from the previous round What color x i shouldn’t be This is basically what [AK] does![AK]

10. June 20th, 2007Message Passing Algorithm for the Coloring Problem 10 Proof Outline – d-regular case Given a sampled graph G from G n, p, 3 –Assume every vertex has np/3 neighbors in the other color classes Claim: W.h.p. there is no U ½ V s.t. | U | np | U | /10 Let U i ={ v |  i ( v )  * ( v ) } –| U 0 |< n/90 –Assume 2 | U j |>| U j-1 | for the first time

11. June 20th, 2007Message Passing Algorithm for the Coloring Problem 11 Yellow Green Blue Proof Outline – d-regular case y2y2 u y1y1 g1g1 b1b1 g2g2 b2b2 Every u 2 U j has ¸ 2np/9 neighbors in U j-1

12. June 20th, 2007Message Passing Algorithm for the Coloring Problem 12 Proof Outline – d-regular case Let U = U j [ U j-1 –| U |< 1.5 | U j |< n/60 – e(U) ¸ 2np | U j | /9 > np | U | /10 Contradiction! This algorithm converges after log n iterations But if only life was perfect…

13. June 20th, 2007Message Passing Algorithm for the Coloring Problem 13 Proof Outline – Remarks Every vertex is expected to have np/3 neighbors in every other color class –if p ¸ logn/ n then the degrees of vertices will be relatively “close” to the expectation –for p = const/n some vertices can have very low degree (less than 3). One cannot expect to recover their color.

14. June 20th, 2007Message Passing Algorithm for the Coloring Problem 14 Definition of H = Core(G) : maximal subset of vertices s.t. – 8 v 2 H has at least ( 1 - ² ) np/3 H -neighbors in the other color classes – v has at most ² ‘ np neighbors outside of H It follows easily that on the vertices of H, the algorithm converges But, is there such an H ? and is it big? Proof Outline – The Core

15. June 20th, 2007Message Passing Algorithm for the Coloring Problem 15 Proof Outline – Outside The Core Corollary: (1-exp{-  (np)})n vertices are frozen in every proper 3-coloring Only one cluster of exponential size V1V1 V3V3 V2V2 V1V1 V2V2 V3V3

16. June 20th, 2007Message Passing Algorithm for the Coloring Problem 16 Concluding Remarks If the factor graph is even “close” to an expander graph, one can recover most of the coloring (codeword) May be interesting to apply this scheme to general constraint systems

17. June 20th, 2007Message Passing Algorithm for the Coloring Problem 17 Merci…