1. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel Computation and Ρ-Completeness Theory by Raymond Greenlaw Jim Hoover Computing Science Department University of Alberta Larry Ruzzo Department of Computer Science University of Washington

2. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 2 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

3. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 3 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

4. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 4 Introduction Sequential computation: Feasible ~ n O(1) time (polynomial time). Parallel computation: Feasible ~ n O(1) time and ~ n O(1) processors.

5. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 5 Introduction Goal of parallel computation: Develop extremely fast ((log n) O(1) ) time algorithms using a reasonable number of processors. Speedup equation: (best sequential time) (number of processors) Allow a polynomial number of processors. (parallel time).

6. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 6 Introduction Roughly, A problem is feasible if it can be solved by a parallel algorithm with worst case time and processor complexity n O(1). A problem is feasible highly parallel if it can be solved by an algorithm with worst case time complexity (log n) O(1) and processor complexity n O(1). A problem is inherently sequential if it is feasible but has no feasible highly parallel algorithm for its solution.

7. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 7 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

8. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 8 Parallel Models of Computation Parallel Random Access Machine Model Boolean Circuit Model Circuits and PRAMs

9. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 9 Parallel Random Access Machine RAM Processors Global Memory Cells P0P0 P1P1 P2P2 C0C0 C1C1 C2C2

10. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 10 Boolean Circuit Model 1 01 1 0 ANDOR AND OR NOT

11. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 11 Circuits and PRAMS Theorem: A function f from {0,1}* to {0,1}* can be computed by a logarithmic space uniform Boolean circuit family { n } with depth ( n ) є (logn) O(1) and size ( n ) є n O(1) if and only if f can be computed by a CREW-PRAM M on inputs of length n in time t(n) є (logn) O(1) using p(n) є n O(1).

12. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 12 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

13. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 13 Basic Complexity Decision, Function, and Search Problems Complexity Classes Reducibility Completeness

14. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 14 Decision, Function, and Search Problems 44 44 33366 Spanning Tree-D Given: An undirected graph G = (V,E) with weights from N labeling edges in E and a natural number k. Problem: Is there a spanning tree of G with cost less than or equal to k ? Spanning Tree-F Given: Same (no k ). Problem: Compute the weight of a minimum cost spanning tree. Spanning Tree-S Given: Same. Problem: Find a minimum cost spanning tree.

15. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 15 Complexity Classes Definitions: P is the set of all languages L that are decidable in sequential time n O(1). NC is the set of all languages L that are decidable in parallel time (logn) O(1) and processors n O(1). FP is the set of all functions from {0,1}* to {0,1}* that are computable in sequential time n O(1). FNC is the set of all functions from {0,1}* to {0,1}* that are computable in parallel time (logn) O(1) and processors n O(1). NC k, k 1, is the set of all languages L such that L is recognized by a uniform Boolean circuit family { n } with size( n ) = n O(1) and depth ( n ) = O((logn) k ).

16. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 16 Definitions: A language L is many-one reducible to a language L, written L P L, if there is a function f such that x є L if and only if f(x) є L. L is P many-one reducible to L, written L P L, if the function f is in FP. For k 1, L is NC k many-one reducible to L, written L NC k L, if the function f is in FNC k. L is NC many-one reducible to L, written L NC L, if the function f is in FNC. Many-One Reducibility m m m m

17. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 17 Many-One Reducibility m Lemma: The reducibilities m, P, NC k (k 1), and NC are transitive. –If L є P and L P L, L NC k L (k > 1), or L NC L then L є P. –If L є NC k (k > 1), and L NC k L then L є NC k. –If L є NC and L NC L then L є NC. m m m m m m m

18. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 18 Turing Reducibility Definition: L is NC Turing reducible to L, written L NC L, if and only if the L-oracle PRAM on inputs of length n uses time (logn) O(1) and processors n O(1). Lemma: The reducibility NC is transitive. –If L NC L then L NC L. –If L NC L and L є NC then L є NC. –If L NC L and L є P then L є P. T T m T T T

19. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 19 Completeness Definitions: A language L is P-hard under NC reducibility if L NC L for every L є P. A language L is P-complete under NC reducibility if L є P and L is P-hard. Theorem: If any P-complete problem is in NC then NC equals P. T

20. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 20 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

21. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 21 Evidence That NC P General Simulations Are Not Fast Fast Simulations Are Not General Natural Approaches Provably Fail

22. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 22 Evidence That NC P Gaps in Simulations Model Resource 1 Resource 2 Max R2 C NC Min R2 C P DTM DTM ATM PDA UC PRAM Time = n O(1) 2 Space = n O(1) 2 Space = n O(1) Size = n O(1) Procs = n O(1) Space Reversals log(Treesize) log(Time) Depth Time log n (log n) O(1) (log n) O(1) (log n) O(1) (log n) O(1) (log n) O(1) n O(1) n O(1) n O(1)

23. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 23 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

24. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 24 P-Complete Problems There are approximately 175 P-complete problems (500 with variations). Categories: –Circuit complexity –Graph theory –Searching graphs –Combinatorial optimization and flow –Local optimality –Logic –Formal languages –Algebra –Geometry –Real analysis –Games –Miscellaneous

25. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 25 Circuit Value Problem Given: An encoding of a Boolean circuit, inputs x 1,…,x n, and a designated output y. Problem: Is output y of TRUE on input x 1,…,x n ? Theorem: [Ladner 75] The Circuit Value Problem is P-complete under NC 1 reductions. m

26. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 26 P-Complete Variations of CVP –Topologically Ordered [Folklore] –Monotone [Goldschlager 77] –Alternating Monotone Fanin 2, Fanout 2 [Folklore] –NAND [Folklore] –Topologically Ordered NOR [Folklore] –Synchronous Alternating Monotone Fanout 2 CVP [Greenlaw, Hoover, and Ruzzo 87] –Planar [Goldschlager 77]

27. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 27 NAND Circuit Value Problem Given: An encoding of a Boolean circuit that consists solely of NAND gates, inputs x 1,…,x n, and a designated output y. Problem : Is output y of TRUE on input x 1,…,x n ? Theorem: The NAND Circuit Value Problem is P-complete.

28. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 28 NAND Circuit Value Problem Proof: Reduce AM2CVP to NAND CVP. Complement all inputs. Relabel all gates as NAND. 01001 1 1 OR AND OR 1 1 10110 1 1 NAND 1 0

29. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 29 Graph Theory –Lexicographically First Maximal Independent Set [Cook 85] –Lexicographically First ( + 1)-Vertex Coloring [Luby 84] –High Degree Subgraph [Anderson and Mayr 84] –Nearest Neighbor Traveling Salesman Heuristic [Kindervater, Lenstra, and Shmoys 89]

30. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 30 Lexicographically First Maximal Independent Set Theorem: [Cook 85] LFMIS is P-complete. Proof: Reduce TopNOR CVP to LFMIS. Add new vertex 0. Connect to all false inputs. 12345 NOR 9 8 7 6 11000 0 1 12345 67 0 8 9 0 0 X X X X

31. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 31 Searching Graphs –Lexicographically First Depth-First Search Ordering [Reif 85] –Stack Breadth-First Search [Greenlaw 92] –Breadth-Depth Search [Greenlaw 93]

32. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 32 Context-Free Grammar Empty Given: A context-free grammar G=(N,T,P,S). Problem: Is L(G) empty? Theorem: [Jones and Laaser 76], [Goldschlager 81], [Tompa 91] CFGempty is P-complete. Proof: Reduce Monotone CVP to CFGempty. Given construct G=(N,T,P,S) with N, T, S, and P as follows:

33. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 33 Context-Free Grammar Empty N = {i | v i is a vertex in } T = {a } S = n, where v n is the output of. P as follows: 1. For input v i, i a i f value of v i is 1, 2. i jk i f v i v j Λ v k, and 3. i j | k if v i v j V v k. Then the value of v i is 1 if and only if i, where є {a } +. *

34. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 34 CFGempty Example x 1 = 0, x 2 = 0, x 3 = 1, and x 4 = 1. G = (N, T, S, P), where N = {1, 2, 3, 4, 5, 6, 7 } T = {a } S = 7 P = {3 a, 4 a, 5 1 | 2, 6 34, 7 56} 1234 56 7 x3x3 x4x4 x1x1 x2x2 OR AND

35. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 35 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

36. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 36 Comparator Circuit Class Definition: [Cook 82] Comparator Circuit Value Problem (CCVP) Given: An encoding of a circuit composed of comparator gates, plus inputs x 1,…,x n, and a designated output y. Problem: Is output y of TRUE on input x 1,…,x n ? Definition: CC is the class of languages NC reducible to CCVP.

37. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 37 CC-Complete Problems –Lexicographically First Maximal Matching [Cook 82] –Telephone Connection [Ramachandran and Wang 91] –Stable Marriage [Mayr and Subramanian 92]

38. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 38 Outline Introduction Parallel Models of Computation Basic Complexity Evidence that NC Ρ Ρ-Complete Problems Comparator Circuit Class Open Problems

39. An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 39 Open Problems Find an NC algorithm or classify as P-complete. –Edge Ranking –Edge-Weighted Matching –Restricted Lexicographically First Maximal Independent Set –Integer Greatest Common Divisor –Modular Powering