Prabhas Chongstitvatana1 NP-Complete What is an algorithm What is a proof What is a hard problem NP-Complete problems -- practical problems that are strongly suspected to be computationally hard.

Prabhas Chongstitvatana2 Computability theory : When can this be computed? Can this be computed at all? Complexity theory : How cheaply can this be computed? How hard is this to compute? Inconsistencies 1900 Georg Cantor, invented set theory.

Prabhas Chongstitvatana3 August, 1900, The Second International Congress of Mathematicians in Paris. Hilbert, formalist school of mathematic, manipulated symbols by fixed rules without thinking of any possible meaning the symbols could have. Strict formalism, by not letting semantic interpretations influence how or when rules are applied, supposedly guarantees that whenever we prove something we are sure it is correct.

Prabhas Chongstitvatana4 Formal system A formal system is a finite set of assumed truths, called axioms, and a finite set of rule, called inference rules. We can use the rules on the axioms to infer new results called theorems. A proof of a theorem is a demonstration of a sequence of inferred theorems starting only with the axioms and using only the inference rules and ending with the theorem.

Prabhas Chongstitvatana5 One of Hilbert’s problem was to establish that arithmetic was consistent, meaning that, under the accepted rules of arithmetic, it is not possible to generate a contradiction Kurt Godel (24), showed that if any system as "powerful" as arithmetic is consistent, then it cannot be complete -- meaning that some true statements are unprovable in the system!

Prabhas Chongstitvatana6 Today we don't know whether arithmetic is consistent; all we can say is that if it is, then there are true things we cannot prove in it. This is similar to the uncertainty principle in quantum mechanics, Heisenberg Hilbert's continuum problem : Cantor, there is more than one infinity. Using the proof called diagonalization, Cantor showed that there were more real numbers than there were integers. But no one knew if there were an infinity between the number of integers and the number of reals.

Prabhas Chongstitvatana7 r 0 = 0.d 00 d 01 d r 1 = 0.d 10 d 11 d r 2 = 0.d 20 d 21 d r n = 0.d n0 d n1 d n2...d nn... diagonal d 00 d 11 d 22 d nn... change each digit into the "opposite". This will be a new number not in the original listing. | N | < | R | Are there any thing between | N | < | S | < | R | ?

Prabhas Chongstitvatana Godel showed that it couldn't be proved false from Cantor's axioms of set theory Cohen showed that it couldn't be proved true from Cantor's axioms. The continuum hypothesis is independent of the usual axioms of set theory. If set theory is consistent to begin with, then we can assume the continuum hypothesis true or false and still have a consistent set theory!

Prabhas Chongstitvatana9 The formalist school was resisted by a group of mathematicians called constructivists (intuitionists) who wanted to toss infinite sets and rebuild mathematics from the bottom up starting from the integers and applying only finite operations to them.

Prabhas Chongstitvatana10 Model of computation Another Hilbert's problem was to find a "finitary way" to solve any diophantine equation. A diophantine equation is a polynomial equation in n unknowns that can have only integer solutions. Hilbert wanted an "algorithm" (not yet invented the name) 1970 Matijasevic (22) showed, using fibonacci numbers, that this problem is computationally unsolvable.

Prabhas Chongstitvatana11 Turing machine, Alan Turing (24) 1936 Herbrand functions, Jacques Herbrand Godel functions, Kurt Godel Church calculus, Alonzo Church Kleen functions, Stephen Kleen Post systems, Emil Post Markov algorithms, Andrei Markov same computational ability

Prabhas Chongstitvatana12 Hilbert's problem : the decision problem -- asked whether there is an algorithm deciding the truth of falsify of any mathematical statement. Church and Turing, 1936, no such algorithm exists. The result is provable from Godel's work on incompleteness in What it meant for something to be computable.

Prabhas Chongstitvatana13 Traveling salesman problem Knapsack problem Graph coloring problem Are some problems harder than the other? Which ones? How much harder are they? 1965 Edmonds, Cobham, definition of a feasible problem : A problem is feasible if it has a solution whose cost is at most polynomial.

Prabhas Chongstitvatana14 Why polynomial Polynomials are closed under composition and addition All sequential digital computers are polynomially related In general, a polynomial algorithm will do a feasible amount of work.

Prabhas Chongstitvatana Dantzig, algorithm for optimize linear programs, simplex. Simplex worst cost is exponential in the number of variables Hacijan (kah-chi-yan) proved that an algorithm, ellipsoid algorithm, is polynomial. (multidimensional binary search) Karmakar, projective scaling, improved ellipsoid to run well in practice. Karmakar's algorithm is polynomial. Although Simplex is exponential, the linear programming problem is polynomial.

Prabhas Chongstitvatana16 NP-completeness polynomial time algorithm O( n^k ) = tractable superpolynomial time algorithm = intractable

Prabhas Chongstitvatana17 Class of problems NP-complete, status unknown No polynomial time algorithm has yet been discovered for any NP-complete problem. P != NP question (posed 1971) Believe NP-complete problems are intractable

Prabhas Chongstitvatana18 If any single NP-complete problem can be solved in polynomial time, then every NP- complete problem has a polynomial time algorithm. problem -> NP-complete -> approximate algorithm

Prabhas Chongstitvatana19 Define complexity P (formal language) complexity NP : decision problem where solutions can be verified in polynomial time. (more work on this slide)

Prabhas Chongstitvatana20 Abstract problems Binary relations of a set of problem instances to a set of solutions. Theory of NP-completeness restricts to decision problem : yes/no answer. Example Shortest Path Give a graph G = (V,E) and u, v in V, non negative integer k, does a path exist in G between u and v whose length is at least k?

Prabhas Chongstitvatana21 Cast optimization problems to decision problems by imposing a bound on the value to be optimized.

Prabhas Chongstitvatana22 Encoding An encoding of a set S of abstract object is a mapping e from S to the set of binary string. A problem whose instance set is the set of binary strings is called a concrete problem. An algorithm "solves" a concrete problem in O(T(n)) if when it is provided a problem instance i of length n = | i | the algorithm can produce the solution in at most O(T(n)).

Prabhas Chongstitvatana23 Complexity class P; the set of concrete decision problem that are solvable in polynomial time. Encoding map abstract problem Q to concrete problem. If solution to an abstract problem i in I is Q(i) in {0,1}, then the solution to the concrete problem instance e(i) in {0,1}* is also Q(i).

Prabhas Chongstitvatana24 We would like the definition to be independent of any particular encoding. An algorithm with the only input k is running in time Zeta(k) if k is unary ( a string of 1's ) its running time is O(n) if k is binary n = ceil( lg k) its running time is Zeta(2^n)

Prabhas Chongstitvatana25 A function f : {0,1}* -> {0,1}* is polynomial time computable if there exists a polynomial time algorithm A that, given input x in {0,1}*, produces as output f(x). For some set I of problem instances, two encoding e 1 and e 2 are polynomially related if there exist two polynomial time computable functions f 12 and f 21 such that for any i in I, we have f 12 (e 1 (i)) = e 2 (i) and f 21 (e 2 (i)) = e 1 (i).

Prabhas Chongstitvatana26 That is, encoding e 2 (i) can be computed from the encoding e 1 (i) by a polynomial time algorithm, and vice versa. Lemma 36.1 Let Q be an abstract decision problem on an instance set I, and let e 1 and e 2 be polynomially related encodings on I. Then, e 1 (Q) is-in P if and only if e 2 (Q) is-in P.

Prabhas Chongstitvatana27 “Standard” encoding we shall generally assume that problem instances are encoded in any reasonable, concise fashion. Encoding of an integer is polynomially related to its binary representation. Encoding of a finite set is polynomially related to its encoding as a list of elements.

Prabhas Chongstitvatana28 Formal language framework An algorithm A accepts a string x in {0,1}* if, given input x, the algorithm outputs A(x) = 1. The language accepted by an algorithm A is the set L = { x in {0,1}* | A(x) = 1 }. An algorithm rejects a string x if A(x) = 0.

Prabhas Chongstitvatana29 A language L is decided by an algorithm A if every binary string is either accepted or rejected by the algorithm. A language L is accepted in polynomial time by an algorithm A if for any length-n string x in L, the algorithm accepts x in time O(n^k) for some constant k. A language L is decided in polynomial time by an algorithm A if for any length-n x in {0,1}*, the algorithm decides x in time O(n^k) for some constant k.

Prabhas Chongstitvatana30 Complexity class P P = {L subset {0,1}* | there exists an algorithm A that decides L in polynomial time}. Theorem 36.2 P = {L | L is accepted by a polynomial time algorithm}.

Prabhas Chongstitvatana31 Polynomial time verification Hamiltonian cycles Does a graph G have a Hamiltonian cycle? HAM-CYCLE = { | G is hamiltonian grah } How an algorithm decide the language HAM- CYCLE?

Prabhas Chongstitvatana32 Given a problem instance, lists all permutations of the vertices of G and then checks each permutation. With "reasonable" encoding of a graph as its adjacency matrix, the number m of vertices in the graph is Ohmega(sqrt(n)), n = | | is the length of the encoding of G. The running time is Ohmega(m!) = Ohmega(sqrt(n)!) = Ohmega( 2 ^ sqrt n).

Prabhas Chongstitvatana33 Another easier problem : Given a graph G, claims to be hamiltonian with the prove as a list of vertices. It is easy to verify the proof by checking if the given list of vertices is a permutation of the vertices of V and whether each of the consecutive edges along the cycle actually exists in the graph. O(n^2). Thus a proof that a hamiltonian cycle exists in a graph can be verified in polynomial time.

Prabhas Chongstitvatana34 Define a verification algorithm, A, a two- argument algorithm, an input string x and a binary string y called a certificate. A verifies an input x if there exists a certificate y such that A(x,y) = 1. The language verified by a verification algorithm A is L = { x in {0,1}* | there exists y in {0,1}* such that A(x,y) = 1 }.

Prabhas Chongstitvatana35 An algorithm A verifies a language L if for any string x in L, there is a certificate y that A can use to prove that x is-in L.