Complexity Theory: Foundations

Complexity Theory: Foundations Problem-solving is generally focused on algorithms. Analyse “space” & “time” requirements : best case/ worst case. Approximation algorithms – how close the solution produced is from the optimal solution? ↪ This analysis doesn’t tell us whether the algorithm is best or good. Since we lack reference point => we need results about the complexity of the problem and not just about the complexity of the algorithm that solves it.

Complexity Theory: Foundations Objective: Complexity Theory objective is to establish bounds on the behavior of the best possible algorithms for solving the problem.= (whether or not such algorithms are known). Complexity Theory is an extension of the computability theory. Computability Theory:- what can/cannot be computed in absolute term. Complexity theory:- what can/cannot be computed within reasonable resource bounds (time & space) ↪ difficult problem: complexity bounds are known only for a few problems.

Complexity Theory: Foundations Eg. How can we derive bounds on the behavior of the best algorithm. > Θ (log n) Eg. Searching / Sorting -> Θ (n log n) Eg. ∃ Algorithms with 𝑂 (𝑒 𝑛 ) exponential upper bound. Ω (𝑛) linear lower bound. How can we characterize this problem ? Characterization of the absolute complexity of problems has proved difficult relative complexity can be more successful.

Complexity Theory: Foundations We attempt to show that: - one problem is harder than another by showing that: the best possible algorithm for solving the former requires more time, space, or other resources than the best possible algorithm to solve the latter. OR that all members of a class of problems are of equal difficulty, at least with regard to their asymptotic behavior.

6.1 Reductions In chapter 5 we saw that: reductions among sets provide a tool for studying computability. To study complexity – need to define reductions among problems and to assess or bound the cost of each reduction. Eg. 1) Reduce sorting to – computation of convex hulls in linear time. 2) Reduce marriage problem to a special instance of network-flow problem and solve that instance (and vice-versa) => Sorting a set of number and finding the convex- hull of a set of points in 2D are computationally equivalent to within a linear-time additive term.

6.1 Reductions Example detailed in text: TRAVELING SALESMAN & HAMILTONIAN CIRCUIT No solution that is guaranteed to run in polynomial time is known for either of these 2 problems (TS & HC) However - if we discover a polynomial solution for (TS) => We could immediately construct a polynomial solution for (HC).

6.1 Reductions Given a particular graph of N vertices: Eg) – we transform this instance of the Hamiltonian circuit problem into an instance of the Travelling Salesman (TS) problem ( by associating each vertex of the graph with a city and setting the distance between 2 cities to are: If the corresponding vertices are connected by all edge and to Otherwise If (TS) returns all optimal tour of N (that is the one corresponding to the Hamiltonian circuit) Does not return N for an optimal tour then no Hamiltonian tour exists.

6.1 Reductions This transformation takes Ο( 𝑁 2 ) time. Time for the (HC) – time to run the transformation + - time to run the (TS) problem. We reduced the (HC) problem to the (TS) problem. HC TS. Easier difficult Then If (HC) intractable (TS) intractable HC tractable If TS tractable Reduced to

6.1 Reductions If problem (A) reduces to another problem (B): 𝐴 ⊆ 𝑡 𝐵 If a solution for B can be used to construct a solution for A. Decision problem. Other decision problems see page 172, 173 textbook. Partition problem – given a set of objects and an integer-value size associated with each object, can the set be partitioned into two subsets, such that the sum of the sizes of the elements in one subset is equal to the sum of the sizes of elements in the other subsets? => DECISION problem.

6.1 Reductions Smallest subsets problem: How many different subsets are there such that the sum of the elements of each subset is no longer than “M” (a positive size bound)? ENUMERATION We reduce the partition problem (P) to the smallest subsets (SS) problem. An instance of (P) admits a solution only if the sum of all the sizes is an even number => start computing sum If it is “add ” answer is “no” and stop. Otherwise use procedure to solve smallest subsets with the same set of elements and the same sizes as in the partition problem. Study proof for exam page 173. Read exercise 6.1, 6.2.

6.1 Reductions If we have A≤𝐵 and B≤𝐴, then (to within the coarseness dictated by the type of the reduction the 2 problems may be considered to be of equivalent complexity, which we denote by 𝐴 ≡ 𝑡 𝐵. ≤ 𝑡 relation is reflexive here. ≤ 𝑡 is also transitive (choose only reductions with that property) relation ≤ 𝑡 partial order (this relation can be seen as a partial order) relation ≤ 𝑡 can compare problems in terms of their complexity. To show that problem (A) is strictly more difficult than problem (B), we need to prove that, while problem (B) reduces to problem (A), problem (A) cannot be reduced to problem B. Strictly more difficult than (B) { B reduces to A { A cannot be reduced to B

6.1 Reductions Which reductions to use depends on the type of comparison (the finer the comparison, the more restricted the reduction) We can restrict the resources available to the reduction. Eg 1) If we wish to distinguish only tractable from intractable We want the relation 𝐴≤ 𝑡 𝐵 to mean that: If B is tractable => A is tractable. => The only restriction on “t” is that it requires no more than polynomial time. In particular: since polynomials are closed under composition Given p(), q(), ∃ v(o) | ∀ x, p(q(x)) = v(x) => Procedure for (A) can call upon procedure for (B) a polynomial number of times.

6.1 Reductions Eg 2) If we want to distinguish more specific attribute: between problems requiring cubic time and those requiring only quadratic time, then: 𝐴 ≤ 𝑡 𝐵 implies that A is solvable in quadratic time if B is solvable in quadratic time. The reduction “t” must run in quadratic time overall, so that it can make only a constant number of calls to the procedure for (B). (Eg 1) and (Eg 2) above show that resources needed for reductions depend directly on the complexity classes being compared.

6.1 Reductions We can choose a specific type of reduction: The chosen type establishes a minimum degree of similarity between the two problems: Eg. STRONG similarity {(HC) and (TS) when both are viewed as decision problems. LOOSER similarity { partition vs smallest subsets Important 2 types of reductions : i. Turing (algorithmic reductions) -> apply to any type of problem ii. Many-one reductions (transformations) -> apply only to decision problems. Note: many-one reductions have been applied earlier on sets. Here we use them on decision problems since a decision problem could be viewed as a set of all “yes” instances of the problem.

6.1 Reductions (D) a. A problem (A) TURING reduces to a problem (B), denoted 𝐴≤ 𝑡 𝐵, if ∃ an algorithm for solving (A) that uses an oracle (i.e., a putative solution algorithm) for (B). b. A decision problem (A) many-reduces to a decision problem (B), denoted 𝐴≤ 𝑚 𝐵 if ∃ a mapping. F: ∑ ∗ ⟶ ∑ ∗ , s.t. “yes” instances of (A) are mapped onto “yes” instances of (B) and “no” instances of (A) (as well as meaningless strings) are mapped onto “no” instances of B.

6.1 Reductions -Turing reductions embody a “subroutine” scheme. - Many-one reductions are much stricter and apply only to decision problems. Page 175. Figure A many-one reduction from problem (A) to problem (B)

6.1 Reductions Discussion on page 175. If “f” mapping is injective => “one-reduction” injective + surjective => the 2 problems are isomorphic. We need to also qualify the reduction by the allowable amount of resources used in the reduction: Polynomial-time transformation. Logarithmic-space Turing reduction Eg. (HC) to (TS) is a polynomial-time transformation partition to smallest subsets is a polynomial-time Turing reduction.

6.1.2 Reductions & complexity classes Complexity classes are characterized simply by one or more resource bounds: (D) A complexity class for some model of computation is the set of all decision problems solvable on this model under same given resources bounds. Each complexity class includes all classes of strictly smaller complexity (i.e., class of intractable problems includes the class of tractable problems) Note: A complexity class (=> includes problems of wide variety in their difficulty) differs from all equivalence class (=> includes problems of similar difficulty)

6.1.2 Reductions & complexity classes (Eg) The class of problems solvable in EXPONENTIAL time on a RAM includes searching [O(log n)] of an ordered array, along with much harder problems. In order to characterize hardest problems in a class, we return to the notion of complete problems. ((D)) Given a class of problems Y, and a type of reduction “t”, a problem (A) is complete for Y (or Y- complete) if: A belongs to Y, and Every problem in Y reduces to A under t: ∀ 𝐵 ∈𝑌, 𝐵≤ 𝑡 𝐴 => A is the hardest problem in Y

6.1.2 Reductions & complexity classes Any complete problem must reduce to any other. The set of all complete problems for Y forms an equivalence class under “t”, since each complete problem is supposed to be the hardest in Y, and thus no particular complete problem could be harder than any other. Requiring a problem to be complete for a class is typically a very stringent condition and not every class has complete problems => however, complete problems for certain classes are common.

6.1.2 Reductions & complexity classes (S) If complete problems exist for a complexity class under suitable type of reduction (one that uses fewer resources than are available in the class), then they characterize the boundaries of a complexity class as follows: If any one of the complete problems can be solved efficiently then all of the problems in the class can (be solved efficiently) If a new problem can be shown to be strictly more difficult than some complete problem, then the new problem cannot be a member of the complexity class.

6.1.2 Reductions & complexity classes ((D)) Given a class of problems Y, a reduction t, and a problem (A), complete for Y under t. - A problem 𝐵 1 is hard for Y (or Y-hard) under t if we have 𝐴≤ 𝑡 𝐵 1 , and. A problem 𝐵 2 is easy for Y (or Y-easy) under t if we have 𝐵 2 ≤ 𝑡 A. A problem that is both Y-hard and Y-easy is named Y-equivalent.