Presentation on theme: "CSCE350 Algorithms and Data Structure"— Presentation transcript:
1 CSCE350 Algorithms and Data Structure Lecture 18Jianjun HuDepartment of Computer Science and EngineeringUniversity of South Carolina
2 Announcement Thanksgiving………… One final homework (for preparing final exam)60-70
3 Chapter 10: Limitations of Algorithm Power Basic Asymptotic Efficiency Classes1constantlog nlogarithmicnlinearn log nn2quadraticn3cubic2nexponentialn!factorial
4 Polynomial-Time Complexity Polynomial-time complexity: the complexity of an algorithm iswith a fixed degree b>0.If a problem can be solved in polynomial time, it is usually considered to be theoretically tractable in current computers.When an algorithm’s complexity is larger than polynomial, i.e., exponential, theoretically it is considered to be too expensive to be useful – intractableThis may not exactly reflect it’s usability in practice!
5 List of Problems Sorting Searching Shortest paths in a graph Minimum spanning treePrimality testingTraveling salesman problemKnapsack problemChessTowers of Hanoi …These were discussed in the first lecture. At this point, the claims made then about“efficient” algorithms can be discussed in more concrete terms.Before launching into this lecture, I usually read to my students the introductory whimsicalexample from Garey and Johnson “Computers and Intractability”, while showingoverhead transparencies of their cartoons. These are not reproduced here because it wouldrequire copyright clearance.
6 Lower Bound Problem A can be solved by algorithms a1, a2,…, ap Problem B can be solved by algorithms b1, b2,…, bqWe may askWhich algorithm is more efficient? This makes more sense when the compared algorithms solve the same problemWhich problem is more complex? We may compare the complexity of the best algorithm for A and the best algorithm for BFor each problem, we want to know the lower bound: the best possible algorithm’s efficiency for a problem Ω(.)Tight lower bound: we have found an algorithm of the this lower-bound complexity
7 Trivial Lower BoundMany problems need to ‘read’ all the necessary items and write the ‘output’Their sizes provide a trivial lower boundExample:Generate all permutations of n distinct items Ω(n!)Why?Is this a tight lower bound?Evaluate the polynomial at a given x Ω(n), because of the input size. Is this tight?
8 Another Example Multiplying two nxn matrices Ω(n2) because we need to process 2n2 elements and output n2 elementsWe do not know whether this is tightMany trivial lower bounds are too low to be usefulTSP Ω(n2) because its input is n(n-1)/2 intercity distance and output is n+1 city in sequenceThere is no known polynomial-time algorithm to solve itTrivial lower bound sometime have problemsWe do not need to process all the input elementsFor example: searching an element in a sorted array. What is its complexity?
9 Information-Theoretic Arguments This approach seeks to establish a lower bound based on the amount of information it has to produceRecall the problem of guess the number from 1..n by asking ‘yes/no’ questionsFundamentally, it is a coding problem. If the input number can be encoded into m bits, each ‘yes/no’ question just resolve one bit and therefore, the lower bound is mWe know that m=log2nWe have also shown its relation to decision tree. We will apply the decision tree to find the lower bound for several other problems
10 Find the Smallest from three numbers using comparison Leaves in the decision tree is the possible output. The output size is at least 3 (maybe larger than 3) hereComplexity: the height of this decision treeGiven l leaves, the height of the binary tree is at least
11 Decision Tree for Sorting Algorithms The number of leaves: n!The lower bound: log n! n log2n. Is this tight?Selection Sort
12 Decision Tree for Searching a Sorted Array Decision tree for binary search in a four-element array# of leaves for n elements 2n+1 lower bound: Ω(logn)
13 P, NP, and NP-Complete Problems As we discussed, problems that can be solved in polynomial time are usually called tractable and the problems that cannot be solved in polynomial time are called intractable, nowIs there a polynomial-time algorithm that solves the problem?Possible answers:yesnobecause it can be proved that all algorithms take exponential timebecause it can be proved that no algorithm exists at all to solve this problemdon’t knowdon’t know, but if such algorithm were to be found, then it would provide a means of solving many other problems in polynomial time
14 Types of problemsOptimization problem: construct a solution that maximizes or minimizes some objective functionDecision problem: answer yes/no to a questionMany problems will have decision and optimization versions.Eg: Traveling Salesman Problemoptimization: find Hamiltonian cycle of minimum weightdecision: Is there a Hamiltonian cycle of weight < k
15 Some More problemsPartition: Given n positive integers, determine whether it is possible to partition them into two disjoint subsets with the same sumBin packing: given n items whose sizes are positive rational numbers not larger than 1, put them into the smallest number of bins of size 1Graph coloring: For a given graph find its chromatic number, i.e., the smallest number of colors that need to be assigned to the graph’s vertices so that no two adjacent vertices are assigned the same colorCNF satisfiability: Given a boolean expression in conjunctive normal form (conjunction of disjunctions of literals), is there a truth assignment to the variables that makes the expression true?
16 The class PP: the class of decision problems that are solvable in O(p(n)), where p(n) is a polynomial on nWhy polynomial?if not, very inefficientnice closure propertiesmachine independent in a strong sense
17 The class NPNP: the class of decision problems that are solvable in polynomial time on a nondeterministic machine(A determinstic computer is what we know)A nondeterministic computer is one that can “guess” the right answer or solutionThus NP can also be thought of as the class of problemswhose solutions can be verified in polynomial time; orthat can be solved in polynomial time on a machine that can pursue infinitely many paths of the computation in parallelNote that NP stands for “Nondeterministic Polynomial-time”
19 Example: Conjunctive Normal Form (CNF) Satisfiability This problem is in NP. Nondeterministic algorithm:Guess truth assignmentCheck assignment to see if it satisfies CNF formulaExample: (Boolean operation)Truth assignments:Checking phase: Θ(n)
20 Where Are We Now?Exhibited nondeterministic poly-time algorithm for CNF- satisfiabilityCNF-sat is in NPSimilar algorithms can be found for TPS, HC, Partition, etc proving that these problems are also in NPAll the problems in P can also be solved in this manner (but no guessing is necessary), so we have:P ⊆ NPBig question: P = NP ?
21 NP-Complete problems A decision problem D is NP-complete iff D ∈ NP every problem in NP is polynomial-time reducible to DCook’s theorem (1971): CNF-sat is NP-completeOther NP-complete problems obtained through polynomial- time reductions of known NP-complete problemsThe class of NP-complete problems is denoted NPC
22 Polynomial Reductions A decision problem D1 is said to be polynomial reducible to a decision problem D2 if there exists a function t that transforms instances of D1 to instances of D2 such thatt maps all yes instances of D1 to yes instances of D2 and all no instances of D1 to no instances of D2t is computable by a polynomial-time algorithmIf D2 can be solved in polynomial time D1 can be solved in polynomial time
23 Polynomial Reductions Example: Polynomial-time reduction of HC-Hamiltonian cycle to decision version of TPS (TPS with total distance no larger than k?) given integer distanceWhat does this prove?HC is harder or easier than TPS decision?HC is NPC TPS(D) is NPC? orTPS(D) is NPC HC is NPC?y1yxx2121vvuu1Of course, both directed and undirected HC are in NPC.Point out also that undirected HC can easily be reduced to directed HC.This is the difficult direction.Poly-time reduction of directed to undirected shows that directed is no harder than undirected.Thus the answer is the first of the two options given(if NPC for directed then also NPC for undirected)
24 To Prove a Decision Problem is in NPC Prove it is in NP (verification takes polynomial time)Prove that all problems in NP is reducible to this problemorProve that a known NPC problem is reducible to this problemIf we can prove any given NPC problem can be solve in polynomial time P=NPFor us, before solving a problem, we may check whether it is a NPC problem. If it is, then …Michael Garey and David Johnson: Computers and Intractability - A Guide to the Theory of NP-completeness; Freeman, 1979.