2 Assignment #8 Submit in PDF format ONLY !!! Assignment will NOT be accepted in format other than pdf.See and blackboard announcements for places to find software for pdf conversion.engineering (ECJ) labsundergraduate library.There are also websites that let you do this for free e.g.-Word doc files get corrupted in the submission system so they will not be accepted for this assignment.
3 Final Exam Course:E E 312 INTRODUCTION TO PROGRAMMING Instructor:BARBER, KWEDNESDAY, MAY 10, 9-12 NExam Room: JES A121A
4 Algorithm Analysis Estimating the time required for a program Determining a relative computation time among programsUnderstanding the limits of computationThe Halting ProblemClassifying particularly hard problemsNP-Complete problems
6 ComputabilityAs we have seen, typical algorithms run the gamut from O(log n) to O(n3), where problems solved using O(n) or O(log n) are considered relatively easyOn the other end of the spectrum are truly hard problems:Impossible ("undecidable") problemsVery hard (NP-Complete) problems
7 Undecidable Problems [http://www.cgl.uwaterloo.ca/~csk/halt/] An algorithm is a solution to a problem if it correctly provides the appropriate answer to the problem and is guaranteed to always run in a finite amount of time.A problem is decidable if it has a solution.If there is no algorithm that solves the problem in a finite amount of time, the problem is undecidable.
8 Generally stated Halting Problem [wikipedia] In computability theory the halting problem is a decision problem which can be informally stated as follows:Given a description of a program and its initial input, determine whether the program, when executed on this input, ever halts (completes). The alternative is that it runs forever without halting.Alan Turing proved in 1936 that a general algorithm to solve the halting problem for all possible inputs cannot exist. We say that the halting problem is undecidable.
9 Impossible "Undecidable" Problems The Halting Problem Probably the most commonly cited "undecidable" problemProblem: Is it possible for your C compiler to have an extra feature that checks not only syntax errors but also checks for infinite loops?Answer: NO!Intuitive Reason: Such a program might have a hard time checking itself.Suggests the term "recursively undecidable"
10 Informal Proof of the Halting Problem If an infinite loop-checking program could be written, it should be able to check itself.Assume a program called Loop(P)Loop takes as input a program P, which takes itself as input (i.e., P(P) )Loop returns "Yes" if P loops when run on itselfLoop goes into an infinite loop if P terminates when run on itself (instead of returning "No")
11 Informal Proof of the Halting Problem (cont'd) What if P = Loop (i.e., input to Loop is Loop itself) ?By definition, Loop either halts or does not haltHowever, both these possibilities lead to contradictions.
12 Informal Proof of the Halting Problem (cont'd) Suppose P = Loop and we were to run Loop(Loop)Case 1:If P(P) terminates then Loop(P) should go into an infinite loopHowever, then we would have a contradiction:Loop(Loop) would both terminate and go into an infinite loop at the same timeCase 2:If P(P) goes into an infinite loop then Loop(P) should terminateOnce again, Loop(Loop) would go into an infinite loop and terminate at the same timeTherefore, Loop cannot exist.Assume a program called Loop(P)Loop takes as input a program P, which takes itself as input (i.e., P(P) )Loop returns "Yes" if P loops when run on itselfLoop goes into an infinite loop if P terminates when run on itself (instead of returning "No")
13 Another informal proof for the Halting problem Now suppose that someone claims to have found an algorithm halt(p, i) that returnstrue if p describes the name of a program that halts when given as input the string Ifalse otherwise.Construct another program trouble(s) that uses halt as a subroutine:function trouble(string s)if halt(s, s) == falsereturn trueelseloop forever
14 Another informal proof for the Halting problem function trouble(string s)if halt(s, s) == falsereturn trueelseloop foreverAll programs can be represented by strings, there is a string t that represents the program trouble. Does trouble(t) halt?Consider both cases:If trouble(t) halts, it must be because halt(t, t) returned false, but that would mean that trouble(t) should not have halted.If trouble(t) runs forever, it is either because halt itself runs forever, or because it returned true. This would mean either that halt does not work for every valid input, or that trouble(t) should have halted.Either case concludes that halt did not give a correct answer, contrary to the original claim.Since the same reasoning applies to any program that someone might offer as a solution to the halting problem, there can be no solution.
15 Pitfalls in Understanding [wikipedia] Would you ask, “Is there an algorithm that can return a third option for some programs, such as "undecidable" or "would lead to a contradiction." ?This reflects a misunderstanding of decidability.It is easy to construct one algorithm that always answers "halts" and another that always answers "doesn't halt."For any specific program and input, one of these two algorithms answers correctly, even though nobody may know which one.The difficulty of the halting problem lies not in particular programs, but in the requirement that a solution must work for all programs.
18 NP problemsComputational complexity theory is part of the theory of computation dealing with the resources required during computation to solve a given problem.time (how many steps it takes to solve a problem)space (how much memory it takes to solve a problem).the class P consists of all those decision problems that can be solved on a deterministic sequential machine in an amount of time that is polynomial in the size of the input;the class NP consists of all those decision problems whose positive solutions can be verified in polynomial time given the right information, or equivalently, whose solution can be found in polynomial time on a non-deterministic machine.
19 Polynomial Timepolynomial time refers to the computation time of a problem where the time, m(n), is no greater than a polynomial function of the problem size, n.Written mathematically, m(n) = O(nk) where k is a constant (which may depend on the problem).Polynomial time is the smallest time-complexity class on a deterministic machine which is robust in terms of machine model changes, and is the smallest class closed under composition of subproblems.Subclasses of polynomial timeLinear time: O(n)Quadratic time: O(n2)Cubic time: O(n3)
20 Very hard (NP-Complete) problems Problems in class NP (nondeterministic polynomial-time)Not impossible, but still very hardA "Nondeterministic computer" is a theoretical constructA deterministic machine executes an instruction and goes to the next instruction, which is uniqueA nondeterministic machine is free to "guess" the next instruction (and guess very well)Can't build a nondeterministic computer, but still a useful theoretical construct
21 Problems in the Class NP Phrase the problem as a yes/no question. The problem is in NP if, in polynomial time, we can prove that any "yes" instance is correct.We don't have to worry about "no" instances because the program always makes the "right" choice.Includes all problems that have polynomial-time solutions.
22 NP-complete problems [wikipedia] In complexity theory, the NP-complete problems are the most difficult problems in NP ("non-deterministic polynomial time") in the sense that they are the ones most likely not to be in P.The complexity class consisting of all NP-complete problems is sometimes referred to as NP-C.One example of an NP-complete problem is the subset sum problem which is: given a finite set of integers, determine whether any non-empty subset of them sums to zero.A supposed answer is very easy to verify for correctness, but no one knows a significantly faster way to solve the problem than to try every single possible subset, which is very slow.
23 NP-CompleteNP-complete problems are a subset of all problems known to be in NP that are the hardest.An NP-complete problem has the property that any problem in NP can be reduced to it.Reducing a problem P1 to P2Provide a mapping to transform an instance of P1 to P2Solve P2Map the P2 answer back to a P1 answerTherefore, an NP-complete problem can be used as a "subroutine" to solve any problem in NPIf any NP-complete problem can be solved in polynomial time, then every problem in NP must have a polynomial time solution
24 Example NP-Complete Problems Hamiltonian Cycle problemFind an ordering of the vertices in an undirected graph such that each vertex is visited exactly once.
25 Example NP-Complete Problems (cont'd) Traveling SalesmanGiven a complete graph with edge costs, and an integer K, is there a simple cycle that visits all vertices and has total cost ≤ K?
26 Example NP-Complete Problems [wikipedia] The Boolean satisfiability problem (SAT) is a decision problem considered in complexity theory. An instance of the problem is a Boolean expression written using only AND, OR, NOT, variables, and parentheses. The question is: given the expression, is there some assignment of TRUE and FALSE values to the variables that will make the entire expression true?In mathematics, a formula of propositional logic is said to be satisfiable if truth-values can be assigned to its variables in a way that makes the formula true. The class of satisfiable propositional formulas is NP-complete.
27 Example NP-Complete Problems SatisfiabilityTakes as input a boolean expression and asks whether the expression has an assignment to the variable that gives a value of 1.
28 Transforming Hamiltonian Cycle into Traveling Salesman Construct a new graph GTS which has same vertices as GH. For GTS, each edge has weight of 1 if it was in GH, and 2 otherwise. Choose K = |V|.If GH has a Hamiltonian cycle if and only if GTS has a traveling salesman tour of total weight |V|.
29 Conclusion As an engineer developing software, you must… Know the limits of computation when attacking a problem.Is the problem decidable?Is your algorithm decidable?Be able to analyze the efficiency of your algorithms.What is the algorithm's Big-Oh growth with respect to input?Understand the relative difficulty of the problem you're trying to solve by comparing to problems with known difficulty.Can the problem be related to an NP-Complete problem?
30 References"The Stony Brook Algorithm Repository," <Data Structures and Algorithm Analysis in C, Weiss