1 CPSC 320: Intermediate Algorithm Design and Analysis July 28, 2014

2 Course Outline Introduction and basic concepts Asymptotic notation Greedy algorithms Graph theory Amortized analysis Recursion Divide-and-conquer algorithms Randomized algorithms Dynamic programming algorithms NP-completeness

3 Dynamic Programming

4 Dynamic Programming Components Analyse the structure of an optimal solution Separate one choice (usually the last) from a subproblem Phrase the value of a choice as a function of the choice and the subproblem Phrase an optimal solution as the value of the best choice Usually a max/min result Implement the calculation of the optimal value Memoization: save optimal values as we compute them Bottom-up: evaluate smaller problems and use them for bigger problems Top-down: evaluate big problem by calling smaller problems recursively and saving result Keep record of the choice made in each level Rebuild the optimal solution from the optimal value result

5 Knapsack Problem

6 Knapsack Algorithm - Complexity

7 Algorithm Strategies - Review Dynamic programming algorithms: Choice is made based on evaluation of all possible results Time and space complexity are usually higher Greedy algorithms: Choice is made based on locally optimal solution Usually faster, but may not result in globally optimal solution Divide and conquer algorithms: Choice of input division is made based on assumption that merging result of subproblems is optimal

8 Global Sequence Alignment Problem Problem: given two sequences, analyse how similar they are Allow both gaps and mismatches Application: Finding suggestions for misspelled words (comparing strings) Comparing files (diff) Analyse if two pieces of DNA match Example: “ocurrance” vs “occurrence” There is a letter “c” missing (gap) An “a” was used instead of an “e” (mismatch) Mismatches may be seen as gaps in both sides “oc-urra-nce” vs “occurr-ence”

9 Formal Definition

10 Finding the Best Alignment

11 Algorithm (Smith-Wasserman)

12 Algorithm (cont.)

13 Longest Common Subsequence

14 Characterizing the LCS

15 Algorithm

16 Algorithm (cont.)

17 NP Complexity

18 Time Complexity for Decision Problems From this point on we analyse time complexity for problems, not algorithms We want to know what is the best possible complexity for the problem Our focus now is on decision problems, not optimization problems Decision problems: Yes/No answer Optimization: “find best”, “find maximum”, “find minimum” We also need to distinguish “finding” and “checking” a solution

19 Time Complexity - Classes

20 Example: Hamiltonian Path Problem: given a graph, is there a path that goes through every node exactly once? Decision problem: answer is yes or no Optimization problem: find a path with minimum cost, etc.; not required Is this problem in NP? Given a path, can we verify that the path is correct in polynomial time? Is this problem in P? Can we solve it in polynomial time?

21 Example: Satisfiability

22 NP Complete Problem Turns out nobody knows! There is no known algorithm that runs in polynomial time There is no proof that such an algorithm doesn’t exist NP complete: set of all decision problems that: Are in NP (solution can be verified in polynomial time) Are at least as hard as any problem in NP NP hard: set of all problems for which the second rule applies Includes non-decision problems (e.g., optimization)

23 Problem Reduction

24 NP Complete

25 Graph Coloring