Computer Sciences Department1

 Property 1: each node can have up to two successor nodes (children)  The predecessor node of a node is called its parent  The "beginning" node is called the root (no parent)  A node without children is called a leaf  Property 2: a unique path exists from the root to every other node 3 Heap tree (Properties) Computer Sciences Department binary tree (Properties) +  max-heap or min-heap property

 Property 1: each node can have up to two successor nodes (children)  The predecessor node of a node is called its parent  The "beginning" node is called the root (no parent)  A node without children is called a leaf  Property 2: a unique path exists from the root to every other node 4 Binary search tree (Properties) Computer Sciences Department binary tree (Property) +

Advanced Design and Analysis Techniques (15.1, 15.2, 15.3, 15.4 and 15.5) 5Computer Sciences Department

 Problem Formulation  Examples  The Basic Problem  Principle of optimality  Important techniques:  dynamic programming (Chapter 15),  greedy algorithms (Chapter 16) 6 Objectives Computer Sciences Department

 This part covers three important techniques for the design and analysis of efficient algorithms:  dynamic programming (Chapter 15),  greedy algorithms (Chapter 16) 7 Techniques -1 Computer Sciences Department

 Earlier parts have presented other widely applicable techniques, such as  divide-and-conquer,  randomization, and  the solution of recurrences. 8 Techniques - 2 Computer Sciences Department

 Dynamic programming is an optimization approach that transforms a complex problem into a sequence of simpler problems.  It is used in many areas such as Computer science “theory, graphics, AI, systems, ….etc”  Dynamic programming typically applies to optimization problems in which a set of choices must be made in order to arrive at an optimal solution.  Dynamic programming is effective when a given subproblem may arise from more than one partial set of choices; the key technique is to store the solution to each such subproblem in case it should reappear. 9 Dynamic programming Computer Sciences Department

 Like dynamic-programming algorithms, greedy algorithms typically apply to optimization problems in which a set of choices must be made in order to arrive at an optimal solution. The idea of a greedy algorithm is to make each choice in a locally optimal manner. 10 Greedy algorithms Computer Sciences Department

 Dynamic programming, like the divide-and-conquer method, solves problems by combining the solutions to subproblems.  Divide and- conquer algorithms partition the problem into independent subproblems, solve the subproblems recursively, and then combine their solutions to solve the original problem. 11 Dynamic programming -1 Computer Sciences Department

 Dynamic programming is applicable when the subproblems are not independent, that is, when subproblems share subsubproblems.  A dynamic-programming algorithm solves every subsubproblem just once and then saves its answer in a table, thereby avoiding the work of recomputing the answer every time the subsubproblem is encountered. 12 Dynamic programming -2 Computer Sciences Department

 Dynamic programming is typically applied to optimization problems. In such problems there can be many possible solutions. Each solution has a value, and we wish to find a solution with the optimal (minimum or maximum) value. We call such a solution an optimal solution to the problem, as opposed to the optimal solution, since there may be several solutions that achieve the optimal value. 13 Dynamic programming -2 Computer Sciences Department

 The development of a dynamic-programming algorithm can be broken into a sequence of four steps. 1. Characterize the structure of an optimal solution. 2. Recursively define the value of an optimal solution. 3. Compute the value of an optimal solution in a bottom-up fashion. 4. Construct an optimal solution from computed information. 14 The development of a dynamic- programming algorithm Computer Sciences Department

Assembly-line scheduling (self study) Computer Sciences Department Note: lecturer should explain the idea of 15.1, in general (you have many ways to get the solution one of them is an optimal).

16 Step 1: The structure of the fastest way through the factory (self study) Computer Sciences Department Note: lecturer should explain the idea of 15.1, in general (you have many ways to get the solution one of them is optimal).

17 Step 2: A recursive solution (self study) Computer Sciences Department Note: lecturer should explain the idea of 15.1, in general (you have many ways to get the solution one of them is optimal).

18 Step 3: Computing the fastest times (self study) Computer Sciences Department

19 Step 4: Constructing the fastest way through the factory (self study) Computer Sciences Department

Matrix-chain multiplication We can multiply two matrices A and B only if they are compatible: the number of columns of A must equal the number of rows of B. If A is a p × q matrix and B is a q × r matrix, the resulting matrix C is a p × r matrix Computer Sciences Department

21 Counting the number of parenthesizations Computer Sciences Department

 Step 1: The structure of an optimal parenthesization  Step 2: A recursive solution  Step 3: Computing the optimal costs 22Computer Sciences Department

23 Step 3: Computing the optimal costs Computer Sciences Department

24Computer Sciences Department

25 Step 4: Constructing an optimal solution Computer Sciences Department

26 Step 4 (explanation)

Constructing an Optimal Solution Analysis of Algorithms27

Constructing an Optimal Solution Analysis of Algorithms28

Constructing an Optimal Solution Analysis of Algorithms29

Example: Recursive Construction of an Optimal Solution Analysis of Algorithms30

Example: Recursive Construction of an Optimal Solution Analysis of Algorithms31

Example: Recursive Construction of an Optimal Solution Analysis of Algorithms32 return A 6 Optimal Solution = ((A 1 (A 2 A 3 ))((A 4 A 5 )A 6 ))

Elements of dynamic programming (self stady) 15.3 Computer Sciences Department33

 The first step in solving an optimization problem by dynamic programming is to characterize the structure of an optimal solution. Recall that a problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to subproblems. Optimal substructure Computer Sciences Department34

 a good rule of thumb is to try to keep the space as simple as possible, and then to expand it as necessary.  For example:  the space of subproblems that we considered for assembly-line scheduling was the fastest way from entry into the factory through stations S 1, j and S 2, j. This subproblem space worked well, and there was no need to try a more general space of subproblems. Characterize the space of subproblems Computer Sciences Department35

 Optimal substructure varies across problem domains in two ways: 1. how many subproblems are used in an optimal solution to the original problem, and 2. how many choices we have in determining which subproblem(s) to use in an optimal solution. Optimal substructure Computer Sciences Department36

 In assembly-line scheduling, an optimal solution uses just one subproblem, but we must consider two choices in order to determine an optimal solution. To find the fastest way through station S i, j, we use either the fastest way through S 1, j−1 or the fastest way through S 2, j−1 ; whichever we use represents the one subproblem that we must optimally solve. Assembly-line scheduling Computer Sciences Department37

 Informally, the running time of a dynamic- programming algorithm depends on the product of two factors: the number of subproblems overall and how many choices we look at for each subproblem. In assembly-line scheduling, we had Theta (n) subproblems overall, and only two choices to examine for each, yielding a Theta (n) running time. Running time of a dynamic- programming Computer Sciences Department38

 One should be careful not to assume that optimal substructure applies when it does not.  Unweighted shortest path:2 Find a path from u to v consisting of the fewest edges. Such a path must be simple, since removing a cycle from a path produces a path with fewer edges.  Unweighted longest simple path: Find a simple path from u to v consisting of the most edges. We need to include the requirement of simplicity because otherwise we can traverse a cycle as many times as we like to create paths with an arbitrarily large number of edges.  A path is called simple if it does not have any repeated vertices. Be careful Computer Sciences Department39

Not simple Computer Sciences Department40

 Overlapping subproblems  Reconstructing an optimal solution  Memoization  15.4 Longest common subsequence read only Computer Sciences Department41

 Suppose that we are designing a program to translate text from English to French.  For each occurrence of each English word in the text, we need to look up its French equivalent. One way to perform these lookup operations is to build a binary search tree with n English words as keys and French equivalents as satellite data Optimal binary search trees Computer Sciences Department42

Computer Sciences Department43

44Computer Sciences Department

45

Computer Sciences Department46

Computer Sciences Department47

 Dynamic programming is an optimization approach that transforms a complex problem into a sequence of simpler problems.  Dynamic programming is effective when a given subproblem may arise from more than one partial set of choices  Steps of Dynamic programming : 1. Characterize the structure of an optimal solution. 2. Recursively define the value of an optimal solution. 3. Compute the value of an optimal solution in a bottom-up fashion. 4. Construct an optimal solution from computed information.  Dynamic programming is applicable when the subproblems are not independent, that is, when subproblems share subsubproblems.  Principle of optimality:…….. 48Computer Sciences Department