1. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Design and Analysis of Algorithms Khawaja Mohiuddin Assistant Professor, Department of Computer Sciences Bahria University, Karachi Campus, Contact: khawaja.mohiuddin@bimcs.edu.pk Lecture # 9 – Recursion

2. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Recursion 2 Topics To Cover  Basic algorithms:  Factorial  Fibonacci numbers  Tower of Hanoi  Backtracking Algorithms:  Eight Queens Problem  Knight’s tour

3. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Recursion 3  Recursion occurs when a method calls itself  Recursion can be direct: when method calls itself  Recursion can be indirect: when method calls some other method that then calls the first method  Recursion can be single: when method calls itself once  Recursion can be multiple: when method calls itself multiple times  Recursive algorithms can be confusing because people don’t naturally think recursively. For example:  To paint a fence, you probably would start at one end and start painting until you reach the other. It is less intuitive to think about breaking the fence into left and right halves and then solving the problem by recursively painting each half.

4. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Recursion (contd.) 4  However, some problems are naturally recursive. They have a structure that allows a recursive algorithm to easily keep track of its progress and find a solution. For example:  A tree is recursive by nature, so algorithms that build, draw, and search trees are often recursive.  Once how to use recursion in understood, it can be utilized in solving many programming situations.

5. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms 5  Some problems have naturally recursive solutions.  We will cover following three naturally recursive algorithms: 1. Calculating factorials 2. Finding Fibonacci numbers 3. Solving Tower of Hanoi puzzle  These relatively straightforward algorithms demonstrate important concepts used by recursive algorithms.  Once these are understood, more complicated algorithms can be looked into.

6. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 6  Calculating Factorials  The factorial of a number N is written N! and pronounced “N factorial”  Factorial function can be defined recursively as follows: 0! = 1 N! = N × (N - 1)!  For example, the following equations show how you can use this definition to calculate 3!: 3! = 3 × 2! = 3 × 2 × 1! = 3 × 2 × 1 × 0! = 3 × 2 × 1 × 1  This definition leads to the following simple recursive algorithm: Integer: Factorial(Integer: n) If (n == 0) Then Return 1 Return n * Factorial(n - 1) End Factorial

7. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 7  Calculating Factorials (contd.)  This is a very simple algorithm, but it demonstrates two important features that all recursive algorithms must have: 1. Each time the method executes, it reduces the current problem to a smaller instance of the same problem and then calls itself to solve the smaller problem. 2. The recursion must eventually stop. To prevent this algorithm from going into infinite loop if negative value is passed as input, some programmers change the first statement in the algorithm to: If (n <= 0) Then Return 1  Analyzing the run time performance of recursive algorithms is sometimes tricky, but it is easy for this particular algorithm.  On input N, the Factorial algorithm calls itself N + 1 times to evaluate N!, (N – 1)!, (N – 2)!,..., 0!.  Each call to the algorithm does a small constant amount of work, so the total run time is O(N).

8. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 8  Calculating Factorials (contd.)  Because the algorithm calls itself N + 1 times, the maximum depth of recursion is also O(N).  However, the factorial function grows very quickly, so there’s a practical limit to how big N can be in a normal program.  For example, 20! ≈ 2.4 × 1018, and 21! is too big to fit in a 64-bit-long integer.  If a program never calculates values larger than 20!, the depth of recursion can be only 20, and there should be no problem.  If you really need to calculate larger factorials, you can use other data types that can hold even larger values.  But in those cases, the maximum depth of recursion could cause a problem by exhausting the stack space and crashing the program.

9. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 9  Finding Fibonacci Numbers  The Fibonacci numbers are defined by these equations: Fibonacci(0) = 0 Fibonacci(1) = 1 Fibonacci(n) = Fibonacci(n–1) + Fibonacci(n–2) for n > 1  For example, the first 12 Fibonacci numbers are: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89.  The recursive definition leads to the following recursive algorithm: Integer: Fibonacci(Integer: n) If (n <= 1) Then Return n Return Fibonacci(n - 1) + Fibonacci(n - 2); End Fibonacci

10. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 10  Finding Fibonacci Numbers (contd.)  This recursive algorithm is reasonably easy to understand but is very slow.  For example, to calculate Fibonacci(6) the program must calculate Fibonacci(5) and Fibonacci(4). But before it can calculate Fibonacci(5), the program must calculate Fibonacci(4) and Fibonacci(3). Here Fibonacci(4) is being calculated twice.  As the recursion continues, the same values must be calculated many times.  For large values of N, Fibonacci(N) calculates the same values an enormous number of times, making the program take a very long time.

11. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 11  Finding Fibonacci Numbers (contd.)  Figure below shows the Fibonacci algorithm’s call tree when it evaluates Fibonacci(6).  You can see that the tree is filled with duplicated calls.  For example, Fibonacci(0) is calculated five times, and Fibonacci(1) is calculated eight times.

12. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 12  Finding Fibonacci Numbers (contd.)  Analyzing this algorithm’s run time is trickier than analyzing Factorial algorithm’s run time, because this algorithm uses multiple recursion.  Suppose T(N) is the run time for the algorithm on input N. If N > 1, the algorithm calculates Fibonacci(N – 1) and Fibonacci(N – 2), performs an extra step to add those values, and returns the result.  That means T(N) = T(N – 1) + T(N – 2) + 1.  Ignoring the extra constant 1 at the end, the run time is the same as the definition of the Fibonacci function, so the algorithm has a run time at least as large as the function itself.  The Fibonacci function doesn’t grow as quickly as the factorial function, but it still grows very quickly. For example, Fibonacci(92) ≈ 7.5x1018 and Fibonacci(93) doesn’t fit in a long integer.  However, the run time of the Fibonacci algorithm grows very quickly.  On a typical computer, calculating Fibonacci(44) takes more than a minute so calculating much larger values of the function is impractical anyway.

13. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 13  Solving Tower of Hanoi Puzzle  In Tower of Hanoi puzzle there a stack of disks, where each disk is smaller than the one below it, sits on one of three pegs. The goal is to transfer the disks from their starting peg to another peg by moving them one at a time and never placing a disk on top of a smaller disk.  Trying to solve the problem as a whole can be confusing  Instead, can reduce the problem size and then recursively solve the rest of the problem.

14. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 14  Solving Tower of Hanoi Puzzle (contd.)  The following pseudocode uses this approach to provide a simple recursive solution: // Move the top n disks from from_peg to to_peg // using other_peg to hold disks temporarily as needed. TowerOfHanoi(Peg: from_peg, Peg: to_peg, Peg: other_peg, Integer: n) // Recursively move the top n - 1 disks from from_peg to other_peg. If (n > 1) Then TowerOfHanoi(from_peg, other_peg, to_peg, n - 1) // Move the last disk from from_peg to to_peg. // Recursively move the top n - 1 disks back from other_peg to to_peg. If (n > 1) Then TowerOfHanoi(other_peg, to_peg, from_peg, n - 1) End TowerOfHanoi

15. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 15  Solving Tower of Hanoi Puzzle (contd.)  The method recursively calls itself repeatedly to move smaller and smaller stacks of disks.  At some point in the sequence of recursive calls, the algorithm is called to move only a single disk.  Then the algorithm doesn’t call itself recursively. It simply moves the disk and returns.  The key is that each recursive call is used to solve a smaller problem.  Eventually the problem size is so small that the algorithm can solve it without calling itself recursively.  As each recursive call returns, the algorithm’s calling instance moves a single disk and then calls itself recursively again to move the smaller stack of disks to its final destination peg.

16. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 16  Solving Tower of Hanoi Puzzle (contd.)

17. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 17  Solving Tower of Hanoi Puzzle (contd.)

18. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 18  Solving Tower of Hanoi Puzzle (contd.)  To analyze the algorithm’s run time, let T(n) be the number of steps required to move n disks from one peg to another.  Clearly T(1) = 1, because it takes one step to move a single disk from one peg to another.  For n > 0, T(n) = T(n – 1) + 1 + T(n – 1) = 2 × T(n – 1) + 1  If you ignore the extra constant 1, T(n) = 2 × T(n – 1) so the function has exponential run time O(2N).  To see this in another way, you can make a table similar to Table 9-1, giving the number of steps for various values of n.  In the table, each value with n > 1 is calculated from the previous value by using the formula T(n) = 2 × T(n – 1) + 1.  If you study the values, you’ll see that T(n) = 2n– 1.

19. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Basic Recursive Algorithms (contd.) 19  Solving Tower of Hanoi Puzzle (contd.)  Like the Fibonacci algorithm, the maximum depth of recursion for this algorithm on input N is N.  Also like the Fibonacci algorithm, this algorithm’s run time increases very quickly as N increases  So the run time limits the effective problem size long before the maximum depth of recursion does.

20. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms 20  Backtracking algorithms use recursion to search for the best solution to the complicated problems.  These algorithms recursively build partial test solutions to solve the problem.  When they find that a test solution cannot lead to a usable final solution, they backtrack, discarding that test solution and continuing the search from farther up the call stack.  Backtracking is useful when you can incrementally build partial solutions  You can sometimes quickly determine that a partial solution cannot lead to a complete solution.  In that case, you can stop improving that partial solution, backtrack to the previous partial solution, and continue the search from there.  We will cover two problems that have natural backtracking algorithms: the eight queens problem and the knight’s tour problem.

21. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 21 General backtracking approach at a high level: // Explore this test solution. // Return false if it cannot be extended to a full solution. // Return true if a recursive call to LeadsToSolution finds a full solution. Boolean: LeadsToSolution(Solution: test_solution) // If we can already tell that this partial solution cannot lead to a full solution, return false. If Then Return false // If this is a full solution, return true. If Then Return true // Extend the partial solution. Loop // Recursively see if this leads to a solution. If (LeadsToSolution(test_solution)) Then Return true // This extension did not lead to a solution. Undo the change. End Loop // If we get here, this partial solution cannot lead to a full solution. Return false End LeadsToSolution

22. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 22  Eight Queens Problem  In the eight queens problem, the goal is to position eight queens on a chessboard so that none of them can attack any of the other queens.  In other words, no two queens can be in the same row, column, or diagonal.  Following figure shows one solution to the eight queens problem.

23. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 23  Eight Queens Problem (contd.)  One way to solve this problem would be to try every possible arrangement of eight queens on a chessboard.  Unfortunately, there are 4,426,165,368 possible arrangements.  You could enumerate all of them, but doing so would be time-consuming.  For this problem, backtracking allows you to eliminate certain possibilities from consideration.  For example, eliminating every possible solution that has the first two queens next to each other in the upper-left corner. The program can backtrack to the point before it added the second queen and search for more promising solutions. One such backtracking step saves you the effort of examining more than 61 million possibilities.

24. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 24  Eight Queens Problem (contd.)  In fact, after placing first queen in the upper-left corner, eliminating all those partial solutions that place next queen in the same row, column, or diagonal removes almost 1.3 billion possible arrangements from consideration.  After you place the second queen somewhere legal, it restricts where the third queen can be placed further and so on.  The following pseudocode shows how you can use backtracking to solve the eight queens problem:

25. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 25 Pseudo-code using backtracking to solve the eight queens problem: Boolean: EightQueens(Boolean: spot_taken[,], Integer: num_queens_positioned) // See If the test solution is already illegal. If (Not IsLegal(spot_taken)) Then Return false // See if we have positioned all of the queens. If (num_queens_positioned == 8) Then Return true // Extend the partial solution. // Try all positions for the next queen. For row = 0 to 7 For col = 0 to 7 // See if this spot is already taken. If (Not spot_taken[row, col]) Then // Put a queen here. spot_taken[row, col] = true // Recursively see if this leads to a solution. If (EightQueens(spot_taken, num_queens_positioned + 1)) Then Return true // The extension did not lead to a solution. // Undo the change. spot_taken[row, col] = false End If Next col Next row // If we get here, we could not find a valid solution. Return false End EightQueens // The IsLegal method, which isn’t shown here, simply // loops through the spot_taken array to see if there // are two queens in the same row, column, or // diagonal.

26. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 26  Knight’s Tour Problem  A knight moves two squares horizontally or vertically and then one square perpendicularly from its current position.  In the knight’s tour problem, the goal is to make a knight visit every position on a chessboard without visiting any square twice.  A tour is considered closed if the final position is one move away from the starting position, and the knight could immediately start the tour again.  A tour that is not closed is considered open.

27. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 27  Knight’s Tour Problem (contd.) Following pseudo-code shows a backtracking solution to the knight’s tour problem: // Move the knight to position [row, col]. Then recursively try // to make other moves. Return true if we find a valid solution. Boolean: KnightsTour(Integer: row, Integer: col, Integer: move_number[,], Integer: num_moves_taken) // Move the knight to this position. num_moves_taken = num_moves_taken + 1 move_number[row, col] = num_moves_taken // See if we have made all the required moves. If (num_moves_taken == 64) Then Return true // Build arrays to determine where legal moves are // with respect to this position. Integer: dRows[] = { -2, -2, -1, 1, 2, 2, 1, -1 } Integer: dCols[] = { -1, 1, 2, 2, 1, -1, -2, -2 } // Try all legal positions for the next move. For i = 0 To 7 Integer: r = row + d_rows[i] Integer: c = col + d_cols[i] If ((r >= 0) And (r = 0) And (c < 8) And (move_number[r, c] == 0)) Then // This move is legal and available. Make this move // and then recursively try other assignments. If (KnightsTour(r, c, move_number, num_moves_taken)) Then Return true End If Next i // This move didn't work out. Undo it. move_number[row, col] = 0 // If we get here, we did not find a valid solution. return false End KnightsTour

28. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Backtracking Algorithms (contd.) 28  Knight’s Tour Problem (contd.)  Unfortunately, the knight’s tour problem isn’t as easy to constrain as the eight queens problem.  With the eight queens problem, it’s fairly easy to tell whether a new position is under attack by another queen and therefore unavailable for a new queen.  In the knight’s tour, any position the knight can reach that has not yet been visited gives a new test solution. So the algorithm often follows a test solution for a long while before discovering that the solution is infeasible.  There is a heuristic approach for knight’s tour problem in which at each step the algorithm should select the next possible move that has the lowest number of possible moves leading out of it.