Rethinking Recursion Prof. Chung-Ta King Department of Computer Science National Tsing Hua University CS1103 電機資訊工程實習 (Contents from Dr. Jürgen Eckerle, Dr. Sameh Elsharkawy, Dr. David Reed,

1 What Is Special about the Tree?

2 How about This?

3 And These?

They all defined/expressed in terms of themselves Recursion

5 How about This? Robot factory

6 Overview  What is recursion?  Why recursion?  Recursive programming  Recursion and iteration  Recursion and induction

7 Recursion  In mathematics and computer science, recursion is a method of defining functions in which the function being defined is applied within its own definition  For example: n! = n  (n-1)!  It is also used more generally to describe a process of repeating objects in a self-similar way

8 Recursion  Recursion is a powerful technique for specifying functions, sets, and programs.  Recursively-defined functions factorial counting combinations (choose r out of n items) differentiation of polynomials  Recursively-defined sets language of expressions  Recursively-defined graphs, images, puzzles, concepts, …

9 Recursive Definitions  Consider the following list of numbers: 24, 88, 40, 37  Such a list can be defined as follows: A LIST is a: number or a: number comma LIST That is, a LIST is defined to be a single number, or a number followed by a comma followed by a LIST A more concise expression: (Grammar) LIST  number LIST  number, LIST  The concept of a LIST is used to define itself

10 Recursive Definitions  The recursive part of the LIST definition is used several times, terminating with the non- recursive part: number comma LIST 24, 88, 40, 37 number comma LIST 88, 40, 37 number comma LIST 40, 37 number 37

11 Infinite Recursion  All recursive definitions have to have a non- recursive part If they didn't, there would be no way to terminate the recursive path Such a definition would cause infinite recursion  This problem is similar to an infinite loop, but the non-terminating "loop" is part of the definition itself  The non-recursive part is often called the base case

12 Recursion Every recursive definition has 2 parts:  BASE CASE(S): case(s) so simple that they can be solved directly  RECURSIVE CASE(S): make use of recursion to solve smaller subproblems and combine into a solution to the larger problem To verify that a recursive definition works:  Check if base case(s) are handled correctly  ASSUME RECURSIVE CALLS WORK ON SMALLER PROBLEMS, then check that results from the recursive calls are combined to solve the whole

13 Recursion  N!, for any positive integer N, is defined to be the product of all integers between 1 and N inclusive  This definition can be expressed recursively in a recurrence equation as: 1! = 1  base case N! = N * (N-1)!  recursive case  A factorial is defined in terms of another factorial  Eventually, the base case of 1! is reached

14 Recursion  Fibonacci numbers: 1, 1, 2, 3, 5, 8, 13, 21, … Fibo(1) = 1 Fibo(2) = 1 Fibo(n) = Fibo(n–1) + Fibo(n–2)  The larger problem is a combination of two smaller problems

15 Python Code def factorial(n): if n == 1: return 1 result = n * factorial(n-1) return result for i in range(1,10): print factorial(i) def fibonacci(N): if N <= 2: return 1 return fibonacci(N-1)+fibonacci(N-2) for i in range(1,10): print fibonacci(i)

16 Overview  What is recursion?  Why recursion?  Recursive programming  Recursion and iteration  Recursion and induction

17 Why Recursion?  Concise in representation and expression: For example: any algebraic expression such as ( x + y ) ( x + y ) * x ( x + y ) * x – z * y / ( x + x ) x * z / y + ( x ) – ( y * z ) + ( y – x * z – y / x ) Can be expressed using the recursive definition: S  x | y | z | S + S | S – S | S * S | S/S | (S) For example: ( x + y )  ( S + S )  ( S )  S Recursion for composition and decomposition

18 Why Recursion?  Simplify solution  divide and conquer Divide a given (complex) problem into a set of smaller problems and solve these and merge them to a complete solution If the smaller problems have the same structure as the originally problem, this problem solving process can be applied recursively. This problem solving process stops as soon as trivial problems are reached which can be solved in one step. Only need to focus on the smaller/simplified subproblems, instead of overwhelming by the (complex) original problem

19 Divide and Conquer Method:  If the problem P is trivial, solve it. Otherwise  [Divide] Divide P into a set of smaller problems P[0],..., P[n-1]  [Conquer] Compute a solution S[i] of all the subproblems P[i]  [Merge] Merge all the subsolutions S[i] to a solution S of P

20 Overview  What is recursion?  Why recursion?  Recursive programming  Recursion and iteration  Recursion and induction

21 Recursive Programming  A procedure can call itself, perhaps indirectly  General structure: if stopping condition then solve base problem else use recursion to solve smaller problem(s) combine solutions from smaller problem(s)  Each call to the procedure sets up a new execution environment (stack frame or activation record), with new parameters and local variables  When the procedure completes, control returns to the calling procedure, which may be an earlier invocation of itself

22 Tower of Hanoi Which is the base case? Which is the recursion case?

23 Tower of Hanoi

24 Tower of Hanoi

25 Python Code def hanoi(n, a='A', b='B', c='C'): # move n discs from a to c thru b if n == 0: return hanoi(n-1, a, c, b) print 'disc', n, ':', a, '->', c hanoi(n-1, b, a, c) hanoi(3)

26 Thinking Recursively 1. Find a way of breaking the given problem into smaller/simpler subproblems Tower of Hanoi: largest disc & remaining n-1 discs 2. Relate the solution of the simpler subproblem with the solution of the larger problem Tower of Hanoi: move n-1 discs to the middle peg; move the largest disc to the destination peg; move n-1 discs from the middle peg to the destination peg 3. Determine the smallest problem that cannot be decomposed any further and terminate there

27 Generating Permutations  Numerous applications require systematically generating permutations (orderings)  Take some sequence of items (e.g. string of characters) and generate every possible arrangement without duplicates "123"  "123", "132", "213", "231", "312", "321"

28 Recursive Generation  Recursive permutation generation for each letter in the word 1. remove that letter 2. get the permutation of the remaining letters 3. add the letter back at the front  Example: "123" "1" + (1st permutation of "23") = "1" + "23" = "123" "1" + (2nd permutation of "23") = "1" + "32" = "132" "2" + (1st permutation of "13") = "2" + "13" = "213" "2" + (2nd permutation of "13") = "2" + "31" = "231" "3" + (1st permutation of "12") = "3" + "12" = "312" "3" + (2nd permutation of "12") = "3" + "21" = "321"

29 Tiled Pictures  Consider the task of repeatedly displaying a set of images in a mosaic Three quadrants contain individual images Upper-left quadrant repeats pattern  The base case is reached when the area for the images shrinks to a certain size

30 Tiled Pictures

31 Fractals  A geometric shape made up of same pattern repeated in different sizes and orientations  Koch curve A curve of order 0 is a straight line A curve of order n consists of 4 curve of order n-1

32 Fractals  Koch curve after five iteration steps (order 4 curve)

33 Fractals  Koch snowflake From 3 Koch curves of order 4

34 Sierpinski Triangle  A confined recursion of triangles to form a geometric lattice

35 Shortest Path from s to v  How to solve it with a recursive procedure?    s uv yx

36 Shortest Path from s to v  Decompose into smaller subproblems  Combine and find the minimum    s uv yx    s uv y x

37 Overview  What is recursion?  Why recursion?  Recursive programming  Recursion and iteration  Recursion and induction

38 Recursion vs. Iteration  Iteration can be used in place of recursion  (Nearly) every recursively defined problem can be solved iteratively Iterative optimization, e.g. by compiler, can be implemented after recursive design  Recursive solutions are often less efficient, in terms of both time and space (next page)  Recursion can simplify the solution of a problem, often resulting in shorter, more easily understood, correct source code What if we have multiple processors?

39 Recursion and Redundancy  Consider the recursive fibonacci method: fib(5) fib(4) + fib(3) fib(3) + fib(2) fib(2) + fib(1) fib(2) + fib(1) SIGNIFICANT amount of redundancy in recursion # recursive calls > # loop iterations (by an exponential amount!) Recursive version is often slower than iterative version

40 Shortest Path from s to v  The recursive procedure builds a search tree of an exponential complexity    s uv yx      s u uy x  x  v  v  v 6 There are more efficient algorithms: e.g. Dijkstra’s algorithm (O(n 2 ))

41 Divide-and-Conquer Again  What have we done with n! in terms of divide- and-conquer? n! = n * (n – 1)! We only divide one number off in each recursion  the two subproblems are imbalanced  Can we do this? n! = (n * (n-1) * … * (n/2 +1)) * (n/2)! even n n! = n * ((n-1) * … * ((n-1)/2+1)) * ((n-1)/2)! odd n Very much like a binary tree Any difference between the two? Tail recursion

42 When Recursion?  When it is the most natural way of thinking about and implementing a solution can solve problem by breaking into smaller instances, solve, combine solutions  When it is roughly equivalent in efficiency to an iterative solution, or When the problems to be solved are so small that efficiency doesn't matter  think only one level deep make sure the recursion handles the base case(s) correctly assume recursive calls work correctly on smaller problems make sure solutions to the recursive problems are combined correctly  avoid infinite recursion make sure there is at least one base case & each recursive call gets closer

43 Overview  What is recursion?  Why recursion?  Recursive programming  Recursion and iteration  Recursion and induction

44 Recursion and Induction  Recursion: A solution strategy that solves a large problem by breaking it up into smaller problems of same kind A concept that make something by itself, e.g. tools that make tools, robots that make robots, Droste pictures  Induction: A mathematical strategy for proving statements about integers (more generally, about sets that can be ordered in some fairly general ways)  Understanding induction is useful for figuring out how to write recursive code.

45 Prove Inductively  Assume equally spaced dominoes, where spacing is less than domino length.  How would you argue that all dominoes would fall? Domino 0 falls because we push it over. Domino 1 falls because domino 0 falls, domino 0 is longer than inter-domino spacing, so it knocks over domino 1 Domino 2 falls because …  Is there a more compact argument?

46 Prove Inductively  Better argument Domino 0 falls because we push it over. Suppose domino k falls over. Because its length is larger than inter-domino spacing, it will knock over domino k+1. Therefore, all dominoes will fall over.  This is an inductive argument.  Not only is it more compact, but it works even for an infinite number of dominoes!

47 Induction over Integers  We want to prove that some property P holds for all integers.  Inductive argument: P(0): show that property P is true for integer 0 P(k) => P(k+1): if property P is true for integer k, it is true for integer k+1 This means P(n) holds for all integers n

48 Consider This  Can we show that these two definitions of SQ(n) are equal? SQ 1 (0) = 0for n = 0 SQ 1 (n) = SQ 1 (n-1) + n 2 for n > 0 SQ 2 (n) = n(n+1)(2n+1)/6 where they all calculate SQ(n) = …+n 2

49 Inductive Proof  Let proposition be P(j): SQ 1 (j) = SQ 2 (j)  Two parts of proof: Prove P(0). Prove P(k+1) assuming P(k). P(0)P(1)P(2)P(k)P(k+1)

50 Inductive Proof  P(0): show SQ 1 (0) = SQ 2 (0) (easy) SQ 1 (0) = 0 = SQ 2 (0)  P(k)  P(k+1): Assume SQ 1 (k) = SQ 2 (k) SQ 1 (k+1) = SQ 1 (k) + (k+1) 2 (definition of SQ 1 ) = SQ 2 (k) + (k+1) 2 (inductive assumption) = k(k+1)(2k+1)/6 + (k+1) 2 (definition of SQ 2 ) =(k+1)(k+2)(2k+3)/6 (algebra) = SQ 2 (k+1) (definition of SQ 2 )

51 Consider the Tiling Problem  Problem: A chess-board has one square cut out of it. Can the remaining board be tiled using tiles of the shape shown in the picture?  Not obvious that we can use induction to solve this problem.

52 Idea  Consider boards of size 2 n x 2 n for n = 1,2,…..  Base case: show that tiling is possible for 2 x 2 board.  Inductive case: assuming 2 n x 2 n board can be tiled, show that 2 n+1 x 2 n+1 board can be tiled.  Chess-board (8x8) is a special case of this argument.

53 Base Case  For a 2x2 board, it is trivial to tile the board regardless of which one of the four pieces has been cut.

54 4x4 Case  Divide 4x4 board into four 2x2 sub-boards.  One of the four sub-boards has the missing piece. That sub-board can be tiled since it is a 2x2 board with a missing piece.  Tile the center squares of the three remaining sub-boards as shown.  This leaves 3 2x2 boards with a missing piece, which can be tiled.

55 Inductive Proof  Claim: Any board of size 2 n x 2 n with one missing square can be tiled.  Base case: (n = 1) trivial  Inductive case: assume inductive hypothesis for (n = k) and consider board of size 2 k+1 x 2 k+1 Divide board into four equal sub-boards of size 2 k X 2 k One of the sub-boards has the missing piece; by inductive assumption, this can be tiled. Tile the central squares of the remaining three sub-boards This leaves three sub-boards with a missing square each, which can be tiled by inductive assumption.

56 When Induction Fails  Proposition: any n x n board with one missing square can be tiled  Problem: a 3 x 3 board with one missing square has 8 remaining squares, but our tile has 3 squares  tiling is impossible  Therefore, any attempt to give an inductive proof of the proposition must fail  This does not say anything about the 2 n x 2 n cases