1. ICS 353: Design and Analysis of Algorithms
King Fahd University of Petroleum & Minerals
Information & Computer Science Department
ICS 353: Design and Analysis of Algorithms
Induction

2. Reading Assignment
M. Alsuwaiyel, Introduction to Algorithms: Design Techniques and Analysis, World Scientific Publishing Co., Inc
Chapter 5 Sections 1-5.

3. Outline
Introduction
Recursive Selection Sort and Recursive Insertion Sort
Radix Sort
Integer Exponentiation
Evaluating Polynomials (Horner's Rule)

4. Induction
Main objective: Using induction or inductive reasoning as a “recursive” algorithm design technique
Why recursion
Concise algorithms for complex problems can be developed
The algorithms are easy to comprehend
Development time
Proof of correctness of the designed algorithm is usually simple.

5. How Does Induction Produce Recursive Algorithms?
Having a problem with input size n, it is sometimes easier to start with a solution to the problem with a smaller size and extend the solution to include the input size n.

6. Problems to Discuss
Selection sort
Insertion sort
Radix sort
Integer exponentiation
Evaluating polynomials

7. Recursive Selection Sort
Induction Hypothesis: We know how to sort A[2..n]
Inductive Reasoning: We sort A[1..n] as follows
Find the minimum A[j], 1  j  n
Swap(A[1],A[j])
Sort A[2..n] // induction hypothesis
What is the base case?
What is the initial call to the sorting algorithm?
This kind of recursion is called tail-recursion.

8. Algorithm SELECTIONSORTREC Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in nondecreasing order.
Procedure sort(i) {Sort A[i..n]}
if i < n then
Begin
2. k := i;
3. for j := i + 1 to n
if A[j] < A[k] then k := j;
5. if k != i then Swap(A[i], A[k]);
6. sort(i + 1);
End

9. Complexity Analysis of Recursive Selection Sort
What is the recurrence relation that describes the number of comparisons carried out by the algorithm?
What is the solution to the recurrence?

10. Recursive Insertion Sort
Induction Hypothesis: We know how to sort A[1..n-1]
Inductive Reasoning: We sort A[1..n] as follows:
Sort A[1..n-1] // induction hypothesis
Insert A[n] in its proper position in A[1..n]
This may involve copying zero or more elements one position ahead in order to insert A[n]

11. Algorithm INSERTIONSORTREC
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in non-decreasing order.
Procedure sort(i) {Sort A[1..i]}
1. if i > 1 then
2. sort(i – 1) {Recursive Call}
3. x := A[i]
4. j := i – 1
5. while j > 0 and A[j] > x
6. A[j + 1] := A[j]
7. j := j – 1
end while
8. A[j + 1] := x
end if

12. Complexity Analysis of Recursive Insertion Sort
Unlike selection sort, the analysis has to differentiate between the best case and the worst case. Why?
Recurrence of the best case:
Solution to the best case:
Recurrence of the worst case:
Solution to the worst case:

13. Radix Sort
Treats keys as numbers in a particular radix or base.
This is NOT an elements comparison-based sorting algorithm
It only works under the following assumption:

14. Radix Sort Derivation Assume that our keys are of the form
dk dk – 1 … d1
Induction Hypothesis: Suppose that we know how to sort numbers lexicographically according to their least k – 1 digits, dk – 1, dk – 2, …, d1 , k > 1.
Inductive reasoning: We sort the numbers based on their first k digits as follows:
Use induction hypothesis to sort the numbers based on their 1..k-1 digits.
Sort the numbers based on their corresponding kth digits

15. Description of Radix Sort Algorithm
Distribute the input numbers into 10 sublists L0, L1,…, L9 according to the least significant digit, d1.
Form a new list, we denote by main list, by removing the input from the 10 lists, starting from L0, L1, …etc. in order.
Distribute the main list into the 10 sublists according to the second digit, d2, and then repeat step 2.
Repeat step 3 for the rest of the digits in order of d3, d4, …, dk.

16. Example
Sort the following numbers using radix sort:

17. Radix Sort Algorithm Algorithm RADIXSORT
Input: A linked list of numbers L = {a1, a2,…,an} and k, the maximum number of digits.
Output: L sorted in nondecreasing order.
1. for j ←1 to k do
2. Prepare 10 empty lists L0, L1, …, L9.
3. while L is not empty
4. a ← next element in L. Delete a from L.
5. i ← jth digit in a. Append a to list Li.
6. end while
7. L ← L0
8. for i ←1 to 9 do
9. L ← L . Li {append list Li to L}
10. end for
11. end for
12. return L

18. Time and Space Complexity Analysis
Time Complexity
Space Complexity

19. Integer Exponentiation
What is the straightforward algorithm to raise x to the power n?
Such an algorithm is exponential in input size!!!!!!!!

20. Algorithm Derivation
Assume we have integer n which is represented in binary as (bkbk-1…b1b0)2
Induction Hypothesis: Assume that we know how to compute xn/2.
Inductive Reasoning: We raise x to the power n as follows:
Use induction hypothesis to compute y=xn/2 . xn/2
If n is even, xn = y
If n is odd, xn = x . y

21. ALGORITHM EXPRECURSIVE
Input: A real number x and a nonnegative integer n.
Output: xn. /* the first call to the algorithm is power(x,n) */
Procedure power(x, m) {Compute xm }
1. if m=0 then y ← 1
2. else
3. y ← power(x,m/2)
y ← y2
if m is odd then y ← x . y end if
7. return y

22. Complexity Analysis of the Algorithm
When n = 0, the number of multiplications is Therefore, T(0)=
Best Case Analysis:
When does the best case occur?
What is the recurrence equation? Solution?
Worst Case Analysis:
When does the worst case occur?

23. Iterative Exponentiation Algorithm
Assume we want to raise x to the power n where n is an integer that is represented in binary as (bkbk-1…b1b0)2
Starting with y = 1, scan the binary digits of the number n from left to right (j = k down to 0):
If bj = 0, square y
If bj = 1, square y and multiply it by x
Example: Compute 212 using the iterative exponentiation algorithm

24. Polynomial Evaluation
A polynomial of degree n is generally written as:
where, a0, a1, …, an and x, is a sequence of n + 2 real numbers. Our objective here is to evaluate this general polynomial at the point x.

25. Two Straightforward Solutions
Evaluate each term aixi separately
What is its time complexity?
Compute xi by multiplying x by the previously computed value xi-1
How many multiplications, assignments, and additions do we have?

26. Horner’s Rule

27. Horner’s Rule
Induction Hypothesis: Suppose we know how to evaluate the following:
Inductive Reasoning: We evaluate Pn(x) as follows:
Using induction hypothesis, we know how to evaluate Pn-1(x)
Pn(x) = x Pn-1(x) + a0
What is the base step?

28. Horner’s Algorithm Horner’s Algorithm
Input: A sequence of n+2 real numbers a0, a1, …, an, x
Output: Pn(x)=anxn+an-1xn-1+…+a1x+a0
p ← an
for j ← 1 to n do
p ← x . p + an-j
return p
How many multiplications, additions, and assignments do we have?