1. CSC 2300 Data Structures & Algorithms January 30, 2007 Chapter 2. Algorithm Analysis

2. Today Binary Search Euclid’s Algorithm Efficient Exponentiation Recursion and recurrences

3. Binary Search Given an integer X and integers A 0, A 1, …, A N-1, which are presorted and already in memory, find i such that A i = X, or return i=-1 if X is not in the input. What is an obvious solution? Scan through the list from left to right. Time? O(N). Better strategy? Check if X is the middle element. If yes, we get the answer. If X is smaller, we apply the same strategy to the sorted array to the left of the middle element. If X is larger, we look to the right half.

4. Binary Search What is the running time? O(logN).

5. Analyzing Binary Search

6. Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=9. lowhigh#ptsmida[mid]return 01516715 06737 463511 441494

7. Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=8. lowhigh#ptsmida[mid]return 01516715 06737 463511 44149 430

8. Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=1. lowhigh#ptsmida[mid]return 01516715 06737 02313 001010

9. Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=31. lowhigh#ptsmida[mid]return 01516715 8 81123 121541327 141521429 15 1 3115

10. Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=33. lowhigh#ptsmida[mid]return 01516715 8 81123 121541327 141521429 15 1 3115 16150

11. Euclid’s Algorithm This algorithm computes gcd(m,n), assuming m≥n. The algorithm works by computing remainders until 0 is reached. The last nonzero remainder is the answer. Example, M=1989 and N=1590. The sequence of remainders is 399, 393, 6, 3, 0. Therefore, gcd(1989,1590)=3.

12. Running Time Analysis The sequence of remainders is 399, 393, 6, 3, 0. The remainder does not decrease by a constant factor in one iteration. What to do? We can prove that after two iterations, the remainder is at most half of its original value. This will show that the number of iterations is at most 2 logN = O(logN). Theorem 2.1. If M>N, then M mod N < M/2. Worst case is 2 logN? No, 1.44 logN, when M and N are consecutive Fibonacci numbers.

13. Efficient Exponentiation The obvious way to compute X N uses N-1 multiplications. A recursive algorithm can do better. What are the base cases? N=0 and N=1. Otherwise, if N is even, we compute X N = X N/2 X N/2, and if N is odd, X N = X N/2 X N/2 X. How many multiplications are needed? At most 2 logN. Why 2? If N is odd.

14. Example Compute X 62. The algorithm performs these calculations: X 3 = (X 2 )X, X 7 = (X 3 ) 2 X, X 15 = (X 7 ) 2 X, X 31 = (X 15 ) 2 X, X 62 = (X 31 ) 2, How many multiplications? Nine. In general, at most 2 logN.

15. Number of Multiplications Does it always require more work to compute X i than X j if i > j? No. Consider X 64 (versus X 62 ). The algorithm performs these calculations: X 2 = (X) 2, X 4 = (X 2 ) 2, X 8 = (X 4 ) 2, X 16 = (X 8 ) 2, X 32 = (X 16 ) 2, X 64 = (X 32 ) 2, How many multiplications? Six (versus nine for X 62 ).

16. Precise Number of Multiplies Can we find the precise number of multiplies used by the algorithm? Consider X 65. The algorithm performs these calculations: X 2 = (X) 2, X 4 = (X 2 ) 2, X 8 = (X 4 ) 2, X 16 = (X 8 ) 2, X 32 = (X 16 ) 2, X 65 = (X 32 ) 2 X, How many multiplications? Seven (versus six for X 64 ).

17. Another Example Consider X 63. The algorithm performs these calculations: X 3 = (X) 2 X, X 7 = (X 3 ) 2 X, X 15 = (X 7 ) 2 X, X 31 = (X 15 ) 2 X, X 63 = (X 31 ) 2 X, How many multiplications? Ten (versus six for X 64 ). What does the number depend on?

18. Precise Number of Multiplies Compute X N. What is the number of multiplies when N=0 or N=1? Zero. What is the important feature on the value of N? Its binary representation. Let b(N) represent the number of ones in the binary representation of N. What is the formula for #mult? [ log 2 N ] + b(N) - 1 N0162636465 #mult0091067

19. Fast Exponentiation Is algorithm optimal? No. Compute X 62. Fast exponentiation requires nine multiplications. Can you calculate X 62 using only 8 multiplies? How? Compute X 2, X 4, X 8, …, X 62. You are asked to solve this problem in Homework 2.

20. Recursion and Recurrences

21. Methods for Recurrences Two straightforward methods for solving recurrences are 1. Repeated expansion accompanied by summation 2. Forming telescopic sums

22. Method 1 Repeated expansion Recurrence: T(n) = T(n-1) + 1

23. Method 2 Telescoping sum Recurrence: T(n) = T(n-1) + 1

24. Telescoping Sum Recall Algorithm 3 to find Maximum Subsequence Sum. Recurrence: T(1) = 1, T(N) = 2 T(N/2) + N. Divide recurrence by N: T(N)/N = T(N/2)/(N/2) + 1. The recurrence is valid for any N that is a power of 2: T(N)/N = T(N/2)/(N/2) + 1 T(N/2)/(N/2) = T(N/4)/(N/4) + 1 T(N/4)/(N/4) = T(N/8)/(N/8) + 1 … T(4)/4 = T(2)/2 + 1 T(2)/2 = T(1)/1 + 1 Telescoping the sum, we get T(N)/N = T(1)/(1) + logN. So, T(N) = N logN + N = O(N logN).