1. Efficiency of Algorithms Csci 107 Lecture 8

2. Last time –Data cleanup algorithms and analysis –  (1),  (n),  (n 2 ) Today –Binary search and analysis –Order of magnitude  (lg n) –Sorting Selection sort

3. Searching Problem: find a target in a list of values Sequential search –Best-case :  (1) comparison target is found immediately –Worst-case:  (n) comparisons Target is not found, or is the last element in the list –Average-case:  (n) comparisons Target is found in the middle Can we do better? –No…unless we have the input list in sorted order

4. Searching a sorted list Problem: find a target in a sorted list –How can we exploit that the list is sorted, and come up with an algorithm faster than sequential search in the worst case? –How do we search in a phone book? –Can we come up with an algorithm? Check the middle value If smaller than target, go right Otherwise go left

5. Binary search Get values for list, A1, A2, ….An, n, target Set start =1, set end = n Set found = NO Repeat until ?? –Set m = middle value between start and end –If target = m then Print target found at position m Set found = YES –If target < Am then end = m-1 –Else start = m+1 If found = NO then print “Not found” End

6. Efficiency of binary search What is the best case? –Found in the middle What is the worst case? –Initially the size of the list is n –After the first iteration through the repeat loop, if not found, then either start = middle or end = middle ==> size of the list on which we search is n/2 –Every time in the repeat loop the size of the list is halved: n, n/2, n/4,…. –How many times can a number be halved before it reaches 1?

7. log 2 x –The number of times you can half a (positive) number x before it goes below 1 –Examples: log 2 16 = 4 [16/2=8, 8/2=4, 4/2=2, 2/2=1] log 2 n = m 2 m = n log 2 8 = 3 2 3 =8

8. log 2 x Increases very slowly log 2 8 = 3 log 2 32 = 5 log 2 128 = 7 log 2 1024 = 10 log 2 1000000 = 20 log 2 1000000000 = 30 …

9. Orders of magnitude Order of magnitude  ( lg n) –Worst-case efficiency of binary search:  ( lg n) Comparing order of magnitudes  (1) <<  (lg n) <<  (n) <<  (n 2 )

10. Comparing  (lg n) and  (n) Does efficiency matter? Say n = 10 9 (1 billion elements) 10 MHz computer ==> 1 instr takes 10 -7 sec –Seq search would take  (n) = 10 9 x 10 -7 sec = 100 sec –Binary search would take  (lg n) = lg 10 9 x 10 -7 sec = 30 x10 -7 sec = 3 microsec

11. Sorting Problem: sort a list of items into alphabetical or numerical order Why sorting? –Sorting is ubiquitous (very common)!! –Examples: Registrar: Sort students by name or by id or by department Post Office: Sort mail by address Bank: Sort transactions by time or customer name or accound number … For simplicity, assume input is a list of n numbers Ideas for sorting?

12. Selection Sort Idea: grow a sorted subsection of the list from the back to the front 5 7 2 1 6 4 8 3 | 5 7 2 1 6 4 3 | 8 5 2 1 6 4 3 | 7 8 5 2 1 3 4 | 6 7 8 … |1 2 3 4 5 6 7 8

13. Selection Sort Pseudocode (at a high level of abstraction) –Get values for n and the list of n items –Set marker for the unsorted section at the end of the list –Repeat until unsorted section is empty Select the largest number in the unsorted section of the list Exchange this number with the last number in unsorted section of list Move the marker of the unsorted section forward one position –End

14. Selection Sort Level of abstraction –It is easier to start thinking of a problem at a high level of abstraction Algorithms as building blocks –We can build an algorithm from “parts” consisting of previous algorithms –Selection sort: Select largest number in the unsorted section of the list We have seen an algorithm to do this last time Exchange 2 values

15. Selection Sort Analysis Iteration 1: –Find largest value in a list of n numbers : n-1 comparisons –Exchange values and move marker Iteration 2: –Find largest value in a list of n-1 numbers: n-2 comparisons –Exchange values and move marker Iteration 3: –Find largest value in a list of n-2 numbers: n-3 comparisons –Exchange values and move marker … Iteration n: –Find largest value in a list of 1 numbers: 0 comparisons –Exchange values and move marker Total: (n-1) + (n-2) + …. + 2 + 1

16. Selection Sort Total work (nb of comparisons): –(n-1) + (n-2) + …. + 2 + 1 –This sum is equal to.5n 2 -.5n (proved by Gauss) => order of magnitude is  ( ? ) Questions –best-case, worst-case ? –we ignored constants, and counted only comparisons.. Does this make a difference? Space efficiency –extra space ?

17. Selection Sort In conclusion: Selection sort –Space efficiency: No extra space used (except for a few variables) –Time efficiency There is no best-case and worst-case the amount of work is the same:  (n 2 ) irrespective of the input Other sorting algorithms? Can we find more efficient sorting algorithms?

18. Exam 1 Wednesday: Lab 4 (more efficiency, binary search, etc) –Due next Wednesday, but… –Strongly encouraged to finish lab before Exam1 Exam 1 (Monday febr 21st) –Material: Algorithms and efficiency –Open books, notes, labs –Practice problems handout –Study group: this time only, Sunday night (instead of Monday) Tom will email