WHICH SEARCH OR SORT IS BETTER?

COMPARING ALGORITHMS Time efficiency refers to how long it takes an algorithm to run Space efficiency refers to the amount of memory space an algorithm uses Algorithms are compared to each other by expressing their efficiency in big-oh notation 2

BIG-O NOTATION ConstantO (1)No matter how large or small the data set, the algorithm will take the same amount of time. Example: 1 item takes 1 second, 10 items takes 1 second, 100 items takes 1 second LogarithmicO (log n)Time increases with larger data sets, but not proportionately Example: 1 item takes 1 second, 10 items takes 2 seconds, 100 items takes 3 seconds. LinearO (n)The larger the data set, the time taken grows proportionately. Example: 1 item takes 1 second, 10 items takes 10 seconds, 100 items takes 100 seconds. LinearithmicO (n log n)Combines the previous two. Normally there’s 2 parts to the sort, the first loop is O(n), the second is O(log n), combining to form O(n log n). Example:1 item takes 2 seconds, 10 items takes 12 seconds, 100 items takes 103 seconds. QuadraticO (n 2 )Things are getting extra slow. Example: 1 item takes 1 second, 10 items takes 100, 100 items takes ExponentialO (2 n )The algorithm takes twice as long for every new element added. Example: 1 item takes 1 second, 10 items takes 1024 seconds, 100 items takes seconds.

BIG-O NOTATION Sequential Search = O(n) Binary Search = O(log n) Selection Sort = O(n²) Insertion Sort = O(n²) Merge Sort = O(n log n) Quick Sort = O (n log n)

ANALYSIS OF SEQUENTIAL SEARCH Problem: Find a target in a list of n values Best case: 1 comparison (target found immediately) Worst case: n comparisons made (target not in list, or target last element) Average case: n/2 comparisons (target found in the middle)

ANALYSIS OF BINARY SEARCH Problem: Find a target in a sorted array of n values (find entry in a phone book?) Algorithm: Search in the middle, every iteration the list is halved (n, n/2, n/4…) Best case: target is found on the first try (in the middle) Worst case: target is not in the list, or is at either end of a sublist How do you know the maximum number of comparisons under the worst case? Example: If the array has 9 elements then find the 2 m closest to 9 (round up to 16)

LOG 2 N log2 n  The number of times you can half a (positive) number n before it goes below 1 log 2 n = m which means 2 m = n Examples: log 2 16 = 4 because 2 4 = 16 16/2=8 and 8/2=4 and 4/2=2 and 2/2=1 log 2 8 = 3 which mean 2 3 =8

LOG 2 N Increases very slowly log 2 8 = 3 log 2 32 = 5 log = 7 log = 10 log = 20 log = 30 …

LOG 2 N Does efficiency matter? Say n = 10 9 (1 billion elements) 10 MHz computer ==> 1 instruction takes seconds Sequential search would take (n) = 10 9 x seconds = 100 seconds Binary search would take (log n) = log x sec = 30 x10 -7 sec = 3 microseconds

COMPARING SORTS Both Selection and Insertion sorts are similar in efficiency They both have outer loops that scan all elements, and inner loops that compare the value of the outer loop with almost all values in the list Approximately n 2 number of comparisons are made to sort a list of size n We therefore say that these sorts have efficiency O(n 2 ), or are of order n 2 Other sorts are more efficient: O(n log 2 n) 10

ANALYSIS OF SELECTION SORT Iteration 1:  Find largest value in a list of n numbers : n-1 comparisons  Exchange values and move marker Iteration 2:  Find largest value in a list of n-1 numbers: n-2 comparisons  Exchange values and move marker Iteration 3:  Find largest value in a list of n-2 numbers: n-3 comparisons  Exchange values and move marker … Iteration n:  Find largest value in a list of 1 numbers: 0 comparisons  Exchange values and move marker Total comparisons: (n-1) + (n-2) + …

ANALYSIS OF SELECTION SORT There is no best case or worst case This is an O (n 2 ) algorithm Selection Sort will be too inefficient to use with a large array

ANALYSIS OF INSERTION SORT Insertion sort uses nested loops. This means that as the number of elements in the array grows it will take approximately n * n longer to sort. In big-O notation, this will be represented like O(n 2 ). Best case scenario is O(n) because if the array is already sorted the inner loop won’t need to go through all of the elements again. Space efficiency: O(1) because it sorts in place Appropriate for very small arrays

ANALYSIS OF MERGE SORT 3 steps: 1. First we computes the midpoint. This is O(1) no matter how large the array. 2. Next we recursively sort two subarrays of approximately half. 3. The final step is to merge the two subarrays together. The merge sort algorithm has time efficiency of O(n log n). Space efficiency: O(n) because it uses a temporary array

ANALYSIS OF QUICK SORT 3 steps: 1. First we select a pivot. This is O(1) no matter how large the array. 2. Next we partition the array with items less than pivot on the left, and items greater than pivot on the right. 3. recursively quicksort until all elements are sorted in place (no temporary arrays needed). Worst case: the pivot partitions the array into a subset of 1 item and a subset of n-1 items, which gives us O(n 2 ) behavior in the worst case. Average case: The quick sort algorithm has time efficiency of O(n log n).

nMerge Sort (milliseconds)Selection Sort (milliseconds) 10, ,000732,148 30, ,796 40, ,192 50, ,321 60, ,299 ANALYZING THE MERGE SORT ALGORITHM

Merge Sort Timing vs. Selection Sort