CMPT 120 Topic: Searching – Part 2 and Intro to Time Complexity (Algorithm Analysis)

In this lecture We shall look at another searching algorithm 2

Learning Outcome At the end of this course, a student is expected to: Create (design), analyze, and explain the behaviour of simple algorithms: … Solve problems by designing simple algorithms, e.g., basic calculations, searching in strings and lists, counting, calculating running sums and products … Describe and illustrate various ways of searching Create (design), analyze, and explain the behaviour of simple algorithms: … Analyze the running time of simple iterative algorithms Compare the running time of algorithms; determine if one algorithm is more efficient than another Create (design) small to medium size programs using Python: … Create programs that search lists and strings 3

From last lecture: Could we search more efficiently? Are there other ways of searching (other search algorithms) that would be more time efficient than O(n)? What if we sort our list first, could this help us search more efficiently? 4

Let’s see … Suppose we have a sorted list Using Binary Search, we can search for target = 7 without having to look at every element

How binary search works – In a nutshell Binary search uses the fact that the list is sorted ! Find element in the middle -> 9 Since we are looking for 7 and 7 < 9, then there is no need to search the second half of the list We can ignore half of the list right away! Then we repeat the above steps Bottom line: using binary search, we do not need to look at every element to search a list 6

Note regarding Slides 8 to 11 The following slides (Slides 8 to 17) are animated so they may not make much sense if they printed To get the most from these slides, one must run them as a “slide show” in PowerPoint (or, perhaps, using other similar applications) 7

How it works We start with a list and a target = We find the middle element 3.Is this element == target? Yes, then we are done! No, then we throw away half of the list in which we know target cannot be located (grey part) We repeat steps 2 and 3 until we found target or run out of list. and we consider only the part in which target could be located 8

How it works We find the middle element 3.Is this element == target? Yes, then we are done! No, then we throw away half of the list in which we know target cannot be located (grey part) We repeat steps 2 and 3 until we found target or run out of list. and we consider only the part in which target could be located 9

How it works We find the middle element 3.Is this element == target? Yes, then we are done! No, then we throw away half of the list in which we know target cannot be located (grey part) We repeat steps 2 and 3 until we found target or run out of list and we consider only the part in which target could be located 10

How it works We find the middle element 3.Is this element == target? Yes, then we are done!

Binary Search algorithm - iterative PreCondition: data must be sorted position = TARGET_NOT_FOUND targetFound = false if list not empty while list is not empty AND not targetFound find middle of list if middle element == target position = position of target in original list targetFound = true else if target < middle element list = first half of list else list = last half of list return position We ignore 2 nd half of the list and middle element We ignore 1 st half of the list and middle element 12

# Precondition: list must be sorted def BinarySearch (aList, target, low, high): result = -1 # returning the index of target or -1 (out of range value) first = 0 # index representing the start of list last = len(aList) - 1 # index representing the end of list # loop until all elements of list have either been discarded or compared while first <= last: mid = (first + last) // 2 # determine middle of list if aList[mid] == target : # target found … return mid # … returns index of list element containing target # otherwise, target must be in second half of list elif aList[mid] < target : first = mid + 1 # otherwise, target must be in first half of list elif aList[mid] > target : last = mid - 1 return result # return invalid index if target not in list Binary Search - Python iterative program 13

# Precondition: list must be sorted # TARGET_ELEMENT_NOT_FOUND = -1 def BinarySearch (aList, target, low, high): TARGET_ELEMENT_NOT_FOUND = -1 if high < low: result = TARGET_ELEMENT_NOT_FOUND else: mid = (low + high) // 2 if aList[mid] > target : result = BinarySearch (aList, target, low, mid-1) elif aList[mid] < target : result = BinarySearch (aList, target, mid+1, high) else: result = mid # target found return result Inspired from Binary Search - Python recursive program 14

Question: Can we quickly tell if target is not in a list? 15

Binary Search - Advantages and disadvantages Advantages 1.Faster because does not have to look at every element (at every iteration, ignores ½ of list) Disadvantages 1.List must be sorted 2.A bit more complicated to implement (iterative or recursive) and test 16

How much faster is Binary Search over Linear Search? What is the time complexity of binary search 17

So far… … we know that worst case scenario of linear search is of order n -> O(n) i.e., has a time complexity of O(n) Review: what does this mean? 18

Review: How we compute the time complexity of an algorithm From our lecture notes on linear search: 1.Count the number of times “critical operations” are executed For linear search, the “critical operations” were the statements in the loop of the linear search 2.Express this count as a function of n 19

Binary search complexity analysis For binary search What are the worst case scenarios? What are its the “critical operations”? 20

Binary search complexity analysis What is the time complexity of the worst case scenario (e.g., target not in list) of the binary search? We compare the middle element with target It is not found We then partition the list in half at each iteration, ignore one half and search the other Size of data in 1 st step: n Size of data in 2 nd step: n/2 Size of data in 3 rd step: n/4 Size of data in 4 th step: n/8 …. Size of data in T-1 th step: 2 Size of data in T th step: 1 21

Binary Search - Worst case scenario 22

Binary Search - Worst case scenario 23

Result of Binary search complexity analysis The worst case scenario of binary search is of order log 2 n i.e., has a time complexity of O(log 2 n) 24

How much faster is Binary Search over Linear Search? We can compare algorithms that perform the same task, like our searching algorithms, by looking at their time complexity We often do this comparison considering n as it goes to infinity For example: how much “faster” is binary search over linear search as n goes to infinity ? To answer this question, let’s have a look at the diagram on next slide 25

Comparing various order of n’s Number of “critical operations” the algorithm executes As n goes to infinity 26 Source:

Comparing binary search to linear search Binary search is much faster than linear search when searching large data But the data must first be sorted Great if data is already sorted, but if this is not the case … How much work does this sorting requires? 27