INTRO2CS Tirgul 8 1

Searching and Sorting  Tips for debugging  Binary search  Sorting algorithms:  Bogo sort  Bubble sort  Quick sort and maybe  Radix sort  Merge sort 2

Tips for Debugging 3  Debugging is a process of finding bugs or defects in a program  We can roughly divide the bugs into two types: 1. Bugs that will cause the program to terminate its execution. This will be followed by an exception, and usually in Python there will be a traceback print of the error, including in which line it occurred and what type of error it is Those are the easy bugs to solve..

Tips for Debugging 4  For example:  Division by zero  Index Error  Undefined variable …  There are some examples in tirgul 6

Tips for Debugging 5 2. Bugs in the implementation – there is no exception thrown but the program does not do what we wanted it to do  Those are hard to find.  This is where we need to debug..

Tips for Debugging - example 6  Say we want to implement a function that gets a list and sums its elements:  No exception will be thrown, but it will not result what we intended

Tips for Debugging - example 7  We can add print commands in places we suspect can go wrong:

Tips for Debugging - example 8  Breakpoint means a point in the program where the execution is paused and we can retrieve the current values of the variables  We can use Pdb (the Python debugger) module and create breakpoints throughout the program using set_trace() function  In each breakpoint we can ask for the current variable values in the stdout  We can continue execution using ‘continue’

Tips for Debugging - example 9 > test.py(4)sum_list() -> for i in lst: (Pdb) 7 7 > test.py(4)sum_list() -> for i in lst: (Pdb) 2 type the variable name i sum_lst continue i Continue to next breakpoint More info:

Sorting  A sorting algorithm is an algorithm that puts elements in a certain order:  Lexicographic  Numeric  Chronologic  Sorting is important since many search algorithms are much more efficient (lower running time complexity) when running on a sorted input  We will talk about lists during the presentation, but it is valid to any type of sequence that keeps the order of its elements 10

Example  Find the youngest person  Unsorted: go over n rows: worst case takes n steps  Find the person who is 10 years old  Unsorted: go over n rows: worst case takes n steps NameAge Bart10 Homer37 Lisa8 Marge35 Meggie1 11

Example  Find the youngest person  Sorted (by age): get the youngest person in the table: takes 1 step  Find the person who is 10 years old  How long would it take on a sorted list? NameAge Meggie1 Lisa8 Bart10 Marge35 Homer37 12

Runtime Analysis  It is like “counting” the number of operations needed to be done, given an input of size n  How many iterations or recursive calls  How many operations are done in each iteration or recursive call  We will learn about it more deeply in the future 13

Binary Search  Given a sorted list and a key to be found:  Get the middle element  if(key == middle) we are done.  If(key < middle) do binary search on the left half of the list  If(key > middle) do binary search on the right half of the list 14

Binary Search - example  A list: [1,1,2,3,5,8,13,21,34,55]  We want to find the index of 3 in the list 1. middle = < 8, therefore we will look for it in the left half: [1,1,2,3,5] 3. Search again on the updated list 15

Binary Search  Find the index of 3: middle middle 16

Binary Search – implementation (1) 17

Binary Search – implementation (2) 18

Binary Search – How long does it take?  In each iteration (or in each recursive call) the search is done on a list that is half the size of the previous one.  Assume the list is of size n  After first iteration the list is of size n/2  After the second iteration the list is of size n/2*(1/2) = n/2 2  After k iterations the list is of size n/2 k  The algorithm is done when the list is of size 1, therefore: n/2 k = 1 Take log: k = log(n) 19

Sorting Motivation:  In many applications we would like to get information on a given input, for example to get the grades of a student  Given list of data of size n:  Searching on an unsorted list requires n steps  Searching on a sorted list requires log(n) steps when we use binary search 20

Stable Sorting  A sorting algorithm is considered to be stable if the initial order of equal items is preserved  That means that if 2 items are equal and one came before the other in the input, then this order would be kept in the output 21

Stable Sorting  Stable sort will always return same solution (permutation) on the same input. 22

Stable Sorting  The order of the two 5 cards is preserved  Actually, it is sorted by a certain key but have more than one comparable property 23

Stable Sorting  We can implement unstable algorithms as stable ones  For example, we can add to each element the original index as an additional property  This, of course would cost more memory  Stable algorithms usually have higher CPU and/or memory usage than unstable algorithms. 24

Bogo Sort  It is also called “random sort”, “slow sort”, or “stupid sort”  Simply shuffle the collection given randomly until it is sorted. 25

Bogo Sort - implementation 26

Bogo Sort Runtime Analysis  How long would it take?  Is it stable? No 1 iteration if we are lucky n! iterations on average Infinite number of iterations in the worst case 27

Bubble Sort  The idea: bubble up the largest element  Iteration 1: Compare each element with its neighbor to the right from index 0 to index n-1  If they are out of order, swap them  At the end of this iteration, the largest element is at the end of the list  If there were no swaps after going over all elements, we are done 28

Bubble Sort  Iteration 2: Compare each element with its neighbor to the right from index 0 to index n-2 …  Continue until there are no more elements to swap 29

Bubble Sort - example (done) 30

Bubble Sort - implementation 31

Bubble Sort Runtime Analysis (I)  For a list of size n, the outer loop does n iterations  In each iteration i of the outer loop, the inner loop does n-i iterations: (n-1) + (n-2) + … + 1 = n(n-1)/2  Note that each operation within the inner loop (comparing and swapping) takes constant time c Number of steps is a polynomial of 2 nd degree (n 2 ) 32

Bubble Sort Runtime Analysis (II)  What would be the number of steps in the best case? (i.e., the input list is sorted)  Start with the first iteration, going over all the list  The list is sorted so there are no swaps n steps are required 33

Bubble Sort  Is it stable? Yes, we will not swap elements of equal values  Relatively inefficient, so we would probably use it only in cases where we know that the input is nearly sorted 34

Quick Sort  Choose an element from the list called pivot  Partition the list:  All elements < pivot will be on the left  All elements ≥ pivot will be on the right  Recursively call the quicksort function on each part of the list 35

Quick Sort - implementation 36

Quick Sort – implementation (II) 37

Quick Sort – implementation (III) 38

Quick Sort – Runtime Analysis (I)  On each level of the recursion, we go over lists that contain total of n elements: About n steps at each level 39

Quick Sort – Runtime Analysis (II)  How many levels are there?  It depends on the pivot value:  Lets say we choose each time the median value  Each time the list is divided by half:  n/2  n/4 …… 11  There will be log(n) levels, and each takes n steps It would take about nlog(n) steps 40

Quick Sort – Runtime Analysis (III)  Lets say we choose each time an extreme value (smallest or largest) – it is unlikely  Each time we get one list of size 1 and one of size n-1:  n-1  n-2 …… 11  There will be n levels, and each takes n steps It would take about n 2 steps 41

Quick Sort  The efficiency is depended on the pivot choice  Is it stable? 42 No, since partition reorders elements within a partition

Find Duplicate Values 43  We would like to find duplicates values in a given list. It can be done by two approaches: 1. Compare all possible pairs 2. Sort the list and then search for neighboring elements with the same value

Find Duplicate Values (1) 44  How long does it take?  Outer loop iterates n times  For each outer iteration i, inner iteration runs i times: … + n-1 = n(n-1)/2

Find Duplicate Values (2) 45  How long does that takes?

Find Duplicate Values (2) 46  Quicksort takes on average nlog(n)  Make one pass on the sorted data – n steps  In total: n + nlog(n) steps This is much more efficient than ~n 2 steps

Radix Sort  Sorts integers by grouping keys by the individual digits which share the same significant position and value  Assumption: the input has d digits ranging from 0 to k : For each position there is a finite number of possible digits, depends on the base of the number (for decimal representation there are 10) 47

Radix Sort (assume decimal representation)  Divide all in integers into 10 groups according the least significant digit (why?)  For two numbers with the same digit keep the original order  Repeat the above steps with the next least significant digit 48

Radix Sort  Input list: [493,812,715,710,195,437,582,340,385] digitsublist 5715,195, [710, 340, 812, 582, 493, 715, 195, 385, 437] Divide by the least significant digit: 49 digitsublist 0710, ,

Radix Sort  Current list: [710, 340, 812,582,493,715,195,385,437] digitsublist , ,195 [710, 812, 715, 437, 340, 582, 385, 493, 195] Divide by the second least significant digit: 50 digitsublist ,812,

Radix Sort  Current list: [710, 812,715,437,340,582,385,493,195] digitsublist , [195, 340, 385, 437, 493, 582, 710, 715, 812] Divide by the third least significant (and last) digit: and we are done! 51 digitsublist , , 493

Radix Sort - implementation 52

Radix Sort Runtime Analysis  Lets note:  n the number of elements in the input  d the number of digits in the largest number  k the base of the number  The outer loop would iterate d times  In each iteration, we go over n elements and rebuild a list from k buckets It would take about d*(n+k) steps For d, k small enough it is linear runtime 53

Merge Sort  Divide and conquer algorithm:  Recursively break down a problem into two or more sub-problems of the same (or related) type, until these become simple enough to be solved directly  The solutions to the sub-problems are then combined to give a solution to the original problem. 54

Merge Sort  Divide the unsorted list into n lists, each contain 1 element  Repeatedly merge sublists to produce new sorted sublists until there is only 1 sublist remaining. 55

Merge Sort Split Merge 56

Merge Sort - implementation 57

Merge Sort Runtime Analysis (I)  How many times is merge called?  Last level of merge is on lists of size n/2  In previous level, the lists are of size n/2*(1/2) = n/2 2 …  First call is on lists of size 1 Assume we have k levels of merge calls, and we will stop when the lists size is 1: n/2 k = 1 Take log: k = log(n) 58

Merge Sort Runtime Analysis (II)  In each merge call, we iterate len(right) + len(left) times  Each merge is done on total of n elements (the number of elements in each level)  In total: we have log(n) levels of merge, each level takes n steps: It would take about nlog(n) steps 59

Merge Sort Runtime Analysis (III) log(n) levels n operations on each level 60

Insertion Sort  Similar to bubble sort, we go over the list n times  Iteration 1: Compare the first element with the second one.  If they are out of order, swap them  At the end of this iteration, the sub-list of the first two elements is sorted.  Iteration 2: Compare the third element with the elements of its left, from index 1 to index 0. …  Continue till the end of the list 61

Insertion Sort sortednext to be inserted temp sorted less than 10 62

Insertion Sort - implementation 63

Insertion Sort Runtime Analysis  For a list of size n, the outer loop does n iterations  In each iteration i of the outer loop, the inner loop does i iterations (worst case): … + (n-2) + (n-1) = n(n-1)/2  Note that each operation within the inner loop (comparing and swapping) takes constant time c  And what about the best case? (i.e, the list is sorted) Number of steps is a polynomial of 2 nd degree (n 2 ) n steps 64

What have we seen? Algorithm Best Case Runtime Worst Case Runtime Stable Bogo sortninfiniteno Bubble sortnn2n2 yes Insertion sortnn2n2 yes Merge sortnlog(n) yes Quick sortnlog(n)n2n2 no Radix sortk*(n+b) yes 65