1. INTRO2CS Tirgul 8 1

2. Searching and Sorting  Tips for debugging  Binary search  Sorting algorithms:  Bogo sort  Bubble sort  Quick sort and maybe  Radix sort  Merge sort 2

3. Tips for Debugging 3  Debugging is a process of finding bugs or defects in a program  We can roughly divide the bugs into two types: 1. Bugs that will cause the program to terminate its execution. This will be followed by an exception, and usually in Python there will be a traceback print of the error, including in which line it occurred and what type of error it is Those are the easy bugs to solve..

4. Tips for Debugging 4  For example:  Division by zero  Index Error  Undefined variable …  There are some examples in tirgul 6

5. Tips for Debugging 5 2. Bugs in the implementation – there is no exception thrown but the program does not do what we wanted it to do  Those are hard to find.  This is where we need to debug..

6. Tips for Debugging - example 6  Say we want to implement a function that gets a list and sums its elements:  No exception will be thrown, but it will not result what we intended

7. Tips for Debugging - example 7  We can add print commands in places we suspect can go wrong:

8. Tips for Debugging - example 8  Breakpoint means a point in the program where the execution is paused and we can retrieve the current values of the variables  We can use Pdb (the Python debugger) module and create breakpoints throughout the program using set_trace() function  In each breakpoint we can ask for the current variable values in the stdout  We can continue execution using ‘continue’

9. Tips for Debugging - example 9 > test.py(4)sum_list() -> for i in lst: (Pdb) 7 7 > test.py(4)sum_list() -> for i in lst: (Pdb) 2 type the variable name i sum_lst continue i Continue to next breakpoint More info: https://docs.python.org/3/library/pdb.html

10. Sorting  A sorting algorithm is an algorithm that puts elements in a certain order:  Lexicographic  Numeric  Chronologic  Sorting is important since many search algorithms are much more efficient (lower running time complexity) when running on a sorted input  We will talk about lists during the presentation, but it is valid to any type of sequence that keeps the order of its elements 10

11. Example  Find the youngest person  Unsorted: go over n rows: worst case takes n steps  Find the person who is 10 years old  Unsorted: go over n rows: worst case takes n steps NameAge Bart10 Homer37 Lisa8 Marge35 Meggie1 11

12. Example  Find the youngest person  Sorted (by age): get the youngest person in the table: takes 1 step  Find the person who is 10 years old  How long would it take on a sorted list? NameAge Meggie1 Lisa8 Bart10 Marge35 Homer37 12

13. Runtime Analysis  It is like “counting” the number of operations needed to be done, given an input of size n  How many iterations or recursive calls  How many operations are done in each iteration or recursive call  We will learn about it more deeply in the future 13

14. Binary Search  Given a sorted list and a key to be found:  Get the middle element  if(key == middle) we are done.  If(key < middle) do binary search on the left half of the list  If(key > middle) do binary search on the right half of the list 14

15. Binary Search - example  A list: [1,1,2,3,5,8,13,21,34,55]  We want to find the index of 3 in the list 1. middle = 8 2. 3 < 8, therefore we will look for it in the left half: [1,1,2,3,5] 3. Search again on the updated list 15

16. Binary Search  Find the index of 3: 11235813213455 middle 11235813213455 11235813213455 middle 16

17. Binary Search – implementation (1) 17

18. Binary Search – implementation (2) 18

19. Binary Search – How long does it take?  In each iteration (or in each recursive call) the search is done on a list that is half the size of the previous one.  Assume the list is of size n  After first iteration the list is of size n/2  After the second iteration the list is of size n/2*(1/2) = n/2 2  After k iterations the list is of size n/2 k  The algorithm is done when the list is of size 1, therefore: n/2 k = 1 Take log: k = log(n) 19

20. Sorting Motivation:  In many applications we would like to get information on a given input, for example to get the grades of a student  Given list of data of size n:  Searching on an unsorted list requires n steps  Searching on a sorted list requires log(n) steps when we use binary search 20

21. Stable Sorting  A sorting algorithm is considered to be stable if the initial order of equal items is preserved  That means that if 2 items are equal and one came before the other in the input, then this order would be kept in the output 21

22. Stable Sorting  Stable sort will always return same solution (permutation) on the same input. 22

23. Stable Sorting  The order of the two 5 cards is preserved  Actually, it is sorted by a certain key but have more than one comparable property 23

24. Stable Sorting  We can implement unstable algorithms as stable ones  For example, we can add to each element the original index as an additional property  This, of course would cost more memory  Stable algorithms usually have higher CPU and/or memory usage than unstable algorithms. 24

25. Bogo Sort  It is also called “random sort”, “slow sort”, or “stupid sort”  Simply shuffle the collection given randomly until it is sorted. 25

26. Bogo Sort - implementation 26

27. Bogo Sort Runtime Analysis  How long would it take?  Is it stable? No 1 iteration if we are lucky n! iterations on average Infinite number of iterations in the worst case 27

28. Bubble Sort  The idea: bubble up the largest element  Iteration 1: Compare each element with its neighbor to the right from index 0 to index n-1  If they are out of order, swap them  At the end of this iteration, the largest element is at the end of the list  If there were no swaps after going over all elements, we are done 28

29. Bubble Sort  Iteration 2: Compare each element with its neighbor to the right from index 0 to index n-2 …  Continue until there are no more elements to swap 29

30. Bubble Sort - example 7 2854 2 7854 2 7854 2 7584 2 7548 2 7548 2 5748 2 5478 2 7548 2 5478 2 4578 2 5478 2 4578 2 4578 (done) 30

31. Bubble Sort - implementation 31

32. Bubble Sort Runtime Analysis (I)  For a list of size n, the outer loop does n iterations  In each iteration i of the outer loop, the inner loop does n-i iterations: (n-1) + (n-2) + … + 1 = n(n-1)/2  Note that each operation within the inner loop (comparing and swapping) takes constant time c Number of steps is a polynomial of 2 nd degree (n 2 ) 32

33. Bubble Sort Runtime Analysis (II)  What would be the number of steps in the best case? (i.e., the input list is sorted)  Start with the first iteration, going over all the list  The list is sorted so there are no swaps n steps are required 33

34. Bubble Sort  Is it stable? Yes, we will not swap elements of equal values  Relatively inefficient, so we would probably use it only in cases where we know that the input is nearly sorted 34

35. Quick Sort  Choose an element from the list called pivot  Partition the list:  All elements < pivot will be on the left  All elements ≥ pivot will be on the right  Recursively call the quicksort function on each part of the list 35

36. Quick Sort - implementation 36

37. Quick Sort – implementation (II) 37

38. Quick Sort – implementation (III) 38

39. Quick Sort – Runtime Analysis (I)  On each level of the recursion, we go over lists that contain total of n elements: About n steps at each level 39

40. Quick Sort – Runtime Analysis (II)  How many levels are there?  It depends on the pivot value:  Lets say we choose each time the median value  Each time the list is divided by half:  n/2  n/4 …… 11  There will be log(n) levels, and each takes n steps It would take about nlog(n) steps 40

41. Quick Sort – Runtime Analysis (III)  Lets say we choose each time an extreme value (smallest or largest) – it is unlikely  Each time we get one list of size 1 and one of size n-1:  n-1  n-2 …… 11  There will be n levels, and each takes n steps It would take about n 2 steps 41

42. Quick Sort  The efficiency is depended on the pivot choice  Is it stable? 42 No, since partition reorders elements within a partition

43. Find Duplicate Values 43  We would like to find duplicates values in a given list. It can be done by two approaches: 1. Compare all possible pairs 2. Sort the list and then search for neighboring elements with the same value

44. Find Duplicate Values (1) 44  How long does it take?  Outer loop iterates n times  For each outer iteration i, inner iteration runs i times: 1 + 2 + … + n-1 = n(n-1)/2

45. Find Duplicate Values (2) 45  How long does that takes?

46. Find Duplicate Values (2) 46  Quicksort takes on average nlog(n)  Make one pass on the sorted data – n steps  In total: n + nlog(n) steps This is much more efficient than ~n 2 steps

47. Radix Sort  Sorts integers by grouping keys by the individual digits which share the same significant position and value  Assumption: the input has d digits ranging from 0 to k : For each position there is a finite number of possible digits, depends on the base of the number (for decimal representation there are 10) 47

48. Radix Sort (assume decimal representation)  Divide all in integers into 10 groups according the least significant digit (why?)  For two numbers with the same digit keep the original order  Repeat the above steps with the next least significant digit 48

49. Radix Sort  Input list: [493,812,715,710,195,437,582,340,385] digitsublist 5715,195,385 6 7437 8 9 [710, 340, 812, 582, 493, 715, 195, 385, 437] Divide by the least significant digit: 49 digitsublist 0710,340 1 2812,582 3493 4

50. Radix Sort  Current list: [710, 340, 812,582,493,715,195,385,437] digitsublist 5 6 7 8582,385 9493,195 [710, 812, 715, 437, 340, 582, 385, 493, 195] Divide by the second least significant digit: 50 digitsublist 0 1710,812,715 2 3437 4340

51. Radix Sort  Current list: [710, 812,715,437,340,582,385,493,195] digitsublist 5582 6 7710, 715 8812 9 [195, 340, 385, 437, 493, 582, 710, 715, 812] Divide by the third least significant (and last) digit: and we are done! 51 digitsublist 0 1195 2 3340, 385 4437, 493

52. Radix Sort - implementation 52

53. Radix Sort Runtime Analysis  Lets note:  n the number of elements in the input  d the number of digits in the largest number  k the base of the number  The outer loop would iterate d times  In each iteration, we go over n elements and rebuild a list from k buckets It would take about d*(n+k) steps For d, k small enough it is linear runtime 53

54. Merge Sort  Divide and conquer algorithm:  Recursively break down a problem into two or more sub-problems of the same (or related) type, until these become simple enough to be solved directly  The solutions to the sub-problems are then combined to give a solution to the original problem. 54

55. Merge Sort  Divide the unsorted list into n lists, each contain 1 element  Repeatedly merge sublists to produce new sorted sublists until there is only 1 sublist remaining. 55

56. Merge Sort Split Merge 56

57. Merge Sort - implementation 57

58. Merge Sort Runtime Analysis (I)  How many times is merge called?  Last level of merge is on lists of size n/2  In previous level, the lists are of size n/2*(1/2) = n/2 2 …  First call is on lists of size 1 Assume we have k levels of merge calls, and we will stop when the lists size is 1: n/2 k = 1 Take log: k = log(n) 58

59. Merge Sort Runtime Analysis (II)  In each merge call, we iterate len(right) + len(left) times  Each merge is done on total of n elements (the number of elements in each level)  In total: we have log(n) levels of merge, each level takes n steps: It would take about nlog(n) steps 59

60. Merge Sort Runtime Analysis (III) log(n) levels n operations on each level 60

61. Insertion Sort  Similar to bubble sort, we go over the list n times  Iteration 1: Compare the first element with the second one.  If they are out of order, swap them  At the end of this iteration, the sub-list of the first two elements is sorted.  Iteration 2: Compare the third element with the elements of its left, from index 1 to index 0. …  Continue till the end of the list 61

62. Insertion Sort 3 471214 2021 33 3810559 232816 sortednext to be inserted 3 47 559 232816 10 temp 3833 21 2014 12 10 sorted less than 10 62

63. Insertion Sort - implementation 63

64. Insertion Sort Runtime Analysis  For a list of size n, the outer loop does n iterations  In each iteration i of the outer loop, the inner loop does i iterations (worst case): 1+ 2 + … + (n-2) + (n-1) = n(n-1)/2  Note that each operation within the inner loop (comparing and swapping) takes constant time c  And what about the best case? (i.e, the list is sorted) Number of steps is a polynomial of 2 nd degree (n 2 ) n steps 64

65. What have we seen? Algorithm Best Case Runtime Worst Case Runtime Stable Bogo sortninfiniteno Bubble sortnn2n2 yes Insertion sortnn2n2 yes Merge sortnlog(n) yes Quick sortnlog(n)n2n2 no Radix sortk*(n+b) yes 65