1. CS 2430 Day 28

2. Announcements We will have class in ULR 111 on Monday Exam 2 next Friday (sample exam will be distributed next week)

3. Agenda (long term) Computational complexity Searching –Linear search –Binary search Sorting

4. Computational complexity and big O notation

5. Algorithm efficiency Need a way to determine how “efficient” an algorithm so we can compare algorithms Could try timings, but they can vary a lot depending on the input and/or hardware Another approach is to count the number of steps required to process N elements

6. Critical steps To get a feel for the behavior of the algorithm, we just count the dominant instructions –for a sort, it would be the comparison of array elements –for a search, it would be the comparison of an array element to the item being searched for The count will be some function of N, i.e., the number of items being processed

7. Big O The “Big O” of an algorithm is the order of the fastest growing term of the number of critical steps performed (as a function of N) Examples: –2N / 5 + 3 = O(N) –42 = 42N 0 = O(1) –100N 2 + 7N / 2 + 15 = O(N 2 ) –Log N + 31 = O(Log 2 N), where “Log 2 N” is the base-2 logarithm of N

8. Big O examples Search an unsorted array? O(N) Push onto a Stack ? O(1) Remove from a Bag ? O(N) Compute the union of two disjoint sets? ???

9. Matrix multiplication

10. public class Matrix { private double[][] data; // matrix entries private int n; // dimension of matrix, assume square... public Matrix times(Matrix rhs) { Matrix lhs = this; Matrix res = new Matrix(lhs.n); // creates empty Matrix for (int i = 0; i < res.n; i++) for (int j = 0; j < res.n; j++) for (int k = 0; k < lhs.n; k++) res.data[i][j] += lhs.data[i][k] * rhs.data[k][j]; return res; } } Let N denote the matrix dimension, i.e., lhs.n What is the big O of times() ? O(N 3 )

11. Do YOU think it’s possible to do better than O(N 3 )?

12. Yes! Middle school matrix multiplication algorithmO(N 3 ) Strassen algorithm (1969)O(N 2.807 ) Coppersmith–Winograd algorithm (1987, 1990) O(N 2.376 ) Williams algorithm (2011)O(N 2.373 ) (20??)O(N 2 ) – conjectured to be possible!

13. Bigger big O and beyond! Agrawal–Kayal–Saxena primality test algorithm (2002) O(N 10 ) –The exponent continues to decrease Traveling Salesman problem (NP-complete) O(N!) – brute force O(2 N ) – “Dynamic programming” Halting problem (undecidable)Cannot be solved by any computer given any amount of time. No big O! Cannot even be solved by humans (in general)

14. Searching

15. Linear search private static int lsearch(Comparable a[], int n, E x) { for (int i = 0; i < n; i++) if (a[i].compareTo(x) == 0) return i; return -1; } Can put this method in any container class that needs it What is the worst case big O? O(N)

16. Linear search on average What is the average case efficiency of linear search? Given a random array that contains the target element X, how many comparisons needed? Answer = 1*Pr[X is in position 0] + 2*Pr[X is in position 1] + ∙ ∙ ∙ + N*Pr[X is in position N – 1] = ??? Answer = O(N) How did we get this?

17. How to get the answer! Answer = 1*Pr[X is in position 0] + 2*Pr[X is in position 1] + ∙ ∙ ∙ + N*Pr[X is in position N – 1] = 1 / N + 2 / N + ∙ ∙ ∙ + N / N = (1 / N) * N*(N+1) / 2 = (N + 1) / 2 = O(N)

18. Can we do better if the array is sorted?

19. Yes!

20. Binary search Array MUST be in sorted order (either ascending or descending) Algorithm: how do you look up a word in the dictionary? At each iteration of the algorithm, disregard half of the input that is currently under consideration Extremely efficient!

21. Search for “17” 235711131719232931374143472357111317111317 (0 + 14) / 2 == 7 (0 + 6) / 2 == 3 (4 + 6) / 2 == 5 01234567891011121314 0123456 456 6

22. Binary search algorithm private static int bsearch(Comparable a[], int n, E x) { int lo = 0, hi = n - 1; while (lo <= hi) { int mid = (lo + hi) / 2; int result = a[mid].compareTo(x); if (result == 0) return mid; if (result > 0) hi = mid - 1; else lo = mid + 1; } return -1; }

23. Another example

24. private static int bsearch(Comparable a[], int n, E x) { int lo = 0, hi = n - 1; while (lo <= hi) { int mid = (lo + hi) / 2; int result = a[mid].compareTo(x); if (result == 0) return mid; if (result > 0) hi = mid - 1; else lo = mid + 1; } return -1; } ► ► ► ► ► ► ► ► ► ► ► ► ► 23571113171923293137414347 lo mid hi 17 x 01234567891011121314

25. private static int bsearch(Comparable a[], int n, E x) { int lo = 0, hi = n - 1; while (lo <= hi) { int mid = (lo + hi) / 2; int result = a[mid].compareTo(x); if (result == 0) return mid; if (result > 0) hi = mid - 1; else lo = mid + 1; } return -1; } ► ► ► ► ► ► ► ► ► ► ► ► ► 23571113171923293137414347 lo mid hi 35 x 01234567891011121314

26. Binary search efficiency

27. Worst case scenario What if target item not present? How many iterations of the loop? Each time through the loop, roughly half the array elements are disregarded

28. Worst case big O How many times can you divide N by 2 before you get 1? Example: N = 100; 100 / 2 = 50; 50 / 2 = 25; 25 / 2 = 12; 12 / 2 = 6; 6 / 2 = 3; 3 / 2 = 1; Answer: 6 times In general, solve for r: N / 2 r = 1 We get: r = Log 2 N (we’ll just write “Log N”) Final answer: O(Log N)

29. Any questions?

30. Review Linear search: O(N) (worst and average case) Binary search: O(Log N) (worst case) Binary search works with sorted input How to get the input sorted?

31. Next: sorting