1. 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection Sort for int’s for string’s Templated Functions Recursion

2. 2 Sorting Elementary sorting algorithms – Bubble – Insertion* – Selection

3. 3 Bubble Sort const size_t N = ; int A[N]; // ‘A’ then is populated // Sort follows for (size_t i = N - 1; i >= 1; --i) for (size_t j = 0; j < i; ++j) if (A[j] > A[j + 1]) std::swap (A[j], A[j + 1]); Complexity?

4. 4 Insertion Sort for (i = 1; i < N; ++i) { // Invariant: A[0..i) sorted v = A[i]; // Element to place j = i; // Index to place ‘v’ while (j >= 1 && v < A[j–1]) { A[j] = A[j–1]; --j; } // Invariant: v >= A[j-1] or j == 0 A[j] = v; }

5. 5 Selection Sort // Find N-1 smallest and place for (i = 0; i < N - 1; ++i) { min = i; for (j = i+1; j < N; ++j) if (A[j] < A[min]) min = j; //if (i != min) swap (A[i], A[min]); }

6. 6 Searching Sequential – Examine 1 st element, 2 nd, … – Assume int A[N]; – int* pos = std::find (A, A+N, val); Binary – Requires sequence be sorted – More efficient – quantify – int* pos = lower_bound (A, A+N, val); // *pos >= val or pos @ end – bool found = binary_search (A, A+N, val); Interpolation – Sorted sequence – O(lg lg (N)) average

7. 7 Binary Search If data are sorted can use Binary Search Look at middle element – If match, return location – If smaller, search left sub-array – If larger, search right sub-array

8. 8 Binary Search Case 1: Match if (target == midValue) return mid;

9. 9 Binary Search Case 2: Target smaller than middle value // Search the left sublist if (target <search sublist arr[first]…arr[mid-1]

10. 10 Binary Search Binary Search Case 3: Target larger than middle value // Search upper sublist if (target > midValue)

11. 11 Illustrating the Binary Search - Successful Search 1.Search for target = 23 Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4. Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

12. 12 Illustrating the Binary Search - Successful Search Step 2: Indices first = 5, last = 9, mid = (5+9)/2 = 7. Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

13. 13 Illustrating the Binary Search - Successful Search Step 3:Indices first = 5, last = 7, mid = (5+7)/2 = 6. Since target = midvalue = 23, a match is found at index mid = 6.

14. 14 Illustrating the Binary Search - Unsuccessful Search Search for target = 4. Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4. Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

15. 15 Illustrating the Binary Search - Unsuccessful Search Step 2: Indices first = 0, last = 4, mid = (0+4)/2 = 2. Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

16. 16 Illustrating the Binary Search - Unsuccessful Search Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1. Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index -1.

17. 17 Binary Search Algorithm int binSearch (const int A[], int first, int last, int target) { int midIndex; int midValue;

18. 18 Binary Search Algorithm // Test for nonempty sublist while (first < last) { midIndex = (first+last)/2; midValue = A[mid]; if (target == midValue) return midIndex; // Determine which sublist to // search

19. 19 Binary Search Algorithm else if (target < midValue) last = midIndex; else first = midIndex + 1; } return -1; } // Note half-open range convention

20. 20 Binary Search (Observation) midIndex = first + 1 * (last – first) / 2; Implicit assumption? Can we do better?

21. 21 Interpolation Search int iSearch (const vector & A, int v) { int loc, s = 0, e = A.size () - 1; if (v < A[s]) return (-1); if (v >= A[e]) s = e; while (s < e) { loc = s + (e - s) * (v – A[s]) / (A[e] – A[s]); if (v > A[loc]) { s = loc + 1; } else if (v < A[loc]) { e = loc - 1; } else return (loc); } if (v != A[s]) s = -1; return (s); }

22. 22 Algorithm Complexity Big-O notation – machine-independent means for expressing efficiency of an algorithm Count key operations or stmts, and relate this count to problem size using growth fn. f(N) = O(g(N)) if there are positive constants c and n 0 such that f(N) = n 0 Express op count as function of input size – f 1 (N) = 3N 2 + 50N + 120 – f 2 (N) = 1000 lg(N) + 50 – f 3 (N) = 5 f 1 (N) = O(N 2 ); n 0 = 1, c = 200  3N 2 + 50N + 120 = n 0 f 3 (N) = O(1); n 0 = 1, c = 6

23. 23 Big-O Notation

24. 24 Constant Time Algorithms An algorithm is O(1) (constant time) when its running time is independent of input size push_back on vector involves a simple assignment statement  complexity O(1) (if capacity sufficient)

25. 25 Linear Time Algorithms An algorithm is O(N) when its running time is proportional to input size

26. 26 Polynomial Algorithms Algorithms with running time O(N 2 ) are quadratic – practical only for relatively small values of N; whenever N triples, the running time increases by ? Algorithms with running time O(N 3 ) are cubic – efficiency is generally poor; tripling N increases the running time by ?

27. 27 Algorithm Complexity Chart

28. 28 Logarithmic Time Algorithms The logarithm of N, base 2, is commonly used when analyzing computer algorithms Ex.log 2 (2) = lg (2) = 1 log 2 (75) = lg (75) ~= 6.2288 When compared to functions N and N 2, the function lg N grows very slowly

29. 29 Function Templates Same logic is appropriate for different types – Templatize function Example: selection sort logic identical for floats, ints, strings, etc.

30. 30 Selection Sort Algorithm Integer Version void selectionSort (int A[], int n) {... int temp; // int temp used for the exchange for (pass = 0; pass < n-1; ++pass) {... if (A[j] < A[smallIndex]) // Compare integer elements... }

31. 31 Selection Sort Algorithm String Version void selectionSort (string A[], int n) {... string temp; // string temp used for the exchange for (pass = 0; pass < n-1; ++pass) {... if (A[j] < A[smallIndex]) // compare string elements...} }

32. 32 Template Syntax Include keyword template followed list of formal types enclosed in angle brackets In the argument list, each type is preceded by the keyword typename (or class) // Argument list with multiple template // types template

33. 33 Template Syntax Example template void selectionSort (T A[], int n) { int smallIndex; // index of smallest element in the // sublist int pass, j; T temp;

34. 34 Template Syntax Example // pass has the range 0 to n-2 for (pass = 0; pass < n-1; ++pass) { // scan the sublist starting at // index pass smallIndex = pass; // j traverses the sublist // A[pass+1] to A[n-1] for (j = pass+1; j < n; ++j) // update if smaller element found

35. 35 Template Syntax Example if (A[j] < A[smallIndex]) smallIndex = j; // if smallIndex and pass are not // the same location, exchange the // smallest item in the sublist with // A[pass] if (smallIndex != pass) swap (A[pass], A[smallIndex]); }

36. 36 Recursion Must ensure – Recursive calls work on smaller problem – Eventually reach base case Many problems have naturally recursive solutions (difficult to express iteratively)

37. 37 Recursion Examples Power function Binary search Towers of Hanoi Fibonacci numbers (*)

38. 38 Recursive Definition of the Power Function Recursive definition – exponent n = 0 (base case) – n  1 which assumes we already know x n-1 Compute successive powers of x by multiplying the previous value by x

39. 39 Implementing the Recursive Power Function Recursive power: double power (double x, int n) // n is a non-negative integer { if (n == 0) return 1.0; // base case else return x * power (x, n-1); // recursive step } // runtime? how to improve?

40. Solving Recurrence T(N) = T(N-1) + 1; T(0) = 0 Solve: – T(N) = T(N-1) + 1 – T(N-1) = T(N-2) + 1 – T(N-2) = T(N-3) + 1 – … – T(1) = T(0) + 1 – T(0) = 0 – :: T(N) = N = O(N) – (not good enough) 40

41. 41 Binary Search // Search A [s, e) int bs (int A[], int s, int e, int v) { if (s >= e) return -1; int m = (s + e) / 2; if (v == A[m]) return m; if (v < A[m]) return bs (A, s, m, v); else return bs (A, m+1, e, v); }

42. Improvement Reformulate definition to cut problem in half each time (?) Suppose N = 2 k for k >= 0 T(N) = T(N/2) + 1; T(0) = 0 Solve: – T(N) = T(N/2) + 1 – T(N/2) = T(N/4) + 1 – T(N/4) = T(N/8) + 1 –…–… – T(1) = T(0) + 1 – T(0) = 0 – :: T(N) = lg(N) + N = O(lg(N)) 42

43. 43 Solving the Tower of Hanoi Puzzle using Recursion Move N-1 discs from A to B (using B as temp) Move disc N from A to C Move N-1 discs from B to C (using A as temp) Recurrence?

44. 44 Fibonacci Numbers using Iteration int fibIter (int n) { // Don’t use recursion! if (n == 0 || n == 1) return n; // Store previous two Fib #s int oneBack = 0, twoBack = 1, current;

45. 45 Fibonacci Numbers using Iteration // Compute successive terms for (int i = 2; i <= n; ++i) { current = oneBack + twoBack; twoBack = oneBack; oneBack = current; } return current; } // Counting adds: N >= 2 // T(N) = N-1 = O(N)