1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity Sorting, Searching, Recursion Intro to Algorithms Selection Sort for int’s for string’s Templated Functions Recursion

2 Sorting Elementary sorting algorithms – Bubble – Insertion* – Selection

3 Bubble Sort const size_t N = ; int A[N]; // ‘A’ then is populated // Sort follows for (size_t i = N - 1; i >= 1; --i) for (size_t j = 0; j < i; ++j) if (A[j] > A[j + 1]) std::swap (A[j], A[j + 1]); Complexity?

4 Insertion Sort for (i = 1; i < N; ++i) { // Invariant: A[0..i) sorted v = A[i]; // Element to place j = i; // Index to place ‘v’ while (j >= 1 && v < A[j–1]) { A[j] = A[j–1]; --j; } // Invariant: v >= A[j-1] or j == 0 A[j] = v; }

5 Selection Sort // Find N-1 smallest and place for (i = 0; i < N - 1; ++i) { min = i; for (j = i+1; j < N; ++j) if (A[j] < A[min]) min = j; //if (i != min) swap (A[i], A[min]); }

6 Searching Sequential – Examine 1 st element, 2 nd, … – Assume int A[N]; – int* pos = std::find (A, A+N, val); Binary – Requires sequence be sorted – More efficient – quantify – int* pos = lower_bound (A, A+N, val); // *pos >= val or end – bool found = binary_search (A, A+N, val); Interpolation – Sorted sequence – O(lg lg (N)) average

7 Binary Search If data are sorted can use Binary Search Look at middle element – If match, return location – If smaller, search left sub-array – If larger, search right sub-array

8 Binary Search Case 1: Match if (target == midValue) return mid;

9 Binary Search Case 2: Target smaller than middle value // Search the left sublist if (target <search sublist arr[first]…arr[mid-1]

10 Binary Search Binary Search Case 3: Target larger than middle value // Search upper sublist if (target > midValue)

11 Illustrating the Binary Search - Successful Search 1.Search for target = 23 Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4. Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

12 Illustrating the Binary Search - Successful Search Step 2: Indices first = 5, last = 9, mid = (5+9)/2 = 7. Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

13 Illustrating the Binary Search - Successful Search Step 3:Indices first = 5, last = 7, mid = (5+7)/2 = 6. Since target = midvalue = 23, a match is found at index mid = 6.

14 Illustrating the Binary Search - Unsuccessful Search Search for target = 4. Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4. Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

15 Illustrating the Binary Search - Unsuccessful Search Step 2: Indices first = 0, last = 4, mid = (0+4)/2 = 2. Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

16 Illustrating the Binary Search - Unsuccessful Search Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1. Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index -1.

17 Binary Search Algorithm int binSearch (const int A[], int first, int last, int target) { int midIndex; int midValue;

18 Binary Search Algorithm // Test for nonempty sublist while (first < last) { midIndex = (first+last)/2; midValue = A[mid]; if (target == midValue) return midIndex; // Determine which sublist to // search

19 Binary Search Algorithm else if (target < midValue) last = midIndex; else first = midIndex + 1; } return -1; } // Note half-open range convention

20 Binary Search (Observation) midIndex = first + 1 * (last – first) / 2; Implicit assumption? Can we do better?

21 Interpolation Search int iSearch (const vector & A, int v) { int loc, s = 0, e = A.size () - 1; if (v < A[s]) return (-1); if (v >= A[e]) s = e; while (s < e) { loc = s + (e - s) * (v – A[s]) / (A[e] – A[s]); if (v > A[loc]) { s = loc + 1; } else if (v < A[loc]) { e = loc - 1; } else return (loc); } if (v != A[s]) s = -1; return (s); }

22 Algorithm Complexity Big-O notation – machine-independent means for expressing efficiency of an algorithm Count key operations or stmts, and relate this count to problem size using growth fn. f(N) = O(g(N)) if there are positive constants c and n 0 such that f(N) = n 0 Express op count as function of input size – f 1 (N) = 3N N – f 2 (N) = 1000 lg(N) + 50 – f 3 (N) = 5 f 1 (N) = O(N 2 ); n 0 = 1, c = 200  3N N = n 0 f 3 (N) = O(1); n 0 = 1, c = 6

23 Big-O Notation

24 Constant Time Algorithms An algorithm is O(1) (constant time) when its running time is independent of input size push_back on vector involves a simple assignment statement  complexity O(1) (if capacity sufficient)

25 Linear Time Algorithms An algorithm is O(N) when its running time is proportional to input size

26 Polynomial Algorithms Algorithms with running time O(N 2 ) are quadratic – practical only for relatively small values of N; whenever N triples, the running time increases by ? Algorithms with running time O(N 3 ) are cubic – efficiency is generally poor; tripling N increases the running time by ?

27 Algorithm Complexity Chart

28 Logarithmic Time Algorithms The logarithm of N, base 2, is commonly used when analyzing computer algorithms Ex.log 2 (2) = lg (2) = 1 log 2 (75) = lg (75) ~= When compared to functions N and N 2, the function lg N grows very slowly

29 Function Templates Same logic is appropriate for different types – Templatize function Example: selection sort logic identical for floats, ints, strings, etc.

30 Selection Sort Algorithm Integer Version void selectionSort (int A[], int n) {... int temp; // int temp used for the exchange for (pass = 0; pass < n-1; ++pass) {... if (A[j] < A[smallIndex]) // Compare integer elements... }

31 Selection Sort Algorithm String Version void selectionSort (string A[], int n) {... string temp; // string temp used for the exchange for (pass = 0; pass < n-1; ++pass) {... if (A[j] < A[smallIndex]) // compare string elements...} }

32 Template Syntax Include keyword template followed list of formal types enclosed in angle brackets In the argument list, each type is preceded by the keyword typename (or class) // Argument list with multiple template // types template

33 Template Syntax Example template void selectionSort (T A[], int n) { int smallIndex; // index of smallest element in the // sublist int pass, j; T temp;

34 Template Syntax Example // pass has the range 0 to n-2 for (pass = 0; pass < n-1; ++pass) { // scan the sublist starting at // index pass smallIndex = pass; // j traverses the sublist // A[pass+1] to A[n-1] for (j = pass+1; j < n; ++j) // update if smaller element found

35 Template Syntax Example if (A[j] < A[smallIndex]) smallIndex = j; // if smallIndex and pass are not // the same location, exchange the // smallest item in the sublist with // A[pass] if (smallIndex != pass) swap (A[pass], A[smallIndex]); }

36 Recursion Must ensure – Recursive calls work on smaller problem – Eventually reach base case Many problems have naturally recursive solutions (difficult to express iteratively)

37 Recursion Examples Power function Binary search Towers of Hanoi Fibonacci numbers (*)

38 Recursive Definition of the Power Function Recursive definition – exponent n = 0 (base case) – n  1 which assumes we already know x n-1 Compute successive powers of x by multiplying the previous value by x

39 Implementing the Recursive Power Function Recursive power: double power (double x, int n) // n is a non-negative integer { if (n == 0) return 1.0; // base case else return x * power (x, n-1); // recursive step } // runtime? how to improve?

Solving Recurrence T(N) = T(N-1) + 1; T(0) = 0 Solve: – T(N) = T(N-1) + 1 – T(N-1) = T(N-2) + 1 – T(N-2) = T(N-3) + 1 – … – T(1) = T(0) + 1 – T(0) = 0 – :: T(N) = N = O(N) – (not good enough) 40

41 Binary Search // Search A [s, e) int bs (int A[], int s, int e, int v) { if (s >= e) return -1; int m = (s + e) / 2; if (v == A[m]) return m; if (v < A[m]) return bs (A, s, m, v); else return bs (A, m+1, e, v); }

Improvement Reformulate definition to cut problem in half each time (?) Suppose N = 2 k for k >= 0 T(N) = T(N/2) + 1; T(0) = 0 Solve: – T(N) = T(N/2) + 1 – T(N/2) = T(N/4) + 1 – T(N/4) = T(N/8) + 1 –…–… – T(1) = T(0) + 1 – T(0) = 0 – :: T(N) = lg(N) + N = O(lg(N)) 42

43 Solving the Tower of Hanoi Puzzle using Recursion Move N-1 discs from A to B (using B as temp) Move disc N from A to C Move N-1 discs from B to C (using A as temp) Recurrence?

44 Fibonacci Numbers using Iteration int fibIter (int n) { // Don’t use recursion! if (n == 0 || n == 1) return n; // Store previous two Fib #s int oneBack = 0, twoBack = 1, current;

45 Fibonacci Numbers using Iteration // Compute successive terms for (int i = 2; i <= n; ++i) { current = oneBack + twoBack; twoBack = oneBack; oneBack = current; } return current; } // Counting adds: N >= 2 // T(N) = N-1 = O(N)