Midterm Review 1

Midterm Exam Thursday, October 15 in classroom 75 minutes Exam structure: –TRUE/FALSE questions –short questions on the topics discussed in class –homework-like problems 2

Asymptotic Notations A way to describe behavior of functions in the limit –Abstracts away low-order terms and constant factors –How we indicate running times of algorithms –Describe the running time of an algorithm as n grows to  O notation: asymptotic “less than”: f(n) “≤” g(n)  notation: asymptotic “greater than”: f(n) “≥” g(n)  notation: asymptotic “equality”: f(n) “=” g(n) 3

Exercise Order the following 6 functions in increasing order of their growth rates: equivalence class iff f(n) = Θ(g(n)) –n log 10 n, 4 lg n, n 2, n n, 3 n + n 2. a=0  f(n) = o(g(n)  O(g(n)) but not Θ(g(n)) a=  f(n) = ω(g(n))  Ω(g(n)) but not Θ(g(n)) 0<a<  f(n) = Θ(g(n)) – n log 10 n, {4 lg n, n 2 }, 3 n + n 2, n n 4

Analyzing Algorithms Alg.: MIN (a[1], …, a[n]) m ← a[1]; for i ← 2 to n if a[i] < m then m ← a[i]; Running time: –the number of primitive operations (steps) executed before termination T(n) =1 [first step] + (n) [for loop] + (n-1) [if condition] + (n-1) [the assignment in then] = 3n - 1 Order (rate) of growth: –The leading term of the formula –Expresses the asymptotic behavior of the algorithm T(n) grows like n 5

Running Time Analysis Algorithm Loop2(n) p=1 for i = 1 to 2n p = p*i Algorithm Loop3(n) p=1 for i = 1 to n 2 p = p*i O(n) O(n 2 ) 6

Running Time Analysis Algorithm Loop4(n) s=0 for i = 1 to 2n for j = 1 to i s = s + i O(n 2 ) 7

Recurrences Def.: Recurrence = an equation or inequality that describes a function in terms of its value on smaller inputs, and one or more base cases Recurrences arise when an algorithm contains recursive calls to itself Methods for solving recurrences –Substitution method –Recursion tree method –Master method 8

Example Recurrences T(n) = T(n-1) + n –Recursive algorithm that loops through the input to eliminate one item T(n) = T(n/2) + c –Recursive algorithm that halves the input in one step Θ(n 2 ) Θ(lg n) 9

Analyzing Divide and Conquer Algorithms The recurrence is based on the three steps of the paradigm: –T(n) = running time on a problem of size n –Divide the problem into a subproblems, each of size n/b: takes –Conquer (solve) the subproblems: takes –Combine the solutions: takes  (1)if n ≤ c T(n) = D(n) aT(n/b) C(n) aT(n/b) + D(n) + C(n)otherwise 10

Master Theorem Theorem 4.1 Let a  1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by  n/b  or  n/b . T(n) can be bounded asymptotically in three cases: 1.If f(n) = O(n log b a–  ) for some constant  > 0, then T(n) =  ( n log b a ). 2.If f(n) =  (n log b a ), then T(n) =  ( n log b a lg n). 3.If f(n) =  (n log b a+  ) for some constant  > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b)  c f(n), then T(n) =  (f(n)). Theorem 4.1 Let a  1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by  n/b  or  n/b . T(n) can be bounded asymptotically in three cases: 1.If f(n) = O(n log b a–  ) for some constant  > 0, then T(n) =  ( n log b a ). 2.If f(n) =  (n log b a ), then T(n) =  ( n log b a lg n). 3.If f(n) =  (n log b a+  ) for some constant  > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b)  c f(n), then T(n) =  (f(n)). Whether the problem cost is dominated by the root or by the leaves 11

Example Two different divide-and-conquer algorithms A and B have been designed for solving the problem . –A partitions  into 4 subproblems each of size n/2, where n is the input size for , and it takes a total of  (n 1.5 ) time for the partition and combine steps. –B partitions  into 4 subproblems each of size n/4, and it takes a total of  (n) time for the partition and combine steps. Which algorithm is preferable? Why? A:,  (case 1)  B:,  (case 2)  12

Exercise T(n)=3T(n/2)+ n lg nCase 1: T(n) =Θ(n lg 3 ) T(n)=4T(n/2)+ n 2 √nCase 3: T(n) =Θ(n 2 √n) 13

Sorting Insertion sort –Design approach: –Sorts in place: –Best case: –Worst case: – n 2 comparisons, n 2 exchanges Bubble Sort –Design approach: –Sorts in place: –Running time: – n 2 comparisons, n 2 exchanges Yes  (n)  (n 2 ) incremental Yes  (n 2 ) incremental 14

Sorting Selection sort –Design approach: –Sorts in place: –Running time: – n 2 comparisons, n exchanges Merge Sort –Design approach: –Sorts in place: –Running time: Yes  (n 2 ) incremental No  (nlgn) divide and conquer 15

Quicksort –Idea: –Design approach: –Sorts in place: –Best case: –Worst case: Partition –Running time Randomized Quicksort Yes  (nlgn)  (n 2 ) Divide and conquer  (n) Partition the array A into 2 subarrays A[p..q] and A[q+1..r], such that each element of A[p..q] is smaller than or equal to each element in A[q+1..r]. Then sort the subarrays recursively.  (n lg n) – on average  (n 2 ) – in the worst case 16

Heap Data Structure Def: A heap is a nearly complete binary tree with the following two properties: –Structural property: all levels are full, except possibly the last one, which is filled from left to right –Order (heap) property: for any node x, parent(x) ≥ x Heap

Array Representation of Heaps A heap can be stored as an array A. –Root of tree is A[1] –Parent of A[i] = A[  i/2  ] –Left child of A[i] = A[2i] –Right child of A[i] = A[2i + 1] –Heapsize[A] ≤ length[A] The elements in the subarray A[(  n/2  +1).. n] are leaves The root is the max/min element of the heap A heap is a binary tree that is filled in order 18

Operations on Heaps –MAX-HEAPIFYO(lgn) –BUILD-MAX-HEAPO(n) –HEAP-SORTO(nlgn) –MAX-HEAP-INSERTO(lgn) –HEAP-EXTRACT-MAXO(lgn) –HEAP-INCREASE-KEYO(lgn) –HEAP-MAXIMUMO(1) –You should be able to show how these algorithms perform on a given heap, and tell their running time 19

Questions TRUEFALSE A reverse sorted array is always a max heap. What is the maximum number of nodes possible in a max heap of height h? - max number reached when all levels are full 20

Questions TRUE FALSE The sequence  23, 17, 14, 6, 13, 10, 1, 5, 7, 12  is a heap. 7, which is the right child of 6, is greater than its parent. 21

HW3. 1 Since a heap is an almost-complete binary tree, it has at most 2 h+1 -1 elements (if it is complete) and at least 2 h -1+1=2 h elements (if the lowest level has just 1 element and the other levels are complete) Any leaf can be the smallest element in a max-heap (tree representation). A[n/2+1,…,n] are leaves in a max- heap (array representation) If the array is sorted in non-increasing order, it is a max- heap. If the array is sorted in non-decreasing order, it is a min-heap. If [ ] a max heap –No: 7 is a child of 6 22

HW3. 2 Since we call Max-Heapify on A[i] in the loop, and Max- Heapify requires that the subtrees rooted at both children of A[i] are max-heaps, we must decrease from length[A]/2 to 1. If we increase from 1, the two subtrees rooted at A[2] and A[3]are not max-heaps yet. 23

24

Readings Chapter: 1-4, 6-9, Appendix: A, C 25