CS6045: Advanced Algorithms Sorting Algorithms

Sorting Input: sequence of numbers Output: a sorted sequence

Insertion Sort

//next current //go left //find place for current // shift sorted right // go left //put current in place

Example

Correctness Which elements are in sorted order after running each iteration? Loop invariant: the subarray A[1 … j-1] consists of the elements originally in A[1 … j-1] but in sorted order

Correctness To use a loop invariant to prove correctness, we must show three things about it: –Initialization: It is true prior to the first iteration of the loop. –Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration. –Termination: When the loop terminates, the invariant—usually along with the reason that the loop terminated—gives us a useful property that helps show that the algorithm is correct.

Insertion Sort Correctness Initialization: Just before the first iteration, j = 2. The subarray A[1 … j-1] is the single element A[1], which is the element originally in A[1], and it is trivially sorted. Maintenance: To be precise, we would need to state and prove a loop invariant for the “inner” while loop. Rather than getting bogged down in another loop invariant, we instead note that the body of the inner while loop works by moving A[1 … j-1], A[1 … j-2], A[1 … j-3], and so on, by one position to the right until the proper position for key (which has the value that started out in A[j]) is found. At that point, the value of key is placed into this position. Termination: The outer for loop ends when j > n, which occurs when j = n+1. Therefore, j – 1 = n. Plugging n in for j - 1 in the loop invariant, the subarray A[1 … n] consists of the elements originally in A[1 … n] but in sorted order. In other words, the entire array is sorted.

Analyze Algorithm’s Running Time Depends on –input size –input quality (partially ordered) Kinds of analysis –Worst case(standard) –Average case(sometimes) –Best case(never)

Asymptotic Analysis Ignore machine dependent constants Look at growth of T(n) while n   –Drop lower-order terms –Ignore the constant coefficient in the leading term O - big O notation to represent the order of growth

Asymptotic Notations BIG O: O –f = O(g) if f is no faster then g –f / g < some constant BIG OMEGA:  –f =  (g) if f is no slower then g –f / g > some constant BIG Theta:  –f =  (g) if f has the same growth rate as g –some constant < f / g < some constant

Analyze Insertion Sort

Insertion Sort Analysis Best Case –O(n) Worst Case –O(n^2) Average Case –O(n^2)

Merge Sort Divide (into two equal parts) Conquer (solve for each part separately) Combine separate solutions Merge sort –Divide into two equal parts –Sort each part using merge-sort (recursion!!!) –Merge two sorted subsequences

Merge Sort

Example 1

Example 2

Merging Design an algorithm, which takes O(n) time?

Analyze Merge Sort log n n comparisons per level log n levels total runtime = n log n

Quicksort Sorts in place like insertion unlike merge Divide into two parts such that –elements of left part < elements of right part Conquer: recursively solve for each part separately Combine: trivial - do not do anything Quicksort(A,p,r) if p <r then q  Partition(A,p,r) Quicksort(A,p,q-1) Quicksort(A,q+1,r) //divide //conquer left //conquer right

Divide = Partition PARTITION(A,p,r) //Partition array from A[p] to A[r] with pivot A[r] //Result: All elements  original A[r] has index  i x = A[r] i = p - 1 for j = p to r - 1 if A[j] <= x i = i + 1 exchange A[i]  A[j] exchange A[i+1] with A[r] return i + 1

Loop Invariant

Runtime of Quicksort Worst case: –Partition cause one sub-problem with n-1 elements and one with 0 elements –O(n^2) n

Runtime of Quicksort Best case: –every time partition in (almost) equal parts –O(n log n) Average case –O(n log n)

Randomized Quicksort Idea: select a randomly chosen element as the pivot Randomized algorithms: –includes (pseudo) random-number generator –the behavior depends not only from the input but from random-number generator also Simple approach: permute randomly the input –same result but more difficult to analyze

Randomized Quicksort

Partition around first element: O(n^2) worst-case Average case: O(n log n)