1. CSC 2300 Data Structures & Algorithms March 16, 2007 Chapter 7. Sorting

2. Today – Sorting Insertion sort –  Algorithm  Analysis  Lower bound Shellsort –  Algorithm  Worst case

3. Insertion Sort – Algorithm We want to sort N integers. There are N – 1 passes. For pass p = 1 through N – 1, insertion sort ensures that the elements in positions 0 through p are in sorted order.

4. Insert Sort Routine

5. Insertion Sort – Analysis How many element comparisons are there in the inner loop? At most p for each value of p. Total number of comparisons = 1+2+3+…+(N – 1) = N(N – 1)/2 = Θ(N 2 ). Can you give an input sequence that achieves this bound? Can you give an input sequence that requires Θ(N) time?

6. Average Case Defining the average case is always difficult. What do you think the average-case running time is? Θ(N 2 ) or Θ(N)?

7. Inversion An inversion is an array of numbers is any ordered pair (i,j) having the property that i a[j]. The input list has nine inversions: (34,8), (34,32), (34,21), (64,51), (64,32), (64,21), (51,32), (51,21), and (32,21). This is exactly the number of swaps that need to be performed by insertion sort. Since there is O(N) other work involved in the algorithm, the running time of insertion sort is O(I+N), where I is the number of inversions in the original array.

8. Theorem 7.1 Statement:  The average number of inversions in an array of N distinct elements is N(N – 1)/4. Proof:  For any list L of elements, consider L r, the list in reverse order.  Consider any pair of two elements (x,y) in the list.  In exactly one of L and L r, the ordered pair represents an inversion.  The total number of ordered pairs in L and L r is N(N – 1)/2.  Thus, an average list has half this amount, or N(N – 1)/4 inversions.

9. Theorem 7.2 Statement:  Any algorithm that sorts by exchanging adjacent elements requires Ω(N 2 ) time on average. Proof:  The average number of inversions is initially N(N – 1)/4.  Each swap removes only one inversion.  So Ω(N 2 ) swaps are required.

10. An Idea Theorem 7.2 is valid for an entire class of sorting algorithms that performs only adjacent exchanges. If a sorting algorithm is to run in sub-quadratic, or o(N 2 ), time, what must it do? It must do comparisons and exchanges between elements that are far apart.

11. Shellsort Named after Donald Shell. It works by comparing elements that are distant. The distance between comparisons decreases as the algorithm runs until its last phase, in which adjacent elements are compared. Also called diminishing increment sort.

12. Shellsort – Increment Sequence Shellsort uses a sequence h 1, h 2, …,h t, called the increment sequence. Any increment sequence will do, as long as h 1 =1. Some choices are better than others. After using the increment h k, for every i, we have a[i] ≤ a[i+h k ]; that is, all elements spaced h k apart are sorted. The file is said to be h k -sorted. What is an important property that should be valid? An h k -sorted file that is then h k–1 -sorted remains h k -sorted.

13. Shellsort – Example

14. Shellsort – Original sequence Increment sequence: h t = [N/2], and h k = [h k+1 /2].

15. Theorem 7.3 Statement:  The worst-case running time of Shellsort, using Shell’s increments, is Θ(N 2 ). Proof:  The proof requires showing not only an upper bound on the worst-case running time, but also showing that there exists some input that actually takes Ω(N 2 ) time to run.  We will prove the lower bound by constructing a bad case.  You should read the textbook for the upper bound.

16. Bad Case Choose N to be a power of 2. This makes all the increments even, except for the last increment. Choose as input an array with the N/2 largest numbers in the even positions and the N/2 smallest numbers in the odd positions. For this example, the first position is position 1. So, when we come to the last pass, the N/2 largest numbers are still in even positions and the N/2 smallest numbers are still in odd positions. The i th smallest number (i ≤ N/2) is thus in position 2i – 1 before the beginning of the last pass. Restoring the i th element to its correct place requires moving it i – 1 spaces in the array. Thus, to merely place the N/2 smallest elements in the correct places requires at least 1 + 2 + 3 + (N/2 – 1) = Ω(N 2 ) work.

17. Bad Case – Example Choose N to be a power of 2. This makes all the increments even, except for the last increment. Choose as input an array with the N/2 largest numbers in the even positions and the N/2 smallest numbers in the odd positions. Can you suggest a better sequence?

18. Hibbert’s Increments 1, 3, 7, 15, 31, …, 2 k – 1. Theorem 7.4. The worst case running time of Shell sort using Hibbard’s increments is what? Θ( N 3/2 ).