1. Algorithm : Design & Analysis [5]
Insertion Sort
Algorithm : Design & Analysis
[5]

2. In the last class… Recurrence Algorithm and Recursion Equations
Solution of the Recurrence Equations
Guess and proving
Recursion tree
Master theorem
Divide-and-conquer

3. Insertion Sort Comparison-based sorting Insertion sort
Analysis of insertion sorting algorithm
Lower bound of local comparison based sorting algorithm
Shell sort

4. Why Sorting Practical use Enough different algorithms for comparing
Easy to derive the strong lower bounds for worst case and average behavior

5. Ordering Is Out of Order
According to an English dictionary, sorting is defined as a process of separating or arranging things according to class or kind.
Here, “sorting” is “ordering”
A sentence from Knuth:
Since only two of our tape drives were in working order, I was ordered to order more tape units in short order, in order to order the data several orders of magnitude faster.

6. Comparison-Based Algorithm
The class of “algorithms that sort by comparison of keys”
comparing (and, perhaps, copying) the key
no other operations are allowed
The measure of work used for analysis is the number of comparison.

7. As Simple as Inserting

8. Shifting Vacancy: the Specification
int shiftVac(Element[ ] E, int vacant, Key x)
Precondition: vacant is nonnegative
Postconditions: Let xLoc be the value returned to the caller, then:
Elements in E at indexes less than xLoc are in their original positions and have keys less than or equal to x.
Elements in E at positions (xLoc+1,…, vacant) are greater than x and were shifted up by one position from their positions when shiftVac was invoked.

9. Shifting Vacancy: Recursion
int shiftVacRec(Element[] E, int vacant, Key x)
int xLoc
1. if (vacant==0)
xLoc=vacant;
3. else if (E[vacant-1].keyx)
xLoc=vacant;
5. else
E[vacant]=E[vacant-1];
xLoc=shiftVacRec(E,vacant-1,x);
8. Return xLoc
The recursive call is working on a smaller range, so terminating;
The second argument is non-negative, so precondition holding
Worst case frame stack size is (n)

10. Shifting Vacancy: Iteration
int shiftVac(Element[] E, int xindex, Key x)
int vacant, xLoc;
vacant=xindex;
xLoc=0; //Assume failure
while (vacant>0)
if (E[vacant-1].keyx)
xLoc=vacant; //Succeed
break;
E[vacant]=E[vacant-1];
vacant--; //Keep Looking
return xLoc

11. Insertion Sorting: the Algorithm
Input: E(array), n0(size of E)
Output: E, ordered nondecreasingly by keys
Procedure:
void insertSort(Element[] E, int n)
int xindex;
for (xindex=1; xindex<n; xindex++)
Element current=E[xindex];
Key x=current.key;
int xLoc=shiftVac(E,xindex,x);
E[xLoc]=current;
return;

12. Worst-Case Analysis
To find the right position for x in the sorted segment, i comparisons must be done in the worst case.
Sorted (i entries) x
At the beginning, there are n-1 entries in the unsorted segment, so:
The input for which the upper bound is reached does exist, so: W(n)(n2)

13. Average Behavior Assumptions: Sorted (i entries) x
All permutations of the keys are equally likely as input.
There are not different entries with the same keys.
Note: For the (i+1)th interval(leftmost), only i comparisons are needed. x
x may be located in any one of the i+1 intervals(inclusive), assumingly, with the same probability

14. Average Complexity
The average number of comparisons in shiftVac to find the location for the ith element:
For all n-1 insertions:
for the leftmost interval

15. Inversion and Sorting An unsorted sequence E:
x1, x2, x3, …, xn-1, xn
If there are no same keys, for the purpose of sorting, it is a reasonable assumption that {x1, x2, x3, …, xn-1, xn}={1,2,3,…,n-1,n}
<xi, xj> is an inversion if xi>xj, but i<j
All the inversions must be eliminated during the process of sorting

16. Eliminating Inverses: Worst Case
Local comparison is done between two adjacent elements.
At most one inversion is removed by a local comparison.
There do exist inputs with n(n-1)/2 inversions, such as (n,n-1,…,3,2,1)
The worst-case behavior of any sorting algorithm that remove at most one inversion per key comparison must in (n2)

17. Eliminating Inverses: Average
Computing the average number of inversions in inputs of size n (n>1):
Transpose: x1, x2, x3, …, xn-1, xn
xn, xn-1, …, x3, x2, x1
For any i, j, (1jin), the inversion (i,j ) is in exactly one sequence in a transpose pair.
The number of inversions (i,j ) is n(n-1)/2.
So, the average number of inversions in all possible inputs is n(n-1)/4.
The average behavior of any sorting algorithm that remove at most one inversion per key comparison must in (n2)

18. Traveling a Long Way Problem
If a1, a2, …an is a random permutation of {1,2,…n}, what is the average value of
| a1-1|+| a2-2|+…+| a1-n|
The answer is the average net distance traveled by all records during a sorting process.

19. Shellsort: the Strategy7 19 24 13 31 8 82 18 44 63 5 29
h=4 7 18 24 13 5 8 82 19 44 63 31 29
h=3 5 8 24 13 7 18 31 19 44 63 82 29 5 7 18 13 8 24 31 19 29 63 82 44
h=2

20. Shellsort: the Strategy (cont.)
For each subsequence, insertion sorting is use, since, generally, in most cycle, there are not many element out of the order. 5 7 8 13 18 19 29 24 31 44 82 63
h=1 5 7 8 13 18 19 24 29 31 44 63 82
The solution

21. Shellsort: the Algorithm
Input: E, an unsorted array of elements, n0, the number of elements, a sequence of diminishing increments ht, ht-1,…, h1, where h1=1, and t, the number of increments.
Output: E, with the elements in nondecreasing order of their keys.
Procedure:
void shellSort(Element [ ] E, int n, int [ ] h, int t);
int xindex, s;
for (s=t; s1; s--)
for (xindex=h[s]; xindex<n; xindex++)
element current=E[xindex];
key x= current.key;
int xLoc=shiftVacH(E, h[s], xindex,x);
E[xLoc]=current;
return;
Is there any possibility that Shellsort is an improvement on Insertion Sort, since only the last pass has to do the same thing as Insertion Sort, let along the passes done before that?
???
A special version for Shellsort

22. Complexity of Shellsort
The efficiency of Shellsort should be higher than insertSort, since:
More than 1 inverse may be removed by one comparison, when h>1
Sorting with one increment will not undo any of the work done previously when a different increment was used
The behavior of Shellsort is mostly unknown
Function of the sequence of increments used
Influences of “pre-processing”(when h>1) is unknown

23. Invariable Ordering Pairs under Sorting
At least r x’s no less than r different y’s
Unordered
No greater than
Ordered independently
“No greater than” hold
In non-decreasing order

24. Previous k-Ordering Kept
r is the largest integer making the subscript maximal no greater than n
Previous k-Ordering Kept
If a k-ordered file is h-ordered, it remains k-ordered.
That is: assume that a k-ordered sequence a1,a2, … ,an has been h-sorted, then for each i=1,2,…n-k, we have to prove that aiai+k
Non-existing if v+rh>n
Let iu, i+kv (mod h), note:
Last r items
No greater than
First r items

25. Some Results about Shellsort
If an h-ordered array is k-sorted, the array will still be h-ordered.
For a two-pass case of Shellsort(i.e. the increments are 1,h)
The average number of inversions in an h-ordered permutation of {1,2,…n} is
Running time is approximately proportional to the best choice of h is about ,which gives O(n5/3)

26. Home Assignment
pp.208-
4.6
4.8
4.9
4.11
4.45