1. Algorithmics - Lecture 61 LECTURE 6: Analysis of sorting methods

2. Algorithmics - Lecture 62 Outline The problem of sorting Insertion sort – correctness and efficiency analysis Selection sort – correctness and efficiency analysis Bubble sort – correctness and efficiency analysis

3. Algorithmics - Lecture 63 The problem of sorting Let us consider: A sequence of items Each item has a characteristic called sorting key. The values of the sorting key belong to a set on which a total order relationship exists Sorting the sequence = arrange its elements such that the sorting keys are in increasing (or decreasing) order

4. Algorithmics - Lecture 64 The problem of sorting Examples: 1.Sequence of numbers: (5,8,3,1,6) The sorting key is the entire entity (an integer value) The result of the sorting process is (1,3,5,6,8) 2. Sequence of records containing two fields (name and mark): ((Peter,9), (John, 10), (Victor,8), (Adam,9)) There are two possible criteria of sorting here: name and mark Increasing sorting by name: ((Adam,9),(John,10),(Peter,9),(Victor,8)) Decreasing sorting by mark: ((John,10),(Peter,9),(Adam,9),(Victor,8))

5. Algorithmics - Lecture 65 The problem of sorting More formally … Ordering (increasingly) a sequence (x 1,x 2,…,x n ) is equivalent to finding a permutation (p(1),p(2),…,p(n)) of the indices such that key(x p(1) )<=key(x p(2) ) <= … <=key(x p(n) ) In the following we shall consider that the sorting key is the element itself, thus the following condition is satisfied: x p(1) <=x p(2) <= … <=x p(n)

6. Algorithmics - Lecture 66 The problem of sorting Other assumptions: We shall consider that the sequence is stored in a random access memory (e.g. in an array) This means that we will discuss about internal sorting We shall analyze only sorting methods which are in place (the additional space needed for sorting has at most the size of an element).

7. Algorithmics - Lecture 67 Properties of sorting methods: 1.Simplicity The method is simple if it is easy to understand and to implement 2.Efficiency The method is efficient if it doesn’t use a large amount of resources (running time) 3.Naturalness The method is natural if the number of operations depends on how out of order is the initial sequence 4.Stability The method is stable if it preserves the initial ordering of any two elements having identical keys

8. Algorithmics - Lecture 68 Stability Example: Initial configuration: ((Adam,9), (John, 10), (Peter,9), (Victor,8)) Stable sorting : ((John,10),(Adam,9),(Peter,9),(Victor,8)) Unstable sorting : ((John,10), (Peter,9),(Adam,9), (Victor,8))

9. Algorithmics - Lecture 69 Basic sorting methods Are simple, intuitive but not very efficient methods… However, they are the starting point for advanced methods. Examples: Insertion sort Selection sort Bubble sort

10. Algorithmics - Lecture 610 Insertion sort Basic idea: Starting with the second element of the array, each element is inserted in the subsequence which precedes it such that this subsequence is kept sorted (x 1,x 2,…,x i-1,x i,x i+1,…x n ) Already sorted subsequence (destination subsequence) Element to be inserted

11. Algorithmics - Lecture 611 Insertion sort - example 8 5 9 8 18 8 9 8 1 aux=5 5 8 9 8 1 aux=9 5 8 9 8 1 aux=8 5 8 9 9 15 8 8 9 1 aux=1 5 8 8 9 95 8 8 8 9 5 5 8 8 9 1 5 8 8 9 i 2 3 4 5 j i-1i-2i-3i-4 WHILE (j>=1) AND (aux<x[j]) FOR i:=2,n

12. Algorithmics - Lecture 612 Insertion sort – algorithm General structure FOR i:=2,n DO ENDFOR Algorithm Insertion(x[1..n]) FOR i:=2,n DO aux:=x[i] j:=i-1 WHILE (j>=1) AND (aux<x[j)) DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=aux ENDFOR RETURN x[1..n]

13. Algorithmics - Lecture 613 Insertion sort – a variant Insertion(x[1..n]) FOR i:=2,n DO aux:=x[i] j:=i-1 WHILE (j>=1) AND (aux<x[j]) DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=aux ENDFOR RETURN x[1..n] Insertion(x[1..n]) FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE (x[0]<x[j]) DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n]

14. Algorithmics - Lecture 614 Insertion sort – correctness analysis Insertion(x[1..n]) i:=2 {x[1..i-1] sorted} WHILE i<=n aux:=x[i] j:=i-1 {x[1..j] sorted, aux<=x[j+1]<=..<=x[i]} WHILE (j>=1) AND (aux<x[j]) DO x[j+1]:=x[j] {x[1..j-1] sorted, aux<x[j]=x[j+1]<= … <=x[i]} j:=j-1 {x[1..j] sorted, aux<x[j+1]=x[j+2]<= … <=x[i]} Either {aux>=x[j] and x[1]<=…x[j]<=aux<x[j+1]=x[j+2]<=…<=x[i]} or {(j=0) and aux<x[1]=x[2]<=…<=x[i]} ENDWHILE x[j+1]:=aux {x[1]<=x[2]<=…x[j+1]<=x[j+2]<=…<=x[i]} (x[1..i] sorted) i:=i+1 {x[1..i-1] sorted} ENDWHILE RETURN x[1..n]

15. Algorithmics - Lecture 615 Insertion sort – correctness analysis Thus we obtained … An invariant for the outer loop: {x[1..i-1] sorted} An invariant for the inner loop: {x[1..j] sorted, aux<=x[j+1]<=x[j+2]<=..<=x[i]} Both loops are finite: termination function for the outer loop: t(p)=n+1-i p termination function for the inner loop: j p if aux <x[j p ] t(p)= 0 if aux>=x[j p ]

16. Algorithmics - Lecture 616 Insertion sort – efficiency analysis Insertion(x[1..n]) FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE x[0]<x[j] DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n] Input size: n Operations: Comparisons (T C (n)) Movements (of the elements) (T M (n)) For each i (2 <=i <= n): 1 <=T C (n) <=i For all i: (n-1) <=T C (n) <=(n+2)(n-1)/2

17. Algorithmics - Lecture 617 Insertion sort – efficiency analysis Insertion(x[1..n]) FOR i:=2,n DO x[0]:=x[i] j:=i-1 WHILE x[0]<x[j] DO x[j+1]:=x[j] j:=j-1 ENDWHILE x[j+1]:=x[0] ENDFOR RETURN x[1..n] Input size: n Operations: Comparisons (T C (n)) Movements (of the elements) (T M (n)) For each i (2 <=i <= n): 0+2 <=T M (n) <=(i-1)+2=i+1 For all i: 2 (n-1) <=T M (n) <=(n+1)(n+2)/2-3

18. Algorithmics - Lecture 618 Insertion sort – efficiency analysis Comparisons: (n-1) <=T C (n) <=(n+2)(n-1)/2 Movements: 2 (n-1) <=T M (n) <=(n+1)(n+2)/2-3 Total: 3(n-1) <=T(n) <=n 2 +2n-3 Efficiency classes: T(n)  Ω(n) T(n)  O(n 2 )

19. Algorithmics - Lecture 619 Insertion sort – stability 8 5 9 8’ 15 8 9 8’ 1 5 8 8’ 9 1 1 5 8 8’ 9 The insertion method is stable

20. Algorithmics - Lecture 620 Outline The problem of sorting Insertion sort Selection sort Bubble sort

21. Algorithmics - Lecture 621 Basic idea: For each position, i (starting with the first one) the subsequence x[i..n] is scanned in order to identify the minimum. This minimum is swapped with the i-th element. (x 1,x 2,…,x i-1,x i,x i+1,…x n ) Selection sort Already sorted subsequence Subsequence searched for the minimum The position where the minimum will be placed

22. Algorithmics - Lecture 622 Selection sort 8 5 9 8 1 i 1 2 3 4 ji i+1 … 5 FOR j:=i+1,n 8 5 9 8 1 1 5 9 8 8 1 5 8 9 8 1 5 8 8 9 FOR i:=1,n-1

23. Algorithmics - Lecture 623 General structure FOR i:=1,n-1 DO ENDFOR Selection sort Algorithm Selection (x[1..n]) FOR i:=1,n-1 DO k:=i; FOR j:=i+1,n DO IF x[k]>x[j] THEN k:=j ENDIF ENDFOR IF k<>i THEN x[i] x[k] ENDIF ENDFOR RETURN x[1..n]

24. Algorithmics - Lecture 624 Selection sort – correctness analysis Algorithm Selection (x[1..n]) i:=1 {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n} WHILE i<=n-1 DO k:=i j:=i+1 {x[k]<=x[r] for all r=i+1..j-1} WHILE j<=n DO IF x[k]>x[j] THEN k:=j ENDIF j:=j+1 ENDWHILE {x[k]<=x[r] for all r=i+1..n} IF k<>i THEN x[i] x[k] {x[1..i] sorted and x[i]<=x[j], j=i+1..n} i:=i+1 ENDWHILE {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n} RETURN x[1..n]

25. Algorithmics - Lecture 625 Selection sort – correctness analysis Thus we obtained … An invariant for the outer loop: {x[1..i-1] sorted and x[i-1]<=x[j], j=i..n} An invariant for the inner loop: {x[k]<=x[r] for all r=i+1..j-1} Both loops are finite: termination function for the outer loop: t(p)=n-i p termination function for the inner loop: t(p)=n+1-j p

26. Algorithmics - Lecture 626 Input size: n Operations: Comparisons (T C (n)) Movements (of the elements) (T M (n)) For each i (1 <=i <= n-1): T C (n,i) =n-i For all i: T C (n) =n(n-1)/2 Selection sort – efficiency analysis Selection (x[1..n]) FOR i:=1,n-1 DO k:=i FOR j:=i+1,n DO IF x[k]>x[j] THEN k:=j ENDFOR IF k<>i THEN x[i] x[k] ENDFOR RETURN x[1..n]

27. Algorithmics - Lecture 627 Input size: n Operations: Comparisons (T C (n)) Movements (of the elements) (T M (n)) For each i (2 <=i <= n): 0 <= T M (n,i) <=3 For all i: 0 <= T M (n) <= 3(n-1) Selection sort – efficiency analysis Selection (x[1..n]) FOR i:=1,n-1 DO k:=i FOR j:=i+1,n DO IF x[k]>x[j] THEN k:=j ENDFOR IF k<>i THEN x[i] x[k] ENDFOR RETURN x[1..n]

28. Algorithmics - Lecture 628 Comparisons: T C (n) =n(n-1)/2 Movements: 0 <= T M (n) <= 3(n-1) Total: n(n-1)/2 <=T(n) <=n(n-1)/2+3(n-1) Efficiency class: T(n)  (n 2 ) Selection sort – efficiency analysis

29. Algorithmics - Lecture 629 Selection sort - stability 8 5 9 8’1 1 5 9 8’8 1 5 8’9 8 1 5 8’8 9 The method is not stable

30. Algorithmics - Lecture 630 Outline The problem of sorting Insertion sort Selection sort Bubble sort

31. Algorithmics - Lecture 631 Bubble sort Basic idea: The array is scanned from left to right and the adjacent elements are compared. If they are out of order then they are swapped. The scanning process is repeated until the array is sorted. (x 1,x 2,…,x m-1,x m,x m+1,…x n ) Already sorted subsequence

32. Algorithmics - Lecture 632 Bubble sort 8 5 9 8 15 8 9 8 1 i 5 4 3 2 j1 2 …4 FOR j:=1,i-1 5 8 8 9 15 8 8 1 9 5 8 1 8 9 5 1 8 8 9 1 5 8 9 8 FOR i:=n,2,-1

33. Algorithmics - Lecture 633 Bubble sort General structure FOR i:=n,2,-1 DO ENDFOR Algorithm Bubblesort(x[1..n]) FOR i:=n,2,-1 DO FOR j:=1,i-1 DO IF x[j]>x[j+1] THEN x[j] x[j+1] ENDIF ENDFOR RETURN x[1..n]

34. Algorithmics - Lecture 634 Bubble sort – correctness analysis Bubble(x[1..n]) i:=n {x[i..n] is sorted and x[i]>=x[j], j=1..i} WHILE i>=2 DO j:=1 {x[j]>=x[k], k=1..j-1} WHILE j<=i-1 DO IF x[j]>x[j+1] THEN x[j] x[j+1] {x[j+1]>=x[k], k=1..j} j:=j+1 {x[j]>=x[k], k=1..j-1} ENDWHILE {x[i-1]>=x[j], j=1..i-1} {x[i-1..n] is sorted and x[i-1]>=x[j], j=1..i-1} i:=i-1 ENDWHILE {x[i..n] is sorted and x[i]>=x[j], j=1..i} RETURN x[1..n]

35. Algorithmics - Lecture 635 Bubble sort – correctness analysis Thus we obtained … An invariant for the outer loop: {x[i..n] is sorted and x[i]>=x[j], j=1..i} An invariant for the inner loop: {x[j]>=x[k], k=1..j-1} Both loops are finite: termination function for the outer loop: t(p)=i p -1 termination function for the inner loop: t(p)=i p -j p

36. Algorithmics - Lecture 636 Bubble sort – efficiency analysis Bubblesort(x[1..n]) FOR i:=n,2,-1 DO FOR j:=1,i-1 DO IF x[j]>x[j+1] THEN x[j] x[j+1] ENDIF ENDFOR RETURN x[1..n] Input size: n Operations: Comparisons (T C (n)) Movements (of the elements) (T M (n)) For each i (1 <=i <= n-1): T C (n,i) =i-1 For all i: T C (n) =n(n-1)/2

37. Algorithmics - Lecture 637 Bubble sort – efficiency analysis Bubblesort(x[1..n]) FOR i:=n,2,-1 DO FOR j:=1,i-1 DO IF x[j]>x[j+1] THEN x[j] x[j+1] ENDIF ENDFOR RETURN x[1..n] Input size: n Operations: Comparisons (T C (n)) Movements (of the elements) (T M (n)) For each i (2 <= i <= n): 0 <= T M (n,i) <= 3(i-1) For all i: 0 <= T M (n) <= 3n(n-1)/2

38. Algorithmics - Lecture 638 Bubble sort – efficiency analysis Comparisons: T C (n) =n(n-1)/2 Movements: 0 <= T M (n) <= 3n(n-1)/2 Total: n(n-1)/2 <= T(n) <= 2n(n-1) Efficiency classes: T(n)  (n 2 )

39. Algorithmics - Lecture 639 Bubble sort – other variants Idea: repeat the array scanning only if at the last scan at least one swap has been made Bubblesort(x[1..n]) sw:=TRUE WHILE sw=True DO sw:=FALSE FOR j:=1,n-1 DO IF x[j]>x[j+1] THEN x[j] x[j+1] sw:=TRUE ENDIF ENDFOR ENDWHILE RETURN x[1..n] n-1<=T C (n)<=n(n-1) 0<=T M (n)<=3n(n-1)/2 Idea: scan the array only up to the index of the last swap Bubblesort(x[1..n]) t:=n WHILE t>1 m:=t; t:=0; FOR j:=1,m-1 DO IF x[j]>x[j+1] THEN x[j] x[j+1]; t:=j ENDIF ENDFOR ENDWHILE RETURN x[1..n] n-1<=T C (n)<=n(n-1)/2 0<=T M (n)<=3n(n-1)/2

40. Algorithmics - Lecture 640 Bubble sort - stability 8 5 9 8’ 15 8 9 8’ 1 5 8 8’9 15 8 8’ 1 9 5 8 1 8’ 9 5 1 8 8’ 9 1 5 8 9 8’ Bubble sort is stable

41. Algorithmics - Lecture 641 Some useful (or just interesting) links… http://www.sorting-algorithms.com/ http://www.softpanorama.org/Algorithms/sorting.shtml http://www.brian-borowski.com/Software/Sorting/ http://www.youtube.com/watch?v=INHF_5RIxTE ☺ http://boingboing.net/2010/08/19/what-does-a-bubble-s.html ☺

42. Algorithmics - Lecture 642 Next lecture will be on … … algorithm design techniques … recursive algorithms … decrease and conquer