2. CSE 221/ICT221 Analysis and Design of Algorithms Lecture 05: Analysis of time Complexity of Sorting Algorithms Dr.Surasak Mungsing E-mail: Surasak.mu@spu.ac.th Oct-151

3. 10/6/2015 2 Sorting Algorithms  Bin Sort  Radix Sort  Insertion Sort  Shell Sort  Selection Sort  Heap Sort  Bubble Sort  Quick Sort  Merge Sort CSE221/ICT221 Analysis and Design of Algorithms

4. 10/6/2015 3 CSE221/ICT221 Analysis and Design of Algorithms

5. 10/6/2015 4 Bin Sort A2C5B4J3H3I4D4E3F0G4 FAE H J B D G I C Bin 0Bin 1Bin 2Bin 3Bin 4Bin 5 F0E3A2C5G4I4H3J3B4D4 (a) Input chain (b) Nodes in bins (c) Sorted chain CSE221/ICT221 Analysis and Design of Algorithms

6. 10/6/20155 Radix Sort with r=10 and d=3 21652142511691515124349624 (a) Input chain 24349196116124216425515521 (d) Chain after sorting on most significant digit 51551521621611611652112424425349196 (c) Chain after sorting on second-least significant digit 5219191124342442551521611696 (b) Chain after sorting on least significant digit

7. 10/6/20156 Insertion Sort Concept

8. 10/6/2015 7 CSE221/ICT221 Analysis and Design of Algorithms

9. 10/6/2015 8 Shell Sort Algorithm CSE221/ICT221 Analysis and Design of Algorithms

10. 10/6/2015 9 Shell Sort Algorithm CSE221/ICT221 Analysis and Design of Algorithms

11. 10/6/2015 10 Shell Sort Algorithm CSE221/ICT221 Analysis and Design of Algorithms

12. 10/6/2015 11 CSE221/ICT221 Analysis and Design of Algorithms

13. 10/6/201512 Shell Sort Demo http://e-learning.mfu.ac.th/mflu/1302251/demo/Chap03/ShellSort/ShellSort.html Shell Sort Demo http://e-learning.mfu.ac.th/mflu/1302251/demo/Chap03/ShellSort/ShellSort.html

14. 10/6/2015 13 Selection Sort Concept CSE221/ICT221 Analysis and Design of Algorithms

15. 10/6/201514

16. 10/6/2015 15 Heap Sort Algorithm CSE221/ICT221 Analysis and Design of Algorithms

17. 10/6/2015 16 CSE221/ICT221 Analysis and Design of Algorithms

18. 10/6/201517 Bubble Sort Concept

19. 10/6/201518

20. 10/6/2015 19 Quick sort 43 13 81 31 92 57 65 75 26 0 43 13 81 31 92 57 65 75 26 0 65 0 31 13 26 57 43 92 75 81 Select pivot Partition The fastest known sorting algorithm in practice. CSE221/ICT221 Analysis and Design of Algorithms

21. 10/6/2015 20 Quick sort 0 3126 57 43 92 75 81 65 13 65 0 31 13 26 57 43 92 75 81 0 3126 57 43 13 65 92 75 81 Quick sort small Quick sort large CSE221/ICT221 Analysis and Design of Algorithms

22. 10/6/201521 Quick sort

23. 10/6/2015 22 Quick Sort Partitions CSE221/ICT221 Analysis and Design of Algorithms

24. 10/6/2015 23 Quick sort CSE221/ICT221 Analysis and Design of Algorithms

25. 10/6/2015 24 External Sort: A simple merge CSE221/ICT221 Analysis and Design of Algorithms

26. 10/6/2015 25 Merge Sort CSE221/ICT221 Analysis and Design of Algorithms

27. 10/6/2015 26 Merge Sort CSE221/ICT221 Analysis and Design of Algorithms

28. 10/6/2015 27 CSE221/ICT221 Analysis and Design of Algorithms

29. 10/6/201528

30. 10/6/2015 29 Analysis of Insertion Sort for (int i = 1; i < a.length; i++) {// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; } CSE221/ICT221 Analysis and Design of Algorithms

31. 10/6/2015 30 Complexity  Space/Memory  Time  Count a particular operation  Count number of steps  Asymptotic complexity CSE221/ICT221 Analysis and Design of Algorithms

32. 10/6/2015 31 Comparison Count for (int i = 1; i < a.length; i++) {// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; } CSE221/ICT221 Analysis and Design of Algorithms

33. 10/6/2015 32 Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; How many comparisons are made? CSE221/ICT221 Analysis and Design of Algorithms

34. 10/6/2015 33 Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; number of compares depends on a[]s and t as well as on i CSE221/ICT221 Analysis and Design of Algorithms

35. 10/6/2015 34 Comparison Count  Worst-case count = maximum count  Best-case count = minimum count  Average count CSE221/ICT221 Analysis and Design of Algorithms

36. 10/6/2015 35 Worst-Case Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a = [1, 2, 3, 4] and t = 0  4 compares a = [1,2,3,…,i] and t = 0  i compares CSE221/ICT221 Analysis and Design of Algorithms

37. 10/6/2015 36 Worst-Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; total compares = 1 + 2 + 3 + … + (n-1) = (n-1)n/2 = O(n 2 ) n ( i-1 )  i=2 T(n) = CSE221/ICT221 Analysis and Design of Algorithms

38. 10/6/2015 37 Average-Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; n  i=2 T = i-1 2 = O(n 2 ) CSE221/ICT221 Analysis and Design of Algorithms

39. 10/6/2015 38 43 13 81 31 92 57 65 75 26 0 43 13 81 31 92 57 65 75 26 0 65 0 31 13 26 57 43 92 75 81 Select pivot Partition Analysis of Quick Sort CSE221/ICT221 Analysis and Design of Algorithms

40. 10/6/2015 39 Main Quick Sort Routine private static void quicksort( Comparable [ ] a, int left, int right ) { /* 1*/ if( left + CUTOFF <= right ) { /* 2*/ Comparable pivot = median3( a, left, right ); // Begin partitioning /* 3*/ int i = left, j = right - 1; /* 4*/ for( ; ; ) { /* 5*/ while( a[ ++i ].compareTo( pivot ) < 0 ) { } /* 6*/ while( a[ --j ].compareTo( pivot ) > 0 ) { } /* 7*/ if( i < j ) /* 8*/ swapReferences( a, i, j ); else /* 9*/ break; } /*10*/ swapReferences( a, i, right - 1 ); // Restore pivot /*11*/ quicksort( a, left, i - 1 ); // Sort small elements /*12*/ quicksort( a, i + 1, right ); // Sort large elements } else // Do an insertion sort on the subarray /*13*/ insertionSort( a, left, right ); } CSE221/ICT221 Analysis and Design of Algorithms

41. 10/6/2015 40 Worst-Case Analysis เวลาที่ใช้ในการ run quick sort เท่ากับเวลาที่ใช้ในการทำ recursive call 2 ครั้ง + linear time ที่ใช้ในการเลือก pivot ซึ่งทำให้ basic quick sort relation เท่ากับ T(n) = T(i) + T(n-i-1) + cn ในกรณี Worst- case เช่น การที่ pivot มีค่าน้อยที่สุดเสมอ เวลา ที่ใช้ในการทำ recursion คือ T(n) = T(n-1) + cnn>1 T(n-1)= T(n-2)+c(n-1) T(n-2)= T(n-3)+c(n-2) … T(2) = T(1)+c(2) รวมเวลาทั้งหมด T(n) = T(1) + c n  i=2 i = O(n 2 ) CSE221/ICT221 Analysis and Design of Algorithms

42. 10/6/2015 41 The pivot is in the middle;T(n) = 2 T(n/2) + cn Divide both sides by n; Add all equations; Best-Case Analysis T(n/2) n/2 T(n) n =+ c T(2) 2 T(1) 1 = + c T(n/4) n/4 T(n/2 ) n/2 =+ c T(n/4) n/4 T(n/8) n/8 =+ c … T(n) n T(1) 1 = + c log n T(n) = c n logn + n = O(n log n) CSE221/ICT221 Analysis and Design of Algorithms

43. 10/6/2015 42 Average-Case Analysis (1/4) 2 N N -1  j=0 T( j ) + cN T(N) = NT(N) = N -1  j=0 T( j ) + cN 2 2 (N-1) T(N-1) = 2 + c(N – 1) 2 T( j ) N -2  j=0 Average time of T(i) and T(N-i-1 ) is 1 N N -1  j=0 T( j ) Total time;T(N) = T(i) + T(N-i-1) + c N …………..(1) Therefore …………..(2) …………..(3) …………..(4) …………..(5) CSE221/ICT221 Analysis and Design of Algorithms

44. 10/6/2015 43 Average-Case Analysis (2/4) …………..(6) NT(N) – (N-1) T(N-1) = 2T(N-1) +2cN - c (5) – (4); (7) divides by N(N+1); T(N) N+1 T(N-1) N 2c N+1 = + …………..(8) Rearrange terms in equation and ignore c on the right-hand side; NT(N) = (N+1) T(N-1) +2cN …………..(7) CSE221/ICT221 Analysis and Design of Algorithms

45. 10/6/2015 44 Average-Case Analysis (3/4) T(N) N+1 T(N-1) N 2c N+1 = + T(N-2) N-1 T(N-3) N-2 2c N-1 = + T(N-1 ) N T(N-2) N-1 2c N = + …………..(8) …………..(9) …………..(10)...... T(2) 3 T(1) 2 2c 3 = + …………..(11) Sun equations (8) to (11); …………..(12) N +1  i=3 T(N) N+1 T(1) 2 2c = + CSE221/ICT221 Analysis and Design of Algorithms

46. 10/6/2015 45 Average-Case Analysis (4/4) N +1  i=3 T(N) N+1 T(1) 2 2c = + …………..(12) Sum in equation (12) ia approximately log C (N+1)+  - 3/2, which  is Euler’s constant  0.577 T(N) = O(Nlog N) …………..(13) and T(N) N+1 = O(log N) therefore …………..(14) In summary, time complexity of Quick sort algorithm for Average-Case is T(n) = O(n log n) CSE221/ICT221 Analysis and Design of Algorithms

47. 6-Oct-1546