Computer Sciences Department1

Sorting algorithm 4 Computer Sciences Department3

Quicksort Chapter 7 4Computer Sciences Department

 The basic idea of quicksort  The run time of quicksort  Analysis of quicksort  Efficient of quicksort 5 Objectives Computer Sciences Department

 The basic idea behind quicksort is: partition; sort one half; sort the other half.  Quicksort is a sorting algorithm whose worst-case running time is O (n 2 ) on an input array of n numbers.  Quicksort, like merge sort, is based on the divide-and- conquer paradigm. 6 Description of quicksort Computer Sciences Department

7 Steps - divide-and-conquer process for sorting a typical sub-array A[p.. r]. Computer Sciences Department

 The steps are:  Pick an element, called a pivot, from the list.  Reorder the list so that all elements with values less than the pivot come before the pivot, while all elements with values greater than the pivot come after it. After this partitioning, the pivot is in its final position. This is called the partition operation.  Recursively sort the sub-list of lesser elements and the sub-list of greater elements. 8 Steps Computer Sciences Department

9 Procedure implements quicksort Computer Sciences Department

10 Partitioning the array Computer Sciences Department

11Computer Sciences Department

 The running time of quicksort depends on whether the partitioning is balanced or unbalanced, and this in turn depends on which elements are used for partitioning.  If the partitioning is balanced, the algorithm runs asymptotically as fast as merge sort.  If the partitioning is unbalanced, however, it can run asymptotically as slowly as insertion sort. 12 Performance of quicksort Computer Sciences Department

13 The worst-case behavior for quicksort occurs when the partitioning routine produces one sub-problem with n − 1 elements and one with 0 elements (only 1 element). In the worst case p will be always be an extreme value: one of the outer partitions will always be empty and the other size n−1 ; total execution time will be O(( n − 1 ) + ( n − 2 ) ) which is O ( n 2 ). Quick sort: Worst Case Computer Sciences Department

 Let us assume that this unbalanced partitioning arises in each recursive call.  The partitioning costs (n) time. Since the recursive call on an array of size 0 just returns, T (0) = (1), and the recurrence for the running time is 14 Partitioning Array Computer Sciences Department

pivot_index = 0 [0] [1] [2] [3] [4] [5] [6] [7] [8] too_big_index too_small_index 15 Example Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 16Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 17Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 18Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 19Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 20Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] [0] [1] [2] [3] [4] [5] [6] [7] [8] 21Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] [0] [1] [2] [3] [4] [5] [6] [7] [8] 22Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. [0] [1] [2] [3] [4] [5] [6] [7] [8] 23Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. [0] [1] [2] [3] [4] [5] [6] [7] [8] 24Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. [0] [1] [2] [3] [4] [5] [6] [7] [8] 25Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. [0] [1] [2] [3] [4] [5] [6] [7] [8] 26Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. [0] [1] [2] [3] [4] [5] [6] [7] [8] 27Computer Sciences Department

pivot_index = 0 too_big_index too_small_index 1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. [0] [1] [2] [3] [4] [5] [6] [7] [8] 28Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 29Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 30Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 31Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 32Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 33Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 34Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 35Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 36Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 37Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. 5.Swap data[too_small_index] and data[pivot_index] pivot_index = 0 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 38Computer Sciences Department

1.While data[too_big_index] <= data[pivot] ++too_big_index 2.While data[too_small_index] > data[pivot] --too_small_index 3.If too_big_index < too_small_index swap data[too_big_index] and data[too_small_index] 4.While too_small_index > too_big_index, go to 1. 5.Swap data[too_small_index] and data[pivot_index] pivot_index = 4 too_big_index too_small_index [0] [1] [2] [3] [4] [5] [6] [7] [8] 39Computer Sciences Department

Partition Result <= data[pivot]> data[pivot] [0] [1] [2] [3] [4] [5] [6] [7] [8] Computer Sciences Department

41 Best-case partitioning In the best case the two partitions will always be about the same size as each other. Computer Sciences Department

 The average-case running time of quicksort is much closer to the best case than to the worst case. Suppose the split is 1/10 : 9/10 (not half and half) “ In the most unbalanced case, each time we perform a partition we divide the list into two sub lists of size 0 and n − 1 ”, “In the most balanced case, each time we perform a partition we divide the list into two nearly equal pieces” and “In fact, it's not necessary to be perfectly balanced; even if each pivot splits the elements with 75% on one side and 25% on the other side (or any other fixed fraction)” 42 Balanced partitioning Computer Sciences Department

43 split: two subarrays of sizes 0 and n − 1. Computer Sciences Department 9

44 A randomized version of quicksort (self study) Computer Sciences Department

 we analyze the behavior of quicksort more rigorously.  We begin with a worst-case analysis, which applies to either QUICKSORT or RANDOMIZED-QUICKSORT, and conclude with an average-case analysis of RANDOMIZED-QUICKSORT 45 Analysis of quicksort Computer Sciences Department

What is the proof of O (n 2 )? Computer Sciences Department46

47Computer Sciences Department

48Computer Sciences Department

 Expected running time and Running time and comparisons (read only) 49Computer Sciences Department

What is the proof of Omega (n lg n)? Computer Sciences Department50

51Computer Sciences Department

52 Solution to Exercise Computer Sciences Department

53Computer Sciences Department

Algorithm steps : Choose a pivot value. Take the value of the middle element as pivot value, but it can be any value, which is in range of sorted values, even if it doesn't present in the array. Partition. Rearrange elements in such a way, that all elements which are lesser than the pivot go to the left part of the array and all elements greater than the pivot, go to the right part of the array. Values equal to the pivot can stay in any part of the array. Notice that array may be divided in to non-equal parts. Sort both parts. Apply quick sort algorithm recursively to the left and the right parts of the array. Computer Sciences Department54

 The basic idea behind quicksort is: partition; sort one half; sort the other half  Quicksort selects one of the entries in the sequence to be the pivot and divides the sequence into two subsequences - one with all elements less than or equal to pivot are placed before it and one with all elements greater than pivot are placed after it.  It is one of the most common sorting algorithms for sequential computers because of its simplicity, low overhead, and optimal average complexity.  Quicksort is O(n lg n) on average and O(n 2 ) in the worst case. At the same time, other sorting algorithms are O(n lg n) in the worst case (like mergesort and heapsort).  A very good partition splits an array up into two equal sized arrays. A bad partition, on other hand, splits an array up into two arrays of very different sizes. The worst partition puts only one element in one array and all other elements in the other array Why is quicksort better than other sorting algorithms in practice? (Efficient Sorting Algorithm) Computer Sciences Department55

Choosing a random pivot minimizes the chance that you will encounter worst-case O(n 2 ) performance (Choosing first or last would cause worst-case performance for nearly-sorted or nearly-reverse-sorted data). Choosing the middle element would also be acceptable in the majority of cases. Computer Sciences Department56 Quicksort “Choosing a pivot”

Never ever choose a fixed pivot - this can be attacked to exploit your algorithm's worst case O(n 2 ) runtime, which is just asking for trouble. Quicksort's worst case runtime occurs when partitioning results in one array of 1 element, and one array of n-1 elements. Suppose you choose the first element as your partition. If someone feeds an array to your algorithm that is in decreasing order, your first pivot will be the biggest, so everything else in the array will move to the left of it. Then when you recursive, the first element will be the biggest again, so once more you put everything to the left of it, and so on.  A better technique is the median-of-3 method, where you pick three elements at random, and choose the middle. You know that the element that you choose won't be the the first or the last, but also, by the central limit theorem, the distribution of the middle element will be normal, which means that you will tend towards the middle (and hence, n lg n time). Computer Sciences Department57 SCIENTIFIC ADVICE