Unit IV : Searching and sorting Techniques Contents : Need of searching and sorting, Concept of internal and external sorting, sort stability. Searching methods: Linear and binary search algorithms their comparison and complexity analysis Sorting methods: Bubble, selection, insertion, merge, quick, bucket sort and their time and space complexity analysis Objective : To understand frequent operations for accessing and manipulating data in information systems. To understand the comparative performance of various operations on varying spread of data. 11/19/2018

Syllabus Need of searching & sorting Concept of internal & external sorting Sort stability Searching methods: Linear & Binary search, comparision & complexity analysis Sorting Techniques: Bubble, Selection, Insertion, Quick, Merge, Bucket sort and their time & space complexity

Searching Search: locate an item in a list of information Two algorithms : Linear search Binary search

Linear Search Also called the sequential search Starting at the first element, this algorithm sequentially steps through an array examining each element until it locates the value it is searching for.

List of Comparable Items: 7 14 22 5 12 18 40 key Item: key = 5

7 14 22 5 12 18 40 key = 18 A = key key key key key key key == a[5]? 0 1 2 3 4 5 7 14 22 5 12 18 40 key = 18 A = key key key key key key key == a[5]? key == a[0]? key == a[1]? key == a[4]? key == a[3]? key == a[2]? Done!

int linearSearch(int [ ] a, int n, int key) { int i = 0; while(i < n)) if (key == a[i]) return i i++; } return -1;

Number of comparisions: Best Case Analysis: 1 key is at A[0] Worst Case Analysis: n key is at A[size-1] OR not found Average Case Analysis: key can appear in each of the n index positions with equal probability (n + 1)/2

Int binarySearch(int a[ ], int first, int last, int key) { int mid; if (first > last) return false; else { mid = (first + last)/2; if (key = = a[mid]) return true; else if (key < a[mid]) return binarySearch(a, first, mid -1, key); return binarySearch(a, mid + 1, last, key); } //end else return -1; } //end binarySearch

Analysis: After 1 comaparisons  Remaining file size = n / 2 = n / 21 After 2 comaparisons  Remaining file size = n / 4 = n / 22 After 3 comaparisons  Remaining file size = n / 8 = n / 23 …… After k comaparisons  Remaining file size = n / 2k The process terminates when no more partitions are possible i.e. remaining file size = 1 n / 2k = 1 k = log2n Thus, the time complexity is O(log2n). which is very efficient.

Sorting One fundamental problems of computer science is ordering a list of items. Many solutions exist to this problem, known as sorting algorithms. Some sorting algorithms are simple such as the bubble sort. Others, such as the quick sort are extremely complicated, but produce lightning-fast results.

Sorting Algorithms are divided into two categories: Internal Sorts and External sorts.

Sorting Internal Sort: Any sort algorithm which uses main memory exclusively during the sort. This assumes high-speed random access to all memory.

Sorting External Sort: Any sort algorithm which uses external memory, such as tape or disk, during the sort.

Internal Vs External sorting Internal sorting is done in internal memory the internal sorting can reside in main memory internal sorting is independent of time to read/write a record internal sorting takes input only which can be fit into its memory...i.e. it takes small input Egs : Heap sort , Bubble sort, Selection sort, quick sort , shell sort, Insertion sort, Bucket sort external sorting is done in external memory like hard disk or magnetic tape. external use secondary memory external sorting can take as much as large input Egs : Mergesort,Tag Sorts ,Four Tape Sort Polyphase Sort ,ExternalRadix Sort,External Merge.

Sort stability A sort algorithm is said to be “stable” if multiple items which compare as equal will stay in the same order they were in after a sort.

Sorting Sort Stable Example: Sorting on surnames Tom Greene Ann George All the George names maintain their position, although Arthur is less than Peter Tom Greene Ann George Peter George Arthur George Joe Summers Ann George Peter George Arthur George Tom Greene Joe Summers Sorting on surnames

In-place In computer science, an in-place algorithm is an algorithm which transforms input using a data structure with a small, constant amount of extra storage space. The input is usually overwritten by the output as the algorithm executes. An algorithm which is not in-place is sometimes called not-in-place or out-of-place. An algorithm is sometimes informally called in-place as long as it overwrites its input with its output.

Bubble sort Compare each element (except the last one) with its neighbor to the right If they are out of order, swap them This puts the largest element at the very end The last element is now in the correct and final place Compare each element (except the last two) with its neighbor to the right This puts the second largest element next to last The last two elements are now in their correct and final places Compare each element (except the last three) with its neighbor to the right Continue as above until you have no unsorted elements on the left

Example of bubble sort Pass 1 Pass 2 Pass 3 Pass 4 7 2 8 5 4 2 7 5 4 8 (done) 2 7 5 8 4 2 5 4 7 8 2 7 5 4 8

Bubble sort C code Void bubblesort(int a[],int n) { int I,j,temp,f=1; for(i=1;i<n && f==1 ;i++) f=0 for(j=0;j<n-i;j++) if(a[j+1]<a[j]) swap(a[j],a[j+1]) f=1 }

Selection sort The selection sort might swap an array element with itself--this is harmless, and not worth checking for Analysis: The outer loop executes n-1 times The inner loop executes about n/2 times on average (from n to 2 times) Work done in the inner loop is constant (swap two array elements) Time required is roughly (n-1)*(n/2) You should recognize this as O(n2) 7 2 8 5 4 2 7 8 5 4 2 4 8 5 7 2 4 5 8 7 2 4 5 7 8

Selection Sort C code void selectionsort(int arr[], int n) { int i,j,k,temp; for(i=0;i<n-1;i++) k=i; for(j=i+1;j<n;j++) if(arr[j]<arr[k]) k=j; } if(k!=i) temp=arr[i]; arr[i]=arr[k]; arr[k]=temp;

Insertion Sort Idea: like sorting a hand of playing cards Start with an empty left hand and the cards facing down on the table. Remove one card at a time from the table, and insert it into the correct position in the left hand compare it with each of the cards already in the hand, from right to left The cards held in the left hand are sorted these cards were originally the top cards of the pile on the table

Insertion Sort To insert 12, we need to make room for it by moving first 36 and then 24. 6 10 24 36 12

Insertion Sort 6 10 24 36 12

Insertion Sort 24 36 10 6 12

Insertion Sort 5 2 4 6 1 3 input array 5 2 4 6 1 3 at each iteration, the array is divided in two sub-arrays: left sub-array right sub-array unsorted sorted

Insertion Sort

Insert Action: i=1 temp 8 20 8 5 10 7 i = 1, first iteration 8 20 20 5 --- 8 20 5 10 7

Insert Action: i=2 temp 5 8 20 5 10 7 i = 2, second iteration 5 8 20 --- 5 8 20 10 7

Insert Action: i=3 temp 10 5 8 20 10 7 i = 3, third iteration 10 5 8 --- 5 8 10 20 7

Insert Action: i=4 temp 7 5 8 10 20 7 i = 4, forth iteration 7 5 8 10 --- 5 7 8 10 20

Insertion sort C code void insertionsort(int a[],int n) { int i,k,temp; for(k=1;k<n;k++) temp=a[k]; for(i=k-1;i>=0&&temp<x[i];i--) a[i+1]=a[i]; a[i+1]=temp; }

Bucket sort Idea: BucketSort on each digit, least significant to most significant (lsd to msd)

Bucket sort Example (1st pass) by 1’s digit After 1st pass Input data 721 3 123 537 67 478 38 9 478 537 9 1 2 3 4 5 6 7 8 9 721 721 3 123 537 67 478 38 9 3 38 123 67

Bucket Sort Example (2nd pass) by 10’s digit After 1st pass After 2nd pass 3 9 721 123 537 38 67 478 721 3 123 537 67 478 38 9 1 2 3 4 5 6 7 8 9 03 09 721 123 537 38 67 478

Bucket Sort Example (3rd pass) by 100’s digit After 2nd pass After 3rd pass 3 9 721 123 537 38 67 478 3 9 38 67 123 478 537 721 1 2 3 4 5 6 7 8 9 003 009 038 067 123 478 537 721 Invariant: after k passes the low order k digits are sorted.

Quicksort Recursive algorithm consists of four steps: If there is one or less element in the array to be sorted, return immediately. 2. Pick an element in the array to serve as a "pivot" point. (Usually the left-most element in the array is used.)

Quicksort Split the array into two parts - one with elements larger than the pivot and the other with elements smaller than the pivot. 4. Recursively repeat the algorithm for both halves of the original array.

Initial list ‘A': 26 22 12 29 19 33 35 1 2 3 4 6 5 Select 26 as pivot Place Pivot at A[3] QS(A,0,6) 19 22 12 26 29 33 35 1 2 3 4 6 5 QS(A,0,2) QS(A,4,6) Select 19 as pivot Place Pivot at A[1] Select 29 as pivot Place Pivot at A[4] 12 19 1 22 2 29 4 35 5 33 6 QS(A,5,6) QS (A,2,2) Select 35 as pivot Place Pivot at A[6] QS(A,0,0) Done Done 33 5 35 6 Done QS(A,5,5) Done

QS( A,0,6 ) 26 22 12 29 19 33 35 i j 26 22 12 19 29 33 35 i j Pivot Left Subarray Right Subarray Swap Pivot 26 22 12 29 19 33 35 i j Pivot 26 22 12 29 19 33 35 i j Pivot 26 22 12 29 19 33 35 i j Pivot Swap 26 22 12 29 19 35 33 Pivot i j

Quick sort Pros: Extremely fast. Cons: Very complex algorithm,massively recursive.

Merge Sort MergeSort is a divide and conquer method of sorting

MergeSort Algorithm MergeSort is a recursive sorting procedure that uses at most O(n lg(n)) comparisons. To sort an array of n elements, we perform the following steps in sequence: If n < 2 then the array is already sorted. Otherwise, n > 1, and we perform the following three steps in sequence: Sort the left half of the the array using MergeSort. Sort the right half of the the array using MergeSort. Merge the sorted left and right halves.

An example Initial: 25 57 48 37 12 92 86 33 Split: 25 57 48 37 12 92 86 33 Split: 25 57 48 37 12 92 86 33 Split: 25 57 48 37 12 92 86 33 Merge: 25 57 37 48 12 92 33 86 Merge: 25 37 48 57 12 33 86 92 Merge: 12 25 33 37 48 57 86 92

MergeSort

Sorting comparison A’m TC(best) TC(avg) TC(worst) passes stability Bubble sort O(n2) n-1 stable Selection sort Insertion sort Quick sort O(nlogn) Logn/n-1 Unstable Merge sort logn Bucket sort O(n) No of digits in largest no