Algorithm Course Algorithms Lecture 3 Sorting Algorithm-1 Dr. Aref Rashad Algorithms Lecture 3 Sorting Algorithm-1

Classification of Algorithms By Function By Implementation By Design Paradigm

Classification by Function Sorting Algorithms Searching Algorithms Selection Algorithms Arithmetic Algorithms Text Algorithms

Classification by implementation Recursive or iterative A recursive algorithm: calls itself repeatedly until a certain limit Iterative algorithms: use repetitive constructs like loops. Deterministic or non-deterministic Deterministic algorithm: solve the problem with a predefined process Non-deterministic algorithm: must perform guesses of best solution at each step through the use of heuristics.

Classification by implementation Logical or procedural Procedural algorithm: follow a certain procedure Logical Algorithm: uses controlled deduction. Apply rules Serial or parallel Serial Algorithms: Based on executing one instruction of an algorithm at a time. Parallel algorithms: take advantage of computer architectures to process several instructions at once

Classification by design paradigm Divide and conquer Repeatedly reduces an instance of a problem to one or more smaller instances of the same problem (usually recursively), until the instances are small enough to solve easily. Sub-problems are independent. Dynamic programming Having the optimal solution to a problem from optimal solutions to subproblems, avoiding recomputing solutions that have already been computed.  Sub-problems are overlaped

Divide-&-conquer works best when all subproblems are independent Divide-&-conquer works best when all subproblems are independent. So, pick partition that makes algorithm most efficient & simply combine solutions to solve entire problem. Divide-&-conquer is best suited for the case when no “overlapping subproblems” are encountered. Dynamic programming is needed when subproblems are dependent; we don’t know where to partition the problem. In dynamic programming algorithms, we typically solve each subproblem only once and store their solutions. But this is at the cost of space

Classification by design paradigm  The greedy method Similar to dynamic programming, but the solutions to the sub-problems do not have to be known at each stage Using graphs Many problems can be modeled as problems on graphs. A graph exploration algorithms are used. This category also includes the search algorithms and backtracking.

Greedy Algorithm solves the sub-problems from top down Greedy Algorithm solves the sub-problems from top down. We first need to find the greedy choice for a problem, then reduce the problem to a smaller one. The solution is obtained when the whole problem disappears. Dynamic Programming solves the sub-problems bottom up. The problem can’t be solved until we find all solutions of sub-problems. The solution comes up when the whole problem appears. Dynamic Programming has to try every possibility before solving the problem. It is much more expensive than greedy. However, there are some problems that greedy cannot solve while dynamic programming can. Therefore, we first try greedy algorithm. If it fails then try dynamic programming.

Classification by design paradigm Linear programming The problem is expressed as a set of linear inequalities and then an attempt is made to maximize or minimize the inputs Probabilistic  Those that make some choices randomly. Heuristic  Whose general purpose is not to find an optimal solution, but an approximate solution where the time or resources to find a perfect solution are not practical.

Algorithm Course Dr. Aref Rashad  Sorting Algorithm- Part 3

Sorting Algorithms classification Comparison based vs Counting based In-Place vs Not-In-Place algorithm Internal structure of the algorithm Data Structure of the algorithm

Comparison based vs Counting based Comparison based sorting technique: Bubble Sort Selection Sort Insertion Sort Heap Sort Quick Sort Merge Sort Counting based sorting technique: Radix Sort Bucket Sort

In-Place vs Not-In-Place algorithm In-place : In In-Place algorithm, no additional data structure or array is required for sorting. Bubble Sort Selection Sort Insertion Sort Heap Sort Quick Sort Not-In-Place : In Not-In-Place algorithm additional ,data structure or array is required for sorting. Radix Sort. Bucket Sort. Merge Sort.

Internal structure of the algorithm Swap-based sorts begin conceptually with the entire list, and exchange particular pairs of elements moving toward a more sorted list. Merge-based sorts creates initial "naturally" or "unnaturally" sorted sequences, and then add either one element (insertion sort) or merge two already sorted sequences. Tree-based sorts store the data, at least conceptually, in a binary tree; either based on heaps, or based on search trees.   Other sorts which use additional key-value information, such as radix or bucket sort.

Techniques of the Algorithm Sorting by Insertion insertion sort, shellsort Sorting by Exchange bubble sort, quicksort Sorting by Selection selection sort, heapsort Sorting by Merging merge sort Sorting by Distribution radix sort

Importance of Sorting 1. Computers spend more time sorting than anything else, historically 25% on mainframes. 2. Sorting is the best studied problem in computer science, with a variety of different algorithms known. 3. Many the interesting ideas can be taught in the context of sorting, such as divide-and-conquer, randomized algorithms, and lower bounds.

Applications of Sorting Searching Binary search lets you test whether an item is in a dictionary Closest pair Given n numbers, find the pair which are closest to each other. Once the numbers are sorted, the closest pair will be next to each other in sorted order Element Uniqueness Given a set of n items, are they all unique or are there any duplicates? Sort them and do a linear scan to check adjacent pairs

Applications of Sorting Mode Given a set of n items, which element occurs the largest number of times? Sort them and do a linear scan to measure the length of all adjacent runs. Median and Selection What is the kth largest item in the set? Once the keys are placed in sorted order in an array, the kth largest can be by looking in the kth position of the array.

Applications of Sorting Convex hulls Given n points in two dimensions, find the smallest area polygon which contains them all. Convex hulls are the most important building block for more sophisticated geometric algorithms. Comparison Functions Alphabetic is the sorting of text strings. There is a built-in sort routine as a library function

The Problem of Sorting

Elementary Sorting Methods (Selection, Insertion, Bubble) Easier to understand the basic mechanisms of sorting. Good for small files. Good for well-structured files that are relatively easy to sort, such as those almost sorted. Can be used to improve efficiency of more powerful methods.

Example of Insertion Sort

Insertion Sort

Insertion Sort Complexity Analysis BEST CASE is when array is already sorted. Best case complexity  O(n) WORST CASE is when the array is already reverse sorted in decreasing order. Worst-Case complexity  O(n2) AVERAGE CASE assumes a random distribution of data. On the average, 1/2 of all the inner loops are performed since 1/2 of the elements A[i] will be greater. Average Case complexity  O(n2)

Example; Insertion Sort {38, 27, 43,  3,  9, 82, 10} 1 2 3 4 5 6 7 {38, 27, 43,  3,  9, 82, 10}   {27, 38, 43,  3,  9, 82, 10},   i: 1 {27, 38, 43,  3,  9, 82, 10},   i: 2 { 3, 27, 38, 43,  9, 82, 10},   i: 3 { 3,  9, 27, 38, 43, 82, 10},   i: 4 { 3,  9, 27, 38, 43, 82, 10},   i: 5 { 3,  9, 10, 27, 38, 43, 82},   i: 6

Selection Sort: Given a list, take the current element and exchange it with the smallest element on the right hand side of the current element.

Selection Sort Analysis The number of comparisons is (n2) in all cases. For each i from 1 to n-1, there is: one exchange and n-i comparisons A total of: n-1 exchanges and (n-1) + (n-2) + . . . + 2 + 1 = n(n-1)/2 comparisons

Example; Selection Sort {38, 27, 43,  3,  9, 82, 10} 1 2 3 4 5 6 7 {38, 27, 43,  3,  9, 82, 10},   i: 0, min: 3,  minValue:  3 { 3, 27, 43, 38,  9, 82, 10},   i: 1, min: 4,  minValue:  9 { 3,  9, 43, 38, 27, 82, 10},   i: 2, min: 6,  minValue: 10 { 3,  9, 10, 38, 27, 82, 43},   i: 3, min: 4,  minValue: 27 { 3,  9, 10, 27, 38, 82, 43},   i: 4, min: 4,  minValue: 38 { 3,  9, 10, 27, 38, 82, 43},   i: 5, min: 6,  minValue: 43 { 3,  9, 10, 27, 38, 43, 82},   i: 6

Step-by-step example Let us take the array of numbers "5 1 4 2 8", and sort the array from lowest number to greatest number using bubble sort. In each step, elements written in bold are being compared. Three passes will be required. First Pass: ( 5 1 4 2 8 )        ( 1 5 4 2 8 ), Here, algorithm compares the first two elements, and swaps since 5 > 1. ( 1 5 4 2 8 )        ( 1 4 5 2 8 ), Swap since 5 > 4 ( 1 4 5 2 8 )        ( 1 4 2 5 8 ), Swap since 5 > 2 ( 1 4 2 5 8 )        ( 1 4 2 5 8 ), Now, since these elements are already in order (8 > 5), algorithm does not swap them. Second Pass: ( 1 4 2 5 8 )        ( 1 4 2 5 8 ) ( 1 4 2 5 8 )        ( 1 2 4 5 8 ), Swap since 4 > 2 ( 1 2 4 5 8 )        ( 1 2 4 5 8 ) ( 1 2 4 5 8 )        ( 1 2 4 5 8 ) Now, the array is already sorted, but our algorithm does not know if it is completed. The algorithm needs one whole pass without any swap to know it is sorted. Third Pass: ( 1 2 4 5 8 )        ( 1 2 4 5 8 ) ( 1 2 4 5 8 )        ( 1 2 4 5 8 ) ( 1 2 4 5 8 )        ( 1 2 4 5 8 ) ( 1 2 4 5 8 )        ( 1 2 4 5 8 )