Introduction to Algorithms

Algorithms Algorithms are ways of solving problems. There is a technical definition that basically says an algorithm is a clear set of instructions which if followed will lead to a correct problem solution in a finite amount of time Sorting is a very important problem and has been studied extensively. We begin by looking at a simple sorting algorithm We build up to the algorithm by starting with finding the smallest element in an array 2

Finding the smallest element in an array min = data(1) 'the smallest so far For k = 2 To lastIndex 'test each element to see if it is the min so far If data(k) < min Then 'new min found min = data(k) End If Next 3

Find the Smallest and Make it the First What if we want to find the smallest element and put it first in the array? We’ll do this by switching it with the first element We need to know the index of the smallest element to do this A slight modification of our code for finding the smallest element will let us do this 4

Find the smallest element and its index min = data(1) 'the smallest so far minIndex = 1 For k = 2 To lastNdx 'test each element to see if it is the min so far If data(k) < min Then 'new min found min = data(k) minIndex = k End If Next k 5

Switching the Values of Variables (PROBLEM!!) Consider the following code: varA = 1 varB = 4 varA = varB varB = varA What are the values of varA and varB after I do this? 6

Switching the Values of Variables Doing it right: varA = 1 varB = 4 temp = varB varB = varA varA = temp What are the values of varA and varB after I do this? 7

Now Using the Array ‘*** find the smallest element, value and index min = data(1) 'the smallest so far minIndex = 1 For k = 2 To lastNdx 'test each element to see if it is the min so far If data(k) < min Then 'new min found min = data(k) minIndex = k End If Next ‘*** exchange the smallest element with the first temp = data(1) data(1) = data(minIndex) data(minIndex) = temp 8

Idea for Sorting Find the smallest element in the array, and switch it with the first one Find the smallest element in the rest of the array, and switch it with the second one Etc. This is called selection sort 9

Algorithm picture (1) Here’s an initial array: The smallest element is in index 3. If we switch it with the element in index 1, we get: We now know the first element is the smallest

Algorithm picture (2) Looking at elements 2-5, the smallest is in index 5 Let’s switch with the element in index 2 Now we know the first two are smallest, and are in the right order

Algorithm picture (3) Consider elements 3-5. The smallest is in position 5. Let’s switch with the element in index

Algorithm picture (4) Consider elements 4-5. The smallest is in position 5. Let’s switch with the element in index 4 This finishes sorting the array (why?)

Algorithm Structure We want to work on the whole array, then the array without the first element, then the array without the second element, etc. If we work on a whole array of n elements, that’s a loop from 1 to n. If we work on a whole array minus the first element, that loop is from 2 to n. Next we do 3 to n, etc. 14

Loop Setup A loop from 1 to n looks like: For k = 1 To n Next k Here’s a loop from 2 to n: For k = 2 To n Next k 15

In general… We need a loop that looks like this: For k = j To n Next k for each j going from 1 to n-1. This we can do by using another loop! 16

The Nested Loop Here’s what the structure looks like For j = 1 To n - 1 For k = j + 1 To n Next k Next j 17

Here’s the Code For j = 1 To lastNdx – 1 ‘start with element j min = data(j) 'the largest so far minIndex = j For k = j + 1 To lastNdx ‘look at elements that follow j If data(k) < min Then min = data(k) minIndex = k End If Next k temp = data(j) ‘exchange the smallest element with element j data(j) = data(minIndex) data(minIndex) = temp Next j 18

Tricky Bits Note the -1 and +1 in the loop limits. Getting those right takes some thought Does the code work on arrays with just one element? With two elements? With no elements? (Nothing to sort, but we want to avoid a runtime error.) What if the data is already sorted? 19

Demo: Simple Sort 20

Other Ways of Sorting There are actually many ways of sorting items Sorting is very important so people have put a lot of thought into it Some well-known methods: – Bubble sort – Quicksort – Heapsort – Mergesort – Bucket sort 21

Which Method is Best? With small data sets, the best method is usually the easiest one to program With large data sets, speed becomes an issue We could measure the time with a stopwatch, but the essential factor is the functional form of the time: if n is the length of the list of data, is the time proportional to n? n log n? n 2 ? 22

Comparison of n, n log n, n 2 23

Time for Selection Sort The number of comparisons of data elements in a sorting algorithm is usually proportional to the time On the first loop in Selection Sort, we do n-1 comparisons. The second loop does n-2, etc; the last loop does 1 So the time is roughly proportional to (n-1) + (n-2) + … + 1 = (n^2 – n)/2 The largest power of n is n^2 which dominates the time for this algorithm This means Selection Sort is actually too slow to use on large amounts of data 24

Importance of Algorithms You now know the basics for programming: assignment statements, conditionals, procedures and functions, loops, and arrays This is like knowing the rules for chess or go What you have only started to learn are the tactics and strategies to use these tools effectively Algorithms are the tactics for how to accomplish tasks quickly and correctly 25

Software Engineering Software engineering is about the strategies to control the complexity of designing large programs We’ve been learning a few of these strategies (e.g. naming conventions, principles of program structure, requirements and specifications) Good software engineering allows one person or a large group to produce a complex program that is correct and cost-effective 26