1. HOW TO SOLVE IT? Algorithms

2. An Algorithm An algorithm is any well-defined (computational) procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. An algorithm is thus a sequence of (computational) steps that transforms the input into the output.

3. Algorithms The output we would like to obtain as the result of the algorithm is usually the solution of a problem. Many problems do not have closed-form solutions that we can just plug-in the input values to obtain the solution. To obtain the solution requires the performance of a procedure.

4. Algorithms Even the simplest of computational tasks requires algorithms when one thinks of it. Multiple digit addition.  Input: Two integers a and b  Output: An integer that is the sum of a and b  Algorithm? The sorting problem  Input: A sequence of n numbers (a 1, a 2, …, a n )  Output: A permutation (reordering) ( a 1 ’, a 2 ’, …, a n ’ ) of the input sequence such that a 1 ’<= a 2 ’<= …<= a n ’  Algorithm?

5. Algorithms Consider the input sequence (33,22,11, 45,42) When sorting algorithm is applied, it returns the output sequence (11,22,33,42,45) Such an input sequence is called an instance of the sorting problem. In general an instance of a problem consists of the input (satisfying the input requirements of the problem) needed to compute a solution to the problem

6. Algorithms Sorting is a fundamental operation that are used in many computer programs A large number of good sorting algorithms have been developed The best algorithm for a given application depends on  The number of times to be sorted  The extent to which the items are already somehow sorted  Possible restrictions on the values  The kind of storage device used to keep the sequence (RAM, hard drive, tapes, …)  …

7. Algorithms An algorithm is called correct if, for every input sequence it stops with the correct output. A correct algorithm solves any given instance of the problem.

8. Examples of Important Algorithms Sorting algorithms DNA sequencing algorithms Data mining algorithms Cryptology algorithms Optimization algorithms  Simplex Algorithm  Metaheuristics  Algorithms for TSP (Travelling Salesman Problem) Algorithms to analyze Markov chains Simulation algorithms ….

9. Efficiency of Algorithms The devices that execute the algorithms (?) are not infinitely fast and the memory capacity is limited Algorithms devised to solve the same problem often differ in their efficiency. Efficiency is about how many basic operations are done and how much memory is used The number of basic operations required by the algorithm determines the execution time

10. Insertion Sort An efficient algorithm for sorting a small number of elements It works the way many people sort a hand of playing cards We start with an empty left hand and the cards face down on the table We then remove one card at a time from the table and insert it into the correct position between the cards already in the hand

11. Insertion Sort How can we adapt this to a sequence? Assume there is a barrier that separates the sorted part of the sequence from the unsorted part (the left hand from the cards on the table) This barrier will move by one at each step of the algorithm.

12. Insertion Sort - Example Input: (5, 2, 4, 6, 1, 3) Step 1: (5,| 2, 4, 6, 1, 3) Step 2: (2, 5,| 4, 6, 1, 3) Step 3: (2, 4, 5,| 6, 1, 3) Step 4: (2, 4, 5, 6,| 1, 3) Step 5: (1, 2, 4, 5, 6,| 3) Step 6: (1, 2, 3, 4, 5, 6|)

13. Insertion Sort- Pseudocode

14. Loop Invariant for Insertion Sort

15. Loop Invariants We use loop invariants to help us understand why an algorithm is correct. We must show three things to check if a loop invariant is true 1. Initialization: It is true prior to the first iteration of the loop. 2. Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration. 3. Termination: When the loop terminates, the invariant gives us useful property that helps us to show that the algorithm is correct. Is it similar to another concept you know?

16. Analysis of Insertion Sort

19. Analysis of Algorithms In our previous analysis, we looked at both the best case, in which the input array was already sorted, and the worst case, in which the input array was reverse sorted. Usually the worst-case analysis is the one that merits most attention, because: 1. It gives an upper bound on the running time for any input. 2. For some algorithm, worst case occurs very frequently (search algorithms) 3. The average case behaves usually the same with the worst case.

20. Merge-Sort Algorithm Another algorithm for sorting Instead of the incremental approach of insertion sort, it makes use of a divide-and-conquer approach Divide: Divide the n-element sequence into two subsequences of n/2 elements each. Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted sequences to produce the sorted answer.

21. Merge-Sort Algorithm

22. MERGE (A, p, q, r)

24. Loop Invariant for Merge Algorithm

25. Running Time of Merge Algorithm

26. MERGE-SORT (A, p, r)

27. Computational Complexity

28. Counting Sort

29. COUNTING-SORT (A)

30. Counting Sort It is a very good algorithm It is able to run in linear time since it does not do comparisons Yet beware of its limitations

31. Project Work Write MATLAB functions for the three sorting algorithms presented:  Insertion Sort  Merge Sort (In order to implement this sort you will need to write a function for the Merge algorithm)  Counting Sort Create random arrays of elements between 0 and 10 of sizes 10, 1000, 1000000 Compare the times it takes to for each algorithm to sort these three arrays. (Don’t forget that the counting sort necessitates integers) Present your findings in a report.