Algorithms 1.Notion of an algorithm 2.Properties of an algorithm 3.The GCD algorithm 4.Correctness of the GCD algorithm 5.Termination of the GCD algorithm 6.Performance 7.Data structures

1-Notion of an algorithm Definition An algorithm is a clearly specified set of instructions describing the solution to a specific problem. An algorithm takes the input and transforms it into an adequate output, must be independent from any programming language, is written in a level of detail that allows to reproduce it in any programming language, has to be designed so it can be reused and understood by others.

2-Properties of an algorithm Some properties must be satisfied by an algorithm in order to allow a successful execution of the corresponding program: 1.Correctness: if the input conditions are satisfied and the algorithm instructions executed, then the correct output is produced. 2.Termination: the algorithm must terminate after a finite number of steps. Thus, it has to be composed by a finite number of steps. This can be ensured if the algorithm avoids an infinite loop. 3.Performance: Quantification of the space and time complexities.

3-The GCD algorithm: Given two positive integers m and n, find the greatest common divisor, gcd(m,n). A view of the problem: All numbers from 1 up to the smallest of m and n, say n. Naive algorithm: Go through search space (from 1 to n) Keep track of largest number that divides both m and n. Is there a more efficient way of doing this? Yes, the Euclidean Algorithm (Euclid – c.350 B.C.E.)

Pseudocode of the GCD algorithm : Algorithm GreatestCommonDivisor Input: Two positive integers, m and n Ouptut: The gcd of m and n repeat r  m mod n m  n n  r until (r == 0) Output m and STOP.

Example: m = 24, n = 9 r  24 mod 9 = 6 m  9 n  6 r  9 mod 6 = 3 m  6 n  3 r  6 mod 3 = 0 m  3 n  0 Output 3 STOP

How does the algorithm works? Given m and n, how many times does “repeat” execute? First, we need the following result: Theorem 1: If m > n, then m mod n < m/2 Proof: If n  m/2, then the claim follows since: r = m mod n < n Ex. 5  11/2  r = 11 mod 5 = 1 < 5  11/2 If n > m/2, then r = m mod n = m – n < m/2 Ex. 7 > 11/2  r = 11 mod 7 = 4 < 11/2 q.e.d.

Consider sequence of remainders: r0 = m mod n < m/2 r1 = n mod r0 < n/2 r2 = r0 mod r1 < r0/2 r3 = r1 mod r2 < r1/2 r4 = r2 mod r3 < r2/2 r5 = r3 mod r4 < r3/2 ….. Second iteration: r1 < n/2 Third iteration: r2 < r0/2 < m/4 Fourth iteration: r3 < r1/2 < n/4 Fifth iteration: r4 < r2/2 < r0/4 < m/8 Sixth iteration: r5 < r3/2 < r1/4 < n/8

Thus, ri will become 0 in at most 2 log n iterations Example: m = 1989 and n = 1590 Remainder sequence: 399 = 1989 mod 1590  399 < 1989/2 393 = 1590 mod 399  393 < 1590/2 6 = 399 mod 393  6 < 399/2 < 1989/4 3 = 393 mod 6  3 < 393/2 < 1590/4 0 = 6 mod 3  0 < 6/2 < 399/4 < 1989/8 We stop here, but if we continue: < 1590/8 < 1989/16 < 1590/16

Number of Steps: In 2 * 3 steps, ri reduced by factor of 8 2 * 4 ri 16 2 * 5 ri 32 2 * 6 ri 64 …… 2 * log n ri n Thus, ri becomes 0 in at most 2 log n iterations.

4-Correctness of the GCD algorithm. Correctness is an important issue in algorithm design. This means proving that the algorithm works for all legal inputs. Correctness is an important issue in algorithm design. This means proving that the algorithm works for all legal inputs. It is analogous to proving the correctness of a theorem in mathematics! The correctness of the GCD algorithm depends on the following loop invariant: The correctness of the GCD algorithm depends on the following loop invariant: gcd(m,n)=gcd(n,r) where r=m%n

Claim 1: gcd(m,n) = gcd(n,r) Proof: The gcd of the new pair is equal to the gcd of the previous pair. How is this correct? Let us write: m = q * n + r where 0  r  n. This implies that: a common divisor of m and n is also a common divisor of n and r, and vice versa.q.e.d.

5-Termination of the GCD algorithm. Show that algorithm terminates in a finite number of steps. Show that algorithm terminates in a finite number of steps. This must be true for every valid input. This must be true for every valid input. Can we show this for Algorithm GCD? Can we show this for Algorithm GCD? We must show that ri goes to 0 in a finite number of steps. We must show that ri goes to 0 in a finite number of steps.Observe: The sequence of remainders strictly decreases. The sequence of remainders strictly decreases. They are all non-negative. They are all non-negative. Thus ri will become 0 in at most 2 log n steps. Thus ri will become 0 in at most 2 log n steps.

6-Performance of the GCD algorithm. Quantification of performance of the algorithm. Quantification of performance of the algorithm. Crucial parameters: time and space. Crucial parameters: time and space. Called time and space complexity of the algorithm. Called time and space complexity of the algorithm. Will be discussed latter in the course. Will be discussed latter in the course. For example: Time complexity of GCD: Takes at most 2 log n steps, where n < m Thus, worst-case time complexity: O(log n)

7-Data structures The study of different ways of organizing data. The study of different ways of organizing data. Why? Why? Efficiency of algorithm depends on how data is organized. Efficiency of algorithm depends on how data is organized. Reason for studying data structures and algorithms together. Reason for studying data structures and algorithms together. Organic connection between the two areas. Organic connection between the two areas. In the programming language JAVA is studied. In the programming language JAVA is studied. Here: An algorithmic perspective. Here: An algorithmic perspective. Independent of programming language (eg. C, C++, JAVA). Independent of programming language (eg. C, C++, JAVA).

For example: The median of a list of n numbers is a number m such that: n/2 numbers in the list are  m, and n/2 … are  m. Many definitions of median, we take: If n is even: Two medians: lower median and upper median, Then, median is average of lower and upper medians. If n is odd: Both medians (lower and upper) are the same.

Consider this problem: Given a sorted list of n numbers, find the median. A crucial question: How should we store the list? We store it in an array, A, then The median is found in constant time, O(1) !! Median = (A[5]+A[6])/2 = (19+22)/2 =

Whereas in a linked list: Traverse half of the list in n/2 steps, which is O(n) ! Quite simple stated: The way in which data is organized is crucial in complexity. More examples like this will be seen later …. first

Example of problems: Design an efficient algorithm to determine if a list has repeated elements. Given a list of n elements find their minimum (or maximum). Given n points in the plane, find the pair(s) of points which are closest to each other. Given n points in the plane determine if any three are contained in a straight line.