1. © Love Ekenberg The Algorithm Concept, Big O Notation, and Program Verification Love Ekenberg

2. © Love Ekenberg In General These slides provide an overview of the algorithm concept and show applications of the big O notation. A notation is introduced that is common for describing algorithms generally and independently of languages. An important point is that all algorithms can be described using only three constructions: assignment, iteration, and selection. It can be shown, though not in this course, that a language that contains these constructs is sufficiently powerful to construct a so called Turing machine, which is a mathematical representation of the concept of an algorithm. This set of slides also touches on program verification and proof by induction.

3. © Love Ekenberg Algorithms Algorithms are finite sequences of instructions. Let us now look at an algorithm that describes how to sum two integers. The algorithm is the same one that we learn in elementary school. For every integer 0  m  100, let d(m) och u(m) signify the decimal and unit digits respectively, i.e. d(76) = 7 and u(76) = 6. More formally: m = 10d + u (0  d  9, 0  u  9). This yields d(m) = d och u(m) = u.

4. © Love Ekenberg Example Ex: Let a and b be two integers with n digits in a base 10 representation of numbers. Let s = a + b. The algorithm for calculating the sum is: 78153a +37429b 11001memory 115582s

5. © Love Ekenberg Adding two Numbers The general form of the algorithm is: a n-1 a n-2 …a 2 a 1 a 0 a +b n-1 b n-2 …b 2 b 1 b 0 b c n c n-1 c n-1 …c 2 c 1 s n s n-1 s n-2 …s 2 s 1 s 0 s Where: s 0 = u(a 0 + b 0 ) c 1 = d(a 0 + b 0 ) s i = u(a i + b i + c i ), 0<i<n. c i+1 = t(a i + b i + c i ), 1<i s n = c n

6. © Love Ekenberg Description of the Algorithm An informal description of the algorithm is as follows: 1. Calculate s 0 2. Calculate c 1 3. Calculate s 1 using s 1 = u(a 1 + b 1 + c 1 ) 4. Calculate c 2 using c 2 = d(a 1 + b 1 + c 1 ) 5. Calculate s 2 using s 2 = u(a 2 + b 2 + c 2 ) Continue with this until s n has been calculated given the values obtained at each step.

7. © Love Ekenberg Programs All programs can be constructed using the following constructs: –assignment –while-do instructions –if-then-else instructions These constitute the foundations of structured programming

8. © Love Ekenberg Assignment Assignment consists of associating a variable with a value either explicitely as a constant or implicitely as the value associated with another variable. Assignment is written x  q, where x is a variable and q is a value. Examples of assignments. x  y, x  65, x  x + 1 An exemple of an algorihm that only uses assignment. x  n; y  x - 1; x  xy; b  x/2 This algorithm calculates n over 2, i.e, how many combinations of two elements can be chosen from a set of n elements. Think it through!

9. © Love Ekenberg While-do Iterations can be expressed with the instruction: while A do S As long as condition A is satisfied, the sequence of instructions S will be carried out. Ex. x  n; y  0; z  0; while y < x-1 do y  y+1; z  z + y; b  z This algorithm calculates n over 2. Think it through! Use known results for expressing n over 2.

10. © Love Ekenberg If-then-else Selection is expressed with the instruction: if A then S else T If condition A is satisfied, the sequence of instructions S is carried out, otherwise the sequence of instructions T is carried out. Example x <- n; if x is even then y <- x - 1; x <- x/2; else y <- (x - 1)/2 b <- xy This algorithm also calculates n over 2. Think it through!!

11. © Love Ekenberg Tables Tables are commonly used in order to keep track of variable assignments during the execution of an algorithm. x <- m; y <- n; z <- 0; while not y = 0 do z <- z + x; y <- y - 1; p <- z zyx 0nm mn-1m 2mn-2m … nm0m

12. © Love Ekenberg Program Verification Exercise x <- n while not x = 1 do if x is even then x <- x/2 else x <- 3x + 1 b <- x What does b become according to this algorithm??

13. © Love Ekenberg Solution x <- n while not x = 1 do if x is even then x <- x/2 else x <- 3x + 1 b <- x Let n = 15. 15 46 23 70 35 106 53 160 80 40 20 10 5 16 8 4 2 1

14. © Love Ekenberg Induction Will every assignment of n result in b = 1? If that is the case, then this is an unnecessarily complicated algorthm for the calculation of the function f : N -> N, f(n) = 1, for all n. This can be shown by induction. So now lets look an explanation of what proof by induction is.

15. © Love Ekenberg Proof by Induction Proof by induction is actually based on a theorem from introductory courses in mathematics. The interested reader may wish to consult an elementary book on algebra for a proof of this theorem. Theorem: Suppose S is a subset of N that satisfies the following conditions: a) 1 belongs to S b) for every k that belongs to N; if k belongs to S, then k + 1 belongs to S This implies that S = N. (Formally the proof builds on the well-ordering axiom: If X is a non- empty subset of Z and has a lower bound, then X has a least member.) From an implementational perspective, this is not much use, so an example now follows of how it can be used.

16. © Love Ekenberg A Proof by Induction Example Show that 1 + 3 + 5 + … + (2n-1) = n 2. (*) Let P(k) mean that (*) is true for k. Induction basis: P(1) is true since 1 = 1 2. Induction hypothesis: Suppose that P(k) is true. Then it must hold that 1 + 3 + 5 + … + (2k-1) = k 2. What about P(k + 1)? 1 + 3 + 5 + … + (2(k + 1)-1) = 1 + 3 + 5 + … + (2k + 1) = 1 + 3 + 5 + … + (2k - 1) + (2k + 1) = k 2 + (2k + 1) = (k + 1) 2. Therefore P(k + 1) is true. By induction it follows that P(n) is true for all n  1.

17. © Love Ekenberg Induction for Program Verification Proof by induction can now be used for program verification. Show that if n  0, then the following program terminates with s = n + m. x <- n; y <- m; while not x = 0 do (*) x <- x - 1 y <- y + 1 s <- y Let P(k) mean: if the program reaches (*) with the value x = k and y holds any value at all, then the program will terminate with s = k + y.

18. © Love Ekenberg Program Verification Induction basis: P(0) is true since if the program reaches (*) with x = 0, then the while-do loop will not execute, s will be assigned the value of y and the program will terminate with s = y (= 0 + y). Induction hypothesis: Suppose P(k) is true. If the program reaches (*) with x = k + 1 and y, then k + 1 not = 0 and the loop executes. The new assignment becomes: x' = x - 1 = k and y' = y + 1 the program returns to (*) with these values. From the assumption P(k) it follows that the program will terminate with s = x' + y' = k + y + 1 = (k + 1) + y. Therefore P(k + 1) is true and by induction P(n) is true for all n  0. Input was now: x = n, y = m. Since P(n) is true, the desired result is implied.

19. © Love Ekenberg Analysis of Algorithms The efficiency of an algorithm is determined by the relationship between the effort required to solve an instance of a problem and the size of that instance. An instance means a particular case. For example (3457,324) is an instance of a multiplication problem. The size of the instance can be set at 4 (i.e. The number of digits in the largest number) In order to determine the effort an algorithm expends in solving a problem, count the number of significant operations it requires.

20. © Love Ekenberg Complexity A complexity analysis consists primarily of three steps Formulate the problem precisely Define the size of an instance - n Calculate f(n) - the number of significant operations the algorithm needs

21. © Love Ekenberg Example Find the binary representation of a positive integer in decimal form. y <- N; i <- 0 while not y = 0 do if y is even then r i <- 0 else r i <- 1; y <- y-1 y <- y/2 i <- i + 1 k <- i - 1 N 2 <- r k r k-1 …r 1 r 0

22. © Love Ekenberg Solution For a given integer the program calculates: N <- 2q 0 + r 0 q 0 <- 2q 1 + r 1 … q k-1 <- 2q k + r k Each remainder r i is 0 or 1 and the program stops when q k = 0. The binary representation is then N 2 = r k r k-1 …r 1 r 0. There are now two possible instance sizes: 1. The value of N 2. The number n of decimal places in N. If N = (x n-1 x n-2 …x 1 x 0 ) 10 then N lies between 10 n-1 and 10 n - 1. n - 1 is thereby the integral part of log 10 N (here written H(log 10 N)). The number n of decimal places is H(log 10 N) + 1. Think this through!!

23. © Love Ekenberg Solution (cont.) Division by 2 is carried out k + 1 times. This is the number of digits in the binary representation N 2 and k + 1 = H(log 2 N 2 ) + 1. Log 2 N = log 10 N/ log 10 2 = (3,3219…) x log 10 N. The number of operations k + 1, is therefore: 10/3 x log 10 N (when size is measured in terms of N) 10/3 n (when size is measured in terms of n)

24. © Love Ekenberg Efficiency Measurement - Big ‘O’ Notation Let f : N -> N f(n) is O(g(n)) iff there is a postive constant k such that f(n)  kg(n) for all n in N/k, where k is a finite set. Ex. 3n 3 + 20n 2 + 5n +23  (3+20+5+23)n 3, i.e. efficiency is O(n 3 ). Complexity is a critical measure of the usefulness of a program. This can be understood by studying the following table of efficiency and time taken:

25. © Love Ekenberg Efficiency f(n)n=20n=40n=60 n0,00002 sec0,00004 sec 0,00006 sec n 2 0,0004 sec 0,0016 sec 0,0036 sec n 3 0,008 sec 0,064 sec 0,216 sec 2 n 1 sec 12,7 days366 centuries