1. The Fundamentals

2. Algorithms

3. What is an algorithm? An algorithm is “a finite set of precise instructions for performing a computation or for solving a problem” Pseudocode (T: Sözde Kod) Describing an algorithm by using a specific computer language: Complex instructions and difficult to understand. Intemadiate step between Natural Language & Programming Language

4. Algorithm 1: Maximum element Algorithm for finding the maximum element in a list: procedure max (a 1, a 2, …, a n : integers) max := a 1 for i := 2 to n if max < a i then max := a i {max is the largest element} How long does this take? If the list has n elements, worst case scenario is that it takes n “steps”

5. Algorithm 2: Linear search Given a list, find a specific element in the list List does NOT have to be sorted! procedure linear_search (x: integer; a 1, a 2, …, a n : integers) i := 1 while ( i ≤ n and x ≠ a i ) i := i + 1 if i ≤ n then location := i else location := 0 {location is the subscript of the term that equals x, or it is 0 if x is not found}

6. Algorithm 3: Binary search Given a list, find a specific element in the list List MUST be sorted! Each time it iterates through, it cuts the list in half procedure binary_search (x: integer; a 1, a 2, …, a n : increasing integers) i := 1{ i is left endpoint of search interval } j := n{ j is right endpoint of search interval } while i < j begin m :=  (i+j)/2  { m is the point in the middle } if x > a m then i := m+1 else j := m end if x = a i then location := i else location := 0 {location is the subscript that equals x, or it is 0 if x is not found}

7. Binary search running time How long does this take (worst case)? If the list has 8 elements It takes 3 steps If the list has 16 elements It takes 4 steps If the list has 64 elements It takes 6 steps If the list has n elements It takes log 2 n steps

8. Sorting Algorithms Sort a sequence A for a given order criteria Bubble Sort, Insertion Sort, …..

9. Growth of Functions Complexity of Algorithms

10. How does one measure algorithms We can measure time how long it takes a computer What if the computer is doing other things? And what happens if you get a faster computer? A 3 Ghz Windows machine chip will run an algorithm at a different speed than a 3 Ghz Macintosh We can measure how many machine instructions an algorithm takes Different CPUs will require different amount of machine instructions for the same algorithm We can loosely define a “step” as a single computer operation A comparison, an assignment, etc. Regardless of how many machine instructions it translates into An efficient algorithm on a slow computer will always beat an inefficient algorithm on a fast computer

11. Binary Search running time The binary search takes log 2 n “steps” Let’s say the binary search takes the following number of steps on specific CPUs: Intel Pentium IV CPU: 58* log 2 n /2 Motorola CPU: 84.4*(log 2 n + 1)/2 Intel Pentium V CPU: 44*(log 2 n)/2 Notice that each has an log 2 n term As n increases, the other terms will drop out As processors change, the constants will always change The exponent on n will not

12. Big-Oh notation If f(x) and g(x) are two functions of a single variable, the statement f(x)  O(g(x)) means that  k  R,  c  R,  x  R, x  k  0  f(x)|  c  g(x)|. Informally c g(x) is greater than f(x) for sufficiently large x. f(x) grows no faster than g(x), as x gets large. How proof goes Need to find k and c to show f(x)  O(g(x)). Conventionally people use f(x)  O(g(x)).

13. Big-Oh proofs

15. A variant of the last question Show that f(x) = 3x+7 is O(x 2 ) In other words, show that 3x+7 ≤ c*x 2

16. What that means If a function is O(x) Then it is also O(x 2 ) And it is also O(x 3 ) Meaning a O(x) function will grow at a slower or equal to the rate x, x 2, x 3, etc.

17. Function growth rates For input size n = 1000 O(1)1 O(log n)≈10 O(n)10 3 O(n log n)≈10 4 O(n 2 )10 6 O(n 3 )10 9 O(n 4 )10 12 O(n c )10 3*c c is a consant 2 n ≈10 301 n!≈10 2568 n n 10 3000 Many interesting problems fall into this category

18. Properties of Big-O If f is O(g) and g is O(f) then O(f) = O(g) Big O, as a relation, is transitive: f is O(g) and g is O(h), then f is O(h) The set O(g) is closed under addition: If f is O(g) and h is O(g) then f + h is O(g) The set O(g) is closed under multiplication by a scalar a (real number): If f is O(g) then af is O(g)

19. Theorem If f1 is O(g1) and f2 is O(g2) then i) f1f2 is O(g1g2) ii) f1 + f2 is O(max{ g1, g2}) Proof of ii There is a k1 and C1 such that 1. f1(n) k1. There is a k2 and C2 such that 2. f2(n) k2. We must find a k3 and C3 such that 3. f1(n)f2(n) k3

20. Proof (cont.) We use the inequality if 0 < a < b and 0 < c < d then ac < bd to conclude that f1(n)f2(n) < C1C2g1(n)g2(n) as long as k > max{k1, k2} so that both inequalities 1 and 2 hold at the same time. Therefore, choose C3 = C1C2 and k3 = max{k1, k2}.

21. Big  and  notation If f(x) and g(x) are two functions of a single variable, the statement f(x)  (g(x)) means that  k  R,  c  R,  x  R, x  k   f(x)|  c  g(x)|. Informally, f(x) is greater than c g(x) for sufficiently large x. f(x) = 8x 3 + 5x 2 + 7 is  (x 3 ). since 8x 3 + 5x 2 + 7 > 8x 3 for all x > 0. f(x)  (g(x)) is equivalent to g(x)  O(f(x)). f(x) =  (g(x)) if f(x) = O(g(x)) and f(x) =  (g(x)). We can find c 1 and c 2 such that c 1 |g(x)|  |f(x)|  c 2 |g(x)|. 3x 2 + 8xlog x =  (x 2 ). 3x 2 +8xlog x 1  3x 2 +8xlog x =O(x 2 ). 3x 2 +8xlog x > x 2 for x > 1.  3x 2 +8xlog x =  (x 2 ).

22. Little bit of Complexity Efficiency of your algorithm? Given input size, Time complexity: Time used by your computer to solve the problem We use number of operations… Memory (space) complexity: Memory used by your computer to solve the problem We use number of variables allocated…

23. Example Find the maximal element in a set max = a1, i = 2 While (i ≤ n) if (max < ai) max = ai i++ Time complexity: 2 (n – 1)+1 comparisons (one to determine at the end of the list has not been reached, another to determine whether to update max)=  (n) Memory complexity: 2 =  (1)

24. Example 2: Linear Search procedure linear search (x: integer, a1, a2, …, an: distinct integers) i := 1t1 while (i ≤ n and x ≠ ai)t2 i := i + 1t3 if i ≤ n then location := it4 else location := 0t5 return locationt6

25. Linear search analysis Worst case time complexity order: Best case: Average case, if item is present:

26. Understanding the complexity of Algorithms Tractable Problems Polynomial worst-case complexity (P Class)  (n c ), c ≥ 1 constant [Eg.Bubble Sort Alg. is  (n 2 )] Intractable Problems Unsolvable Problems No algorithms exists for solving them Halting Problem Class NP: Problems which a solution can be checked in polynomial time Class NP-complete: any of these problems can be solved in polynomial time, then problems in class NP can be solved

27. Integers and Division

28. Division Def: a (a  0) divides b if  c such that b = ac. a | b: a divides b 3 | 7? 3 | 12? 3 | 0? 0 | 3? Theorem: Let a, b, c be integers. Then a | b, a | c  a | (b + c). a | b  a | bc  integer c. a | b, b | c  a | c.

29. Division Let a, b, c be integers. Then a | b, a | c  a | (b + c).

30. Division Let a, b, c be integers. Then a | b  a | bc  integer c. Proof Suppose a|b. By definition, there is a number n such that b = an. Multiply both sides by c to get bc = anc = a (nc ). Consequently, bc has been expressed as a times the integer nc so by definition of “|”, a|bc

31. Division Let a, b, c be integers. Then a | b, b | c  a | c.

32. Division a | b and b | c  a | (mb + nc) for all integer m, n.

33. Primes and Greatest Common Divisors

34. Showing a number is prime Show that 113 is prime Solution The only prime factors less than  113 = 10.63 are 2, 3, 5, and 7 Neither of these divide 113 evenly Thus, by the fundamental theorem of arithmetic, 113 must be prime

35. Primes are infinite Theorem (Euclid): There are infinitely many prime numbers (Proof) Proof by contradiction Assume there are a finite number of primes List them as follows: p 1, p 2 …, p n. Consider the number q = p 1 p 2 … p n + 1 Since we have only finite number of primes and q is not one of them, p i should divide q for some i. Therefore, p i | (q - p 1 p 2 … p n ) = 1.

36. The prime number theorem The radio of the number of primes not exceeding x and x/ln(x) approaches 1 as x grows without bound Rephrased: the number of prime numbers less than x is approximately x/ln(x) When x = 2 512, # of primes = 2 512 /512  2 503

37. Greatest common divisor The greatest common divisor of two integers a and b is the largest integer d such that : d | a and d | b Denoted by gcd(a,b) Examples gcd (24, 36) = 12 gcd (17, 22) = 1 gcd (100, 17) = 1

38. Relative primes Two numbers are relatively prime if they don’t have any common factors (other than 1) Rephrased: a and b are relatively prime if gcd (a,b) = 1 gcd (25, 39) = 1, so 25 and 39 are relatively prime

39. Pairwise relative prime A set of integers a 1, a 2, … a n are pairwise relatively prime if, for all pairs of numbers, they are relatively prime Formally: The integers a 1, a 2, … a n are pairwise relatively prime if gcd(a i, a j ) = 1 whenever 1 ≤ i < j ≤ n. Example: are 10, 17, and 21 pairwise relatively prime? gcd(10,17) = 1, gcd (17, 21) = 1, and gcd (21, 10) = 1 Thus, they are pairwise relatively prime Example: are 10, 19, and 24 pairwise relatively prime? Since gcd(10,24) ≠ 1, they are not

40. More on gcd’s Given two numbers a and b, rewrite them as: Example: gcd (120, 500) 120 = 2 3 *3*5 = 2 3 *3 1 *5 1 500 = 2 2 *5 3 = 2 2 *3 0 *5 3 Then compute the gcd by the following formula: Example: gcd(120,500) = 2 min(3,2) 3 min(1,0) 5 min(1,3) = 2 2 3 0 5 1 = 20

41. Least common multiple The least common multiple of the positive integers a and b is the smallest positive integer that is divisible by both a and b. Denoted by lcm (a, b) Example: lcm(10, 25) = 50 What is lcm (95256, 432)? 95256 = 2 3 3 5 7 2, 432=2 4 3 3 lcm (2 3 3 5 7 2, 2 4 3 3 ) = 2 max(3,4) 3 max(5,3) 7 max(2,0) = 2 4 3 5 7 2 = 190512

42. lcm and gcd theorem Let a and b be positive integers. Then a*b = gcd(a,b) * lcm (a, b) Example: gcd (10,25) = 5, lcm (10,25) = 50 10*25 = 5*50 Example: gcd (95256, 432) = 216, lcm (95256, 432) = 190512 95256*432 = 216*190512

43. Integers and Algorithms

44. Eucledean Algorithm For all integers a, b, gcd(a, b) = gcd((a mod b), b). Sort a,b so that a>b, and then (given b>1) (a mod b) < a, so problem is simplified. Example: gcd (287, 91)=? 287 mod 91 = 14 (287=91*3+14)  gcd (91, 14) 91 mod 14 = 7 (91=14*6+7)  gcd (14, 7) gcd (287, 91)= gcd (14, 7) = 7

45. Eucledean Algorithm Let a = b q + r, where a, b, q, r be integers. Then gcd (a, b) = gcd (b, r). Proof : Suppose d divides both a and b Then d divides a-bq = r Hence, any common divisor a and b is also a common divisor of b and r Likewise, suppose d divides both b and r Then d divides bq + r = a Hence, any common divisor b and r is also a common divisor of b and a Consequently, gcd (a,b) = gcd (b,r)

46. Eucledian Algorithm procedure gcd (a, b: positive integer) x := a, y := b while y  0 r := x mod y x := y y := r gcd (a, b) = x