Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Lecture No. 39 Number Theoretic Algorithms (Theorems and Algorithms) Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

Today Covered Some More Proofs GCD as a Linear Combination Finding GCD, a Recursive Theorem Euclid’s Algorithm Extended Euclid’s Algorithm Time Complexity of Euclid’s Algorithm Residues and Reduced set of Residues Groups and Rings Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Method of Proof by Contraposition Steps in proving by contraposition Express the statement to be proved in the form:  x  D, P(x)  Q(x) Rewrite the statement in the contrapositive form  x  D,  Q(x)   P(x) Prove the contrapositive by direct proof Suppose that x is an arbitrary but particular element of D such that Q(x) is false Show that P(x) is false Dr. Nazir A. Zafar 9

Examples: Proof by Contraposition Prove that for all integers n, if n2 is even then n is also even Proof Express the above statement in the form:  x  D, P(x)  Q(x) Suppose that D = Z, Even(n, 2)  n2 is even Even(n)  n is even We have to prove that  n  Z, Even(n, 2)  Even(n) Dr. Nazir A. Zafar 9

Examples: Proof by Contraposition Contraposition of the above statement  n  Z,  Even(n)   Even(n, 2) is even Now we prove above contrapositive by direct proof Suppose that n is an arbitrary element of Z such that,  Even(n) (n is not even) i.e., n is odd n2 = n.n = odd. odd = odd n2 is odd  Even(n, 2) is even Hence,  n  Z,  Even(n)   Even(n, 2) is even Therefore,  n  Z, Even(n, 2)  Even(n) is even Hence  n  Z, if n2 is even then n is even Dr. Nazir A. Zafar 9

Examples: Proof by Contraposition Prove that for all integers n, if n2 is divisible by 7 then n is divisible by 7. Proof Express the above statement in the form:  x  D, P(x)  Q(x) Suppose that D = Z, Div(n, 2, 7)  n2 is divisible by 7 Div(n, 7)  n is divisible by 7 We have to prove that  n  Z, Div(n, 2, 7)  Div(n, 7) Dr. Nazir A. Zafar 9

Examples: Proof by Contraposition Contraposition of the above statement  n  Z,  Div(n, 7)   Div(n, 2, 7) Now we prove above contrapositive by direct proof Suppose that n is an arbitrary element of Z such that,  Div(n, 7) (n is not divisible by 7) n does contain any factor of 7 n2 does contain any factor of 7 Hence,  n  Z,  Div(n, 7)   Div(n, 2, 7) Therefore,  n  Z, Div(n, 2, 7)  Div(n, 7) Hence,  n  Z, if n2 is divisible by 7 then n is divisible by 7. Dr. Nazir A. Zafar 9

Lemma 1 Statement : The square of an odd integer is of the form 8m + 1 for some integer m. Proof: Suppose n is an arbitrary odd integer. By quotient remainder theorem any integer has the form 4m, 4m + 1, 4m + 2 OR 4m+3 Now since n is an odd integer, hence n can be represented as 4m + 1 OR 4m+3 Now we have to prove that squares of 4m + 1 and 4m + 3 are of the form 8m + 1. Dr. Nazir A. Zafar 9

Lemma 1 Case 1 Square of 4m + 1 (4m + 1)2 = 16m2 + 8m + 1 = 8(2m2 + m) + 1 = 8m’ + 1, where m ‘ = (2m2 + m) Case 2 Square of 4m + 3 (4m + 3)2 = 16m2 + 24m + 9 = 8(2m2 + 3m + 1) + 1 = 8m’’ + 1, where m’’ = (2m2 + 3m + 1) Hence any odd integer has the form 8m + 1 for some m Dr. Nazir A. Zafar 9

Theorem 1 Statement: If a and b are any integers, not both zero, then gcd(a, b) is the smallest positive element of the set {ax + by : x, y  Z} of linear combinations of a and b. Proof Let s be the smallest positive such linear combination of a and b, i.e. s = ax + by, for some x, y  Z By quotient remainder theorem a = qs + r = qs + a mod s, where q = ⌊a/s⌋. a mod s = a – qs = a - q(ax + by) = a (1 - qx) + b(-qy) Dr. Nazir A. Zafar 10

Theorem 1 Hence a mod s is a linear combination of a and b. But, since a mod s < s, therefore, a mod s = 0 Now a mod s = 0  s | a Similarly we can prove that, s | b. Thus, s is a common divisor of both a and b, Therefore, s  gcd(a, b) (1) We know that if d | a and d | b then d | ax + by for all x, y integers. Since gcd(a, b)| a and gcd(a, b) | b, hence gcd(a, b) | s and s > 0 imply that gcd(a, b) ≤ s. (2) By (1) and (2), gcd(a, b) = s Dr. Nazir A. Zafar 10

Corollary Statement; For all integers a and b and any nonnegative integer n, gcd(an, bn) = n gcd(a, b). Proof If n = 0, the corollary is trivial. If n > 0, then gcd(an, bn) is the smallest positive element of the set {anx + bny}, i.e. gcd(an, bn) = min {anx + bny} = min{n.{ax + by}} = n. min{ax + by} n times smallest positive element of set {ax + by}. Hence gcd(an, bn) = n.gcd(x, y) Dr. Nazir A. Zafar 10

Relatively Prime Integers Two integers a, b are said to be relatively prime if their only common divisor is 1, i. e, if gcd(a, b) = 1. Generalized Form of Relatively Prime Integers We say that integers n1, n2, ..., nk are pairwise relatively prime if, whenever i ≠ j, we have gcd(ni, nj) = 1. Dr. Nazir A. Zafar 10

Lemma 2 Statement For any integers a, b, and p, if both gcd(a, p) = 1 and gcd(b, p) = 1, then gcd(ab, p) = 1. Proof As, gcd(a, p) = 1, there exist integers x, y such that ax + py = 1 (1) gcd(b, p) = 1, there exist integers x’, y’ such that bx’ + py’ = 1 (2) Multiplying equations (1) and (2) and rearranging, ab(x x′) + p(ybx′ + y′ax + pyy′) = 1, abx’’ + py’’ = 1 Since 1 is a positive linear combination of ab and p, Hence gcd(ab , p) = 1,which completes the proof Dr. Nazir A. Zafar 10

Lemma 3 Statement For all primes p and all integers a, b, if p | ab, then p | a or p | b (or p divides both and b). Proof Let P = set of all primes; Z = set of all integers P(p, ab)  p | ab; Q(p, a, b)  p | a or p | b Express above statement to be proved in the form:  a, b, p, P(p, ab)  Q(p, a, b)  p  P, a, b  Z, p | ab  (p | a or p | b) Assume for the purpose of contradiction that p | ab but that p ∤ a and p ∤ b. Dr. Nazir A. Zafar 10

Lemma 3 Now p ∤ a  gcd(a, p) = 1 And, p ∤ b  gcd(b, p) = 1 Since only divisors of p are 1 and p, and by assumption p divides neither a nor b. Above Lemma 2, states that for any integers a, b, and p, if both gcd(a, p) = 1 and gcd(b, p) = 1, then gcd(ab, p) = 1. Now, gcd(ab, p) = 1, contradicts our assumption that p | ab, since p | ab implies gcd(ab, p) = p. This contradiction completes the proof. Dr. Nazir A. Zafar 10

Theorem 2: GCD Recursion Theorem Statement For any nonnegative integer a and any positive integer b, gcd(a, b) = gcd(b, a mod b). Proof If we will be able to prove that gcd(a, b) and gcd(b, a mod b) divide each other, It will complete the proof of the theorem. This is because both are nonnegative. Case 1 We first show that gcd(a, b) | gcd(b, a mod b). If we let d = gcd(a, b). By quotient remainder theorem: (a mod b) = a - qb, where q = ⌊a/b⌋. Dr. Nazir A. Zafar 10

Theorem 2: GCD Recursion Theorem Now d = gcd(a, b)  d | a and d | b, Hence, d | (a – qb), (this is because, a – qb is a linear combination of a and b, where x = 1, y = -q) And consequently d | (a mod b), this is because (a mod b = a – qb) Now, d | b and d | (a mod b), implies that: d | gcd(b, a mod b) Hence gcd(a, b) | gcd(a, a mod b). (A) Dr. Nazir A. Zafar 10

Theorem 2: GCD Recursion Theorem Case 2 We now show that: gcd(a, a mod b) | gcd(a, b). If we let, d = gcd(b, a mod b), then d | b and d | (a mod b). By quotient remainder theorem a = qb + (a mod b), where q = ⌊a/b⌋, a is a linear combination of b and a mod b,  d | a Now, d | a and d | b  d | gcd(a, b) Hence, gcd(a, a mod b) | gcd(a, b) (B) By (A) and (B): gcd(a, b) = gcd(b, a mod b). Dr. Nazir A. Zafar 10

Example: Compute gcd (1970, 1066) a = 1970, b = 1066 1970 = 1 x 1066 + 904 = gcd(1066, 904), R = 904 1066 = 1 x 904 + 162 = gcd(904, 162), R = 162 904 = 5 x 162 + 94 = gcd(162, 94), R = 94 162 = 1 x 94 + 68 = gcd(94, 68), R = 68 94 = 1 x 68 + 26 = gcd(68, 26), R = 26 68 = 2 x 26 + 16 = gcd(26, 16), R = 16 26 = 1 x 16 + 10 = gcd(16, 10), R = 10 16 = 1 x 10 + 6 = gcd(10, 6), R = 6 10 = 1 x 6 + 4 = gcd(6, 4), R = 4 6 = 1 x 4 + 2 = gcd(4, 2), R = 2 4 = 2 x 2 + 0 = gcd(2, 0), R = 0 Hence gcd(1970, 1066) = 2 Dr. Nazir A. Zafar 14

Euclid’s Algorithm EUCLID(a, b) 1 if b = 0 2 then return a 3 else return EUCLID(b, a mod b) Example Compute the gcd of 30 and 21 Solution EUCLID(30, 21) = EUCLID(21, 9) = EUCLID(9, 3) = EUCLID(3, 0) = 3 Here, there are three recursive invocations of EUCLID. The correctness of EUCLID follows from Theorem 2 And the fact that if the algorithm returns a in line 2, then b = 0, and so gcd(a, b) = gcd(a, 0) = a Dr. Nazir A. Zafar 10

Euclid’s Algorithm Note: The algorithm cannot recurse indefinitely This is because the second argument strictly decreases in each recursive call And this second argument is also always nonnegative. Hence it must be 0 after some number of calls Therefore, EUCLID always terminates with the correct answer. Dr. Nazir A. Zafar 10

Running Time of Euclid’s Algorithm We analyze the worst-case running time of EUCLID as a function of the size of a and b. We assume without loss of generality that a > b ≥ 0. This assumption justified because if b > a ≥ 0, then EUCLID(a, b) makes recursive call EUCLID(b, a). That is, if first argument is less than second one, EUCLID spends one recursive call swapping a, b Similarly, if b = a > 0, the procedure terminates after one recursive call, since a mod b = 0. The overall running time of EUCLID is proportional to the number of recursive calls it makes. Our analysis makes use of the Fibonacci numbers Fk, defined earlier in the first part of our course Dr. Nazir A. Zafar 10

Running Time of Euclid’s Algorithm Statement If a > b ≥ 1 and the invocation EUCLID(a, b) takes k ≥ 1 recursive calls, then a ≥ Fk+2 and b ≥ Fk+1. Proof The proof is by induction on k. Case 1 For base case, let k = 1. Then, b ≥ 1 = F2, and since a > b, we must have a ≥ 2 = F3. Hence the statement is true for k = 1 Please note that, b > (a mod b), in each recursive call, i.e., first argument is strictly larger than the second and hence the assumption that a > b therefore holds for each recursive call. Dr. Nazir A. Zafar 10

Running Time of Euclid’s Algorithm Case 2 Now suppose that the lemma is true for k – 1 i.e., if a > b ≥ 1 and invocation EUCLID(a, b) takes k-1 ≥ 1 recursive calls, then a ≥ Fk+1 and b ≥ Fk. Case 3 Now we have to prove that statement is true for k i.e. if a > b ≥ 1 and invocation EUCLID(a, b) takes k ≥ 1 recursive calls, then a ≥ Fk+2 and b ≥ Fk+1. Since k > 0, and b > 0, and EUCLID(a, b) calls EUCLID(b, a mod b) recursively, which in turn makes k - 1 recursive calls. Since we know that statement is true for k-1, hence b ≥ Fk+1, and (a mod b) ≥ Fk. Dr. Nazir A. Zafar 10

Running Time of Euclid’s Algorithm Now we have b + (a mod b) = b + (a - ⌊a/b⌋ b) (1) Since, a > b > 0, therefore, ⌊a/b⌋ ≥ 1  ⌊a/b⌋ b ≥ b  b - ⌊a/b⌋ b  0  a + b - ⌊a/b⌋ b  0 + a  b + (a - ⌊a/b⌋ b)  a By (1), b + (a mod b) = b + (a - ⌊a/b⌋ b) ≤ a b + (a mod b) ≤ a Thus, a ≥ b + (a mod b) ≥ Fk+1 + Fk = Fk+2 . Hence, a ≥ Fk+2, It completes proof of the theorem Dr. Nazir A. Zafar 10

Extended Euclid’s Algorithm EXTENDED-EUCLID(a, b) 1 if b = 0 then return (a, 1, 0) (d’, x’, y’)  EXTENDED-EUCLID(b, a mod b) (d, x, y)  (d’, y’, x’ -  a/b y’) return (d, x, y) Proof of Correctness d’ = bx’+ (a mod b)y’ d = bx’+ (a -  a/bb)y’ gcd(a, b) = gcd(b, a mod b) d = ay’ + b(x’ -  a/by’) Dr. Nazir A. Zafar 10

Reduced set of residues mod n Complete set of residues is: 0 . . . n-1 Reduced set of residues consists of all those numbers (residues) which are relatively prime to n And it is denoted by Zn* = {k : gcd(k, n) = 1, 0  k < n} The number of elements in reduced set of residues is called the Euler Totient Function (n) Example For n = 10, find reduced list of residues of n All residues: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} Reduced residues (primes) = {1, 3, 7, 9}, (n) = 4 Dr. Nazir A. Zafar 10

Group Definition of a Group Group is a set, G, together with a binary operation : G * G  G, usually denoted by a*b, such that the following properties are satisfied : Associativity : (a*b)*c = a*(b*c) for all a, b, c  G Identity :  e  G, such that e*g = g = g*e for all g  G. Inverse : For each g  G, there exists the g’, inverse of g, such that g’*g = g*g’ = e Dr. Nazir A. Zafar 11

The Multiplicative Group Z*n Zn* = {k : gcd(k, n) = 1, 1  k < n} For any positive integer n, Zn* forms a group under multiplication modulo n. Proof: Binary Operation Let a, b Zn*, gcd(a, n) = 1; gcd(b, n) = 1 gcd(ab, n) = gcd(a, n)*gcd(b,n) = 1*1 = 1 Associativity holds, 1 is the identity element. inverse of each element exits Hence (Zn* ,*) forms a group. Dr. Nazir A. Zafar 11

Rings Definition A ring is a set R with two binary operations + : R × R → R and · : R × R → R (where × denotes the Cartesian product), called addition and multiplication, such that: (R, +) is an abelian group with identity element 0 (a + b) + c = a + (b + c) 0 + a = a + 0 = a For every a in R, there exists an element denoted −a, such that a + −a = −a + a = 0 a + b = b + a Dr. Nazir A. Zafar 22

Rings Definition (Contd..) (R, ·) is a monoid with identity element 1: (a·b)·c = a·(b·c) 1·a = a·1 = a Multiplication distributes over addition: a·(b + c) = (a·b) + (a·c) (a + b)·c = (a·c) + (b·c) Note Ring addition is commutative so that a + b = b + a But ring with multiplication is not required to be commutative i.e. a·b need not equal b·a. Rings that satisfy commutative property for multiplication are called commutative rings. Not all rings are commutative. Dr. Nazir A. Zafar 22

Rings Rings need not have multiplicative inverses either. An element a in a ring is called a unit if it is invertible with respect to multiplication An element a is called invertible under multiplication if there is an element b in the ring such that a·b = b·a = 1, This b is uniquely determined by a and we write a−1 = b. Lemma The set of all units in R forms a group under ring multiplication Dr. Nazir A. Zafar 22

Example: Rings Example Prove that Z (+, *) ( the set of integers) is a ring. Solution + and * are binary operation on Z because sum and product of two integers are also an integer Now,  a, b, c  Z (a + b) + c = a + (b + c), 0 + a = a + 0 = a a + (−a) = (−a) + a = 0 a + b = b + a Hence (Z, +) is an abelian group with identity element 0 Dr. Nazir A. Zafar 22

Example: Rings Since,  a, b, c  Z (a·b)·c = a·(b·c) 1·a = a·1 = a Hence (Z, ·) is a monoid with identity element 1 Finally  a, b, c  Z a·(b + c) = (a·b) + (a·c) (a + b)·c = (a·c) + (b·c) i.e., multiplication is distributive over addition Hence we can conclude that Z (+, *) is a ring Dr. Nazir A. Zafar 22