1 Fingerprint

2 Verifying set equality Verifying set equality v String Matching – Rabin-Karp Algorithm

3 Verifying set equality Verifying set equality

4

5

6

7 Fingerprinting Fingerprinting

8

9 Fingerprinting Computation Fingerprinting Computation

10 Fingerprinting Computation Fingerprinting Computation Horner’s Rule

11 Protocol Protocol

12 Prime Number q Prime Number q

13 False Positive False Positive

14 Prime Divisors Prime Divisors

15 Density of Primes Density of Primes

16 Density of Primes Density of Primes v  (x) = número de primos menores ou iguais a x –  (13) = 6 –Primos < = do que 13 = 2, 3, 5, 7, 11 e 13 v O valor de  não muda até chegarmos ao próximo primo. –  (13) =  (14) =  (15) =  (16) –Ou seja,  aumenta em salto de 1, mas o intervalo entre esses saltos é irregular

17 Density of Primes Density of Primes Esses intervalos tornam-se cada vez maiores, isto é, a chance de um inteiro escolhido ao acaso ser primo diminui quando avançamos para os números maiores. PERGUNTA: O valor de  não poderia ser aproximado por alguma função conhecida?

18 Density of Primes Density of Primes Para um valor elevado de x,  (x) ~ x/ ln x. Ou seja, lim  (x) = 1 x   x/ln x

19 Sample Space Sample Space

20 Probability of a bad prime Probability of a bad prime

21 Final Protocol Properties Final Protocol Properties

22 String Matching String Matching

23 String Matching String Matching Many applications –While using editor/word processor/browser –Login name & password checking –Virus detection –Header analysis in data communications –DNA sequence analysis

24 Naïve O(nm) algorithm Naïve O(nm) algorithm

25 Rabin-Karp Algorithm Rabin-Karp Algorithm

26 Fingerprinting Fingerprinting

27 Fingerprinting function Fingerprinting function

28 Fingerprinting computation Fingerprinting computation The only expensive operation

29 False Positives? False Positives?

30 Sample Space Sample Space

31 False Positives False Positives

32 Fingerprinting Fingerprinting

33 Primality testing v A natural number n is prime iff the only natural numbers dividing n are 1 and n

34 Primality testing v A natural number n is prime iff the only natural numbers dividing n are 1 and n v The following are prime: 2, 3, 5, 7, 11, 13,

35 Primality testing v A natural number n is prime iff the only natural numbers dividing n are 1 and n v The following are prime: 2, 3, 5, 7, 11, 13, …and so are , , , …

36 Primality testing v A natural number n is prime iff the only natural numbers dividing n are 1 and n v The following are prime: 2, 3, 5, 7, 11, 13, …and so are , , , … There is an infinite number of prime numbers

37 Primality testing There is an infinite number of prime numbers Proof: Let us suppose the number of primes is Finite.

38 Primality testing There is an infinite number of prime numbers Proof: Let us suppose the number of primes is Finite. Let p 1, p 2, … p k be all primes. Let n = p 1 p 2 … p k +1,

39 Primality testing There is an infinite number of prime numbers Proof: Let us suppose the number of primes is Finite. Let p 1, p 2, … p k be all primes. Let n = p 1 p 2 … p k +1,  n must be composite.

40 Primality testing There is an infinite number of prime numbers Proof: Let us suppose the number of primes is Finite. Let p 1, p 2, … p k be all primes. Let n = p 1 p 2 … p k +1,  n must be composite.  there exists a prime p s.t. p | n (fund theo. arithmetic), and p cannot be any of the p 1, p 2, … p k

41 Primality testing There is an infinite number of prime numbers Proof: Let us suppose the number of primes is Finite. Let p 1, p 2, … p k be all primes. Let n = p 1 p 2 … p k +1,  n must be composite.  there exists a prime p s.t. p | n (fund theo. arithmetic), and p cannot be any of the p 1, p 2, … p k Therefore, p 1, … p k were not all the prime numbers.

42 Some questions? v Is = prime? v How do we check whether a number is prime? v How do we generate huge prime numbers? v Why do we care?

43 Some questions? v Is = prime? v How do we check whether a number is prime? v How do we generate huge prime numbers? v Why do we care?

44 Some questions? v Is = prime? v How do we check whether a number is prime? v How do we generate huge prime numbers? v Why do we care?

45 Some questions? v Is = prime? v How do we check whether a number is prime? v How do we generate huge prime numbers? v Why do we care?

46 Naïve solution: Finding the smallest divisor of n –For i=2,..., n do u Divide n by i until n mod i = 0 Check if i is a divisor of n for some i = 2,..., n

47 An improvement   Check if i is a divisor of n for some i = 2,...,  n

48 An improvement   Check if i is a divisor of n for some i = 2,...,  n Why can we do that?

49 Theorem: Composit numbers have a divisor bellow their square root Theorem: Composit numbers have a divisor bellow their square root

50 Theorem: Composit numbers have a divisor bellow their square root Theorem: Composit numbers have a divisor bellow their square root Proof Idea: n composite  n = ab, 0 < a  b < n

51 Theorem: Composit numbers have a divisor bellow their square root Theorem: Composit numbers have a divisor bellow their square root Proof Idea: n composite  n = ab, 0 < a  b < n  a  sqrt(n)

52 Theorem: Composit numbers have a divisor bellow their square root Theorem: Composit numbers have a divisor bellow their square root Proof Idea: n composite  n = ab, 0 < a  b < n  a  sqrt(n) Otherwise, we obtain ab > n (contradiction!!)

53 Is there a more efficient way of checking primality? Is there a more efficient way of checking primality?

54 Is there a more efficient way of checking primality? Is there a more efficient way of checking primality? Yes! At least if we are willing to accept a tiny probability of error.

55 We can prove that a number is not prime without explicitly finding a divisor of it Is there a more efficient way of checking primality? Is there a more efficient way of checking primality? Yes! At least if we are willing to accept a tiny probability of error.

56 We can prove that a number is not prime without explicitly finding a divisor of it Is there a more efficient way of checking primality? Is there a more efficient way of checking primality? Yes! At least if we are willing to accept a tiny probability of error. RANDOMNESS IS USEFUL IN COMPUTATION

57 The Fermat Primality Test Fermat’s little theorem: If p is a prime and p does not divide the integer a, then: a p-1  1(mod p)

58 Suppose that ra e sa have are the same modulo p, then we have r = s (mod p) Contradiction!! Aa, 2a, 3a,..., (p-1)a quando divididos por p possuem restos diferentes:1, 2,..., p-1 Proof: List the first p-1 positive multiple of a: a, 2a, 3a, 4a,..., (p-1) a The Fermat Primality Test Fermat’s little theorem: If p is a prime and p does not divide the integer a, then: a p-1  1(mod p)

59 Suppose that ra and sa are the same modulo p, then we have r = s (mod p) Contradiction!! Aa, 2a, 3a,..., (p-1)a quando divididos por p possuem restos diferentes:1, 2,..., p-1 Proof: List the first p-1 positive multiple of a: a, 2a, 3a, 4a,..., (p-1) a The Fermat Primality Test Fermat’s little theorem: If p is a prime and p does not divide the integer a, then: a p-1  1(mod p)

60 Suppose that ra and sa are the same modulo p, then we have r = s (mod p) Contradiction!! Aa, 2a, 3a,..., (p-1)a when divided by p have the different reminders:1, 2,..., p-1 Proof: List the first p-1 positive multiple of a: a, 2a, 3a, 4a,..., (p-1) a The Fermat Primality Test Fermat’s little theorem: If p is a prime and p does not divide the integer a, then: a p-1  1(mod p)

61 a (p-1) (p-1)! = (p-1)! (mod p) Proof: a. 2a. 3a..... (p-1)a  (p-1) (mod p) The Fermat Primality Test Fermat’s little theorem: If p is a prime and p does not divide the integer a, then: a p-1  1(mod p)

62 a (p-1) (p-1)! = (p-1)! (mod p) Dividing by (p-1)! we get the result Proof: a. 2a. 3a..... (p-1)a  (p-1) (mod p) The Fermat Primality Test Fermat’s little theorem: If p is a prime and p does not divide the integer a, then: a p-1  1(mod p)

63 A Corollary: If p is a prime then, for any integer a: a p  a (mod p)

64 A Corollary: If p is a prime then, for any integer a: a p  a (mod p) The result is trivial if p divides a: a(a p-1 – 1)  0 (mod p) If a does not divide a, then we need only multiply the congruence in Fermat´s little theorem by a to complete the proof

65 A Corollary: If p is a prime then, for any integer a: a p  a (mod p) The result is trivial if p divides a: a(a p-1 – 1)  0 (mod p) If p does not divide a, then we need only multiply the congruence in Fermat´s little theorem by a to complete the proof

66 Corollary: Corollary: If a n ≠  a (mod n), for some a, then n is not a prime! Such an a is a witness to the compositeness of n. The Fermat Test: Do 100 times: Pick a random 1<a<n and compute a n (mod n). If a n  a (mod n), then n is not a prime. If all 100 tests passed, declare n to be a prime.

67 Fast computation of modular exponentiation (define (expmod a b m) (cond ((= b 0) 1) ((= b 0) 1) ((even? b) ((even? b) (remainder (expmod (remainder (expmod (remainder (* a a) m) (remainder (* a a) m) (/ b 2) (/ b 2) m) m)) m) m)) (else (else (remainder (* a (expmod a (- b 1) m)) (remainder (* a (expmod a (- b 1) m)) m)))) m))))

68 Implementing Fermat test (define (test a n)(= (expmod a n n) a)) (define (rand-test n) (test (+ 1 (random (- n 1))) n)) (define (fermat-test n t); (cond ((= t 0) #t) ((rand-test n) (fermat-test n (- t 1))) (else #f))) Worst-case time complexity:  (log n) Even if n is a 1000 digit number, it is still okay!

69 Is the Fermat test correct? v If the Fermat test says that a number n is composite, then the number n is indeed a composite number. v If n is a prime number, the Fermat test will always say that n is prime. But, v Can the Fermat test say that a composite number is prime? v What is the probability that this will happen?

70 Carmichael numbers A composite number n is a Carmichael number iff a n  a (mod n) for every integer a. The first Carmichael numbers are: 561, 1105, 1729, 2465, 2821, 6601, 8911, 10585, 15841, … On Carmichael numbers, the Fermat test is always wrong! Carmichael numbers are fairly rare.

71 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

72 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n. Proof: Consider Z * n = {1, 2,..., n-1} Let B={x / x  Z * n and x n-1  1 (mod n)} We are going to show that B is subgroup of Z * n For this: 1.1  B 2.x 1, x 2  B  x 1. x 2  B 3.x  B  x -1  B

73 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n. Proof: Consider Z * n = {1, 2,..., n-1} Let B={x / x  Z * n and x n-1  1 (mod n)} We are going to show that B is subgroup of Z * n For this: 1.1  B : 1 n-1  1 (mod n) 2.x 1, x 2  B  x 1. x 2  B 3.x  B  x -1  B

74 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n. Proof: Consider Z * n = {1, 2,..., n-1} Let B={x / x  Z * n and x n-1  1 (mod n)} We are going to show that B is subgroup of Z * n For this: 1.1  B : 1 n-1  1 (mod n) 2.x 1, x 2  B  x 1. x 2  B 3.x  B  x -1  B (x 1 ) n-1  1 (mod n) (x 2 ) n-1  1 (mod n) (x 1.x 2 ) n-1  1 (mod n)

75 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n. Proof: Consider Z * n = {1, 2,..., n-1} Let B={x / x  Z * n and x n-1  1 (mod n)} We are going to show that B is subgroup of Z * n For this: 1.1  B : 1 n-1  1 (mod n) 2.x 1, x 2  B  x 1. x 2  B 3.x  B  x -1  B (1) n-1  1 (mod n) (x.x -1 ) n-1  1 (mod n) (x -1 ) n-1  1 (mod n)

76 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n. Proof: Pr(x n-1  1 (mod n)) = Pr(x  B) It can be proved that 1  B and n-1  B and therefore, |B|  2 Since the order of a subgroup divides the subgroup we have that |B|  |Z * n | / 2  Pr(x  B)  1/2

77 Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n. Corollary: Let n be a composite number that is not a Carmichael number. If we pick a random number a, 1<a<n, then a is a witness with a probability of at least a 1/2 !

78 “Correctness” of the Fermat test “Correctness” of the Fermat test v If n is prime, the Fermat test is always right. v If n is a Carmichael number, the Fermat test is always wrong! v If n is composite number that is not a Carmichael number, the Fermat test is wrong with a probability of at most Is an error probability of acceptable? Yes!

79 The Rabin-Miller test v A fairly simple modification of the Fermat test that is correct with a probability of at least also on Carmichael numbers. v Will not be covered in this course.

80 A probabilistic algorithm An algorithm that uses random choices but outputs the correct result, with high probability, for every input! Randomness is a very useful algorithmic tool. Up to 2002, there were no efficient deterministic primality testing algorithms. In 2002, Agarwal, Kayal and Saxena found a fast deterministic primality testing algorithm.

81 Finding large prime numbers Finding large prime numbers The prime number Theorem: The number of prime numbers smaller than n is asymptotically n / ln n. Thus, for every number n, there is “likely” to be a prime number between n and n + ln n. To find a prime number roughly the size of n, simply test n, n+2, n+4, … for primality.

82 Primality testing versus Factoring v Fast primality testing algorithms determine that a number n is composite without finding any of its factors. v No efficient factoring algorithms are known. v Factoring a number is believed to be a much harder task. Primality testing - Easy Factoring - Hard But, factoring is not that hard on a quantum computer!