1. Probabilistic Algorithms
Complexity
22-1
Probabilistic Algorithms
Complexity
Andrei Bulatov

2. Non-Deterministic vs. Probabilistic
Complexity
22-2
Non-Deterministic vs. Probabilistic
All algorithms we nave seen so far are either deterministic or impractical (non-deterministic)
To make non-deterministic algorithms more practical we introduce probabilistic algorithms
A probabilistic algorithm (Turing Machine) is a non-deterministic algorithm that makes non-deterministic choices randomly, e.g. by flipping a coin
This is still not practical, because sometimes the algorithm should be extremely lucky to solve problems

3. Interactive Proofs Prover Verifier Has unlimited computational power
Complexity
22-3
Interactive Proofs
Prover
Verifier
Has unlimited computational
power
Can perform polynomial time
computations
Wants to convince Verifier
in something
Accepts or rejects after
performing some computation
They can exchange massages

4. Proofs for Problems in NP
Complexity
22-4
Proofs for Problems in NP
SAT
Prover and Verifier get an instance of SAT
Prover solves the instance using his unlimited computational power and send a satisfying assignment to Verifier
Verifier checks (in polynomial time) if what obtained is a satisfying assignment, and accepts if it is or rejects otherwise

5. Problems from coNP Graph Non-Isomorphism Instance: Graphs G and H.
Complexity
22-5
Problems from coNP
Graph Non-Isomorphism
Instance: Graphs G and H.
Question: Are G and H isomorphic?
This problem belongs to coNP, but is not believed to be coNP-complete
Apparently, there is no way to prove interactively that two graphs are not isomorphic

6. Randomized Verifier Now suppose that Verifier has a fair coin
Complexity
22-6
Randomized Verifier
Now suppose that Verifier has a fair coin
Given graphs G and H
Verifier choose one of G and H by flipping a coin
Verifier then rename somehow the vertices of the chosen
graph and send it to Prover
Prover decides which graph it received
Prover send the answer to Verifier
Repeat the procedure

7. Complexity
22-7
Analysis
If the graphs are not isomorphic then Prover always gives the right
answer
If the graphs are isomorphic then Prover gives a correct answer with probability 1/2
Therefore if Prover is wrong we conclude that the graphs are isomorphic
If after n repetitions of the protocol, Prover gives only right answers, then Verifier can conclude with probability that graphs are not
isomorphic

8. Probabilistic Turing Machines
Complexity
22-8
Probabilistic Turing Machines
Definition
A Probabilistic Turing Machine is a nondeterministic polynomial
time Turing Machine PT such that
• from each configuration of PT, there are at most two
possible next configurations
• PT chooses which of the two possible next configurations
to take by flipping a fair coin
Thus, with each computational path, we can associate the probability of taking this path. This probability is equal to where k is the number of coin flips made along this path
Denote this probability by Pr[p]

9. Define the probability that PT accepts w to be
Complexity
22-9
Define the probability that PT accepts w to be
Clearly

10. Complexity
22-10
Class BPP
Definition
A Probabilistic Turing Machine PT recognizes language L with
error probability  if
• w  L implies Pr[PT accepts w]  1 – 
• w  L implies Pr[PT rejects w]  1 – 
We say that PT operates with error probability
if the above inequalities hold for where n is the
length of w
Definition
BPP is the class of languages that are recognizable by
probabilistic Turing Machines with error probability of 1/3

11. Amplification The error probability 1/3 may seem random
Complexity
22-11
Amplification
The error probability 1/3 may seem random
Actually, we can choose any value 0    1
Amplification Lemma
Let 0    1. Then for any polynomial p(n) and a
probabilistic TM that operates with error probability ,
there is a probabilistic TM that operates with an error
probability
The main idea is to run many times and then output the majority of votes

12. Complexity
22-12
Math Prerequisites
Let be a series of independent experiments (for example, coin flips) such that the probability of success in each of them is p
Theorem (Chernoff Bound)
If for some   , then the probability that the
number of successes in a series of n experiments is less than
is at most

13. Proof of Amplification Lemma
Complexity
22-13
Proof of Amplification Lemma
Machine works as follows
On input w
for i = 1 to t(|w|) do
- simulate on w
if most runs of accept, then accept; otherwise reject

14. Complexity
22-14
Analysis
The number t(|w|) must be such that

15. Complexity
22-15
Primes
Complexity
Andrei Bulatov

16. The Problem Primes Instance: A positive integer k.
Complexity
22-16
The Problem
Instance: A positive integer k.
Question: Is k prime?
Primes
The complement of Primes, the Composite problem, belongs to NP. Therefore Primes is in coNP
Recently M.Agarwal et al. Proved that Primes can be solved in polynomial time
(see
However, the probabilistic algorithm we are going describe is far more efficient

17. Residues For a positive integer n, we denote the set {0,1,2,…,n –1}
Complexity
22-17
Residues
For a positive integer n, we denote
the set {0,1,2,…,n –1}
the set {1,2,…,n – 1}
addition, multiplication and exponentiation modulo n
together with these operations is called the set of residues modulo n
Every integer m, positive or negative, has a corresponding residue —
m mod n
For example,
17 mod 5 = 2
20 mod 5 = 0
-1 mod 5 = 4

18. Complexity of Arithmetic
22-18
Complexity of Arithmetic
Given two integers, a and b, we can compute
a + b in O(max(log a, log b))
a  b in O(log a  log b)
cannot be computed in polynomial time, because the size of this number is blog a
It is possible modulo n
Let be the binary representation of b (k = log b)
Then that implies
First, we consecutively compute in
Then we compute the product again in

19. Complexity
22-19
Prime and Coprime
Integers a and b are called coprime if their greatest common divisor is 1
For example, 16 and 27 are coprime, and 15 and 18 are not
Theorem (Chinese Remainder Theorem)
If p and q are coprime then, for any a and b, there is x
such that
For example, if p = 5, q = 3, and a = 2, b = 1, then x can be chosen to be 7

20. Fermat’s Theorem Theorem (Fermat’s Little Theorem)
Complexity
22-20
Fermat’s Theorem
Theorem (Fermat’s Little Theorem)
If p is prime then, for any we have
If the converse were true, we could use it for a probabilistic primality test:
Choose k residues modulo n;
Compute their n –1 powers;
Accept if all results are 1 (mod n), reject otherwise

21. Carmichael Numbers Unfortunately, the converse is true just “almost”
Complexity
22-21
Carmichael Numbers
Unfortunately, the converse is true just “almost”
Definition
A number n passes Fermat’s test if for all a
coprime with n
A number that passes Fermat’s test is called pseudo-prime
One can straightforwardly check that, for any , coprime with 561,
561 is a Carmichael number
n is said to be a Carmichael number if, for any prime divisor p of n, p –1 | n – 1
Pseudo-prime = Prime + Carmichael

22. Roots of 1 A square root of 1 modulo n is a number a such that
Complexity
22-22
Roots of 1
A square root of 1 modulo n is a number a such that
Clearly, 1 and -1 (that is n – 1) are always roots of 1, but if n is composite, then it may have more than two roots of 1
For example,
8 has four roots of 1: 1, -1, 3, and 5
561 has eight: 1, -1, 188, (find the remaining four)
Lemma
Any Carmichael number has at least 8 roots of 1

23. Complexity
22-23
Algorithm
On input n
if n is even, then if n = 2 accept, otherwise reject
select randomly
for i = 1 to k do
- if then reject
- let n – 1 = st where s is odd and is a power of 2
- compute the sequence modulo n
- if then
let j be the maximal with this property
if then reject
accept

24. Complexity
22-24
Analysis
First we show that the algorithm does not give false negatives, that is it accepts all prime numbers
If n = 2 then n is accepted. Let n be an odd prime number
Then n passes Fermat test
n cannot be rejected in the last line, because n has only two roots of 1

25. Next we show that if n is composite, then Pr[n accepted]
Complexity
22-25
Next we show that if n is composite, then Pr[n accepted]
A number such that a does not pass either Fermat test or the square root test, is called a witness
It is enough to prove that Pr[a is a witness]  1/2, or, in other words, that at least half of the elements of are witnesses
For every nonwitness d we find a witness d´ such that if then
For a nonwitness a the sequence either contains 1s only, or it contains -1 followed by 1s
Nonwitnesses of both types are present: 1 is a nonwitness of the first type, and -1 is a nonwitness of the second type

26. Since n is composite, n = qr for some coprime q and r
Complexity
22-26
Let d be a nonwitness of the second type such that the –1 appears in the largest position in the sequence
Let and
Since n is composite, n = qr for some coprime q and r
Note that
and
By the Chinese Reminder Theorem, there is t such that
therefore
Hence t is a witness, because but

27. Now, for every nonwitness a we set a´ = a · t
Complexity
22-26
Now, for every nonwitness a we set a´ = a · t
a´ is a witness, because and
but
if then
Assume the contrary
Then, since we have
Finally, we have