1. Time Complexity

2. Define M: a det. Turing machine that halts on all input
Measuring Complexity
Worst-case analysis
Average-case analysis
Define M: a det. Turing machine that halts on all input
The running time or time complexity of M is the function
f: NN, where f(n) is the maximum number of steps that M uses on any input of length n
If f(n) is the running time of M, we say M runs in time f(n) and that M is an f(n) time Turing machine.

3. Big-O and small-o Notation
Def:

4. Complexity relationships among models

5. The class P
P is the class of languages that are decidable in polynomial time on a deterministic single-tape TM
In other words,

6. s t

7. The Class NP
Hamiltonian path in a directed graph G is a directed path that goes through each node once
Note:
It is not hard to find an exponential time algorithm for the HAMPATH problem
The HAMPATH problem can be verified in polynomial time s t

8. Def: A verifier for a language A is an algorithm V, where
Not all problem can be verified in poly. time.
Def: A verifier for a language A is an algorithm V, where
A = { | V accepts < ,c> for some string c}
A polynomial time verifier runs in polynomial time in the length of
A language A is polynomially verifiable if it has a polynomial time verifier.
A verifier uses additional information c to verify that a string is member of A
C: certificate or proof of membership in A
12653 = 123 x 111
verifiable in poly. time

9. Def: NP is the class of languages that have polynomial time verifiers
NP: Non-deterministic Polynomial time
Def:
Thm: A language is in NP iff it is decided by some non-deterministic polynomial time Turing machine
Pf:

11. Examples of problem in NP
Cor:
Examples of problem in NP
Def: clique is a graph, where every two nodes are connected by an edge
A k-clique is a clique that contains k nodes
A graph of 5-clique

12. Def:
Thm: CLIQUE is in NP
pf :

13. Def:
Thm: SUBSET-SUM is in NP
Pf:

14. P = the class of languages where membership can be decided quickly
NP= the class of languages where membership can be verified quickly
BIG Question!! P = NP ?

15. NP-completeness
The satisfiability problem is to test whether a Boolean formula is satisfiable

16. Thm: Cook-Levin Theorem
Def:

17. Def: A B

18. Thm:
Pf:

19. A language B is NP-complete if it satisfies two condition:
Def:
A language B is NP-complete if it satisfies two condition:
1. B is in NP
2. Every A in NP is polynomial time reducible to B
(with only 2, it is called NP-hard)
Thm:
Pf:

20. Thm:
Pf:

21. Def

22. Thm: 3SAT is polynomial time reducible to CLIQUE
Pf:

23. Cook-Levin Theorem： SAT is NP-complete. Proof ： 1> SAT NP：
A NTM can guess an assignment to a given formula and accept if the assignment satisfies .
2> For any A NP and show that A is polynomial time reducible to SAT.
Let N be a non-deterministic TM that decides A in nk time for some constant k.
A tableau for N on w is an nkXnk table whose rows are the configurations of a branch of the
Computation of N on input w. # q0 w1 #
start configuration
cell # #
2nd configuration # #
window # #
nkth configuration
A tableau is accepting if any row of the tableau is accepting configuration.

24. * Determine whether N accepts w is equivalent to determine whether an
Every accepting tableau for N on w corresponds to a computation branch of N on w.
* Determine whether N accepts w is equivalent to determine whether an
accepting tableau for N on w exists.
f：polynomial time reduction from A to SAT.
On input w, the reduction produces a formula .
Let Q and be the state set and the tape alphabet of N.
Let For and each we have a variable such variables.
If , it means cell[i,j] contains an s.
Design so that a satisfying assignment to the variable does correspond to an accepting tableau
for N on w.
(1) i s j

25. Ensure that the first row of the table is the starting configuration of N on w.
(3) Guarantees that an accepting configuration occurs in the tableau.
(4) This part is more complicated !
guarantee that each row of the table corresponds to a configuration that legally follows the preceding
Row’s configuration according to N’s rules. # b a q1 b a a # b q2 a c a a # b b q2 a a
(a)
(b)
(c) q1 c a q1 b q2 a q1 b a a a a q2 a a b
legal windows
(c)
(d)
(b) # b a a b a b b b a b q2 c b b # b a

26. Examples of illegal windows：
(b)
(c) a q1 b a q1 a a b a q1 q2 b q2 a a a a a
Claim：If the top row of the table is the start configuration and every window in the table is legal, each row
of the table is a configuration that legally follows the preceding one. # b a q1 b a a # # b q2 a c a a #
is a legal window j a1 a2 a3
j+1 a4 a5 a6
i-1 i
i+1
Size of ：
Total number of variables
Thus the reduction is poly.

27. Cor： 3SAT is NP-complete. CLIQUE is NP-complete. Proof ：
3SAT is in NP.
How about

28. Vertex cover of G：
If G is an undirected graph, a vertex cover of G is a subset of the nodes where
every edge of G touches one of those nodes.
{2,3} is a vertex cover. 1 4
{1,3} is not a vertex cover. 3
VERTEX-COVER={<G,k>|G is an undirected graph that has a k-node vertex cover}.

29. Thm： VERTEX-COVER is NP-complete. Proof ： VERTEX-COVER is in NP.
Variable gadgets
Clause gadgets
Need to show that is satisfiable iff G has a vertex cover with k nodes.
Let have m variables and l clauses. Let k=m+2l.
One true variable from each variable gadget.
two node from each clause gadget.
(2) Each of the 3 edges connecting the variable gadgets with each clause gadget is covered.

30. SUBSET-SUM：

31. Thm： SUBSET-SUM is NP-complete. Proof ： SUBSET-SUM is in NP.
Let be a boolean formula with variables x1,…,xl and clauses c1,…,ck.
….. l
C1 C2 ….. Ck
….. 0 …
….. 0 … …
….. 1 ….
….. 0 …
….. 0 … …
….. 0 ….
….. 0 …
….. 0 …
clause cj contains xi 1 … 1 …
….. 0
….. 0
….. 0
….. 0 1 1
….. 1 …

32. (1)
Suppose is satisfiable.
Construct a subset S as follows.
If xi is assigned TRUE, select yi
else select zi.
I.e. for each I, we select either yi or zi.
Last k digits add up between 1 and 3.
Select enough of g and h numbers to make each of the last k digits up to 3.
(2)
Suppose a subset of S sums to t.
We construct a satisfying assignment to .
If the subset contains yi , we assigned xi TRUE
else assign xi FALSE.
This assignment satisfied ! Why??
The table has size

33. Thm: MAX-SAT is NP-complete
Given a boolean formula , and an integer k, is there a truth
assignment that satisfies at least k clauses?
Thm: MAX-SAT is NP-complete
Pf:
SAT p MAX-SAT
 = c1  c2  …  cm
k = m 
Thm: DOUBLE-SAT:
Given a boolean formula , are there at least 2 truth
assignment for ?
DOUBLE-SAT is NP-complete

34. Thm: HAMPATH is NP-complete
Def:
Hamiltonian path in a directed graph G is a directed path that goes through each node once.
HAMPATH:
Given a directed graph and its 2 vertices s and t, is there
a Hamiltonian path from s to t?
Thm: HAMPATH is NP-complete
Pf: (long…later)
3SAT p HAMPATH 
UHAMPATH:
Given a undirected graph and its 2 vertices s and t, is there

35. Proof: (HAMPATH is NPC)
The following is a 3cnf-formula with k clauses:
=(a1∨b1∨c1) ∧ (a2∨b2∨c2) ∧ …∧ (ak∨bk∨ck)
Each a,b,c is a literal xi or , and x1,…,xl are l variables of .
xi : ci :

36. s c1
Proof conti. c2 cl t

37. Proof conti.

38. Proof conti.
If there is a satisfying assignment, “select exactly one of the literals” in a clause and assign it TRUE.
If there is a truth assignment, then there exists an Hamiltonian path from s to t.

39. Proof conti.
If the Hamiltonian path is normal, that is, it goes through the diamonds in order from the top one to the bottom one, we can easily get the satisfying assignment.
Because each clause node appears on the path, by observing the diamond at which the detour to it is taken, we may determine which of the literals in the corresponding clause is TRUE.
If the path zigzag, we assign corresponding variable TRUE
If the path zagzig, we assign FALSE.

40. We show that a Hamiltonian path must be normal.
Proof conti.
We show that a Hamiltonian path must be normal. c

41. Thm: UHAMPATH is NP-complete
Pf:
HAMPATH p UHAMPATH
directed G  undirected G’
u  V(G)
vertices of G’: u  uin, umid, uout
s  sout ; t  tin
edges of G’: u  v  E(G)
 uin – umid – uout – vin – vmid – vout
if s  u1  u2  …  uk  t is a Hamiltonian path in G,
then sout – u1in – u1mid – u1out – u2in – u2mid – u2out – …
– ukin – ukmid – ukout – tin is an undirected Hamiltonian
path in G’ s u v t
vmid
sout
uin
umid
uout
vin
vout
tin 

42. Undirected Hamiltonian Cycle Thm:
Undirected Hamiltonian Cycle is NP-comlplete
Pf: UHAMPATH p UHAMCYCLE
Traveling Salesman Problem: (TSP)
Given an integer n2, an nn distance matrix of some cities,
and an integer B0, is there a tour that visits every city
exactly once and returns to the starting city by traveling
within distance B?
Thm: TSP is NP-complete
Pf: UHAMCYCLE p TSP

43. LONGEST CYCLE is NP-complete SUBGRAPH ISOMORPHISM:
Given a graph and integer k, is there a cycle, with no
repeated nodes, of length at least k?
Thm:
LONGEST CYCLE is NP-complete
SUBGRAPH ISOMORPHISM:
Given 2 undirected graphs G and H, is G a subgraph of H?
SUBGRAPH ISOMORPHISM is NP-complete

44. Coping with NP-completeness:
Approximation algorithms:
Let x be an instance of an optimization problem
Opt(x): the optimum solution of x
A: a poly. time algorithm for x
: positive real number
If A satisfies
For all x, then we say A is an -approximation algorithm

45. 2. while there is an edge [u,v] in G
Eg.
The following is a 1-approximation algorithm for the vertex cover problem.
G: C={2,3}
Algorithm: 1. C :=;
2. while there is an edge [u,v] in G
add u, v to C and delete them from G
Let be the optimum vertex cover
c={ , , ,…, } 2 4 3 1

46. Polynomial Time Approximation Scheme(PTAS)
We say that an approximation scheme is PTAS, if for any fixed
 >0, the scheme runs in time polynomial in the size n of its
input instance.
Ratio bound:
Inapproximable problem:
If there is no -approximation algorithm for them with
however large , unless P=NP

47. Thm: TSP is inapproximable unless P=NP
Pf:
Let G be a graph with n nodes.
UHAMCYCLE p TSP
If G has a Hamiltonian cycle, then the optimum cost of a tour is n; otherwise if G has no Hamiltonian cycle, then the optimum cost of a tour > n(1+) 1 
2+3 1

48. If TSP had an -approximation algorithm A, then we would be able to tell whether G has a Hamiltonian cycle.
Run A for TSP:
i> if the returned cost  n(1+)+1: No cycle
ii> if the returned cost  n(1+): has a cycle
Unless P=NP, TSP is inapproximable.

49. Backtracking and Branch&Bound:
A={S0}
while A is not empty do
choose a subproblem S and delete it from A
choose a way of branching out of S , say to subproblems
S1, S2,…, Sr
for each subproblem Si in this list do
if test(Si) returns “solution found”
else if test(Si) returns “?” then add Si to A
Return “no solution”

50. Eg.
x = T
x = F
y = T
y = F
z = T
z = F

51. Branch&Bound algorithm:
A:={S0} , best_so_far:=
while A is not empty do
choose a subproblem S and delete it from A
choose a way of branching out of S , say to subproblems
S1, S2,…, Sr
for each subproblem Si in this list do
if |Si|=1 then update best_so_far
else if lowerbound(Si) < best_so_far then add Si to A
Return best_so_far

52. Local improvement algorithm:
S= initial solution
while there is a solution S’ such that
S’ is a neighbor of S and cost(S’) < cost(S)
do S:=S’
Return S

53. Simulated annealing: S= initial solution; T:=T0 repeat
generate a random solution S’ such that
S’ is a neighbor of S and let  = cost(S’)cost(S)
if   0 then S:=S’
else S:=S’ with probability e-/T
update (T)
until T=0
Return the best solution seen!
T: temperature