Presentation is loading. Please wait.

Presentation is loading. Please wait.

Theoretical Foundation of Computation

Similar presentations


Presentation on theme: "Theoretical Foundation of Computation"— Presentation transcript:

1 Theoretical Foundation of Computation
Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm Tuesday

2 Contents Computational models and languages
finite automata, push automata regular languages, context free languages Computability theory decidable problem, un-decidable problem Complexity theory time, space, P, NP, PSpace

3 Theory of Computation What is the computation?
What problems are computable by computer in finite steps? What problems are computable by small number of steps?

4 Mathematical Model Real computers are very complicate
Develop simple mathematical model to define computation The mathematical models are “equivalent to” real computer under some transformation

5 Un-computable problems
Important Boundary Computability theory boundary Un-computable problems Computable problems

6 Hierarchy inside computable area

7 Basic Concepts Set: a set is a group of objects represented as a unit.
{7,21,57} Element: A object of a set is called an element. Subset: A is a subset of B if every member of A is also a member of B.

8 Some sets Natural numbers set: N={1,2,3,…} Integers set
Z={…, -2,-1,0,1,2,…} Empty set: it is a set with no elements

9 Set operations Union: A={2,4,9} B={1,2,5} ={1,2,4,5,9} Intersection:
={2}

10 Tuple Sequence: a list of objects in some order (7, 21, 57)
Tuple: finite sequence (7, 21,57) Cartesian product: A x B is the set of all pairs with first element from A and second element from B A={1,2} B={x,y,z} A x B= {(1,x),(1,y), (1,z), (2,x), (2,y), (2,z) }

11 Power Set Let A be a set. The power set of A is the set of subsets of A. For A={a,b}, its power set P(A) is { , {a}, {b}, {a,b}}

12 Cartesian product

13 Function A function f is mapping from one set D to another set R
f: DR for every a in the set D, there is another element b in R such that a is mapped to b by f f(a)=b Domain: D Range: R

14 Function example Function f: {0,1,2,3,4}{0,1,2,3,4} n f(n) 0 1 1 2

15 Relation For two sets A and B, a binary relation R is a subset AxB
Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

16 Represent binary relation
If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

17 Relation For two sets A and B, a binary relation R is a subset AxB
Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

18 Represent binary relation
If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

19 Equivalence relation A binary relation R is equivalence relation if
Reflexive: xRx for every x Symmetric: if xRy, then yRx Transitive: if xRy and yRz, then xRz Example 1: = on {1,2,3}x{1,2,3} Example 2: = on NxN

20 Example for equivalence
Relation For two integers x and y, x y if (x-y) is a multiple of 7 In other words, there is another integer z such that (x-y)=7z.

21 Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ }

22 Path Graph G=(V,E) A path is a series of edges linked one by one Loop:

23 Tree A graph is connected if every two nodes have a path to connect them A tree is a connected graph without loop

24 Connected Graph Tree Every connected graph can be converted into tree by removing some edges Removing one edge on a loop does not damage the connectivity.

25 A tree is a minimal connected graph
Removing any edge on a tree damages the connectivity Proof. Tree T=(V,E). Let (v1, v2) be removed from T. T  T’=(V, E-{(v1,v2)}). If T’ is still connected, T has a loop containing v1 and v2 . Contradiction!

26 Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ }

27 Node degree: The number of edges connecting to the node
v2 v1 v3 v4

28 Mathematical approach
Definition Mathematical statement Express some object with certain property Theorem A statement proved to be true Proof

29 Mathematical proof Convincing logical argument that a statement is true Usually consists a series logical statements There is a small logical gap between the current logic statement with previous statements.

30 3 Styles of Mathematical Proofs
Proof by construction Proof by contradiction Proof by induction

31 Sum of node degrees For every graph, the sum of the degrees of all nodes in G is an even number Sum=2+2+2= sum= =14

32 Proof Let v1, v2, …, vn are the n nodes of the graph.
deg(vi) is the degree of node vi The sum of node degrees is sum=deg(v1)+deg(v2)+…+deg(vn) For each edge e=(vi, vj), it makes one contribution to both deg(vi) and deg(vj). If there are k edges, the sum is 2k.

33 Regular graph A graph is k-regular if every node has degree equal to k
Theorem: For each even number n>2, there exists a 3-regular graph with n nodes.

34 Proof by construction n/2 nodes
Let every point at top half connect to a point in the bottom half

35 Proof Let be the n nodes of the graph.
Add edges for i=0, 1, …,(n/2)-1 , and Add edges for i=0,1, …,(n/2)-1

36 Proof by contradiction
Assume the theorem is false Lead to an obviously false consequence Example: Jill just came in from outdoor and is complete dry Try to Prove: No rain Proof: Assume it were raining Jill would be wet. A contradiction!

37 No 3-regular graph with odd nodes
Theorem: There is no 3-regular graph with odd number of nodes Proof (by contradiction) If the graph G is a 3-regular graph with 2m+1 nodes, where m is an integer at least 0. The sum of the degrees of nodes is sum=3+3+….+3=3(2m+1)=6m+3=2(3m+1)+1 It is a contradiction to our previous theorem.

38 3 Styles of Mathematical Proofs
Proof by construction Proof by contradiction Proof by induction

39 Proof by induction Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

40 Example for induction Theorem: For every natural number n,

41 Proof Basis: when n=1. The left side is 1 The right is Induction step:
Assume

42 Proof

43 Computation models Finite automata Pushdown automata Turing machine

44 3 Styles of Mathematical Proofs
Proof by construction Proof by contradiction Proof by induction

45 Proof by induction Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

46 Example for induction Theorem: For every natural number n,

47 Proof Basis: when n=1. The left side is 1 The right is Induction step:
Assume

48 Proof

49 Computation models Finite automata Pushdown automata Turing machine

50 Finite automata Supermarket entrance
Front: A person is coming from the front. Rear: …… Both: front and rear Neither: neither front nor rear closed open

51 Formal definition of automata
A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

52 Automata example q1 q3 q2

53 Acceptance Any sequence with at least one 1 and even number of 0s following the last 1

54 Notions for automata start state: q1
accept state: q2. In other words, F={q2} The set The transition function is q1 q1 q2 q2 q3 q2 q3 q2 q2 Q={q1,q2,q3}

55 Run the automata Start from the start state
Follow the state transition based on the current state and symbol accepts if it enters accept state, rejects otherwise

56 Run the machine at inputs
q1q2 q1q1q2 q1q2q3q2q3q2q3q2q3q2 q1q2 q1q2q3 101000 q1q2q3q2q3q3q3

57 Language and Machine Let A be a set of strings. Let M be a machine
If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

58 Example for language acceptance
A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

59 Formal definition of automata
A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

60 Formal definition of computation
A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

61 Automata What is the language accepted by the automata?

62 Answer L(M)={w| w ends in a 1}

63 Automata What is the language accepted by the automata?

64 Answer L(M)={w| w is an empty string or ends in a 0}

65 Automata What is the language accepted by the automata?

66 Answer L(M)={w| w starts and ends at the same symbol}

67 Designing Automata Language A consists all {0,1} strings with even number of 1s. Problem: design an automata M with L(M)=A.

68 Automata States:

69 Designing Automata Language A consists all {0,1} strings with 001 as substring. Problem: design an automata M with L(M)=A.

70 Automata States:

71 Regular Language M accepts language A if A={w| M accepts w}
A language is regular if some finite automata accepts it. Example: A1={w| w is {0,1} string and ends in a 1} A2={w| w is a {a,b} string that starts and ends with the same symbol}

72 Language and Machine Let A be a set of strings. Let M be a machine
If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

73 Example for language acceptance
A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

74 Formal definition of computation
A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

75 Non-determinism blind monkey

76 Symbol It represents the empty symbol.
If used , one state moves to the next without consuming any symbol q1 q2

77 Automata example q2 q1 q3 q3 q4 q2

78

79 Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

80 Language recognized Let A be a set of strings that contain either 101 or 11 as a substring. E.G , 10111

81 Automata example q2 q1 q3 q3 q4 q2

82 Language recognized Let A be a set of strings containing a 1 in the third position from the end E.G

83 Automata example q2 q3 q1 q3 q2 q3

84 Problem: What language does it accept?
q2 q3 q2 q1 q2 q5

85 Language recognized Accept all strings , where k is a multiple of 2 and 3.

86 Some notations For a set Q, P(Q) is the collection of all subsets of Q
Example, Q={q1,q2} P(Q)={empty, {q1}, {q2}, {q1,q2}} For alphabet , write

87 Formal definition of automata
A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

88 Equivalence between NFA and DFA
Theorem: Every NFA has an equivalent DFA Proof. Given an NFA We will construct an to accept the same language.

89 Proof Q’=P(Q), which is the set of all subsets of Q
The transition function or and for some The start state F’={R in Q’| R has accept state in N}

90 Example Given NFA, convert it into DFA 2 1

91 Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 1 2 1,2

92 Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 2 1 1,2

93 Problem: What language does it accept?
q2 q3 q2 q1 q2 q5

94 Language Operations Let A and B be two languages Union: Concatenation:
Star:

95 Closure under Union Theorem: If A and B are regular languages, then
is also regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

96 Construct N N1: N2:

97 Construct N N accepts iff one of N1 and N2 accepts

98 Closure under Catenation
Theorem: If A and B are regular languages, then is also a regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

99 Construct N N1: N2:

100 Construct N N2 is linked to the accept state of N1

101 Closure under Concatenation
Theorem: If A is regular language, then is also a regular language Proof. Let A be accepted by finite automata N1, Find another finite automata N to accept

102 Construct N N1:

103 Construct N N1:

104 Exercise 1. Given NFA, convert it into DFA
2. Let A be the language recognized by the NFA. Design the automata to recognize A* 1 2

105 Regular Operations R is a regular expression if R is A for some a in
The special string The empty set ,where R1 and R2 are regular expressions , where R1 and R2 are regular expressions ,where R1 is a regular expression

106 Regular Operations Examples
0*10* 3) 4) 5) 6)

107 Regular  Automata Theorem: Every regular expression is regular language Proof. Let R be a regular expression. Construct a DFA to accept R

108 6 Cases a ,where R1 and R2 are regular expressions
,where R1 is a regular expression

109 Case 1 a

110 Case 2

111 Case 3

112 Case 4 DFA N1 accepts R1 DFA N2 accepts R2

113 Construct N N accepts

114 Case 5 DFA N1 accepts R1 DFA N2 accepts R2

115 Construct N N accepts

116 Construct N N1 accepts R

117 Construct N N accepts R*

118 Example

119 Regular Operations R is a regular expression if R is A for some a in
The special string The empty set ,where R1 and R2 are regular expressions , where R1 and R2 are regular expressions ,where R1 is a regular expression

120 Regular Operations Examples
0*10* 3) 4) 5) 6)

121 Automata  Regular expression
If a language is regular, then it is described by a regular expression Proof. Let A be recognized by a DFA A, find a regular expression R for A.

122 Some observation States transition 1 2 3

123 Some observation States transition 1 3

124 Some observation States transition 1 2 3

125 Some observations States transition 1 3

126 Some observation States transition 1 2 3

127 Some observations States transition 1 3

128 Some observations States transition 1 3

129 Proof Idea Convert DFA by shrinking it step by step
Replace the symbols by regular expressions on the state transition

130 Proof Idea Given DFA, covert it into regular expression

131 Proof Idea Add start state s and accept state a
Convert DFA by removing those old state one by one Replace the symbols by regular expressions on the state transition

132 Example States transition 1 2

133 Example States transition 1 2 s a

134 Example States transition 1 s a

135 Example States transition s a

136 Regular Operations For and for and

137 Remove a state Convert left to right

138 Verify The old and new machines accept the same language

139 Construct Regular expression
Add one start state and one accept state:

140 Pumping Lemma Lemma: If A is regular language, there is a number p such that if s is in A and of length at least p, s may be divided into s=xyz, satisfying 1) for each 2) 3)

141 Some notations For a string s, is the length of s (the number of letters) For example, |adb|=3, |a|=1, |afdsaf|=6 For a string s and integer i, is a string to repeats i times For example, if s=dgh then

142 Proof Idea a q9 q1 q13

143 Analysis Run the input string on a automata
Input string: s1 s2 s3 s4 s5 s6 … sn State q1 q2 q3 q9 q5 q9… q13 Let p be the number of states If n>p, two of the states must be equal, say q9 Repeat the substring between the two q9s reach the same accept state q13

144 Proof Let p be the number of the states in the Automata M for A
Let (n>=p) be a string in A Let be the state transition sequence There are two equal states (j<k) Let x takes M from r1 to rj, y takes M from rj to rj z takes M from rj to r(n+1) M must accepts for all

145 Application of Pumping Lemma
is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

146 Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

147 Application of Pumping Lemma
The language L={w | w has an equal number of 0s and 1s} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

148 Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. y contains only 0s. By 1) of the pumping, the string is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

149 Application of Pumping Lemma
The language {ww | w is a {0,1} string} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

150 Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the first 0s area is more than the number of 0s in the second 0s area. This is a contradiction.

151 Application of Pumping Lemma
The language is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

152 Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 1s. By 1) of the pumping, is also in the language. The number of 1s in the string is at least and at most This is a contradiction since the language L does not contain any string with the number of 1s between and .

153 Problem Prove that is not a regular language.

154 Two ways for language A language can be accepted by a machine
A language can be also generated by some rules

155 Context Free Grammar Rules for generating a language: A0A1 A#
Variables: A Terminals: 0,1,# Start variable: A Rules: A0A1, A#

156 Conversion AXY X0 YAZ Z1 A#

157 Context Free Grammar Rules for generating a language: A0A1 AB B#
Variables: A, B Terminals: 0,1,# Start variable: A Rules: A0A1, AB, B#

158 Derivation A0A10B10#1 A0A100A11000A111000B111000#111

159 Context free grammar A context free grammar is
V is a finite set called the variables is a finite set of terminals R is a finite set of rultes is the start variable

160 Language of the grammar
If Aw is a rule, u and v are strings, then , called uAv yields uwv if for some strings The language of the grammar is

161 Context Free Grammar Rules for generating a language: A0A1 AB
Variables: A, B Terminals: 0,1, Start variable: A The language of this grammar is

162 Convert DFA to grammar Automata accepts all strings that end in 1 A B

163 Grammar A0A A1B B1B B0A (because B represents the accept state)
Start variable A, Variables A,B Terminals: 0,1,

164 DFA to grammar conversion
For every transition , add a rule For every accept state , add rule

165 Theorem Every regular language can be generated by a grammar with the rules like AaB, and

166 Chomsky normal form A context-free grammar is in chomsky normal form if every rule is of the form ABC Aa where “a” is a terminal and A,B,and C are variables. In addition, we permit if S is the start variable.

167 Removing empty Every context free grammar G has another equivalent context free grammar G’, which has no rule for any non-start variable A

168 Proof Add a new start symbol and rule
For any rule that A is not a start symbol, 1) replace every RuAv by Ruv, every RuAvAw by Ruvw, etc. 2) replace RA by Repeat the steps above until with non-start variable A does not appear

169 Remove Unit Rule Every context free grammar G has another equivalent context free grammar G’, which has no rule

170 Proof For every A B, replace every Bu by Au
Repeat the step above until there is no AB

171 Chomsky Normal Form Every context free grammar G has an equivalent chomsky normal form

172 Proof Remove the for non-start symbol A Remove the unit rule AB
For each at k>2, replace it by rules

173 Proof For each at k>1, replace each terminal by the new symbol and add new rule

174 Example A0A1 Conversion:

175 Remove the empty

176 Remove the empty

177

178

179 Arithmetic expression
Grammar

180 Example Generate

181 Generate the expression
Steps

182 Arithmetic expression
Grammar Start variable:<EXPR> Variable: <EXPR> Terminals: a,+,x,(,)

183 Example Generate

184 Example Generate

185 Leftmost derivation A derivation of a string w is leftmost derivation if the left most variable is replaced at every step <EXPR><EXPR> x <EXPR> <EXPR>+<EXPR> x <EXPR> a+<EXPR> x <EXPR> a+a x <EXPR> a+a x a

186 Ambiguity A string w is derived ambiguously in context-free grammar G if it has more than one leftmost derivation Grammar G is ambiguous if it generates some string ambiguously Ambiguity gives different interpretations by computer program

187 Example Design the context free grammar for

188 Grammar Partition the problem into two parts. One part is for
The second part is Let and S1 is used to get via S2 is used to get via

189 Pushdown Automata State control aaaabbbbbabaaabb a b

190 Example The grammar generates the language
It can be accepted by pushdown automata

191 Remove input 0, push 0 into stack
State control

192 Remove input 0, push 0 into stack
State control

193 Match stack 0 with input 1. Then remove both of them
State control

194 Match stack 0 with input 1. Then remove both of them, accept
State control

195 Pushdown Automata Let M be is the set of states is the input alphabet
is the stack alphabet is the state transition function is the start state, is the set of accept states q2 q1 q3 q4

196 States Explanation Q1 is the start state Q2 is used to push 0 symbol
Q3 is used to match 0,1 pairs between stack and input tape Q4 is the accept state

197 Transition a,bc: Used when the current input symbol a and the stack top symbol b, Remove the top symbol on the stack and push c on it The input is moved to the next after a

198 Difference between Pushdown and finite state automata
Pushdown automata has unlimited memory, which is last in, first out. Nondeterministic finite state automata is a special case of pushdown automata that has no memory

199 Pushdown Automata 6-tuple Q is the finite set of states
is the input alphabet (finite) is the stack alphabet (finite) is the start state, is the set of accept states

200 A computation of pushdown automata
Input string: State transition sequence: For i=0,…,m-1, where and Final state accepts

201 Theorem Every context free language can be accepted by a non-deterministic pushdown automata

202 Example The grammar generates the language
It generates the string 0011

203 Pushdown Automata State control

204 Replace S by 0S1 at stack State control

205 Match the stack top symbol with the input symbol, remove both if matched
State control

206 Replace S by 0S1 State control

207 Remove after matching State control

208 Replace S by empty State control

209 Remove after matching State control

210 Remove after matching, Accept
State control

211 Proof Idea Push $ and start symbol to the stack in the beginning.
Repeat the three steps below Replace the top variable A on the stack with a the right side of a rule Match the top terminal on the stack with the input symbol, reject if not matched When stack has $ on the top and all input has been read, accepts

212 How to replace the variable
If the current stack has the top element s, replace it with the right of the rule State q a…………….. s .

213 New States The states are new states, specially added for the rule
State moves from q to r after the variable is replaced on the top of the stack

214 Pumping Lemma Lemma: If A is context-free language, there is a number p such that if s is in A and of length at least p, s may be divided into s=uvxyz, satisfying 1) for each 2) 3)

215 Proof Idea Two variables are the same on a path

216 Proof Idea A path from root to a leaf has all variables except the last one, which is a terminal When the path is too long, same variable has to happen twice. Repeat the part of the two equal variables area.

217 Analysis Let V be the set of variables in a grammar.
|V| is the number of variables in V If a path has at least |V|+1 variables, two of them will be equal. A path from root to a leaf has least |V|+1 variables if its length is at least |V|+2 A tree of depth |V|+2 has leaves, where b is the maximal length of right side among all rules.

218 Proof Let If s has length , the parse tree T has height
It has a path has at least |V|+1 variables. Two of the variables on the path are equal (=R). We have So,

219 Proof The tree T is the parsing tree for s and has least size
It is impossible that is empty otherwise, T is not least. Select the bottom |V|+2 symbols on the longest path of T, this makes the

220 Application The language is not context free
Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

221 Proof The language is not context free Proof. Let
By Pumping lemma, s can be expressed as s=uvxyz s. t And Case 1. v and y are in the area of the same symbol, say a. Contradiction for having more a than b. Case 2. v or y contains more than one symbol, contradiction for the incorrect order of symbols in

222 Regular Pumping The language is not regular
Proof. Assume it is a regular language. It can be accepted by automata M. Let p be the length of the pumping lemma. Consider s can be expressed s=xyz, where The string xz has the number of 0s no more than 1s (Pump it down to get the contradiction)

223 Application The language is not context free
Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

224 Proof The language is not context free Proof. Let
By Pumping lemma, s can be expressed as s=uvxyz s. t And Case 1. vy have intersection with the a area. Contradiction for having more a or b than c (pumping up). Case 2. vy does not contain symbol a, contradiction for having more a than b or c (pumping down).

225 Application The language is not context free
Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

226 Proof The language is not context free Proof. Let
By Pumping lemma, s can be expressed as s=uvxyz s. t And Case 1. vy is in the first half. Contradiction for moving 1 to the right area(pumping up). Case 2. vy is in the second half. Contradiction for moving 0 to the left area(pumping up).

227 Proof The language is not context free
Proof. Case 3. vy crosses the middle line. Contradiction for reducing 0 or 1 in the middle area (pumping down). The left half and right half have different number of 1s or 0s.

228 Problem Write a context-free grammar for the following language

229 Problem Using pumping lemma to disprove the following language is context-free:

230 Algorithm An algorithm is a collection of simple instructions for carrying out some task

231 Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

232 Unlimited Register Machine
Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m Each register can save an unbounded integer

233 Turing Machine Proposed in 1936
An accurate model for the general purpose computer.

234 Turing Machine Write on the tape and read from it
Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

235 Language ww Design Turing machine for L={w#w|w is in {0,1}*}
For example: 011#011 is in L 10011#10011 is in L

236 Movement on the tape Move the head back and forth to match all pairs

237 Movement on the tape Move the head

238 Turing Machine 7-tuple Q is the finite set of states
is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

239 State transition function
For

240 State transition function
For

241 Configuration Current state: q7
Current head position on the tape: 4th cell Current tape content: abab q7 a b a b

242 Configuration A configuration is represented by
Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

243 Configuration Transition
For

244 Configuration Transition
For

245 Configuration Start configuration: , where w is the input
Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

246 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where is the start configuration of M on input w, 2. each yields , and is an accepting configuration

247 Language recognized by TM
For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

248 a q1 q3 q2 q4 q5 q7 q6 q9 q8 q11 q10 q12 q13 q_accept q14

249 Multi-tape Turing Machine
1 1 Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control a a a b a

250 Multi-tape Turing Machine
Transition for multi-tape Turing machine

251 Simulate Multitape Turing machine
Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 # 1 1 # a a a # b a #

252 Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

253 Another simulation Assume there are k tapes
Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 a b 1 a a a

254 Simulate 3 tapes q7 a b 1 a a a

255 Simulate 3 tapes q7 a b 1 a a a

256 K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j-1)k+2i-1 and 2(j-1)k+2i

257 Midterm October 18, 2010 Class Time Close book Chapter 0-Chapter 3

258 Problem 5 Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains an equal number of 0s and 1s}.

259 Turing Machine Write on the tape and read from it
Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

260 Language L Design Turing machine for L={w|w is in {0,1}* and the same number 1s as the same number of 0s} For example: is in L is not in L

261 Movement on the tape Move the head back and forth to pair up 0 and 1.

262 Movement on the tape Move the head

263 Turing Machine Write on the tape and read from it
Head can move left and right Tape is infinite Rejecting and accepting states Control 1 1

264 Language L Design Turing machine for L={w|w is in {0,1}* and the same number 0s is two times the number of 1s} For example: is in L is not in L

265 Movement on the tape Move the head back and forth to pair up 0 and 1.

266 Movement on the tape Move the head

267 Problem 6 Problem 6 (10) If A and B are languages, define
and Prove that if A and B are regular languages, then is a context free language.

268 Problem 6 DFA N1 accepts A DFA N2 accepts B

269 Construct N N accepts

270 Problem 7 Problem 7 (10) If A and B are languages, define
and and |x|=|y|} Prove that if A and B are regular languages, then is a context free language. You will get 10 more points for midterm.

271 Deterministic Turing Machine
7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

272 Nondeterministic Turing Machine
7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

273 Nondeterministic Turing Machine
One configuration can have multiple choice for entering the next configuration because of

274 Non-determinism blind monkey

275 Automata example q2 q1 q3 q3 q4 q2

276

277 Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

278 Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

279 Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

280 Simulate NTM with DTM Each dot is one configuration
Find an accepting path

281 Simulate NTM Think about the problem from the Java programming point of view Convert the strategy into deterministic Turing machine.

282 Simulate NTM Key points: search accepting path with width first until it finds the first one. Search from the left to right Search from small level 0, to level 1, to level 2,…

283 Simulate NTM 1: first branch, 2: second branch, etc
1 1 1: first branch, 2: second branch, etc 121: take the first branch at level 2, then take the second at level 3, then take the first branch at level 4. Control a a a 1 2 1

284 Turing machine and Computer
The computational power of Turing machine is equivalent to regular computer with unlimited memory

285 Successor Funtion f(x)=x+1

286 Successor Funtion f(x)=x+1

287 Successor Turing machine is able to simulate the operation xx+1 1 1 1
1 1 1 1 1

288 Comparison Turing machine is able to check if x=y 1 1 1 1 1 1 1 1 1 1

289 Transfer Turing machine is able to simulate yx 1 1 1 1 1

290 Transfer Turing machine is able to simulate xy 1 1 1 1 1 1 1 1 1 1

291 Set zero Turing machine is able to simulate x0 1 1 1 1 1

292 Program statements Statement1: xx+1 Statement2: xy Statement3: x0
Statement4: if (x==y) jump m

293 Compute Algorithm: Let x be added by 1 y times Program for x+y: 1: z0
2: If (z==y) jump 6 3: xx+1 4: zz+1 5: If (z==z) jump 2 6:

294 Jump TM  program state Program q1 1: I1 q2 2: I2 q3 3: I3 …. ……
… …… qk k: jump 3 The jump can be achieved via state transition qkq3

295 Compute Algorithm: Let x be added by original x y times
Program for x*y: 1: z0 2: p0 3: If (z==y) jump 7 4: pp+x 5: zz+1 6: If (z==z) jump 3 7:

296 Compute Algorithm: Let x be multiplied by x y times Program for :
1: z0 2: p (via p0 and pp+1) 3: If (z==y) jump 7 4: pp*x 5: zz+1 6: If (z==z) jump 3

297 Algorithm An algorithm is a collection of simple instructions for carrying out some task

298 Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

299 Unlimited Register Machine
Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m Each register can save an unbounded integer

300 Other computation model
Lambda calculus (Church) Recursion (Godel and Kleene) Post and Markov’s model

301 Evidences Many approaches led to the same algorithmic computable class
No one has found an algorithm that is accepted in informal sense, but it can not be implemented in Turing model

302 Hilbert 10th Problem Find an algorithm to decide if a polynomial has integer root Input: a polynomial e.x. (it has root x=5,y=3,z=0) Output: yes or no

303 Midterm Problem 1 (20) Give the state diagram of a DFA recognizing {w|w is 0,1-string with at least five 0s}. Problem 2 (20) Prove that is not regular language with the pumping lemma.

304 Midterm Problem 3 (20) Prove the following facts:
a) If A and B are context free languages, then so is their union . b) If A and B are context free languages, then so is their concatenation.

305 Midterm Problem 4 (20) a) Design a context-free grammar to recognize the language b) Prove that is not a context-free language by using pumping lemma.

306 Midterm Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains more 0s than 1s}.

307 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

308 DFA ={<B,w>| B is a DFA that accepts input string w}
Theorem: is a decidable language.

309 Proof The input is <B,w>, where B is a DFA , and w is a string.
Simulate B on the input w: Start from the state of B and leftmost symbol w. Follow to transit the state and move the input symbols one by one Accepts if ends at an accept state; rejects otherwise.

310 NFA ={<B,w>| B is a NFA that accepts input string w}
Theorem: is a decidable language.

311 Proof The input is <B,w>, where B is a NFA and w is a string.
Convert B into a DFA C via our previous algorithm Use our last TM to decide if C accepts w Accepts if it accepts; rejects otherwise.

312 ={<A>| L(A) is not empty}
Theorem: is a decidable language.

313 Proof The input is <A>, where A is a DFA. Mark the start state
Repeat Mark a new state that has a transition to it from a marked one. Until no new state can be added Accept if an accept state is marked; otherwise reject

314 ={<A,B>| A and B are DFA and L(A)=L(B)}
Theorem: is a decidable language.

315 Proof The input is <A,B>, where A and B are DFA Check if
is empty

316 Problem Define L={ <A,B,C>: L(A) is the union of L(B) and L(C)}, where A, B, and C are DFAs. Prove that L is decidable.

317 One- one and onto Let A and B be two sets.
For function f: AB, if whenever then f is called one-one. For function f: A B, say f is onto if f hits every element of B(In other words, for very b in B, there is a in A such that b=f(a))

318 Correspondence Let A and B be two sets.
A and B are of the same size if there is a one-one and onto function f: AB For function f: AB, if it is both one-one and onto, then f is called correspondence.

319 Examples {1,2,3,…} and {2,4,6,…} are of the same size via f(x)=2x.

320 Countable A set is countable if it is finite or it has the same size as N={1,2,3,…} Theorem: The positive rational numbers set is coutable

321 Proof Every positive rational number is in the table below

322 Proof List all of them by and avoid repetition

323 Examples For two positive rational numbers , ( p and q have no common divisor>1, and p’ and q’ have no common divisor >1), The number is listed before if p+q<p’+q’, or p+q=p’+q’ and q<q’

324 (0,1) is not countable Theorem: (0,1) is not countable

325 Proof. Proof by contradiction.
Assume N={1,2,3,…} and (0,1) have the same size. There is one-one and onto map f: N (0,1)

326 Proof. Select such that and Since

327 Infinite binary strings set is not countable
Let be the set of all infinite binary strings Theorem: is not countable

328 Proof. Proof by contradiction.
Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

329 Proof. Select such that and Since

330 The set of finite binary strings is countable
Correspondence N= {0, 1 , 2, 3, 4, 5, 6, 7, , , 10,…}

331 Language of Binary Strings
Every set A of binary strings is countable Proof. Let A be a set of binary strings. Its elements can be listed according to their orders in A={ s1, s2, s3, s4, ….} It is easy to see the correspondence between N and A

332 An infinite binary string uniquely determines a language
The language with positive even number of 1s strings

333 Correspondence For each infinite binary string B, it uniquely determines a binary language L(B). If B1 and B2 are different binary strings, then L(B1) and L(B2) are different language Each binary language uniquely determines an infinite binary string.

334 Correspondence There is a correspondence between the set of all infinite binary strings and the set of all binary languages. The set of all binary languages is not countable

335 Turing machine to binary string
Each Turing machine can be encoded into a binary string. The set of all Turing machines can be encoded into a set of binary strings. The set of Turing machines is countable. The set of binary languages recognized by TM is also countable

336 Un-computable language by TM
There is a binary language that is not Turing recognizable Proof. The set of binary languages is not countable, but the set of language recognized by Turing machine is countable

337 Undecidable Problem ={<M,w>| M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

338 Every entry of the table can be obtained in finite steps
accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

339 Proof Assume that is decidable There is a TM H such that
H(<M,w>) accepts if M accepts w H(<M,w>) rejects if M rejects w. Consider a TM D D(<M>) accepts if M rejects <M> D(<M>) rejects if M accepts <M>

340 Proof Since D(<M>) accepts if M rejects <M>
D(<M>) rejects if M accepts <M> We have D(<D>) accepts if D rejects <D> D(<D>) rejects if D accepts <D> A contradiction

341 Every entry of the table can be obtained in finite steps
accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

342 Every diagonal entry of the table can be obtained in finite steps
accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

343 Diagonal method reject reject accept accept accept reject ……

344 Call H in R If TM H exists, the TM R also exits via using H
(Software R uses an existing software H)

345 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

346 Problem Is there any one-one and onto map from the set of integers in [1,10] and the set of odd integers in [1,10]? Why? Prove that there is a one-one and onto map from the set of all integers and the set of all odd integers.

347 Reduction Solution for Problem 1 Solution for Problem 2 Help

348 Example: Map and Direction

349 Reduction Problem 2 Solution for Problem 1 Question Answer

350 Reduction Software for Problem 2: Software for Problem 1 call Return

351 Reduction & Undecidability
It is known that P2 is undecidable We want to prove P1 is undecidable Proof by contradiction Assume P1 is decidable. There is a software solving P1 Design a software for P2 by calling the software for P1 Contradiction!!!!

352 Halting Problem P1: ={<M,w>| M is a Turing machine and M halts on input w} P2: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

353 P2 Input <M,w> If M does not stop on w, reject it
If M stops, simulate M. If M acceps, accpets Otherwise

354 Halting Problem ={<M,w>| M is a Turing machine and M halts on
input w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

355 Undecidable Problem ={<M,w>| M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

356 Proof Assume the Turing machine R decides We can use R to decides
For input <M, w> Run R on <M,w> if R rejects, “reject” if R accepts, simulate M until it stops if M accepts, “accept” else “reject”

357 Empty Problem ={<M>| M is a Turing machine and }
Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

358 Proof Assume the Turing machine R decides
For input <M,w>, design another TM If , reject simulate M on input w, accepts if M accepts w

359 Proof If M accepts w, then w belongs to L( ) Otherwise, L( ) is empty

360 Proof Use R to decide Input <M,w> Make from <M,w>
Run R on the input If R rejects (it means L( ) is not empty), accepts I R accepts (it means L( ) is empty), rejects

361 Problem ={<M>| M is a Turing machine and L(M) is regular}
Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

362 Proof Assume the Turing machine R decides
For input <M,w>, design another TM (x) If x has format , accept simulate M on input w, accepts if M accepts x

363 Proof If M accepts w, then Otherwise,

364 Proof Use R to decide Input <M,w> Make from <M,w>
Run R on the input If R rejects (it means L( ) is not regular), accepts I R accepts (it means L( ) is regular), rejects

365 Turing machine equivalence Problem
={ | M1 and M2 are Turing machines and L(M1)=L(M2) } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

366 Proof For input <M,w> for TM M1: rejects any input
TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

367 Mapping Reducibility Convert instances of problem to instances of problem B A B

368 Computable function Assume that is a set of finite number of symbols
is the set of all finite strings with symbols from The function is computable if some Turing machine M, on every input w, halts with just f(w) on the tape. Example: +,x,/ are all computable functions

369 Mapping Reducibility Language A is mapping reducible to language B, if there is a computable function such that for every w

370 Example A={1,3,5,….} B={0,2,4,…} f(x)=x+1 via the funtion f

371 Theorem If , and B is decidable, then A is also decidable A B

372 Proof Let f be the reduction from A to B since
Let M be the decider for B. Decider N: Input w, Compute f(w) Run M on f(w) and accept iff M accepts

373 Corollary If , and A is undecidable, then B is also undecidable A B

374 Proof Proof by contradiction. If B is decidable, then so is A

375 Halting Problem P1: ={<M,w>| M is a Turing machine and M halts on input w} P2: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

376 A reduction from to B: ={<M,w>| M is a Turing machine and M halts on input w} A: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

377 Reduction On input <M,w> Construct Turing machine M’ Run M on w
If M accepts, accept If M rejects, enter an infinite loop Output <M’,w>

378 Empty Problem ={<M>| M is a Turing machine and }
Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

379 Reduction from to For <M,w>, design another TM M’ Input x
If x is not equal to w, reject else simulate M on input w, accepts if M accepts w

380 Reduction from to <M,w> is in L(M’) is not empty
<M,w> is not in L(M’) is empty

381 Reduction from to <M,w> is in M’ is in
<M,w> is not in M’ is not in

382 Problem Let ={M| M is a Turing machine and M prints 1 for some input w} Show that L is undecidable.

383 Turing Machine Write on the tape and read from it
Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

384 Turing Machine 7-tuple Q is the finite set of states
is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

385 Configuration Current state: q7
Current head position on the tape: 4th cell Current tape content: abab q7 a b a b

386 Configuration A configuration is represented by
Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

387 Configuration Transition
For

388 Configuration Transition
For

389 Configuration Start configuration: , where w is the input
Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

390 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where is the start configuration of M on input w, 2. each yields , and is an accepting configuration

391 Language recognized by TM
For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

392 Turing Recognizable Turing machine M recognizes language L

393 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

394 Turing Decidable Turing machine M decides language L

395 Observation If L is Turing decidable, then L is Turing recognizable

396 A Turing recognizable problem
={<M,w>| M is a Turing machine and M accepts w} Turing machine R recognizable For input <M,w> Simulate M on w R accepts <M,w> if M accepts w.

397 Theorem Theorem: If L is a language in {0,1}*, there is a Turing machine to print out all elements in L

398 Proof Let M recognize L Step i(i=1,2,3…)
Simulate M each of the first i strings i steps If a string is accepted, print it out.

399 Theorem If , and B is Turing recognizable, so is A A B

400 Proof Let f be the reduction from A to B since Let M recognize B.
TM N: Input w, Compute f(w) Run M on f(w) and accept if M accepts

401 Theorem If , and A is not Turing recognizable, B is not Turing recognizable A B

402 Complement set Set and its complement

403 Theorem If A and are Turing recognizable, then A is decidable

404 Proof Let Turing machine M1 recognize A
For input x, Run M1 and M2 on x in parallel If M1 accepts, accept If M2 accepts, reject.

405 ={<A,B>| A and B are DFA and L(A)=L(B)}
.

406 ={<A,B>| A and B are DFA and L(A)=L(B)}
Theorem: is not Turing decidable language.

407 . Theorem: Neither nor its complement is Turing recognizable

408 Proof We first prove For input <M,w> for
TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

409 Proof We prove For input <M,w> for TM M1: accepts any input
TM M2: accepts any input if M accepts w. It is easy to see L(M1)=L(M2) iff M accepts w So,

410 Problem Show that the language F={M| L(M) contains infinite elements} is not Turing recognizable}.

411 Problem 4.2 The equivalence of a DFA and a regular expression is decidable

412 Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

413 Problem 4.7 The set of all infinite binary strings is not countable

414 Problem 4.8 N={1,2,3,…} NxNxN is countable

415 Problem 5.9 All Turing recognizable problems are mapping reducible to

416 Problem 5.12 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

417 Problem 5.13 Test if a Turing machine has useless state, which never enters.

418 Language Design Turing machine to recognize

419 Complexity Let M be a deterministic Turing machine. The running time of M is the function f:N N such that f(n) is the maximum number of steps that M uses on input of length n.

420 Big O-notation Let f and g be functions Say f(n)=O(g(n)) if integers c and exist so that for all

421 Examples Let

422 Small o-notation Let f and g be functions Say f(n)=o(g(n)) if

423 One tape Turing machine time
One tape Turing machine can recognize In steps

424 Two tapes Turing machine time
Two tapes Turing machine can recognize In O(n) steps

425 Multi-tape Turing Machine
1 1 Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control a a a b a

426 Multi-tape Turing Machine
Transition for multi-tape Turing machine

427 Simulate Multitape Turing machine
Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 # 1 1 # a a a # b a #

428 Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

429 Another simulation Assume there are k tapes
Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 a b 1 a a a

430 Simulate 3 tapes q7 a b 1 a a a

431 Simulate 3 tapes q7 a b 1 a a a

432 K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j-1)k+2i-1 and 2(j-1)k+2i

433 Simulate multi-tape by one tape
One tape Turing machine can simulate t(n) time multi-tape Turing machine in time

434 Time Complexity Class Let .
is the class of languages decided by O(t(n) time Turing machines. In other words is a language decided by O(t(n)) time Turing machine}

435 One tape Turing machine time
One tape Turing machine can decide In steps

436 Complexity Class P P is the class of languages decided by single tape Turing machine in polynomial time

437 Directed Path Problem G is a directed graph that has a directed path from s to t} a t c s d b

438 Algorithm Input mark “s” Repeat
For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

439 Greatest common divisor
Divisor: For two integers b and c, if b=c*z for some integer z, c is a divisor of b. Greatest common divisor: Given two integers a and b, gcd(a,b) is the greatest positive integer c such that c is the divisor for both a and b. Examples: gcd(10,4)=2, gcd(16,100)=4 Problem: How to find gcd(a,b)?

440 Relatively Prime Problem
Two integers x and y are relatively prime if gcd(x,y)=1. RELPRIME={<x,y>| x and y are relatively prime} Theorem:

441 Euclid algorithm For two integer b and a a=q*b+c with
gcd(a,b)=gcd(b,c) a and b are relatively prime iff b and c are relatively prime

442 Euclid algorithm Assume a1 and a2 are two positive integers
If , then and Therefore, always true

443 Algorithm for gcd(x,y) Input Repeat rx mod y xy yr until y=0
output x

444 Modular Assume a and b are two positive integers
This is a recursive equation since the second item goes down

445 Example Find gcd(1970,1066)

446 Speed of Euclid algorithm
Assume a1 and a2 are two positive integers If , we have In another words,

447 Euclid algorithm Assume a1 and a2 are two positive integers

448 Time For input <a,b> The total time of steps is O(log a+log b)

449 Non-determinism blind monkey

450 Automata example q2 q1 q3 q3 q4 q2

451

452 Deterministic Turing Machine
7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

453 Nondeterministic Turing Machine
7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

454 Nondeterministic Turing Machine
One configuration can have multiple choice for entering the next configuration because of

455 Non-determinism blind monkey

456 Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

457 Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

458 Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

459 Simulate NTM with DTM Each dot is one configuration
Find an accepting path

460 Class NP NP is the class of languages that can be recognized by nondeterministic Turing machine in polynomial time

461 Hamiltonian Path Hamiltonian path goes through each node exactly once
HAMPATH={<G,s,t>| G is a directed graph with a Hamiltonian path from s to t}

462 Composite A natural number is composite if it is the product of two integers >1 10=2* =2*9 Composite={x| x=pq, for integers p,q>1}

463 Verifier A verifier for a language L is an algorithm V,
L={w| V accepts <w,c> for some string c} For the verifier V for L, c is a certificate of w if V accepts <w,c> If the verifier V for the language L runs in polynomial time, V is the polynomial time verifier for L.

464 Verifier for Hamiltonian Path
For <G,s,t>, a certificate is a list of nodes of G: Verifier: check if m is the number of nodes of G check if and check if each is a directed edge of G for i=1,…,m-1 If all pass, accept . Otherwise, reject.

465 Verifier for Composite
For integer x, a certificate is two integers p,q: Verifier: check if p>1 and q>1 check if x=pq If all pass, accept . Otherwise, reject.

466 Class NP NP is the class of languages that have polynomial time verifiers. Examples: COMPOSITE is in NP HAMPATH is in NP

467 Theorem A language has polynomial verifier iff it can be recognized by polynomial time nondeterministic Turing machine Proof: Assume L has verifier V, …  Assume L has NTM M,…

468 Proof Assume that L has polynomial time verifier V, which runs in time, where k is a constant NTM M Nondeterministically select string c of length Run V on <w,c> If V accepts, accept; Otherwise, reject.

469 Proof Assume that L is recognized by polynomial time NTM M, which runs in time, where k is a constant Verifier <w,c> Let c determine a computation path of M Simulate M on the path c If M accepts, accept. Otherwise, reject.

470 Clique Problem Given undirected graph G, a clique is a set of nodes of G such that every two nodes are connected by an edge. A k-clique is a clique with k nodes

471 Clique Problem CLIQUE={<G,k>| G iss an undirected graph with k-clique} CLIQUE is in NP.

472 Subset Sum Problem SUBSET-SUM={<S,t>| S= and for some , we have

473 Polynomial Time Computable
A function is a polynomial time computable function if some polynomial time Turing machine M exists that outputs on the tape for input w.

474 Polynomial Time Reduction
Assume that A and B are two languages on A is polynomial time mapping reducible to A if a polynomial time computable function exists such that

475 Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

476 SAT A boolean formula is satisfiable if there exists assignments to its variables to make the formula true SAT={ | is satisfiable boolean formula}

477 3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

478 3SAT to CLIQUE Example:

479 NP-completeness A language B is NP-complete if B is in NP, and
Every A in NP is polynomial time reducible to B Theorem. If B is NP-complete and B is in P, then P=NP.

480 Cook-Leving Theorem Theorem: SAT is NP-complete Proof.
1. SAT is in NP. 2. For every problem A in NP,

481 Proof The start configuration is legal The final state is accept.
The movement is legal. Each cell takes one legal symbol.

482 Proof 1 if The cell[i,j] holds symbol s; 0 otherwise
Time bound for the NTM M with constant k. The movement is legal. NTM M for accepting A.

483 Nondeterministic Turing Machine
Q is the finite set of states is the tape alphabet is the start state, is the accept state.

484 Configuration Transition
For

485 Configuration Transition
For

486 Configuration Start configuration: , where w is the input
Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

487 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where is the start configuration of M on input w, 2. each yields , and is an accepting configuration

488 Language recognized by TM
For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

489 Proof Each cell has only one symbol The symbol is selected from C:
Only one symbol is selected: It is true for all cell at all configuration:

490 Proof The start configuration is

491 Proof Accept computation has reached.
It makes sure the accept state will appear among the configuration transitions.

492 Proof Characterize the legal move
The whole move is legal if all windows are legal. Characterize one window is legal

493 Proof The state transition

494 Logic Demorgan Law:

495 Truth table for y y x2

496 Convert to CNF Conversion:

497 Convert to CNF Conversion:

498 Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

499 Prepare for the Final Regular language and automata
Context free language Decidability Undecidability Complexity theory

500 Regular Language Concepts: Automata, regular expression
Skills: Design automata to accept a regular language Disprove a language is a regular

501 Context-free Language
Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

502 Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

503 Undecidability Concepts: Countable, Turing undecidable, reduction
Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

504 Complexity Concepts: Time on Turing machine PTIME(t(n))
NP-completeness Polynomial time reduction Polynomial time verifier

505 Complexity Skill: Prove a problem is in P Prove a problem is in NP
Use reduction to prove a problem is NP-complete.

506 Grade A:… B:… C: Miss exam or homework

507 SAT’ A conjunctive normal form is a conjunction of some clauses
SAT’={ | is satisfiable conjunctive normal form}

508 Cook-Leving Theorem’ Theorem: SAT’ is NP-complete
Proof. Same as that for SAT is NP-complete

509 3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

510 3SAT is NP-complete Theorem: There is polynomial time reduction from SAT’ to 3SAT.

511 3SAT is NP-complete is satisfiable if and only if the following is satisfiable

512 3SAT is NP-complete is satisfiable if and only if the following is satisfiable

513 3SAT is NP-complete Convert every clause into 3cnf:

514 3SAT is NP-complete Conjunctive normal form
Each clause is convert into is satisfiable if and only if the following is satisfiable

515 Problem Convert the formula F into 3SAT formula F’ such that F is satisfiable iff and F’ is satisfiable.

516 Approximation Algorithms

517 Outline and Reading Approximation Algorithms for NP-Complete Problems
Approximation ratios Polynomial-Time Approximation Schemes 2-Approximation for Vertex Cover Approximate Scheme for Subset Sum 2-Approximation for TSP special case Log n-Approximation for Set Cover

518 Approximation Ratios Optimization Problems
We have some problem instance x that has many feasible “solutions”. We are trying to minimize (or maximize) some cost function c(S) for a “solution” S to x. For example, Finding a minimum spanning tree of a graph Finding a smallest vertex cover of a graph Finding a smallest traveling salesperson tour in a graph

519 Approximation Ratios An approximation produces a solution T
T is a k-approximation to the optimal solution OPT if c(T)/c(OPT) < k (assuming a min. prob.; a maximization approximation would be the reverse)

520 Polynomial-Time Approximation Schemes
A problem L has a polynomial-time approximation scheme (PTAS) if it has a polynomial-time (1+)-approximation algorithm, for any fixed  >0 (this value can appear in the running time). Subset Sum has a PTAS.

521 Vertex Cover A vertex cover of graph G=(V,E) is a subset W of V, such that, for every (a,b) in E, a is in W or b is in W. OPT-VERTEX-COVER: Given an graph G, find a vertex cover of G with smallest size. OPT-VERTEX-COVER is NP-hard.

522 Copyright © The McGraw-Hill Companies, Inc
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

523 Copyright © The McGraw-Hill Companies, Inc
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

524 A 2-Approximation for Vertex Cover
Every chosen edge e has both ends in C But e must be covered by an optimal cover; hence, one end of e must be in OPT Thus, there is at most twice as many vertices in C as in OPT. That is, C is a 2-approx. of OPT Running time: O(m) Algorithm VertexCoverApprox(G) Input graph G Output a vertex cover C for G C  empty set H  G while H has edges e  H.removeEdge(H.anEdge()) v  H.origin(e) w  H.destination(e) C.add(v) C.add(w) for each f incident to v or w H.removeEdge(f) return C

525 Problem 1 Show that T={(i,j)| both I and j are positive rational numbers} is countable.

526 Problem 2. Find a correspondence between (0,1) and [0,1]

527 Solution Let for n=1, 2, … Define the correspondence f(x) below:

528 Problem 3 3. Let A={<R,S> | R and S are regular expressions and L(R ) is a subset of L( S)}. Show that A is decidable.

529 R S Solution For two regular expression R and S, L( R) is a subset of L(S) if and only if the L(R ) has no intersection with . Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Check if L(M4) is empty

530 Problem 4 I={x| x is an irrational number in (0,1)}. Show that I is not countable.

531 Homework 2009

532 Problem 4.2 The equivalence of a DFA and a regular expression is decidable

533 Solution Proof. Given a regular language L and DFA M.
We can construct a DFA N to accept L. The problem is converted into checking if two DFAs are equivalent. By Theorem 4.5, there is an algorithm to decide if L(M)=L(N).

534 Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

535 Solution Assume M is a DFA with n states.
L(M) is infinite iff it has path from the start state to accept with a loop on it, and the loop length is no more than n. L(M) is infinite iff M accepts a string of length between n and 2n. For every string x with length between n and 2n Run M on x If M accepts x, then output “L(M) is infinite” else output “no”

536 Problem 4.7 The set of all infinite binary strings is not countable

537 Proof. Proof by contradiction.
Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

538 Proof. Select such that and Since

539 Problem 1 N={1,2,3,…} NxNxN is countable

540 Proof List all of them by

541 Proof Use the similar method like the last slide to prove the NxN is countable Let f: NxNN be one-one and onto We have g: NxNxNN is one-one and onto, where g(i,j,k)=f(f(i,j),k)

542 Problem 2. Find a correspondence between (0,1) and [0,1]

543 Solution Let for n=1, 2, … Define the correspondence f(x) below:

544 Problem 3 3. Let A={<R,S> | R and S are regular expression and L(R ) = L( S)}. Show that A is decidable.

545 Solution For two regular expression R and S, L( R)=L(S) if and only if their symmetric difference is empty. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Construct automata M5 to accept Check if L(M5) is empty

546 Problem 4 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

547 Proof We are going to design a reduction from to S. For <M,w>
Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts <M,w> is in iff N accepts both 01 and 10.

548 Problem 5 5. Let S={<M> | M is a TM and L(M) = {<M>}}. Show that neither S nor its complement is Turing-recognizable.

549 Solution We first show that can be reducible to S.
Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal <Y> then reject If x=<Y> then accepts if M(w) accepts.

550 Solution For an input (M, w), derive M1(x, M1).

551 Solution We show that can be reducible to . Let (M, w) be an input for
Let Turing machine M2 be constructed below: M1 Input x If x =<M1> then accept If x is not equal <M1> then accept if M(w) accepts.

552 Solution For an input (M, w), derive M2(x, M2).

553 Problem 6 Show that A is a Turing recognizable if and only if

554 Proof Part a)  Let A be a Turing recognizable language.
There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =<M,w>. It is easy to see that

555 Proof Part b)  Assume that via computable function g.
It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

556 Proof We are going to design a reduction from to it For <M,w>
Design the TM N as follows Input x if x=0, enters all states except if x=1 run M on w, if M accepts w, enter <M,w> is in iff N enters all states.

557 Problem 4 c) The language is not a context free language
Proof: Assume it is a context free language. Let p be the number from the pumping lemma Consider the string

558 Final Exam December 13, 5:45-7:30pm, Monday

559 Homework 4, 20

560 Problem 1 1.Let T1={<M>| M is a Turing machines and accepts infinite number of 0,1-strings of finite length}. Prove that T1 is undecidable.

561 Proof We are going to design a reduction from to T1.
For input Turing machine <M,w> for Design the TM N as follows Input x run M on w, if M accepts w, acceptx <M,w> is in iff N acccepts each input x.

562 Problem 2 Let T2={<M>| M is a Turing machines and accepts five of 0,1-strings of finite length}. Prove that T2 is undecidable.

563 Proof We are going to design a reduction from to T2. For <M,w>
Design the TM N as follows Input x if x is not 0,1, 00, 01 or 10, rejects. run M on w if M accepts w, accept x <M,w> is in iff N accepts 0,1,00,01, and 10. Otherwise, N accepts no string

564 Problem 3 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

565 Proof We are going to design a reduction from to S. For <M,w>
Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts <M,w> is in iff N accepts both 01 and 10.

566 Problem 4 Show that A is a Turing recognizable if and only if

567 Proof Part a)  Let A be a Turing recognizable language.
There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =<M,w>. It is easy to see that

568 Proof Part b)  Assume that via computable function g.
It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

569 Problem 5 5. Let S={<M> | M is a TM and L(M) = {<M>}}. Show that neither S nor its complement is Turing-recognizable.

570 Solution We first show that can be reducible to S.
Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal <Y> then reject If x=<Y> then accepts if M(w) accepts.

571 Solution For an input (M, w), derive M1(x, M1).

572 Solution We show that can be reducible to . Let (M, w) be an input for
Let Turing machine M2 be constructed below: M1 Input x If x =<M1> then accept If x is not equal <M1> then accept if M(w) accepts.

573 Solution For an input (M, w), derive M2(x, M2).

574 Homework 5, 20

575 Problem 1 Call graphs G and H isomorphic if the nodes of G may be reordered so that it is identical to H. Let ISO={<G,H> | G and H are isomorphic graphs}. Show that ISO is in NP.

576 Solution for Problem 1 We design a polynomial time verification algorithm. A mapping f from the vertices of G to those of H is a witness. Check if 1) f is one-one, 2) f is onto, 3) (v,u) is an edge of G iff (f(u),f(v)) is an edge of H. It is easy to see that the verification takes polynomial time.

577 Problem 2 Let MAX-CIQUE={<G, k>| the largest clique of G has k vertices}. Whether MAX-CLIQUE is in NP is unknown. Show that if P=NP, then MAX-CLIQUE is in P, and a polynomial time algorithm exists that, for a graph G, finds one of its largest cliques.

578 Solution of Problem 2 Fist step is to find the largest k with (G,k) is in Clique. Try k from 1,2,… Check if (G,k) is in Clique Select the largest k. Assume the largest k for (G,k) in Clique is obtained.

579 Solution of Problem 2 Assume the largest k for (G,k) in Clique is obtained. Formulate the problem: (H, k,G) Determine if there is a clique of size k in G and contains all veritces in H. The problem is in NP. Extend H one by one until its size reaches k.

580 Problem 3 Let G represent an undirected graph and let SPATH={<G, a,b,k> | G contains a simple path of length at most k from a to b} and LPATH={<G, a,b,k} | G contains a simple path of length at least k from a to b}. Show that SPATH is in P. Show that LPATH is NP-complete.

581 Solution Problem 3 Show that SPATH is in P.

582 Directed Path Problem G is a directed graph that has a directed path from s to t} a t c s d b

583 Algorithm Input mark “s” Repeat
For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

584 Solution Problem 3 Show that LPATH is NP-complete.
Hamiltonian path problem is NP-complete. There is an easy reduction from Hamiltonian path to it.

585 Prepare for the Final Regular language and automata
Context free language Decidability Undecidability Complexity theory

586 Regular Language Concepts: Automata, regular expression
Skills: Design automata to accept a regular language Disprove a language is a regular

587 Context-free Language
Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

588 Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

589 Undecidability Concepts: Countable, Turing undecidable, reduction
Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

590 Complexity Concepts: Time on Turing machine PTIME(t(n))
NP-completeness Polynomial time reduction Polynomial time verifier

591 Complexity Skill: Prove a problem is in P Prove a problem is in NP
Use reduction to prove a problem is NP-complete.

592 Grade A:… B:… C: Miss exam or homework

593 Space Complexity Complexity ©D.Moshkovits

594 Motivation Complexity classes correspond to bounds on resources
One such resource is space: the number of tape cells a TM uses when solving a problem Complexity ©D.Moshkovits

595 Introduction Objectives: To define space complexity classes Overview:
Low space classes: L, NL Savitch’s Theorem Immerman’s Theorem TQBF Complexity ©D.Moshkovits

596 Space Complexity Classes
For any function f:NN, we define: SPACE(f(n))={ L : L is decidable by a deterministic O(f(n)) space TM} NSPACE(f(n))={ L : L is decidable by a non-deterministic O(f(n)) space TM} Complexity ©D.Moshkovits

597 Low Space Classes Definitions (logarithmic space classes):
L = SPACE(logn) NL = NSPACE(logn) Complexity ©D.Moshkovits

598 Problem! How can a TM use only logn space if the input itself takes n cells?! !? Complexity ©D.Moshkovits

599 3Tape Machines a b _ b _ b a _ input . . . work output . . . . . .
read-only Only the size of the work tape is counted for complexity purposes read/ write b _ . . . write-only b a _ . . . Complexity ©D.Moshkovits

600 Example Question: How much space would a TM that decides {anbn | n>0} require? Note: to count up to n, we need logn bits Complexity ©D.Moshkovits

601 Graph Connectivity CONN
An undirected version is also worth considering CONN Instance: a directed graph G=(V,E) and two vertices s,tV Problem: To decide if there is a path from s to t in G? Complexity ©D.Moshkovits

602 Graph Connectivity t s Complexity ©D.Moshkovits

603 CONN is in NL Start at s For i = 1, .., |V| {
Non-deterministically choose a neighbor and jump to it Accept if you get to t } If you got here – reject! Counting up to |V| requires log|V| space Storing the current position requires log|V| space Complexity ©D.Moshkovits

604 Configurations Which objects determine the configuration of a TM of the new type? The content of the work tape The machine’s state The head position on the input tape The head position on the work tape The head position on the output tape If the TM uses logarithmic space, there are polynomially many configurations Complexity ©D.Moshkovits

605 Log-Space Reductions Definition:
A is log-space reducible to B, written ALB, if there exists a log space TM M that, given input w, outputs f(w) s.t. wA iff f(w)B the reduction Complexity ©D.Moshkovits

606 Do Log-Space Reductions Imply what they should?
Suppose A1 ≤L A2 and A2L; how to construct a log space TM which decides A1? Wrong Solution: w Too Large! f(w) Use the TM for A2 to decide if f(w)A2 Complexity ©D.Moshkovits

607 Log-Space reductions Claim: if Then, A1 is in L
A1 ≤L A2 – f is the log-space reduction A2  L – M is a log-space machine for A2 Then, A1 is in L Proof: on input x, in or not-in A1: Simulate M and whenever M reads the ith symbol of its input tape run f on x and wait for the ith bit to be outputted Complexity ©D.Moshkovits

608 NL Completeness Definition: A language B is NL-Complete if BNL
For every ANL, ALB. If (2) holds, B is NL-hard Complexity ©D.Moshkovits

609 Savitch’s Theorem Theorem: S(n) ≥ log(n) NSPACE(S(n))  SPACE(S(n)2)
Proof: First we’ll prove NLSPACE(log2n) then, show this implies the general case Complexity ©D.Moshkovits

610 Savitch’s Theorem Theorem: NSPACE(logn) SPACE(log2n) Proof:
First prove CONN is NL-complete (under log-space reductions) Then show an algorithm for CONN that uses log2n space Complexity ©D.Moshkovits

611 CONN is NL-Complete Theorem: CONN is NL-Complete
Proof: by the following reduction: s L t “Is there a path from s to t?” “Does M accept x?” Complexity ©D.Moshkovits

612 Technicality Observation:
Without loss of generality, we can assume all NTM’s have exactly one accepting configuration. Complexity ©D.Moshkovits

613 Configurations Graph A Computation of a NTM M on an input x can be described by a graph GM,x: A vertex per configuration the start configuration s t the accepting configuration (u,v)E if M can move from u to v in one step Complexity ©D.Moshkovits

614 Correctness Claim: For every non-deterministic log-space Turing machine M and every input x, M accepts x iff there is a path from s to t in GM,x Complexity ©D.Moshkovits

615 CONN is NL-Complete Corollary: CONN is NL-Complete
Proof: We’ve shown CONN is in NL. We’ve also presented a reduction from any NL language to CONN which is computable in log space (Why?)  Complexity ©D.Moshkovits

616 A Byproduct Claim: NLP Proof:
Any NL language is log-space reducible to CONN Thus, any NL language is poly-time reducible to CONN CONN is in P Thus any NL language is in P.  Complexity ©D.Moshkovits

617 What Next? We need to show CONN can be decided by a deterministic TM in O(log2n) space. Complexity ©D.Moshkovits

618 The Trick “Is there a path from u to v of length d?”
“Is there a vertex z, so there is a path from u to z of size d/2 and one from z to v of size d/2?” d/2 d/2 u z v d Complexity ©D.Moshkovits

619 Recycling Space The two recursive invocations can use the same space
Complexity ©D.Moshkovits

620 The Algorithm Boolean PATH(a,b,d) {
if there is an edge from a to b then return TRUE else { if d=1 return FALSE for every vertex v { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE Complexity ©D.Moshkovits

621 Example of Savitch’s algorithm
boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE 2 3 1 4 (a,b,c)=Is there a path from a to b, that takes no more than c steps. (1,4,3)(1,3,2)(2,3,1)TRUE (1,4,3)(1,2,2)TRUE (1,4,3)(1,2,2) (1,4,3)(2,4,1) (1,4,3)(2,4,1)FALSE (1,4,3)(1,3,2)TRUE (1,4,3) (1,4,3) TRUE (1,4,3)(3,4,1)TRUE (1,4,3)(1,3,2)(1,2,1) (1,4,3)(1,3,2)(2,3,1) (1,4,3)(3,4,1) (1,4,3)(1,3,2) (1,4,3)(1,3,2)(1,2,1)TRUE Complexity ©D.Moshkovits 3Log2(d)

622 O(log2n) Space DTM Claim: There is a deterministic TM which decides CONN in O(log2n) space. Proof: To solve CONN, we invoke PATH(s,t,|V|) The space complexity: S(n)=S(n/2)+O(logn)=O(log2n)  Complexity ©D.Moshkovits

623 Conclusion Theorem: NSPACE(logn) SPACE(log2n)
How about the general case NSPACE(S(n))SPACE(S2(n))? Complexity ©D.Moshkovits

624 The Padding Argument Motivation: Scaling-Up Complexity Claims
We have: can be simulated by… space space + non-determinism + determinism We want: can be simulated by… space space + non-determinism + determinism Complexity ©D.Moshkovits

625 Formally  NSPACE(s1(f(n)))  SPACE(s2(f(n)))
si(n) can be computed with space si(n) Claim: For any two space constructible functions s1(n),s2(n)logn, f(n)n: NSPACE(s1(n))  SPACE(s2(n)) NSPACE(s1(f(n)))  SPACE(s2(f(n))) simulation overhead E.g NSPACE(n)SPACE(n2)  NSPACE(n2)SPACE(n4) Complexity ©D.Moshkovits

626 Idea NTM DTM n n . . . f(n) . space: s1(.) in the size of its input
space: O(s2(f(n))) NTM n n . . . space: O(s1(f(n))) f(n) . Complexity ©D.Moshkovits

627 Padding argument Let LNPSPACE(s1(f(n))) There is a 3-Tape-NTM ML: |x|
Input babba Work  O(s1(f(|x|))) Complexity ©D.Moshkovits

628 Padding argument Let L’ = { x0f(|x|)-|x| | xL }
We’ll show a NTM ML’ which decides L’ in the same number of cells as ML. f(|x|) babba#  Input Work  O(s1(f(|x|)) Complexity ©D.Moshkovits

629 Padding argument – ML’ In O(log(f(|x|)) space
Count backwards the number of 0’s and check there are f(|x|)-|x| such. 2. Run ML on x. in O(s1(f(|x|))) space f(|x|) Input babba  Work  O(s1(f(|x|))) Complexity ©D.Moshkovits

630 Padding argument Total space: O(s1(f(|x|))) f(|x|) Input
babba  Work  O(s1(f(|x|))) Complexity ©D.Moshkovits

631 Padding Argument We started with LNSPACE(s1(f(n)))
We showed: L’NSPACE(s1(n)) Thus, L’SPACE(s2(n)) Using the DTM for L’ we’ll construct a DTM for L, which will work in O(s2(f(n))) space. Complexity ©D.Moshkovits

632 Padding Argument The DTM for L will simulate the DTM for L’ when working on its input concatenated with zeros Input babba Complexity ©D.Moshkovits

633 Padding Argument When the input head leaves the input part, just pretend it encounters 0s. maintaining the simulated position (on the imaginary part of the tape) takes O(log(f(|x|))) space. Thus our machine uses O(s2(f(|x|))) space.  NSPACE(s1(f(n)))SPACE(s2(f(n))) Complexity ©D.Moshkovits

634 Savitch: Generalized Version
Theorem (Savitch): S(n) ≥ log(n) NSPACE(S(n))  SPACE(S(n)2) Proof: We proved NLSPACE(log2n). The theorem follows from the padding argument.  Complexity ©D.Moshkovits

635 Corollary Corollary: PSPACE = NPSPACE Proof: Clearly, PSPACENPSPACE.
By Savitch’s theorem, NPSPACEPSPACE.  Complexity ©D.Moshkovits

636 Space Vs. Time We’ve seen space complexity probably doesn’t resemble time complexity: Non-determinism doesn’t decrease the space complexity drastically (Savitch’s theorem). We’ll next see another difference: Non-deterministic space complexity classes are closed under completion (Immerman’s theorem). Complexity ©D.Moshkovits

637 NON-CONN NON-CONN Instance: A directed graph G and two vertices s,tV.
Problem: To decide if there is no path from s to t. Complexity ©D.Moshkovits

638 NON-CONN Clearly, NON-CONN is coNL-Complete.
(Because CONN is NL-Complete. See the coNP lecture) If we’ll show it is also in NL, then NL=coNL. (Again, see the coNP lecture) Complexity ©D.Moshkovits

639 An Algorithm for NON-CONN
We’ll see a log space algorithm for counting reachability Count how many vertices are reachable from s. Take out t and count again. Accept if the two numbers are the same. Complexity ©D.Moshkovits

640 N.D. Algorithm for reachs(v, l)
1. length = l; u = s 2. while (length > 0) { 3. if u = v return ‘YES’ 4. else, for all (u’  V) { 5. if (u, u’) E nondeterministic switch: 5.1 u = u’; --length; break 5.2 continue } } 6. return ‘NO’ Takes up logarithmic space This N.D. algorithm might never stop Complexity ©D.Moshkovits

641 N.D. Algorithm for CRs CRs ( d ) 1. count = 0 2. for all uV {
3. countd-1 = 0 4. for all vV { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++countd-1 else fail if (v,u)  E then ++count; break 5.2 continue } 6. if countd-1 < CRs (d-1) fail 7.return count Assume (v,v)  E Recursive call! Complexity ©D.Moshkovits

642 N.D. Algorithm for CRs parameter , C) CRs ( d 1. count = 0
2. for all uV { 3. countd-1 = 0 4. for all vV { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++countd-1 else fail if (v,u)  E then ++count; break 5.2 continue } 6. if countd-1 < fail 7.return count Main Algorithm: CRs C = 1 for d = 1..|V| C = CR(d, C) return C C parameter Complexity ©D.Moshkovits

643 Efficiency Lemma: The algorithm uses O(log(n)) space. Proof:
There is a constant number of variables ( d, count, u, v, countd-1). Each requires O(log(n)) space (range |V|).  Complexity ©D.Moshkovits

644 Immerman’s Theorem Theorem[Immerman/Szelepcsenyi]: NL=coNL Proof:
(1) NON-CONN is NL-Complete (2) NON-CONNNL Hence, NL=coNL.  Complexity ©D.Moshkovits

645 Corollary Corollary: s(n)log(n), NSPACE(s(n))=coNSPACE(s(n))
Proof: By a padding argument. Complexity ©D.Moshkovits

646 TQBF We can use the insight of Savich’s proof to show a language which is complete for PSPACE. We present TQBF, which is the quantified version of SAT. Complexity ©D.Moshkovits

647 TQBF Instance: a fully quantified Boolean formula 
Problem: to decide if  is true Example: a fully quantified Boolean formula xyz[(xyz)(xy)] Variables` range is {0,1} Complexity ©D.Moshkovits

648 TQBF is in PSPACE Theorem: TQBFPSPACE
Proof: We’ll describe a poly-space algorithm A for evaluating : If  has no quantifiers: evaluate it If =x((x)) call A on (0) and on (1); Accept if both are true. If =x((x)) call A on (0) and on (1); Accept if either is true. in poly time Complexity ©D.Moshkovits

649 Algorithm for TQBF 1 1 1 1 1 xy[(xy)(xy)] y[(0y)(0y)]
(00)(00) (01)(01) (10)(10) (11)(11) 1 1 Complexity ©D.Moshkovits

650 Efficiency Since both recursive calls use the same space,
the total space needed is polynomial in the number of variables (the depth of the recursion)  TQBF is polynomial-space decidable  Complexity ©D.Moshkovits

651 PSAPCE Completeness Definition: A language B is PSPACE-Complete if
BPSPACE For every APSAPCE, APB. standard Karp reduction If (2) holds, then B is PSPACE-hard Complexity ©D.Moshkovits

652 TQBF is PSPACE-Complete
Theorem: TQBF is PSAPCE-Complete Proof: It remains to show TQBF is PSAPCE-hard: P x1x2x3…[…] “Will the poly-space M accept x?” “Is the formula true?” Complexity ©D.Moshkovits

653 TQBF is PSPACE-Hard Given a TM M for a language L PSPACE, and an input x, let fM,x(u, v), for any two configurations u and v, be the function evaluating to TRUE iff M on input x moves from configuration u to configuration v fM,x(u, v) is efficiently computable Complexity ©D.Moshkovits

654 Formulating Connectivity
The following formula, over variables u,vV and path’s length d, is TRUE iff G has a path from u to v of length ≤d (u,v,1)  fM,x(u, v)  u=v (u,v,d)  wxy[((x=uy=w)(x=wy=v))(x,y,d/2)] w is reachable from u in d/2 steps. v is reachable from w in d/2 steps. simulates AND of (u,w,d/2) and (w,v,d/2) Complexity ©D.Moshkovits

655 TQBF is PSPACE-Complete
Claim: TQBF is PSPACE-Complete Proof:   (s,t,|V|) is TRUE iff there is a path from s to t.  is constructible in poly-time. Thus, any PSPACE language is poly-time reducible to TQBF, i.e – TQBF is PSAPCE-hard. Since TQBFPSPACE, it’s PSAPCE-Complete  Complexity ©D.Moshkovits

656 Summary We introduced a new way to classify problems: according to the space needed for their computation. We defined several complexity classes: L, NL, PSPACE. Complexity ©D.Moshkovits

657  Summary Our main results were: Connectivity is NL-Complete
TQBF is PSPACE-Complete Savitch’s theorem (NLSPACE(log2)) The padding argument (extending results for space complexity) Immerman’s theorem (NL=coNL) By reducing decidability to reachability Complexity ©D.Moshkovits


Download ppt "Theoretical Foundation of Computation"