Download presentation

Presentation is loading. Please wait.

Published byHarmony Heald Modified about 1 year ago

1

2
Theoretical Foundation of Computation Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm Tuesday

3
Contents Computational models and languages finite automata, push automata regular languages, context free languages Computability theory decidable problem, un-decidable problem Complexity theory time, space, P, NP, PSpace

4
Theory of Computation What is the computation? What problems are computable by computer in finite steps? What problems are computable by small number of steps?

5
Mathematical Model Real computers are very complicate Develop simple mathematical model to define computation The mathematical models are “equivalent to” real computer under some transformation

6
Important Boundary Computability theory boundary Un-computable problems Computable problems

7
Hierarchy inside computable area

8
Basic Concepts Set: a set is a group of objects represented as a unit. {7,21,57} Element: A object of a set is called an element. Subset: A is a subset of B if every member of A is also a member of B.

9
Some sets Natural numbers set: N={1,2,3,…} Integers set Z={…, -2,-1,0,1,2,…} Empty set: it is a set with no elements

10
Set operations Union: A={2,4,9} B={1,2,5} ={1,2,4,5,9} Intersection: ={2 }

11
Tuple Sequence: a list of objects in some order (7, 21, 57) Tuple: finite sequence (7, 21,57) Cartesian product: A x B is the set of all pairs with first element from A and second element from B A={1,2} B={x,y,z} A x B= {(1,x),(1,y), (1,z), (2,x), (2,y), (2,z) }

12
Power Set Let A be a set. The power set of A is the set of subsets of A. For A={a,b}, its power set P(A) is {, {a}, {b}, {a,b}}

13
Cartesian product

14
Function A function f is mapping from one set D to another set R f: D R for every a in the set D, there is another element b in R such that a is mapped to b by f f(a)=b Domain: D Range: R

15
Function example Function f: {0,1,2,3,4} {0,1,2,3,4} n f(n)

16
Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

17
Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

18
Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

19
Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

20
Equivalence relation A binary relation R is equivalence relation if Reflexive: xRx for every x Symmetric: if xRy, then yRx Transitive: if xRy and yRz, then xRz Example 1: = on {1,2,3}x{1,2,3} Example 2: = on NxN

21
Example for equivalence Relation For two integers x and y, x y if (x-y) is a multiple of 7 In other words, there is another integer z such that (x-y)=7z.

22
Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ } v1 v2 v3 v4

23
Path Graph G=(V,E) A path is a series of edges linked one by one Loop:

24
Tree A graph is connected if every two nodes have a path to connect them A tree is a connected graph without loop

25
Connected Graph Tree Every connected graph can be converted into tree by removing some edges Removing one edge on a loop does not damage the connectivity.

26
A tree is a minimal connected graph Removing any edge on a tree damages the connectivity Proof. Tree T=(V,E). Let (v1, v2) be removed from T. T T’=(V, E-{(v1,v2)}). If T’ is still connected, T has a loop containing v1 and v2. Contradiction!

27
Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ } v1 v2 v3 v4

28
Node degree: The number of edges connecting to the node v1 v2 v3 v4

29
Mathematical approach Definition Mathematical statement Express some object with certain property Theorem A statement proved to be true Proof

30
Mathematical proof Convincing logical argument that a statement is true Usually consists a series logical statements There is a small logical gap between the current logic statement with previous statements.

31
3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

32
Sum of node degrees For every graph, the sum of the degrees of all nodes in G is an even number Sum=2+2+2=6 sum= =14

33
Proof Let v1, v2, …, vn are the n nodes of the graph. deg(vi) is the degree of node vi The sum of node degrees is sum=deg(v1)+deg(v2)+…+deg(vn) For each edge e=(vi, vj), it makes one contribution to both deg(vi) and deg(vj). If there are k edges, the sum is 2k.

34
Regular graph A graph is k-regular if every node has degree equal to k Theorem: For each even number n>2, there exists a 3-regular graph with n nodes.

35
Proof by construction n/2 nodes Let every point at top half connect to a point in the bottom half

36
Proof Let be the n nodes of the graph. Add edges for i=0, 1, …,(n/2)-1, and Add edges for i=0,1, …,(n/2)-1

37
Proof by contradiction Assume the theorem is false Lead to an obviously false consequence Example: Jill just came in from outdoor and is complete dry Try to Prove: No rain Proof: Assume it were raining Jill would be wet. A contradiction!

38
No 3-regular graph with odd nodes Theorem: There is no 3-regular graph with odd number of nodes Proof (by contradiction) If the graph G is a 3-regular graph with 2m+1 nodes, where m is an integer at least 0. The sum of the degrees of nodes is sum=3+3+….+3=3(2m+1)=6m+3=2(3m+1)+1 It is a contradiction to our previous theorem.

39
3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

40
Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

41
Example for induction Theorem: For every natural number n,

42
Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume

43
Proof

44
Computation models Finite automata Pushdown automata Turing machine

45
3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

46
Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

47
Example for induction Theorem: For every natural number n,

48
Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume

49
Proof

50
Computation models Finite automata Pushdown automata Turing machine

51
Finite automata Supermarket entrance Front: A person is coming from the front. Rear: …… Both: front and rear Neither: neither front nor rear closed open

52
Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

53
Automata example q2 q3q1

54
Acceptance Any sequence with at least one 1 and even number of 0s following the last 1

55
Notions for automata start state: q1 accept state: q2. In other words, F={q2} The set The transition function is 0 1 q1 q1 q2 q2 q3 q2 q3 q2 q2 Q={q1,q2,q3}

56
Run the automata Start from the start state Follow the state transition based on the current state and symbol accepts if it enters accept state, rejects otherwise

57
Run the machine at inputs 1 q1 q2 01 q1 q1 q q1 q2 q3 q2 q3 q2 q3 q2 q3 q2 0 q1 q2 10 q1 q2 q q1 q2 q3 q2 q3 q3 q3

58
Language and Machine Let A be a set of strings. Let M be a machine If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

59
Example for language acceptance A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

60
Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

61
Formal definition of computation A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

62
Automata What is the language accepted by the automata?

63
Answer L(M)={w| w ends in a 1}

64
Automata What is the language accepted by the automata?

65
Answer L(M)={w| w is an empty string or ends in a 0}

66
Automata What is the language accepted by the automata?

67
Answer L(M)={w| w starts and ends at the same symbol}

68
Designing Automata Language A consists all {0,1} strings with even number of 1s. Problem: design an automata M with L(M)=A.

69
Automata States:

70
Designing Automata Language A consists all {0,1} strings with 001 as substring. Problem: design an automata M with L(M)=A.

71
Automata States:

72
Regular Language M accepts language A if A={w| M accepts w} A language is regular if some finite automata accepts it. Example: A1={w| w is {0,1} string and ends in a 1} A2={w| w is a {a,b} string that starts and ends with the same symbol}

73
Language and Machine Let A be a set of strings. Let M be a machine If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

74
Example for language acceptance A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

75
Formal definition of computation A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

76
Non-determinism blind monkey

77
Symbol It represents the empty symbol. If used, one state moves to the next without consuming any symbol q1 q2

78
Automata example q2 q3q1 q3 q4

79

80
Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

81
Language recognized Let A be a set of strings that contain either 101 or 11 as a substring. E.G , 10111

82
Automata example q2 q3q1 q3 q4

83
Language recognized Let A be a set of strings containing a 1 in the third position from the end E.G

84
Automata example q2 q3 q1 q3 q2q3

85
Problem: What language does it accept? q2 q1 q5 q2q3 q2

86
Language recognized Accept all strings, where k is a multiple of 2 and 3.

87
Some notations For a set Q, P(Q) is the collection of all subsets of Q Example, Q={q1,q2} P(Q)={empty, {q1}, {q2}, {q1,q2}} For alphabet, write

88
Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

89
Equivalence between NFA and DFA Theorem: Every NFA has an equivalent DFA Proof. Given an NFA We will construct an to accept the same language.

90
Proof Q’=P(Q), which is the set of all subsets of Q The transition function or and for some The start state F’={R in Q’| R has accept state in N}

91
Example Given NFA, convert it into DFA 1 2

92
Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 1 1,2 2

93
Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 2 1,2 1

94
Problem: What language does it accept? q2 q1 q5 q2q3 q2

95
Language Operations Let A and B be two languages Union: Concatenation: Star:

96
Closure under Union Theorem: If A and B are regular languages, then is also regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

97
Construct N N1: N2:

98
Construct N N accepts iff one of N1 and N2 accepts

99
Closure under Catenation Theorem: If A and B are regular languages, then is also a regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

100
Construct N N1: N2:

101
Construct N N2 is linked to the accept state of N1

102
Closure under Concatenation Theorem: If A is regular language, then is also a regular language Proof. Let A be accepted by finite automata N1, Find another finite automata N to accept

103
Construct N N1:

104
Construct N N1:

105
Exercise 1. Given NFA, convert it into DFA 2. Let A be the language recognized by the NFA. Design the automata to recognize A* 1 2

106
Regular Operations R is a regular expression if R is 1)A for some a in 2)The special string 3)The empty set 4),where R1 and R2 are regular expressions 5), where R1 and R2 are regular expressions 6),where R1 is a regular expression

107
Regular Operations Examples 1)0*10* 2) 3) 4) 5) 6)

108
Regular Automata Theorem: Every regular expression is regular language Proof. Let R be a regular expression. Construct a DFA to accept R

109
6 Cases 1)a 2) 3) 4),where R1 and R2 are regular expressions 5), where R1 and R2 are regular expressions 6),where R1 is a regular expression

110
Case 1 a

111
Case 2

112
Case 3

113
Case 4 DFA N1 accepts R1 DFA N2 accepts R2

114
Construct N N accepts

115
Case 5 DFA N1 accepts R1 DFA N2 accepts R2

116
Construct N N accepts

117
Construct N N1 accepts R

118
Construct N N accepts R*

119
Example

120
Regular Operations R is a regular expression if R is 1)A for some a in 2)The special string 3)The empty set 4),where R1 and R2 are regular expressions 5), where R1 and R2 are regular expressions 6),where R1 is a regular expression

121
Regular Operations Examples 1)0*10* 2) 3) 4) 5) 6)

122
Automata Regular expression If a language is regular, then it is described by a regular expression Proof. Let A be recognized by a DFA A, find a regular expression R for A.

123
Some observation States transition 123

124
Some observation States transition 13

125
Some observation States transition 123

126
Some observations States transition 13

127
Some observation States transition 123

128
Some observations States transition 13

129
Some observations States transition 13

130
Proof Idea Convert DFA by shrinking it step by step Replace the symbols by regular expressions on the state transition

131
Proof Idea Given DFA, covert it into regular expression

132
Proof Idea Add start state s and accept state a Convert DFA by removing those old state one by one Replace the symbols by regular expressions on the state transition s a

133
Example States transition 1 2

134
Example States transition 12 s a

135
Example States transition 1 s a

136
Example States transition s a

137
Regular Operations For and for and

138
Remove a state Convert left to right

139
Verify The old and new machines accept the same language

140
Construct Regular expression Add one start state and one accept state:

141
Pumping Lemma Lemma: If A is regular language, there is a number p such that if s is in A and of length at least p, s may be divided into s=xyz, satisfying 1) for each 2) 3)

142
Some notations For a string s, is the length of s (the number of letters) For example, |adb|=3, |a|=1, |afdsaf|=6 For a string s and integer i, is a string to repeats i times For example, if s=dgh then

143
Proof Idea a q1 q9 q13

144
Analysis Run the input string on a automata Input string: s1 s2 s3 s4 s5 s6 … sn State q1 q2 q3 q9 q5 q9… q13 Let p be the number of states If n>p, two of the states must be equal, say q9 Repeat the substring between the two q9s reach the same accept state q13

145
Proof Let p be the number of the states in the Automata M for A Let (n>=p) be a string in A Let be the state transition sequence There are two equal states (j

146
Application of Pumping Lemma is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

147
Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

148
Application of Pumping Lemma The language L={w | w has an equal number of 0s and 1s} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

149
Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. y contains only 0s. By 1) of the pumping, the string is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

150
Application of Pumping Lemma The language {ww | w is a {0,1} string} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

151
Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the first 0s area is more than the number of 0s in the second 0s area. This is a contradiction.

152
Application of Pumping Lemma The language is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

153
Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 1s. By 1) of the pumping, is also in the language. The number of 1s in the string is at least and at most. This is a contradiction since the language L does not contain any string with the number of 1s between and.

154
Problem Prove that is not a regular language.

155
Two ways for language A language can be accepted by a machine A language can be also generated by some rules

156
Context Free Grammar Rules for generating a language: A 0A1 A # Variables: A Terminals: 0,1,# Start variable: A Rules: A 0A1, A #

157
Conversion A XY X 0 Y AZ Z 1 A #

158
Context Free Grammar Rules for generating a language: A 0A1 A B B # Variables: A, B Terminals: 0,1,# Start variable: A Rules: A 0A1, A B, B #

159
Derivation A 0A1 0B1 0#1 A 0A1 00A11 000A111 000B111 000#111

160
Context free grammar A context free grammar is V is a finite set called the variables is a finite set of terminals R is a finite set of rultes is the start variable

161
Language of the grammar If A w is a rule, u and v are strings, then, called uAv yields uwv if for some strings The language of the grammar is

162
Context Free Grammar Rules for generating a language: A 0A1 A B Variables: A, B Terminals: 0,1, Start variable: A The language of this grammar is

163
Convert DFA to grammar Automata accepts all strings that end in 1 A B

164
Grammar A 0A A 1B B 1B B 0A (because B represents the accept state) Start variable A, Variables A,B Terminals: 0,1,

165
DFA to grammar conversion For every transition, add a rule For every accept state, add rule

166
Theorem Every regular language can be generated by a grammar with the rules like A aB, and

167
Chomsky normal form A context-free grammar is in chomsky normal form if every rule is of the form A BC A a where “a” is a terminal and A,B,and C are variables. In addition, we permit if S is the start variable.

168
Removing empty Every context free grammar G has another equivalent context free grammar G’, which has no rule for any non-start variable A

169
Proof Add a new start symbol and rule For any rule that A is not a start symbol, 1) replace every R uAv by R uv, every R uAvAw by R uvw, etc. 2) replace R A by Repeat the steps above until with non-start variable A does not appear

170
Remove Unit Rule Every context free grammar G has another equivalent context free grammar G’, which has no rule

171
Proof For every A B, replace every B u by A u Repeat the step above until there is no A B

172
Chomsky Normal Form Every context free grammar G has an equivalent chomsky normal form

173
Proof Remove the for non-start symbol A Remove the unit rule A B For each at k>2, replace it by rules

174
Proof For each at k>1, replace each terminal by the new symbol and add new rule

175
Example A 0A1 Conversion:

176
Remove the empty

177

178

179

180
Arithmetic expression Grammar

181
Example Generate

182
Generate the expression Steps

183
Arithmetic expression Grammar Start variable: Variable: Terminals: a,+,x,(,)

184
Example Generate

185
Example Generate

186
Leftmost derivation A derivation of a string w is leftmost derivation if the left most variable is replaced at every step x + x a+ x a+a x a+a x a

187
Ambiguity A string w is derived ambiguously in context-free grammar G if it has more than one leftmost derivation Grammar G is ambiguous if it generates some string ambiguously Ambiguity gives different interpretations by computer program

188
Example Design the context free grammar for

189
Grammar Partition the problem into two parts. One part is for The second part is Let and S1 is used to get via S2 is used to get via

190
Pushdown Automata State control aabbbaabbb aaaabbbbbabaaabb

191
Example The grammar generates the language It can be accepted by pushdown automata

192
Remove input 0, push 0 into stack State control

193
Remove input 0, push 0 into stack State control

194
Match stack 0 with input 1. Then remove both of them State control

195
Match stack 0 with input 1. Then remove both of them, accept State control

196
Pushdown Automata Let M be 1. is the set of states 2. is the input alphabet 3. is the stack alphabet 4. is the state transition function 5. is the start state, 6. is the set of accept states q2 q3 q4 q1

197
States Explanation Q1 is the start state Q2 is used to push 0 symbol Q3 is used to match 0,1 pairs between stack and input tape Q4 is the accept state

198
Transition a,b c: Used when the current input symbol a and the stack top symbol b, Remove the top symbol on the stack and push c on it The input is moved to the next after a

199
Difference between Pushdown and finite state automata Pushdown automata has unlimited memory, which is last in, first out. Nondeterministic finite state automata is a special case of pushdown automata that has no memory

200
Pushdown Automata 6-tuple 1.Q is the finite set of states 2. is the input alphabet (finite) 3. is the stack alphabet (finite) is the start state, 6. is the set of accept states

201
A computation of pushdown automata Input string: State transition sequence: For i=0,…,m-1, where and Final state accepts

202
Theorem Every context free language can be accepted by a non- deterministic pushdown automata

203
Example The grammar generates the language It generates the string 0011

204
Pushdown Automata State control

205
Replace S by 0S1 at stack State control

206
Match the stack top symbol with the input symbol, remove both if matched State control

207
Replace S by 0S1 State control

208
Remove after matching State control

209
Replace S by empty State control

210
Remove after matching State control

211
Remove after matching, Accept State control

212
Proof Idea Push $ and start symbol to the stack in the beginning. Repeat the three steps below Replace the top variable A on the stack with a the right side of a rule Match the top terminal on the stack with the input symbol, reject if not matched When stack has $ on the top and all input has been read, accepts

213
How to replace the variable If the current stack has the top element s, replace it with the right of the rule State q a…………….. s....s....

214
New States The states are new states, specially added for the rule State moves from q to r after the variable is replaced on the top of the stack

215
Pumping Lemma Lemma: If A is context-free language, there is a number p such that if s is in A and of length at least p, s may be divided into s=uvxyz, satisfying 1) for each 2) 3)

216
Proof Idea Two variables are the same on a path

217
Proof Idea A path from root to a leaf has all variables except the last one, which is a terminal When the path is too long, same variable has to happen twice. Repeat the part of the two equal variables area.

218
Analysis Let V be the set of variables in a grammar. |V| is the number of variables in V If a path has at least |V|+1 variables, two of them will be equal. A path from root to a leaf has least |V|+1 variables if its length is at least |V|+2 A tree of depth |V|+2 has leaves, where b is the maximal length of right side among all rules.

219
Proof Let If s has length, the parse tree T has height It has a path has at least |V|+1 variables. Two of the variables on the path are equal (=R). We have So,

220
Proof The tree T is the parsing tree for s and has least size It is impossible that is empty otherwise, T is not least. Select the bottom |V|+2 symbols on the longest path of T, this makes the

221
Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

222
Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. v and y are in the area of the same symbol, say a. Contradiction for having more a than b. Case 2. v or y contains more than one symbol, contradiction for the incorrect order of symbols in

223
Regular Pumping The language is not regular Proof. Assume it is a regular language. It can be accepted by automata M. Let p be the length of the pumping lemma. Consider s can be expressed s=xyz, where The string xz has the number of 0s no more than 1s (Pump it down to get the contradiction)

224
Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

225
Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy have intersection with the a area. Contradiction for having more a or b than c (pumping up). Case 2. vy does not contain symbol a, contradiction for having more a than b or c (pumping down).

226
Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

227
Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy is in the first half. Contradiction for moving 1 to the right area(pumping up). Case 2. vy is in the second half. Contradiction for moving 0 to the left area(pumping up).

228
Proof The language is not context free Proof. Case 3. vy crosses the middle line. Contradiction for reducing 0 or 1 in the middle area (pumping down). The left half and right half have different number of 1s or 0s.

229
Problem Write a context-free grammar for the following language

230
Problem Using pumping lemma to disprove the following language is context-free:

231
Algorithm An algorithm is a collection of simple instructions for carrying out some task

232
Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

233
Unlimited Register Machine Statement1: x x+1 Statement2: x y Statement3: x 0 Statement4: if (x==y) jump m Each register can save an unbounded integer

234
Turing Machine Proposed in 1936 An accurate model for the general purpose computer.

235
Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab

236
Language ww Design Turing machine for L={w#w|w is in {0,1}*} For example: 011#011 is in L 10011#10011 is in L

237
Movement on the tape Move the head back and forth to match all pairs

238
Movement on the tape Move the head

239
Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

240
State transition function For

241
State transition function For

242
Configuration Current state: q7 Current head position on the tape: 4 th cell Current tape content: abab q7 abab

243
Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

244
Configuration Transition For

245
Configuration Transition For

246
Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

247
Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration

248
Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

249
a q1 q2 q3 q4 q6 q8 q10 q12 q13 q14 q5 q7 q9 q11 q_accept

250
Multi-tape Turing Machine Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control ba aaa 01010

251
Multi-tape Turing Machine Transition for multi-tape Turing machine

252
Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 0101#aaa#b#a#

253
Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

254
Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 ab1aa0a0

255
Simulate 3 tapes q7 ab1aa0a0

256
Simulate 3 tapes q7 ab1aa0a0

257
K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j- 1)k+2i-1 and 2(j-1)k+2i

258
Midterm October 18, 2010 Class Time Close book Chapter 0-Chapter 3

259
Problem 5 Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains an equal number of 0s and 1s}.

260
Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab

261
Language L Design Turing machine for L={w|w is in {0,1}* and the same number 1s as the same number of 0s} For example: is in L is not in L

262
Movement on the tape Move the head back and forth to pair up 0 and 1.

263
Movement on the tape Move the head

264
Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control

265
Language L Design Turing machine for L={w|w is in {0,1}* and the same number 0s is two times the number of 1s} For example: is in L is not in L

266
Movement on the tape Move the head back and forth to pair up 0 and 1.

267
Movement on the tape Move the head

268
Problem 6 Problem 6 (10) If A and B are languages, define and. Prove that if A and B are regular languages, then is a context free language.

269
Problem 6 DFA N1 accepts A DFA N2 accepts B

270
Construct N N accepts

271
Problem 7 Problem 7 (10) If A and B are languages, define and and |x|=|y|}. Prove that if A and B are regular languages, then is a context free language. You will get 10 more points for midterm.

272
Deterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

273
Nondeterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

274
Nondeterministic Turing Machine One configuration can have multiple choice for entering the next configuration because of

275
Non-determinism blind monkey

276
Automata example q2 q3q1 q3 q4

277

278
Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

279
Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

280
Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

281
Simulate NTM with DTM Each dot is one configuration Find an accepting path

282
Simulate NTM Think about the problem from the Java programming point of view Convert the strategy into deterministic Turing machine.

283
Simulate NTM Key points: search accepting path with width first until it finds the first one. Search from the left to right Search from small level 0, to level 1, to level 2,…

284
Simulate NTM 1: first branch, 2: second branch, etc 121: take the first branch at level 2, then take the second at level 3, then take the first branch at level 4. Control 121 aaa 01010

285
Turing machine and Computer The computational power of Turing machine is equivalent to regular computer with unlimited memory

286
Successor Funtion f(x)=x+1

287
Successor Funtion f(x)=x+1

288
Successor Turing machine is able to simulate the operation x x

289
Comparison Turing machine is able to check if x=y

290
Transfer Turing machine is able to simulate y x

291
Transfer Turing machine is able to simulate x y

292
Set zero Turing machine is able to simulate x

293
Program statements Statement1: x x+1 Statement2: x y Statement3: x 0 Statement4: if (x==y) jump m

294
Compute Algorithm: Let x be added by 1 y times Program for x+y: 1: z 0 2: If (z==y) jump 6 3: x x+1 4: z z+1 5: If (z==z) jump 2 6:

295
Jump TM program state Program q1 1: I1 q2 2: I2 q3 3: I3 …. …… qk k: jump 3 The jump can be achieved via state transition qk q3

296
Compute Algorithm: Let x be added by original x y times Program for x*y: 1: z 0 2: p 0 3: If (z==y) jump 7 4: p p+x 5: z z+1 6: If (z==z) jump 3 7:

297
Compute Algorithm: Let x be multiplied by x y times Program for : 1: z 0 2: p 1 (via p 0 and p p+1) 3: If (z==y) jump 7 4: p p*x 5: z z+1 6: If (z==z) jump 3

298
Algorithm An algorithm is a collection of simple instructions for carrying out some task

299
Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

300
Unlimited Register Machine Statement1: x x+1 Statement2: x y Statement3: x 0 Statement4: if (x==y) jump m Each register can save an unbounded integer

301
Other computation model Lambda calculus (Church) Recursion (Godel and Kleene) Post and Markov’s model

302
Evidences Many approaches led to the same algorithmic computable class No one has found an algorithm that is accepted in informal sense, but it can not be implemented in Turing model

303
Hilbert 10 th Problem Find an algorithm to decide if a polynomial has integer root Input: a polynomial e.x. (it has root x=5,y=3,z=0) Output: yes or no

304
Midterm Problem 1 (20) Give the state diagram of a DFA recognizing {w|w is 0,1-string with at least five 0s}. Problem 2 (20) Prove that is not regular language with the pumping lemma.

305
Midterm Problem 3 (20) Prove the following facts: a) If A and B are context free languages, then so is their union. b) If A and B are context free languages, then so is their concatenation.

306
Midterm Problem 4 (20) a) Design a context-free grammar to recognize the language. b) Prove that is not a context- free language by using pumping lemma.

307
Midterm Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains more 0s than 1s}.

308
Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

309
DFA ={ | B is a DFA that accepts input string w} Theorem: is a decidable language.

310
Proof The input is, where B is a DFA, and w is a string. Simulate B on the input w: Start from the state of B and leftmost symbol w. Follow to transit the state and move the input symbols one by one Accepts if ends at an accept state; rejects otherwise.

311
NFA ={ | B is a NFA that accepts input string w} Theorem: is a decidable language.

312
Proof The input is, where B is a NFA and w is a string. Convert B into a DFA C via our previous algorithm Use our last TM to decide if C accepts w Accepts if it accepts; rejects otherwise.

313
={ | L(A) is not empty} Theorem: is a decidable language.

314
Proof The input is, where A is a DFA. Mark the start state Repeat Mark a new state that has a transition to it from a marked one. Until no new state can be added Accept if an accept state is marked; otherwise reject

315
={ | A and B are DFA and L(A)=L(B)} Theorem: is a decidable language.

316
Proof The input is, where A and B are DFA Check if is empty

317
Problem Define L={ : L(A) is the union of L(B) and L(C)}, where A, B, and C are DFAs. Prove that L is decidable.

318
One- one and onto Let A and B be two sets. For function f: A B, if whenever then f is called one-one. For function f: A B, say f is onto if f hits every element of B(In other words, for very b in B, there is a in A such that b=f(a))

319
Correspondence Let A and B be two sets. A and B are of the same size if there is a one-one and onto function f: A B For function f: A B, if it is both one-one and onto, then f is called correspondence.

320
Examples {1,2,3,…} and {2,4,6,…} are of the same size via f(x)=2x. (0,1) and are of the same size

321
Countable A set is countable if it is finite or it has the same size as N={1,2,3,…} Theorem: The positive rational numbers set is coutable

322
Proof Every positive rational number is in the table below

323
Proof List all of them by and avoid repetition

324 Examples For two positive rational numbers, ( p and q have no common divisor>1, and p’ and q’ have no common divisor >1), The number is listed before if p+q

325
(0,1) is not countable Theorem: (0,1) is not countable

326
Proof. Proof by contradiction. Assume N={1,2,3,…} and (0,1) have the same size. There is one-one and onto map f: N (0,1)

327
Proof. Select such that and Since

328
Infinite binary strings set is not countable Let be the set of all infinite binary strings Theorem: is not countable

329
Proof. Proof by contradiction. Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

330
Proof. Select such that and Since

331
The set of finite binary strings is countable Correspondence N= {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,…}

332
Language of Binary Strings Every set A of binary strings is countable Proof. Let A be a set of binary strings. Its elements can be listed according to their orders in A={ s1, s2, s3, s4, ….} It is easy to see the correspondence between N and A

333
An infinite binary string uniquely determines a language N= … L= { 11, 011,….} The language with positive even number of 1s strings

334
Correspondence For each infinite binary string B, it uniquely determines a binary language L(B). If B1 and B2 are different binary strings, then L(B1) and L(B2) are different language Each binary language uniquely determines an infinite binary string.

335
Correspondence There is a correspondence between the set of all infinite binary strings and the set of all binary languages. The set of all binary languages is not countable

336
Turing machine to binary string Each Turing machine can be encoded into a binary string. The set of all Turing machines can be encoded into a set of binary strings. The set of Turing machines is countable. The set of binary languages recognized by TM is also countable

337
Un-computable language by TM There is a binary language that is not Turing recognizable Proof. The set of binary languages is not countable, but the set of language recognized by Turing machine is countable

338
Undecidable Problem ={ | M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

339
Every entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

340
Proof Assume that is decidable There is a TM H such that H( ) accepts if M accepts w H( ) rejects if M rejects w. Consider a TM D D( ) accepts if M rejects D( ) rejects if M accepts

341
Proof Since D( ) accepts if M rejects D( ) rejects if M accepts We have D( ) accepts if D rejects D( ) rejects if D accepts A contradiction

342
Every entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

343
Every diagonal entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

344
Diagonal method reject reject accept accept accept reject ……

345
Call H in R If TM H exists, the TM R also exits via using H (Software R uses an existing software H)

346
Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

347
Problem Is there any one-one and onto map from the set of integers in [1,10] and the set of odd integers in [1,10]? Why? Prove that there is a one-one and onto map from the set of all integers and the set of all odd integers.

348
Reduction Solution for Problem 1 Solution for Problem 2 Help

349
Example: Map and Direction MapDirection?

350
Reduction Solution for Problem 1 Problem 2 Question Answer

351
Reduction Software for Problem 1 Software for Problem 2: call Return

352
Reduction & Undecidability It is known that P2 is undecidable We want to prove P1 is undecidable Proof by contradiction Assume P1 is decidable. There is a software solving P1 Design a software for P2 by calling the software for P1 Contradiction!!!!

353
Halting Problem P1: ={ | M is a Turing machine and M halts on input w} P2:. ={ | M is a Turing machine and M accepts input w} Undecidable

354
P2 Input If M does not stop on w, reject it If M stops, simulate M. If M acceps, accpets Otherwise

355
Halting Problem ={ | M is a Turing machine and M halts on input w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

356
Undecidable Problem ={ | M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

357
Proof Assume the Turing machine R decides We can use R to decides For input Run R on if R rejects, “reject” if R accepts, simulate M until it stops if M accepts, “accept” else “reject”

358
Empty Problem ={ | M is a Turing machine and } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

359
Proof Assume the Turing machine R decides For input, design another TM If, reject simulate M on input w, accepts if M accepts w

360
Proof If M accepts w, then w belongs to L( ) Otherwise, L( ) is empty

361
Proof Use R to decide Input Make from Run R on the input If R rejects (it means L( ) is not empty), accepts I R accepts (it means L( ) is empty), rejects

362
Problem ={ | M is a Turing machine and L(M) is regular} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

363
Proof Assume the Turing machine R decides For input, design another TM (x) If x has format, accept simulate M on input w, accepts if M accepts x

364
Proof If M accepts w, then Otherwise,

365
Proof Use R to decide Input Make from Run R on the input If R rejects (it means L( ) is not regular), accepts I R accepts (it means L( ) is regular), rejects

366
Turing machine equivalence Problem ={ | M1 and M2 are Turing machines and L(M1)=L(M2) } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

367
Proof For input for TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

368
Mapping Reducibility Convert instances of problem to instances of problem B A B

369
Computable function Assume that is a set of finite number of symbols is the set of all finite strings with symbols from The function is computable if some Turing machine M, on every input w, halts with just f(w) on the tape. Example: +,x,/ are all computable functions

370
Mapping Reducibility Language A is mapping reducible to language B, if there is a computable function such that for every w

371
Example A={1,3,5,….} B={0,2,4,…} f(x)=x+1 via the funtion f

372
Theorem If, and B is decidable, then A is also decidable A B

373
Proof Let f be the reduction from A to B since Let M be the decider for B. Decider N: Input w, Compute f(w) Run M on f(w) and accept iff M accepts

374
Corollary If, and A is undecidable, then B is also undecidable A B

375
Proof Proof by contradiction. If B is decidable, then so is A

376
Halting Problem P1: ={ | M is a Turing machine and M halts on input w} P2:. ={ | M is a Turing machine and M accepts input w} Undecidable

377
A reduction from to B: ={ | M is a Turing machine and M halts on input w} A:. ={ | M is a Turing machine and M accepts input w} Undecidable

378
Reduction On input Construct Turing machine M’ Run M on w If M accepts, accept If M rejects, enter an infinite loop Output

379
Empty Problem ={ | M is a Turing machine and } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

380
Reduction from to For, design another TM M’ Input x If x is not equal to w, reject else simulate M on input w, accepts if M accepts w

381
Reduction from to is in L(M’) is not empty is not in L(M’) is empty

382
Reduction from to is in M’ is in is not in M’ is not in

383
Problem Let ={M| M is a Turing machine and M prints 1 for some input w} Show that L is undecidable.

384
Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab

385
Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

386
Configuration Current state: q7 Current head position on the tape: 4 th cell Current tape content: abab q7 abab

387
Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

388
Configuration Transition For

389
Configuration Transition For

390
Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

391
Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration

392
Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

393
Turing Recognizable Turing machine M recognizes language L

394
Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

395
Turing Decidable Turing machine M decides language L

396
Observation If L is Turing decidable, then L is Turing recognizable

397
A Turing recognizable problem ={ | M is a Turing machine and M accepts w} Turing machine R recognizable For input Simulate M on w R accepts if M accepts w.

398
Theorem Theorem: If L is a language in {0,1}*, there is a Turing machine to print out all elements in L

399
Proof Let M recognize L Step i(i=1,2,3…) Simulate M each of the first i strings i steps If a string is accepted, print it out.

400
Theorem If, and B is Turing recognizable, so is A A B

401
Proof Let f be the reduction from A to B since Let M recognize B. TM N: Input w, Compute f(w) Run M on f(w) and accept if M accepts

402
Theorem If, and A is not Turing recognizable, B is not Turing recognizable A B

403
Complement set Set and its complement

404
Theorem If A and are Turing recognizable, then A is decidable

405
Proof Let Turing machine M1 recognize A Let Turing machine M2 recognize For input x, Run M1 and M2 on x in parallel If M1 accepts, accept If M2 accepts, reject.

406
={ | A and B are DFA and L(A)=L(B)}.

407
={ | A and B are DFA and L(A)=L(B)} Theorem: is not Turing decidable language.

408
. Theorem: Neither nor its complement is Turing recognizable

409
Proof We first prove For input for TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

410
Proof We prove For input for TM M1: accepts any input TM M2: accepts any input if M accepts w. It is easy to see L(M1)=L(M2) iff M accepts w So,

411
Problem Show that the language F={M| L(M) contains infinite elements} is not Turing recognizable}.

412
Problem 4.2 The equivalence of a DFA and a regular expression is decidable

413
Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

414
Problem 4.7 The set of all infinite binary strings is not countable

415
Problem 4.8 N={1,2,3,…} NxNxN is countable

416
Problem 5.9 All Turing recognizable problems are mapping reducible to

417
Problem 5.12 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable.

418
Problem 5.13 Test if a Turing machine has useless state, which never enters.

419
Language Design Turing machine to recognize

420
Complexity Let M be a deterministic Turing machine. The running time of M is the function f:N N such that f(n) is the maximum number of steps that M uses on input of length n.

421
Big O-notation Let f and g be functions Say f(n)=O(g(n)) if integers c and exist so that for all

422
Examples Let

423
Small o-notation Let f and g be functions Say f(n)=o(g(n)) if

424
One tape Turing machine time One tape Turing machine can recognize In steps

425
Two tapes Turing machine time Two tapes Turing machine can recognize In O(n) steps

426
Multi-tape Turing Machine Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control ba aaa 01010

427
Multi-tape Turing Machine Transition for multi-tape Turing machine

428
Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 0101#aaa#b#a#

429
Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

430
Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 ab1aa0a0

431
Simulate 3 tapes q7 ab1aa0a0

432
Simulate 3 tapes q7 ab1aa0a0

433
K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j- 1)k+2i-1 and 2(j-1)k+2i

434
Simulate multi-tape by one tape One tape Turing machine can simulate t(n) time multi-tape Turing machine in time

435
Time Complexity Class Let. is the class of languages decided by O(t(n) time Turing machines. In other words is a language decided by O(t(n)) time Turing machine}

436
One tape Turing machine time One tape Turing machine can decide In steps

437
Complexity Class P P is the class of languages decided by single tape Turing machine in polynomial time

438
Directed Path Problem G is a directed graph that has a directed path from s to t} s a b c d t

439
Algorithm Input mark “s” Repeat For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

440
Greatest common divisor Divisor: For two integers b and c, if b=c*z for some integer z, c is a divisor of b. Greatest common divisor: Given two integers a and b, gcd(a,b) is the greatest positive integer c such that c is the divisor for both a and b. Examples: gcd(10,4)=2, gcd(16,100)=4 Problem: How to find gcd(a,b)?

441
Relatively Prime Problem Two integers x and y are relatively prime if gcd(x,y)=1. RELPRIME={ | x and y are relatively prime} Theorem:

442
Euclid algorithm For two integer b and a a=q*b+c with gcd(a,b)=gcd(b,c) a and b are relatively prime iff b and c are relatively prime

443
Euclid algorithm Assume a1 and a2 are two positive integers If, then and Therefore, always true

444
Algorithm for gcd(x,y) Input Repeat r x mod y x y y r until y=0 output x

445
Modular Assume a and b are two positive integers This is a recursive equation since the second item goes down

446
Example Find gcd(1970,1066)

447
Speed of Euclid algorithm Assume a1 and a2 are two positive integers If, we have In another words,

448
Euclid algorithm Assume a1 and a2 are two positive integers

449
Time For input The total time of steps is O(log a+log b)

450
Non-determinism blind monkey

451
Automata example q2 q3q1 q3 q4

452

453
Deterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

454
Nondeterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

455
Nondeterministic Turing Machine One configuration can have multiple choice for entering the next configuration because of

456
Non-determinism blind monkey

457
Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

458
Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

459
Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

460
Simulate NTM with DTM Each dot is one configuration Find an accepting path

461
Class NP NP is the class of languages that can be recognized by nondeterministic Turing machine in polynomial time

462
Hamiltonian Path Hamiltonian path goes through each node exactly once HAMPATH={ | G is a directed graph with a Hamiltonian path from s to t}

463
Composite A natural number is composite if it is the product of two integers >1 10=2*5 18=2*9 Composite={x| x=pq, for integers p,q>1}

464
Verifier A verifier for a language L is an algorithm V, L={w| V accepts for some string c} For the verifier V for L, c is a certificate of w if V accepts If the verifier V for the language L runs in polynomial time, V is the polynomial time verifier for L.

465
Verifier for Hamiltonian Path For, a certificate is a list of nodes of G: Verifier: check if m is the number of nodes of G check if and check if each is a directed edge of G for i=1,…,m-1 If all pass, accept. Otherwise, reject.

466
Verifier for Composite For integer x, a certificate is two integers p,q: Verifier: check if p>1 and q>1 check if x=pq If all pass, accept. Otherwise, reject.

467
Class NP NP is the class of languages that have polynomial time verifiers. Examples: COMPOSITE is in NP HAMPATH is in NP

468
Theorem A language has polynomial verifier iff it can be recognized by polynomial time nondeterministic Turing machine Proof: Assume L has verifier V, … Assume L has NTM M,…

469
Proof Assume that L has polynomial time verifier V, which runs in time, where k is a constant NTM M Nondeterministically select string c of length Run V on If V accepts, accept; Otherwise, reject.

470
Proof Assume that L is recognized by polynomial time NTM M, which runs in time, where k is a constant Verifier Let c determine a computation path of M Simulate M on the path c If M accepts, accept. Otherwise, reject.

471
Clique Problem Given undirected graph G, a clique is a set of nodes of G such that every two nodes are connected by an edge. A k-clique is a clique with k nodes

472
Clique Problem CLIQUE={ | G iss an undirected graph with k-clique} CLIQUE is in NP.

473
Subset Sum Problem SUBSET-SUM={ | S= and for some, we have

474
Polynomial Time Computable A function is a polynomial time computable function if some polynomial time Turing machine M exists that outputs on the tape for input w.

475
Polynomial Time Reduction Assume that A and B are two languages on A is polynomial time mapping reducible to A if a polynomial time computable function exists such that

476
Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

477
SAT A boolean formula is satisfiable if there exists assignments to its variables to make the formula true SAT={ | is satisfiable boolean formula}

478
3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

479
3SAT to CLIQUE Example:

480
NP-completeness A language B is NP-complete if 1.B is in NP, and 2.Every A in NP is polynomial time reducible to B Theorem. If B is NP-complete and B is in P, then P=NP.

481
Cook-Leving Theorem Theorem: SAT is NP-complete Proof. 1. SAT is in NP. 2. For every problem A in NP,

482
Proof 1. The start configuration is legal 2. The final state is accept. 3. The movement is legal. 4. Each cell takes one legal symbol.

483
Proof 1. 1 if The cell[i,j] holds symbol s; 0 otherwise 2. Time bound for the NTM M with constant k. 3. The movement is legal. 4. NTM M for accepting A.

484
Nondeterministic Turing Machine 1.Q is the finite set of states 2. is the tape alphabet is the start state, 5. is the accept state.

485
Configuration Transition For

486
Configuration Transition For

487
Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

488
Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration

489
Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

490
Proof Each cell has only one symbol 1.The symbol is selected from C: 2.Only one symbol is selected: 3.It is true for all cell at all configuration:

491
Proof The start configuration is

492
Proof Accept computation has reached. It makes sure the accept state will appear among the configuration transitions.

493
Proof Characterize the legal move The whole move is legal if all windows are legal. Characterize one window is legal

494
Proof The state transition

495
Logic Demorgan Law:

496
Truth table for y1 y2 x

497
Convert to CNF Conversion:

498
Convert to CNF Conversion:

499
Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

500
Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory

501
Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular

502
Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

503
Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

504
Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

505
Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier

506
Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete.

507
Grade A:… B:… C: Miss exam or homework

508
SAT’ A conjunctive normal form is a conjunction of some clauses SAT’={ | is satisfiable conjunctive normal form}

509
Cook-Leving Theorem’ Theorem: SAT’ is NP-complete Proof. Same as that for SAT is NP-complete

510
3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

511
3SAT is NP-complete Theorem: There is polynomial time reduction from SAT’ to 3SAT.

512
3SAT is NP-complete is satisfiable if and only if the following is satisfiable

513
3SAT is NP-complete is satisfiable if and only if the following is satisfiable

514
3SAT is NP-complete Convert every clause into 3cnf:

515
3SAT is NP-complete Conjunctive normal form Each clause is convert into is satisfiable if and only if the following is satisfiable

516
Problem Convert the formula F into 3SAT formula F’ such that F is satisfiable iff and F’ is satisfiable.

517
Approximation Algorithms

518
Outline and Reading Approximation Algorithms for NP-Complete Problems –Approximation ratios –Polynomial-Time Approximation Schemes –2-Approximation for Vertex Cover –Approximate Scheme for Subset Sum –2-Approximation for TSP special case –Log n-Approximation for Set Cover

519
Approximation Ratios Optimization Problems –We have some problem instance x that has many feasible “ solutions ”. –We are trying to minimize (or maximize) some cost function c(S) for a “ solution ” S to x. For example, Finding a minimum spanning tree of a graph Finding a smallest vertex cover of a graph Finding a smallest traveling salesperson tour in a graph

520
Approximation Ratios An approximation produces a solution T –T is a k-approximation to the optimal solution OPT if c(T)/c(OPT) < k (assuming a min. prob.; a maximization approximation would be the reverse)

521
Polynomial-Time Approximation Schemes A problem L has a polynomial-time approximation scheme (PTAS) if it has a polynomial-time (1+ )-approximation algorithm, for any fixed >0 (this value can appear in the running time). Subset Sum has a PTAS.

522
Vertex Cover A vertex cover of graph G=(V,E) is a subset W of V, such that, for every (a,b) in E, a is in W or b is in W. OPT-VERTEX-COVER: Given an graph G, find a vertex cover of G with smallest size. OPT-VERTEX-COVER is NP-hard.

523
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

524

525
A 2-Approximation for Vertex Cover Every chosen edge e has both ends in C But e must be covered by an optimal cover; hence, one end of e must be in OPT Thus, there is at most twice as many vertices in C as in OPT. That is, C is a 2-approx. of OPT Running time: O(m) Algorithm VertexCoverApprox(G) Input graph G Output a vertex cover C for G C empty set H G while H has edges e H.removeEdge(H.anEdge()) v H.origin(e) w H.destination(e) C.add(v) C.add(w) for each f incident to v or w H.removeEdge(f) return C

526
Problem 1 Show that T={(i,j)| both I and j are positive rational numbers} is countable.

527
Problem 2. Find a correspondence between (0,1) and [0,1]

528
Solution Let for n=1, 2, … Define the correspondence f(x) below:

529
Problem 3 3. Let A={ | R and S are regular expressions and L(R ) is a subset of L( S)}. Show that A is decidable.

530
Solution For two regular expression R and S, L( R) is a subset of L(S) if and only if the L(R ) has no intersection with. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Check if L(M4) is empty R S

531
Problem 4 I={x| x is an irrational number in (0,1)}. Show that I is not countable.

532
Homework 2009

533
Problem 4.2 The equivalence of a DFA and a regular expression is decidable

534
Solution Proof. Given a regular language L and DFA M. We can construct a DFA N to accept L. The problem is converted into checking if two DFAs are equivalent. By Theorem 4.5, there is an algorithm to decide if L(M)=L(N).

535
Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

536
Solution Assume M is a DFA with n states. L(M) is infinite iff it has path from the start state to accept with a loop on it, and the loop length is no more than n. L(M) is infinite iff M accepts a string of length between n and 2n. For every string x with length between n and 2n Run M on x If M accepts x, then output “L(M) is infinite” else output “no”

537
Problem 4.7 The set of all infinite binary strings is not countable

538
Proof. Proof by contradiction. Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

539
Proof. Select such that and Since

540
Problem 1 N={1,2,3,…} NxNxN is countable

541
Proof List all of them by

542
Proof Use the similar method like the last slide to prove the NxN is countable Let f: NxN N be one-one and onto We have g: NxNxN N is one-one and onto, where g(i,j,k)=f(f(i,j),k)

543
Problem 2. Find a correspondence between (0,1) and [0,1]

544
Solution Let for n=1, 2, … Define the correspondence f(x) below:

545
Problem 3 3. Let A={ | R and S are regular expression and L(R ) = L( S)}. Show that A is decidable.

546
Solution For two regular expression R and S, L( R)=L(S) if and only if their symmetric difference is empty. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Construct automata M5 to accept Check if L(M5) is empty

547
Problem 4 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable.

548
Proof We are going to design a reduction from to S. For Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts is in iff N accepts both 01 and 10.

549
Problem 5 5. Let S={ | M is a TM and L(M) = { }}. Show that neither S nor its complement is Turing-recognizable.

550
Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal then reject If x= then accepts if M(w) accepts.

551
Solution For an input (M, w), derive M1(x, M1).

552
Solution We show that can be reducible to. Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x = then accept If x is not equal then accept if M(w) accepts.

553
Solution For an input (M, w), derive M2(x, M2).

554
Problem 6 Show that A is a Turing recognizable if and only if

555
Proof Part a) Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =. It is easy to see that

556
Proof Part b) Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize. We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

557
Proof We are going to design a reduction from to it For Design the TM N as follows Input x if x=0, enters all states except if x=1 run M on w, if M accepts w, enter is in iff N enters all states.

558
Problem 4 c) The language is not a context free language Proof: Assume it is a context free language. Let p be the number from the pumping lemma Consider the string

559
Final Exam December 13, 5:45-7:30pm, Monday

560
Homework 4, 20

561
Problem 1 1.Let T1={ | M is a Turing machines and accepts infinite number of 0,1-strings of finite length}. Prove that T1 is undecidable.

562
Proof We are going to design a reduction from to T1. For input Turing machine for Design the TM N as follows Input x run M on w, if M accepts w, acceptx is in iff N acccepts each input x.

563
Problem 2 Let T2={ | M is a Turing machines and accepts five of 0,1-strings of finite length}. Prove that T2 is undecidable.

564
Proof We are going to design a reduction from to T2. For Design the TM N as follows Input x if x is not 0,1, 00, 01 or 10, rejects. run M on w if M accepts w, accept x is in iff N accepts 0,1,00,01, and 10. Otherwise, N accepts no string

565
Problem 3 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable.

566
Proof We are going to design a reduction from to S. For Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts is in iff N accepts both 01 and 10.

567
Problem 4 Show that A is a Turing recognizable if and only if

568
Proof Part a) Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =. It is easy to see that

569
Proof Part b) Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize. We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

570
Problem 5 5. Let S={ | M is a TM and L(M) = { }}. Show that neither S nor its complement is Turing-recognizable.

571
Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal then reject If x= then accepts if M(w) accepts.

572
Solution For an input (M, w), derive M1(x, M1).

573
Solution We show that can be reducible to. Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x = then accept If x is not equal then accept if M(w) accepts.

574
Solution For an input (M, w), derive M2(x, M2).

575
Homework 5, 20

576
Problem 1 Call graphs G and H isomorphic if the nodes of G may be reordered so that it is identical to H. Let ISO={ | G and H are isomorphic graphs}. Show that ISO is in NP.

577
Solution for Problem 1 We design a polynomial time verification algorithm. A mapping f from the vertices of G to those of H is a witness. Check if 1) f is one-one, 2) f is onto, 3) (v,u) is an edge of G iff (f(u),f(v)) is an edge of H. It is easy to see that the verification takes polynomial time.

578
Problem 2 Let MAX-CIQUE={ | the largest clique of G has k vertices}. Whether MAX-CLIQUE is in NP is unknown. Show that if P=NP, then MAX-CLIQUE is in P, and a polynomial time algorithm exists that, for a graph G, finds one of its largest cliques.

579
Solution of Problem 2 Fist step is to find the largest k with (G,k) is in Clique. Try k from 1,2,… Check if (G,k) is in Clique Select the largest k. Assume the largest k for (G,k) in Clique is obtained.

580
Solution of Problem 2 Assume the largest k for (G,k) in Clique is obtained. Formulate the problem: (H, k,G) Determine if there is a clique of size k in G and contains all veritces in H. The problem is in NP. Extend H one by one until its size reaches k.

581
Problem 3 Let G represent an undirected graph and let SPATH={ | G contains a simple path of length at most k from a to b} and LPATH={

582
Solution Problem 3 Show that SPATH is in P.

583
Directed Path Problem G is a directed graph that has a directed path from s to t} s a b c d t

584
Algorithm Input mark “s” Repeat For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

585
Solution Problem 3 Show that LPATH is NP-complete. Hamiltonian path problem is NP-complete. There is an easy reduction from Hamiltonian path to it.

586
Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory

587
Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular

588
Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

589
Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

590
Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

591
Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier

592
Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete.

593
Grade A:… B:… C: Miss exam or homework

594
Complexity ©D.Moshkovits 593 Space Complexity

595 Complexity ©D.Moshkovits 594 Motivation Complexity classes c