Presentation is loading. Please wait.

Presentation is loading. Please wait.

Theoretical Foundation of Computation Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm.

Similar presentations


Presentation on theme: "Theoretical Foundation of Computation Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm."— Presentation transcript:

1

2 Theoretical Foundation of Computation Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm Tuesday

3 Contents Computational models and languages finite automata, push automata regular languages, context free languages Computability theory decidable problem, un-decidable problem Complexity theory time, space, P, NP, PSpace

4 Theory of Computation What is the computation? What problems are computable by computer in finite steps? What problems are computable by small number of steps?

5 Mathematical Model Real computers are very complicate Develop simple mathematical model to define computation The mathematical models are “equivalent to” real computer under some transformation

6 Important Boundary Computability theory boundary Un-computable problems Computable problems

7 Hierarchy inside computable area

8 Basic Concepts Set: a set is a group of objects represented as a unit. {7,21,57} Element: A object of a set is called an element. Subset: A is a subset of B if every member of A is also a member of B.

9 Some sets Natural numbers set: N={1,2,3,…} Integers set Z={…, -2,-1,0,1,2,…} Empty set: it is a set with no elements

10 Set operations Union: A={2,4,9} B={1,2,5} ={1,2,4,5,9} Intersection: ={2 }

11 Tuple Sequence: a list of objects in some order (7, 21, 57) Tuple: finite sequence (7, 21,57) Cartesian product: A x B is the set of all pairs with first element from A and second element from B A={1,2} B={x,y,z} A x B= {(1,x),(1,y), (1,z), (2,x), (2,y), (2,z) }

12 Power Set Let A be a set. The power set of A is the set of subsets of A. For A={a,b}, its power set P(A) is {, {a}, {b}, {a,b}}

13 Cartesian product

14 Function A function f is mapping from one set D to another set R f: D  R for every a in the set D, there is another element b in R such that a is mapped to b by f f(a)=b Domain: D Range: R

15 Function example Function f: {0,1,2,3,4}  {0,1,2,3,4} n f(n)

16 Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

17 Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

18 Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

19 Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

20 Equivalence relation A binary relation R is equivalence relation if Reflexive: xRx for every x Symmetric: if xRy, then yRx Transitive: if xRy and yRz, then xRz Example 1: = on {1,2,3}x{1,2,3} Example 2: = on NxN

21 Example for equivalence Relation For two integers x and y, x y if (x-y) is a multiple of 7 In other words, there is another integer z such that (x-y)=7z.

22 Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ } v1 v2 v3 v4

23 Path Graph G=(V,E) A path is a series of edges linked one by one Loop:

24 Tree A graph is connected if every two nodes have a path to connect them A tree is a connected graph without loop

25 Connected Graph  Tree Every connected graph can be converted into tree by removing some edges Removing one edge on a loop does not damage the connectivity.

26 A tree is a minimal connected graph Removing any edge on a tree damages the connectivity Proof. Tree T=(V,E). Let (v1, v2) be removed from T. T  T’=(V, E-{(v1,v2)}). If T’ is still connected, T has a loop containing v1 and v2. Contradiction!

27 Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ } v1 v2 v3 v4

28 Node degree: The number of edges connecting to the node v1 v2 v3 v4

29 Mathematical approach Definition Mathematical statement Express some object with certain property Theorem A statement proved to be true Proof

30 Mathematical proof Convincing logical argument that a statement is true Usually consists a series logical statements There is a small logical gap between the current logic statement with previous statements.

31 3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

32 Sum of node degrees For every graph, the sum of the degrees of all nodes in G is an even number Sum=2+2+2=6 sum= =14

33 Proof Let v1, v2, …, vn are the n nodes of the graph. deg(vi) is the degree of node vi The sum of node degrees is sum=deg(v1)+deg(v2)+…+deg(vn) For each edge e=(vi, vj), it makes one contribution to both deg(vi) and deg(vj). If there are k edges, the sum is 2k.

34 Regular graph A graph is k-regular if every node has degree equal to k Theorem: For each even number n>2, there exists a 3-regular graph with n nodes.

35 Proof by construction n/2 nodes Let every point at top half connect to a point in the bottom half

36 Proof Let be the n nodes of the graph. Add edges for i=0, 1, …,(n/2)-1, and Add edges for i=0,1, …,(n/2)-1

37 Proof by contradiction Assume the theorem is false Lead to an obviously false consequence Example: Jill just came in from outdoor and is complete dry Try to Prove: No rain Proof: Assume it were raining Jill would be wet. A contradiction!

38 No 3-regular graph with odd nodes Theorem: There is no 3-regular graph with odd number of nodes Proof (by contradiction) If the graph G is a 3-regular graph with 2m+1 nodes, where m is an integer at least 0. The sum of the degrees of nodes is sum=3+3+….+3=3(2m+1)=6m+3=2(3m+1)+1 It is a contradiction to our previous theorem.

39 3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

40 Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

41 Example for induction Theorem: For every natural number n,

42 Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume

43 Proof

44 Computation models Finite automata Pushdown automata Turing machine

45 3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

46 Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

47 Example for induction Theorem: For every natural number n,

48 Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume

49 Proof

50 Computation models Finite automata Pushdown automata Turing machine

51 Finite automata Supermarket entrance Front: A person is coming from the front. Rear: …… Both: front and rear Neither: neither front nor rear closed open

52 Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

53 Automata example q2 q3q1

54 Acceptance Any sequence with at least one 1 and even number of 0s following the last 1

55 Notions for automata start state: q1 accept state: q2. In other words, F={q2} The set The transition function is 0 1 q1 q1 q2 q2 q3 q2 q3 q2 q2 Q={q1,q2,q3}

56 Run the automata Start from the start state Follow the state transition based on the current state and symbol accepts if it enters accept state, rejects otherwise

57 Run the machine at inputs 1 q1  q2 01 q1  q1  q q1  q2  q3  q2  q3  q2  q3  q2  q3  q2 0 q1  q2 10 q1  q2  q q1  q2  q3  q2  q3  q3  q3

58 Language and Machine Let A be a set of strings. Let M be a machine If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

59 Example for language acceptance A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

60 Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

61 Formal definition of computation A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

62 Automata What is the language accepted by the automata?

63 Answer L(M)={w| w ends in a 1}

64 Automata What is the language accepted by the automata?

65 Answer L(M)={w| w is an empty string or ends in a 0}

66 Automata What is the language accepted by the automata?

67 Answer L(M)={w| w starts and ends at the same symbol}

68 Designing Automata Language A consists all {0,1} strings with even number of 1s. Problem: design an automata M with L(M)=A.

69 Automata States:

70 Designing Automata Language A consists all {0,1} strings with 001 as substring. Problem: design an automata M with L(M)=A.

71 Automata States:

72 Regular Language M accepts language A if A={w| M accepts w} A language is regular if some finite automata accepts it. Example: A1={w| w is {0,1} string and ends in a 1} A2={w| w is a {a,b} string that starts and ends with the same symbol}

73 Language and Machine Let A be a set of strings. Let M be a machine If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

74 Example for language acceptance A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

75 Formal definition of computation A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

76 Non-determinism blind monkey

77 Symbol It represents the empty symbol. If used, one state moves to the next without consuming any symbol q1 q2

78 Automata example q2 q3q1 q3 q4

79

80 Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

81 Language recognized Let A be a set of strings that contain either 101 or 11 as a substring. E.G , 10111

82 Automata example q2 q3q1 q3 q4

83 Language recognized Let A be a set of strings containing a 1 in the third position from the end E.G

84 Automata example q2 q3 q1 q3 q2q3

85 Problem: What language does it accept? q2 q1 q5 q2q3 q2

86 Language recognized Accept all strings, where k is a multiple of 2 and 3.

87 Some notations For a set Q, P(Q) is the collection of all subsets of Q Example, Q={q1,q2} P(Q)={empty, {q1}, {q2}, {q1,q2}} For alphabet, write

88 Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

89 Equivalence between NFA and DFA Theorem: Every NFA has an equivalent DFA Proof. Given an NFA We will construct an to accept the same language.

90 Proof Q’=P(Q), which is the set of all subsets of Q The transition function or and for some The start state F’={R in Q’| R has accept state in N}

91 Example Given NFA, convert it into DFA 1 2

92 Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 1 1,2 2

93 Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 2 1,2 1

94 Problem: What language does it accept? q2 q1 q5 q2q3 q2

95 Language Operations Let A and B be two languages Union: Concatenation: Star:

96 Closure under Union Theorem: If A and B are regular languages, then is also regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

97 Construct N N1: N2:

98 Construct N N accepts iff one of N1 and N2 accepts

99 Closure under Catenation Theorem: If A and B are regular languages, then is also a regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

100 Construct N N1: N2:

101 Construct N N2 is linked to the accept state of N1

102 Closure under Concatenation Theorem: If A is regular language, then is also a regular language Proof. Let A be accepted by finite automata N1, Find another finite automata N to accept

103 Construct N N1:

104 Construct N N1:

105 Exercise 1. Given NFA, convert it into DFA 2. Let A be the language recognized by the NFA. Design the automata to recognize A* 1 2

106 Regular Operations R is a regular expression if R is 1)A for some a in 2)The special string 3)The empty set 4),where R1 and R2 are regular expressions 5), where R1 and R2 are regular expressions 6),where R1 is a regular expression

107 Regular Operations Examples 1)0*10* 2) 3) 4) 5) 6)

108 Regular  Automata Theorem: Every regular expression is regular language Proof. Let R be a regular expression. Construct a DFA to accept R

109 6 Cases 1)a 2) 3) 4),where R1 and R2 are regular expressions 5), where R1 and R2 are regular expressions 6),where R1 is a regular expression

110 Case 1 a

111 Case 2

112 Case 3

113 Case 4 DFA N1 accepts R1 DFA N2 accepts R2

114 Construct N N accepts

115 Case 5 DFA N1 accepts R1 DFA N2 accepts R2

116 Construct N N accepts

117 Construct N N1 accepts R

118 Construct N N accepts R*

119 Example

120 Regular Operations R is a regular expression if R is 1)A for some a in 2)The special string 3)The empty set 4),where R1 and R2 are regular expressions 5), where R1 and R2 are regular expressions 6),where R1 is a regular expression

121 Regular Operations Examples 1)0*10* 2) 3) 4) 5) 6)

122 Automata  Regular expression If a language is regular, then it is described by a regular expression Proof. Let A be recognized by a DFA A, find a regular expression R for A.

123 Some observation States transition 123

124 Some observation States transition 13

125 Some observation States transition 123

126 Some observations States transition 13

127 Some observation States transition 123

128 Some observations States transition 13

129 Some observations States transition 13

130 Proof Idea Convert DFA by shrinking it step by step Replace the symbols by regular expressions on the state transition

131 Proof Idea Given DFA, covert it into regular expression

132 Proof Idea Add start state s and accept state a Convert DFA by removing those old state one by one Replace the symbols by regular expressions on the state transition s a

133 Example States transition 1 2

134 Example States transition 12 s a

135 Example States transition 1 s a

136 Example States transition s a

137 Regular Operations For and for and

138 Remove a state Convert left to right

139 Verify The old and new machines accept the same language

140 Construct Regular expression Add one start state and one accept state:

141 Pumping Lemma Lemma: If A is regular language, there is a number p such that if s is in A and of length at least p, s may be divided into s=xyz, satisfying 1) for each 2) 3)

142 Some notations For a string s, is the length of s (the number of letters) For example, |adb|=3, |a|=1, |afdsaf|=6 For a string s and integer i, is a string to repeats i times For example, if s=dgh then

143 Proof Idea a q1 q9 q13

144 Analysis Run the input string on a automata Input string: s1 s2 s3 s4 s5 s6 … sn State q1 q2 q3 q9 q5 q9… q13 Let p be the number of states If n>p, two of the states must be equal, say q9 Repeat the substring between the two q9s reach the same accept state q13

145 Proof Let p be the number of the states in the Automata M for A Let (n>=p) be a string in A Let be the state transition sequence There are two equal states (j

146 Application of Pumping Lemma is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

147 Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

148 Application of Pumping Lemma The language L={w | w has an equal number of 0s and 1s} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

149 Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. y contains only 0s. By 1) of the pumping, the string is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

150 Application of Pumping Lemma The language {ww | w is a {0,1} string} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

151 Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the first 0s area is more than the number of 0s in the second 0s area. This is a contradiction.

152 Application of Pumping Lemma The language is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

153 Proof Let s=. It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 1s. By 1) of the pumping, is also in the language. The number of 1s in the string is at least and at most. This is a contradiction since the language L does not contain any string with the number of 1s between and.

154 Problem Prove that is not a regular language.

155 Two ways for language A language can be accepted by a machine A language can be also generated by some rules

156 Context Free Grammar Rules for generating a language: A  0A1 A  # Variables: A Terminals: 0,1,# Start variable: A Rules: A  0A1, A  #

157 Conversion A  XY X  0 Y  AZ Z  1 A  #

158 Context Free Grammar Rules for generating a language: A  0A1 A  B B  # Variables: A, B Terminals: 0,1,# Start variable: A Rules: A  0A1, A  B, B  #

159 Derivation A  0A1  0B1  0#1 A  0A1  00A11  000A111  000B111  000#111

160 Context free grammar A context free grammar is V is a finite set called the variables is a finite set of terminals R is a finite set of rultes is the start variable

161 Language of the grammar If A  w is a rule, u and v are strings, then, called uAv yields uwv if for some strings The language of the grammar is

162 Context Free Grammar Rules for generating a language: A  0A1 A  B Variables: A, B Terminals: 0,1, Start variable: A The language of this grammar is

163 Convert DFA to grammar Automata accepts all strings that end in 1 A B

164 Grammar A  0A A  1B B  1B B  0A (because B represents the accept state) Start variable A, Variables A,B Terminals: 0,1,

165 DFA to grammar conversion For every transition, add a rule For every accept state, add rule

166 Theorem Every regular language can be generated by a grammar with the rules like A  aB, and

167 Chomsky normal form A context-free grammar is in chomsky normal form if every rule is of the form A  BC A  a where “a” is a terminal and A,B,and C are variables. In addition, we permit if S is the start variable.

168 Removing empty Every context free grammar G has another equivalent context free grammar G’, which has no rule for any non-start variable A

169 Proof Add a new start symbol and rule For any rule that A is not a start symbol, 1) replace every R  uAv by R  uv, every R  uAvAw by R  uvw, etc. 2) replace R  A by Repeat the steps above until with non-start variable A does not appear

170 Remove Unit Rule Every context free grammar G has another equivalent context free grammar G’, which has no rule

171 Proof For every A  B, replace every B  u by A  u Repeat the step above until there is no A  B

172 Chomsky Normal Form Every context free grammar G has an equivalent chomsky normal form

173 Proof Remove the for non-start symbol A Remove the unit rule A  B For each at k>2, replace it by rules

174 Proof For each at k>1, replace each terminal by the new symbol and add new rule

175 Example A  0A1 Conversion:

176 Remove the empty

177

178

179

180 Arithmetic expression Grammar

181 Example Generate

182 Generate the expression Steps

183 Arithmetic expression Grammar Start variable: Variable: Terminals: a,+,x,(,)

184 Example Generate

185 Example Generate

186 Leftmost derivation A derivation of a string w is leftmost derivation if the left most variable is replaced at every step  x  + x  a+ x  a+a x  a+a x a

187 Ambiguity A string w is derived ambiguously in context-free grammar G if it has more than one leftmost derivation Grammar G is ambiguous if it generates some string ambiguously Ambiguity gives different interpretations by computer program

188 Example Design the context free grammar for

189 Grammar Partition the problem into two parts. One part is for The second part is Let and S1 is used to get via S2 is used to get via

190 Pushdown Automata State control aabbbaabbb aaaabbbbbabaaabb

191 Example The grammar generates the language It can be accepted by pushdown automata

192 Remove input 0, push 0 into stack State control

193 Remove input 0, push 0 into stack State control

194 Match stack 0 with input 1. Then remove both of them State control

195 Match stack 0 with input 1. Then remove both of them, accept State control

196 Pushdown Automata Let M be 1. is the set of states 2. is the input alphabet 3. is the stack alphabet 4. is the state transition function 5. is the start state, 6. is the set of accept states q2 q3 q4 q1

197 States Explanation Q1 is the start state Q2 is used to push 0 symbol Q3 is used to match 0,1 pairs between stack and input tape Q4 is the accept state

198 Transition a,b  c: Used when the current input symbol a and the stack top symbol b, Remove the top symbol on the stack and push c on it The input is moved to the next after a

199 Difference between Pushdown and finite state automata Pushdown automata has unlimited memory, which is last in, first out. Nondeterministic finite state automata is a special case of pushdown automata that has no memory

200 Pushdown Automata 6-tuple 1.Q is the finite set of states 2. is the input alphabet (finite) 3. is the stack alphabet (finite) is the start state, 6. is the set of accept states

201 A computation of pushdown automata Input string: State transition sequence: For i=0,…,m-1, where and Final state accepts

202 Theorem Every context free language can be accepted by a non- deterministic pushdown automata

203 Example The grammar generates the language It generates the string 0011

204 Pushdown Automata State control

205 Replace S by 0S1 at stack State control

206 Match the stack top symbol with the input symbol, remove both if matched State control

207 Replace S by 0S1 State control

208 Remove after matching State control

209 Replace S by empty State control

210 Remove after matching State control

211 Remove after matching, Accept State control

212 Proof Idea Push $ and start symbol to the stack in the beginning. Repeat the three steps below Replace the top variable A on the stack with a the right side of a rule Match the top terminal on the stack with the input symbol, reject if not matched When stack has $ on the top and all input has been read, accepts

213 How to replace the variable If the current stack has the top element s, replace it with the right of the rule State q a…………….. s....s....

214 New States The states are new states, specially added for the rule State moves from q to r after the variable is replaced on the top of the stack

215 Pumping Lemma Lemma: If A is context-free language, there is a number p such that if s is in A and of length at least p, s may be divided into s=uvxyz, satisfying 1) for each 2) 3)

216 Proof Idea Two variables are the same on a path

217 Proof Idea A path from root to a leaf has all variables except the last one, which is a terminal When the path is too long, same variable has to happen twice. Repeat the part of the two equal variables area.

218 Analysis Let V be the set of variables in a grammar. |V| is the number of variables in V If a path has at least |V|+1 variables, two of them will be equal. A path from root to a leaf has least |V|+1 variables if its length is at least |V|+2 A tree of depth |V|+2 has leaves, where b is the maximal length of right side among all rules.

219 Proof Let If s has length, the parse tree T has height It has a path has at least |V|+1 variables. Two of the variables on the path are equal (=R). We have So,

220 Proof The tree T is the parsing tree for s and has least size It is impossible that is empty otherwise, T is not least. Select the bottom |V|+2 symbols on the longest path of T, this makes the

221 Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

222 Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. v and y are in the area of the same symbol, say a. Contradiction for having more a than b. Case 2. v or y contains more than one symbol, contradiction for the incorrect order of symbols in

223 Regular Pumping The language is not regular Proof. Assume it is a regular language. It can be accepted by automata M. Let p be the length of the pumping lemma. Consider s can be expressed s=xyz, where The string xz has the number of 0s no more than 1s (Pump it down to get the contradiction)

224 Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

225 Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy have intersection with the a area. Contradiction for having more a or b than c (pumping up). Case 2. vy does not contain symbol a, contradiction for having more a than b or c (pumping down).

226 Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

227 Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy is in the first half. Contradiction for moving 1 to the right area(pumping up). Case 2. vy is in the second half. Contradiction for moving 0 to the left area(pumping up).

228 Proof The language is not context free Proof. Case 3. vy crosses the middle line. Contradiction for reducing 0 or 1 in the middle area (pumping down). The left half and right half have different number of 1s or 0s.

229 Problem Write a context-free grammar for the following language

230 Problem Using pumping lemma to disprove the following language is context-free:

231 Algorithm An algorithm is a collection of simple instructions for carrying out some task

232 Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

233 Unlimited Register Machine Statement1: x  x+1 Statement2: x  y Statement3: x  0 Statement4: if (x==y) jump m Each register can save an unbounded integer

234 Turing Machine Proposed in 1936 An accurate model for the general purpose computer.

235 Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab

236 Language ww Design Turing machine for L={w#w|w is in {0,1}*} For example: 011#011 is in L 10011#10011 is in L

237 Movement on the tape Move the head back and forth to match all pairs

238 Movement on the tape Move the head

239 Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

240 State transition function For

241 State transition function For

242 Configuration Current state: q7 Current head position on the tape: 4 th cell Current tape content: abab q7 abab

243 Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

244 Configuration Transition For

245 Configuration Transition For

246 Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

247 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration

248 Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

249 a q1 q2 q3 q4 q6 q8 q10 q12 q13 q14 q5 q7 q9 q11 q_accept

250 Multi-tape Turing Machine Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control ba aaa 01010

251 Multi-tape Turing Machine Transition for multi-tape Turing machine

252 Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 0101#aaa#b#a#

253 Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

254 Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 ab1aa0a0

255 Simulate 3 tapes q7 ab1aa0a0

256 Simulate 3 tapes q7 ab1aa0a0

257 K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j- 1)k+2i-1 and 2(j-1)k+2i

258 Midterm October 18, 2010 Class Time Close book Chapter 0-Chapter 3

259 Problem 5 Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains an equal number of 0s and 1s}.

260 Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab

261 Language L Design Turing machine for L={w|w is in {0,1}* and the same number 1s as the same number of 0s} For example: is in L is not in L

262 Movement on the tape Move the head back and forth to pair up 0 and 1.

263 Movement on the tape Move the head

264 Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control

265 Language L Design Turing machine for L={w|w is in {0,1}* and the same number 0s is two times the number of 1s} For example: is in L is not in L

266 Movement on the tape Move the head back and forth to pair up 0 and 1.

267 Movement on the tape Move the head

268 Problem 6 Problem 6 (10) If A and B are languages, define and. Prove that if A and B are regular languages, then is a context free language.

269 Problem 6 DFA N1 accepts A DFA N2 accepts B

270 Construct N N accepts

271 Problem 7 Problem 7 (10) If A and B are languages, define and and |x|=|y|}. Prove that if A and B are regular languages, then is a context free language. You will get 10 more points for midterm.

272 Deterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

273 Nondeterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

274 Nondeterministic Turing Machine One configuration can have multiple choice for entering the next configuration because of

275 Non-determinism blind monkey

276 Automata example q2 q3q1 q3 q4

277

278 Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

279 Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

280 Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

281 Simulate NTM with DTM Each dot is one configuration Find an accepting path

282 Simulate NTM Think about the problem from the Java programming point of view Convert the strategy into deterministic Turing machine.

283 Simulate NTM Key points: search accepting path with width first until it finds the first one. Search from the left to right Search from small level 0, to level 1, to level 2,…

284 Simulate NTM 1: first branch, 2: second branch, etc 121: take the first branch at level 2, then take the second at level 3, then take the first branch at level 4. Control 121 aaa 01010

285 Turing machine and Computer The computational power of Turing machine is equivalent to regular computer with unlimited memory

286 Successor Funtion f(x)=x+1

287 Successor Funtion f(x)=x+1

288 Successor Turing machine is able to simulate the operation x  x

289 Comparison Turing machine is able to check if x=y

290 Transfer Turing machine is able to simulate y  x

291 Transfer Turing machine is able to simulate x  y

292 Set zero Turing machine is able to simulate x 

293 Program statements Statement1: x  x+1 Statement2: x  y Statement3: x  0 Statement4: if (x==y) jump m

294 Compute Algorithm: Let x be added by 1 y times Program for x+y: 1: z  0 2: If (z==y) jump 6 3: x  x+1 4: z  z+1 5: If (z==z) jump 2 6:

295 Jump TM  program state Program q1 1: I1 q2 2: I2 q3 3: I3 …. …… qk k: jump 3 The jump can be achieved via state transition qk  q3

296 Compute Algorithm: Let x be added by original x y times Program for x*y: 1: z  0 2: p  0 3: If (z==y) jump 7 4: p  p+x 5: z  z+1 6: If (z==z) jump 3 7:

297 Compute Algorithm: Let x be multiplied by x y times Program for : 1: z  0 2: p  1 (via p  0 and p  p+1) 3: If (z==y) jump 7 4: p  p*x 5: z  z+1 6: If (z==z) jump 3

298 Algorithm An algorithm is a collection of simple instructions for carrying out some task

299 Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

300 Unlimited Register Machine Statement1: x  x+1 Statement2: x  y Statement3: x  0 Statement4: if (x==y) jump m Each register can save an unbounded integer

301 Other computation model Lambda calculus (Church) Recursion (Godel and Kleene) Post and Markov’s model

302 Evidences Many approaches led to the same algorithmic computable class No one has found an algorithm that is accepted in informal sense, but it can not be implemented in Turing model

303 Hilbert 10 th Problem Find an algorithm to decide if a polynomial has integer root Input: a polynomial e.x. (it has root x=5,y=3,z=0) Output: yes or no

304 Midterm Problem 1 (20) Give the state diagram of a DFA recognizing {w|w is 0,1-string with at least five 0s}. Problem 2 (20) Prove that is not regular language with the pumping lemma.

305 Midterm Problem 3 (20) Prove the following facts: a) If A and B are context free languages, then so is their union. b) If A and B are context free languages, then so is their concatenation.

306 Midterm Problem 4 (20) a) Design a context-free grammar to recognize the language. b) Prove that is not a context- free language by using pumping lemma.

307 Midterm Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains more 0s than 1s}.

308 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

309 DFA ={ | B is a DFA that accepts input string w} Theorem: is a decidable language.

310 Proof The input is, where B is a DFA, and w is a string. Simulate B on the input w: Start from the state of B and leftmost symbol w. Follow to transit the state and move the input symbols one by one Accepts if ends at an accept state; rejects otherwise.

311 NFA ={ | B is a NFA that accepts input string w} Theorem: is a decidable language.

312 Proof The input is, where B is a NFA and w is a string. Convert B into a DFA C via our previous algorithm Use our last TM to decide if C accepts w Accepts if it accepts; rejects otherwise.

313 ={ | L(A) is not empty} Theorem: is a decidable language.

314 Proof The input is, where A is a DFA. Mark the start state Repeat Mark a new state that has a transition to it from a marked one. Until no new state can be added Accept if an accept state is marked; otherwise reject

315 ={ | A and B are DFA and L(A)=L(B)} Theorem: is a decidable language.

316 Proof The input is, where A and B are DFA Check if is empty

317 Problem Define L={ : L(A) is the union of L(B) and L(C)}, where A, B, and C are DFAs. Prove that L is decidable.

318 One- one and onto Let A and B be two sets. For function f: A  B, if whenever then f is called one-one. For function f: A  B, say f is onto if f hits every element of B(In other words, for very b in B, there is a in A such that b=f(a))

319 Correspondence Let A and B be two sets. A and B are of the same size if there is a one-one and onto function f: A  B For function f: A  B, if it is both one-one and onto, then f is called correspondence.

320 Examples {1,2,3,…} and {2,4,6,…} are of the same size via f(x)=2x. (0,1) and are of the same size

321 Countable A set is countable if it is finite or it has the same size as N={1,2,3,…} Theorem: The positive rational numbers set is coutable

322 Proof Every positive rational number is in the table below

323 Proof List all of them by and avoid repetition

324 Examples For two positive rational numbers, ( p and q have no common divisor>1, and p’ and q’ have no common divisor >1), The number is listed before if p+q

325 (0,1) is not countable Theorem: (0,1) is not countable

326 Proof. Proof by contradiction. Assume N={1,2,3,…} and (0,1) have the same size. There is one-one and onto map f: N  (0,1)

327 Proof. Select such that and Since

328 Infinite binary strings set is not countable Let be the set of all infinite binary strings Theorem: is not countable

329 Proof. Proof by contradiction. Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N 

330 Proof. Select such that and Since

331 The set of finite binary strings is countable Correspondence N= {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,…}

332 Language of Binary Strings Every set A of binary strings is countable Proof. Let A be a set of binary strings. Its elements can be listed according to their orders in A={ s1, s2, s3, s4, ….} It is easy to see the correspondence between N and A

333 An infinite binary string uniquely determines a language N= … L= { 11, 011,….} The language with positive even number of 1s strings

334 Correspondence For each infinite binary string B, it uniquely determines a binary language L(B). If B1 and B2 are different binary strings, then L(B1) and L(B2) are different language Each binary language uniquely determines an infinite binary string.

335 Correspondence There is a correspondence between the set of all infinite binary strings and the set of all binary languages. The set of all binary languages is not countable

336 Turing machine to binary string Each Turing machine can be encoded into a binary string. The set of all Turing machines can be encoded into a set of binary strings. The set of Turing machines is countable. The set of binary languages recognized by TM is also countable

337 Un-computable language by TM There is a binary language that is not Turing recognizable Proof. The set of binary languages is not countable, but the set of language recognized by Turing machine is countable

338 Undecidable Problem ={ | M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

339 Every entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

340 Proof Assume that is decidable There is a TM H such that H( ) accepts if M accepts w H( ) rejects if M rejects w. Consider a TM D D( ) accepts if M rejects D( ) rejects if M accepts

341 Proof Since D( ) accepts if M rejects D( ) rejects if M accepts We have D( ) accepts if D rejects D( ) rejects if D accepts A contradiction

342 Every entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

343 Every diagonal entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

344 Diagonal method reject reject accept accept accept reject ……

345 Call H in R If TM H exists, the TM R also exits via using H (Software R uses an existing software H)

346 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

347 Problem Is there any one-one and onto map from the set of integers in [1,10] and the set of odd integers in [1,10]? Why? Prove that there is a one-one and onto map from the set of all integers and the set of all odd integers.

348 Reduction Solution for Problem 1 Solution for Problem 2 Help

349 Example: Map and Direction MapDirection?

350 Reduction Solution for Problem 1 Problem 2 Question Answer

351 Reduction Software for Problem 1 Software for Problem 2: call Return

352 Reduction & Undecidability It is known that P2 is undecidable We want to prove P1 is undecidable Proof by contradiction Assume P1 is decidable. There is a software solving P1 Design a software for P2 by calling the software for P1 Contradiction!!!!

353 Halting Problem P1: ={ | M is a Turing machine and M halts on input w} P2:. ={ | M is a Turing machine and M accepts input w} Undecidable

354 P2 Input If M does not stop on w, reject it If M stops, simulate M. If M acceps, accpets Otherwise

355 Halting Problem ={ | M is a Turing machine and M halts on input w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

356 Undecidable Problem ={ | M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

357 Proof Assume the Turing machine R decides We can use R to decides For input Run R on if R rejects, “reject” if R accepts, simulate M until it stops if M accepts, “accept” else “reject”

358 Empty Problem ={ | M is a Turing machine and } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

359 Proof Assume the Turing machine R decides For input, design another TM If, reject simulate M on input w, accepts if M accepts w

360 Proof If M accepts w, then w belongs to L( ) Otherwise, L( ) is empty

361 Proof Use R to decide Input Make from Run R on the input If R rejects (it means L( ) is not empty), accepts I R accepts (it means L( ) is empty), rejects

362 Problem ={ | M is a Turing machine and L(M) is regular} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

363 Proof Assume the Turing machine R decides For input, design another TM (x) If x has format, accept simulate M on input w, accepts if M accepts x

364 Proof If M accepts w, then Otherwise,

365 Proof Use R to decide Input Make from Run R on the input If R rejects (it means L( ) is not regular), accepts I R accepts (it means L( ) is regular), rejects

366 Turing machine equivalence Problem ={ | M1 and M2 are Turing machines and L(M1)=L(M2) } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

367 Proof For input for TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

368 Mapping Reducibility Convert instances of problem to instances of problem B A B

369 Computable function Assume that is a set of finite number of symbols is the set of all finite strings with symbols from The function is computable if some Turing machine M, on every input w, halts with just f(w) on the tape. Example: +,x,/ are all computable functions

370 Mapping Reducibility Language A is mapping reducible to language B, if there is a computable function such that for every w

371 Example A={1,3,5,….} B={0,2,4,…} f(x)=x+1 via the funtion f

372 Theorem If, and B is decidable, then A is also decidable A B

373 Proof Let f be the reduction from A to B since Let M be the decider for B. Decider N: Input w, Compute f(w) Run M on f(w) and accept iff M accepts

374 Corollary If, and A is undecidable, then B is also undecidable A B

375 Proof Proof by contradiction. If B is decidable, then so is A

376 Halting Problem P1: ={ | M is a Turing machine and M halts on input w} P2:. ={ | M is a Turing machine and M accepts input w} Undecidable

377 A reduction from to B: ={ | M is a Turing machine and M halts on input w} A:. ={ | M is a Turing machine and M accepts input w} Undecidable

378 Reduction On input Construct Turing machine M’ Run M on w If M accepts, accept If M rejects, enter an infinite loop Output

379 Empty Problem ={ | M is a Turing machine and } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

380 Reduction from to For, design another TM M’ Input x If x is not equal to w, reject else simulate M on input w, accepts if M accepts w

381 Reduction from to is in L(M’) is not empty is not in L(M’) is empty

382 Reduction from to is in M’ is in is not in M’ is not in

383 Problem Let ={M| M is a Turing machine and M prints 1 for some input w} Show that L is undecidable.

384 Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control abab

385 Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

386 Configuration Current state: q7 Current head position on the tape: 4 th cell Current tape content: abab q7 abab

387 Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

388 Configuration Transition For

389 Configuration Transition For

390 Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

391 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration

392 Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

393 Turing Recognizable Turing machine M recognizes language L

394 Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

395 Turing Decidable Turing machine M decides language L

396 Observation If L is Turing decidable, then L is Turing recognizable

397 A Turing recognizable problem ={ | M is a Turing machine and M accepts w} Turing machine R recognizable For input Simulate M on w R accepts if M accepts w.

398 Theorem Theorem: If L is a language in {0,1}*, there is a Turing machine to print out all elements in L

399 Proof Let M recognize L Step i(i=1,2,3…) Simulate M each of the first i strings i steps If a string is accepted, print it out.

400 Theorem If, and B is Turing recognizable, so is A A B

401 Proof Let f be the reduction from A to B since Let M recognize B. TM N: Input w, Compute f(w) Run M on f(w) and accept if M accepts

402 Theorem If, and A is not Turing recognizable, B is not Turing recognizable A B

403 Complement set Set and its complement

404 Theorem If A and are Turing recognizable, then A is decidable

405 Proof Let Turing machine M1 recognize A Let Turing machine M2 recognize For input x, Run M1 and M2 on x in parallel If M1 accepts, accept If M2 accepts, reject.

406 ={ | A and B are DFA and L(A)=L(B)}.

407 ={ | A and B are DFA and L(A)=L(B)} Theorem: is not Turing decidable language.

408 . Theorem: Neither nor its complement is Turing recognizable

409 Proof We first prove For input for TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

410 Proof We prove For input for TM M1: accepts any input TM M2: accepts any input if M accepts w. It is easy to see L(M1)=L(M2) iff M accepts w So,

411 Problem Show that the language F={M| L(M) contains infinite elements} is not Turing recognizable}.

412 Problem 4.2 The equivalence of a DFA and a regular expression is decidable

413 Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

414 Problem 4.7 The set of all infinite binary strings is not countable

415 Problem 4.8 N={1,2,3,…} NxNxN is countable

416 Problem 5.9 All Turing recognizable problems are mapping reducible to

417 Problem 5.12 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable.

418 Problem 5.13 Test if a Turing machine has useless state, which never enters.

419 Language Design Turing machine to recognize

420 Complexity Let M be a deterministic Turing machine. The running time of M is the function f:N  N such that f(n) is the maximum number of steps that M uses on input of length n.

421 Big O-notation Let f and g be functions Say f(n)=O(g(n)) if integers c and exist so that for all

422 Examples Let

423 Small o-notation Let f and g be functions Say f(n)=o(g(n)) if

424 One tape Turing machine time One tape Turing machine can recognize In steps

425 Two tapes Turing machine time Two tapes Turing machine can recognize In O(n) steps

426 Multi-tape Turing Machine Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control ba aaa 01010

427 Multi-tape Turing Machine Transition for multi-tape Turing machine

428 Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 0101#aaa#b#a#

429 Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

430 Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 ab1aa0a0

431 Simulate 3 tapes q7 ab1aa0a0

432 Simulate 3 tapes q7 ab1aa0a0

433 K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j- 1)k+2i-1 and 2(j-1)k+2i

434 Simulate multi-tape by one tape One tape Turing machine can simulate t(n) time multi-tape Turing machine in time

435 Time Complexity Class Let. is the class of languages decided by O(t(n) time Turing machines. In other words is a language decided by O(t(n)) time Turing machine}

436 One tape Turing machine time One tape Turing machine can decide In steps

437 Complexity Class P P is the class of languages decided by single tape Turing machine in polynomial time

438 Directed Path Problem G is a directed graph that has a directed path from s to t} s a b c d t

439 Algorithm Input mark “s” Repeat For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

440 Greatest common divisor Divisor: For two integers b and c, if b=c*z for some integer z, c is a divisor of b. Greatest common divisor: Given two integers a and b, gcd(a,b) is the greatest positive integer c such that c is the divisor for both a and b. Examples: gcd(10,4)=2, gcd(16,100)=4 Problem: How to find gcd(a,b)?

441 Relatively Prime Problem Two integers x and y are relatively prime if gcd(x,y)=1. RELPRIME={ | x and y are relatively prime} Theorem:

442 Euclid algorithm For two integer b and a a=q*b+c with gcd(a,b)=gcd(b,c) a and b are relatively prime iff b and c are relatively prime

443 Euclid algorithm Assume a1 and a2 are two positive integers If, then and Therefore, always true

444 Algorithm for gcd(x,y) Input Repeat r  x mod y x  y y  r until y=0 output x

445 Modular Assume a and b are two positive integers This is a recursive equation since the second item goes down

446 Example Find gcd(1970,1066)

447 Speed of Euclid algorithm Assume a1 and a2 are two positive integers If, we have In another words,

448 Euclid algorithm Assume a1 and a2 are two positive integers

449 Time For input The total time of steps is O(log a+log b)

450 Non-determinism blind monkey

451 Automata example q2 q3q1 q3 q4

452

453 Deterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

454 Nondeterministic Turing Machine 7-tuple 1.Q is the finite set of states 2. is the input alphabet not containing special blank 3. is the tape alphabet is the start state, 6. is the accept state 7. is the reject state, where

455 Nondeterministic Turing Machine One configuration can have multiple choice for entering the next configuration because of

456 Non-determinism blind monkey

457 Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

458 Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

459 Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

460 Simulate NTM with DTM Each dot is one configuration Find an accepting path

461 Class NP NP is the class of languages that can be recognized by nondeterministic Turing machine in polynomial time

462 Hamiltonian Path Hamiltonian path goes through each node exactly once HAMPATH={ | G is a directed graph with a Hamiltonian path from s to t}

463 Composite A natural number is composite if it is the product of two integers >1 10=2*5 18=2*9 Composite={x| x=pq, for integers p,q>1}

464 Verifier A verifier for a language L is an algorithm V, L={w| V accepts for some string c} For the verifier V for L, c is a certificate of w if V accepts If the verifier V for the language L runs in polynomial time, V is the polynomial time verifier for L.

465 Verifier for Hamiltonian Path For, a certificate is a list of nodes of G: Verifier: check if m is the number of nodes of G check if and check if each is a directed edge of G for i=1,…,m-1 If all pass, accept. Otherwise, reject.

466 Verifier for Composite For integer x, a certificate is two integers p,q: Verifier: check if p>1 and q>1 check if x=pq If all pass, accept. Otherwise, reject.

467 Class NP NP is the class of languages that have polynomial time verifiers. Examples: COMPOSITE is in NP HAMPATH is in NP

468 Theorem A language has polynomial verifier iff it can be recognized by polynomial time nondeterministic Turing machine Proof:  Assume L has verifier V, …  Assume L has NTM M,…

469 Proof  Assume that L has polynomial time verifier V, which runs in time, where k is a constant NTM M Nondeterministically select string c of length Run V on If V accepts, accept; Otherwise, reject.

470 Proof  Assume that L is recognized by polynomial time NTM M, which runs in time, where k is a constant Verifier Let c determine a computation path of M Simulate M on the path c If M accepts, accept. Otherwise, reject.

471 Clique Problem Given undirected graph G, a clique is a set of nodes of G such that every two nodes are connected by an edge. A k-clique is a clique with k nodes

472 Clique Problem CLIQUE={ | G iss an undirected graph with k-clique} CLIQUE is in NP.

473 Subset Sum Problem SUBSET-SUM={ | S= and for some, we have

474 Polynomial Time Computable A function is a polynomial time computable function if some polynomial time Turing machine M exists that outputs on the tape for input w.

475 Polynomial Time Reduction Assume that A and B are two languages on A is polynomial time mapping reducible to A if a polynomial time computable function exists such that

476 Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

477 SAT A boolean formula is satisfiable if there exists assignments to its variables to make the formula true SAT={ | is satisfiable boolean formula}

478 3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

479 3SAT to CLIQUE Example:

480 NP-completeness A language B is NP-complete if 1.B is in NP, and 2.Every A in NP is polynomial time reducible to B Theorem. If B is NP-complete and B is in P, then P=NP.

481 Cook-Leving Theorem Theorem: SAT is NP-complete Proof. 1. SAT is in NP. 2. For every problem A in NP,

482 Proof 1. The start configuration is legal 2. The final state is accept. 3. The movement is legal. 4. Each cell takes one legal symbol.

483 Proof 1. 1 if The cell[i,j] holds symbol s; 0 otherwise 2. Time bound for the NTM M with constant k. 3. The movement is legal. 4. NTM M for accepting A.

484 Nondeterministic Turing Machine 1.Q is the finite set of states 2. is the tape alphabet is the start state, 5. is the accept state.

485 Configuration Transition For

486 Configuration Transition For

487 Configuration Start configuration:, where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

488 Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields, and 3. is an accepting configuration

489 Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

490 Proof Each cell has only one symbol 1.The symbol is selected from C: 2.Only one symbol is selected: 3.It is true for all cell at all configuration:

491 Proof The start configuration is

492 Proof Accept computation has reached. It makes sure the accept state will appear among the configuration transitions.

493 Proof Characterize the legal move The whole move is legal if all windows are legal. Characterize one window is legal

494 Proof The state transition

495 Logic Demorgan Law:

496 Truth table for y1 y2 x

497 Convert to CNF Conversion:

498 Convert to CNF Conversion:

499 Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

500 Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory

501 Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular

502 Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

503 Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

504 Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

505 Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier

506 Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete.

507 Grade A:… B:… C: Miss exam or homework

508 SAT’ A conjunctive normal form is a conjunction of some clauses SAT’={ | is satisfiable conjunctive normal form}

509 Cook-Leving Theorem’ Theorem: SAT’ is NP-complete Proof. Same as that for SAT is NP-complete

510 3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

511 3SAT is NP-complete Theorem: There is polynomial time reduction from SAT’ to 3SAT.

512 3SAT is NP-complete is satisfiable if and only if the following is satisfiable

513 3SAT is NP-complete is satisfiable if and only if the following is satisfiable

514 3SAT is NP-complete Convert every clause into 3cnf:

515 3SAT is NP-complete Conjunctive normal form Each clause is convert into is satisfiable if and only if the following is satisfiable

516 Problem Convert the formula F into 3SAT formula F’ such that F is satisfiable iff and F’ is satisfiable.

517 Approximation Algorithms

518 Outline and Reading Approximation Algorithms for NP-Complete Problems –Approximation ratios –Polynomial-Time Approximation Schemes –2-Approximation for Vertex Cover –Approximate Scheme for Subset Sum –2-Approximation for TSP special case –Log n-Approximation for Set Cover

519 Approximation Ratios Optimization Problems –We have some problem instance x that has many feasible “ solutions ”. –We are trying to minimize (or maximize) some cost function c(S) for a “ solution ” S to x. For example, Finding a minimum spanning tree of a graph Finding a smallest vertex cover of a graph Finding a smallest traveling salesperson tour in a graph

520 Approximation Ratios An approximation produces a solution T –T is a k-approximation to the optimal solution OPT if c(T)/c(OPT) < k (assuming a min. prob.; a maximization approximation would be the reverse)

521 Polynomial-Time Approximation Schemes A problem L has a polynomial-time approximation scheme (PTAS) if it has a polynomial-time (1+  )-approximation algorithm, for any fixed  >0 (this value can appear in the running time). Subset Sum has a PTAS.

522 Vertex Cover A vertex cover of graph G=(V,E) is a subset W of V, such that, for every (a,b) in E, a is in W or b is in W. OPT-VERTEX-COVER: Given an graph G, find a vertex cover of G with smallest size. OPT-VERTEX-COVER is NP-hard.

523 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

524

525 A 2-Approximation for Vertex Cover Every chosen edge e has both ends in C But e must be covered by an optimal cover; hence, one end of e must be in OPT Thus, there is at most twice as many vertices in C as in OPT. That is, C is a 2-approx. of OPT Running time: O(m) Algorithm VertexCoverApprox(G) Input graph G Output a vertex cover C for G C  empty set H  G while H has edges e  H.removeEdge(H.anEdge()) v  H.origin(e) w  H.destination(e) C.add(v) C.add(w) for each f incident to v or w H.removeEdge(f) return C

526 Problem 1 Show that T={(i,j)| both I and j are positive rational numbers} is countable.

527 Problem 2. Find a correspondence between (0,1) and [0,1]

528 Solution Let for n=1, 2, … Define the correspondence f(x) below:

529 Problem 3 3. Let A={ | R and S are regular expressions and L(R ) is a subset of L( S)}. Show that A is decidable.

530 Solution For two regular expression R and S, L( R) is a subset of L(S) if and only if the L(R ) has no intersection with. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Check if L(M4) is empty R S

531 Problem 4 I={x| x is an irrational number in (0,1)}. Show that I is not countable.

532 Homework 2009

533 Problem 4.2 The equivalence of a DFA and a regular expression is decidable

534 Solution Proof. Given a regular language L and DFA M. We can construct a DFA N to accept L. The problem is converted into checking if two DFAs are equivalent. By Theorem 4.5, there is an algorithm to decide if L(M)=L(N).

535 Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

536 Solution Assume M is a DFA with n states. L(M) is infinite iff it has path from the start state to accept with a loop on it, and the loop length is no more than n. L(M) is infinite iff M accepts a string of length between n and 2n. For every string x with length between n and 2n Run M on x If M accepts x, then output “L(M) is infinite” else output “no”

537 Problem 4.7 The set of all infinite binary strings is not countable

538 Proof. Proof by contradiction. Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N 

539 Proof. Select such that and Since

540 Problem 1 N={1,2,3,…} NxNxN is countable

541 Proof List all of them by

542 Proof Use the similar method like the last slide to prove the NxN is countable Let f: NxN  N be one-one and onto We have g: NxNxN  N is one-one and onto, where g(i,j,k)=f(f(i,j),k)

543 Problem 2. Find a correspondence between (0,1) and [0,1]

544 Solution Let for n=1, 2, … Define the correspondence f(x) below:

545 Problem 3 3. Let A={ | R and S are regular expression and L(R ) = L( S)}. Show that A is decidable.

546 Solution For two regular expression R and S, L( R)=L(S) if and only if their symmetric difference is empty. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Construct automata M5 to accept Check if L(M5) is empty

547 Problem 4 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable.

548 Proof We are going to design a reduction from to S. For Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts is in iff N accepts both 01 and 10.

549 Problem 5 5. Let S={ | M is a TM and L(M) = { }}. Show that neither S nor its complement is Turing-recognizable.

550 Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal then reject If x= then accepts if M(w) accepts.

551 Solution For an input (M, w), derive M1(x, M1).

552 Solution We show that can be reducible to. Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x = then accept If x is not equal then accept if M(w) accepts.

553 Solution For an input (M, w), derive M2(x, M2).

554 Problem 6 Show that A is a Turing recognizable if and only if

555 Proof Part a)  Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =. It is easy to see that

556 Proof Part b)  Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize. We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

557 Proof We are going to design a reduction from to it For Design the TM N as follows Input x if x=0, enters all states except if x=1 run M on w, if M accepts w, enter is in iff N enters all states.

558 Problem 4 c) The language is not a context free language Proof: Assume it is a context free language. Let p be the number from the pumping lemma Consider the string

559 Final Exam December 13, 5:45-7:30pm, Monday

560 Homework 4, 20

561 Problem 1 1.Let T1={ | M is a Turing machines and accepts infinite number of 0,1-strings of finite length}. Prove that T1 is undecidable.

562 Proof We are going to design a reduction from to T1. For input Turing machine for Design the TM N as follows Input x run M on w, if M accepts w, acceptx is in iff N acccepts each input x.

563 Problem 2 Let T2={ | M is a Turing machines and accepts five of 0,1-strings of finite length}. Prove that T2 is undecidable.

564 Proof We are going to design a reduction from to T2. For Design the TM N as follows Input x if x is not 0,1, 00, 01 or 10, rejects. run M on w if M accepts w, accept x is in iff N accepts 0,1,00,01, and 10. Otherwise, N accepts no string

565 Problem 3 S={ | M is a TM that accepts whenever it accepts w}. Show S is undecidable.

566 Proof We are going to design a reduction from to S. For Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts is in iff N accepts both 01 and 10.

567 Problem 4 Show that A is a Turing recognizable if and only if

568 Proof Part a)  Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =. It is easy to see that

569 Proof Part b)  Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize. We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

570 Problem 5 5. Let S={ | M is a TM and L(M) = { }}. Show that neither S nor its complement is Turing-recognizable.

571 Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal then reject If x= then accepts if M(w) accepts.

572 Solution For an input (M, w), derive M1(x, M1).

573 Solution We show that can be reducible to. Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x = then accept If x is not equal then accept if M(w) accepts.

574 Solution For an input (M, w), derive M2(x, M2).

575 Homework 5, 20

576 Problem 1 Call graphs G and H isomorphic if the nodes of G may be reordered so that it is identical to H. Let ISO={ | G and H are isomorphic graphs}. Show that ISO is in NP.

577 Solution for Problem 1 We design a polynomial time verification algorithm. A mapping f from the vertices of G to those of H is a witness. Check if 1) f is one-one, 2) f is onto, 3) (v,u) is an edge of G iff (f(u),f(v)) is an edge of H. It is easy to see that the verification takes polynomial time.

578 Problem 2 Let MAX-CIQUE={ | the largest clique of G has k vertices}. Whether MAX-CLIQUE is in NP is unknown. Show that if P=NP, then MAX-CLIQUE is in P, and a polynomial time algorithm exists that, for a graph G, finds one of its largest cliques.

579 Solution of Problem 2 Fist step is to find the largest k with (G,k) is in Clique. Try k from 1,2,… Check if (G,k) is in Clique Select the largest k. Assume the largest k for (G,k) in Clique is obtained.

580 Solution of Problem 2 Assume the largest k for (G,k) in Clique is obtained. Formulate the problem: (H, k,G) Determine if there is a clique of size k in G and contains all veritces in H. The problem is in NP. Extend H one by one until its size reaches k.

581 Problem 3 Let G represent an undirected graph and let SPATH={ | G contains a simple path of length at most k from a to b} and LPATH={

582 Solution Problem 3 Show that SPATH is in P.

583 Directed Path Problem G is a directed graph that has a directed path from s to t} s a b c d t

584 Algorithm Input mark “s” Repeat For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

585 Solution Problem 3 Show that LPATH is NP-complete. Hamiltonian path problem is NP-complete. There is an easy reduction from Hamiltonian path to it.

586 Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory

587 Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular

588 Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

589 Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

590 Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

591 Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier

592 Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete.

593 Grade A:… B:… C: Miss exam or homework

594 Complexity ©D.Moshkovits 593 Space Complexity

595 Complexity ©D.Moshkovits 594 Motivation Complexity classes c