Download presentation

Presentation is loading. Please wait.

1
**Theoretical Foundation of Computation**

Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm Tuesday

2
**Contents Computational models and languages**

finite automata, push automata regular languages, context free languages Computability theory decidable problem, un-decidable problem Complexity theory time, space, P, NP, PSpace

3
**Theory of Computation What is the computation?**

What problems are computable by computer in finite steps? What problems are computable by small number of steps?

4
**Mathematical Model Real computers are very complicate**

Develop simple mathematical model to define computation The mathematical models are “equivalent to” real computer under some transformation

5
**Un-computable problems**

Important Boundary Computability theory boundary Un-computable problems Computable problems

6
**Hierarchy inside computable area**

7
**Basic Concepts Set: a set is a group of objects represented as a unit.**

{7,21,57} Element: A object of a set is called an element. Subset: A is a subset of B if every member of A is also a member of B.

8
**Some sets Natural numbers set: N={1,2,3,…} Integers set**

Z={…, -2,-1,0,1,2,…} Empty set: it is a set with no elements

9
**Set operations Union: A={2,4,9} B={1,2,5} ={1,2,4,5,9} Intersection:**

={2}

10
**Tuple Sequence: a list of objects in some order (7, 21, 57)**

Tuple: finite sequence (7, 21,57) Cartesian product: A x B is the set of all pairs with first element from A and second element from B A={1,2} B={x,y,z} A x B= {(1,x),(1,y), (1,z), (2,x), (2,y), (2,z) }

11
Power Set Let A be a set. The power set of A is the set of subsets of A. For A={a,b}, its power set P(A) is { , {a}, {b}, {a,b}}

12
Cartesian product

13
**Function A function f is mapping from one set D to another set R**

f: DR for every a in the set D, there is another element b in R such that a is mapped to b by f f(a)=b Domain: D Range: R

14
**Function example Function f: {0,1,2,3,4}{0,1,2,3,4} n f(n) 0 1 1 2**

15
**Relation For two sets A and B, a binary relation R is a subset AxB**

Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

16
**Represent binary relation**

If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

17
**Relation For two sets A and B, a binary relation R is a subset AxB**

Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

18
**Represent binary relation**

If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

19
**Equivalence relation A binary relation R is equivalence relation if**

Reflexive: xRx for every x Symmetric: if xRy, then yRx Transitive: if xRy and yRz, then xRz Example 1: = on {1,2,3}x{1,2,3} Example 2: = on NxN

20
**Example for equivalence**

Relation For two integers x and y, x y if (x-y) is a multiple of 7 In other words, there is another integer z such that (x-y)=7z.

21
**Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ }**

22
Path Graph G=(V,E) A path is a series of edges linked one by one Loop:

23
Tree A graph is connected if every two nodes have a path to connect them A tree is a connected graph without loop

24
Connected Graph Tree Every connected graph can be converted into tree by removing some edges Removing one edge on a loop does not damage the connectivity.

25
**A tree is a minimal connected graph**

Removing any edge on a tree damages the connectivity Proof. Tree T=(V,E). Let (v1, v2) be removed from T. T T’=(V, E-{(v1,v2)}). If T’ is still connected, T has a loop containing v1 and v2 . Contradiction!

26
**Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ }**

27
**Node degree: The number of edges connecting to the node**

v2 v1 v3 v4

28
**Mathematical approach**

Definition Mathematical statement Express some object with certain property Theorem A statement proved to be true Proof

29
Mathematical proof Convincing logical argument that a statement is true Usually consists a series logical statements There is a small logical gap between the current logic statement with previous statements.

30
**3 Styles of Mathematical Proofs**

Proof by construction Proof by contradiction Proof by induction

31
Sum of node degrees For every graph, the sum of the degrees of all nodes in G is an even number Sum=2+2+2= sum= =14

32
**Proof Let v1, v2, …, vn are the n nodes of the graph.**

deg(vi) is the degree of node vi The sum of node degrees is sum=deg(v1)+deg(v2)+…+deg(vn) For each edge e=(vi, vj), it makes one contribution to both deg(vi) and deg(vj). If there are k edges, the sum is 2k.

33
**Regular graph A graph is k-regular if every node has degree equal to k**

Theorem: For each even number n>2, there exists a 3-regular graph with n nodes.

34
**Proof by construction n/2 nodes**

Let every point at top half connect to a point in the bottom half

35
**Proof Let be the n nodes of the graph.**

Add edges for i=0, 1, …,(n/2)-1 , and Add edges for i=0,1, …,(n/2)-1

36
**Proof by contradiction**

Assume the theorem is false Lead to an obviously false consequence Example: Jill just came in from outdoor and is complete dry Try to Prove: No rain Proof: Assume it were raining Jill would be wet. A contradiction!

37
**No 3-regular graph with odd nodes**

Theorem: There is no 3-regular graph with odd number of nodes Proof (by contradiction) If the graph G is a 3-regular graph with 2m+1 nodes, where m is an integer at least 0. The sum of the degrees of nodes is sum=3+3+….+3=3(2m+1)=6m+3=2(3m+1)+1 It is a contradiction to our previous theorem.

38
**3 Styles of Mathematical Proofs**

Proof by construction Proof by contradiction Proof by induction

39
Proof by induction Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

40
Example for induction Theorem: For every natural number n,

41
**Proof Basis: when n=1. The left side is 1 The right is Induction step:**

Assume

42
Proof

43
Computation models Finite automata Pushdown automata Turing machine

44
**3 Styles of Mathematical Proofs**

Proof by construction Proof by contradiction Proof by induction

45
Proof by induction Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

46
Example for induction Theorem: For every natural number n,

47
**Proof Basis: when n=1. The left side is 1 The right is Induction step:**

Assume

48
Proof

49
Computation models Finite automata Pushdown automata Turing machine

50
**Finite automata Supermarket entrance**

Front: A person is coming from the front. Rear: …… Both: front and rear Neither: neither front nor rear closed open

51
**Formal definition of automata**

A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

52
Automata example q1 q3 q2

53
Acceptance Any sequence with at least one 1 and even number of 0s following the last 1

54
**Notions for automata start state: q1**

accept state: q2. In other words, F={q2} The set The transition function is q1 q1 q2 q2 q3 q2 q3 q2 q2 Q={q1,q2,q3}

55
**Run the automata Start from the start state**

Follow the state transition based on the current state and symbol accepts if it enters accept state, rejects otherwise

56
**Run the machine at inputs**

q1q2 q1q1q2 q1q2q3q2q3q2q3q2q3q2 q1q2 q1q2q3 101000 q1q2q3q2q3q3q3

57
**Language and Machine Let A be a set of strings. Let M be a machine**

If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

58
**Example for language acceptance**

A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

59
**Formal definition of automata**

A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

60
**Formal definition of computation**

A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

61
Automata What is the language accepted by the automata?

62
Answer L(M)={w| w ends in a 1}

63
Automata What is the language accepted by the automata?

64
Answer L(M)={w| w is an empty string or ends in a 0}

65
Automata What is the language accepted by the automata?

66
Answer L(M)={w| w starts and ends at the same symbol}

67
Designing Automata Language A consists all {0,1} strings with even number of 1s. Problem: design an automata M with L(M)=A.

68
Automata States:

69
Designing Automata Language A consists all {0,1} strings with 001 as substring. Problem: design an automata M with L(M)=A.

70
Automata States:

71
**Regular Language M accepts language A if A={w| M accepts w}**

A language is regular if some finite automata accepts it. Example: A1={w| w is {0,1} string and ends in a 1} A2={w| w is a {a,b} string that starts and ends with the same symbol}

72
**Language and Machine Let A be a set of strings. Let M be a machine**

If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

73
**Example for language acceptance**

A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

74
**Formal definition of computation**

A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

75
Non-determinism blind monkey

76
**Symbol It represents the empty symbol.**

If used , one state moves to the next without consuming any symbol q1 q2

77
Automata example q2 q1 q3 q3 q4 q2

79
Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

80
Language recognized Let A be a set of strings that contain either 101 or 11 as a substring. E.G , 10111

81
Automata example q2 q1 q3 q3 q4 q2

82
Language recognized Let A be a set of strings containing a 1 in the third position from the end E.G

83
Automata example q2 q3 q1 q3 q2 q3

84
**Problem: What language does it accept?**

q2 q3 q2 q1 q2 q5

85
Language recognized Accept all strings , where k is a multiple of 2 and 3.

86
**Some notations For a set Q, P(Q) is the collection of all subsets of Q**

Example, Q={q1,q2} P(Q)={empty, {q1}, {q2}, {q1,q2}} For alphabet , write

87
**Formal definition of automata**

A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

88
**Equivalence between NFA and DFA**

Theorem: Every NFA has an equivalent DFA Proof. Given an NFA We will construct an to accept the same language.

89
**Proof Q’=P(Q), which is the set of all subsets of Q**

The transition function or and for some The start state F’={R in Q’| R has accept state in N}

90
Example Given NFA, convert it into DFA 2 1

91
Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 1 2 1,2

92
Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 2 1 1,2

93
**Problem: What language does it accept?**

q2 q3 q2 q1 q2 q5

94
**Language Operations Let A and B be two languages Union: Concatenation:**

Star:

95
**Closure under Union Theorem: If A and B are regular languages, then**

is also regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

96
Construct N N1: N2:

97
Construct N N accepts iff one of N1 and N2 accepts

98
**Closure under Catenation**

Theorem: If A and B are regular languages, then is also a regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

99
Construct N N1: N2:

100
Construct N N2 is linked to the accept state of N1

101
**Closure under Concatenation**

Theorem: If A is regular language, then is also a regular language Proof. Let A be accepted by finite automata N1, Find another finite automata N to accept

102
Construct N N1:

103
Construct N N1:

104
**Exercise 1. Given NFA, convert it into DFA**

2. Let A be the language recognized by the NFA. Design the automata to recognize A* 1 2

105
**Regular Operations R is a regular expression if R is A for some a in**

The special string The empty set ,where R1 and R2 are regular expressions , where R1 and R2 are regular expressions ,where R1 is a regular expression

106
**Regular Operations Examples**

0*10* 3) 4) 5) 6)

107
Regular Automata Theorem: Every regular expression is regular language Proof. Let R be a regular expression. Construct a DFA to accept R

108
**6 Cases a ,where R1 and R2 are regular expressions**

,where R1 is a regular expression

109
Case 1 a

110
Case 2

111
Case 3

112
Case 4 DFA N1 accepts R1 DFA N2 accepts R2

113
Construct N N accepts

114
Case 5 DFA N1 accepts R1 DFA N2 accepts R2

115
Construct N N accepts

116
Construct N N1 accepts R

117
Construct N N accepts R*

118
Example

119
**Regular Operations R is a regular expression if R is A for some a in**

The special string The empty set ,where R1 and R2 are regular expressions , where R1 and R2 are regular expressions ,where R1 is a regular expression

120
**Regular Operations Examples**

0*10* 3) 4) 5) 6)

121
**Automata Regular expression**

If a language is regular, then it is described by a regular expression Proof. Let A be recognized by a DFA A, find a regular expression R for A.

122
Some observation States transition 1 2 3

123
Some observation States transition 1 3

124
Some observation States transition 1 2 3

125
Some observations States transition 1 3

126
Some observation States transition 1 2 3

127
Some observations States transition 1 3

128
Some observations States transition 1 3

129
**Proof Idea Convert DFA by shrinking it step by step**

Replace the symbols by regular expressions on the state transition

130
Proof Idea Given DFA, covert it into regular expression

131
**Proof Idea Add start state s and accept state a**

Convert DFA by removing those old state one by one Replace the symbols by regular expressions on the state transition

132
Example States transition 1 2

133
Example States transition 1 2 s a

134
Example States transition 1 s a

135
Example States transition s a

136
Regular Operations For and for and

137
Remove a state Convert left to right

138
Verify The old and new machines accept the same language

139
**Construct Regular expression**

Add one start state and one accept state:

140
Pumping Lemma Lemma: If A is regular language, there is a number p such that if s is in A and of length at least p, s may be divided into s=xyz, satisfying 1) for each 2) 3)

141
Some notations For a string s, is the length of s (the number of letters) For example, |adb|=3, |a|=1, |afdsaf|=6 For a string s and integer i, is a string to repeats i times For example, if s=dgh then

142
Proof Idea a q9 q1 q13

143
**Analysis Run the input string on a automata**

Input string: s1 s2 s3 s4 s5 s6 … sn State q1 q2 q3 q9 q5 q9… q13 Let p be the number of states If n>p, two of the states must be equal, say q9 Repeat the substring between the two q9s reach the same accept state q13

144
**Proof Let p be the number of the states in the Automata M for A**

Let (n>=p) be a string in A Let be the state transition sequence There are two equal states (j<k) Let x takes M from r1 to rj, y takes M from rj to rj z takes M from rj to r(n+1) M must accepts for all

145
**Application of Pumping Lemma**

is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

146
Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

147
**Application of Pumping Lemma**

The language L={w | w has an equal number of 0s and 1s} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

148
Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. y contains only 0s. By 1) of the pumping, the string is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

149
**Application of Pumping Lemma**

The language {ww | w is a {0,1} string} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

150
Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the first 0s area is more than the number of 0s in the second 0s area. This is a contradiction.

151
**Application of Pumping Lemma**

The language is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

152
Proof Let s= It is in the language L By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 1s. By 1) of the pumping, is also in the language. The number of 1s in the string is at least and at most This is a contradiction since the language L does not contain any string with the number of 1s between and .

153
Problem Prove that is not a regular language.

154
**Two ways for language A language can be accepted by a machine**

A language can be also generated by some rules

155
**Context Free Grammar Rules for generating a language: A0A1 A#**

Variables: A Terminals: 0,1,# Start variable: A Rules: A0A1, A#

156
Conversion AXY X0 YAZ Z1 A#

157
**Context Free Grammar Rules for generating a language: A0A1 AB B#**

Variables: A, B Terminals: 0,1,# Start variable: A Rules: A0A1, AB, B#

158
Derivation A0A10B10#1 A0A100A11000A111000B111000#111

159
**Context free grammar A context free grammar is**

V is a finite set called the variables is a finite set of terminals R is a finite set of rultes is the start variable

160
**Language of the grammar**

If Aw is a rule, u and v are strings, then , called uAv yields uwv if for some strings The language of the grammar is

161
**Context Free Grammar Rules for generating a language: A0A1 AB**

Variables: A, B Terminals: 0,1, Start variable: A The language of this grammar is

162
Convert DFA to grammar Automata accepts all strings that end in 1 A B

163
**Grammar A0A A1B B1B B0A (because B represents the accept state)**

Start variable A, Variables A,B Terminals: 0,1,

164
**DFA to grammar conversion**

For every transition , add a rule For every accept state , add rule

165
Theorem Every regular language can be generated by a grammar with the rules like AaB, and

166
Chomsky normal form A context-free grammar is in chomsky normal form if every rule is of the form ABC Aa where “a” is a terminal and A,B,and C are variables. In addition, we permit if S is the start variable.

167
Removing empty Every context free grammar G has another equivalent context free grammar G’, which has no rule for any non-start variable A

168
**Proof Add a new start symbol and rule**

For any rule that A is not a start symbol, 1) replace every RuAv by Ruv, every RuAvAw by Ruvw, etc. 2) replace RA by Repeat the steps above until with non-start variable A does not appear

169
Remove Unit Rule Every context free grammar G has another equivalent context free grammar G’, which has no rule

170
**Proof For every A B, replace every Bu by Au**

Repeat the step above until there is no AB

171
Chomsky Normal Form Every context free grammar G has an equivalent chomsky normal form

172
**Proof Remove the for non-start symbol A Remove the unit rule AB**

For each at k>2, replace it by rules

173
Proof For each at k>1, replace each terminal by the new symbol and add new rule

174
Example A0A1 Conversion:

175
Remove the empty

176
Remove the empty

179
**Arithmetic expression**

Grammar

180
Example Generate

181
**Generate the expression**

Steps

182
**Arithmetic expression**

Grammar Start variable:<EXPR> Variable: <EXPR> Terminals: a,+,x,(,)

183
Example Generate

184
Example Generate

185
Leftmost derivation A derivation of a string w is leftmost derivation if the left most variable is replaced at every step <EXPR><EXPR> x <EXPR> <EXPR>+<EXPR> x <EXPR> a+<EXPR> x <EXPR> a+a x <EXPR> a+a x a

186
Ambiguity A string w is derived ambiguously in context-free grammar G if it has more than one leftmost derivation Grammar G is ambiguous if it generates some string ambiguously Ambiguity gives different interpretations by computer program

187
Example Design the context free grammar for

188
**Grammar Partition the problem into two parts. One part is for**

The second part is Let and S1 is used to get via S2 is used to get via

189
Pushdown Automata State control aaaabbbbbabaaabb a b

190
**Example The grammar generates the language**

It can be accepted by pushdown automata

191
**Remove input 0, push 0 into stack**

State control

192
**Remove input 0, push 0 into stack**

State control

193
**Match stack 0 with input 1. Then remove both of them**

State control

194
**Match stack 0 with input 1. Then remove both of them, accept**

State control

195
**Pushdown Automata Let M be is the set of states is the input alphabet**

is the stack alphabet is the state transition function is the start state, is the set of accept states q2 q1 q3 q4

196
**States Explanation Q1 is the start state Q2 is used to push 0 symbol**

Q3 is used to match 0,1 pairs between stack and input tape Q4 is the accept state

197
Transition a,bc: Used when the current input symbol a and the stack top symbol b, Remove the top symbol on the stack and push c on it The input is moved to the next after a

198
**Difference between Pushdown and finite state automata**

Pushdown automata has unlimited memory, which is last in, first out. Nondeterministic finite state automata is a special case of pushdown automata that has no memory

199
**Pushdown Automata 6-tuple Q is the finite set of states**

is the input alphabet (finite) is the stack alphabet (finite) is the start state, is the set of accept states

200
**A computation of pushdown automata**

Input string: State transition sequence: For i=0,…,m-1, where and Final state accepts

201
Theorem Every context free language can be accepted by a non-deterministic pushdown automata

202
**Example The grammar generates the language**

It generates the string 0011

203
Pushdown Automata State control

204
Replace S by 0S1 at stack State control

205
**Match the stack top symbol with the input symbol, remove both if matched**

State control

206
Replace S by 0S1 State control

207
Remove after matching State control

208
Replace S by empty State control

209
Remove after matching State control

210
**Remove after matching, Accept**

State control

211
**Proof Idea Push $ and start symbol to the stack in the beginning.**

Repeat the three steps below Replace the top variable A on the stack with a the right side of a rule Match the top terminal on the stack with the input symbol, reject if not matched When stack has $ on the top and all input has been read, accepts

212
**How to replace the variable**

If the current stack has the top element s, replace it with the right of the rule State q a…………….. s .

213
**New States The states are new states, specially added for the rule**

State moves from q to r after the variable is replaced on the top of the stack

214
Pumping Lemma Lemma: If A is context-free language, there is a number p such that if s is in A and of length at least p, s may be divided into s=uvxyz, satisfying 1) for each 2) 3)

215
Proof Idea Two variables are the same on a path

216
Proof Idea A path from root to a leaf has all variables except the last one, which is a terminal When the path is too long, same variable has to happen twice. Repeat the part of the two equal variables area.

217
**Analysis Let V be the set of variables in a grammar.**

|V| is the number of variables in V If a path has at least |V|+1 variables, two of them will be equal. A path from root to a leaf has least |V|+1 variables if its length is at least |V|+2 A tree of depth |V|+2 has leaves, where b is the maximal length of right side among all rules.

218
**Proof Let If s has length , the parse tree T has height**

It has a path has at least |V|+1 variables. Two of the variables on the path are equal (=R). We have So,

219
**Proof The tree T is the parsing tree for s and has least size**

It is impossible that is empty otherwise, T is not least. Select the bottom |V|+2 symbols on the longest path of T, this makes the

220
**Application The language is not context free**

Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

221
**Proof The language is not context free Proof. Let**

By Pumping lemma, s can be expressed as s=uvxyz s. t And Case 1. v and y are in the area of the same symbol, say a. Contradiction for having more a than b. Case 2. v or y contains more than one symbol, contradiction for the incorrect order of symbols in

222
**Regular Pumping The language is not regular**

Proof. Assume it is a regular language. It can be accepted by automata M. Let p be the length of the pumping lemma. Consider s can be expressed s=xyz, where The string xz has the number of 0s no more than 1s (Pump it down to get the contradiction)

223
**Application The language is not context free**

Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

224
**Proof The language is not context free Proof. Let**

By Pumping lemma, s can be expressed as s=uvxyz s. t And Case 1. vy have intersection with the a area. Contradiction for having more a or b than c (pumping up). Case 2. vy does not contain symbol a, contradiction for having more a than b or c (pumping down).

225
**Application The language is not context free**

Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

226
**Proof The language is not context free Proof. Let**

By Pumping lemma, s can be expressed as s=uvxyz s. t And Case 1. vy is in the first half. Contradiction for moving 1 to the right area(pumping up). Case 2. vy is in the second half. Contradiction for moving 0 to the left area(pumping up).

227
**Proof The language is not context free**

Proof. Case 3. vy crosses the middle line. Contradiction for reducing 0 or 1 in the middle area (pumping down). The left half and right half have different number of 1s or 0s.

228
Problem Write a context-free grammar for the following language

229
Problem Using pumping lemma to disprove the following language is context-free:

230
Algorithm An algorithm is a collection of simple instructions for carrying out some task

231
Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

232
**Unlimited Register Machine**

Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m Each register can save an unbounded integer

233
**Turing Machine Proposed in 1936**

An accurate model for the general purpose computer.

234
**Turing Machine Write on the tape and read from it**

Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

235
**Language ww Design Turing machine for L={w#w|w is in {0,1}*}**

For example: 011#011 is in L 10011#10011 is in L

236
Movement on the tape Move the head back and forth to match all pairs

237
Movement on the tape Move the head

238
**Turing Machine 7-tuple Q is the finite set of states**

is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

239
**State transition function**

For

240
**State transition function**

For

241
**Configuration Current state: q7**

Current head position on the tape: 4th cell Current tape content: abab q7 a b a b

242
**Configuration A configuration is represented by**

Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

243
**Configuration Transition**

For

244
**Configuration Transition**

For

245
**Configuration Start configuration: , where w is the input**

Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

246
Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where is the start configuration of M on input w, 2. each yields , and is an accepting configuration

247
**Language recognized by TM**

For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

248
a q1 q3 q2 q4 q5 q7 q6 q9 q8 q11 q10 q12 q13 q_accept q14

249
**Multi-tape Turing Machine**

1 1 Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control a a a b a

250
**Multi-tape Turing Machine**

Transition for multi-tape Turing machine

251
**Simulate Multitape Turing machine**

Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 # 1 1 # a a a # b a #

252
Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

253
**Another simulation Assume there are k tapes**

Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 a b 1 a a a

254
Simulate 3 tapes q7 a b 1 a a a

255
Simulate 3 tapes q7 a b 1 a a a

256
K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j-1)k+2i-1 and 2(j-1)k+2i

257
Midterm October 18, 2010 Class Time Close book Chapter 0-Chapter 3

258
Problem 5 Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains an equal number of 0s and 1s}.

259
**Turing Machine Write on the tape and read from it**

Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

260
Language L Design Turing machine for L={w|w is in {0,1}* and the same number 1s as the same number of 0s} For example: is in L is not in L

261
Movement on the tape Move the head back and forth to pair up 0 and 1.

262
Movement on the tape Move the head

263
**Turing Machine Write on the tape and read from it**

Head can move left and right Tape is infinite Rejecting and accepting states Control 1 1

264
Language L Design Turing machine for L={w|w is in {0,1}* and the same number 0s is two times the number of 1s} For example: is in L is not in L

265
Movement on the tape Move the head back and forth to pair up 0 and 1.

266
Movement on the tape Move the head

267
**Problem 6 Problem 6 (10) If A and B are languages, define**

and Prove that if A and B are regular languages, then is a context free language.

268
Problem 6 DFA N1 accepts A DFA N2 accepts B

269
Construct N N accepts

270
**Problem 7 Problem 7 (10) If A and B are languages, define**

and and |x|=|y|} Prove that if A and B are regular languages, then is a context free language. You will get 10 more points for midterm.

271
**Deterministic Turing Machine**

7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

272
**Nondeterministic Turing Machine**

7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

273
**Nondeterministic Turing Machine**

One configuration can have multiple choice for entering the next configuration because of

274
Non-determinism blind monkey

275
Automata example q2 q1 q3 q3 q4 q2

277
Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

278
Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

279
Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

280
**Simulate NTM with DTM Each dot is one configuration**

Find an accepting path

281
Simulate NTM Think about the problem from the Java programming point of view Convert the strategy into deterministic Turing machine.

282
Simulate NTM Key points: search accepting path with width first until it finds the first one. Search from the left to right Search from small level 0, to level 1, to level 2,…

283
**Simulate NTM 1: first branch, 2: second branch, etc**

1 1 1: first branch, 2: second branch, etc 121: take the first branch at level 2, then take the second at level 3, then take the first branch at level 4. Control a a a 1 2 1

284
**Turing machine and Computer**

The computational power of Turing machine is equivalent to regular computer with unlimited memory

285
Successor Funtion f(x)=x+1

286
Successor Funtion f(x)=x+1

287
**Successor Turing machine is able to simulate the operation xx+1 1 1 1**

1 1 1 1 1

288
Comparison Turing machine is able to check if x=y 1 1 1 1 1 1 1 1 1 1

289
Transfer Turing machine is able to simulate yx 1 1 1 1 1

290
Transfer Turing machine is able to simulate xy 1 1 1 1 1 1 1 1 1 1

291
Set zero Turing machine is able to simulate x0 1 1 1 1 1

292
**Program statements Statement1: xx+1 Statement2: xy Statement3: x0**

Statement4: if (x==y) jump m

293
**Compute Algorithm: Let x be added by 1 y times Program for x+y: 1: z0**

2: If (z==y) jump 6 3: xx+1 4: zz+1 5: If (z==z) jump 2 6:

294
**Jump TM program state Program q1 1: I1 q2 2: I2 q3 3: I3 …. ……**

… …… qk k: jump 3 The jump can be achieved via state transition qkq3

295
**Compute Algorithm: Let x be added by original x y times**

Program for x*y: 1: z0 2: p0 3: If (z==y) jump 7 4: pp+x 5: zz+1 6: If (z==z) jump 3 7:

296
**Compute Algorithm: Let x be multiplied by x y times Program for :**

1: z0 2: p (via p0 and pp+1) 3: If (z==y) jump 7 4: pp*x 5: zz+1 6: If (z==z) jump 3

297
Algorithm An algorithm is a collection of simple instructions for carrying out some task

298
Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

299
**Unlimited Register Machine**

Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m Each register can save an unbounded integer

300
**Other computation model**

Lambda calculus (Church) Recursion (Godel and Kleene) Post and Markov’s model

301
**Evidences Many approaches led to the same algorithmic computable class**

No one has found an algorithm that is accepted in informal sense, but it can not be implemented in Turing model

302
Hilbert 10th Problem Find an algorithm to decide if a polynomial has integer root Input: a polynomial e.x. (it has root x=5,y=3,z=0) Output: yes or no

303
Midterm Problem 1 (20) Give the state diagram of a DFA recognizing {w|w is 0,1-string with at least five 0s}. Problem 2 (20) Prove that is not regular language with the pumping lemma.

304
**Midterm Problem 3 (20) Prove the following facts:**

a) If A and B are context free languages, then so is their union . b) If A and B are context free languages, then so is their concatenation.

305
Midterm Problem 4 (20) a) Design a context-free grammar to recognize the language b) Prove that is not a context-free language by using pumping lemma.

306
Midterm Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains more 0s than 1s}.

307
Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

308
**DFA ={<B,w>| B is a DFA that accepts input string w}**

Theorem: is a decidable language.

309
**Proof The input is <B,w>, where B is a DFA , and w is a string.**

Simulate B on the input w: Start from the state of B and leftmost symbol w. Follow to transit the state and move the input symbols one by one Accepts if ends at an accept state; rejects otherwise.

310
**NFA ={<B,w>| B is a NFA that accepts input string w}**

Theorem: is a decidable language.

311
**Proof The input is <B,w>, where B is a NFA and w is a string.**

Convert B into a DFA C via our previous algorithm Use our last TM to decide if C accepts w Accepts if it accepts; rejects otherwise.

312
**={<A>| L(A) is not empty}**

Theorem: is a decidable language.

313
**Proof The input is <A>, where A is a DFA. Mark the start state**

Repeat Mark a new state that has a transition to it from a marked one. Until no new state can be added Accept if an accept state is marked; otherwise reject

314
**={<A,B>| A and B are DFA and L(A)=L(B)}**

Theorem: is a decidable language.

315
**Proof The input is <A,B>, where A and B are DFA Check if**

is empty

316
Problem Define L={ <A,B,C>: L(A) is the union of L(B) and L(C)}, where A, B, and C are DFAs. Prove that L is decidable.

317
**One- one and onto Let A and B be two sets.**

For function f: AB, if whenever then f is called one-one. For function f: A B, say f is onto if f hits every element of B(In other words, for very b in B, there is a in A such that b=f(a))

318
**Correspondence Let A and B be two sets.**

A and B are of the same size if there is a one-one and onto function f: AB For function f: AB, if it is both one-one and onto, then f is called correspondence.

319
**Examples {1,2,3,…} and {2,4,6,…} are of the same size via f(x)=2x.**

320
Countable A set is countable if it is finite or it has the same size as N={1,2,3,…} Theorem: The positive rational numbers set is coutable

321
Proof Every positive rational number is in the table below

322
Proof List all of them by and avoid repetition

323
Examples For two positive rational numbers , ( p and q have no common divisor>1, and p’ and q’ have no common divisor >1), The number is listed before if p+q<p’+q’, or p+q=p’+q’ and q<q’

324
(0,1) is not countable Theorem: (0,1) is not countable

325
**Proof. Proof by contradiction.**

Assume N={1,2,3,…} and (0,1) have the same size. There is one-one and onto map f: N (0,1)

326
Proof. Select such that and Since

327
**Infinite binary strings set is not countable**

Let be the set of all infinite binary strings Theorem: is not countable

328
**Proof. Proof by contradiction.**

Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

329
Proof. Select such that and Since

330
**The set of finite binary strings is countable**

Correspondence N= {0, 1 , 2, 3, 4, 5, 6, 7, , , 10,…}

331
**Language of Binary Strings**

Every set A of binary strings is countable Proof. Let A be a set of binary strings. Its elements can be listed according to their orders in A={ s1, s2, s3, s4, ….} It is easy to see the correspondence between N and A

332
**An infinite binary string uniquely determines a language**

The language with positive even number of 1s strings

333
Correspondence For each infinite binary string B, it uniquely determines a binary language L(B). If B1 and B2 are different binary strings, then L(B1) and L(B2) are different language Each binary language uniquely determines an infinite binary string.

334
Correspondence There is a correspondence between the set of all infinite binary strings and the set of all binary languages. The set of all binary languages is not countable

335
**Turing machine to binary string**

Each Turing machine can be encoded into a binary string. The set of all Turing machines can be encoded into a set of binary strings. The set of Turing machines is countable. The set of binary languages recognized by TM is also countable

336
**Un-computable language by TM**

There is a binary language that is not Turing recognizable Proof. The set of binary languages is not countable, but the set of language recognized by Turing machine is countable

337
Undecidable Problem ={<M,w>| M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

338
**Every entry of the table can be obtained in finite steps**

accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

339
**Proof Assume that is decidable There is a TM H such that**

H(<M,w>) accepts if M accepts w H(<M,w>) rejects if M rejects w. Consider a TM D D(<M>) accepts if M rejects <M> D(<M>) rejects if M accepts <M>

340
**Proof Since D(<M>) accepts if M rejects <M>**

D(<M>) rejects if M accepts <M> We have D(<D>) accepts if D rejects <D> D(<D>) rejects if D accepts <D> A contradiction

341
**Every entry of the table can be obtained in finite steps**

accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

342
**Every diagonal entry of the table can be obtained in finite steps**

accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

343
Diagonal method reject reject accept accept accept reject ……

344
**Call H in R If TM H exists, the TM R also exits via using H**

(Software R uses an existing software H)

345
Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

346
Problem Is there any one-one and onto map from the set of integers in [1,10] and the set of odd integers in [1,10]? Why? Prove that there is a one-one and onto map from the set of all integers and the set of all odd integers.

347
Reduction Solution for Problem 1 Solution for Problem 2 Help

348
**Example: Map and Direction**

349
Reduction Problem 2 Solution for Problem 1 Question Answer

350
Reduction Software for Problem 2: Software for Problem 1 call Return

351
**Reduction & Undecidability**

It is known that P2 is undecidable We want to prove P1 is undecidable Proof by contradiction Assume P1 is decidable. There is a software solving P1 Design a software for P2 by calling the software for P1 Contradiction!!!!

352
Halting Problem P1: ={<M,w>| M is a Turing machine and M halts on input w} P2: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

353
**P2 Input <M,w> If M does not stop on w, reject it**

If M stops, simulate M. If M acceps, accpets Otherwise

354
**Halting Problem ={<M,w>| M is a Turing machine and M halts on**

input w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

355
Undecidable Problem ={<M,w>| M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

356
**Proof Assume the Turing machine R decides We can use R to decides**

For input <M, w> Run R on <M,w> if R rejects, “reject” if R accepts, simulate M until it stops if M accepts, “accept” else “reject”

357
**Empty Problem ={<M>| M is a Turing machine and }**

Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

358
**Proof Assume the Turing machine R decides**

For input <M,w>, design another TM If , reject simulate M on input w, accepts if M accepts w

359
Proof If M accepts w, then w belongs to L( ) Otherwise, L( ) is empty

360
**Proof Use R to decide Input <M,w> Make from <M,w>**

Run R on the input If R rejects (it means L( ) is not empty), accepts I R accepts (it means L( ) is empty), rejects

361
**Problem ={<M>| M is a Turing machine and L(M) is regular}**

Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

362
**Proof Assume the Turing machine R decides**

For input <M,w>, design another TM (x) If x has format , accept simulate M on input w, accepts if M accepts x

363
Proof If M accepts w, then Otherwise,

364
**Proof Use R to decide Input <M,w> Make from <M,w>**

Run R on the input If R rejects (it means L( ) is not regular), accepts I R accepts (it means L( ) is regular), rejects

365
**Turing machine equivalence Problem**

={ | M1 and M2 are Turing machines and L(M1)=L(M2) } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

366
**Proof For input <M,w> for TM M1: rejects any input**

TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

367
Mapping Reducibility Convert instances of problem to instances of problem B A B

368
**Computable function Assume that is a set of finite number of symbols**

is the set of all finite strings with symbols from The function is computable if some Turing machine M, on every input w, halts with just f(w) on the tape. Example: +,x,/ are all computable functions

369
Mapping Reducibility Language A is mapping reducible to language B, if there is a computable function such that for every w

370
Example A={1,3,5,….} B={0,2,4,…} f(x)=x+1 via the funtion f

371
Theorem If , and B is decidable, then A is also decidable A B

372
**Proof Let f be the reduction from A to B since**

Let M be the decider for B. Decider N: Input w, Compute f(w) Run M on f(w) and accept iff M accepts

373
Corollary If , and A is undecidable, then B is also undecidable A B

374
Proof Proof by contradiction. If B is decidable, then so is A

375
Halting Problem P1: ={<M,w>| M is a Turing machine and M halts on input w} P2: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

376
A reduction from to B: ={<M,w>| M is a Turing machine and M halts on input w} A: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

377
**Reduction On input <M,w> Construct Turing machine M’ Run M on w**

If M accepts, accept If M rejects, enter an infinite loop Output <M’,w>

378
**Empty Problem ={<M>| M is a Turing machine and }**

Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

379
**Reduction from to For <M,w>, design another TM M’ Input x**

If x is not equal to w, reject else simulate M on input w, accepts if M accepts w

380
**Reduction from to <M,w> is in L(M’) is not empty**

<M,w> is not in L(M’) is empty

381
**Reduction from to <M,w> is in M’ is in**

<M,w> is not in M’ is not in

382
Problem Let ={M| M is a Turing machine and M prints 1 for some input w} Show that L is undecidable.

383
**Turing Machine Write on the tape and read from it**

Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

384
**Turing Machine 7-tuple Q is the finite set of states**

is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

385
**Configuration Current state: q7**

Current head position on the tape: 4th cell Current tape content: abab q7 a b a b

386
**Configuration A configuration is represented by**

Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

387
**Configuration Transition**

For

388
**Configuration Transition**

For

389
**Configuration Start configuration: , where w is the input**

Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

390
Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where is the start configuration of M on input w, 2. each yields , and is an accepting configuration

391
**Language recognized by TM**

For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

392
Turing Recognizable Turing machine M recognizes language L

393
Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

394
Turing Decidable Turing machine M decides language L

395
Observation If L is Turing decidable, then L is Turing recognizable

396
**A Turing recognizable problem**

={<M,w>| M is a Turing machine and M accepts w} Turing machine R recognizable For input <M,w> Simulate M on w R accepts <M,w> if M accepts w.

397
Theorem Theorem: If L is a language in {0,1}*, there is a Turing machine to print out all elements in L

398
**Proof Let M recognize L Step i(i=1,2,3…)**

Simulate M each of the first i strings i steps If a string is accepted, print it out.

399
Theorem If , and B is Turing recognizable, so is A A B

400
**Proof Let f be the reduction from A to B since Let M recognize B.**

TM N: Input w, Compute f(w) Run M on f(w) and accept if M accepts

401
Theorem If , and A is not Turing recognizable, B is not Turing recognizable A B

402
Complement set Set and its complement

403
Theorem If A and are Turing recognizable, then A is decidable

404
**Proof Let Turing machine M1 recognize A**

For input x, Run M1 and M2 on x in parallel If M1 accepts, accept If M2 accepts, reject.

405
**={<A,B>| A and B are DFA and L(A)=L(B)}**

.

406
**={<A,B>| A and B are DFA and L(A)=L(B)}**

Theorem: is not Turing decidable language.

407
. Theorem: Neither nor its complement is Turing recognizable

408
**Proof We first prove For input <M,w> for**

TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

409
**Proof We prove For input <M,w> for TM M1: accepts any input**

TM M2: accepts any input if M accepts w. It is easy to see L(M1)=L(M2) iff M accepts w So,

410
Problem Show that the language F={M| L(M) contains infinite elements} is not Turing recognizable}.

411
Problem 4.2 The equivalence of a DFA and a regular expression is decidable

412
Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

413
Problem 4.7 The set of all infinite binary strings is not countable

414
Problem 4.8 N={1,2,3,…} NxNxN is countable

415
Problem 5.9 All Turing recognizable problems are mapping reducible to

416
Problem 5.12 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

417
Problem 5.13 Test if a Turing machine has useless state, which never enters.

418
Language Design Turing machine to recognize

419
Complexity Let M be a deterministic Turing machine. The running time of M is the function f:N N such that f(n) is the maximum number of steps that M uses on input of length n.

420
Big O-notation Let f and g be functions Say f(n)=O(g(n)) if integers c and exist so that for all

421
Examples Let

422
Small o-notation Let f and g be functions Say f(n)=o(g(n)) if

423
**One tape Turing machine time**

One tape Turing machine can recognize In steps

424
**Two tapes Turing machine time**

Two tapes Turing machine can recognize In O(n) steps

425
**Multi-tape Turing Machine**

1 1 Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control a a a b a

426
**Multi-tape Turing Machine**

Transition for multi-tape Turing machine

427
**Simulate Multitape Turing machine**

Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 # 1 1 # a a a # b a #

428
Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

429
**Another simulation Assume there are k tapes**

Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 a b 1 a a a

430
Simulate 3 tapes q7 a b 1 a a a

431
Simulate 3 tapes q7 a b 1 a a a

432
K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j-1)k+2i-1 and 2(j-1)k+2i

433
**Simulate multi-tape by one tape**

One tape Turing machine can simulate t(n) time multi-tape Turing machine in time

434
**Time Complexity Class Let .**

is the class of languages decided by O(t(n) time Turing machines. In other words is a language decided by O(t(n)) time Turing machine}

435
**One tape Turing machine time**

One tape Turing machine can decide In steps

436
Complexity Class P P is the class of languages decided by single tape Turing machine in polynomial time

437
Directed Path Problem G is a directed graph that has a directed path from s to t} a t c s d b

438
**Algorithm Input mark “s” Repeat**

For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

439
**Greatest common divisor**

Divisor: For two integers b and c, if b=c*z for some integer z, c is a divisor of b. Greatest common divisor: Given two integers a and b, gcd(a,b) is the greatest positive integer c such that c is the divisor for both a and b. Examples: gcd(10,4)=2, gcd(16,100)=4 Problem: How to find gcd(a,b)?

440
**Relatively Prime Problem**

Two integers x and y are relatively prime if gcd(x,y)=1. RELPRIME={<x,y>| x and y are relatively prime} Theorem:

441
**Euclid algorithm For two integer b and a a=q*b+c with**

gcd(a,b)=gcd(b,c) a and b are relatively prime iff b and c are relatively prime

442
**Euclid algorithm Assume a1 and a2 are two positive integers**

If , then and Therefore, always true

443
**Algorithm for gcd(x,y) Input Repeat rx mod y xy yr until y=0**

output x

444
**Modular Assume a and b are two positive integers**

This is a recursive equation since the second item goes down

445
Example Find gcd(1970,1066)

446
**Speed of Euclid algorithm**

Assume a1 and a2 are two positive integers If , we have In another words,

447
Euclid algorithm Assume a1 and a2 are two positive integers

448
Time For input <a,b> The total time of steps is O(log a+log b)

449
Non-determinism blind monkey

450
Automata example q2 q1 q3 q3 q4 q2

452
**Deterministic Turing Machine**

7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

453
**Nondeterministic Turing Machine**

7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

454
**Nondeterministic Turing Machine**

One configuration can have multiple choice for entering the next configuration because of

455
Non-determinism blind monkey

456
Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

457
Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

458
Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

459
**Simulate NTM with DTM Each dot is one configuration**

Find an accepting path

460
Class NP NP is the class of languages that can be recognized by nondeterministic Turing machine in polynomial time

461
**Hamiltonian Path Hamiltonian path goes through each node exactly once**

HAMPATH={<G,s,t>| G is a directed graph with a Hamiltonian path from s to t}

462
Composite A natural number is composite if it is the product of two integers >1 10=2* =2*9 Composite={x| x=pq, for integers p,q>1}

463
**Verifier A verifier for a language L is an algorithm V,**

L={w| V accepts <w,c> for some string c} For the verifier V for L, c is a certificate of w if V accepts <w,c> If the verifier V for the language L runs in polynomial time, V is the polynomial time verifier for L.

464
**Verifier for Hamiltonian Path**

For <G,s,t>, a certificate is a list of nodes of G: Verifier: check if m is the number of nodes of G check if and check if each is a directed edge of G for i=1,…,m-1 If all pass, accept . Otherwise, reject.

465
**Verifier for Composite**

For integer x, a certificate is two integers p,q: Verifier: check if p>1 and q>1 check if x=pq If all pass, accept . Otherwise, reject.

466
Class NP NP is the class of languages that have polynomial time verifiers. Examples: COMPOSITE is in NP HAMPATH is in NP

467
Theorem A language has polynomial verifier iff it can be recognized by polynomial time nondeterministic Turing machine Proof: Assume L has verifier V, … Assume L has NTM M,…

468
Proof Assume that L has polynomial time verifier V, which runs in time, where k is a constant NTM M Nondeterministically select string c of length Run V on <w,c> If V accepts, accept; Otherwise, reject.

469
Proof Assume that L is recognized by polynomial time NTM M, which runs in time, where k is a constant Verifier <w,c> Let c determine a computation path of M Simulate M on the path c If M accepts, accept. Otherwise, reject.

470
Clique Problem Given undirected graph G, a clique is a set of nodes of G such that every two nodes are connected by an edge. A k-clique is a clique with k nodes

471
Clique Problem CLIQUE={<G,k>| G iss an undirected graph with k-clique} CLIQUE is in NP.

472
Subset Sum Problem SUBSET-SUM={<S,t>| S= and for some , we have

473
**Polynomial Time Computable**

A function is a polynomial time computable function if some polynomial time Turing machine M exists that outputs on the tape for input w.

474
**Polynomial Time Reduction**

Assume that A and B are two languages on A is polynomial time mapping reducible to A if a polynomial time computable function exists such that

475
Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

476
SAT A boolean formula is satisfiable if there exists assignments to its variables to make the formula true SAT={ | is satisfiable boolean formula}

477
3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

478
3SAT to CLIQUE Example:

479
**NP-completeness A language B is NP-complete if B is in NP, and**

Every A in NP is polynomial time reducible to B Theorem. If B is NP-complete and B is in P, then P=NP.

480
**Cook-Leving Theorem Theorem: SAT is NP-complete Proof.**

1. SAT is in NP. 2. For every problem A in NP,

481
**Proof The start configuration is legal The final state is accept.**

The movement is legal. Each cell takes one legal symbol.

482
**Proof 1 if The cell[i,j] holds symbol s; 0 otherwise**

Time bound for the NTM M with constant k. The movement is legal. NTM M for accepting A.

483
**Nondeterministic Turing Machine**

Q is the finite set of states is the tape alphabet is the start state, is the accept state.

484
**Configuration Transition**

For

485
**Configuration Transition**

For

486
**Configuration Start configuration: , where w is the input**

Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

487
Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where is the start configuration of M on input w, 2. each yields , and is an accepting configuration

488
**Language recognized by TM**

For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

489
**Proof Each cell has only one symbol The symbol is selected from C:**

Only one symbol is selected: It is true for all cell at all configuration:

490
Proof The start configuration is

491
**Proof Accept computation has reached.**

It makes sure the accept state will appear among the configuration transitions.

492
**Proof Characterize the legal move**

The whole move is legal if all windows are legal. Characterize one window is legal

493
Proof The state transition

494
Logic Demorgan Law:

495
Truth table for y y x2

496
Convert to CNF Conversion:

497
Convert to CNF Conversion:

498
Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

499
**Prepare for the Final Regular language and automata**

Context free language Decidability Undecidability Complexity theory

500
**Regular Language Concepts: Automata, regular expression**

Skills: Design automata to accept a regular language Disprove a language is a regular

501
**Context-free Language**

Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

502
Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

503
**Undecidability Concepts: Countable, Turing undecidable, reduction**

Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

504
**Complexity Concepts: Time on Turing machine PTIME(t(n))**

NP-completeness Polynomial time reduction Polynomial time verifier

505
**Complexity Skill: Prove a problem is in P Prove a problem is in NP**

Use reduction to prove a problem is NP-complete.

506
Grade A:… B:… C: Miss exam or homework

507
**SAT’ A conjunctive normal form is a conjunction of some clauses**

SAT’={ | is satisfiable conjunctive normal form}

508
**Cook-Leving Theorem’ Theorem: SAT’ is NP-complete**

Proof. Same as that for SAT is NP-complete

509
3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

510
3SAT is NP-complete Theorem: There is polynomial time reduction from SAT’ to 3SAT.

511
3SAT is NP-complete is satisfiable if and only if the following is satisfiable

512
3SAT is NP-complete is satisfiable if and only if the following is satisfiable

513
3SAT is NP-complete Convert every clause into 3cnf:

514
**3SAT is NP-complete Conjunctive normal form**

Each clause is convert into is satisfiable if and only if the following is satisfiable

515
Problem Convert the formula F into 3SAT formula F’ such that F is satisfiable iff and F’ is satisfiable.

516
**Approximation Algorithms**

517
**Outline and Reading Approximation Algorithms for NP-Complete Problems**

Approximation ratios Polynomial-Time Approximation Schemes 2-Approximation for Vertex Cover Approximate Scheme for Subset Sum 2-Approximation for TSP special case Log n-Approximation for Set Cover

518
**Approximation Ratios Optimization Problems**

We have some problem instance x that has many feasible “solutions”. We are trying to minimize (or maximize) some cost function c(S) for a “solution” S to x. For example, Finding a minimum spanning tree of a graph Finding a smallest vertex cover of a graph Finding a smallest traveling salesperson tour in a graph

519
**Approximation Ratios An approximation produces a solution T**

T is a k-approximation to the optimal solution OPT if c(T)/c(OPT) < k (assuming a min. prob.; a maximization approximation would be the reverse)

520
**Polynomial-Time Approximation Schemes**

A problem L has a polynomial-time approximation scheme (PTAS) if it has a polynomial-time (1+)-approximation algorithm, for any fixed >0 (this value can appear in the running time). Subset Sum has a PTAS.

521
Vertex Cover A vertex cover of graph G=(V,E) is a subset W of V, such that, for every (a,b) in E, a is in W or b is in W. OPT-VERTEX-COVER: Given an graph G, find a vertex cover of G with smallest size. OPT-VERTEX-COVER is NP-hard.

522
**Copyright © The McGraw-Hill Companies, Inc**

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

523
**Copyright © The McGraw-Hill Companies, Inc**

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

524
**A 2-Approximation for Vertex Cover**

Every chosen edge e has both ends in C But e must be covered by an optimal cover; hence, one end of e must be in OPT Thus, there is at most twice as many vertices in C as in OPT. That is, C is a 2-approx. of OPT Running time: O(m) Algorithm VertexCoverApprox(G) Input graph G Output a vertex cover C for G C empty set H G while H has edges e H.removeEdge(H.anEdge()) v H.origin(e) w H.destination(e) C.add(v) C.add(w) for each f incident to v or w H.removeEdge(f) return C

525
Problem 1 Show that T={(i,j)| both I and j are positive rational numbers} is countable.

526
Problem 2. Find a correspondence between (0,1) and [0,1]

527
Solution Let for n=1, 2, … Define the correspondence f(x) below:

528
Problem 3 3. Let A={<R,S> | R and S are regular expressions and L(R ) is a subset of L( S)}. Show that A is decidable.

529
R S Solution For two regular expression R and S, L( R) is a subset of L(S) if and only if the L(R ) has no intersection with . Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Check if L(M4) is empty

530
Problem 4 I={x| x is an irrational number in (0,1)}. Show that I is not countable.

531
Homework 2009

532
Problem 4.2 The equivalence of a DFA and a regular expression is decidable

533
**Solution Proof. Given a regular language L and DFA M.**

We can construct a DFA N to accept L. The problem is converted into checking if two DFAs are equivalent. By Theorem 4.5, there is an algorithm to decide if L(M)=L(N).

534
Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

535
**Solution Assume M is a DFA with n states.**

L(M) is infinite iff it has path from the start state to accept with a loop on it, and the loop length is no more than n. L(M) is infinite iff M accepts a string of length between n and 2n. For every string x with length between n and 2n Run M on x If M accepts x, then output “L(M) is infinite” else output “no”

536
Problem 4.7 The set of all infinite binary strings is not countable

537
**Proof. Proof by contradiction.**

Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

538
Proof. Select such that and Since

539
Problem 1 N={1,2,3,…} NxNxN is countable

540
Proof List all of them by

541
Proof Use the similar method like the last slide to prove the NxN is countable Let f: NxNN be one-one and onto We have g: NxNxNN is one-one and onto, where g(i,j,k)=f(f(i,j),k)

542
Problem 2. Find a correspondence between (0,1) and [0,1]

543
Solution Let for n=1, 2, … Define the correspondence f(x) below:

544
Problem 3 3. Let A={<R,S> | R and S are regular expression and L(R ) = L( S)}. Show that A is decidable.

545
Solution For two regular expression R and S, L( R)=L(S) if and only if their symmetric difference is empty. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Construct automata M5 to accept Check if L(M5) is empty

546
Problem 4 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

547
**Proof We are going to design a reduction from to S. For <M,w>**

Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts <M,w> is in iff N accepts both 01 and 10.

548
Problem 5 5. Let S={<M> | M is a TM and L(M) = {<M>}}. Show that neither S nor its complement is Turing-recognizable.

549
**Solution We first show that can be reducible to S.**

Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal <Y> then reject If x=<Y> then accepts if M(w) accepts.

550
Solution For an input (M, w), derive M1(x, M1).

551
**Solution We show that can be reducible to . Let (M, w) be an input for**

Let Turing machine M2 be constructed below: M1 Input x If x =<M1> then accept If x is not equal <M1> then accept if M(w) accepts.

552
Solution For an input (M, w), derive M2(x, M2).

553
Problem 6 Show that A is a Turing recognizable if and only if

554
**Proof Part a) Let A be a Turing recognizable language.**

There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =<M,w>. It is easy to see that

555
**Proof Part b) Assume that via computable function g.**

It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

556
**Proof We are going to design a reduction from to it For <M,w>**

Design the TM N as follows Input x if x=0, enters all states except if x=1 run M on w, if M accepts w, enter <M,w> is in iff N enters all states.

557
**Problem 4 c) The language is not a context free language**

Proof: Assume it is a context free language. Let p be the number from the pumping lemma Consider the string

558
Final Exam December 13, 5:45-7:30pm, Monday

559
Homework 4, 20

560
Problem 1 1.Let T1={<M>| M is a Turing machines and accepts infinite number of 0,1-strings of finite length}. Prove that T1 is undecidable.

561
**Proof We are going to design a reduction from to T1.**

For input Turing machine <M,w> for Design the TM N as follows Input x run M on w, if M accepts w, acceptx <M,w> is in iff N acccepts each input x.

562
Problem 2 Let T2={<M>| M is a Turing machines and accepts five of 0,1-strings of finite length}. Prove that T2 is undecidable.

563
**Proof We are going to design a reduction from to T2. For <M,w>**

Design the TM N as follows Input x if x is not 0,1, 00, 01 or 10, rejects. run M on w if M accepts w, accept x <M,w> is in iff N accepts 0,1,00,01, and 10. Otherwise, N accepts no string

564
Problem 3 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

565
**Proof We are going to design a reduction from to S. For <M,w>**

Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts <M,w> is in iff N accepts both 01 and 10.

566
Problem 4 Show that A is a Turing recognizable if and only if

567
**Proof Part a) Let A be a Turing recognizable language.**

There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =<M,w>. It is easy to see that

568
**Proof Part b) Assume that via computable function g.**

It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

569
Problem 5 5. Let S={<M> | M is a TM and L(M) = {<M>}}. Show that neither S nor its complement is Turing-recognizable.

570
**Solution We first show that can be reducible to S.**

Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal <Y> then reject If x=<Y> then accepts if M(w) accepts.

571
Solution For an input (M, w), derive M1(x, M1).

572
**Solution We show that can be reducible to . Let (M, w) be an input for**

Let Turing machine M2 be constructed below: M1 Input x If x =<M1> then accept If x is not equal <M1> then accept if M(w) accepts.

573
Solution For an input (M, w), derive M2(x, M2).

574
Homework 5, 20

575
Problem 1 Call graphs G and H isomorphic if the nodes of G may be reordered so that it is identical to H. Let ISO={<G,H> | G and H are isomorphic graphs}. Show that ISO is in NP.

576
Solution for Problem 1 We design a polynomial time verification algorithm. A mapping f from the vertices of G to those of H is a witness. Check if 1) f is one-one, 2) f is onto, 3) (v,u) is an edge of G iff (f(u),f(v)) is an edge of H. It is easy to see that the verification takes polynomial time.

577
Problem 2 Let MAX-CIQUE={<G, k>| the largest clique of G has k vertices}. Whether MAX-CLIQUE is in NP is unknown. Show that if P=NP, then MAX-CLIQUE is in P, and a polynomial time algorithm exists that, for a graph G, finds one of its largest cliques.

578
Solution of Problem 2 Fist step is to find the largest k with (G,k) is in Clique. Try k from 1,2,… Check if (G,k) is in Clique Select the largest k. Assume the largest k for (G,k) in Clique is obtained.

579
Solution of Problem 2 Assume the largest k for (G,k) in Clique is obtained. Formulate the problem: (H, k,G) Determine if there is a clique of size k in G and contains all veritces in H. The problem is in NP. Extend H one by one until its size reaches k.

580
Problem 3 Let G represent an undirected graph and let SPATH={<G, a,b,k> | G contains a simple path of length at most k from a to b} and LPATH={<G, a,b,k} | G contains a simple path of length at least k from a to b}. Show that SPATH is in P. Show that LPATH is NP-complete.

581
Solution Problem 3 Show that SPATH is in P.

582
Directed Path Problem G is a directed graph that has a directed path from s to t} a t c s d b

583
**Algorithm Input mark “s” Repeat**

For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

584
**Solution Problem 3 Show that LPATH is NP-complete.**

Hamiltonian path problem is NP-complete. There is an easy reduction from Hamiltonian path to it.

585
**Prepare for the Final Regular language and automata**

Context free language Decidability Undecidability Complexity theory

586
**Regular Language Concepts: Automata, regular expression**

Skills: Design automata to accept a regular language Disprove a language is a regular

587
**Context-free Language**

Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

588
Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

589
**Undecidability Concepts: Countable, Turing undecidable, reduction**

Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

590
**Complexity Concepts: Time on Turing machine PTIME(t(n))**

NP-completeness Polynomial time reduction Polynomial time verifier

591
**Complexity Skill: Prove a problem is in P Prove a problem is in NP**

Use reduction to prove a problem is NP-complete.

592
Grade A:… B:… C: Miss exam or homework

593
Space Complexity Complexity ©D.Moshkovits

594
**Motivation Complexity classes correspond to bounds on resources**

One such resource is space: the number of tape cells a TM uses when solving a problem Complexity ©D.Moshkovits

595
**Introduction Objectives: To define space complexity classes Overview:**

Low space classes: L, NL Savitch’s Theorem Immerman’s Theorem TQBF Complexity ©D.Moshkovits

596
**Space Complexity Classes**

For any function f:NN, we define: SPACE(f(n))={ L : L is decidable by a deterministic O(f(n)) space TM} NSPACE(f(n))={ L : L is decidable by a non-deterministic O(f(n)) space TM} Complexity ©D.Moshkovits

597
**Low Space Classes Definitions (logarithmic space classes):**

L = SPACE(logn) NL = NSPACE(logn) Complexity ©D.Moshkovits

598
Problem! How can a TM use only logn space if the input itself takes n cells?! !? Complexity ©D.Moshkovits

599
**3Tape Machines a b _ b _ b a _ input . . . work output . . . . . .**

read-only Only the size of the work tape is counted for complexity purposes read/ write b _ . . . write-only b a _ . . . Complexity ©D.Moshkovits

600
Example Question: How much space would a TM that decides {anbn | n>0} require? Note: to count up to n, we need logn bits Complexity ©D.Moshkovits

601
**Graph Connectivity CONN**

An undirected version is also worth considering CONN Instance: a directed graph G=(V,E) and two vertices s,tV Problem: To decide if there is a path from s to t in G? Complexity ©D.Moshkovits

602
Graph Connectivity t s Complexity ©D.Moshkovits

603
**CONN is in NL Start at s For i = 1, .., |V| {**

Non-deterministically choose a neighbor and jump to it Accept if you get to t } If you got here – reject! Counting up to |V| requires log|V| space Storing the current position requires log|V| space Complexity ©D.Moshkovits

604
Configurations Which objects determine the configuration of a TM of the new type? The content of the work tape The machine’s state The head position on the input tape The head position on the work tape The head position on the output tape If the TM uses logarithmic space, there are polynomially many configurations Complexity ©D.Moshkovits

605
**Log-Space Reductions Definition:**

A is log-space reducible to B, written ALB, if there exists a log space TM M that, given input w, outputs f(w) s.t. wA iff f(w)B the reduction Complexity ©D.Moshkovits

606
**Do Log-Space Reductions Imply what they should?**

Suppose A1 ≤L A2 and A2L; how to construct a log space TM which decides A1? Wrong Solution: w Too Large! f(w) Use the TM for A2 to decide if f(w)A2 Complexity ©D.Moshkovits

607
**Log-Space reductions Claim: if Then, A1 is in L**

A1 ≤L A2 – f is the log-space reduction A2 L – M is a log-space machine for A2 Then, A1 is in L Proof: on input x, in or not-in A1: Simulate M and whenever M reads the ith symbol of its input tape run f on x and wait for the ith bit to be outputted Complexity ©D.Moshkovits

608
**NL Completeness Definition: A language B is NL-Complete if BNL**

For every ANL, ALB. If (2) holds, B is NL-hard Complexity ©D.Moshkovits

609
**Savitch’s Theorem Theorem: S(n) ≥ log(n) NSPACE(S(n)) SPACE(S(n)2)**

Proof: First we’ll prove NLSPACE(log2n) then, show this implies the general case Complexity ©D.Moshkovits

610
**Savitch’s Theorem Theorem: NSPACE(logn) SPACE(log2n) Proof:**

First prove CONN is NL-complete (under log-space reductions) Then show an algorithm for CONN that uses log2n space Complexity ©D.Moshkovits

611
**CONN is NL-Complete Theorem: CONN is NL-Complete**

Proof: by the following reduction: s L t “Is there a path from s to t?” “Does M accept x?” Complexity ©D.Moshkovits

612
**Technicality Observation:**

Without loss of generality, we can assume all NTM’s have exactly one accepting configuration. Complexity ©D.Moshkovits

613
Configurations Graph A Computation of a NTM M on an input x can be described by a graph GM,x: A vertex per configuration the start configuration s t the accepting configuration (u,v)E if M can move from u to v in one step Complexity ©D.Moshkovits

614
Correctness Claim: For every non-deterministic log-space Turing machine M and every input x, M accepts x iff there is a path from s to t in GM,x Complexity ©D.Moshkovits

615
**CONN is NL-Complete Corollary: CONN is NL-Complete**

Proof: We’ve shown CONN is in NL. We’ve also presented a reduction from any NL language to CONN which is computable in log space (Why?) Complexity ©D.Moshkovits

616
**A Byproduct Claim: NLP Proof:**

Any NL language is log-space reducible to CONN Thus, any NL language is poly-time reducible to CONN CONN is in P Thus any NL language is in P. Complexity ©D.Moshkovits

617
What Next? We need to show CONN can be decided by a deterministic TM in O(log2n) space. Complexity ©D.Moshkovits

618
**The Trick “Is there a path from u to v of length d?”**

“Is there a vertex z, so there is a path from u to z of size d/2 and one from z to v of size d/2?” d/2 d/2 u z v d Complexity ©D.Moshkovits

619
**Recycling Space The two recursive invocations can use the same space**

Complexity ©D.Moshkovits

620
**The Algorithm Boolean PATH(a,b,d) {**

if there is an edge from a to b then return TRUE else { if d=1 return FALSE for every vertex v { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE Complexity ©D.Moshkovits

621
**Example of Savitch’s algorithm**

boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE 2 3 1 4 (a,b,c)=Is there a path from a to b, that takes no more than c steps. (1,4,3)(1,3,2)(2,3,1)TRUE (1,4,3)(1,2,2)TRUE (1,4,3)(1,2,2) (1,4,3)(2,4,1) (1,4,3)(2,4,1)FALSE (1,4,3)(1,3,2)TRUE (1,4,3) (1,4,3) TRUE (1,4,3)(3,4,1)TRUE (1,4,3)(1,3,2)(1,2,1) (1,4,3)(1,3,2)(2,3,1) (1,4,3)(3,4,1) (1,4,3)(1,3,2) (1,4,3)(1,3,2)(1,2,1)TRUE Complexity ©D.Moshkovits 3Log2(d)

622
O(log2n) Space DTM Claim: There is a deterministic TM which decides CONN in O(log2n) space. Proof: To solve CONN, we invoke PATH(s,t,|V|) The space complexity: S(n)=S(n/2)+O(logn)=O(log2n) Complexity ©D.Moshkovits

623
**Conclusion Theorem: NSPACE(logn) SPACE(log2n)**

How about the general case NSPACE(S(n))SPACE(S2(n))? Complexity ©D.Moshkovits

624
**The Padding Argument Motivation: Scaling-Up Complexity Claims**

We have: can be simulated by… space space + non-determinism + determinism We want: can be simulated by… space space + non-determinism + determinism Complexity ©D.Moshkovits

625
**Formally NSPACE(s1(f(n))) SPACE(s2(f(n)))**

si(n) can be computed with space si(n) Claim: For any two space constructible functions s1(n),s2(n)logn, f(n)n: NSPACE(s1(n)) SPACE(s2(n)) NSPACE(s1(f(n))) SPACE(s2(f(n))) simulation overhead E.g NSPACE(n)SPACE(n2) NSPACE(n2)SPACE(n4) Complexity ©D.Moshkovits

626
**Idea NTM DTM n n . . . f(n) . space: s1(.) in the size of its input**

space: O(s2(f(n))) NTM n n . . . space: O(s1(f(n))) f(n) . Complexity ©D.Moshkovits

627
**Padding argument Let LNPSPACE(s1(f(n))) There is a 3-Tape-NTM ML: |x|**

Input babba Work O(s1(f(|x|))) Complexity ©D.Moshkovits

628
**Padding argument Let L’ = { x0f(|x|)-|x| | xL }**

We’ll show a NTM ML’ which decides L’ in the same number of cells as ML. f(|x|) babba# Input Work O(s1(f(|x|)) Complexity ©D.Moshkovits

629
**Padding argument – ML’ In O(log(f(|x|)) space**

Count backwards the number of 0’s and check there are f(|x|)-|x| such. 2. Run ML on x. in O(s1(f(|x|))) space f(|x|) Input babba Work O(s1(f(|x|))) Complexity ©D.Moshkovits

630
**Padding argument Total space: O(s1(f(|x|))) f(|x|) Input**

babba Work O(s1(f(|x|))) Complexity ©D.Moshkovits

631
**Padding Argument We started with LNSPACE(s1(f(n)))**

We showed: L’NSPACE(s1(n)) Thus, L’SPACE(s2(n)) Using the DTM for L’ we’ll construct a DTM for L, which will work in O(s2(f(n))) space. Complexity ©D.Moshkovits

632
Padding Argument The DTM for L will simulate the DTM for L’ when working on its input concatenated with zeros Input babba Complexity ©D.Moshkovits

633
Padding Argument When the input head leaves the input part, just pretend it encounters 0s. maintaining the simulated position (on the imaginary part of the tape) takes O(log(f(|x|))) space. Thus our machine uses O(s2(f(|x|))) space. NSPACE(s1(f(n)))SPACE(s2(f(n))) Complexity ©D.Moshkovits

634
**Savitch: Generalized Version**

Theorem (Savitch): S(n) ≥ log(n) NSPACE(S(n)) SPACE(S(n)2) Proof: We proved NLSPACE(log2n). The theorem follows from the padding argument. Complexity ©D.Moshkovits

635
**Corollary Corollary: PSPACE = NPSPACE Proof: Clearly, PSPACENPSPACE.**

By Savitch’s theorem, NPSPACEPSPACE. Complexity ©D.Moshkovits

636
Space Vs. Time We’ve seen space complexity probably doesn’t resemble time complexity: Non-determinism doesn’t decrease the space complexity drastically (Savitch’s theorem). We’ll next see another difference: Non-deterministic space complexity classes are closed under completion (Immerman’s theorem). Complexity ©D.Moshkovits

637
**NON-CONN NON-CONN Instance: A directed graph G and two vertices s,tV.**

Problem: To decide if there is no path from s to t. Complexity ©D.Moshkovits

638
**NON-CONN Clearly, NON-CONN is coNL-Complete.**

(Because CONN is NL-Complete. See the coNP lecture) If we’ll show it is also in NL, then NL=coNL. (Again, see the coNP lecture) Complexity ©D.Moshkovits

639
**An Algorithm for NON-CONN**

We’ll see a log space algorithm for counting reachability Count how many vertices are reachable from s. Take out t and count again. Accept if the two numbers are the same. Complexity ©D.Moshkovits

640
**N.D. Algorithm for reachs(v, l)**

1. length = l; u = s 2. while (length > 0) { 3. if u = v return ‘YES’ 4. else, for all (u’ V) { 5. if (u, u’) E nondeterministic switch: 5.1 u = u’; --length; break 5.2 continue } } 6. return ‘NO’ Takes up logarithmic space This N.D. algorithm might never stop Complexity ©D.Moshkovits

641
**N.D. Algorithm for CRs CRs ( d ) 1. count = 0 2. for all uV {**

3. countd-1 = 0 4. for all vV { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++countd-1 else fail if (v,u) E then ++count; break 5.2 continue } 6. if countd-1 < CRs (d-1) fail 7.return count Assume (v,v) E Recursive call! Complexity ©D.Moshkovits

642
**N.D. Algorithm for CRs parameter , C) CRs ( d 1. count = 0**

2. for all uV { 3. countd-1 = 0 4. for all vV { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++countd-1 else fail if (v,u) E then ++count; break 5.2 continue } 6. if countd-1 < fail 7.return count Main Algorithm: CRs C = 1 for d = 1..|V| C = CR(d, C) return C C parameter Complexity ©D.Moshkovits

643
**Efficiency Lemma: The algorithm uses O(log(n)) space. Proof:**

There is a constant number of variables ( d, count, u, v, countd-1). Each requires O(log(n)) space (range |V|). Complexity ©D.Moshkovits

644
**Immerman’s Theorem Theorem[Immerman/Szelepcsenyi]: NL=coNL Proof:**

(1) NON-CONN is NL-Complete (2) NON-CONNNL Hence, NL=coNL. Complexity ©D.Moshkovits

645
**Corollary Corollary: s(n)log(n), NSPACE(s(n))=coNSPACE(s(n))**

Proof: By a padding argument. Complexity ©D.Moshkovits

646
TQBF We can use the insight of Savich’s proof to show a language which is complete for PSPACE. We present TQBF, which is the quantified version of SAT. Complexity ©D.Moshkovits

647
**TQBF Instance: a fully quantified Boolean formula **

Problem: to decide if is true Example: a fully quantified Boolean formula xyz[(xyz)(xy)] Variables` range is {0,1} Complexity ©D.Moshkovits

648
**TQBF is in PSPACE Theorem: TQBFPSPACE**

Proof: We’ll describe a poly-space algorithm A for evaluating : If has no quantifiers: evaluate it If =x((x)) call A on (0) and on (1); Accept if both are true. If =x((x)) call A on (0) and on (1); Accept if either is true. in poly time Complexity ©D.Moshkovits

649
**Algorithm for TQBF 1 1 1 1 1 xy[(xy)(xy)] y[(0y)(0y)]**

(00)(00) (01)(01) (10)(10) (11)(11) 1 1 Complexity ©D.Moshkovits

650
**Efficiency Since both recursive calls use the same space,**

the total space needed is polynomial in the number of variables (the depth of the recursion) TQBF is polynomial-space decidable Complexity ©D.Moshkovits

651
**PSAPCE Completeness Definition: A language B is PSPACE-Complete if**

BPSPACE For every APSAPCE, APB. standard Karp reduction If (2) holds, then B is PSPACE-hard Complexity ©D.Moshkovits

652
**TQBF is PSPACE-Complete**

Theorem: TQBF is PSAPCE-Complete Proof: It remains to show TQBF is PSAPCE-hard: P x1x2x3…[…] “Will the poly-space M accept x?” “Is the formula true?” Complexity ©D.Moshkovits

653
TQBF is PSPACE-Hard Given a TM M for a language L PSPACE, and an input x, let fM,x(u, v), for any two configurations u and v, be the function evaluating to TRUE iff M on input x moves from configuration u to configuration v fM,x(u, v) is efficiently computable Complexity ©D.Moshkovits

654
**Formulating Connectivity**

The following formula, over variables u,vV and path’s length d, is TRUE iff G has a path from u to v of length ≤d (u,v,1) fM,x(u, v) u=v (u,v,d) wxy[((x=uy=w)(x=wy=v))(x,y,d/2)] w is reachable from u in d/2 steps. v is reachable from w in d/2 steps. simulates AND of (u,w,d/2) and (w,v,d/2) Complexity ©D.Moshkovits

655
**TQBF is PSPACE-Complete**

Claim: TQBF is PSPACE-Complete Proof: (s,t,|V|) is TRUE iff there is a path from s to t. is constructible in poly-time. Thus, any PSPACE language is poly-time reducible to TQBF, i.e – TQBF is PSAPCE-hard. Since TQBFPSPACE, it’s PSAPCE-Complete Complexity ©D.Moshkovits

656
Summary We introduced a new way to classify problems: according to the space needed for their computation. We defined several complexity classes: L, NL, PSPACE. Complexity ©D.Moshkovits

657
** Summary Our main results were: Connectivity is NL-Complete**

TQBF is PSPACE-Complete Savitch’s theorem (NLSPACE(log2)) The padding argument (extending results for space complexity) Immerman’s theorem (NL=coNL) By reducing decidability to reachability Complexity ©D.Moshkovits

Ppt on solar system for class 8 Ppt on event handling in javascript all objects Ppt on various types of web browsers and their features Ppt on computer languages list Ppt on intelligent manufacturing Ppt on different occupations in art Ppt on bill gates as a leader Ppt on carbon and its compounds elements Signal generator and display ppt on tv Download ppt on basic concepts of chemistry