Theoretical Foundation of Computation Instructor: Bin Fu Textbook: Introduction to the theory of computation, by Michael Siper Class Time: 5:45-8:25pm Tuesday

Contents Computational models and languages finite automata, push automata regular languages, context free languages Computability theory decidable problem, un-decidable problem Complexity theory time, space, P, NP, PSpace

Theory of Computation What is the computation? What problems are computable by computer in finite steps? What problems are computable by small number of steps?

Mathematical Model Real computers are very complicate Develop simple mathematical model to define computation The mathematical models are “equivalent to” real computer under some transformation

Un-computable problems Important Boundary Computability theory boundary Un-computable problems Computable problems

Hierarchy inside computable area

Basic Concepts Set: a set is a group of objects represented as a unit. {7,21,57} Element: A object of a set is called an element. Subset: A is a subset of B if every member of A is also a member of B.

Some sets Natural numbers set: N={1,2,3,…} Integers set Z={…, -2,-1,0,1,2,…} Empty set: it is a set with no elements

Set operations Union: A={2,4,9} B={1,2,5} ={1,2,4,5,9} Intersection: ={2}

Tuple Sequence: a list of objects in some order (7, 21, 57) Tuple: finite sequence (7, 21,57) Cartesian product: A x B is the set of all pairs with first element from A and second element from B A={1,2} B={x,y,z} A x B= {(1,x),(1,y), (1,z), (2,x), (2,y), (2,z) }

Power Set Let A be a set. The power set of A is the set of subsets of A. For A={a,b}, its power set P(A) is { , {a}, {b}, {a,b}}

Cartesian product

Function A function f is mapping from one set D to another set R f: DR for every a in the set D, there is another element b in R such that a is mapped to b by f f(a)=b Domain: D Range: R

Function example Function f: {0,1,2,3,4}{0,1,2,3,4} n f(n) 0 1 1 2 0 1 1 2 2 3 3 4 4 0

Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

Relation For two sets A and B, a binary relation R is a subset AxB Example: A={scissor, paper, stone} scissor beats paper paper beats stone stone beats scissor R= {(scissor, paper), (paper, stone), (stone, scissor)}

Represent binary relation If (a,b) in the binary relation R, we often use aRb to represent them. Example A={1,2,3}. The relation = is the set {(1,1),(2,2),(3,3)} 1=1 2=2 3=3

Equivalence relation A binary relation R is equivalence relation if Reflexive: xRx for every x Symmetric: if xRy, then yRx Transitive: if xRy and yRz, then xRz Example 1: = on {1,2,3}x{1,2,3} Example 2: = on NxN

Example for equivalence Relation For two integers x and y, x y if (x-y) is a multiple of 7 In other words, there is another integer z such that (x-y)=7z.

Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ }

Path Graph G=(V,E) A path is a series of edges linked one by one Loop:

Tree A graph is connected if every two nodes have a path to connect them A tree is a connected graph without loop

Connected Graph Tree Every connected graph can be converted into tree by removing some edges Removing one edge on a loop does not damage the connectivity.

A tree is a minimal connected graph Removing any edge on a tree damages the connectivity Proof. Tree T=(V,E). Let (v1, v2) be removed from T. T  T’=(V, E-{(v1,v2)}). If T’ is still connected, T has a loop containing v1 and v2 . Contradiction!

Graph Graph: A set of nodes V A set of edges E from V x V V={ } E={ }

Node degree: The number of edges connecting to the node v2 v1 v3 v4

Mathematical approach Definition Mathematical statement Express some object with certain property Theorem A statement proved to be true Proof

Mathematical proof Convincing logical argument that a statement is true Usually consists a series logical statements There is a small logical gap between the current logic statement with previous statements.

3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

Sum of node degrees For every graph, the sum of the degrees of all nodes in G is an even number Sum=2+2+2=6 sum=2+3+4+3+2=14

Proof Let v1, v2, …, vn are the n nodes of the graph. deg(vi) is the degree of node vi The sum of node degrees is sum=deg(v1)+deg(v2)+…+deg(vn) For each edge e=(vi, vj), it makes one contribution to both deg(vi) and deg(vj). If there are k edges, the sum is 2k.

Regular graph A graph is k-regular if every node has degree equal to k Theorem: For each even number n>2, there exists a 3-regular graph with n nodes.

Proof by construction n/2 nodes Let every point at top half connect to a point in the bottom half

Proof Let be the n nodes of the graph. Add edges for i=0, 1, …,(n/2)-1 , and Add edges for i=0,1, …,(n/2)-1

Proof by contradiction Assume the theorem is false Lead to an obviously false consequence Example: Jill just came in from outdoor and is complete dry Try to Prove: No rain Proof: Assume it were raining Jill would be wet. A contradiction!

No 3-regular graph with odd nodes Theorem: There is no 3-regular graph with odd number of nodes Proof (by contradiction) If the graph G is a 3-regular graph with 2m+1 nodes, where m is an integer at least 0. The sum of the degrees of nodes is sum=3+3+….+3=3(2m+1)=6m+3=2(3m+1)+1 It is a contradiction to our previous theorem.

3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

Proof by induction Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

Example for induction Theorem: For every natural number n,

Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume

Proof

Computation models Finite automata Pushdown automata Turing machine

3 Styles of Mathematical Proofs Proof by construction Proof by contradiction Proof by induction

Proof by induction Try to prove the statement P(k) is true for all nature numbers k in N={1, 2, 3,…} Basis: Prove P(1) is true Induction step: Prove if P(i) is true, then so is P(i+1)

Example for induction Theorem: For every natural number n,

Proof Basis: when n=1. The left side is 1 The right is Induction step: Assume

Proof

Computation models Finite automata Pushdown automata Turing machine

Finite automata Supermarket entrance Front: A person is coming from the front. Rear: …… Both: front and rear Neither: neither front nor rear closed open

Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

Automata example q1 q3 q2

Acceptance Any sequence with at least one 1 and even number of 0s following the last 1

Notions for automata start state: q1 accept state: q2. In other words, F={q2} The set The transition function is 0 1 q1 q1 q2 q2 q3 q2 q3 q2 q2 Q={q1,q2,q3}

Run the automata Start from the start state Follow the state transition based on the current state and symbol accepts if it enters accept state, rejects otherwise

Run the machine at inputs 1 q1q2 01 q1q1q2 0101010101 q1q2q3q2q3q2q3q2q3q2 0 q1q2 10 q1q2q3 101000 q1q2q3q2q3q3q3

Language and Machine Let A be a set of strings. Let M be a machine If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

Example for language acceptance A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

Formal definition of computation A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

Automata What is the language accepted by the automata?

Answer L(M)={w| w ends in a 1}

Automata What is the language accepted by the automata?

Answer L(M)={w| w is an empty string or ends in a 0}

Automata What is the language accepted by the automata?

Answer L(M)={w| w starts and ends at the same symbol}

Designing Automata Language A consists all {0,1} strings with even number of 1s. Problem: design an automata M with L(M)=A.

Automata States:

Designing Automata Language A consists all {0,1} strings with 001 as substring. Problem: design an automata M with L(M)=A.

Automata States:

Regular Language M accepts language A if A={w| M accepts w} A language is regular if some finite automata accepts it. Example: A1={w| w is {0,1} string and ends in a 1} A2={w| w is a {a,b} string that starts and ends with the same symbol}

Language and Machine Let A be a set of strings. Let M be a machine If A is the set of strings that M accepts, A is the language of M, denoted by L(M)=A We also say “M recognizes A” or “M accepts A”

Example for language acceptance A={ w| w has at least one 1 and an even number of 0s follow the last 1} Let M be the automata example with 3 states A=L(M)

Formal definition of computation A finite automata String w= be a string on the alphabet M accepts w if there is a sequence of states with following conditions: (1) (2) (3)

Non-determinism blind monkey

Symbol It represents the empty symbol. If used , one state moves to the next without consuming any symbol q1 q2

Automata example q2 q1 q3 q3 q4 q2

Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

Language recognized Let A be a set of strings that contain either 101 or 11 as a substring. E.G. 001010, 10111

Automata example q2 q1 q3 q3 q4 q2

Language recognized Let A be a set of strings containing a 1 in the third position from the end E.G. 00100

Automata example q2 q3 q1 q3 q2 q3

Problem: What language does it accept? q2 q3 q2 q1 q2 q5

Language recognized Accept all strings , where k is a multiple of 2 and 3.

Some notations For a set Q, P(Q) is the collection of all subsets of Q Example, Q={q1,q2} P(Q)={empty, {q1}, {q2}, {q1,q2}} For alphabet , write

Formal definition of automata A finite automata has Q is a finite set called the states is a finite set called the alphabet is the transition function is the start state, and is the set of accept states

Equivalence between NFA and DFA Theorem: Every NFA has an equivalent DFA Proof. Given an NFA We will construct an to accept the same language.

Proof Q’=P(Q), which is the set of all subsets of Q The transition function or and for some The start state F’={R in Q’| R has accept state in N}

Example Given NFA, convert it into DFA 2 1

Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 1 2 1,2

Example Let Q={1,2}. P(Q)={empty, {1},{2}, {1,2}} 2 1 1,2

Problem: What language does it accept? q2 q3 q2 q1 q2 q5

Language Operations Let A and B be two languages Union: Concatenation: Star:

Closure under Union Theorem: If A and B are regular languages, then is also regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

Construct N N1: N2:

Construct N N accepts iff one of N1 and N2 accepts

Closure under Catenation Theorem: If A and B are regular languages, then is also a regular language Proof. Let A be accepted by finite automata N1, and B be accepted by finite automata N2 Find another finite automata N to accept

Construct N N1: N2:

Construct N N2 is linked to the accept state of N1

Closure under Concatenation Theorem: If A is regular language, then is also a regular language Proof. Let A be accepted by finite automata N1, Find another finite automata N to accept

Construct N N1:

Construct N N1:

Exercise 1. Given NFA, convert it into DFA 2. Let A be the language recognized by the NFA. Design the automata to recognize A* 1 2

Regular Operations R is a regular expression if R is A for some a in The special string The empty set ,where R1 and R2 are regular expressions , where R1 and R2 are regular expressions ,where R1 is a regular expression

Regular Operations Examples 0*10* 3) 4) 5) 6)

Regular  Automata Theorem: Every regular expression is regular language Proof. Let R be a regular expression. Construct a DFA to accept R

6 Cases a ,where R1 and R2 are regular expressions ,where R1 is a regular expression

Case 1 a

Case 2

Case 3

Case 4 DFA N1 accepts R1 DFA N2 accepts R2

Construct N N accepts

Case 5 DFA N1 accepts R1 DFA N2 accepts R2

Construct N N accepts

Construct N N1 accepts R

Construct N N accepts R*

Example

Regular Operations R is a regular expression if R is A for some a in The special string The empty set ,where R1 and R2 are regular expressions , where R1 and R2 are regular expressions ,where R1 is a regular expression

Regular Operations Examples 0*10* 3) 4) 5) 6)

Automata  Regular expression If a language is regular, then it is described by a regular expression Proof. Let A be recognized by a DFA A, find a regular expression R for A.

Some observation States transition 1 2 3

Some observation States transition 1 3

Some observation States transition 1 2 3

Some observations States transition 1 3

Some observation States transition 1 2 3

Some observations States transition 1 3

Some observations States transition 1 3

Proof Idea Convert DFA by shrinking it step by step Replace the symbols by regular expressions on the state transition

Proof Idea Given DFA, covert it into regular expression

Proof Idea Add start state s and accept state a Convert DFA by removing those old state one by one Replace the symbols by regular expressions on the state transition

Example States transition 1 2

Example States transition 1 2 s a

Example States transition 1 s a

Example States transition s a

Regular Operations For and for and

Remove a state Convert left to right

Verify The old and new machines accept the same language

Construct Regular expression Add one start state and one accept state:

Pumping Lemma Lemma: If A is regular language, there is a number p such that if s is in A and of length at least p, s may be divided into s=xyz, satisfying 1) for each 2) 3)

Some notations For a string s, is the length of s (the number of letters) For example, |adb|=3, |a|=1, |afdsaf|=6 For a string s and integer i, is a string to repeats i times For example, if s=dgh then

Proof Idea a q9 q1 q13

Analysis Run the input string on a automata Input string: s1 s2 s3 s4 s5 s6 … sn State q1 q2 q3 q9 q5 q9… q13 Let p be the number of states If n>p, two of the states must be equal, say q9 Repeat the substring between the two q9s reach the same accept state q13

Proof Let p be the number of the states in the Automata M for A Let (n>=p) be a string in A Let be the state transition sequence There are two equal states (j<k) Let x takes M from r1 to rj, y takes M from rj to rj z takes M from rj to r(n+1) M must accepts for all

Application of Pumping Lemma is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

Proof Let s= . It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

Application of Pumping Lemma The language L={w | w has an equal number of 0s and 1s} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

Proof Let s= . It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. y contains only 0s. By 1) of the pumping, the string is also in the language L. The number of 0s in the string is more than the number of 1s. This is a contradiction.

Application of Pumping Lemma The language {ww | w is a {0,1} string} is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

Proof Let s= . It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 0s. By 1) of the pumping, is also in the language L. The number of 0s in the first 0s area is more than the number of 0s in the second 0s area. This is a contradiction.

Application of Pumping Lemma The language is not a regular language Proof: Assume it is a regular language. There is a automata M to accept it. Let p be the number from the pumping lemma Consider the string

Proof Let s= . It is in the language L. By the pumping lemma, s can be divided into s=xyz that satisfies 1), 2) and 3) in the pumping lemma. By 3), |xy| is at most p. So, y contains only 1s. By 1) of the pumping, is also in the language. The number of 1s in the string is at least and at most . This is a contradiction since the language L does not contain any string with the number of 1s between and .

Problem Prove that is not a regular language.

Two ways for language A language can be accepted by a machine A language can be also generated by some rules

Context Free Grammar Rules for generating a language: A0A1 A# Variables: A Terminals: 0,1,# Start variable: A Rules: A0A1, A#

Conversion AXY X0 YAZ Z1 A#

Context Free Grammar Rules for generating a language: A0A1 AB B# Variables: A, B Terminals: 0,1,# Start variable: A Rules: A0A1, AB, B#

Derivation A0A10B10#1 A0A100A11000A111000B111000#111

Context free grammar A context free grammar is V is a finite set called the variables is a finite set of terminals R is a finite set of rultes is the start variable

Language of the grammar If Aw is a rule, u and v are strings, then , called uAv yields uwv if for some strings The language of the grammar is

Context Free Grammar Rules for generating a language: A0A1 AB Variables: A, B Terminals: 0,1, Start variable: A The language of this grammar is

Convert DFA to grammar Automata accepts all strings that end in 1 A B

Grammar A0A A1B B1B B0A (because B represents the accept state) Start variable A, Variables A,B Terminals: 0,1,

DFA to grammar conversion For every transition , add a rule For every accept state , add rule

Theorem Every regular language can be generated by a grammar with the rules like AaB, and

Chomsky normal form A context-free grammar is in chomsky normal form if every rule is of the form ABC Aa where “a” is a terminal and A,B,and C are variables. In addition, we permit if S is the start variable.

Removing empty Every context free grammar G has another equivalent context free grammar G’, which has no rule for any non-start variable A

Proof Add a new start symbol and rule For any rule that A is not a start symbol, 1) replace every RuAv by Ruv, every RuAvAw by Ruvw, etc. 2) replace RA by Repeat the steps above until with non-start variable A does not appear

Remove Unit Rule Every context free grammar G has another equivalent context free grammar G’, which has no rule

Proof For every A B, replace every Bu by Au Repeat the step above until there is no AB

Chomsky Normal Form Every context free grammar G has an equivalent chomsky normal form

Proof Remove the for non-start symbol A Remove the unit rule AB For each at k>2, replace it by rules

Proof For each at k>1, replace each terminal by the new symbol and add new rule

Example A0A1 Conversion:

Remove the empty

Remove the empty

Arithmetic expression Grammar

Example Generate

Generate the expression Steps

Arithmetic expression Grammar Start variable:<EXPR> Variable: <EXPR> Terminals: a,+,x,(,)

Example Generate

Example Generate

Leftmost derivation A derivation of a string w is leftmost derivation if the left most variable is replaced at every step <EXPR><EXPR> x <EXPR> <EXPR>+<EXPR> x <EXPR> a+<EXPR> x <EXPR> a+a x <EXPR> a+a x a

Ambiguity A string w is derived ambiguously in context-free grammar G if it has more than one leftmost derivation Grammar G is ambiguous if it generates some string ambiguously Ambiguity gives different interpretations by computer program

Example Design the context free grammar for

Grammar Partition the problem into two parts. One part is for The second part is Let and S1 is used to get via S2 is used to get via

Pushdown Automata State control aaaabbbbbabaaabb a b

Example The grammar generates the language It can be accepted by pushdown automata

Remove input 0, push 0 into stack State control

Remove input 0, push 0 into stack State control

Match stack 0 with input 1. Then remove both of them State control

Match stack 0 with input 1. Then remove both of them, accept State control

Pushdown Automata Let M be is the set of states is the input alphabet is the stack alphabet is the state transition function is the start state, is the set of accept states q2 q1 q3 q4

States Explanation Q1 is the start state Q2 is used to push 0 symbol Q3 is used to match 0,1 pairs between stack and input tape Q4 is the accept state

Transition a,bc: Used when the current input symbol a and the stack top symbol b, Remove the top symbol on the stack and push c on it The input is moved to the next after a

Difference between Pushdown and finite state automata Pushdown automata has unlimited memory, which is last in, first out. Nondeterministic finite state automata is a special case of pushdown automata that has no memory

Pushdown Automata 6-tuple Q is the finite set of states is the input alphabet (finite) is the stack alphabet (finite) is the start state, is the set of accept states

A computation of pushdown automata Input string: State transition sequence: For i=0,…,m-1, where and Final state accepts

Theorem Every context free language can be accepted by a non-deterministic pushdown automata

Example The grammar generates the language It generates the string 0011

Pushdown Automata State control

Replace S by 0S1 at stack State control

Match the stack top symbol with the input symbol, remove both if matched State control

Replace S by 0S1 State control

Remove after matching State control

Replace S by empty State control

Remove after matching State control

Remove after matching, Accept State control

Proof Idea Push $ and start symbol to the stack in the beginning. Repeat the three steps below Replace the top variable A on the stack with a the right side of a rule Match the top terminal on the stack with the input symbol, reject if not matched When stack has $ on the top and all input has been read, accepts

How to replace the variable If the current stack has the top element s, replace it with the right of the rule State q a…………….. s .

New States The states are new states, specially added for the rule State moves from q to r after the variable is replaced on the top of the stack

Pumping Lemma Lemma: If A is context-free language, there is a number p such that if s is in A and of length at least p, s may be divided into s=uvxyz, satisfying 1) for each 2) 3)

Proof Idea Two variables are the same on a path

Proof Idea A path from root to a leaf has all variables except the last one, which is a terminal When the path is too long, same variable has to happen twice. Repeat the part of the two equal variables area.

Analysis Let V be the set of variables in a grammar. |V| is the number of variables in V If a path has at least |V|+1 variables, two of them will be equal. A path from root to a leaf has least |V|+1 variables if its length is at least |V|+2 A tree of depth |V|+2 has leaves, where b is the maximal length of right side among all rules.

Proof Let If s has length , the parse tree T has height It has a path has at least |V|+1 variables. Two of the variables on the path are equal (=R). We have So,

Proof The tree T is the parsing tree for s and has least size It is impossible that is empty otherwise, T is not least. Select the bottom |V|+2 symbols on the longest path of T, this makes the

Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. v and y are in the area of the same symbol, say a. Contradiction for having more a than b. Case 2. v or y contains more than one symbol, contradiction for the incorrect order of symbols in

Regular Pumping The language is not regular Proof. Assume it is a regular language. It can be accepted by automata M. Let p be the length of the pumping lemma. Consider s can be expressed s=xyz, where The string xz has the number of 0s no more than 1s (Pump it down to get the contradiction)

Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy have intersection with the a area. Contradiction for having more a or b than c (pumping up). Case 2. vy does not contain symbol a, contradiction for having more a than b or c (pumping down).

Application The language is not context free Proof. Assume it is a context free language. It can be generated by a context free grammar G. Apply pumping lemma, to derive some strings not in the language can be also derived by G.

Proof The language is not context free Proof. Let By Pumping lemma, s can be expressed as s=uvxyz s. t. And Case 1. vy is in the first half. Contradiction for moving 1 to the right area(pumping up). Case 2. vy is in the second half. Contradiction for moving 0 to the left area(pumping up).

Proof The language is not context free Proof. Case 3. vy crosses the middle line. Contradiction for reducing 0 or 1 in the middle area (pumping down). The left half and right half have different number of 1s or 0s.

Problem Write a context-free grammar for the following language

Problem Using pumping lemma to disprove the following language is context-free:

Algorithm An algorithm is a collection of simple instructions for carrying out some task

Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

Unlimited Register Machine Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m Each register can save an unbounded integer

Turing Machine Proposed in 1936 An accurate model for the general purpose computer.

Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

Language ww Design Turing machine for L={w#w|w is in {0,1}*} For example: 011#011 is in L 10011#10011 is in L

Movement on the tape Move the head back and forth to match all pairs

Movement on the tape Move the head

Turing Machine 7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

State transition function For

State transition function For

Configuration Current state: q7 Current head position on the tape: 4th cell Current tape content: abab q7 a b a b

Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

Configuration Transition For

Configuration Transition For

Configuration Start configuration: , where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields , and 3. is an accepting configuration

Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

a q1 q3 q2 q4 q5 q7 q6 q9 q8 q11 q10 q12 q13 q_accept q14

Multi-tape Turing Machine 1 1 Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control a a a b a

Multi-tape Turing Machine Transition for multi-tape Turing machine

Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 # 1 1 # a a a # b a #

Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 a b 1 a a a

Simulate 3 tapes q7 a b 1 a a a

Simulate 3 tapes q7 a b 1 a a a

K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j-1)k+2i-1 and 2(j-1)k+2i

Midterm October 18, 2010 Class Time Close book Chapter 0-Chapter 3

Problem 5 Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains an equal number of 0s and 1s}.

Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

Language L Design Turing machine for L={w|w is in {0,1}* and the same number 1s as the same number of 0s} For example: 110010 is in L 1111000 is not in L

Movement on the tape Move the head back and forth to pair up 0 and 1.

Movement on the tape Move the head

Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control 1 1

Language L Design Turing machine for L={w|w is in {0,1}* and the same number 0s is two times the number of 1s} For example: 100010 is in L 11000 is not in L

Movement on the tape Move the head back and forth to pair up 0 and 1.

Movement on the tape Move the head

Problem 6 Problem 6 (10) If A and B are languages, define and . Prove that if A and B are regular languages, then is a context free language.

Problem 6 DFA N1 accepts A DFA N2 accepts B

Construct N N accepts

Problem 7 Problem 7 (10) If A and B are languages, define and and |x|=|y|} . Prove that if A and B are regular languages, then is a context free language. You will get 10 more points for midterm.

Deterministic Turing Machine 7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

Nondeterministic Turing Machine 7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

Nondeterministic Turing Machine One configuration can have multiple choice for entering the next configuration because of

Non-determinism blind monkey

Automata example q2 q1 q3 q3 q4 q2

Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

Simulate NTM with DTM Each dot is one configuration Find an accepting path

Simulate NTM Think about the problem from the Java programming point of view Convert the strategy into deterministic Turing machine.

Simulate NTM Key points: search accepting path with width first until it finds the first one. Search from the left to right Search from small level 0, to level 1, to level 2,…

Simulate NTM 1: first branch, 2: second branch, etc 1 1 1: first branch, 2: second branch, etc 121: take the first branch at level 2, then take the second at level 3, then take the first branch at level 4. Control a a a 1 2 1

Turing machine and Computer The computational power of Turing machine is equivalent to regular computer with unlimited memory

Successor Funtion f(x)=x+1

Successor Funtion f(x)=x+1

Successor Turing machine is able to simulate the operation xx+1 1 1 1 1 1 1 1 1

Comparison Turing machine is able to check if x=y 1 1 1 1 1 1 1 1 1 1

Transfer Turing machine is able to simulate yx 1 1 1 1 1

Transfer Turing machine is able to simulate xy 1 1 1 1 1 1 1 1 1 1

Set zero Turing machine is able to simulate x0 1 1 1 1 1

Program statements Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m

Compute Algorithm: Let x be added by 1 y times Program for x+y: 1: z0 2: If (z==y) jump 6 3: xx+1 4: zz+1 5: If (z==z) jump 2 6:

Jump TM  program state Program q1 1: I1 q2 2: I2 q3 3: I3 …. …… …. …… qk k: jump 3 The jump can be achieved via state transition qkq3

Compute Algorithm: Let x be added by original x y times Program for x*y: 1: z0 2: p0 3: If (z==y) jump 7 4: pp+x 5: zz+1 6: If (z==z) jump 3 7:

Compute Algorithm: Let x be multiplied by x y times Program for : 1: z0 2: p1 (via p0 and pp+1) 3: If (z==y) jump 7 4: pp*x 5: zz+1 6: If (z==z) jump 3

Algorithm An algorithm is a collection of simple instructions for carrying out some task

Church-Turing Thesis Intuitive Algorithm is equal to Turing machine algorithms

Unlimited Register Machine Statement1: xx+1 Statement2: xy Statement3: x0 Statement4: if (x==y) jump m Each register can save an unbounded integer

Other computation model Lambda calculus (Church) Recursion (Godel and Kleene) Post and Markov’s model

Evidences Many approaches led to the same algorithmic computable class No one has found an algorithm that is accepted in informal sense, but it can not be implemented in Turing model

Hilbert 10th Problem Find an algorithm to decide if a polynomial has integer root Input: a polynomial e.x. (it has root x=5,y=3,z=0) Output: yes or no

Midterm Problem 1 (20) Give the state diagram of a DFA recognizing {w|w is 0,1-string with at least five 0s}. Problem 2 (20) Prove that is not regular language with the pumping lemma.

Midterm Problem 3 (20) Prove the following facts: a) If A and B are context free languages, then so is their union . b) If A and B are context free languages, then so is their concatenation.

Midterm Problem 4 (20) a) Design a context-free grammar to recognize the language . b) Prove that is not a context-free language by using pumping lemma.

Midterm Problem 5 (20) Give a description of a Turing machine that decides the following language over the alphabet {0,1} {w|w contains more 0s than 1s}.

Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

DFA ={<B,w>| B is a DFA that accepts input string w} Theorem: is a decidable language.

Proof The input is <B,w>, where B is a DFA , and w is a string. Simulate B on the input w: Start from the state of B and leftmost symbol w. Follow to transit the state and move the input symbols one by one Accepts if ends at an accept state; rejects otherwise.

NFA ={<B,w>| B is a NFA that accepts input string w} Theorem: is a decidable language.

Proof The input is <B,w>, where B is a NFA and w is a string. Convert B into a DFA C via our previous algorithm Use our last TM to decide if C accepts w Accepts if it accepts; rejects otherwise.

={<A>| L(A) is not empty} Theorem: is a decidable language.

Proof The input is <A>, where A is a DFA. Mark the start state Repeat Mark a new state that has a transition to it from a marked one. Until no new state can be added Accept if an accept state is marked; otherwise reject

={<A,B>| A and B are DFA and L(A)=L(B)} Theorem: is a decidable language.

Proof The input is <A,B>, where A and B are DFA Check if is empty

Problem Define L={ <A,B,C>: L(A) is the union of L(B) and L(C)}, where A, B, and C are DFAs. Prove that L is decidable.

One- one and onto Let A and B be two sets. For function f: AB, if whenever then f is called one-one. For function f: A B, say f is onto if f hits every element of B(In other words, for very b in B, there is a in A such that b=f(a))

Correspondence Let A and B be two sets. A and B are of the same size if there is a one-one and onto function f: AB For function f: AB, if it is both one-one and onto, then f is called correspondence.

Examples {1,2,3,…} and {2,4,6,…} are of the same size via f(x)=2x.

Countable A set is countable if it is finite or it has the same size as N={1,2,3,…} Theorem: The positive rational numbers set is coutable

Proof Every positive rational number is in the table below

Proof List all of them by and avoid repetition

Examples For two positive rational numbers , ( p and q have no common divisor>1, and p’ and q’ have no common divisor >1), The number is listed before if p+q<p’+q’, or p+q=p’+q’ and q<q’

(0,1) is not countable Theorem: (0,1) is not countable

Proof. Proof by contradiction. Assume N={1,2,3,…} and (0,1) have the same size. There is one-one and onto map f: N (0,1)

Proof. Select such that and Since

Infinite binary strings set is not countable Let be the set of all infinite binary strings Theorem: is not countable

Proof. Proof by contradiction. Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

Proof. Select such that and Since

The set of finite binary strings is countable Correspondence N= {0, 1 , 2, 3, 4, 5, 6, 7, 8, 9, 10,…}

Language of Binary Strings Every set A of binary strings is countable Proof. Let A be a set of binary strings. Its elements can be listed according to their orders in A={ s1, s2, s3, s4, ….} It is easy to see the correspondence between N and A

An infinite binary string uniquely determines a language The language with positive even number of 1s strings

Correspondence For each infinite binary string B, it uniquely determines a binary language L(B). If B1 and B2 are different binary strings, then L(B1) and L(B2) are different language Each binary language uniquely determines an infinite binary string.

Correspondence There is a correspondence between the set of all infinite binary strings and the set of all binary languages. The set of all binary languages is not countable

Turing machine to binary string Each Turing machine can be encoded into a binary string. The set of all Turing machines can be encoded into a set of binary strings. The set of Turing machines is countable. The set of binary languages recognized by TM is also countable

Un-computable language by TM There is a binary language that is not Turing recognizable Proof. The set of binary languages is not countable, but the set of language recognized by Turing machine is countable

Undecidable Problem ={<M,w>| M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Every entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

Proof Assume that is decidable There is a TM H such that H(<M,w>) accepts if M accepts w H(<M,w>) rejects if M rejects w. Consider a TM D D(<M>) accepts if M rejects <M> D(<M>) rejects if M accepts <M>

Proof Since D(<M>) accepts if M rejects <M> D(<M>) rejects if M accepts <M> We have D(<D>) accepts if D rejects <D> D(<D>) rejects if D accepts <D> A contradiction

Every entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

Every diagonal entry of the table can be obtained in finite steps accept reject accept reject accept accept accept accept reject reject reject reject accept accept reject reject ……

Diagonal method reject reject accept accept accept reject ……

Call H in R If TM H exists, the TM R also exits via using H (Software R uses an existing software H)

Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

Problem Is there any one-one and onto map from the set of integers in [1,10] and the set of odd integers in [1,10]? Why? Prove that there is a one-one and onto map from the set of all integers and the set of all odd integers.

Reduction Solution for Problem 1 Solution for Problem 2 Help

Example: Map and Direction

Reduction Problem 2 Solution for Problem 1 Question Answer

Reduction Software for Problem 2: Software for Problem 1 call Return

Reduction & Undecidability It is known that P2 is undecidable We want to prove P1 is undecidable Proof by contradiction Assume P1 is decidable. There is a software solving P1 Design a software for P2 by calling the software for P1 Contradiction!!!!

Halting Problem P1: ={<M,w>| M is a Turing machine and M halts on input w} P2: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

P2 Input <M,w> If M does not stop on w, reject it If M stops, simulate M. If M acceps, accpets Otherwise

Halting Problem ={<M,w>| M is a Turing machine and M halts on input w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Undecidable Problem ={<M,w>| M is a Turing machine and M accepts w} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Proof Assume the Turing machine R decides We can use R to decides For input <M, w> Run R on <M,w> if R rejects, “reject” if R accepts, simulate M until it stops if M accepts, “accept” else “reject”

Empty Problem ={<M>| M is a Turing machine and } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Proof Assume the Turing machine R decides For input <M,w>, design another TM If , reject simulate M on input w, accepts if M accepts w

Proof If M accepts w, then w belongs to L( ) Otherwise, L( ) is empty

Proof Use R to decide Input <M,w> Make from <M,w> Run R on the input If R rejects (it means L( ) is not empty), accepts I R accepts (it means L( ) is empty), rejects

Problem ={<M>| M is a Turing machine and L(M) is regular} Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Proof Assume the Turing machine R decides For input <M,w>, design another TM (x) If x has format , accept simulate M on input w, accepts if M accepts x

Proof If M accepts w, then Otherwise,

Proof Use R to decide Input <M,w> Make from <M,w> Run R on the input If R rejects (it means L( ) is not regular), accepts I R accepts (it means L( ) is regular), rejects

Turing machine equivalence Problem ={ | M1 and M2 are Turing machines and L(M1)=L(M2) } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Proof For input <M,w> for TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

Mapping Reducibility Convert instances of problem to instances of problem B A B

Computable function Assume that is a set of finite number of symbols is the set of all finite strings with symbols from The function is computable if some Turing machine M, on every input w, halts with just f(w) on the tape. Example: +,x,/ are all computable functions

Mapping Reducibility Language A is mapping reducible to language B, if there is a computable function such that for every w

Example A={1,3,5,….} B={0,2,4,…} f(x)=x+1 via the funtion f

Theorem If , and B is decidable, then A is also decidable A B

Proof Let f be the reduction from A to B since Let M be the decider for B. Decider N: Input w, Compute f(w) Run M on f(w) and accept iff M accepts

Corollary If , and A is undecidable, then B is also undecidable A B

Proof Proof by contradiction. If B is decidable, then so is A

Halting Problem P1: ={<M,w>| M is a Turing machine and M halts on input w} P2: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

A reduction from to B: ={<M,w>| M is a Turing machine and M halts on input w} A: . ={<M,w>| M is a Turing machine and M accepts input w} Undecidable

Reduction On input <M,w> Construct Turing machine M’ Run M on w If M accepts, accept If M rejects, enter an infinite loop Output <M’,w>

Empty Problem ={<M>| M is a Turing machine and } Theorem: is an undecidable language. Proof by contraction. Assume it is decidable.

Reduction from to For <M,w>, design another TM M’ Input x If x is not equal to w, reject else simulate M on input w, accepts if M accepts w

Reduction from to <M,w> is in L(M’) is not empty <M,w> is not in L(M’) is empty

Reduction from to <M,w> is in M’ is in <M,w> is not in M’ is not in

Problem Let ={M| M is a Turing machine and M prints 1 for some input w} Show that L is undecidable.

Turing Machine Write on the tape and read from it Head can move left and right Tape is infinite Rejecting and accepting states Control a b a b

Turing Machine 7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

Configuration Current state: q7 Current head position on the tape: 4th cell Current tape content: abab q7 a b a b

Configuration A configuration is represented by Where is the left part of the tape content, is the right part of the tape content, a is the symbol at the head position, q is the current state

Configuration Transition For

Configuration Transition For

Configuration Start configuration: , where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields , and 3. is an accepting configuration

Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

Turing Recognizable Turing machine M recognizes language L

Decidability A language L is Turing decidable if there is a deterministic Turing machine M such that If x is in L, then M accepts x in finite number of steps If x is not in L, then M rejects x in finite number of steps Example: {w#w| w is in {0,1}*} is Turing decidable

Turing Decidable Turing machine M decides language L

Observation If L is Turing decidable, then L is Turing recognizable

A Turing recognizable problem ={<M,w>| M is a Turing machine and M accepts w} Turing machine R recognizable For input <M,w> Simulate M on w R accepts <M,w> if M accepts w.

Theorem Theorem: If L is a language in {0,1}*, there is a Turing machine to print out all elements in L

Proof Let M recognize L Step i(i=1,2,3…) Simulate M each of the first i strings i steps If a string is accepted, print it out.

Theorem If , and B is Turing recognizable, so is A A B

Proof Let f be the reduction from A to B since Let M recognize B. TM N: Input w, Compute f(w) Run M on f(w) and accept if M accepts

Theorem If , and A is not Turing recognizable, B is not Turing recognizable A B

Complement set Set and its complement

Theorem If A and are Turing recognizable, then A is decidable

Proof Let Turing machine M1 recognize A For input x, Run M1 and M2 on x in parallel If M1 accepts, accept If M2 accepts, reject.

={<A,B>| A and B are DFA and L(A)=L(B)} .

={<A,B>| A and B are DFA and L(A)=L(B)} Theorem: is not Turing decidable language.

. Theorem: Neither nor its complement is Turing recognizable

Proof We first prove For input <M,w> for TM M1: rejects any input TM M2: accepts any input if M accepts w. It is easy to see L(M1) not equal L(M2) iff M accepts w So,

Proof We prove For input <M,w> for TM M1: accepts any input TM M2: accepts any input if M accepts w. It is easy to see L(M1)=L(M2) iff M accepts w So,

Problem Show that the language F={M| L(M) contains infinite elements} is not Turing recognizable}.

Problem 4.2 The equivalence of a DFA and a regular expression is decidable

Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

Problem 4.7 The set of all infinite binary strings is not countable

Problem 4.8 N={1,2,3,…} NxNxN is countable

Problem 5.9 All Turing recognizable problems are mapping reducible to

Problem 5.12 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

Problem 5.13 Test if a Turing machine has useless state, which never enters.

Language Design Turing machine to recognize

Complexity Let M be a deterministic Turing machine. The running time of M is the function f:N N such that f(n) is the maximum number of steps that M uses on input of length n.

Big O-notation Let f and g be functions Say f(n)=O(g(n)) if integers c and exist so that for all

Examples Let

Small o-notation Let f and g be functions Say f(n)=o(g(n)) if

One tape Turing machine time One tape Turing machine can recognize In steps

Two tapes Turing machine time Two tapes Turing machine can recognize In O(n) steps

Multi-tape Turing Machine 1 1 Each tape has its own head Initially the input is at tape 1 and other tapes are blank Reading, writing and moving at all tapes Control a a a b a

Multi-tape Turing Machine Transition for multi-tape Turing machine

Simulate Multitape Turing machine Theorem: Every multitape Turing machine has an equivalent single tape Turing machine q7 # 1 1 # a a a # b a #

Simulation Initially, For one move simulation, scan the first #, then the second #, etc. It one tape goes out of its #, move the rest of tape in order to get one space for it. (stupid, slow)

Another simulation Assume there are k tapes Let the positions i, k+i, 2k+i, …., mk+i,… be used for simulating the tape i. q7 a b 1 a a a

Simulate 3 tapes q7 a b 1 a a a

Simulate 3 tapes q7 a b 1 a a a

K tapes Use two consecutive cells for one symbol, one is for storing the symbol and the other one is for storing the dot mark The tape i’s first j-cell information is at 2(j-1)k+2i-1 and 2(j-1)k+2i

Simulate multi-tape by one tape One tape Turing machine can simulate t(n) time multi-tape Turing machine in time

Time Complexity Class Let . is the class of languages decided by O(t(n) time Turing machines. In other words is a language decided by O(t(n)) time Turing machine}

One tape Turing machine time One tape Turing machine can decide In steps

Complexity Class P P is the class of languages decided by single tape Turing machine in polynomial time

Directed Path Problem G is a directed graph that has a directed path from s to t} a t c s d b

Algorithm Input mark “s” Repeat For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

Greatest common divisor Divisor: For two integers b and c, if b=c*z for some integer z, c is a divisor of b. Greatest common divisor: Given two integers a and b, gcd(a,b) is the greatest positive integer c such that c is the divisor for both a and b. Examples: gcd(10,4)=2, gcd(16,100)=4 Problem: How to find gcd(a,b)?

Relatively Prime Problem Two integers x and y are relatively prime if gcd(x,y)=1. RELPRIME={<x,y>| x and y are relatively prime} Theorem:

Euclid algorithm For two integer b and a a=q*b+c with gcd(a,b)=gcd(b,c) a and b are relatively prime iff b and c are relatively prime

Euclid algorithm Assume a1 and a2 are two positive integers If , then and Therefore, always true

Algorithm for gcd(x,y) Input Repeat rx mod y xy yr until y=0 output x

Modular Assume a and b are two positive integers This is a recursive equation since the second item goes down

Example Find gcd(1970,1066)

Speed of Euclid algorithm Assume a1 and a2 are two positive integers If , we have In another words,

Euclid algorithm Assume a1 and a2 are two positive integers

Time For input <a,b> The total time of steps is O(log a+log b)

Non-determinism blind monkey

Automata example q2 q1 q3 q3 q4 q2

Deterministic Turing Machine 7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

Nondeterministic Turing Machine 7-tuple Q is the finite set of states is the input alphabet not containing special blank is the tape alphabet is the start state, is the accept state is the reject state, where

Nondeterministic Turing Machine One configuration can have multiple choice for entering the next configuration because of

Non-determinism blind monkey

Acceptance of NFA The nondeterministic machine accepts if at least one of the computation branches ends in an accept state.

Acceptance of NTM The nondeterministic Turing machine accepts if at least one of the computation branches ends in an accept state.

Simulate NTM Theorem: Every Nondeterministic Turing machine has an equivalent deterministic Turing machine.

Simulate NTM with DTM Each dot is one configuration Find an accepting path

Class NP NP is the class of languages that can be recognized by nondeterministic Turing machine in polynomial time

Hamiltonian Path Hamiltonian path goes through each node exactly once HAMPATH={<G,s,t>| G is a directed graph with a Hamiltonian path from s to t}

Composite A natural number is composite if it is the product of two integers >1 10=2*5 18=2*9 Composite={x| x=pq, for integers p,q>1}

Verifier A verifier for a language L is an algorithm V, L={w| V accepts <w,c> for some string c} For the verifier V for L, c is a certificate of w if V accepts <w,c> If the verifier V for the language L runs in polynomial time, V is the polynomial time verifier for L.

Verifier for Hamiltonian Path For <G,s,t>, a certificate is a list of nodes of G: Verifier: check if m is the number of nodes of G check if and check if each is a directed edge of G for i=1,…,m-1 If all pass, accept . Otherwise, reject.

Verifier for Composite For integer x, a certificate is two integers p,q: Verifier: check if p>1 and q>1 check if x=pq If all pass, accept . Otherwise, reject.

Class NP NP is the class of languages that have polynomial time verifiers. Examples: COMPOSITE is in NP HAMPATH is in NP

Theorem A language has polynomial verifier iff it can be recognized by polynomial time nondeterministic Turing machine Proof: Assume L has verifier V, …  Assume L has NTM M,…

Proof Assume that L has polynomial time verifier V, which runs in time, where k is a constant NTM M Nondeterministically select string c of length Run V on <w,c> If V accepts, accept; Otherwise, reject.

Proof Assume that L is recognized by polynomial time NTM M, which runs in time, where k is a constant Verifier <w,c> Let c determine a computation path of M Simulate M on the path c If M accepts, accept. Otherwise, reject.

Clique Problem Given undirected graph G, a clique is a set of nodes of G such that every two nodes are connected by an edge. A k-clique is a clique with k nodes

Clique Problem CLIQUE={<G,k>| G iss an undirected graph with k-clique} CLIQUE is in NP.

Subset Sum Problem SUBSET-SUM={<S,t>| S= and for some , we have

Polynomial Time Computable A function is a polynomial time computable function if some polynomial time Turing machine M exists that outputs on the tape for input w.

Polynomial Time Reduction Assume that A and B are two languages on A is polynomial time mapping reducible to A if a polynomial time computable function exists such that

Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

SAT A boolean formula is satisfiable if there exists assignments to its variables to make the formula true SAT={ | is satisfiable boolean formula}

3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

3SAT to CLIQUE Example:

NP-completeness A language B is NP-complete if B is in NP, and Every A in NP is polynomial time reducible to B Theorem. If B is NP-complete and B is in P, then P=NP.

Cook-Leving Theorem Theorem: SAT is NP-complete Proof. 1. SAT is in NP. 2. For every problem A in NP,

Proof The start configuration is legal The final state is accept. The movement is legal. 4. Each cell takes one legal symbol.

Proof 1 if The cell[i,j] holds symbol s; 0 otherwise Time bound for the NTM M with constant k. 3. The movement is legal. 4. NTM M for accepting A.

Nondeterministic Turing Machine Q is the finite set of states is the tape alphabet is the start state, is the accept state.

Configuration Transition For

Configuration Transition For

Configuration Start configuration: , where w is the input Accepting configuration: a configuration with state Rejecting configuration: a configuration with state

Accept Computation A Turing machine M accepts input w if a sequence of configurations exists where 1. is the start configuration of M on input w, 2. each yields , and 3. is an accepting configuration

Language recognized by TM For a Turing machine M, L(M) denotes the set of all strings accepted by M. A language is Turing recognizable if some Turing machine recognizes it.

Proof Each cell has only one symbol The symbol is selected from C: Only one symbol is selected: It is true for all cell at all configuration:

Proof The start configuration is

Proof Accept computation has reached. It makes sure the accept state will appear among the configuration transitions.

Proof Characterize the legal move The whole move is legal if all windows are legal. Characterize one window is legal

Proof The state transition

Logic Demorgan Law:

Truth table for y1 y2 x2 1 1 1 0 1 1 1 0 1 0 1 0 1 0 1 1 0 0 0 1 0 1 1 1 0 0 1 0 0 1 0 0 1 1 0 0 0 0 1 0

Convert to CNF Conversion:

Convert to CNF Conversion:

Boolean Formula A literal is either a boolean variable or its negation: A clause is the disjunction of several literals Conjunctive normal form is the conjunction of several clauses

Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory

Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular

Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier

Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete.

Grade A:… B:… C: Miss exam or homework

SAT’ A conjunctive normal form is a conjunction of some clauses SAT’={ | is satisfiable conjunctive normal form}

Cook-Leving Theorem’ Theorem: SAT’ is NP-complete Proof. Same as that for SAT is NP-complete

3SAT A 3nd conjunctive normal formula (3nd-formula) is a conjunction form with at most 3 literals at each clause 3SAT={ | is satisfiable 3nd-formula}

3SAT is NP-complete Theorem: There is polynomial time reduction from SAT’ to 3SAT.

3SAT is NP-complete is satisfiable if and only if the following is satisfiable

3SAT is NP-complete is satisfiable if and only if the following is satisfiable

3SAT is NP-complete Convert every clause into 3cnf:

3SAT is NP-complete Conjunctive normal form Each clause is convert into is satisfiable if and only if the following is satisfiable

Problem Convert the formula F into 3SAT formula F’ such that F is satisfiable iff and F’ is satisfiable.

Approximation Algorithms

Outline and Reading Approximation Algorithms for NP-Complete Problems Approximation ratios Polynomial-Time Approximation Schemes 2-Approximation for Vertex Cover Approximate Scheme for Subset Sum 2-Approximation for TSP special case Log n-Approximation for Set Cover

Approximation Ratios Optimization Problems We have some problem instance x that has many feasible “solutions”. We are trying to minimize (or maximize) some cost function c(S) for a “solution” S to x. For example, Finding a minimum spanning tree of a graph Finding a smallest vertex cover of a graph Finding a smallest traveling salesperson tour in a graph

Approximation Ratios An approximation produces a solution T T is a k-approximation to the optimal solution OPT if c(T)/c(OPT) < k (assuming a min. prob.; a maximization approximation would be the reverse)

Polynomial-Time Approximation Schemes A problem L has a polynomial-time approximation scheme (PTAS) if it has a polynomial-time (1+)-approximation algorithm, for any fixed  >0 (this value can appear in the running time). Subset Sum has a PTAS.

Vertex Cover A vertex cover of graph G=(V,E) is a subset W of V, such that, for every (a,b) in E, a is in W or b is in W. OPT-VERTEX-COVER: Given an graph G, find a vertex cover of G with smallest size. OPT-VERTEX-COVER is NP-hard.

Copyright © The McGraw-Hill Companies, Inc Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright © The McGraw-Hill Companies, Inc Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

A 2-Approximation for Vertex Cover Every chosen edge e has both ends in C But e must be covered by an optimal cover; hence, one end of e must be in OPT Thus, there is at most twice as many vertices in C as in OPT. That is, C is a 2-approx. of OPT Running time: O(m) Algorithm VertexCoverApprox(G) Input graph G Output a vertex cover C for G C  empty set H  G while H has edges e  H.removeEdge(H.anEdge()) v  H.origin(e) w  H.destination(e) C.add(v) C.add(w) for each f incident to v or w H.removeEdge(f) return C

Problem 1 Show that T={(i,j)| both I and j are positive rational numbers} is countable.

Problem 2. Find a correspondence between (0,1) and [0,1]

Solution Let for n=1, 2, … Define the correspondence f(x) below:

Problem 3 3. Let A={<R,S> | R and S are regular expressions and L(R ) is a subset of L( S)}. Show that A is decidable.

R S Solution For two regular expression R and S, L( R) is a subset of L(S) if and only if the L(R ) has no intersection with . Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Check if L(M4) is empty

Problem 4 I={x| x is an irrational number in (0,1)}. Show that I is not countable.

Homework 2009

Problem 4.2 The equivalence of a DFA and a regular expression is decidable

Solution Proof. Given a regular language L and DFA M. We can construct a DFA N to accept L. The problem is converted into checking if two DFAs are equivalent. By Theorem 4.5, there is an algorithm to decide if L(M)=L(N).

Problem 4.5 A is a DFA and L(A) is infinite} Show is decidable

Solution Assume M is a DFA with n states. L(M) is infinite iff it has path from the start state to accept with a loop on it, and the loop length is no more than n. L(M) is infinite iff M accepts a string of length between n and 2n. For every string x with length between n and 2n Run M on x If M accepts x, then output “L(M) is infinite” else output “no”

Problem 4.7 The set of all infinite binary strings is not countable

Proof. Proof by contradiction. Assume N={1,2,3,…} and have the same size. There is one-one and onto map f: N

Proof. Select such that and Since

Problem 1 N={1,2,3,…} NxNxN is countable

Proof List all of them by

Proof Use the similar method like the last slide to prove the NxN is countable Let f: NxNN be one-one and onto We have g: NxNxNN is one-one and onto, where g(i,j,k)=f(f(i,j),k)

Problem 2. Find a correspondence between (0,1) and [0,1]

Solution Let for n=1, 2, … Define the correspondence f(x) below:

Problem 3 3. Let A={<R,S> | R and S are regular expression and L(R ) = L( S)}. Show that A is decidable.

Solution For two regular expression R and S, L( R)=L(S) if and only if their symmetric difference is empty. Find two automata M1 and M2 to accept R and S respectively. Construct automata M3 to accept Construct automata M4 to accept Construct automata M5 to accept Check if L(M5) is empty

Problem 4 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

Proof We are going to design a reduction from to S. For <M,w> Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts <M,w> is in iff N accepts both 01 and 10.

Problem 5 5. Let S={<M> | M is a TM and L(M) = {<M>}}. Show that neither S nor its complement is Turing-recognizable.

Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal <Y> then reject If x=<Y> then accepts if M(w) accepts.

Solution For an input (M, w), derive M1(x, M1).

Solution We show that can be reducible to . Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x =<M1> then accept If x is not equal <M1> then accept if M(w) accepts.

Solution For an input (M, w), derive M2(x, M2).

Problem 6 Show that A is a Turing recognizable if and only if

Proof Part a)  Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =<M,w>. It is easy to see that

Proof Part b)  Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize . We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

Proof We are going to design a reduction from to it For <M,w> Design the TM N as follows Input x if x=0, enters all states except if x=1 run M on w, if M accepts w, enter <M,w> is in iff N enters all states.

Problem 4 c) The language is not a context free language Proof: Assume it is a context free language. Let p be the number from the pumping lemma Consider the string

Final Exam December 13, 5:45-7:30pm, Monday

Homework 4, 20

Problem 1 1.Let T1={<M>| M is a Turing machines and accepts infinite number of 0,1-strings of finite length}. Prove that T1 is undecidable.

Proof We are going to design a reduction from to T1. For input Turing machine <M,w> for Design the TM N as follows Input x run M on w, if M accepts w, acceptx <M,w> is in iff N acccepts each input x.

Problem 2 Let T2={<M>| M is a Turing machines and accepts five of 0,1-strings of finite length}. Prove that T2 is undecidable.

Proof We are going to design a reduction from to T2. For <M,w> Design the TM N as follows Input x if x is not 0,1, 00, 01 or 10, rejects. run M on w if M accepts w, accept x <M,w> is in iff N accepts 0,1,00,01, and 10. Otherwise, N accepts no string

Problem 3 S={<M>| M is a TM that accepts whenever it accepts w}. Show S is undecidable.

Proof We are going to design a reduction from to S. For <M,w> Design the TM N as follows Input x if x is not 01 or 10, rejects. if x=01, accepts if x=10, run M on w, if M accepts w, accepts <M,w> is in iff N accepts both 01 and 10.

Problem 4 Show that A is a Turing recognizable if and only if

Proof Part a)  Let A be a Turing recognizable language. There is a Turing machine M such that if w is in A iff M accepts w Let f be the reduction f(w) =<M,w>. It is easy to see that

Proof Part b)  Assume that via computable function g. It is easy to see that is Turing recognizable. Let M1 be a Turing machine that recognize . We have Turing machine M2 to recognize A: M2: Input w Compute g(w) Run M1 on input g(w), accepts w if M1 accepts g(w).

Problem 5 5. Let S={<M> | M is a TM and L(M) = {<M>}}. Show that neither S nor its complement is Turing-recognizable.

Solution We first show that can be reducible to S. Let (M, w) be an input for Let Turing machine M1 be constructed below: M1 (x, Y) Input x and another TM Y If x is not equal <Y> then reject If x=<Y> then accepts if M(w) accepts.

Solution For an input (M, w), derive M1(x, M1).

Solution We show that can be reducible to . Let (M, w) be an input for Let Turing machine M2 be constructed below: M1 Input x If x =<M1> then accept If x is not equal <M1> then accept if M(w) accepts.

Solution For an input (M, w), derive M2(x, M2).

Homework 5, 20

Problem 1 Call graphs G and H isomorphic if the nodes of G may be reordered so that it is identical to H. Let ISO={<G,H> | G and H are isomorphic graphs}. Show that ISO is in NP.

Solution for Problem 1 We design a polynomial time verification algorithm. A mapping f from the vertices of G to those of H is a witness. Check if 1) f is one-one, 2) f is onto, 3) (v,u) is an edge of G iff (f(u),f(v)) is an edge of H. It is easy to see that the verification takes polynomial time.

Problem 2 Let MAX-CIQUE={<G, k>| the largest clique of G has k vertices}. Whether MAX-CLIQUE is in NP is unknown. Show that if P=NP, then MAX-CLIQUE is in P, and a polynomial time algorithm exists that, for a graph G, finds one of its largest cliques.

Solution of Problem 2 Fist step is to find the largest k with (G,k) is in Clique. Try k from 1,2,… Check if (G,k) is in Clique Select the largest k. Assume the largest k for (G,k) in Clique is obtained.

Solution of Problem 2 Assume the largest k for (G,k) in Clique is obtained. Formulate the problem: (H, k,G) Determine if there is a clique of size k in G and contains all veritces in H. The problem is in NP. Extend H one by one until its size reaches k.

Problem 3 Let G represent an undirected graph and let SPATH={<G, a,b,k> | G contains a simple path of length at most k from a to b} and LPATH={<G, a,b,k} | G contains a simple path of length at least k from a to b}. Show that SPATH is in P. Show that LPATH is NP-complete.

Solution Problem 3 Show that SPATH is in P.

Directed Path Problem G is a directed graph that has a directed path from s to t} a t c s d b

Algorithm Input mark “s” Repeat For each edge (a,b), if “a” is marked, the mark “b” until no node is marked If (“t” is marked) then accept else reject

Solution Problem 3 Show that LPATH is NP-complete. Hamiltonian path problem is NP-complete. There is an easy reduction from Hamiltonian path to it.

Prepare for the Final Regular language and automata Context free language Decidability Undecidability Complexity theory

Regular Language Concepts: Automata, regular expression Skills: Design automata to accept a regular language Disprove a language is a regular

Context-free Language Concepts: Context-free grammar, parsing tree Skills: Design automata to accept a context-free language Disprove a language is context-free

Decidability Concepts: Turing machine, algorithm, Church-Turing Thesis, Turing recognizable, Turing Decidable Skills: Prove a language is decidable (design algorithm) Prove a language is Turing recognizable

Undecidability Concepts: Countable, Turing undecidable, reduction Skills: Diagonal method: Prove is undeciable Use reduction to prove a language is undecidable

Complexity Concepts: Time on Turing machine PTIME(t(n)) NP-completeness Polynomial time reduction Polynomial time verifier

Complexity Skill: Prove a problem is in P Prove a problem is in NP Use reduction to prove a problem is NP-complete.

Grade A:… B:… C: Miss exam or homework

Space Complexity Complexity ©D.Moshkovits

Motivation Complexity classes correspond to bounds on resources One such resource is space: the number of tape cells a TM uses when solving a problem Complexity ©D.Moshkovits

Introduction Objectives: To define space complexity classes Overview: Low space classes: L, NL Savitch’s Theorem Immerman’s Theorem TQBF Complexity ©D.Moshkovits

Space Complexity Classes For any function f:NN, we define: SPACE(f(n))={ L : L is decidable by a deterministic O(f(n)) space TM} NSPACE(f(n))={ L : L is decidable by a non-deterministic O(f(n)) space TM} Complexity ©D.Moshkovits

Low Space Classes Definitions (logarithmic space classes): L = SPACE(logn) NL = NSPACE(logn) Complexity ©D.Moshkovits

Problem! How can a TM use only logn space if the input itself takes n cells?! !? Complexity ©D.Moshkovits

3Tape Machines a b _ b _ b a _ input . . . work output . . . . . . read-only Only the size of the work tape is counted for complexity purposes read/ write b _ . . . write-only b a _ . . . Complexity ©D.Moshkovits

Example Question: How much space would a TM that decides {anbn | n>0} require? Note: to count up to n, we need logn bits Complexity ©D.Moshkovits

Graph Connectivity CONN An undirected version is also worth considering CONN Instance: a directed graph G=(V,E) and two vertices s,tV Problem: To decide if there is a path from s to t in G? Complexity ©D.Moshkovits

Graph Connectivity t s Complexity ©D.Moshkovits

CONN is in NL Start at s For i = 1, .., |V| { Non-deterministically choose a neighbor and jump to it Accept if you get to t } If you got here – reject! Counting up to |V| requires log|V| space Storing the current position requires log|V| space Complexity ©D.Moshkovits

Configurations Which objects determine the configuration of a TM of the new type? The content of the work tape The machine’s state The head position on the input tape The head position on the work tape The head position on the output tape If the TM uses logarithmic space, there are polynomially many configurations Complexity ©D.Moshkovits

Log-Space Reductions Definition: A is log-space reducible to B, written ALB, if there exists a log space TM M that, given input w, outputs f(w) s.t. wA iff f(w)B the reduction Complexity ©D.Moshkovits

Do Log-Space Reductions Imply what they should? Suppose A1 ≤L A2 and A2L; how to construct a log space TM which decides A1? Wrong Solution: w Too Large! f(w) Use the TM for A2 to decide if f(w)A2 Complexity ©D.Moshkovits

Log-Space reductions Claim: if Then, A1 is in L A1 ≤L A2 – f is the log-space reduction A2  L – M is a log-space machine for A2 Then, A1 is in L Proof: on input x, in or not-in A1: Simulate M and whenever M reads the ith symbol of its input tape run f on x and wait for the ith bit to be outputted Complexity ©D.Moshkovits

NL Completeness Definition: A language B is NL-Complete if BNL For every ANL, ALB. If (2) holds, B is NL-hard Complexity ©D.Moshkovits

Savitch’s Theorem Theorem: S(n) ≥ log(n) NSPACE(S(n))  SPACE(S(n)2) Proof: First we’ll prove NLSPACE(log2n) then, show this implies the general case Complexity ©D.Moshkovits

Savitch’s Theorem Theorem: NSPACE(logn) SPACE(log2n) Proof: First prove CONN is NL-complete (under log-space reductions) Then show an algorithm for CONN that uses log2n space Complexity ©D.Moshkovits

CONN is NL-Complete Theorem: CONN is NL-Complete Proof: by the following reduction: s L t “Is there a path from s to t?” “Does M accept x?” Complexity ©D.Moshkovits

Technicality Observation: Without loss of generality, we can assume all NTM’s have exactly one accepting configuration. Complexity ©D.Moshkovits

Configurations Graph A Computation of a NTM M on an input x can be described by a graph GM,x: A vertex per configuration the start configuration s t the accepting configuration (u,v)E if M can move from u to v in one step Complexity ©D.Moshkovits

Correctness Claim: For every non-deterministic log-space Turing machine M and every input x, M accepts x iff there is a path from s to t in GM,x Complexity ©D.Moshkovits

CONN is NL-Complete Corollary: CONN is NL-Complete Proof: We’ve shown CONN is in NL. We’ve also presented a reduction from any NL language to CONN which is computable in log space (Why?)  Complexity ©D.Moshkovits

A Byproduct Claim: NLP Proof: Any NL language is log-space reducible to CONN Thus, any NL language is poly-time reducible to CONN CONN is in P Thus any NL language is in P.  Complexity ©D.Moshkovits

What Next? We need to show CONN can be decided by a deterministic TM in O(log2n) space. Complexity ©D.Moshkovits

The Trick “Is there a path from u to v of length d?” “Is there a vertex z, so there is a path from u to z of size d/2 and one from z to v of size d/2?” d/2 d/2 u . . . z v d Complexity ©D.Moshkovits

Recycling Space The two recursive invocations can use the same space Complexity ©D.Moshkovits

The Algorithm Boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if d=1 return FALSE for every vertex v { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE Complexity ©D.Moshkovits

Example of Savitch’s algorithm boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then } return FALSE 2 3 1 4 (a,b,c)=Is there a path from a to b, that takes no more than c steps. (1,4,3)(1,3,2)(2,3,1)TRUE (1,4,3)(1,2,2)TRUE (1,4,3)(1,2,2) (1,4,3)(2,4,1) (1,4,3)(2,4,1)FALSE (1,4,3)(1,3,2)TRUE (1,4,3) (1,4,3) TRUE (1,4,3)(3,4,1)TRUE (1,4,3)(1,3,2)(1,2,1) (1,4,3)(1,3,2)(2,3,1) (1,4,3)(3,4,1) (1,4,3)(1,3,2) (1,4,3)(1,3,2)(1,2,1)TRUE Complexity ©D.Moshkovits 3Log2(d)

O(log2n) Space DTM Claim: There is a deterministic TM which decides CONN in O(log2n) space. Proof: To solve CONN, we invoke PATH(s,t,|V|) The space complexity: S(n)=S(n/2)+O(logn)=O(log2n)  Complexity ©D.Moshkovits

Conclusion Theorem: NSPACE(logn) SPACE(log2n) How about the general case NSPACE(S(n))SPACE(S2(n))? Complexity ©D.Moshkovits

The Padding Argument Motivation: Scaling-Up Complexity Claims We have: can be simulated by… space space + non-determinism + determinism We want: can be simulated by… space space + non-determinism + determinism Complexity ©D.Moshkovits

Formally  NSPACE(s1(f(n)))  SPACE(s2(f(n))) si(n) can be computed with space si(n) Claim: For any two space constructible functions s1(n),s2(n)logn, f(n)n: NSPACE(s1(n))  SPACE(s2(n))  NSPACE(s1(f(n)))  SPACE(s2(f(n))) simulation overhead E.g NSPACE(n)SPACE(n2)  NSPACE(n2)SPACE(n4) Complexity ©D.Moshkovits

Idea NTM DTM n n . . . f(n) . space: s1(.) in the size of its input space: O(s2(f(n))) NTM n n . . . space: O(s1(f(n))) f(n) . Complexity ©D.Moshkovits

Padding argument Let LNPSPACE(s1(f(n))) There is a 3-Tape-NTM ML: |x| Input babba Work  O(s1(f(|x|))) Complexity ©D.Moshkovits

Padding argument Let L’ = { x0f(|x|)-|x| | xL } We’ll show a NTM ML’ which decides L’ in the same number of cells as ML. f(|x|) babba#00000000000000000000000000000000 Input Work  O(s1(f(|x|)) Complexity ©D.Moshkovits

Padding argument – ML’ In O(log(f(|x|)) space Count backwards the number of 0’s and check there are f(|x|)-|x| such. 2. Run ML on x. in O(s1(f(|x|))) space f(|x|) Input babba00000000000000000000000000000000 Work  O(s1(f(|x|))) Complexity ©D.Moshkovits

Padding argument Total space: O(s1(f(|x|))) f(|x|) Input babba00000000000000000000000000000000 Work  O(s1(f(|x|))) Complexity ©D.Moshkovits

Padding Argument We started with LNSPACE(s1(f(n))) We showed: L’NSPACE(s1(n)) Thus, L’SPACE(s2(n)) Using the DTM for L’ we’ll construct a DTM for L, which will work in O(s2(f(n))) space. Complexity ©D.Moshkovits

Padding Argument The DTM for L will simulate the DTM for L’ when working on its input concatenated with zeros Input babba 00000000000000000000000 Complexity ©D.Moshkovits

Padding Argument When the input head leaves the input part, just pretend it encounters 0s. maintaining the simulated position (on the imaginary part of the tape) takes O(log(f(|x|))) space. Thus our machine uses O(s2(f(|x|))) space.  NSPACE(s1(f(n)))SPACE(s2(f(n))) Complexity ©D.Moshkovits

Savitch: Generalized Version Theorem (Savitch): S(n) ≥ log(n) NSPACE(S(n))  SPACE(S(n)2) Proof: We proved NLSPACE(log2n). The theorem follows from the padding argument.  Complexity ©D.Moshkovits

Corollary Corollary: PSPACE = NPSPACE Proof: Clearly, PSPACENPSPACE. By Savitch’s theorem, NPSPACEPSPACE.  Complexity ©D.Moshkovits

Space Vs. Time We’ve seen space complexity probably doesn’t resemble time complexity: Non-determinism doesn’t decrease the space complexity drastically (Savitch’s theorem). We’ll next see another difference: Non-deterministic space complexity classes are closed under completion (Immerman’s theorem). Complexity ©D.Moshkovits

NON-CONN NON-CONN Instance: A directed graph G and two vertices s,tV. Problem: To decide if there is no path from s to t. Complexity ©D.Moshkovits

NON-CONN Clearly, NON-CONN is coNL-Complete. (Because CONN is NL-Complete. See the coNP lecture) If we’ll show it is also in NL, then NL=coNL. (Again, see the coNP lecture) Complexity ©D.Moshkovits

An Algorithm for NON-CONN We’ll see a log space algorithm for counting reachability Count how many vertices are reachable from s. Take out t and count again. Accept if the two numbers are the same. Complexity ©D.Moshkovits

N.D. Algorithm for reachs(v, l) 1. length = l; u = s 2. while (length > 0) { 3. if u = v return ‘YES’ 4. else, for all (u’  V) { 5. if (u, u’) E nondeterministic switch: 5.1 u = u’; --length; break 5.2 continue } } 6. return ‘NO’ Takes up logarithmic space This N.D. algorithm might never stop Complexity ©D.Moshkovits

N.D. Algorithm for CRs CRs ( d ) 1. count = 0 2. for all uV { 3. countd-1 = 0 4. for all vV { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++countd-1 else fail if (v,u)  E then ++count; break 5.2 continue } 6. if countd-1 < CRs (d-1) fail 7.return count Assume (v,v)  E Recursive call! Complexity ©D.Moshkovits

N.D. Algorithm for CRs parameter , C) CRs ( d 1. count = 0 2. for all uV { 3. countd-1 = 0 4. for all vV { 5. nondeterministic switch: 5.1 if reach(v, d - 1) then ++countd-1 else fail if (v,u)  E then ++count; break 5.2 continue } 6. if countd-1 < fail 7.return count Main Algorithm: CRs C = 1 for d = 1..|V| C = CR(d, C) return C C parameter Complexity ©D.Moshkovits

Efficiency Lemma: The algorithm uses O(log(n)) space. Proof: There is a constant number of variables ( d, count, u, v, countd-1). Each requires O(log(n)) space (range |V|).  Complexity ©D.Moshkovits

Immerman’s Theorem Theorem[Immerman/Szelepcsenyi]: NL=coNL Proof: (1) NON-CONN is NL-Complete (2) NON-CONNNL Hence, NL=coNL.  Complexity ©D.Moshkovits

Corollary Corollary: s(n)log(n), NSPACE(s(n))=coNSPACE(s(n)) Proof: By a padding argument. Complexity ©D.Moshkovits

TQBF We can use the insight of Savich’s proof to show a language which is complete for PSPACE. We present TQBF, which is the quantified version of SAT. Complexity ©D.Moshkovits

TQBF Instance: a fully quantified Boolean formula  Problem: to decide if  is true Example: a fully quantified Boolean formula xyz[(xyz)(xy)] Variables` range is {0,1} Complexity ©D.Moshkovits

TQBF is in PSPACE Theorem: TQBFPSPACE Proof: We’ll describe a poly-space algorithm A for evaluating : If  has no quantifiers: evaluate it If =x((x)) call A on (0) and on (1); Accept if both are true. If =x((x)) call A on (0) and on (1); Accept if either is true. in poly time Complexity ©D.Moshkovits

Algorithm for TQBF 1 1 1 1 1 xy[(xy)(xy)] y[(0y)(0y)] (00)(00) (01)(01) (10)(10) (11)(11) 1 1 Complexity ©D.Moshkovits

Efficiency Since both recursive calls use the same space, the total space needed is polynomial in the number of variables (the depth of the recursion)  TQBF is polynomial-space decidable  Complexity ©D.Moshkovits

PSAPCE Completeness Definition: A language B is PSPACE-Complete if BPSPACE For every APSAPCE, APB. standard Karp reduction If (2) holds, then B is PSPACE-hard Complexity ©D.Moshkovits

TQBF is PSPACE-Complete Theorem: TQBF is PSAPCE-Complete Proof: It remains to show TQBF is PSAPCE-hard: P x1x2x3…[…] “Will the poly-space M accept x?” “Is the formula true?” Complexity ©D.Moshkovits

TQBF is PSPACE-Hard Given a TM M for a language L PSPACE, and an input x, let fM,x(u, v), for any two configurations u and v, be the function evaluating to TRUE iff M on input x moves from configuration u to configuration v fM,x(u, v) is efficiently computable Complexity ©D.Moshkovits

Formulating Connectivity The following formula, over variables u,vV and path’s length d, is TRUE iff G has a path from u to v of length ≤d (u,v,1)  fM,x(u, v)  u=v (u,v,d)  wxy[((x=uy=w)(x=wy=v))(x,y,d/2)] w is reachable from u in d/2 steps. v is reachable from w in d/2 steps. simulates AND of (u,w,d/2) and (w,v,d/2) Complexity ©D.Moshkovits

TQBF is PSPACE-Complete Claim: TQBF is PSPACE-Complete Proof:   (s,t,|V|) is TRUE iff there is a path from s to t.  is constructible in poly-time. Thus, any PSPACE language is poly-time reducible to TQBF, i.e – TQBF is PSAPCE-hard. Since TQBFPSPACE, it’s PSAPCE-Complete.  Complexity ©D.Moshkovits

 Summary We introduced a new way to classify problems: according to the space needed for their computation. We defined several complexity classes: L, NL, PSPACE. Complexity ©D.Moshkovits

 Summary Our main results were: Connectivity is NL-Complete TQBF is PSPACE-Complete Savitch’s theorem (NLSPACE(log2)) The padding argument (extending results for space complexity) Immerman’s theorem (NL=coNL) By reducing decidability to reachability Complexity ©D.Moshkovits