Recall last lecture and Nondeterministic TMs Ola Svensson

Computational Complexity

Turing Machines

The goals A model that can compute anything computable A model so simple and clean so that we can prove interesting results

A Turing machine 𝑀 is described by a tuple (Γ, 𝑄, 𝛿) A finite alphabet Γ containing (⊳,∎, 0, 1, …) start symbol blank symbol

A Turing machine 𝑀 is described by a tuple (Γ, 𝑄, 𝛿) A finite alphabet Γ containing (⊳,∎, 0, 1, …) A finite set 𝑄 of states containing 𝑞 𝑠𝑡𝑎𝑟𝑡 and 𝑞 ℎ𝑎𝑙𝑡 A function 𝛿: Γ 2 ×𝑄→ Γ × 𝐿,𝑆,𝑅 𝑘 ×𝑄 describing rules of each step: IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Start configuration of 𝑀 on input 𝑥 Input: ⊳ 1 ∎ … ∎ … Work/output: ⊳ ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input ∎ … Work/output: ⊳ ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞  Hey, don’t miss this! IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 IF THEN Input ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 IF THEN Input ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 ℎ𝑎𝑙𝑡 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 ℎ𝑎𝑙𝑡 What do we compute? IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right Stay Left 𝑞 1 𝑞 ℎ𝑎𝑙𝑡

Efficiency and Running Time Let 𝑓: 0,1 ∗ → 0,1 ∗ and let 𝑇:ℕ→ℕ Let 𝑀 be a Turing machine

Efficiency and Running Time Let 𝑓: 0,1 ∗ → 0,1 ∗ and let 𝑇:ℕ→ℕ Let 𝑀 be a Turing machine We say that 𝑀 computes 𝑓 if for every 𝑥∈ 0,1 ∗ , whenever 𝑀 is initialized to the start configuration on input 𝑥, then it halts with f(𝑥) written on its output type.

Efficiency and Running Time Let 𝑓: 0,1 ∗ → 0,1 ∗ and let 𝑇:ℕ→ℕ Let 𝑀 be a Turing machine We say that 𝑀 computes 𝑓 if for every 𝑥∈ 0,1 ∗ , whenever 𝑀 is initialized to the start configuration on input 𝑥, then it halts with f(𝑥) written on its output type. We say that 𝑀 computes 𝑓 in T n -time if its computation on every input 𝑥 requires at most T( 𝑥 ) steps.

Efficiency and Running Time Let 𝑓: 0,1 ∗ → 0,1 ∗ and let 𝑇:ℕ→ℕ Let 𝑀 be a Turing machine We say that 𝑀 computes 𝑓 if for every 𝑥∈ 0,1 ∗ , whenever 𝑀 is initialized to the start configuration on input 𝑥, then it halts with f(𝑥) written on its output type. We say that 𝑀 computes 𝑓 in T n -time if its computation on every input 𝑥 requires at most T( 𝑥 ) steps. DTIME(f(n)) = Set of languages that can be decided by a TM in time 𝑐⋅𝑓(𝑛)

Robustness of definition of Turing Machine The size of the alphabet doesn’t really matter The number of tapes do not really matter (up to a square in the running time) It is able to simulate all other realistic models with polynomial slow down (except potentially quantum computers that still need to be built) Church-Turing Thesis: Any computable function can be computed by a Turing machine

Universal Turing Machine Every binary string represents some Turing machine (write down 𝛿) A Turing machine has infinitely many representations (pad with ones) There exists a TM 𝑈 such that for every 𝑥, 𝛼∈ 0,1 ∗ 𝑈 𝑥, 𝛼 = M 𝛼 (𝑥) Moreover, if 𝑀 𝛼 halts on input 𝑥 within 𝑇 steps then 𝑈(𝑥,𝛼) halts within 𝐶⋅𝑇⋅𝑙𝑜𝑔(𝑇) steps where 𝐶 is a number independent of 𝑥 . (𝐶 depends on the number of states, alphabet, and tapes of 𝑀 𝛼 ).

DIAGONALIZATION

Cantor’s argument The set of infinite binary strings are uncountable 𝑠 1 =(𝟎, 0, 0, 0, 0, 0, 0, … 𝑠 2 =(0, 𝟏, 0, 1, 0, 1, 0, … 𝑠 3 =(0, 1, 𝟏, 0, 1, 1, 0, … 𝑠 4 =(1, 1, 1, 𝟎, 1, 1, 0, … 𝑠 5 =(0, 0, 1, 0, 𝟏, 0, 1, … 𝑠 6 =(0, 0, 1, 0, 0, 𝟏, 1, … 𝑠 7 =(0, 0, 1, 0, 0, 1, 𝟎, … … 𝑠=(1, 0, 0, 1, 0, 0, 1, … s is different from any string in the enumeration so it is not part of the enumeration

Halting Problem The output of 𝑀 𝛼 3 ( 𝛼 5 ) (? if doesn’t halt) 𝜶 𝟏 𝜶 𝟐 𝜶 𝟑 𝜶 𝟒 𝜶 𝟓 𝜶 𝟔 𝟎 1 𝟏 ? … 𝐿= { 𝛼 1 , 𝛼 2 , 𝛼 4 𝛼 5 , 𝛼 6 , …} The set of strings 𝛼 for which M 𝛼 𝛼 =0 or ?

Halting Problem The output of 𝑀 𝛼 3 ( 𝛼 5 ) (? if doesn’t halt) 𝜶 𝟏 𝜶 𝟐 𝜶 𝟑 𝜶 𝟒 𝜶 𝟓 𝜶 𝟔 𝟎 1 𝟏 ? … 𝐿= { 𝛼 1 , 𝛼 2 , 𝛼 4 𝛼 5 , 𝛼 6 , …} The set of strings 𝛼 for which M 𝛼 𝛼 =0 or ? Suppose a TM M decides L. Let 𝛼 be its encoding. Then we get a contradiction on input 𝛼 since 𝛼∈𝐿⇔ 𝑀 𝛼 𝛼 =0

Deterministic Time Hierarchy The output of 𝑀 𝛼 3 ( 𝛼 5 ) (? if does not halt in 𝒏 𝟏.𝟒 steps) 𝜶 𝟏 𝜶 𝟐 𝜶 𝟑 𝜶 𝟒 𝜶 𝟓 𝜶 𝟔 𝟎 1 𝟏 ? … 𝐿= { 𝛼 1 , 𝛼 2 , 𝛼 4 𝛼 5 , 𝛼 6 , …} The set of strings 𝛼 for which M 𝛼 𝛼 =0 or ?

Deterministic Time Hierarchy The output of 𝑀 𝛼 3 ( 𝛼 5 ) (? if does not halt in 𝒏 𝟏.𝟒 steps) 𝜶 𝟏 𝜶 𝟐 𝜶 𝟑 𝜶 𝟒 𝜶 𝟓 𝜶 𝟔 𝟎 1 𝟏 ? … 𝐿= { 𝛼 1 , 𝛼 2 , 𝛼 4 𝛼 5 , 𝛼 6 , …} The set of strings 𝛼 for which M 𝛼 𝛼 =0 or ? Suppose a TM M decides L in time n. Let 𝛼 be its encoding. Then we get a contradiction on input 𝛼 since 𝛼∈𝐿⇔ 𝑀 𝛼 𝛼 =0

Diagonalization The proof technique used two things about TMs They can be encoded efficiently There is a TM that can emulate any other without increasing running time or space too much Otherwise, they treated TMs as ”black boxes” We will show today that such a proof cannot alone resolve P vs NP

Nondeterministic Turing Machines

Nondeterministic Turing machines (NDTMs) Same definition as TMs except that NDTM has two transition functions 𝛿 0 and 𝛿 1 and a special state called 𝑞 𝑎𝑐𝑐𝑒𝑝𝑡 When NDTM computes a function we envision that at each computational step it makes an arbitrary choice as to which of its two transition functions to apply For every input 𝑥, we say that M 𝑥 =1 if there exists some sequences of these choices (called nondeterministic choices) that would make M reach 𝑞 𝑎𝑐𝑐𝑒𝑝𝑡 on input 𝑥 Otherwise if every sequence of choices makes 𝑀 halt without reaching 𝑞 𝑎𝑐𝑐𝑒𝑝𝑡 then we say 𝑀 𝑥 =0 We say that 𝑀 runs in 𝑇 𝑛 -time if for every input 𝑥∈ 0,1 ∗ and every sequence of nondeterministic choices, 𝑀 reaches either the halting state or 𝑞 𝑎𝑐𝑐𝑒𝑝𝑡 within 𝑇( 𝑥 ) steps NTIME(f(n)) = Set of languages that can be decided by a NDTM in time 𝑐⋅𝑓(𝑛)

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞 𝑠𝑡𝑎𝑟𝑡 IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .

If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 . Example What language do we accept? Input: ⊳ 1 ∎ … Work/output: ⊳ 1 ∎ … State: 𝑞  Now normal deterministic execution… IF THEN Input symbol read Work (output) tape read Current state Move input head New work (output) symbol Move work head New state ⊳/0/1 ∎ 𝑞 𝑠𝑡𝑎𝑟𝑡 Right 0 for 𝛿 0 1 for 𝛿 1 Stay Left 𝑞 Once we reach state 𝑞, we run “normal” TM to check if number on work tape divides input. If then move to 𝑞 𝑎𝑐𝑒𝑝𝑡 . If not move to 𝑞 ℎ𝑎𝑙𝑡 .