CSC 3130: Automata theory and formal languages Andrej Bogdanov The Chinese University of Hong Kong Undecidable problems for CFGs and descriptive complexity Fall 2008

Decidable vs. undecidable “ TM M accepts w” “ TM M accepts some input ” “ TM M and M’ accept same inputs ” “ TM M accepts all inputs ” undecidable “ TM M halts on w” “ PDA M accepts w” “ DFA M accepts w” decidable “PDA P accepts all inputs” “CFG G is ambiguous” other kinds of problems? ? “ DFA M accepts all inputs ”

›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐ Computation is local lotus ootus oktus okrus okras okra M q6q6 q0q0 q3q3 q0q0 q1q1 q acc The changes between rows occur in a 2x3 window computation tableau

Computation histories as strings If M halts on w, We can represent the computation tableau by a string t over alphabet  ∪ Q ∪ {#, ›} ›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐ ›q 0 lotus#›oq 6 otus#...#›okrq a a ☐ # M accepts wq a occurs in string t M rejects w q a does not occur in t

Undecidable problems for PDAs Theorem Proof: We will show that ALL PDA = { 〈 P 〉 : P is a PDA that accepts all inputs } The language ALL PDA is undecidable. If ALL PDA can be decided, so can A TM.

Undecidable problems for PDAs 〈 M 〉, w reject if M accepts w accept if M rej/loops w A reject if not accept if P accepts all inputs 〈P〉〈P〉 A 〈P〉〈P〉 P accepts all inputs if M rejects or loops on w P does not accept some input if M accepts w

Undecidability via computation histories P accepts all inputs if M rejects or loops on w P does not accept some input if M accepts w P candidate computation history of M on w reject accepting histories ›q 0 lotus#›oq 6 otus#...#›okrq a a ☐ reject accept every other string M accepts wP rejects t M rej/loops on w no accepting histories P accepts everything

Undecidability via computation histories Task: Design a PDA P such that P candidate computation history t of M on w reject accepting histories ›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐ Expect t of the form w 1 #w 2 #...#w k # If w 1 ≠ ›q 0 w, accept t. If t does not contain q a, accept t. If two consecutive blocks w i #w i+1 do not correspond to a proper transition of M, accept t.

Implementing P If w 1 ≠ ›q 0 w, accept t. If two consecutive blocks w i #w i+1 do not represent a valid transition of M, accept t. On input t : Nondeterministically make one of the following choices If t does not contain q a, accept t. Look for the beginning of the i th block of t Look in the first block w 1 of t Look for the appearance of q a › 0 o 0 k 6 q 3 t 0 u 0 s # › 0 o 0 k 6 r 0 q 0 u 0 s 0 w i #w i+1 represents a valid transition if all 3x2 windows correspond to possible transitions of M valid transition

Valid and invalid windows … 6 c 3 a 0 t 0 … … 0 c 6 a 0 p 0 … 0 … 6 t 3 t 0 u 0 … … 0 t 6 t 0 u 0 … 0 valid window invalid window … 6 t 3 t 0 u 0 … … 0 t 6 t 0 q 3 … 0 valid window … 6 t 3 q 3 u 0 … … 0 t 6 a 0 q 7 … 0 valid if  (q 3, u ) = (q 7, a, R) … 6 q 3 t 0 u 0 … … 0 k 6 t 0 q 0 … 0 invalid window … 6 c 3 a 0 t 0 … … 0 b 6 a 0 t 0 … 0 valid window

Implementing P To check this it is better to write t in boustrophedon w i # w i+1 represent a valid transition of M ›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐›0q0l0o0t0u0s0›0o0q6o0t0u0s0›0o0k6q3t0u0s›0o0k6r0q0u0s0›0o0k6r0a0q1s›0o0k6r0qaa0☐ ›q 0 lotus#›oq 6 otus#...#›okrq a a ☐ # ›q 0 lotus#sutoq 6 o›#...#›okrq a a ☐ # Alternate rows are written in reverse

Implementing P › 0 o 0 k 6 q 3 t 0 u 0 s # › 0 o 0 k 6 r 0 q 0 u 0 s 0 proper transition … #›okq 3 tus#suq 0 rko›# … wiwi w i+1 Nondeterministically look for beginning of 3x2 window w i # w i+1 represent a valid transition of M # Remember first row of window in state Use stack to detect beginning of second row Remember second row of window in state If window is not valid, accept, otherwise reject.

The Post Correspondence Problem Input: A set of tiles like this Given an infinite supply of such tiles, can you match top and bottom? bab cc c ab a ab baa a bab cc c ab a ab baa a c ab a baba a baba bab  bab 

Undecidability of PCP Theorem Proof: We will show that PCP = {D: D is a collection of tiles that contains a top-bottom match } The language PCP is undecidable. If PCP can be decided, so can A TM.

Undecidability of PCP Idea: Matches represent accepting histories 〈M〉,w〈M〉,w T (collection of tiles) If M accepts w, then T can be matched If M rej/loops on w, then T cannot be matched ›q 0 lotus#›oq 6 otus#›okq 3 t...#›q a ☐☐☐☐ ›q 0 lotus#›oq 6 otus#›okq 3 r...#›q a ☐☐☐☐  ›q 0 lotus# ›q 0 l ›oq 6 tttt uuuu ssss #### ›››› oq 6 0 okq 3 oooo …

Some technicalities We will assume that –Before accepting, TM M erases its tape –One of the PCP tiles is marked as a starting tile These assumptions can be made without loss of generality (we will see why later) bab cc c ab a ab baa a baba s

Undecidability of PCP To decide A TM, we construct these tiles for PCP 〈M〉,w〈M〉,w T (collection of tiles) If M accepts w, then T can be matched If M rej/loops on w, then T cannot be matched  ›q 0 w# s a1qia3b1b2b3a1qia3b1b2b3 for each valid window of this form aaaa for all a in  ∪ {#, ›} # ›q a  ☐##☐## ☐☐ “final” tiles

›q 0 lotus#›oq 6 otus#...#›oq 1 ☐☐☐ #›q a ☐☐☐☐ Undecidability of PCP ›q 0 lotus#›oq 6 otus#...#›oq 1 ☐☐☐ #›q a ☐☐☐☐  ›q 0 w# s a1qia3b1b2b3a1qia3b1b2b3 aaaa # ›q a  ☐##☐## ☐☐ accepting computation history

Undecidability of PCP If M rejects on input w, then qr appears on bottom at some point, but it cannot be matched on top If M loops on w, then matching keeps going forever  ›q 0 w# s a1qia3b1b2b3a1qia3b1b2b3 aaaa # ›q a  ☐##☐## ☐☐

A technicality We assumed that one tile marked as starting tile We can remove assumption by changing tiles a bit bab cc c ab a baba s b*a*b* *c*c c* *a*b *a* *b*a*b*a ** “starting tile” begins with * “ending tile” matches last * “middle tiles”

Ambiguity of CFGs AMB = {G: G is an ambiguous CFG } Theorem Proof: We will show that The language AMB is undecidable. If AMB can be decided, so can PCP.

Ambiguity of CFGs Proof: Step 1: Number the tiles TG If T can be matched, then G is ambiguous If T cannot be matched, then G is unambiguous (collection of tiles) bab cc c ab a ab 1 23 (CFG)

Ambiguity of CFGs TG (collection of tiles) bab cc c ab a ab 1 23 Productions: T → bab T 1 Terminals: B → cc S 1 T → c T 2 B → ab B 2 S → a T 3 B → ab B 3 a,b,c,1,2,3 (CFG) Variables: S, T, B T →  B →  S → T | B

Ambiguity of CFGs Each sequence of tiles gives two derivations If the tiles match, these two derive the same string bab cc c ab 1 2 c ab 2 S → T → bab T 1 → babc T2 1 → babcc 22 1 S → B → cc B 1 → ccab B2 1 → ccabab 22 1

Ambiguity of CFGs Argue by contradiction: –If G is ambiguous then ambiguity must look like this TG If T can be matched, then G is ambiguous If T cannot be matched, then G is unambiguous (collection of tiles)(CFG) ✓ S T T a1a1 n1n1 aiai nini T a2a2 n2n2 … S B B b1b1 m1m1 bjbj mjmj B b2b2 m2m2 … Then n 1...n i = m 1 …m j So there is a match a1b1a1b1 a2b2a2b2 aibiaibi n1n1 n2n2 nini ✓ …

Descriptive complexity

Roulette In a game of roulette, you bet $1 on even or odd The outcome is a number between 1 and 36 –If you guessed correctly, double your bet –Otherwise, you lose

Randomness If we write E for even, O for odd, what we saw is It seems the wheel is crooked. If it wasn’t we would expect something more like But both sequences have same probability! Why does one appear less random than the other? OEOEOEOEOEOEOEOEOEOE OOOEEOEOOEOEOOOEEEOE

Turing Machines with output The goal of a Turing Machine with output is to write something on the output tape and go into state q halt M output tape … 010 work tape … 010

Descriptive complexity The descriptive complexity K(x) of x is the shortest description of any Turing Machine that outputs x We will assume x is long Andrey Kolmogorov ( )

Example of descriptive complexity Turing machine implementation: x = “OE...OE” = (OE) n Repeat for n steps: At odd step print O At even step print E Write n in binary on work tape While work tape not equal to 0, Subtract 1 from number on work tape If number is odd, write O If number is even, write E (n = 1,000,000,000) ≈ log 2 n states ≈ 3 states ≈ 15 states ≈ 2 states ≈ log 2 n + 20 K(x)

Bounds on descriptive complexity Theorem 1 Proof: Let x = x 1...x n and consider the following TM: For every x of length n, K(x) is at most O(n) Write x 1 to output tape and move right Write x 2 to output tape and move right Write x n to output tape and halt.... n + O(1)

Descriptive complexity and randomness Theorem 2 For 99% of strings of length n, K(x) ≥ n – 10. 0O(log n) n – 10 “simple” strings , OEOE...OE, , “random-looking” strings n  O(1) “randomness-deficient” strings

Evaluating randomness How do we know if the casino is crooked? Idea: Compute K(sequence). If much less than n, indicates sequence is not random

Computing descriptive complexity Proof: Suppose it is, fix n and consider this TM M : Let x = output of M, then So (when n is large) we get K(x) > K(x), impossible! It is not possible to compute K(x). Output the first x of length n (in lexicographic order) such that K(x) ≥ n – 10 K(x) ≥ n – 10 K(x) ≤  〈 M 〉 | = log 2 n + O(1) but