FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY

PUSH-DOWN AUTOMATA AND CONTEXT-FREE GRAMMARS

Σ = {0, 1}, L = { 0 n 1 n | n ≥ 0 } Σ = {a, b, c, …, z}, L = { w | w = w R } Σ = { (, ) }, L = { balanced strings of parens } NONE OF THESE ARE REGULAR (), ()(), (()()) are in L, (, ()), ())(() are not in L madamimadam  L zeus sees suez  L L = { palindromes}

Σ = {a, b, c, …, z}, L = { w | w = w R } NONE OF THESE ARE REGULAR madamimadam  L zeus sees suez  L L = { palindromes} Σ = {0, 1}, L = { 0 n 1 n | n ≥ 0 }

PUSHDOWN AUTOMATA (PDA) FINITE STATE CONTROL STACK (Last in, first out) INPUT

a, b → c qiqi qjqj Input, pop, push In state q i reading symbol a from input Seeing b on top of stack, Replace b (pop) by c (push) and go to state q j If a = ε, make transition without reading input If b = ε, “ “ “ popping from stack If c = ε, “ “ “ writing on stack Non-deterministic

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε inputpoppush 0011 STACK$ $0 11 $ 0 1 Non-deterministic

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε inputpoppush 001 STACK$$0$ PDA that recognizes L = { 0 n 1 n | n ≥ 0 }

Definition: A (non-deterministic) PDA is a tuple P = (Q, Σ, Γ, , q 0, F), where: Q is a finite set of states Γ is the stack alphabet q 0  Q is the start state F  Q is the set of accept states Σ is the input alphabet  : Q  Σ ε  Γ ε → 2 Q  Γ ε 2 Q  Γ ε is the set of subsets of Q x Γ ε where Γ ε = Γ  {ε}

Let w  Σ* and suppose w can be written as w 1... w n where w i  Σ ε (recall Σ ε = Σ  {ε}) Then P accepts w if there are r 0, r 1,..., r n  Q and s 0, s 1,..., s n  Γ* (sequence of stacks) such that 1.r 0 = q 0 and s 0 = ε ( P starts in q 0 with empty stack) 2.For i = 0,..., n-1: (r i+1, b)   (r i, w i+1, a), where s i =at and s i+1 = bt for some a, b  Γ ε and t  Γ* (P moves correctly according to state, stack and symbol read) 3. r n  F (P is in an accept state at the end of its input)

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε q0q0 q1q1 q2q2 q3q3 Q = {q 0, q 1, q 2, q 3 }Γ =Σ =  : Q  Σ ε  Γ ε → 2 Q  Γ ε {0,1}{$,0,1}  (q 1,1,0) = { (q 2,ε) }  (q 2,1,1) = 

EVEN-LENGTH PALINDROMES Σ = {a, b, c, …, z} ε,ε → $ ε,ε → εε,ε → ε , → ε, → ε ε,$ → ε q0q0 q1q1 q2q2 q3q3 ,ε → ,ε →  Note: Also accepts the empty string

Build a PDA to recognize L = { a i b j c k | i, j, k ≥ 0 and (i = j or i = k) } ε,ε → $ b,a → ε ε,$ → ε q0q0 q5q5 q1q1 q3q3 a,ε → a q2q2 q4q4 ε,ε → ε q6q6 ε,$ → ε b,ε → ε c,a → ε c,ε → ε choose i=j choose i=k

CONTEXT-FREE GRAMMARS “Colorless green ideas sleep furiously.”

A → 0A1 A → B B → # CONTEXT-FREE GRAMMARS A variables terminals production rules start variable  0A1  (yields)  00A11  00B11  00#11 Non-deterministic A  * 00#11 (derives) Derivation We say: 00#11 is generated by the Grammar

CONTEXT-FREE GRAMMARS A → 0A1 A → B B → # A → 0A1 | B B → #

CONTEXT-FREE GRAMMARS A context-free grammar (CFG) is a tuple G = (V, Σ, R, S), where: V is a finite set of variables R is set of production rules of the form A → W, where A  V and W  (V  Σ)* S  V is the start variable Σ is a finite set of terminals (disjoint from V) L(G) = {w  Σ* | S  * w} Strings Generated by G

CONTEXT-FREE LANGUAGES A context-free grammar (CFG) is a tuple G = (V, Σ, R, S), where: V is a finite set of variables R is set of production rules of the form A → W, where A  V and W  (V  Σ)* S  V is the start variable Σ is a finite set of terminals (disjoint from V) G = { {S}, {0,1}, R, S }R = { S → 0S1, S → ε } L(G) =

CONTEXT-FREE LANGUAGES A context-free grammar (CFG) is a tuple G = (V, Σ, R, S), where: V is a finite set of variables R is set of production rules of the form A → W, where A  V and W  (V  Σ)* S  V is the start variable Σ is a finite set of terminals (disjoint from V) G = { {S}, {0,1}, R, S }R = { S → 0S1, S → ε } L(G) = { 0 n 1 n | n ≥ 0 } Strings Generated by G

WRITE A CFG FOR EVEN-LENGTH PALINDROMES S →  S  for all   Σ S → ε

WRITE A CFG FOR THE EMPTY SET G = { {S}, Σ, , S }

PARSE TREES A  0A1  00A11  00B11  00#11 A B 01# A A 01

→ + → x → ( ) → a Build a parse tree for a + a x a ax+a a a+xa a

Definition: a string is derived ambiguously in a context-free grammar if it has more than one parse tree Definition: a grammar is ambiguous if it generates some string ambiguously Can you give an unambiguous grammar with standard arithmetic precedence ?

Σ = {0, 1}, L = { 0 n 1 n | n ≥ 0 } WHAT ABOUT? But L is CONTEXT FREE NOT REGULAR Σ = {0, 1}, L 1 = { 0 n 1 n 0 m | m,n ≥ 0 } Σ = {0, 1}, L 2 = { 0 n 1 m 0 n | m,n ≥ 0 } Σ = {0, 1}, L 3 = { 0 m 1 n 0 n | m=n ≥ 0 } A → 0A1 A → ε

THE PUMPING LEMMA FOR CFGs Let L be a context-free language Then there is a P such that if w  L and |w| ≥ P 1. |v y| > 0 then can write w = u v x y z, where: 3. For every i ≥ 0, u v i x y i z  L 2. |v x y| ≤ P

THE PUMPING LEMMA FOR CFGs Let L be a context-free language Then there is a P such that if w  L and |w| ≥ P 1. |v y| > 0 then can write w = u v x y z, where: 3. For every i ≥ 0, u v i x y i z  L 2. |v x y| ≤ P WHAT ABOUT? Σ = {0, 1}, L 3 = { 0 m 1 n 0 n | m=n ≥ 0 }

Idea of Proof: If w is long enough, then any parse tree for w must have a path that contains a variable more than once T R R uvxzy T uz R R v y R R vxy

If the height of a parse tree is h, the length of the string generated by that tree is at most: Formal Proof: Let b be the maximum number of symbols on the right-hand side of any rule bhbh Let |V| be the number of variables in G Define P = b |V|+1 Let w be a string of length at least P Let T be a parse tree for w with a minimum number of nodes. T must have height at least |V|+1

Let T be a parse tree for w with a minimum number of nodes. T must have height at least |V|+1 The longest path in T must have ≥ |V|+1 variables Select R to be the variable that repeats among the lowest |V|+1 variables (in the path) T R R uvxzy T uz R R v y R R vxy 1. |vy| > 0 2. |vxy| ≤ P

THE PUMPING LEMMA FOR CFGs Let L be a context-free language Then there is a P such that if w  L and |w| ≥ P 1. |v y| > 0 then can write w = u v x y z, where: 3. For every i ≥ 0, u v i x y i z  L 2. |v x y| ≤ P WHAT ABOUT? Σ = {0, 1}, L 3 = { 0 m 1 n 0 n | m=n ≥ 0 }

PDAs ARE EQUIVALENT TO CFGs

A Language L is generated by a CFG  L is recognized by a PDA

A Language L is generated by a CFG  L is recognized by a PDA Suppose L is generated by a CFG G = (V, Σ, R, S) Construct P = (Q, Σ, Γ, , q, F) that recognizes L

Suppose L is generated by a CFG G = (V, Σ, R, S) Construct P = (Q, Σ, Γ, , q, F) that recognizes L ε,ε → S$ ε,$ → ε For each rule 'A → w’  R: For each terminal a  Σ: ε,A → w R a,a → ε

S → aTb T → Ta | ε ε,ε → $ ε,$ → ε ε,ε → S ε,S → b ε,ε → T ε,T → a ε,ε → a ε,ε → T ε,T → ε a,a → ε b,b → ε

S → aTb T → Ta | ε ε,ε → $ ε,$ → ε ε,ε → S ε,S → b ε,ε → T ε,T → a ε,ε → a ε,ε → T ε,T → ε a,a → ε b,b → ε S  * ab (derives)

Suppose L is generated by a CFG G = (V, Σ, R, S) Describe P = (Q, Σ, Γ, , q, F) that recognizes L : (1) Push $ and then S on the stack (2) Repeat the following steps forever: (b) If X is a variable A, guess a rule that matches A and push result into the stack (c) If X is a terminal, read next symbol from input and compare it to terminal. If they’re different, reject. (d) If X is $: then accept iff no more input (a) Pop the stack, call the result X.

A Language L is generated by a CFG  L is recognized by a PDA

A Language L is generated by a CFG  L is recognized by a PDA Given PDA P = (Q, Σ, Γ, , q, F) Construct a CFG G = (V, Σ, R, S) such that L(G) = L(P) First, simplify P to have the following form: (1) It has a unique accept state, q acc (2) It empties the stack before accepting (3) Each transition either pushes a symbol or pops a symbol, but not both at the same time

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε SIMPLIFY q0q0 q1q1 q2q2 q3q3

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε SIMPLIFY q0q0 q1q1 q2q2 q3q3 ε,ε → ε ε,E → ε q4q4 q5q5 Q ε,ε → E ε,σ → ε

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε SIMPLIFY q0q0 q1q1 q2q2 q3q3 ε,ε → D ε,E → ε q4q4 q5q5 Q ε,ε → E ε,σ → ε q’ 0 q’ 3 ε,D → ε

V = {A pq | p,q  Q } S = A q 0 q acc Idea For Our Grammar G: For every pair of states p and q in PDA P, G will have a variable A pq whose production rules will generate all strings x that can take: P from p with an empty stack to q with an empty stack

A q 0 q 1 generates?  A q 1 q 2 generates? {0 n 1 n | n > 0} A q 1 q 3 generates?  ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε q0q0 q1q1 q2q2 q3q3 ε,ε → D ε,E → ε q4q4 q5q5 Q ε,ε → E ε,σ → ε q’ 0 q’ 3 ε,D → ε WANT: A pq generates all strings that take p with an empty stack to q with empty stack

A q 0 q 1 generates?  A q 1 q 2 generates? {0 n 1 n | n > 0} A Qq 5 generates? ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε q0q0 q1q1 q2q2 q3q3 ε,ε → D ε,E → ε q4q4 q5q5 Q ε,ε → E ε,σ → ε q’ 0 q’ 3 ε,D → ε WANT: A pq generates all strings that take p with an empty stack to q with empty stack

Let x be such a string P’s first move on x must be a push P’s last move on x must be a pop Two possibilities: 1. The symbol popped at the end is exactly the one pushed at the beginning 2. The symbol popped at the end is not the one pushed at the beginning

stack height input string pq A pq → aA rs b rs 1. The symbol t popped at the end is exactly the one pushed at the beginning ba push t pop t x = ayb takes p with empty stack to q with empty stack δ(p, a, ε) → (r, t) δ(s, b, t) → (q, ε) ─ ─ ─ ─ x ─ ─ ─ ─

stack height input string prq A pq → A pr A rq 2. The symbol popped at the end is not the one pushed at the beginning

V = {A pq | p, q  Q } S = A q 0 q acc Formally: For every p, q, r, s  Q, t  Γ and a, b  Σ ε If (r, t)   (p, a, ε) and (q, ε)   (s, b, t) Then add the rule A pq → aA rs b For every p, q, r  Q, add the rule A pq → A pr A rq For every p  Q, add the rule A pp → ε

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε q0q0 q1q1 q2q2 q3q3

ε,ε → $ 0,ε → 0 1,0 → ε ε,$ → ε q0q0 q1q1 q2q2 q3q3 L(P) = ? G(P) = ?

A pq generates x  x can bring P from p with an empty stack to q with an empty stack  Proof (by induction on the number of steps in the derivation of x from A pq ): Base Case:The derivation has 1 step: A pp  ε Inductive Step: Assume true for derivations of length ≤ k and prove true for derivations of length k+1: A pq  * x in k+1 steps For all x,

A pq generates x  x can bring P from p with an empty stack to q with an empty stack  Proof (by induction on the number of steps in the derivation of x from A pq ): Assume true for derivations of length ≤ k and prove true for derivations of length k+1: A pq  * x in k+1 steps so x = yz where A pr  * y & A rq  * z Case 1: First step in derivation: A pq → A pr A rq  * x For all x, Inductive Step: Now use induction hypothesis

A pq generates x  x can bring P from p with an empty stack to q with an empty stack  Proof (by induction on the number of steps in the derivation of x from A pq ): Assume true for derivations of length ≤ k and prove true for derivations of length k+1: A pq  * x in k+1 steps so x = ayb where A rs  * y Case 2: A pq → aA rs b For all x, Inductive Step: By induction, y can bring P from r with empty stack to s with empty stack (or with any symbol t on top stack to t on top stack).

A pq generates x  x can bring P from p with an empty stack to q with an empty stack  Proof (by induction on the number of steps in the derivation of x from A pq ): Assume true for derivations of length ≤ k and prove true for derivations of length k+1: A pq  * x in k+1 steps so x = ayb where A rs  * y Case 2: A pq → aA rs b For all x, Inductive Step: (r,t)   (p,a,ε) and (q, ε)   (s,b,t) statepushstatealphabet Because A pq → aA rs b is a rule,there is a t: pop

A pq generates x  x can bring P from p with an empty stack to q with an empty stack  Proof (by induction on the number of steps in the computation of P from p to q with empty stacks, on input x): Base Case:The computation has 0 steps So it starts and ends in the same state: We must show that A pp  * x But it must be that x = ε, so we are done For all x,

Inductive Step: Assume true for computations of length ≤ k, we’ll prove true for computations of length k+1 Suppose that P has a computation where x brings p to q with empty stacks in k+1 steps Two cases: 1. The stack is empty somewhere in the middle of the computation, say at state r. Write x as yz (where the stack is empty after reading y from p to r and after reading z from r to q). So, A pr  * y, A rq  * z by I.H. Recall, A pq → A pr A rq is a rule (by construction). So A pq → A pr A rq → yA rq → yz. So A pq  * x

Inductive Step: Assume true for computations of length ≤ k, we’ll prove true for computations of length k+1 Suppose that P has a computation where x brings p to q with empty stacks in k+1 steps 2. The stack is empty only at the beginning and the end of this computation Write x as ayb. There must be states r (immediately following p) and s (immediately before q) where y goes from r on empty stack to s on empty stack. p r…s q (why?) So, A rs  * y by I.H. Now, P must push some t at p and pop t at s. (why?) So grammar must have rule A pq → aA rs b. So A pq  * x Two cases:

A Language L is generated by a CFG  L is recognized by a PDA

Corollary: Every regular language is context-free