1. FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

2. Homework 2 Grades
35 homeworks handed in.
Histogram:
40 | **
50 | *
60 | *****
70 | ***
80 | **********
90 | *************
100 | *
mean=81.8, median=86

3. REVIEW for MIDTERM 1
THURSDAY Feb 7

4. Midterm 1 will cover everything we have seen so far
The PROBLEMS will be similar to Sipser in Chapter 1, 2, 3 and HWs 1, 2, 3.
It will be Closed-Book, Closed-Everything

5. 1. Deterministic Finite Automata and Regular Languages
2. Non-Deterministic Finite Automata
3. Pumping Lemma for Regular Languages; Regular Expressions
4. Minimizing DFAs
5. PDAs, CFGs; Pumping Lemma for CFLs
6. Equivalence of PDAs and CFGs
7. Chomsky Normal Form
8. Turing Machines

6. Context-Free Grammars
Machines
Syntactic Rules
DFAs
Regular Expressions
NFAs
Context-Free Grammars
PDAs

7. THE REGULAR OPERATIONS
Union: A  B = { w | w  A or w  B }
Intersection: A  B = { w | w  A and w  B }
Negation: A = { w  Σ* | w  A }
Reverse: AR = { w1 …wk | wk …w1  A }
Can A* ever be the empty set? Why or why not?
Concatenation: A  B = { vw | v  A and w  B }
Star: A* = { s1 … sk | k ≥ 0 and each si  A }

8. REGULAR EXPRESSIONS  is a regexp representing {}
If R1 and R2 are regular expressions representing L1 and L2 then:
(R1R2) represents L1  L2
We give an Inductive Definition of Reg Exp
(R1  R2) represents L1  L2
(R1)* represents L1*

9. How can we test if two regular expressions are the same?
Length n R1 R2
O(n) states N1 N2
O(2n) states M1 M2
M1 MIN ?=
M2 MIN

10. THEOREMS and CONSTRUCTIONS

11. (for Regular Languages)
THE PUMPING LEMMA
(for Regular Languages)
Let L be a regular language with |L| = 
Then there is an integer P such that
if w  L and |w| ≥ P
then can write w = xyz, where:
1. |y| > 0
2. |xy| ≤ P
3. xyiz  L for any i ≥ 0 y z x

12. then can write w = uvxyz, where:
THE PUMPING LEMMA
(for Context Free Grammars)
Let L be a context-free language with |L| = 
Then there is an integer P such that
if w  L and |w| ≥ P
then can write w = uvxyz, where: T u z R v y x
1. |vy| > 0 T R u v x z y
2. |vxy| ≤ P
3. uvixyiz  L, for any i ≥ 0

13. F = { R  Q | f  R for some f  F }
CONVERTING NFAs TO DFAs
Input: NFA N = (Q, Σ, , Q0, F)
Output: DFA M = (Q, Σ, , q0, F)
Q = 2Q
 : Q  Σ → Q
(R,) =  ε( (r,) )
rR *
The transition function has to simulate all the possible transitions that can be taken out of all states in the set R.
Its output will again be a new set of states:
all the possible states you could be in after being in one of the input states and reading the input symbol.
The eps-closure is necessary to take care of the effect of eps-transitions.
q0 = ε(Q0)
F = { R  Q | f  R for some f  F } *
For R  Q, the ε-closure of R, ε(R) = {q that can be reached from some r  R by traveling along zero or more ε arrows}

14. N Given: NFA N = ( {1,2,3}, {a,b},  , {1}, {1} )
Construct: Equivalent DFA M
M = (2{1,2,3}, {a,b}, , {1,3}, …) N a a 1 b
{1,3}
{3}  a b b a b b
a,b ε
a , b
{2} 2 3 a
{2,3} b a a
{1}, {1,2} ?
ε({1}) = {1,3}
{1,2,3}

15. L can be represented by a regexp
EQUIVALENCE
L can be represented by a regexp 
L is a regular language

16. L can be represented by a regexp Induction on the length of R:
L is a regular language 
Induction on the length of R:
Base Cases (R has length 1): 
R = 
R = ε
R = 

17. Assume R has length k > 1, and that every regexp of length < k
Inductive Step:
Assume R has length k > 1,
and that every regexp of length < k
represents a regular language
Three possibilities for what R can be:
R = R1  R2
(Closure under Union)
R = R1 R2
(Closure under Concat.)
(Closure under Star)
R = (R1)*
Therefore: L can be represented by a regexp
 L is regular

18.  L is a regular language  L can be represented by a regexp
Proof idea: Transform an NFA for L into a regular expression by removing states and re-labeling the arrows with regular expressions
Add unique and distinct start and accept states ε

19. NFA ε While machine has more than 2 states:
Pick an internal state, rip it out and
re-label the arrows with regexps,
to account for the missing state 1
01*0

20. q1 b a ε q2
a,b ε q0 q3
a*b
(a*b)(ab)*
R(q0,q3) = (a*b)(ab)*

21. Transform (1(0  1))* to an NFAε 1
1,0 ε

22. THEOREM
For every regular language L, there exists a UNIQUE (up to re-labeling of the states) minimal DFA M such that L = L(M)

23. EXTENDING  Given DFA M = (Q, Σ, , q0, F) extend 
to  : Q  Σ* → Q as follows: ^ ^
(q, ε) = q ^
(q, ) =
(q, ) ^ ^
(q, w1 …wk+1 ) = ( (q, w1 …wk ), wk+1 )
Note: (q0, w)  F  M accepts w
String w  Σ* distinguishes states q1 and q2 iff ^ ^
exactly ONE of (q1, w), (q2, w) is a final state

24. Fix M = (Q, Σ, , q0, F) and let p, q, r  Q
Definition:
p ~ q iff p is indistinguishable from q
p ~ q iff p is distinguishable from q /
Proposition: ~ is an equivalence relation
p ~ p (reflexive)
p ~ q  q ~ p (symmetric)
p ~ q and q ~ r  p ~ r (transitive)

25. Proposition: ~ is an equivalence relation
so ~ partitions the set of states of M into disjoint equivalence classes
[q] = { p | p ~ q } q

26. 1

27. (1) DM = { (p,q) | p,q  Q and p ~ q }
TABLE-FILLING ALGORITHM
Input: DFA M = (Q, Σ, , q0, F)
Output:
(1) DM = { (p,q) | p,q  Q and p ~ q } /
(2) EM = { [q] | q  Q }
We know how to find those pairs of states that ε distinguishes…
Use this and recursion to find those pairs distinguishable with longer strings
Pairs left over will be indistinguishable
IDEA:
It is easy to find those states that are inaccessible. What about the pairs of states that are distinguishable?
To find them we’ll give an algorithm we call the “table-filling algorithm”

28. (1) DM = { (p,q) | p,q  Q and p ~ q }
TABLE-FILLING ALGORITHM
Input: DFA M = (Q, Σ, , q0, F)
Output:
(1) DM = { (p,q) | p,q  Q and p ~ q } /
(2) EM = { [q] | q  Q } q0 q1 qi qn
Base Case: p accepts
and q rejects  p ~ q /
Recursion: if there is σ  Σ
and states p, q satisfying D D
To find pairs of states that are distinguishable by longer strings, we do a recursion step.
So p and q are distinguishable, if by reading a symbol we get to another pair of states that we know to be distinguishable.
We just apply this rule until we can’t any longer.
Why does this algorithm stop?
 (p, ) = p D ~ / 
p ~ q /
 (q, ) = q
Repeat until no more new D’s

29. (1) Remove all inaccessible states from M
Algorithm MINIMIZE
Input: DFA M
Output: DFA MMIN
(1) Remove all inaccessible states from M
(2) Apply Table-Filling algorithm to get
EM = { [q] | q is an accessible state of M }
MMIN = (QMIN, Σ, MIN, q0 MIN, FMIN)
QMIN = EM, q0 MIN = [q0], FMIN = { [q] | q  F }
MIN( [q], ) = [ ( q, ) ]
Claim: MMIN  M

30. A Language L is generated by a CFG
L is recognized by a PDA

31. Suppose L is generated by a CFG G = (V, Σ, R, S)
Construct P = (Q, Σ, Γ, , q, F) that recognizes L
ε,ε → $S
For each rule 'A → w’  R:
ε,A → wR
For each terminal a  Σ:
a,a → ε
Here’s a “sketch” of what the PDA P will look like.
We start by pushing $ on the stack, and then we push start variable S.
When $ is at the top of the stack again, we will accept if there’s no more input.
In the meantime, we want to add transitions of the following kind.
The top of the stack will be either a variable or a terminal.
For each variable A, and production rule involving A,
if we pop A we’ll push the symbols in w on the stack, in reverse.
For each terminal a,
if it matches the current input symbol, we pop it, otherwise reject.
ε,$ → ε

32. S → aTb T → Ta | ε ε,ε → $ ε,ε → T ε,S → b ε,ε → T ε,ε → S ε,T → a
First we push $, then S.
If we pop S, then we push aTb on the stack, in reverse.
If we pop T, we have two choices: we can push Ta, or push nothing.
We pop a letter if it corresponds to the input symbol.
Anything else, we reject.
We accept if we reach $ again, and there is no more input.
ε,$ → ε
ε,ε → a
ε,T → ε
a,a → ε
b,b → ε

33. A Language L is generated by a CFG
L is recognized by a PDA
Given PDA P = (Q, Σ, Γ, , q, F)
Construct a CFG G = (V, Σ, R, S) such that
L(G) = L(P)
First, simplify P to have the following form:
(1) It has a single accept state, qaccept
(2) It empties the stack before accepting
(3) Each transition either pushes a symbol or pops a symbol, but not both at the same time

34. SIMPLIFY
ε,ε → $ q0 q1
0,ε → 0
1,0 → ε
ε,$ → ε q3 q2
1,0 → ε

35. SIMPLIFY ε,ε → E ε,ε → $ Q q0 q1 0,ε → 0 ε,ε → D 1,0 → ε q’0 ε,$ → ε
Finally, to make sure that there is always a symbol popped or pushed onto the stack…
we add an additional state that pushes and pops a dummy symbol
ε,D → ε q4
ε,E → ε q5
ε,σ → ε

36. V = {Apq | p,qQ } S = Aq0qacc
Idea For Our Grammar G:
For every pair of states p and q in PDA P,
G will have a variable Apq which generates all strings x that can take:
P from p with an empty stack
to q with an empty stack
So if we can generate a string from the start variable,
this means the string will take P from start state with an empty stack
to final state with an empty stack!
I.e. it accepts!
V = {Apq | p,qQ }
S = Aq0qacc

37. x = ayb takes p with empty stack to q with empty stack
1. The symbol t popped at the end is exactly the one pushed at the beginning
stack
height r s a b
input
string
push t
pop t
Suppose we have a string x, that begins with letter a, ends with letter b.
We can chart the height of the stack over time as a graph. Each symbol of the input is on the x-axis.
Suppose we run the PDA from state p, and read x, then some computation path ends with q in empty stack.
Suppose after we read input a, we push t and go to state r.
Then after reading y, we are in some state s, and the stack is the same as before (it just has t on it).
Finally, popping t and reading b, we are in q.
How can we capture this behavior with a production rule? p
─ ─ ─ ─ x ─ ─ ─ ─ q
δ(p, a, ε) → (r, t)
Apq → aArsb
δ(s, b, t) → (q, ε)

38. 2. The symbol popped at the end is not the one pushed at the beginning
stack
height p
input
string r q
Let r be the state the PDA is in when it emptied the stack completely.
Then we’d have Apq -> Apr Arq
From p with an empty stack, we get to r with an empty stack,
then from r with an empty stack we get to q with an empty stack.
This is just like going from p to q with empty stacks.
Apq → AprArq

39. For every p, q, r, s  Q, t  Γ and a, b  Σε
Formally:
V = {Apq | p, qQ }
S = Aq0qacc
For every p, q, r, s  Q, t  Γ and a, b  Σε
If (r, t)  (p, a, ε) and (q, ε)  (s, b, t)
Then add the rule Apq → aArsb
For every p, q, r  Q,
add the rule Apq → AprArq
For every p  Q,
add the rule App → ε

40. THE CHOMSKY NORMAL FORM
A context-free grammar is in Chomsky normal form if every rule is of the form:
A → BC
B, C are variables (not the start var)
a is a terminal
A → a
S → ε
S is the start variable
S → TU
T → 0
U → SV | 1
S0 → TU | ε
V → 1
S → 0S1
S → TT
T → ε

41. “Can transform any CFG into Chomsky normal form”
Theorem: If G is in CNF, w  L(G) and |w| > 0, then any derivation of w in G has length 2|w| - 1
Theorem: Any context-free language can be generated by a context-free grammar in Chomsky normal form
“Can transform any CFG into Chomsky normal form”

42. Theorem: Any CFL can be generated by a CFG in Chomsky normal form
Algorithm:
1. Add a new start variable (S0S)
2. Eliminate all Aε rules: For each occurrence of A on the RHS of a rule, add a new rule that removes that occurrence (unless this new rule was previously removed)
3. Eliminate all AB rules: For each rule with B on LHS of a rule, add a new rule that puts A on the LHS instead (unless this new rule was previously removed)
4. Convert Au1u2... uk to A u1A1, A1u2A2, ...
If ui is a terminal, replace ui with Ui and add Uiui
The high level idea is to just remove those rules that aren’t in the proper form,
and replace them with equivalent rules that are in the proper form.

43. Convert the following into Chomsky normal form:
A → BAB | B | ε
B → 00 | ε
A → BAB | B | BB | AB | BA
B → 00
S0 → A | ε
A → BAB | B | ε
B → 00 | ε
S0 → A
A → BAB | 00 | BB | AB | BA
B → 00
S0 → BAB | 00 | BB | AB | BA | ε
1. Get rid of epsilon-productions.
2. Get rid of the unit productions.
3. Then clean up the remaining rules so they have two variables or one symbol on each RHS.
S0 → BC | DD | BB | AB | BA | ε, C → AB,
A → BC | DD | BB | AB | BA , B → DD, D → 0

44. FORMAL DEFINITIONS

45. A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, , q0, F)
deterministic
DFA
A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, , q0, F)
Q is the set of states (finite)
Σ is the alphabet (finite)
 : Q  Σ → Q is the transition function
q0  Q is the start state
F  Q is the set of accept states
Let w1, ... , wn  Σ and w = w1... wn  Σ*
Then M accepts w if there are r0, r1, ..., rn  Q, s.t.
r0=q0
(ri, wi+1 ) = ri+1, for i = 0, ..., n-1, and
rn  F

46. A non-deterministic finite automaton (NFA) is a 5-tuple N = (Q, Σ, , Q0, F)
Q is the set of states
Σ is the alphabet
 : Q  Σε → 2Q is the transition function
Q0  Q is the set of start states
F  Q is the set of accept states
2Q is the set of all possible subsets of Q
Σε = Σ  {ε}

47. A language L is recognized by an NFA N
Let w Σ* and suppose w can be written as
w1... wn where wi  Σε (ε = empty string)
Then N accepts w if there are r0, r1, ..., rn  Q such that
r0  Q0
ri+1  (ri, wi+1 ) for i = 0, ..., n-1, and
rn  F
L(N) = the language recognized by N
= set of all strings machine N accepts
A language L is recognized by an NFA N
if L = L(N).

48. PUSHDOWN AUTOMATA (PDA)
FINITE STATE CONTROL
STACK
(Last in,
first out)
INPUT
A Pushdown Automaton is an NFA armed with a stack.
On a single step, it reads the current symbol from the input
and it pops the symbol on the top of the stack.
Based on its state, and these two symbols, it moves to a new state, moves its input head to the right,
and pushes a new symbol onto the stack.
(It’s possible for it to pop nothing, or push nothing, it depends on the rules in its transition function.)

49. string
pop
push
ε,ε → $
0,ε → 0
1,0 → ε
ε,$ → ε
1,0 → ε

50. 2Q is the set of subsets of Q and Σε = Σ  {ε}
Definition: A (non-deterministic) PDA is a tuple P = (Q, Σ, Γ, , q0, F), where:
Q is a finite set of states
Σ is the input alphabet
Γ is the stack alphabet
 : Q  Σε  Γε → 2 Q  Γε
q0  Q is the start state
F  Q is the set of accept states
2Q is the set of subsets of Q and Σε = Σ  {ε}

51. Let w Σ* and suppose w can be written as
w1... wn where wi  Σε (recall Σε = Σ  {ε})
Then P accepts w if there are
r0, r1, ..., rn  Q and
s0, s1, ..., sn  Γ* (sequence of stacks) such that
r0 = q0 and s0 = ε (P starts in q0 with empty stack)
For i = 0, ..., n-1:
(ri+1 , b) (ri, wi+1, a), where si =at and si+1 = bt for some a, b  Γε and t  Γ*
(P moves correctly according to state, stack and symbol read)
3. rn  F (P is in an accept state at the end of its input)
The definition of acceptance is more complicated too!
2. Suppose we’re in state ri, reading wi+1, and we popped stack symbol a.
And suppose that one possible transition is to move to ri+1 and push symbol b.

52. q0 q1 q3 q2 Q = {q0, q1, q2, q3} Σ = {0,1} Γ = {$,0,1}
ε,ε → $ q0 q1
0,ε → 0
1,0 → ε
ε,$ → ε q3 q2
1,0 → ε
Q = {q0, q1, q2, q3}
Σ =
{0,1}
Γ =
{$,0,1}
So, to give a formal definition of this PDA, we could say…
 : Q  Σε  Γε → 2 Q  Γε
(q1,1,0) =
{ (q2,ε) }
(q2,1,1) = 
(q2,ε,$) = { (q3,ε) }

53. L(G) = {w  Σ* | S * w} Strings Generated by G
CONTEXT-FREE GRAMMARS
A context-free grammar (CFG) is a tuple
G = (V, Σ, R, S), where:
V is a finite set of variables
Σ is a finite set of terminals (disjoint from V)
R is set of production rules of the form A → W, where A  V and W  (VΣ)*
S  V is the start variable
L(G) = {w  Σ* | S * w} Strings Generated by G
G = { {S}, {0,1}, R, S }
R = { S → 0S1, S → ε }
L(G) = { 0n1n | n ≥ 0 } Strings Generated by G

54. TURING MACHINE FINITE STATE CONTROL q1 q0 A I N P U T INFINITE TAPE
That concludes our discussion of context-free languages.
Now we’ll shift to a much more general computational model, called the Turing machine.
Named after its inventor…
It has a finite control, like that of a DFA, but it can read and write to its input, which we think of as being presented on an infinite tape.
Only a finite part of the tape has input symbols on it, and the rest of the tape is filled with special “blank” symbols in both directions.
This pointer here is the TAPE HEAD. It begins at the leftmost input symbol.
Here’s what a “step” of a Turing machine looks like:
from a state q0, it will read the symbol under the head, write a symbol, change its state, and move its head left or right by one. A I N P U T
INFINITE TAPE

55. read write move  → , R qaccept  → , R qreject

56. Definition: A Turing Machine is a 7-tuple
T = (Q, Σ, Γ, , q0, qaccept, qreject), where:
Q is a finite set of states
Σ is the input alphabet, where   Σ
Γ is the tape alphabet, where   Γ and Σ  Γ
 : Q  Γ → Q  Γ  {L,R}
q0  Q is the start state
qaccept  Q is the accept state
qreject  Q is the reject state, and qreject  qaccept

57. A language L is called Turing-recognizable or recursively enumerable
A TM recognizes a language iff it accepts all and only those strings in the language
A language L is called Turing-recognizable or recursively enumerable
iff some TM recognizes L
A TM decides a language L iff it accepts all strings in L and rejects all strings not in L
Now, since a TM can run forever on some inputs,
we have two different ways for a TM to “compute” a language.
A language L is called decidable or recursive iff some TM decides L

58. A language is called Turing-recognizable or recursively enumerable (r
A language is called Turing-recognizable or recursively enumerable (r.e.) if some TM recognizes it
A language is called decidable or recursive if some TM decides it
recursive
languages
r.e.

59.
Happy studying!