1. FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY
15-453
FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY

2. M = ({p,q}, {0,1}, , p, {q}) 1 THEOREM: p q 1Σ q0 F  1 p q
M = ({p,q}, {0,1}, , p, {q}) 1
THEOREM:
L(M) = {w | w has odd
number of 1s } p q 1
Proof: By induction on n, the length of a string.
Base Case: n=0: ε  L and ε  L(M).
Induction Step: Suppose for all w  Σ*, |w| = n,
w  L(M) iff w has odd number of 1s
Any string of length n+1 has the form w0 or w1
If w has odd # of 1’s, M is in state q after reading w
 M is in state p after reading w1  M rejects w1
 M is in state q after reading w0  M accepts w0
Let’s give a simple example of how you’d PROVE that a DFA recognizes a certain language.
Take this DFA from last lecture. What language does it recognize, again?
How can we prove that L(M) is this language? One way to do it is by induction on the length of the string.

3. UNION THEOREM INTERSECTION THEOREM
The union of two regular languages is also a regular language
“Regular Languages Are Closed Under Union”
INTERSECTION THEOREM
How did we prove these?
The intersection of two regular languages is also a regular language

4. Complement THEOREM
The complement of a regular language is also a regular language
In other words,
if L is regular than so is L,
where L= { w  Σ* | w  L }
Proof ?

5. THE REGULAR OPERATIONS
Union: A  B = { w | w  A or w  B }
Intersection: A  B = { w | w  A and w  B }
Negation: A = { w  Σ* | w  A }
Reverse: AR = { w1 …wk | wk …w1  A }
Can A* ever be the empty set? Why or why not?
Concatenation: A  B = { vw | v  A and w  B }
Star: A* = { s1 … sk | k ≥ 0 and each si  A }

6. REVERSE THEOREM
The reverse of a regular language is also a regular language
``Regular Languages Are Closed Under Reverse”
If a language can be recognized by a DFA that reads strings from right to left,
then there is an “normal” DFA that accepts the same language
Counterintuitive! DFAs have finite memory…

7. We’ll build a machine MR that accepts LR
REVERSING DFAs
Assume L is a regular language.
Let M be a DFA that recognizes L
We’ll build a machine MR that accepts LR
If M accepts w, then w describes a directed path in M from start to an accept
First Attempt: Try to define MR as M with the arrows reversed, turn start state into a final state, turn final states into starts

8. MR IS NOT ALWAYS A DFA! It could have many start states
Some states may have too many outgoing edges,
or none at all!

9. 0,1 1 1 1

10. NONDETERMINISM 0,1 1 1 1 What happens with 100?1 1
What happens with 100?
We will say that this machine accepts a string if there is some path that reaches an accept state from a start state

11. Turing Award winning paper
IBM JOURNAL APRIL 1959
Turing Award winning paper

12. NFA EXAMPLES
0,1
0,1
0, ε
What language does this NFA recognize?
L = {w | w contains a 0}
At each state, we can have any number of out arrows for some letter   Σ, including ε

13. Possibly many start states
NFA EXAMPLES 1 1
0,1 ε
What language does this NFA recognize?
L = 1^* or 01^*
L(M) = {0i1j | i  {0,1}, j ≥ 0}
Possibly many start states

14. NFA EXAMPLES 1
L(M)={1,00}

15. A non-deterministic finite automaton (NFA) is a 5-tuple N = (Q, Σ, , Q0, F)
Q is the set of states
Σ is the alphabet
 : Q  Σε → 2Q is the transition function
Q0  Q is the set of start states
F  Q is the set of accept states
2Q is the set of all possible subsets of Q
Σε = Σ  {ε}

16. A language L is recognized by an NFA N
Let w Σ* and suppose w can be written as
w1... wn where wi  Σε (ε = empty string)
Then N accepts w if there are r0, r1, ..., rn  Q such that
r0  Q0
ri+1  (ri, wi+1 ) for i = 0, ..., n-1, and
rn  F
L(N) = the language recognized by N
= set of all strings machine N accepts
A language L is recognized by an NFA N
if L = L(N).

17. 1 q2 q4
N = (Q, Σ, , Q0, F) ε q3
Q = {q1, q2, q3, q4}
Σ = {0,1}
Q0 = {q1, q2} q1
F = {q4}  Q
(q2,1) =
{q4}
00  L(N)?
01  L(N)?
(q3,1) = 
{ q3}
(q1,0) =

18. Deterministic Computation Non-Deterministic Computation reject
Last time: we wondered what happens when you change a DFA so that it reads words from right to left.
We wanted to know if the languages recognized becomes different in this case.
We claimed that no, any right-to-left DFA can be simulated by a left-to-right one.
We tried to show this by defining a “reverse” automation which reversed the directions of arrows, and swapped start and final states.
But what we got wasn’t necessarily a DFA anymore!
Rather than giving up, we gave it a name and started studying it. We called it an NFA.
accept or reject
accept

19. We allow multiple start states for NFAs,
Can easily convert NFA with many start states into one with a single start state: ε ε ε

20. UNION THEOREM FOR NFAs? 1 1

21. An NFA that recognizes the language {1}:
NFAs ARE SIMPLER THAN DFAs
An NFA that recognizes the language {1}: 1
A DFA that recognizes the language {1}:
0,1 1
0,1

22. BUT DFAs CAN SIMULATE NFAs!
Theorem: Every NFA has an equivalent* DFA
Corollary: A language is regular iff
it is recognized by an NFA
Corollary: L is regular iff LR is regular
* N is equivalent to M if L(N) = L (M)

23. Q = 2Q FROM NFA TO DFA Input: N = (Q, Σ, , Q0, F)
Output: M = (Q, Σ, , q0, F)
To learn if NFA accepts, we could do the computation in parallel, maintaining the set of all possible states that can be reached
Q = 2Q
reject
Idea:
Q = 2Q
accept

24. F = { R  Q | f  R for some f  F }
FROM NFA TO DFA
Input: NFA N = (Q, Σ, , Q0, F)
Output: DFA M = (Q, Σ, , q0, F)
Q = 2Q
 : Q  Σ → Q
(R,) =  ε( (r,) )
rR *
The transition function has to simulate all the possible transitions that can be taken out of all states in the set R.
Its output will again be a new set of states:
all the possible states you could be in after being in one of the input states and reading the input symbol.
The eps-closure is necessary to take care of the effect of eps-transitions.
q0 = ε(Q0)
F = { R  Q | f  R for some f  F } *
For R  Q, the ε-closure of R, ε(R) = {q that can be reached from some r  R by traveling along zero or more ε arrows}

25. EXAMPLE OF ε-CLOSURE 0,1 0,1 0,ε 0,ε q0 q1 q2 ε({q0}) = {q0 , q1, q2}

26. N Given: NFA N = ( {1,2,3}, {a,b},  , {1}, {1} )
Construct: Equivalent DFA M
M = (2{1,2,3}, {a,b}, , {1,3}, …) N a a b 1
{1,3}
{3}  a b b a b b
a,b ε
a , b
{2} 2 3 a
{2,3} b a a
{1}, {1,2} ?
ε({1}) = {1,3}
{1,2,3}

27. NFAs CAN MAKE PROOFS MUCH EASIER!
Remember this on your Homework!

28. REGULAR LANGUAGES ARE CLOSED UNDER THE REGULAR OPERATIONS
Union: A  B = { w | w  A or w  B }
Intersection: A  B = { w | w  A and w  B }
Negation: A = { w  Σ* | w  A }
Reverse: AR = { w1 …wk | wk …w1  A }
Concatenation: A  B = { vw | v  A and w  B }
Star: A* = { w1 …wk | k ≥ 0 and each wi  A }

29. B = {0n1n | n ≥ 0} is NOT regular!
SOME LANGUAGES ARE
NOT REGULAR
B = {0n1n | n ≥ 0} is NOT regular!

30. WHICH OF THESE ARE REGULAR
C = { w | w has equal number of 1s and 0s}
NOT REGULAR
D = { w | w has equal number of occurrences of 01 and 10 }
REGULAR!!!
Why might C be not regular?
A DFA has only a finite number of states.
If we feed it a string that is longer than its number of states, then over time it should “forget” how many 0’s it has seen, and how many 1’s it has seen. How can we prove this?

31. THE PUMPING LEMMA Let L be a regular language with |L| = 
Then there is a positive integer P s.t.
if w  L and |w| ≥ P
then can write w = xyz, where:
1. |y| > 0 (y isn’t ε)
2. |xy| ≤ P
3. For every i ≥ 0, xyiz  L
Why is it called the pumping lemma? The word w gets PUMPED into something longer…

32. Assume w  L is such that |w| ≥ P There must be j and k such that
Proof: Let M be a DFA that recognizes L
Let P be the number of states in M
Assume w  L is such that |w| ≥ P
We show: w = xyz
1. |y| > 0
2. |xy| ≤ P
3. xyiz  L for all i ≥ 0 y x z
Why must there be such j and k?
PIGEONHOLE PRINCIPLE! … q0 qj qk
q|w|
There must be j and k such that
j < k ≤ P, and qj = qk

33. B = {0n1n | n ≥ 0} is not regular Assume B is regular. Let w = 0P1P
USING THE PUMPING LEMMA
Let’s prove that
B = {0n1n | n ≥ 0} is not regular
Assume B is regular. Let w = 0P1P
If B is regular, can write w = xyz, |y| > 0, |xy| ≤ P, and for any i ≥ 0, xyiz is also in B
Contradiction!
If y is all 0s:
xyyz has more 0s than 1s
Note that w is in B.
If y is all 1s:
THIS CASE CAN’T HAPPEN!
If y has both 1s and 0s:
|xy| ≤ P
THIS CASE CAN’T HAPPEN!

34. Assume C is regular. Let w = 0P1P (w is in C!)
USING THE PUMPING LEMMA
C = { w | w has equal
number of 1s and 0s} is not regular
Assume C is regular. Let w = 0P1P (w is in C!)
If C is regular, can write w = xyz, |y| > 0, |xy| ≤ P, where for any i ≥ 0, xyiz is also in C
If y is all 0s:
xyyz has more 0s than 1s
Note that w is in B.
If y is all 1s:
CAN’T HAPPEN
If y has both 1s and 0s:
CAN’T HAPPEN

35. FLAC
Read Chapters 1.3 and 2.1 of the book for next time