FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY 15-453 FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY

M = ({p,q}, {0,1}, , p, {q}) 1 THEOREM: p q 1 Σ q0 F  1 p q M = ({p,q}, {0,1}, , p, {q}) 1 THEOREM: L(M) = {w | w has odd number of 1s } p q 1 Proof: By induction on n, the length of a string. Base Case: n=0: ε  L and ε  L(M). Induction Step: Suppose for all w  Σ*, |w| = n, w  L(M) iff w has odd number of 1s Any string of length n+1 has the form w0 or w1 If w has odd # of 1’s, M is in state q after reading w  M is in state p after reading w1  M rejects w1  M is in state q after reading w0  M accepts w0 Let’s give a simple example of how you’d PROVE that a DFA recognizes a certain language. Take this DFA from last lecture. What language does it recognize, again? How can we prove that L(M) is this language? One way to do it is by induction on the length of the string.

UNION THEOREM INTERSECTION THEOREM The union of two regular languages is also a regular language “Regular Languages Are Closed Under Union” INTERSECTION THEOREM How did we prove these? The intersection of two regular languages is also a regular language

Complement THEOREM The complement of a regular language is also a regular language In other words, if L is regular than so is L, where L= { w  Σ* | w  L } Proof ?

THE REGULAR OPERATIONS Union: A  B = { w | w  A or w  B } Intersection: A  B = { w | w  A and w  B } Negation: A = { w  Σ* | w  A } Reverse: AR = { w1 …wk | wk …w1  A } Can A* ever be the empty set? Why or why not? Concatenation: A  B = { vw | v  A and w  B } Star: A* = { s1 … sk | k ≥ 0 and each si  A }

REVERSE THEOREM The reverse of a regular language is also a regular language ``Regular Languages Are Closed Under Reverse” If a language can be recognized by a DFA that reads strings from right to left, then there is an “normal” DFA that accepts the same language Counterintuitive! DFAs have finite memory…

We’ll build a machine MR that accepts LR REVERSING DFAs Assume L is a regular language. Let M be a DFA that recognizes L We’ll build a machine MR that accepts LR If M accepts w, then w describes a directed path in M from start to an accept First Attempt: Try to define MR as M with the arrows reversed, turn start state into a final state, turn final states into starts

MR IS NOT ALWAYS A DFA! It could have many start states Some states may have too many outgoing edges, or none at all!

0,1 1 1 1

NONDETERMINISM 0,1 1 1 1 What happens with 100? 1 1 What happens with 100? We will say that this machine accepts a string if there is some path that reaches an accept state from a start state

Turing Award winning paper IBM JOURNAL APRIL 1959 Turing Award winning paper

NFA EXAMPLES 0,1 0,1 0, ε What language does this NFA recognize? L = {w | w contains a 0} At each state, we can have any number of out arrows for some letter   Σ, including ε

Possibly many start states NFA EXAMPLES 1 1 0,1 ε What language does this NFA recognize? L = 1^* or 01^* L(M) = {0i1j | i  {0,1}, j ≥ 0} Possibly many start states

NFA EXAMPLES 1 L(M)={1,00}

A non-deterministic finite automaton (NFA) is a 5-tuple N = (Q, Σ, , Q0, F) Q is the set of states Σ is the alphabet  : Q  Σε → 2Q is the transition function Q0  Q is the set of start states F  Q is the set of accept states 2Q is the set of all possible subsets of Q Σε = Σ  {ε}

A language L is recognized by an NFA N Let w Σ* and suppose w can be written as w1... wn where wi  Σε (ε = empty string) Then N accepts w if there are r0, r1, ..., rn  Q such that r0  Q0 ri+1  (ri, wi+1 ) for i = 0, ..., n-1, and rn  F L(N) = the language recognized by N = set of all strings machine N accepts A language L is recognized by an NFA N if L = L(N).

1 q2 q4 N = (Q, Σ, , Q0, F) ε q3 Q = {q1, q2, q3, q4} Σ = {0,1} Q0 = {q1, q2} q1 F = {q4}  Q (q2,1) = {q4} 00  L(N)? 01  L(N)? (q3,1) =  { q3} (q1,0) =

Deterministic Computation Non-Deterministic Computation reject Last time: we wondered what happens when you change a DFA so that it reads words from right to left. We wanted to know if the languages recognized becomes different in this case. We claimed that no, any right-to-left DFA can be simulated by a left-to-right one. We tried to show this by defining a “reverse” automation which reversed the directions of arrows, and swapped start and final states. But what we got wasn’t necessarily a DFA anymore! Rather than giving up, we gave it a name and started studying it. We called it an NFA. accept or reject accept

We allow multiple start states for NFAs, Can easily convert NFA with many start states into one with a single start state: ε ε ε

UNION THEOREM FOR NFAs? 1 1

An NFA that recognizes the language {1}: NFAs ARE SIMPLER THAN DFAs An NFA that recognizes the language {1}: 1 A DFA that recognizes the language {1}: 0,1 1 0,1

BUT DFAs CAN SIMULATE NFAs! Theorem: Every NFA has an equivalent* DFA Corollary: A language is regular iff it is recognized by an NFA Corollary: L is regular iff LR is regular * N is equivalent to M if L(N) = L (M)

Q = 2Q FROM NFA TO DFA Input: N = (Q, Σ, , Q0, F) Output: M = (Q, Σ, , q0, F) To learn if NFA accepts, we could do the computation in parallel, maintaining the set of all possible states that can be reached Q = 2Q reject Idea: Q = 2Q accept

F = { R  Q | f  R for some f  F } FROM NFA TO DFA Input: NFA N = (Q, Σ, , Q0, F) Output: DFA M = (Q, Σ, , q0, F) Q = 2Q  : Q  Σ → Q (R,) =  ε( (r,) ) rR * The transition function has to simulate all the possible transitions that can be taken out of all states in the set R. Its output will again be a new set of states: all the possible states you could be in after being in one of the input states and reading the input symbol. The eps-closure is necessary to take care of the effect of eps-transitions. q0 = ε(Q0) F = { R  Q | f  R for some f  F } * For R  Q, the ε-closure of R, ε(R) = {q that can be reached from some r  R by traveling along zero or more ε arrows}

EXAMPLE OF ε-CLOSURE 0,1 0,1 0,ε 0,ε q0 q1 q2 ε({q0}) = {q0 , q1, q2}

N Given: NFA N = ( {1,2,3}, {a,b},  , {1}, {1} ) Construct: Equivalent DFA M M = (2{1,2,3}, {a,b}, , {1,3}, …) N a a b 1 {1,3} {3}  a b b a b b a,b ε a , b {2} 2 3 a {2,3} b a a {1}, {1,2} ? ε({1}) = {1,3} {1,2,3}

NFAs CAN MAKE PROOFS MUCH EASIER! Remember this on your Homework!

REGULAR LANGUAGES ARE CLOSED UNDER THE REGULAR OPERATIONS Union: A  B = { w | w  A or w  B } Intersection: A  B = { w | w  A and w  B } Negation: A = { w  Σ* | w  A } Reverse: AR = { w1 …wk | wk …w1  A } Concatenation: A  B = { vw | v  A and w  B } Star: A* = { w1 …wk | k ≥ 0 and each wi  A }

B = {0n1n | n ≥ 0} is NOT regular! SOME LANGUAGES ARE NOT REGULAR B = {0n1n | n ≥ 0} is NOT regular!

WHICH OF THESE ARE REGULAR C = { w | w has equal number of 1s and 0s} NOT REGULAR D = { w | w has equal number of occurrences of 01 and 10 } REGULAR!!! Why might C be not regular? A DFA has only a finite number of states. If we feed it a string that is longer than its number of states, then over time it should “forget” how many 0’s it has seen, and how many 1’s it has seen. How can we prove this?

THE PUMPING LEMMA Let L be a regular language with |L| =  Then there is a positive integer P s.t. if w  L and |w| ≥ P then can write w = xyz, where: 1. |y| > 0 (y isn’t ε) 2. |xy| ≤ P 3. For every i ≥ 0, xyiz  L Why is it called the pumping lemma? The word w gets PUMPED into something longer…

Assume w  L is such that |w| ≥ P There must be j and k such that Proof: Let M be a DFA that recognizes L Let P be the number of states in M Assume w  L is such that |w| ≥ P We show: w = xyz 1. |y| > 0 2. |xy| ≤ P 3. xyiz  L for all i ≥ 0 y x z Why must there be such j and k? PIGEONHOLE PRINCIPLE! … q0 qj qk q|w| There must be j and k such that j < k ≤ P, and qj = qk

B = {0n1n | n ≥ 0} is not regular Assume B is regular. Let w = 0P1P USING THE PUMPING LEMMA Let’s prove that B = {0n1n | n ≥ 0} is not regular Assume B is regular. Let w = 0P1P If B is regular, can write w = xyz, |y| > 0, |xy| ≤ P, and for any i ≥ 0, xyiz is also in B Contradiction! If y is all 0s: xyyz has more 0s than 1s Note that w is in B. If y is all 1s: THIS CASE CAN’T HAPPEN! If y has both 1s and 0s: |xy| ≤ P THIS CASE CAN’T HAPPEN!

Assume C is regular. Let w = 0P1P (w is in C!) USING THE PUMPING LEMMA C = { w | w has equal number of 1s and 0s} is not regular Assume C is regular. Let w = 0P1P (w is in C!) If C is regular, can write w = xyz, |y| > 0, |xy| ≤ P, where for any i ≥ 0, xyiz is also in C If y is all 0s: xyyz has more 0s than 1s Note that w is in B. If y is all 1s: CAN’T HAPPEN If y has both 1s and 0s: CAN’T HAPPEN

FLAC Read Chapters 1.3 and 2.1 of the book for next time