1. Nondeterministic Finite Automata
Nondeterminism
Subset Construction
ε-Transitions

2. Nondeterminism
A nondeterministic finite automaton has the ability to be in several states at once.
Transitions from a state on an input symbol can be to any set of states.
DRAW a state with several transitions on input a.

3. Nondeterminism – (2) Start in one start state.
Accept if any sequence of choices leads to a final state.
Intuitively: the NFA always “guesses right.”
Like the DFA, a nondeterministic finite automaton, or NFA, has one start state, where computation begins. The NFA can have any number of final states, and an input is accepted if any sequence of choices leads from the start state to some final state.
The intuition is that the NFA is allowed to guess which way to go, but it is able always to guess right, since all the guesses are followed in parallel and the NFA gets credit for the right guesses, no matter how many wrong guesses it also makes.

4. Example: Moves on a Chessboard
States = squares.
Inputs = r (move to an adjacent red square) and b (move to an adjacent black square).
Start state, final state are in opposite corners.
In our example of a nondeterministic finite automaton, the states are the squares of a chessboard. In general, you can occupy several different squares at a time.
In a red move, represented by input symbol r, you get to move to any adjacent red square. So in effect, you replicate yourself and move to all of them. Similarly, if the input is b, you can move to any adjacent black square, so in effect you move to all of them.
The question we ask is whether the given sequence of r and b inputs can get you from the start state – the upper left corner, to the one final state – the lower right corner. The answer is “yes” if any sequence of the choices we get with each input leads us from upper left to lower right. It is not necessary that all such choices do, and in general, there will always be some choices that lead us astray.

5. Example: Chessboard – (2)
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, * 1 2 5 7 9 3 4 8 6 1 r b
In our example, the chessboard will be rather tiny – 3 by 3 instead of 8 by 8, but the ideas are all the same.
Click 1
The sequence of inputs we will simulate is rbb, that is, one red move followed by two black moves. We start with the start state, state 1, only.
Click 2
We get an r input, so we can move to any of the red squares adjacent to square 1. We’ll take “adjacent” to mean one King move, that is, any square that can be reached by one vertical, horizontal, or diagonal move. For example, square 5 is adjacent to all the other squares.
The red squares adjacent to 1 are 2 and 4, so that is the entry in the transition table for row 1 and column r. As we simulate the nondeterministic automaton, after reading the r we are in states 2 and 4, and no other states.
WING IT AFTER THAT 4 2 1 5 3 5 7 1 3 9 7
Accept, since final state reached

6. Formal NFA A finite set of states, typically Q.
An input alphabet, typically Σ.
A transition function, typically δ.
A start state in Q, typically q0.
A set of final states F ⊆ Q.
Here are the components of an NFA. They look exactly the same as for the DFA, and we’ll typically use the same letters to represent them. The big difference is in the type of the transition function, and we’ll see that on the next slide.

7. Transition Function of an NFA
δ(q, a) is a set of states.
Extend to strings as follows:
Basis: δ(q, ε) = {q}
Induction: δ(q, wa) = the union over all states p in δ(q, w) of δ(p, a)
Delta(q,a) is now a set of states, possibly empty, rather than a single state as it was for the DFA.
The extension of delta to strings is a bit more complex. The basis is still easy. Delta(q, epsilon) is just the set containing q, since the only state you can reach on no input is the state you are in.
For the induction, suppose we start in state q and read string w followed by symbol “a”. We first compute delta(q,w), the set of states you get to from q by following w. Then, for each of these states, we use the given delta to find the set of states they each get to on “a” and we take the union of those sets. The resulting union is delta(q, wa).
DRAW a state q going to many states p_1, p_2,.. on input w, then to other states on further input a.

8. Language of an NFA
A string w is accepted by an NFA if δ(q0, w) contains at least one final state.
The language of the NFA is the set of strings it accepts.

9. Example: Language of an NFA1 2 5 7 9 3 4 8 6
Example: Language of an NFA
For our chessboard NFA we saw that rbb is accepted.
If the input consists of only b’s, the set of accessible states alternates between {5} and {1,3,7,9}, so only even-length, nonempty strings of b’s are accepted.
What about strings with at least one r?
The language of the NFA we designed to represent moves on a chessboard is actually quite tricky to describe. For strings consisting only of b’s, it is easy. You start in {1}. On the net move you go to {5} only. Another b takes you to {1,3,7,9}, and the next b back to {5}. You alternate between these two sets, and only the {1,3,7,9} set contains a final state, so you accept bb, bbbb, and so on – the even-length strings of b’s, except for the empty string.
It is less clear what happens when there are r’s in the input. Obviously, an accepted string must end in b, because only state 9 is accepting, and you only get there on a b-move. But I’ll leave it to you to figure out exactly what strings with one or more r’s are accepted.

10. Equivalence of DFA’s, NFA’s
A DFA can be turned into an NFA that accepts the same language.
If δD(q, a) = p, let the NFA have δN(q, a) = {p}.
Then the NFA is always in a set containing exactly one state – the state the DFA is in after reading the same input.
In a sense, a DFA is an NFA that doesn’t use any nondeterminism. Formally, given a DFA with transition function delta-D, you can create an NFA with the same states and inputs as the DFA, and the same start and final states. The only thing different about the NFA is that the form of its transition function is what it has to be for an NFA – it gives you a set of states rather than a single state. But the set of states delta-N gives you for state q and input a is the set containing only the one state that delta-D gives you. As a result, the NFA, after reading some sequence of inputs, is in the set of states that contains only the one state the DFA is in.

11. Equivalence – (2)
Surprisingly, for any NFA there is a DFA that accepts the same language.
Proof is the subset construction.
The number of states of the DFA can be exponential in the number of states of the NFA.
Thus, NFA’s accept exactly the regular languages.
This construction was the thing that, as a graduate student, convinced me that there was something to Computer-Science theory. It had only been discovered 5 years before, and it boggled my mind to see a construction that, while it could be described easily, resulted in something that could not be grasped. The problem is that if the NFA has 3 states, the DFA can have 8 – that’s not a big deal – I can visualize an 8-state automaton. But if the NFA has 10 states, the DFA has a thousand, and that is already becoming hard to imagine. And if the NFA has 20 states – still something we can visualize – the DFA can have a million states, and we are completely lost, even though we know the DFA exists.
Oh, and by the way, the situation is not nearly as bad as it looks for NFA’s that we construct in practice. For example, the Chess NFA we talked about actually has an equivalent DFA with FEWER states.

12. Subset Construction
Given an NFA with states Q, inputs Σ, transition function δN, start state q0, and final states F, construct equivalent DFA with:
States 2Q (Set of subsets of Q).
Inputs Σ.
Start state {q0}.
Final states = all those with a member of F.
Let’s start with a typical NFA. It has the conventional names for its components, although we’ll use delta-N for its transition function to distinguish it from delta-D, the transition function for the equivalent DFA we’ll construct.
Click 2
2Q is mathematical notation for the POWER SET of Q, the set of all subsets of Q. Notice that if set Q has n elements, then its power set has 2n elements, so the notation sort of makes sense.
Click 4
Remember that the states of the deterministic automaton are sets of states of the nondeterministic automaton. Thus, the start state, which is a single state of the DFA is written as a set of states of the NFA. Of course this set contains only the start state of the NFA.

13. Critical Point The DFA states have names that are sets of NFA states.
But as a DFA state, an expression like {p,q} must be understood to be a single symbol, not as a set.
Just to make sure we understand what is going on, the DFA states have names that look like sets of states. However, they are single objects. An analogy that might be useful to make is with a class of objects in a language like Java or C++, whose values are sets of objects of another class.

14. Subset Construction – (2)
The transition function δD is defined by:
δD({q1,…,qk}, a) is the union over all i = 1,…,k of δN(qi, a).
Example: We’ll construct the DFA equivalent of our “chessboard” NFA.
DRAW the picture of states q_1, q_2,… all going to various unnamed states on input a.

15. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6,
r b
{1}
{2,4} {5}
{2,4}
{5}
We’re going to do a lazy construction of DFA states, rather than assuming that we need all the subsets of NFA states. You see, if we start by creating all the subsets of the NFA’s states, we often find that many of these states cannot be reached on any sequence of inputs from the start state. Such states are useless and create unneeded work for us.
Obviously, we need the start state of the DFA, which is the set containing only the NFA state 1, so we’ll begin the construction with a row for the set containing 1. From the NFA table, we see that 1 goes to 2 and 4 on r and 5 on b, so we enter these two sets, as names of DFA states, into the DFA table, and we know we need rows for these two DFA states, so we create them now.
Alert: What we’re doing here is
the lazy form of DFA construction,
where we only construct a state
if we are forced to. *

16. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, *
r b
{1}
{2,4} {5}
{2,4}
{2,4,6,8} {1,3,5,7}
{5}
{2,4,6,8}
{1,3,5,7}
Now, we’ve filled out the row for the DFA state whose name is {2,4}. On input r, 2 goes to 4 and 6, while 4 goes to 2 and 8. Thus, {2,4} goes to {2,4,6,8}. We enter that in the row for {2,4} and the column for r, and we add a row for {2,3,6,8}, which we will fill out later.
Similarly, on input b, we look at the NFA transitions from 2 and 4 and take their union. This union is {1,3,5,7}, which is also entered as a row to be filled out.

17. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, *
r b
{1}
{2,4} {5}
{2,4}
{2,4,6,8} {1,3,5,7}
{5}
{2,4,6,8} {1,3,7,9}
{2,4,6,8}
{1,3,5,7}
* {1,3,7,9}
Now we fill out the row for {5}. The transition from 5 on r is to {2,4,6,8}. We already have started a row for that state, so we don’t have to do anything more at this time. From 5 on b, the NFA goes to {1,3,7,9}, and that is a new state whose row we must add. Note that we put a star next to this state, since it is a final state – it contains NFA final state 9.

18. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, *
r b
{1}
{2,4} {5}
{2,4}
{2,4,6,8} {1,3,5,7}
{5}
{2,4,6,8} {1,3,7,9}
{2,4,6,8}
{2,4,6,8} {1,3,5,7,9}
{1,3,5,7}
* {1,3,7,9}
We fill out the row for {2,4,,6,8}. That requires addition of a new final state {1,3,5,7,9}.
* {1,3,5,7,9}

19. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, *
r b
{1}
{2,4} {5}
{2,4}
{2,4,6,8} {1,3,5,7}
{5}
{2,4,6,8} {1,3,7,9}
{2,4,6,8}
{2,4,6,8} {1,3,5,7,9}
{1,3,5,7}
{2,4,6,8} {1,3,5,7,9}
* {1,3,7,9}
Here is the row for {1,3,5,7}. Its transitions are each to states we’ve seen before, so no rows are added.
* {1,3,5,7,9}

20. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, *
r b
{1}
{2,4} {5}
{2,4}
{2,4,6,8} {1,3,5,7}
{5}
{2,4,6,8} {1,3,7,9}
{2,4,6,8}
{2,4,6,8} {1,3,5,7,9}
{1,3,5,7}
{2,4,6,8} {1,3,5,7,9}
* {1,3,7,9}
{2,4,6,8} {5}
The same for state {1,3, 7, 9}.
* {1,3,5,7,9}

21. Example: Subset Construction
r b 2,
4, ,3,5 2,
2, ,5,7
2,4,6,8 1,3,7,9
2, ,5,9 4,
4, ,7,9 6, *
r b
{1}
* {1,3,5,7,9}
{2,4,6,8} {1,3,5,7,9}
* {1,3,7,9}
{2,4,6,8} {5}
{2,4,6,8}
{2,4,6,8} {1,3,7,9}
{5}
{2,4}
{2,4,6,8} {1,3,5,7}
{1,3,5,7}
{2,4} {5}
And finally, state {1,3,5,7,9} yields no new states. We’ve now filled out the DFA transition table for all the states we need, and we are done.

22. NFA’s With ε-Transitions
We can allow state-to-state transitions on ε input.
These transitions are done spontaneously, without looking at the input string.
A convenience at times, but still only regular languages are accepted.
We’re now going to add an additional capability to NFA’s – the ability to make a spontaneous transition on epsilon, that is, without using any input. We just saw that the NFA, while it can be a convenience in designing automata, still accepts only regular languages. The same is true for the new model, which we’ll call epsilon-NFA’s. They can be a real convenience in constructing automata, but they still accept only the regular languages.

23. Example: ε-NFA ε 0 1 ε A {E} {B} ∅ B ∅ {C} {D} C ∅ {D} ∅ D ∅ ∅ ∅ε
A {E} {B} ∅
B ∅ {C} {D}
C ∅ {D} ∅
D ∅ ∅ ∅
E {F} ∅ {B, C}
F {D} ∅ ∅ * C E F A B D 1 ε
Here is an example epsilon-NFA. The transition diagram has some arcs labeled epsilon, and we can follow any such arc without adding to the sequence of inputs that the automaton has processed. That is, epsilon is “invisible” as far as input strings are concerned.
We also see the transition table for this automaton. Notice it has a separate column for epsilon, but epsilon is not an input symbol.
For example, from the start state A, we can go only to state B on input 1, and only to E on input 0. There are no transitions out of A on epsilon, which is why the entry for A and epsilon in the table is the empty set.
From state E, there is no transition on input 1, and on input 0 there is only a transition to F. However, there are transitions to both B and C on epsilon. So, for example, if 1 is the next input, and we are in state E, then we can go to B on epsilon and then to C on 1. Or we can go to C on epsilon and then to D on 1.

24. Closure of States
CL(q) = set of states you can reach from state q following only arcs labeled ε.
Example: CL(A) = {A};
CL(E) = {B, C, D, E}.
Closure of a set of states = union of the closure of each state. C E F A B D 1 ε
The closure of a state q, which we write CL(q), is the set of states we can get to, starting in q and following only epsilon transitions.
For example, from A you can get nowhere else on epsilon, so CL(A) = {A}. The closure of E is trickier. Surely we can reach E itself because we ‘re there without following any arcs. There are epsilon-transitions from E to B and C, and we can go even further, since there is an epsilon transition from B to D. That is as far as we can get, so CL(E) = {B,C,D,E}.
We shall also need to apply the closure operator to sets of states, and the closure of a set of states is the union of the closures of the individual states.

25. Example: ε δ (A, ε) = CL(A) = {A}. (A, 0) = CL({E}) = {B, C, D, E}.F A B D 1 ε
Example: ˄ δ
(A, ε) = CL(A) = {A}.
(A, 0) = CL({E}) = {B, C, D, E}.
(A, 01) = CL({C, D}) = {C, D}.
Language of an ε-NFA is the set of strings w such that (q0, w) contains a final state. ˄ δ ˄ δ ˄ δ
For first bullet: Delta-hat(A,epsilon) is the closure of A, which is just the set containing A.
For second bullet: Why closure({E})? Because closure(A) = {A}, and on 0 the only transition from A is to E.
For third bullet: Why closure({C,D})? Because we start with delta-hat(A,0), which is {B,C,D,E} and then look for transitions out of these states on input 1. We find transitions to only C and D on 1.
The language of an epsilon-NFA is defined in the expected way. For any string w, you compute the extended delta of the start state and that string w, and you see if any of the resulting set of states is a final state. You accept w if so, and if not, then not.

26. Equivalence of NFA, ε-NFA
Every NFA is an ε-NFA.
It just has no transitions on ε.
Converse requires us to take an ε-NFA and construct an NFA that accepts the same language.
We do so by combining ε–transitions with the next transition on a real input.
We now want to show that the NFA and epsilon-NFA models yield the same languages – the regular languages.
One direction is easy, since an ordinary NFA is an epsilon-NFA that happens to have no transitions on epsilon.
But proving that for every epsilon-NFA there is an ordinary NFA that accepts the same language. We’re going to get rid of the epsilon-transitions by combining them with the next transition on a real input.
If you’re following the text, you should notice that this construction is somewhat different from the one in the text, but both constructions work. Here, we have to change the set of final states, while the one in the book doesn’t, but this construction is a bit simpler.

27. Picture of ε-Transition Removal
To here a
We’ll go
from here a
This is a picture of how we iron out the epsilon transitions. We start from a state, follow all the epsilons we can, using the closure operator (POINT to the yellow). We then follow all the transitions from any of these states on a real input symbol, say “a”. The ordinary NFA is able to get to, on input “a”, any of the states that can be reached using these transitions of the epsilon-NFA on input “a”.
Since we don’t close the states after a transition on a, we have to make additional final states – those that can reach a final state on epsilons only. (POINT to a root of one orange triangle and say that this state will be final if the epsilon-NFA has any final states within the orange triangle) a
Transitions
on ε
Transitions
on ε

28. Summary
DFA’s, NFA’s, and ε–NFA’s all accept exactly the same set of languages: the regular languages.
The NFA types are easier to design and may have exponentially fewer states than a DFA.
But only a DFA can be implemented!
Click 1
So we now have three different formalisms for describing languages, the DFA, the NFA, and the epsilon-NFA. They look progressively more powerful, but in fact, they all give us exactly the class of regular languages. Very soon, we’ll set a fourth formalism – regular expressions – that look quite different but in fact also give us exactly the regular languages. So we might be getting the idea that the regular languages are quite a natural class of languages.
Click 2
We should notice that the added power of NFA’s and epsilon-NFA’s are quite useful. For example, we shall talk about designing automata to recognize sets of keywords, say the reserved words in C or some other programming language. That is an important task for building a compiler for the language. We could design a DFA for the task, but it is much easier to start with simple DFA’s for each keyword – it’s just a chain of states (DRAW for e-l-s-e). Then, connect them all with epsilon-transitions from a single start state.
Click 3
However, conversion of these more powerful automata to a DFA is necessary if we are to create code that simulates the automata. That’s because no one has yet invented a nondeterministic computer. I’ve heard Intel is working on it.