1. Lecture 05: Theory of Automata:08 Kleene’s Theorem and NFA

2. Lecture 05: Theory of Automata:08 2 Before we proceed, let’s have a quick review of the formal definition of regular expressions. The set of regular expressions is defined by the following rules: Rule 1: Every letter of the alphabet ∑ can be made into a regular expression by writing it in boldface: Λ itself is a regular expression. Rule 2: If r 1 and r 2 are regular expressions, then so are: (i) (r 1 ) (ii) r 1 r 2 (iii) r 1 + r 2 (iv) r 1 Rule 3: Nothing else is a regular expression. We now present proof of part 3 recursively.

3. Lecture 05: Theory of Automata:08 3 Rule 1 There is an FA that accepts any particular letter of the alphabet. There is an FA that accepts only the word Λ.

4. Lecture 05: Theory of Automata:08 4 Proof of rule 1 If letter x is in ∑, then the following FA accepts only the word x. The following FA accepts only :

5. Lecture 05: Theory of Automata:08 5 Rule 2 If there is an FA called FA 1 that accepts the language defined by the regular expression r 1, and there is an FA called FA 2 that accepts the language defined by the regular expression r 2, then there is an FA that we shall call FA 3 that accepts the language defined by the regular expression (r 1 + r 2 ).

6. Lecture 05: Theory of Automata:08 6 Proof of Rule 2 We shall show that FA 3 exists by presenting an algorithm showing how to construct FA 3. Algorithm: – Starting with two machines, FA 1 with states x 1 ; x 2 ; x 3 ;…, and FA 2 with states y 1 ; y 2 ; y 3 ; …, we construct a new machine FA 3 with states z 1 ; z 2 ; z 3 ; … where each z i is of the form x something or y something. – The combination state x start or y start is the start state of the new machine FA 3. – If either the x part or the y part is a final state, then the corresponding z is a final state.

7. Lecture 05: Theory of Automata:08 7 Algorithm (cont.) – To go from one state z to another by reading a letter from the input string, we observe what happens to the x part and what happens to the y part and go to the new state z accordingly. We could write this as a formula: z new after reading letter p = (x new after reading letter p on FA 1 ) or (y new after reading letter p on FA 2 )

8. Lecture 05: Theory of Automata:08 8 Remarks The new machine FA 3 constructed by the above algorithm will simultaneously keep track of where the input would be if it were running on FA 1 alone, and where the input would be if it were running on FA 2 alone. If a string traces through the new machine FA 3 and ends up at a final state, it means that it would also end at a final state either on machine FA 1 or on machine FA 2. Also, any string accepted by either FA 1 or FA 2 will be accepted by this FA 3. So, the language FA 3 accepts is the union of the languages accepted by FA 1 and FA 2, respectively. Note that since there are only finitely many states x’s and finitely many states y’s, there can be only finitely many possible states z’s. Let us look at an example illustrating how the algorithm works.

9. Lecture 05: Theory of Automata:08 9 Example Consider the following two FAs: FA 1 accepts all words with a double a in them. FA 2 accepts all words ending with b. Let’s follow the algorithm to build FA 3 that accepts the union of the two languages.

10. Lecture 05: Theory of Automata:08 10 Combining the FAs The start (-) state of FA 3 is z 1 = x 1 or y 1. In z 1, if we read an a, we go to x 2 (observing FA 1 ), or we go to y 1 (observing FA 2 ). Let z 2 = x 2 or y 1. In z 1, if we read a b, we go to x 1 (observing FA 1 ), or to y 2 (observing FA 2 ). Let z 3 = x 1 or y 2. Note that z 3 must be a final state since y 2 is a final state.

11. Lecture 05: Theory of Automata:08 11 In z 2, if we read an a, we go to x 3 or y 1. Let z 4 = x 3 or y 1. z 4 is a final state because x 3 is. In z 2, if we read a b, we go to x 1 or y 2, which is z 3.

12. Lecture 05: Theory of Automata:08 12 In z 3, if we read an a, we go to x 2 or y 1, which is z 2. In z 3, if we read a b, we go to x 1 or y 2, which is z 3. In z 4, if we read an a, we go to x 3 or y 1, which is z 4. Hence, we have an a-loop at z 4. In z 4, if we read a b, we go to x 3 or y 2. Let z 5 = x 3 or y 2. Note that z 5 is a final state because x 3 (and y 2 ) are. In z 5, if we read an a, we go to x 3 or y 1, which is z 4. In z 5, if we read a b, we go to x 3 or y 2, which is z 5. Hence, there is a b-loop at z 5. The whole machine looks like the following:

13. Lecture 05: Theory of Automata:08 13 This machine accepts all words that have a double a or that end with b. The labels z 1 = x 1 or y 1, z 2 = x 2 or y 1, etc. can be removed if you want.

14. Lecture 05: Theory of Automata:08 14 Example Consider the following two FAs: FA 1 accepts all words that end in a. FA 2 accepts all words with an odd number of letters (odd length). Can you use the algorithm to build a machine FA3 that accepts all words that either have an odd number of letters or end in a?

15. Lecture 05: Theory of Automata:08 15 Using the algorithm, we can produce FA 3 that accepts all words that either have an odd number of letters or end in a, as follows: The only state that is not a + state is the - state. To get back to that start state, a word must have an even number of letters and end in b.

16. Lecture 05: Theory of Automata:08 16 Rule 3 If there is an FA 1 that accepts the language defined by the regular expression r 1, and there is an FA 2 that accepts the language defined by the regular expression r 2, then there is an FA 3 that accepts the language defined by the (concatenation) regular expression (r 1 r 2 ), i.e. the product language.

17. Lecture 05: Theory of Automata:08 17 We shall show that such an FA 3 exists by presenting an algorithm showing how to construct it from FA 1 and FA 2. The idea is to construct a machine that starts out like FA 1 and follows along it until it enters a final state at which time an option is reached. Either we continue along FA 1, waiting to reach another +, or else we switch over to the start state of FA 2 and begin circulating there.

18. Lecture 05: Theory of Automata:08 18 Algorithm First, create a state z for every state of FA 1 that we may go through before arriving at a final state. 2. For each final state x final of FA 1, add a state z = x final or y 1, where y 1 is the start state of FA 2. 3. From the states added in step 2, add states 4. Label every state z that contains a final state from FA 2 as a final state.

19. Lecture 05: Theory of Automata:08 19 Example FA 1 accepts all words that start with a b. FA 2 accepts all words that end with a b. We will use the above algorithm to construct FA 3 that accepts the product of the languages of FA 1 and FA 2, respectively. That is, FA 3 will accept all words that both start and end with the letter b.

20. Lecture 05: Theory of Automata:08 20 Initially, we must begin with x 1 = z 1. In z 1, if we read an a, we go to x 2 = z 2. In z 1, if we read a b, we go to x 3, a final state, which gives us the option to jump to y 1. Hence, we label z 3 = x 3 or y 1. From z 2 just like x 2, both an a or a b take us back to z 2, i.e., we have a loop here.

21. Lecture 05: Theory of Automata:08 21 In z3, if we read an a then the following happens. If z3 is x3, we can stay in x3 or jump to y1 (because x3 is a final state). If z3 is y1, we would loop back to y1. In any of the events, we end up at either x3 or y1, which is still z3. Hence, we have an a-loop at z3. In z3, if we read a b, then a different event takes place. If z3 is x3 we either stay in x3 or jump to y1. If z3 is y1, then we go to y2, a final state. Hence, we need a new final state z4 = x3 or y1 or y2. In z4, if we read an a, what happens? If z4 is x3 then we go back to x3 or jump to y1. If z4 is y1 then we loop back to y1. If z4 is y2, we go to y1. Thus, from z4, an a takes us to x3 or y1, which is z3. In z4, if we read a b, what happens? If z4 is x3, we go back to x3 or jump to y1. If z4 is y1, we go to y2, a final state. If z4 is y2, we loop back to y2, a final state. Hence, from z4 a b takes us to x3 or y1 or y2, which is still z4 (i.e., we have a b-loop here).

22. Lecture 05: Theory of Automata:08 22 This machine accepts all words that both begin and end with the letter b, which is what the product of the two languages (defined by FA 1 and FA 2 respectively) would be. If you multiply the two languages in opposite order (i.e. first FA 2 then FA 1 ), then the product language will be different. What is that language? Can you build a machine for that product language

23. Lecture 05: Theory of Automata:08 23 Rule 4 If r is a regular expression and FA 1 is a finite automaton that accepts exactly the language defined by r, then there is an FA, called FA 2, that will accepts exactly the language defined by r.

24. Lecture 05: Theory of Automata:08 24 Proof of Rule 4 We shall show that such an FA 2 exists by showing how to construct it from FA 1. The language defined by r must always contain the null word. To accept, we must indicate that the start state is also a final state. This must be done carefully. The general rule is that each z-state (of FA 2 ) corresponds to some collection of x-states (of FA 1 ). We must remember each time we reach a final state, it is possible that we have to start over again at x 1. The transmissions from one collection of x-states to another based on reading certain input letters is determined by the transition rules for FA 1. There are only finitely many possible collections of x-states, so the machine FA 2 has only finitely many states.

25. Lecture 05: Theory of Automata:08 25 Example Consider the regular expression r = aa*bb*. This defines the language where all the a’s come before all the b’s. The FA that accepts this language is:

26. Lecture 05: Theory of Automata:08 26 Let us now build FA 2 that accepts r* = (aa*bb*)*. We begin with the start state z 1 = x 1. In z 1, reading an a takes us to x 2 = z 2. Reading a b takes us to x 3 = z 3.

27. Lecture 05: Theory of Automata:08 27 In z 3, just like x 3, reading either an a or a b, we loop back to z 3. In z 2, if we read an a we go back to z 2. If we read a b, we go to x 4, or we have the option of jumping to the start state x 1 (since x 4 is a final state). Hence, let z 4 += x 4 or x 1.

28. Lecture 05: Theory of Automata:08 28 In z 4, what happens if we read an a? If z 4 = x 1, we go to x 2. If z 4 = x 4, we go to x 3. Hence, we will be in x 2 or x 3. So, let z 5 = x 2 or x 3. In z 4, if we read a b? If z 4 means x 1, we go x 3. If z 4 means x 4, we go to x 4 or jump to x 1 (due to final x 4 ). Thus, let z 6 = x 1 or x 3 or x 4. z 6 must be a final state since x 4 is. In z 5, reading an a takes us to x 2 or x 3, which is still z 5. So, we have an a-loop at z 5. In z 5, reading a b takes us to x 4 or x 1, or x 3, which is z 6. In z 6, reading an a, take us to x 2 or x 3, which is z 5. In z 6, reading a b takes us to x 3, x 4, or x 1, which is still z 6. So, we have a b-loop at z 6.

29. Lecture 05: Theory of Automata:08 29 The final picture is as follows:

30. Lecture 05: Theory of Automata:08 30 Example: Special Case Consider the FA below that accepts the language of all words having an odd number of b’s. Notice that this machine does not accept Λ and does allow the start state x 1 to be re-entered.

31. Lecture 05: Theory of Automata:08 31 In this case we need to represent x 1 as two separate z-states in FA 2, one as a start and final state ±, and the other as the non-final start state. The ± state is necessary for FA 2 to accept. The non-final start state is necessary for FA 2 to operate correctly, since some strings that return to the start state x 1 may not be valid words and therefore should not be accepted. Thus, we represent x1 (-) as two separate states: – z1 ± = x1 and a final state. – z2 = x1 and a non-final state.

32. Lecture 05: Theory of Automata:08 32 So, in z 1 reading an a, we go back to x 1 as a non-final state, which we labeled as z 2. In z 1, reading a b takes us to x 2, a final state, with the option of jumping back to the non-final start state x 1. So, let z 3 = x 1 or x 2. Note that z 3 must be a final state since x 2 is.

33. Lecture 05: Theory of Automata:08 33 In z2, if we read an a, we stay in z2. If we read a b, we go to x2 or back in x1 (as x2 is a final state); that is we go to z3. In z3, reading an a takes us to x1 or x2, which is z3 itself. In z3, reading a b takes us to x2 or x1, which is also z3. Hence, there is a a; b-loop at z3. The complete machine is

34. Lecture 05: Theory of Automata:08 34 We have finished the proof of part 3 of Kleene’s theorem. Because of Rules 1, 2, 3, and 4, we know that all regular expressions have corresponding finite automata that define the same language. This is because while we are constructing the regular expression from elementary building blocks using recursive definition, we can simultaneously be constructing the corresponding FA using the algorithms discussed above.

35. Lecture 05: Theory of Automata:08 35 Definition: A nondeterministic finite automaton (or NFA) is a TG with a unique start state and with the property that each of its edge labels is a single alphabet letter. The regular deterministic finite automata are referred to as DFAs, to distinguish them from NFAs. As a TG, an NFA can have arbitrarily many a-edges and arbitrarily many b-edges coming out of each state. An input string is accepted by an NFA if there exists any possible path from - to +.

36. Lecture 05: Theory of Automata:08 36 Examples of NFAs

37. Lecture 05: Theory of Automata:08 37 Theorem 7 for every NFA, there is some FA that accepts exactly the same language. Proof 1 By the proof of part 2 of Kleene’s theorem, we can convert an NFA into a regular expression, since an NFA is a TG. By the proof of part 3 of Kleene’s theorem, we can construct an FA that accepts the same language as the regular expression. Hence, for every NFA, there is a corresponding FA.

38. Lecture 05: Theory of Automata:08 38 Notes: Theorem 7 means that all NFAs can be converted into FAs. Clearly, all FAs can be considered as NFAs that do not make use of the option of extra freedom of edge production. Hence, as language acceptors, NFA = FA.

39. Lecture 05: Theory of Automata:08 39 Proof 2 of Theorem 7 We present the following constructive algorithm showing how to build a FA for a given NFA: Algorithm: Each state in FA is a collection of states from the original NFA, e.g. x i, or x j, or x k,... For every state z in the FA, the new state that an a-edge (or a b-edge) will take us to is just the collection of possible states that result from being in x i and taking the a-edge, or being in x j and taking the a-edge and so on.

40. Lecture 05: Theory of Automata:08 40 The start state of the FA that we are constructing is the same old start state we had to begin with in the NFA. Its a-edge (or b-edge) goes to the collection of the x-states that can be reached by an a- edge (or a b-edge) from the start state in the NFA. Since there may be no a-edges (or no b-edges) leaving the start state in the NFA (or leaving any particular state), we must add a state in the FA for the a-edge (or the b-edge) to reach in this case. The Φ state in the FA must have an a; b-loop going back to itself. A state in the FA is a final state if the collection of the x-states that it represents has an old final state in it.

41. Lecture 05: Theory of Automata:08 41 Example

42. Lecture 05: Theory of Automata:08 42 NFAs and Kleene’s theorem Rules 1, 2, 3, and 4 in Part 3 of Kleene’s theorem can be proved differently, using NFAs. We will show here how to prove Rule 1 and Rule 2. Proofs for Rule 3 and 4 using NFAs are considered in the problems section.

43. Lecture 05: Theory of Automata:08 43 Proof 2 of Rule 1, Part 3, Theorem 6 Rule 1 states that there are FAs for the languages {a}, {b}, and {Λ}. Proof: Step 1: The above three languages can all be accepted respectively by the NFAs below: Step 2: By Theorem 7, for every NFA, there is an equivalent FA. Hence, there must be FAs for these three languages as well.

44. Lecture 05: Theory of Automata:08 44 Proof 2 of Rule 2, Part 3, Theorem 6 Given FA 1 and FA 2, we shall present an algorithm for constructing a union machine FA 1 + FA 2. Algorithm: Step 1: Introduce a new and unique start state with two outgoing a-edges and two outgoing b-edges but no incoming edges. Connect them to the states that the start states of FA 1 and FA 2 are connected to. Do not erase the start states of FA 1 and FA 2, but erase their - signs, leaving all their edges intact. The new machine is an NFA that clearly accepts exactly language(FA 1 ) + language(FA 2 ). Step 2: Using the algorithm of Theorem 7, convert the NFA into an FA.

45. Lecture 05: Theory of Automata:08 45 Example Consider the FA 1 and FA 2 below:

46. Lecture 05: Theory of Automata:08 46 Example contd. Using the above algorithm (Step 1) we produce the following NFA.