1. CS 3813: Introduction to Formal Languages and Automata Chapter 11 A Hierarchy of Formal Languages and Automata These class notes are based on material from our textbook, An Introduction to Formal Languages and Automata, 3 rd ed., by Peter Linz, published by Jones and Bartlett Publishers, Inc., Sudbury, MA, 2001. They are intended for classroom use only and are not a substitute for reading the textbook.

2. Diagrams from some slides are from a previous year’s textbook: Martin, John C., Introduction to Languages and the Theory of Computation. Boston: WCG McGraw- Hill, 1991. Slides are for use of this class only.

3. Functions A function is a mapping from a set of elements (called the domain) to another set of elements (called the range). If the domain and range are the set of strings over an alphabet, we call it a string function. If the domain and range are the set of natural numbers, we call it a number-theoretic function. Any natural number can be represented by a string. We will see later that any string can be represented by a natural number. (This will turn out to be important.)

4. Computability A function is partial Turing computable if there is a TM that computes it and the TM stops on all inputs in the domain of the function (which may be a subset of all possible inputs) A function is Turing computable if there is a TM that computes it and the TM stops on all inputs

5. Languages Given the set of all possible strings over an alphabet, a language is a subset of this set. A language can be represented by a characteristic function that has the set of all strings as its domain and {0, 1} as its range. It maps a string to 1 if it is in the language, and otherwise maps it to 0. When we extend the concept of computability to languages, we usually call it “decidability.”

6. Decidability A Turing computable language has a characteristic function that is Turing computable. A Turing computable language is also called a decidable language. A semi-decidable language has a TM that outputs 1 (or equivalently, halts) for every input string in the language, and does not halt for any input string that is not in the language. So, we talk about computability for functions, and decidability for languages. But it’s the same idea.

7. Review of definitions A function can be Turing computable, partial Turing computable, or uncomputable. What are the differences? A language can be decidable, semi- decidable, or undecidable. What are the differences?

8. Enumerability A language is said to be Turing enumerable if there is a TM that lists all the strings of the language. (Note that the TM never terminates if the language is infinite.) Some facts: – –A language is Turing enumerable if and only if it is semi-decidable. – –If a language and its complement are Turing enumerable, then the language is decidable. – –If a language is decidable, then its complement is decidable.

9. Church-Turing thesis This thesis (not theorem!) holds that any algorithmic procedure that can be performed by a human or computer can be performed by a TM. It can’t be proved, but is widely believed. First implication: instead of describing a TM in detail, we can describe a high-level algorithm and assume a TM can be described that computes it. Second implication: if we can show that a problem cannot be solved by any TM, we may conclude that it can’t be solved by any computer.

10. Universal Turing machine A TM that takes as input the description of a TM (a “program”) and an input string, simulates (“runs”) the TM on the input, and returns result. Can be viewed as a programmable TM. Equivalently, can be viewed as “interpreter” for TM programming language. Just as you can write an interpreter for C in C, you can construct a universal TM that is interpreter for TM programs. Although Turing developed the concept of a universal TM for theoretical reasons, it helped stimulate the development of stored-program computers.

11. “Programming” a universal Turing machine We can encode any TM as a unique string (or program) over some fixed alphabet, say {0,1}. We can encode any input to the TM as a string over the same alphabet There are many ways to do this and it doesn’t matter what method we use … what matters is that we can do this at all.

12. Important questions How many Turing machines are there? How many functions are there? How many computable functions are there? How many languages are there? How many decidable languages are there? We’ll come back to these questions later. To answer them, we first need to discuss what it means for a set to be countably or uncountably infinite. And for that, we begin with a review of set theory.

13. Review of set theory The cardinality of a set is the number of elements in a set. For example, Let S = {2, 4, 6}. Then |S| = 3. The powerset of S is the set of all subsets of S. For example, 2 S = {{}, {2}, {4}, {6}, {2,4}, {2,6}, {4,6}, {2,4,6}}

14. The cardinality of powersets We can use mathematical induction to prove that the cardinality of the powerset of a finite set, S, is 2 |S|. What about a more difficult question: what is the cardinality of the powerset of an infinite set?

15. Countable sets Two sets have the same cardinality if their elements can be put in 1-1 correspondence with each other An infinite set is countable if its elements can be placed in 1-1 correspondence with the natural numbers, that is, if its elements can be listed sequentially. Basically, this amounts to being able to specify what the first element of the set is, what the second is, etc.

16. The even, natural numbers are countable 1 2 3 4 5 6 7 … n... 2 4 6 8 10 12 14 … 2n... The set of even, natural numbers has the same cardinality as the set of natural numbers, although it is a strict subset of the set of natural numbers.

17. The integers are countable … -4 -3 -2 -1 0 1 2 3 4... … 9 7 5 3 1 2 4 6 8...

18. The rational numbers are countable 1/1 1/2 1/3 1/4 1/5 1/6 1/7... 2/1 2/2 2/3 2/4 2/5 2/6... 3/1 3/2 3/3 3/4 3/5... 4/1 4/2 4/3 4/4 … 5/1 5/2 5/3 … 6/1 6/2 … 7/1... What is the first rational number? 1/1 What is the second rational number?2/1 What is the third rational number? 1/2 What is the fourth rational number?3/1 etc. Here are the rational numbers:

19. The real numbers are uncountable (Cantor’s diagonal argument) f 0 : 0. f 0 (0) f 0 (1) f 0 (2) f 0 (3) … f 1 : 0. f 1 (0) f 1 (1) f 1 (2) f 1 (3) … f 2 : 0. f 2 (0) f 2 (1) f 2 (2) f 2 (3) … f 3 : 0. f 3 (0) f 3 (1) f 3 (2) f 3 (3) …... Consider the real number f defined as f(n) = f n (n) +1. Note that for every i, f(i) f i (i). Therefore f is not in list. This contradiction disproves the assumption that real numbers between 0 and 1 are countable. Assume the real numbers between 0 and 1 can be listed in order as infinite decimals.

20. The real numbers are uncountable Didn’t get that? OK; let’s try again. We can define the first real number. Let’s arbitrarily make 0.0 the first real number. That means that we can put it into one-to- one correspondence with the number 1. real #0.0 counting #1 But now what is the second real number? real #0.0X counting #12 No matter what number we pick for X, we can always find another real number in between the previous real number and X. For example, we can divide X by 2. That gives us another real number in between 0.0 and X.

21. The real numbers are uncountable Since we cannot specify what the second, third, fourth, etc. elements of the set of real numbers are, the set of real numbers in uncountable, or uncountably infinite. Definition: A set is uncountably infinite if it is impossible to sequentially list its elements Georg Cantor used this argument to distinguish between different levels of infinity. א 0 (aleph null) = infinity of integers א 1 (aleph one) = infinity of real numbers

22. The powerset of an infinite set S is uncountable The proof is by contradiction using diagonalization. Assume an infinite set S is countable; this means that the subsets of S can be listed in sequence. Order the elements of S sequentially. Represent each subset of S by an infinite row of 0’s and 1’s, where 1 indicates that the corresponding element of S occurs in it. Element # of elements in the original set 1 2 3 4 5 6... S 1 : 1 0 1 1 0 1 … S 2 : 0 0 1 1 0 0 … S 3 : 1 1 1 0 0 1 … S 4 : 1 0 1 0 1 1 … …

23. The powerset of an infinite set S is uncountable Element # of elements in the original set 1 2 3 4 5 6... S 1 : 1 0 1 1 0 1 … S 2 : 0 0 1 1 0 0 … S 3 : 1 1 1 0 0 1 … S 4 : 1 0 1 0 1 1 … … Consider, S x, a subset of S that differs from each of these at some point along the diagonal. It will be represented by: S x : 0 1 0 1 … Note that S x is a valid subset of S, but it is not identical to any of the subsets already listed. Its existence contradicts the assumption that the powerset of an infinite set is countable.

24. The powerset of an infinite set S is uncountable Don’t worry if you don’t get this right away; we will see this in more detail a few slides later on.

25. Formal languages and countability A formal language is a set of strings over an alphabet. Is this set countably or uncountably infinite? If the symbols of an alphabet are arranged in order, we can define a lexicographical ordering over the strings in any language over that alphabet. – –“alphabetical order” is an example of a lexicographic ordering What does this imply about the countability of the strings in any language?

26. Formal languages and countability Answer: The number of strings in a language is countably infinite. Proof: – –Divide the strings of the language into subsets based on their length; i.e., put all strings of length 1 together, all strings of length 2 together, etc. – –Within each set, put the strings in lexicographical order – –Merge the subsets, preserving their order – –Now put the strings into one-to-one correspondence with the counting numbers

27. Formal languages and countability Example: L = ww, where  = {a, b} 1aa 2bb 3aaaa 4abab 5baba 6bbbb 7aaaaaa... (the strings are listed in canonical order)

28. How many TMs are there? Because we can encode each TM as a string over an alphabet, the number of possible TMs is countably infinite. From this we may also conclude that the number of possible programs in any programming language is countably infinite.

29. Formal languages and countability (continued) Any language over  is a subset of  *. How many possible languages over  are there? (In other words, how many subsets of  * are there?)

30. How many languages are there? Answer: There are an uncountably infinite number of languages Proof: – –Any language over  is a subset of  * – –  * is an infinite set – –The powerset of  * is the number of subsets of  * – –The powerset of an infinite set is uncountable

31. How many TMs are there? What does this imply about whether all languages are decidable?

32. How many TMs are there? We have shown that: – – The number of strings in a language is countably infinite – –We can represent any Turing Machine as a string over the alphabet  = {0, 1} – –Therefore, the number of TMs is countably infinite – –But there are an uncountably infinite number of languages – –Consequently, we cannot put the number of TMs into one-to-one correspondence with the number of languages

33. How many TMs are there? This means that there are more languages than there are TMs. Every TM accepts all and only the strings of one specific language. Therefore, there must be some languages that cannot be recognized by any TM. Next chapter will talk about specific languages that are not decidable (and specific functions that are not computable).

34. 11.1: Recursive and recursively enumerable languages Remember that the strings that a TM accepts constitute the language of the Turing machine. We represent this as L(T). A Turing machine always accepts the words of its language by stopping in the halting state. However, it is allowed to reject strings that don’t belong to its language either by crashing (in a finite number of steps), or by looping forever.

35. Recursive and recursively enumerable languages Infinite loops are bad for us, because if we are trying to decide whether a string belongs to the language of a TM or not, we can’t tell after waiting a finite amount of time whether the TM is going to halt on the very next step, or whether it is going to go on forever. We would prefer to have our TMs crash to reject a string.

36. Recursive and recursively enumerable It turns out that these distinctions exactly correspond to the last two major classes of languages that we want to discuss in this course: Recursively enumerable = accepted by a TM that may loop (or may crash) to reject Recursive = accepted by a TM that always crashes to reject

37. Definition 11.1: If L   * is a language, then a Turing machine T with input alphabet  is said to accept L if L(T) = L. The Turing machine T recognizes or decides L if T computes the characteristic function  L :  *  {0, 1}. In other words, T halts for every string x in  *, outputting a 1 if x  L, and outputting a 0 otherwise.

38. Definitions: Definition 11.1: A language is recursively enumerable if there is a TM that accepts L. Definition 11.2: A language is recursive if there is a TM that recognizes L. This means that a language is recursive iff there exists a membership algorithm for it. Otherwise the language is recursively enumerable.

39. We also know: The set of recursive languages is a proper subset of the set of recursively enumerable languages.

40. Theorem: If L 1 and L 2 are recursively enumerable languages over , then L 1  L 2 and L 1  L 2 are also recursively enumerable languages.

41. Theorem: If L 1 and L 2 are recursive languages over , then L 1  L 2 and L 1  L 2 are also recursive languages. If L is a recursive language, then L is a recursive language. (the  means “complement”). (Proof: Obviously, just change the output of the TM from 0 to 1.)

42. Theorem: If L is a recursively enumerable language, and L is also recursively enumerable, then L must be recursive. Another way to say this is that the only way that a language L and its complement L can both be recursively enumerable is if both are recursive. Think about this. This implies that the complement of a non-recursive recursively enumerable language is... what?

43. Theorem: The complement of a non-recursive recursively enumerable language is a language that is not recursively enumerable. This means that the language cannot be accepted by a Turing Machine........ which means that NO automaton can accept the language.

44. 11.1: Enumerating a language Putting a set of strings in canonical order means listing the shortest strings first, and listing the strings of the same length alphabetically. So the set of strings {abb, a, ba, aa, b} would look like this in canonical order: {a, b, aa, ba, abb}. Enumerating a set means to list the elements of the set one at a time – to put them into one-to- one correspondence with the positive integers.

45. Theorem: A language L   * is recursively enumerable (that is, can be accepted by some TM) if and only if L can be enumerated by some TM. How would the TM do this?

46. Theorem: One way is to list every possible string in canonical order: {, a, b, aa, ab, ba, bb, aab, …} Next, construct a universal TM that contains within it a simulation of the TM that accepts L. Have it write 0 on its on tape to start off. Now run the UTM on the strings. To avoid infinite loops, we make a series of passes:

47. Theorem: 1 st pass: The UTM generates the string and simulates one move of the TM on that input. 2 nd pass: The UTM simulates two moves of the TM on the string, then generates the string a and simulates one move of the TM on that input. 3 rd pass: The UTM simulates three moves of the TM on the string, two moves of the TM on the string a, then generates the string b and simulates one move of the TM on that input.... and so on.

48. Theorem: Whenever the TM accepts a string, the UTM writes the next integer on its tape. Every string that is accepted by the TM is accepted after a finite number of moves. You can see that eventually, after a finite series of moves, all the strings belonging to L will be accepted by the TM, and the UTM will have written a series of integers on its tape.

49. Another observation: Note that, for some languages, the TM may accept a longer string in fewer passes than a shorter string. However, if the language is recursive (not just recursively enumerable) then it turns out that all strings will be accepted in canonical order.

50. Theorem: L is recursive if and only if there is a TM that enumerates L in canonical order.

51. Theorem 11.2: Not all languages are recursively enumerable Theorem 11.1: If S is an infinite countable set, its powerset 2 S is not countable. Proof: Use Cantor’s diagonalization demonstration in a proof by contradiction.

52. Cantor’s diagonalization proof: Assume that 2 S is countable. If S = {s 1, s 2, s 3, …}, then we can represent any element t of 2 S by a binary number in which the 1’s represent the elements of S that are in t and the 0’s represent the elements of S that are not in this particular t.

53. Cantor’s diagonalization proof: For example: t 1 00000… t 2 10000… t 3 01000… t 4 11000… t 5 00100… t 6 10100… t 7 11100…...

54. Cantor’s diagonalization proof: So 0101000… represents the set {s 2, s 4 }. But you can look at this pattern of 1’s and 0’s as if it were a binary number printed in reverse order: …0001010, which represents the integer 9. Each different element of 2 S will correspond to a unique integer. In the table above, t 1 corresponds to 0, t 2 corresponds to 1, etc.

55. Cantor’s diagonalization proof: After we have listed all of the elements of 2 S in a matrix, Cantor says we can create a new subset of 2 S that does not appear in our matrix. Simply take the first number of the first element, the second number of the second element, and so on, and reverse them. The resulting number does not represent any element of 2 S, because it differs from each element by at least one number. However, by definition, the powerset of S contains all possible subsets of S. Ergo, 2 S cannot be countable.

56. Cantor’s diagonalization proof: Cantor used the diagonalization argument to show that the real numbers could not be put into one-to-one correspondence with the integers. The real numbers are not enumerable, or countable.

57. Cantor’s diagonalization proof: What does this have to do with formal languages, you may ask.

58. Cantor’s diagonalization proof: We showed earlier how you can encode any TM in the form of a finite binary pattern. Each different TM will be represented by a unique pattern of 1’s and 0’s. We can interpret that pattern as a binary number, corresponding to a specific integer. There is an infinite number of different possible TMs (just as there is an infinite number of integers), but the set of all possible TM’s is countable – infinite, but countably infinite.

59. Cantor’s diagonalization proof: A language L is a set of strings; you can think of each different language as a different subset of  *. Conversely, each different subset of  * constitutes a different language. Languages may have an infinite number of strings in them. (Even regular languages may be infinite: how about the language of all strings that begin with the letter a?)

60. Cantor’s diagonalization proof: The set of all possible languages is the set of all possible subsets of strings: 2  *. Since  * is infinite, the set of all languages on  is not countable.

61. Cantor’s diagonalization proof: Every recursively enumerable language can be represented by a TM. But the number of TMs is countable, while the number of different languages is not countable. Hence, there must exist some languages which cannot be represented by any TM. These languages are not recursively enumerable.

62. Cantor’s diagonalization proof: I know what you’re going to ask: can you give a good, clear example of a language which is not recursively enumerable? My answer is, “NO”! (But see your book, pages 279-280).

63. Theorem: Since we can encode any TM as a string of 1’s and 0’s, we can talk about a TM which accepts the string that represents itself. Let’s call this TM SA, for “self- accepting”.

64. Theorem: The language SA is recursively enumerable but not recursive. That is, there is at least one input string that will cause it to loop forever to reject the string.

65. Theorem: Now let’s imagine a TM that accepts all strings except the string that represents itself. Call this TM NSA, for “non-self accepting). We know that if L is a recursively enumerable language, and L is also recursively enumerable, then L must be recursive. The language NSA is not recursively enumerable.

66. Grammars Now let’s take a final look at grammars.

67. Unrestricted Grammars A grammar G = (V, T, S, P) is called unrestricted if all the productions are of the form u  v where u is in (V  T) + and v is in (V  T) * Basically, the only restriction is that λ is not allowed as the left side of a production.

68. Example: Let L = {ww | w  {a, b}*}. We already know that a context-free grammar cannot produce this language. But the following unrestricted grammar will produce all and only the strings of L: S  FMAM  Ma F  FaABM  Mb F  FbB F  Aa  aA M  Ab  bA Ba  aB Bb  bB

69. Unrestricted Grammars Any language generated by an unrestricted grammar is recursively enumerable.

70. Theorem: A Turing machine may be constructed to accept any recursively enumerable language. Formally: If G = (V, T, S, P) is any unrestricted grammar, then there is a Turing machine T = (Q, , , q 0,  ) with L(T) = L(G).

71. Theorem: Proof by construction: Construct an NTM T to accept L(G). It will be a composite TM: T= MovePastInput  Simulate  Equal Prior to processing, the input string is copied onto the tape.

72. Theorem: Proof by construction: MovePastInput moves the tape head to the first blank after the input string. Simulate simulates a derivation of the string in the grammar G and leaves the derivation of the string on the tape. (Since it is an NTM, it can “guess” the right string to generate.) Equal compares them. If the strings are equal, the machine erases the second string, leaving the original input string on the tape as the output of the TM. The TM will reject every string that can’t be generated by G, either by crashing during Simulate, or by looping forever while trying to generate the string.

73. Simulate: The Simulate part of this composite TM will be different for every different grammar that we want to build a TM for. The next slide shows what the Simulate part would look like for the grammar S  aBS | aB  Ba Ba  aB B  b which generates strings with equal numbers of a’s and b’s.

74. Next slide:

75. Context-sensitive grammars: Definition: A context sensitive grammar (CSG) is an unrestricted grammar in which every production has the form:   where |  |  (Note that, if we interpret this rule strictly, can never be part of a context-sensitive language.)

76. Context-sensitive Grammars A grammar G = (V, T, S, P) is called context sensitive if all the productions are of the form x  y where x and y are in (V  T) + and |x|  |y| (Note that, if we interpret this rule strictly, λ can never be part of a context-sensitive language.)

77. Context-sensitive grammars: The reason why we call these grammars context- sensitive is that we can rewrite the productions of any context-sensitive grammar to be in this form: xAy  xvy where x, y, and v are strings of any combination of variables and terminals, v is nonnull, and A is a single variable

78. Context-sensitive grammars: Let’s look more closely at this rule: xAy  xvy We can describe this rule by saying that: A goes to v in the context of x on the left and y on the right

79. Compare to context-free: Remember that, in a context-free grammar, the productions all have a single variable on the left, such as: A  aa This rule states that we can always replace the variable A with two terminal a’s anywhere A happens to occur in an intermediate string.

80. Context-sensitive: However, in a context-sensitive grammar, we might have two rules such as: aAa  aba bAb  bbabb which say that we can replace A with a single b when it is in the middle of two a’s, but we replace it with bab when it is in the middle of two b’s.

81. Context-sensitive: Clearly, context-sensitive rules give a grammar more power. A context-sensitive grammar can use the surrounding characters to decide to do different things with a variable, instead of always having to do the same thing every time.

82. Context-sensitive: All productions in context-sensitive grammars are non-decreasing or non-contracting; that is, they never result in the length of the intermediate string being reduced.

83. Context-sensitive: Suppose that a TM, T, simulating a context- sensitive grammar is trying to generate a particular string s to see if the string belongs to L(T), the language of the Turing machine. Assume that |s| is 6, and the TM has just written 7 characters on the tape. Is there any way that this intermediate string is ever going to shrink back to 6 characters? No, not if the TM is a correct implementation of the context-sensitive grammar.

84. This means two important things: 1. In a TM that is simulating a context-sensitive grammar, we never need to have more cells on our tape than the number of characters in the string that the TM has been given to process.

85. This means two important things: 2. We never have to loop infinitely to reject a string. As soon as the TM has checked all the valid intermediate strings of its language that are less than or equal to the length of the input string, if it hasn’t found a match then it is never going to. With a minor modification, the TM can crash immediately to reject the string, and does not have to go on examining additional longer intermediate strings ad infinitum.

86. Note: This is NOT true for standard TMs that simulate unrestricted grammars; unrestricted grammars can have any kind of grammar rule, including rules that shrink the size of the intermediate string. Thus we might need a tape hundreds of cells longs to process a string that ends up being only 2 terminals long. So we can’t arbitrarily restrict the length of a tape for unrestricted grammars and recursively enumerable languages.

87. Linear-bounded automaton: A Turing machine that has the length of its tape limited to the length of the input string is called a linear-bounded automaton (LBA).

88. Definition: A linear bounded automaton is a 5-tuple M = (Q, , , q 0,  ) that is a nondeterministic Turing machine except that: 1. There are two extra tape symbols, which are not elements of . 2. The TM begins in the configuration (q 0, ), with its tape head scanning the symbol symbol is in the cell immediately to the right of the input string x. 3. The TM cannot replace with anything else, nor move the tape head left of.

89. LBAs and CSLs: The languages corresponding to LBAs are precisely the class of context-sensitive languages. We can break this down into two theorems:

90. Theorem 11.4: If L   * is a context-sensitive language, then there is a linear-bounded automaton accepting L.

91. Theorem 11.5: If there is a linear bounded automaton M = (Q, , , q 0,  ) accepting the language L   *, then there is a context-sensitive grammar generating L – { }.

92. LBAs: One last fact about LBAs: The definition of LBAs specifies that they be nondeterministic. It is not known if nondeterminism is necessary or not, but no proof currently exists that a deterministic version of an LBA would be sufficiently powerful to recognize all context- sensitive grammars.

93. Context-sensitive Grammars Most characteristics of human languages can be described by context-sensitive grammars. However, human languages are creative (we don’t have to obey the rules of the language, or can make up new rules), so they are probably really not even recursively enumerable.

94. But what about recursive languages? Theorem: Every context-sensitive language is recursive. Why? Because there is a TM (specifically, an LBA) that computes the characteristic function (often called a membership algorithm) for each CSL.

95. Recursive vs. context-sensitive: Theorem: There exists at least one recursive language L over {a, b} such that L – { } is not context- sensitive. (see book for proof) The previous two theorems together show that the context-sensitive languages are a proper subset of the recursive languages: {context-sensitive languages}  {recursive languages}

96. Recursive languages: This means that LBAs can’t recognize all recursive languages. In fact, we don’t have a different type of automaton that that can recognize all and only the recursive languages – just TMs that always halt or crash, and never loop. Also, recursive languages don’t seem to have a grammar that corresponds to them. (Strange, but true.)

97. Summary: languages We have studied seven types of languages in this course. They can be represented by a set of concentric circles, each circle representing the fact that the languages form proper subsets of one another as you go from the outer to the inner circles.

98. Languages: regular deterministic context-free non-deterministic context-free context-sensitive recursive recursively-enumerable non-recursively enumerable

99. All of the automata we study in this class have a finite number of states. They differ in the “auxiliary memory” they have and how it is organized. DFA / NFA DPDA NPDA LBA DTM / NTM no auxiliary memory infinite stack memory infinite tape memory proportional tape memory Summary: Automata

100. Automata: finite-state automata deterministic push-down automata linear-bounded automata Turing machines non-deterministic push-down automata

101. Summary: grammars Noam Chomsky presented four classes of grammars for generating languages. They can be represented by a set of concentric circles, each circle representing the fact that the grammars form proper subsets of one another and become weaker as you go from the outer to the inner circles. Each corresponds to a class of languages and to a type of automaton.

102. Grammars: regular (Type 3) context-free (Type 2) Phrase-structure, or unrestricted (Type 0) context-sensitive (Type 1)

103. Chomsky Hierarchy of Grammars Language Type Languages Generated Production Rules X  Y Automaton 0Phrase structure (recursively enumerable) X = any string with  1 nonterminal Y = any string TM 1Context-sensitive X = any string with  1 nonterminal Y = any string with length  |X| LBA 2Context-freeX = 1 nonterminal Y = any string PDA 3RegularX = 1 nonterminal Y = 1 terminal OR Y = 1 terminal and 1 nonterminal FA