1. CS/IT 138 THEORY OF COMPUTATION Chapter 1 Introduction to the Theory of Computation

2. Topic of course What are the fundamental capabilities and limitations of computers? To answer this, we will study abstract mathematical models of computers These mathematical models abstract away many of the details of computers to allow us to focus on the essential aspects of computation It allows us to develop a mathematical theory of computation

3. Review of set theory Can specify a set in two ways: - list of elements: A = {6, 12, 28} - characteristic property: B = {x | x is a positive, even integer} Set membership: 12  A, 9  A Set inclusion: A  B (A is a subset of B) A  B (A is a proper subset of B) Set operations: union: A  {9, 12} = {6, 9, 12, 28} intersection: A  {9, 12} = {12} difference: A - {9, 12} = {6, 28}

4. Set theory (continued) Another set operation, called “taking the complement of a set”, assumes a universal set. Let U = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} be the universal set. Let A = {2, 4, 6, 8} Then = U - A = {0, 1, 3, 5, 7, 9} The empty set: 

5. Set theory (continued) The cardinality of a set is the number of elements in a set. Let S = {2, 4, 6} Then |S| = 3 The powerset of S, represented by 2 S, is the set of all subsets of S. 2 S = {{}, {2}, {4}, {6},{2,4}, {2,6}, {4,6}, {2,4,6}} The number of elements in a powerset is |2 S | = 2 |S|

6. What does the title of this course mean? Formal language –a subset of the set of all possible strings from from a set of symbols –example: the set of all syntactically correct C programs Automata –abstract, mathematical model of computer –examples: finite automata, pushdown automata Turing machine

7. Alphabet = finite set of symbols or characters examples:  = {a,b}, binary, ASCII String = finite sequence of symbols from an alphabet examples: aab, bbaba, also computer programs A formal language is a set of strings over an alphabet Examples of formal languages over alphabet  = {a, b}: L 1 = {aa, aba, aababa, aa} L 2 = {all strings containing just two a’s and any number of b’s} A formal language can be finite or infinite. Formal language

8. We often use string variables; u = aab, v = bbaba Operations on strings length: |u| = 3 reversal: u R = baa concatenation: uv = aabbbaba The empty string, denoted, has some special properties: | | = 0 w  w w Formal languages (continued)

9. Operations on languages Set operations: L 1  L 2 = {x | x  L 1 or x  L 2 } is union L 1  L 2 = {x | x  L 1 and x  L 2 } is intersection L 1  L 2 = {x | x  L 1 and x  L 2 } is difference =  * - L is complement L 1  L 2 = (L 1 - L 2 )  (L 2 - L 1 ) is “symmetric difference” String operations: L R = {w R | w  L} is “reverse of language” L 1 L 2 = {xy | x  L 1, y  L 2 } is “concatenation of languages” L * = {x = x 1 …x k | k  0 and x 1, …, x k  L} = L 0  L 1  L 2.... is “Kleene star” or "star closure" L + = L 1  L 2.... is positive closure

10. Important example of a formal language alphabet: ASCII symbols string: a particular C++ program formal language: set of all legal C++ programs

11. Language-recognition problem There are many types of computational problem. We will focus on the simplest, called the “language-recognition problem.” Given a string, determine whether it belongs to a language or not. (Practical application for compilers: Is this a valid C++ program?) We study simple models of computation called “automata,” and measure their computational power in terms of the class of languages they can recognize.

12. Grammars A grammar G is defined as a quadruple: G = (V, T, S, P) Where V is a finite set of objects called variables T is a finite set of objects called terminal symbols S  V is a special symbol called the Start symbol P is a finite set of productions or "production rules" Sets V and T are nonempty and disjoint

13. Grammars Production rules have the form: x  y where x is an element of (V  T) + and y is in (V  T) * Given a string of the form w = uxv and a production rule x  y we can apply the rule, replacing x with y, giving z = uyv We can then say that w  z Read as "w derives z", or "z is derived from w"

14. Grammars If u  v, v  w, w  x, x  y, and y  z, then we say: * u  z This says that u derives z in an unspecified number of steps. Along the way, we may generate strings which contain variables as well as terminals. These are called sentential forms.

15. Grammars What is the relationship between a language and a grammar? Let G = (V, T, S, P) The set * L(G) = {w  T* : S  w} is the language generated by G.

16. Grammars Consider the grammar G = (V, T, S, P), where: V = {S} T = {a, b} S = S, P = S  aSb S 

17. Grammars What are some of the strings in this language? S  aSb  ab S  aSb  aaSbb  aabb S  aSb  aaSbb  aaaSbbb  aaabbb It is easy to see that the language generated by this grammar is: L(G) = {a n b n : n  0}

18. Grammars Let's go the other way, from a description of a language to a grammar that generates it. Find a grammar that generates: L = {a n b n+1 : n  0} So the strings of this language will be: b (0 a's and 1 b) abb (1 a and 2 b's) aabbb (2 a's and 3 b's)... In order to generate a string with no a's and 1 b, you might want to write rules for the grammar that say: S  ab a  But you can't do this; a is a terminal, and you can't change a terminal, only variables

19. Grammars So, instead of: S  ab a  we create another variable, A (we often use capital letters to stand for variables), to use in place of the terminal, a: S  Ab A  Now you might think that we can use another S rule here to generate the other part of the string, the a n b n part S  aSb But you can't, because that will generate ab, aabb, etc. Note, however, that if we use A in place of S, that will solve our problem: A  aAb

20. Grammars So, here are our rules: S  Ab A  aAb A  The S  Ab rule creates a single b terminal on the right, preceded by other strings (including possibly the empty string) on the left. The A  rule allows the single b string to be generated. The A  aAb rule and the A  rule allows ab, aabb, aaabbb, etc. to be generated on the left side of the string.

21. Automata

22. “Computer” or Turing machine (Alan Turing 1936) 0 1 2 3 X0XB0 Finite-state control Infinite tape or “memory” Read/write head

23. Finite automata Developed in 1940’s and 1950’s for neural net models of brain and computer hardware design Finite memory! Many applications: –text-editing software: search and replace –many forms of pattern-recognition (including use in WWW search engines) –compilers: recognizing keywords (lexical analysis) –sequential circuit design –software specification and design –communications protocols

24. Pushdown automata Noam Chomsky’s work in 1950’s and 1960’s on grammars for natural languages infinite memory, organized as a stack Applications: –compilers: parsing computer programs –programming language design

25. Automata, languages, and grammars In this course, we will study the relationship between automata, languages, and grammars Recall that a formal language is a set of strings over a finite alphabet Automata are used to recognize languages Grammars are used to generate languages (All these concepts fit together)

26. Classification of automata, languages, and grammars AutomataLanguageGrammar Turing machineRecursively enumerable Linear-bounded automaton Context sensitive Nondeterministic push-down automaton Context free Finite-state automaton regular

27. Besides developing a theory of classes of languages and automata, we will study the limits of computation. We will consider the following two important questions: –What problems are impossible for a computer to solve? –What problems are too difficult for a computer to solve in practice (although possible to solve in principle)?

28. Uncomputable (undecidable) problems Many well-defined (and apparently simple) problems cannot be solved by any computer Examples: –For any program x, does x have an infinite loop? –For any two programs x and y, do these two programs have the same input/output behavior? –For any program x, does x meet its specification? (i.e., does it have any bugs?)

29. Intractable problems We will learn how to mathematically characterize the difficulty of computational problems. There is a class of problems that can be solved in a reasonable amount of time and another class that cannot (What good is it for a problem to be solvable, if it cannot be solved in the lifetime of the universe?) The field of cryptography, for example, relies on the fact that the computational problem of “breaking a code” is intractable

30. Why study the theory of computing? Core mathematics of CS (has not changed in over 30 years) Many applications, especially in design of compilers and programming languages Important to be able to recognize uncomputable and intractable problems Need to know this in order to be a computer scientist, not simply a computer programmer