1. 1 CSSE 350 Automata, Formal Languages, and Computability

2. 2 Overview of Theory of Computation Many important problems don’t have easy solutions –Vertex Cover Given an undirected graph G and a positive integer k, is there a subset V C of k nodes (in G) such that every arc has at least one of its endpoints in V C? –Hamiltonian Circuit Given a graph G is there a tour of G which passes through every vertex exactly once? –3-Coloring Given a graph G = (V;E) can you color each node with one of three colors so that no two adjacent nodes (i.e. nodes joined by an edge) have the same color? –Integer Programming Given a system of linear inequalities, in n variables and with integer coefficients, is there a non-negative integer solution?

3. 3 Overview of Theory of Computation –Rational Linear Programming Given a system of linear inequalities, in n variables and with integer coefficients, is there a solution (allowing fractional values as well as integers)? –2-Colouring given a graph G = (V,E) can you color each node with one of two colors so that no two adjacent nodes (i.e. nodes joined by an edge) have the same color? –……

4. 4 Overview of Theory of Computation Q1:What problems are solvable (in principle) by computers and what problems are not? –Note: Solvability in principle does not necessarily mean solvability in practice. Complexity theory (NP-hardness) answers. Q2: What problems are solvable (in practice) by computers and what problems are not? –This course will not try to answer Q2. Rather, we will concentrate on Q1.

5. 5 Overview of Theory of Computation Theory of Computation: –Mathematical study of computing machines and their capabilities. Why Theory of Computation? –Implementations come and go.

6. 6 Overview of Theory of Computation IBM 7090 Programming in the 1950’s ENTRY SXA 4,RETURN LDQ X FMP A FAD B XCA FMP X FAD C STO RESULT RETURN TRA 0 A BSS 1 B BSS 1 C BSS 1 X BSS 1 TEMP BSS 1 STORE BSS 1 END

7. 7 Overview of Theory of Computation Programming in the 1970’s (IBM 360) //MYJOB JOB (COMPRESS), 'VOLKER BANDKE',CLASS=P,COND=(0,NE) //BACKUP EXEC PGM=IEBCOPY //SYSPRINT DD SYSOUT=* //SYSUT1 DD DISP=SHR,DSN=MY.IMPORTNT.PDS //SYSUT2 DD DISP=(,CATLG), DSN=MY.IMPORTNT.PDS.BACKUP, // UNIT=3350,VOL=SER=DISK01, // DCB=MY.IMPORTNT.PDS, SPACE=(CYL,(10,10,20)) //COMPRESS EXEC PGM=IEBCOPY //SYSPRINT DD SYSOUT=* //MYPDS DD DISP=OLD,DSN=*.BACKUP.SYSUT1 //SYSIN DD * COPY INDD=MYPDS,OUTDD=MYPDS //DELETE2 EXEC PGM=IEFBR14 //BACKPDS DD DISP=(OLD,DELETE,DELETE), DSN=MY.IMPORTNT.PDS.BACKUP

8. 8 Overview of Theory of Computation Mathematical study of computing machines –Mathematically, how do we describe computing machines? –Mathematically, how do we describe the data computer manipulate? –Mathematically, what do computers compute?

9. 9 Overview of Theory of Computation Mathematically Describe Computing Machines Computation CPU memory

10. 10 CPU input memory output memory Program memory temporary memory Overview of Theory of Computation

11. 11 CPU input memory output memory Program memory temporary memory compute Example:

12. 12 CPU input memory output memory Program memory temporary memory compute

13. 13 CPU input memory output memory Program memory temporary memory compute

14. 14 CPU input memory output memory Program memory temporary memory compute

15. 15 CPU input memory output memory Program memory temporary memory Automaton Mathematically Describe Computing Machines

16. 16 Different Kinds of Automata Automata are distinguished by the temporary memory Finite Automata: no temporary memory Pushdown Automata: stack Turing Machines: random access memory

17. 17 input memory output memory temporary memory Finite Automaton Finite Automaton Example: Vending Machines (small computing power)

18. 18 input memory output memory Stack Pushdown Automaton Pushdown Automaton Example: Compilers for Programming Languages (medium computing power) Push, Pop

19. 19 input memory output memory Random Access Memory Turing Machine Turing Machine Examples: Any Algorithm (highest computing power)

20. 20 Finite Automata Pushdown Automata Turing Machine Power of Automata Less powerMore power Solve more computational problems

21. 21 Languages Overview of Theory of Computation Mathematically Describe the Data Computer Manipulate

22. 22 (1) Lexical analysis: Scan the program and break it up into variable names, numbers, etc. (2) Parsing: Create a tree that corresponds to the sequence of operations that should be executed, e.g., / + 10 2 5 (3) Optimization: Realize that we can skip the first assignment since the value is never used and that we can precompute the arithmetic expression, since it contains only constants. (4) Termination: Decide whether the program is guaranteed to halt. (5) Interpretation: Figure out what (if anything) useful it does. Let's Look at Some Problems int alpha, beta; alpha = 3; beta = (2 + 5) / 10;

23. 23 A Framework for Analyzing Problems We need a single framework in which we can analyze a very diverse set of problems. The framework we will use is Language Recognition A language is a (possibly infinite) set of finite length strings over a finite alphabet.

24. 24 Overview of Theory of Computation Strings –a finite sequence, possibly empty, of symbols drawn from some alphabet . Alphabet nameAlphabet symbolsExample strings The English alphabet { a, b, c, …, z } , aabbcg, aaaaa The binary alphabet { 0, 1 } , 0, 001100  is the empty string.  * is the set of all possible strings over an alphabet .

25. 25 Defining a Language A language is a (finite or infinite) set of strings over a finite alphabet . Examples: Let  = { a, b } Some languages over  : , {  }, { a, b }, { , a, aa, aaa, aaaa, aaaaa } The language  * contains an infinite number of strings, including: , a, b, ab, ababaa.

26. 26 Example Language Definitions L = {x  { a, b }* : all a ’s precede all b ’s}  a aa aba aabbb ba bb abc

27. 27 Example Language Definitions L = {x :  y  { a, b }* : x = y a } Simple English description:

28. 28 More Example Language Definitions L = {} =  L = {  }

29. 29 English L = {w: w is a sentence in English}. Examples: Kerry hit the ball. Colorless green ideas sleep furiously. The window needs fixed. Ball the Stacy hit blue.

30. 30 A Halting Problem Language L = {w: w is a C program that halts on all inputs}. Well specified. Can we decide what strings it contains?

31. 31 How Large is a Language? The smallest language over any  is , with cardinality 0. The largest is  *. How big is it?

32. 32 How Large is a Language? Theorem: If    then  * is countably infinite. Proof: The elements of  * can be lexicographically enumerated by the following procedure: Enumerate all strings of length 0, then length 1, then length 2, and so forth. Within the strings of a given length, enumerate them in dictionary order. This enumeration is infinite since there is no longest string in  *. Since there exists an infinite enumeration of  *, it is countably infinite.

33. 33 How Large is a Language? So the smallest language has cardinality 0. The largest is countably infinite. So every language is either finite or countably infinite.

34. 34 How Many Languages Are There? Theorem: If    then the set of languages over  is uncountably infinite. Proof: The set of languages defined on  is P (  *).  * is countably infinite. If S is a countably infinite set, P (S) is uncountably infinite. So P (  *) is uncountably infinite.

35. 35 What About Meaning? A n B n = { a n b n : n  0}. Do these strings mean anything?

36. 36 Semantic Interpretation Functions A semantic interpretation function assigns meanings to the strings of a language. English: I brogelled the yourtish. The semantic interpretation function for English is mostly compositional. He’s all thumbs.

37. 37 Semantic Interpretation Functions For formal languages: Programming languages Network protocol languages Database query languages HTML BNF For other kinds of “natural” languages: DNA

38. 38 Overview of Theory of Computation Mathematical study of computing machines –Mathematically, how do we describe computing machines? Finite automata (FA) Pushdown automata (PA or PDA) Turing machines –Mathematically, how do we describe the data computer manipulate? Strings of symbols (languages) Regular expressions Context-free languages (Unrestricted) grammars –Mathematically, what do computers compute? Functions from strings to strings Recursive functions

39. 39 The Big Picture

40. 40 A decision problem is simply a problem for which the answer is yes or no (True or False). A decision procedure answers a decision problem. Examples: Given an integer n, does n have a pair of consecutive integers as factors? The language recognition problem: Given a language L and a string w, is w in L? Our focus Decision Problems

41. 41 The Power of Encoding Everything is a string. Problems that don’t look like decision problems can be recast into new problems that do look like that.

42. 42 The Power of Encoding Pattern matching on the web: Problem: Given a search string w and a web document d, do they match? In other words, should a search engine, on input w, consider returning d? The language to be decided: { : d is a candidate match for the query w}

43. 43 The Power of Encoding Does a program always halt? Problem: Given a program p, written in some some standard programming language, is p guaranteed to halt on all inputs? The language to be decided: HP ALL = {p : p halts on all inputs}

44. 44 What If We’re Not Working with Strings? Anything can be encoded as a string. is the string encoding of X. is the string encoding of the pair X, Y.

45. 45 Primality Testing Problem: Given a nonnegative integer n, is it prime? An instance of the problem: Is 9 prime? To encode the problem we need a way to encode each instance: We encode each nonnegative integer as a binary string. The language to be decided: PRIMES = {w : w is the binary encoding of a prime number}.

46. 46 Problem: Given an undirected graph G, is it connected? Instance of the problem: 1 2 3 4 5 Encoding of the problem: Let V be a set of binary numbers, one for each vertex in G. Then we construct  G  as follows: Write |V| as a binary number, Write a list of edges, Separate all such binary numbers by “/”. 101/1/10/10/11/1/100/10/101 The language to be decided: CONNECTED = {w  {0, 1, /}* : w = n 1 /n 2 /…n i, where each n i is a binary string and w encodes a connected graph, as described above}. The Power of Encoding

47. 47 Protein sequence alignment: Problem: Given a protein fragment f and a complete protein molecule p, could f be a fragment from p? Encoding of the problem: Represent each protein molecule or fragment as a sequence of amino acid residues. Assign a letter to each of the 20 possible amino acids. So a protein fragment might be represented as AGHTYWDNR. The language to be decided: { : f could be a fragment from p}. The Power of Encoding

48. 48 Casting multiplication as decision: Problem: Given two nonnegative integers, compute their product. Encoding of the problem: Transform computing into verification. The language to be decided: L = {w of the form: x =, where: is any well formed integer, and integer 3 = integer 1  integer 2 } 12x9=108 12=12 12x8=108 Turning Problems Into Decision Problems

49. 49 Casting sorting as decision: Problem: Given a list of integers, sort it. Encoding of the problem: Transform the sorting problem into one of examining a pair of lists. The language to be decided: L ={w 1 # w 2 :  n  1 (w 1 is of the form, w 2 is of the form, and w 2 contains the same objects as w 1 and w 2 is sorted)} Examples: 1,5,3,9,6#1,3,5,6,9  L 1,5,3,9,6#1,2,3,4,5,6,7  L Turning Problems Into Decision Problems

50. 50 Casting database querying as decision: Problem: Given a database and a query, execute the query. Encoding of the problem: Transform the query execution problem into evaluating a reply for correctness. The language to be decided: L = {d # q # a: d is an encoding of a database, q is a string representing a query, and a is the correct result of applying q to d} Example: (name, age, phone), (John, 23, 567-1234) (Mary, 24, 234-9876)#(select name age=23)# (John)  L Turning Problems Into Decision Problems

51. 51 By equivalent we mean that either problem can be reduced to the other. If we have a machine to solve one, we can use it to build a machine to do the other using just the starting machine and other functions that can be built using a machine of equal or lesser power. The Traditional Problems and their Language Formulations are Equivalent

52. 52 Consider the multiplication example: L ={w of the form: x =, where: is any well formed integer, and integer 3 = integer 1  integer 2 } Given a multiplication machine, we can build the language recognition machine: Given the language recognition machine, we can build a multiplication machine: An Example

53. 53 Goal: Languages and Machines

54. 54 Applications of the Theory of Computation Finite State Machines for parity checkers, vending machines, communication protocols, and building security devices. Interactive games as nondeterministic Finite State Machines. Programming languages, compilers, and context-free grammars. Natural languages are mostly context-free. Speech understanding systems use probabilistic Finite State Machines. Computational biology: DNA and proteins are strings. The undecidability of a simple security model. Artificial intelligence: the undecidability of first- order logic.