1. 1 Unit 1: Automata Theory and Formal Languages Readings 1, 2.2, 2.3

2. 2 What is automata theory Automata theory is the study of abstract computational devices Abstract devices are (simplified) models of real computations Computations happen everywhere: On your laptop, on your cell phone, in nature, … Why do we need abstract models?

3. 3 A simple computer BATTERY SWITCH input: switch output: light bulb actions: flip switch states: on, off

4. 4 A simple “computer” BATTERY SWITCH off on start f f input: switch output: light bulb actions: f for “flip switch” states: on, off bulb is on if and only if there was an odd number of flips

5. 5 Another “computer” BATTERY off start 1 inputs: switches 1 and 2 actions: 1 for “flip switch 1” actions: 2 for “flip switch 2” states: on, off bulb is on if and only if both switches were flipped an odd number of times 1 2 1 off on 1 1 2 2 2 2

6. 6 A design problem Can you design a circuit where the light is on if and only if all the switches were flipped exactly the same number of times? 4 BATTERY 1 2 3 5 ?

7. 7 A design problem Such devices are difficult to reason about, because they can be designed in an infinite number of ways By representing them as abstract computational devices, or automata, we will learn how to answer such questions

8. 8 These devices can model many things They can describe the operation of any “small computer”, like the control component of an alarm clock or a microwave They are also used in lexical analyzers to recognize well formed expressions in programming languages: ab1 is a legal name of a variable in C 5u= is not

9. 9 Different kinds of automata This was only one example of a computational device, and there are others We will look at different devices, and look at the following questions: –What can a given type of device compute, and what are its limitations? –Is one type of device more powerful than another?

10. 10 Some devices we will see finite automataDevices with a finite amount of memory. Used to model “small” computers. push-down automata Devices with infinite memory that can be accessed in a restricted way. Used to model parsers, etc. Turing MachinesDevices with infinite memory. Used to model any computer. time-bounded Turing Machines Infinite memory, but bounded running time. Used to model any computer program that runs in a “reasonable” amount of time.

11. 11 Some highlights of the course Finite automata –We will understand what kinds of things a device with finite memory can do, and what it cannot do –Introduce simulation: the ability of one device to “imitate” another device –Introduce nondeterminism: the ability of a device to make arbitrary choices Push-down automata –These devices are related to grammars, which describe the structure of programming (and natural) languages

12. 12 Some highlights of the course Turing Machines –This is a general model of a computer, capturing anything we could ever hope to compute –Surprisingly, there are many things that we cannot compute, for example: –It seems that you should be able to tell just by looking at the program, but it is impossible to do! Write a program that, given the code of another program in C, tells if this program ever outputs the word “ hello ”

13. 13 Some highlights of the course Time-bounded Turing Machines –Many problems are possible to solve on a computer in principle, but take too much time in practice –Traveling salesman: Given a list of cities, find the shortest way to visit them and come back home –Easy in principle: Try the cities in every possible order –Hard in practice: For 100 cities, this would take 100+ years even on the fastest computer! Hong Kong Beijing Shanghai Xian Guangzhou Chengdu

14. 14 Preliminaries of automata theory How do we formalize the question First, we need a formal way of describing the problems that we are interested in solving Can device A solve problem B?

15. 15 Problems Examples of problems we will consider –Given a word s, does it contain the subword “ fool ”? –Given a number n, is it divisible by 7? –Given a pair of words s and t, are they the same? –Given an expression with brackets, e.g. (()()), does every left bracket match with a subsequent right bracket? All of these have “yes/no” answers. There are other types of problems, that ask “Find this” or “How many of that” but we won’t look at those.

16. 16 Alphabets and strings A common way to talk about words, number, pairs of words, etc. is by representing them as strings To define strings, we start with an alphabet Examples An alphabet is a finite set of symbols.  1 = {a, b, c, d, …, z} : the set of letters in English  2 = {0, 1, …, 9} : the set of (base 10) digits  3 = {a, b, …, z, #} : the set of letters plus the special symbol #  4 = { (, ) } : the set of open and closed brackets

17. 17 Strings The empty string will be denoted by  Examples A string over alphabet  is a finite sequence of symbols in . abfbz is a string over  1 = {a, b, c, d, …, z} 9021 is a string over  2 = {0, 1, …, 9} ab#bc is a string over  3 = {a, b, …, z, #} ))()(() is a string over  4 = { (, ) }

18. 18 Languages Languages can be used to describe problems with “yes/no” answers, for example: A language is a set of strings over an alphabet. L 1 = The set of all strings over  1 that contain the substring “fool” L 2 = The set of all strings over  2 that are divisible by 7 ={7, 14, 21, …} L 3 = The set of all strings of the form s#s where s is any string over {a, b, …, z} L 4 = The set of all strings over  4 where every ( can be matched with a subsequent )

19. 19 Finite Automata

20. 20 Example of a finite automaton There are states off and on, the automaton starts in off and tries to reach the “good state” on What sequences of f s lead to the good state? Answer: {f, fff, fffff, …} = {f n : n is odd } This is an example of a deterministic finite automaton over alphabet {f} off on f f

21. 21 Deterministic finite automata A deterministic finite automaton (DFA) is a 5-tuple (Q, , , q 0, F) where – Q is a finite set of states –  is an alphabet –  : Q ×  → Q is a transition function – q 0  Q is the initial state – F  Q is a set of accepting states (or final states). In diagrams, the accepting states will be denoted by double loops

22. 22 Example q0q0 q1q1 q2q2 10 00,11 alphabet  = {0, 1} set of states Q = {q 0, q 1, q 2 } initial state q 0 accepting states F = {q 0, q 1 } states inputs 0 1 q0q0 q1q1 q2q2 q0q0 q1q1 q2q2 q2q2 q2q2 q1q1 transition function 

23. 23 Language of a DFA The language of a DFA (Q, , , q 0, F) is the set of all strings over  that, starting from q 0 and following the transitions as the string is read left to right, will reach some accepting state. Language of M is {f, fff, fffff, …} = {f n : n is odd } off on f f M:M:

24. 24 q0q0 q1q1 q0q0 q1q1 q0q0 q1q1 q2q2 00 1 1 01 1 0 10 00,11 What are the languages of these DFAs? Examples

25. 25 Examples Construct a DFA that accepts the language L = {010, 1} (  = {0, 1} )

26. 26 Examples Construct a DFA that accepts the language Answer L = {010, 1} (  = {0, 1} ) qq q0q0 q1q1 q 01 q 010 q die 0, 1 0 1 0 1 01

27. 27 Examples Construct a DFA over alphabet {0, 1} that accepts all strings that end in 101

28. 28 Examples Construct a DFA over alphabet {0, 1} that accepts all strings that end in 101 Hint: The DFA must “remember” the last 3 bits of the string it is reading

29. 29 Examples 0 1 … … … … qq qq qq q  q  q  q  q  q  q  q  0 1 0 1 0 1 1 1 1 1 0 Construct a DFA over alphabet {0, 1} that accepts all strings that end in 101 Sketch of answer:

30. 30 Would be easier if… Suppose we could guess when the string we are reading has only 3 symbols left Then we could simply look for the sequence 101 and accept if we see it q die 1 0 1 3 symbols left This is not a DFA!

31. 31 Nondeterminism Nondeterminism is the ability to make guesses, which we can later verify Informal nondeterministic algorithm for language of strings that end in 101 : 1.Guess if you are approaching end of input 2.If guess is yes, look for 101 and accept if you see it 3.If guess is no, read one more symbol and go to step 1

32. 32 Nondeterministic finite automaton This is a kind of automaton that allows you to make guesses Each state can have zero, one, or more transitions out labeled by the same symbol 1 0 1 0, 1 qq qq qq qq

33. 33 Semantics of guessing 1 0 1 0, 1 qq qq qq qq State q 0 has two transitions labeled 1 Upon reading 1, we have the choice of staying in q 0 or moving to q 1

34. 34 Semantics of guessing 1 0 1 0, 1 qq qq qq qq State q 1 has no transition labeled 1 Upon reading 1 in q 1, we die; upon reading 0, we continue to q 2

35. 35 Semantics of guessing 1 0 1 0, 1 qq qq qq qq State q 3 has no transition going out Upon reading anything in q 3, we die

36. 36 Meaning of automaton 1 0 1 0, 1 qq qq qq qq Guess if you are 3 symbols away from end of input If so, guess you will see the pattern 101 Check that you are at the end of input

37. 37 Formal definition A nondeterministic finite automaton (NFA) is a 5-tuple (Q, , , q 0, F) where – Q is a finite set of states –  is an alphabet –  : Q ×  → subsets of Q is a transition function – q 0  Q is the initial state – F  Q is a set of accepting states (or final states). Only difference from DFA is that output of  is a set of states

38. 38 Example 1 0 1 0, 1 qq qq qq qq alphabet  = {0, 1} states Q = {q 0, q 1, q 2, q 3 } initial state q 0 accepting states F = {q 3 } states inputs 0 1 q0q0 q1q1 q2q2 {q 0, q 1 } transition function  q3q3 {q 0 } {q 2 }    {q 3 } 

39. 39 Language of an NFA The language of an NFA is the set of all strings for which there is some path that, starting from the initial state, leads to an accepting state as the string is read left to right. 1 0 1 0, 1 qq qq qq qq Example –1101 is accepted, but 0110 is not

40. 40 NFAs are as powerful as DFAs Obviously, an NFA can do everything a DFA can do But can it do more?

41. 41 NFAs are as powerful as DFAs Obviously, an NFA can do everything a DFA can do But can it do more? Theorem A language L is accepted by some DFA if and only if it is accepted by some NFA. NO!

42. 42 Proof of theorem To prove the theorem, we have to show that for every NFA there is a DFA that accepts the same language We will give a general method for simulating any NFA by a DFA Let’s do an example first

43. 43 Simulation example 1 0 0, 1 qq qq qq NFA: DFA: 1 qq q 0 or q 1 1 q 0 or q 2 1 0 0 0

44. 44 General method NFADFA states q 0, q 1, …, q n  q 0 }, {q 1 }, {q 0,q 1 }, …, {q 0,…,q n } one for each subset of states in the NFA initial state q0q0 q0}q0} transitions  ’({q i1,…,q ik }, a) =  (q i1, a) ∪ … ∪  (q ik, a) accepting states F  Q F’ = {S: S contains some state in F}

45. 45 Proof of correctness Lemma Proof by induction on n At the end, the DFA accepts iff it is in a state that contains some accepting state of NFA By lemma, this is true iff the NFA can reach an accepting state After reading n symbols, the DFA is in state {q i1,…,q ik } if and only if the NFA is in one of the states q i1,…,q ik

46. 46 Exercises Construct NFAs for the following languages over the alphabet {a, b, …, z} : –All strings that contain eat or sea or easy –All strings that contain both sea and tea –All strings that do not contain fool