1. 1 Formal Languages and Theory of Computation Wen-Hsiang Tsai 蔡文祥 講座教授 2011/9 Taj Mahal in India

2. 2 Chapter 0 Introduction Opera House in Sidney, Australia

3. Chapter. 0 – p. 3 Outline 0.0 Reasons for Taking This Course 0.0 Reasons for Taking This Course 0.1 General Concepts 0.1 General Concepts 0.2 Problems Studied in Theory of Computation 0.2 Problems Studied in Theory of Computation 0.3 Fields Related to Theory of Computation 0.3 Fields Related to Theory of Computation 0.4 Applications of Theory of Computation 0.4 Applications of Theory of Computation 0.5 Brief History of Theory of Computation 0.5 Brief History of Theory of Computation 0.6 Textbooks and Course Grading 0.6 Textbooks and Course Grading

4. Chapter. 0 – p. 4 0.0 Reasons for Taking This Course Philosophy of computer science (CS) Philosophy of computer science (CS) Indispensable for complete study of CS Indispensable for complete study of CS Broadening views in all fields of CS Broadening views in all fields of CS Enhancing problem solving capability Enhancing problem solving capability Basic training for undergraduate students Basic training for undergraduate students Background course for graduate students Background course for graduate students Qualification exam topic for Ph. D. students. Qualification exam topic for Ph. D. students.

5. Chapter. 0 – p. 5 0.1 General Concepts Course title --- Course title --- “Formal Languages and Theory of Computation” ( 正規語言與計算理論 ) Scope of course: Scope of course: Formal languages Formal languages Automata theory Automata theory Computability Computability Computational complexity Computational complexity

6. Chapter. 0 – p. 6 0.1 General Concepts Formal Languages --- Formal Languages --- Definition --- Definition --- Study of “strings of symbols,” including their descriptions, properties, descriptions, properties, generations, recognitions, generations, recognitions, compiling, applications, … compiling, applications, …

7. Chapter. 0 – p. 7 0.1 General Concepts Formal Languages --- Formal Languages --- Concepts (1/2) --- Concepts (1/2) --- Strings of symbols include those of natural languages, natural languages, computer languages, computer languages, discrete signals, … discrete signals, …

8. Chapter. 0 – p. 8 0.1 General Concepts Formal Languages --- Formal Languages --- Concepts (2/2) --- Concepts (2/2) --- Results of study of automatic processing of symbolic information Results of study of automatic processing of symbolic information Parallel to numerical mathematical processing Parallel to numerical mathematical processing Similar to the role of set theory ( 集合論 ) in mathematics Similar to the role of set theory ( 集合論 ) in mathematics

9. Chapter. 0 – p. 9 0.1 General Concepts Automata Theory --- Automata Theory --- A question --- A question --- Do you know how a vending machine works? Can you design one? Do you know how a vending machine works? Can you design one? Vending machine room seen in Hokkaido, Japan 2004

10. Chapter. 0 – p. 10 0.1 General Concepts Automata Theory --- Automata Theory --- An example (1/3) --- An example (1/3) --- How to design a vending machine? How to design a vending machine?  Use a finite automaton!  Use a finite automaton!

11. Chapter. 0 – p. 11 0.1 General Concepts Automata Theory --- Automata Theory --- An example (2/3) --- An example (2/3) --- Assumptions (for simplicity): Assumptions (for simplicity): Only NT 5-dollar and 10-dollar coins are used. Only NT 5-dollar and 10-dollar coins are used. Only drinks all of 20 dollars are sold. Only drinks all of 20 dollars are sold.

12. Chapter. 0 – p. 12 0.1 General Concepts Automata Theory --- Automata Theory --- An example (3/3) --- An example (3/3) --- Requiring “memory” called “states” for the design. Requiring “memory” called “states” for the design. Notes: see next page. Notes: see next page. $0 start $5 $10 $5 $15 $5 $10 $10 $10 $5 $20 transition diagram

13. Chapter. 0 – p. 13 0.1 General Concepts $10 $5 is returned as “output” (not shown) $20 “Final state” Automata Theory --- Automata Theory --- Notes for the last diagram: Notes for the last diagram: $0 start “Start state”

14. Chapter. 0 – p. 14 0.1 General Concepts Automata Theory --- Automata Theory --- Definition --- Definition --- Study of dynamic behaviors of “discrete- parameter information systems” in form of “abstract computing devices” or “machines.”

15. Chapter. 0 – p. 15 0.1 General Concepts Automata Theory --- Automata Theory --- Examples of discrete-parameter information systems --- Examples of discrete-parameter information systems --- digital systems, nerve systems, languages digital systems, nerve systems, languages information transmission systems information transmission systems human-environment interactions, … human-environment interactions, …

16. Chapter. 0 – p. 16 0.1 General Concepts Automata Theory --- Automata Theory --- Three major models of automata --- Three major models of automata --- generator --- with output and without input generator --- with output and without input acceptor --- with input and without output acceptor --- with input and without output transducer --- both with input and with output transducer --- both with input and with output generatoracceptortransducer trigger signal Yes or No

17. Chapter. 0 – p. 17 0.1 General Concepts Automata Theory --- Automata Theory --- Examples --- Examples --- generator --- generator --- “natural language” grammar (generating “sentences” spoken by people) “natural language” grammar (generating “sentences” spoken by people) reception robot reception robot (speaking organized words and sentences) (speaking organized words and sentences) context-free grammar context-free grammar (generating strings of symbols) *** (generating strings of symbols) *** (*** abstract models studied in this course) Reception robot --- Expo 2005

18. Chapter. 0 – p. 18 0.1 General Concepts Automata Theory --- Automata Theory --- Examples --- Examples --- acceptor --- acceptor --- digital lock digital lock (accepting digits) (accepting digits) lexical analyzer lexical analyzer (recognizing computer language keywords) (recognizing computer language keywords) finite automaton finite automaton (accepting valid strings of symbols) *** (accepting valid strings of symbols) *** (*** abstract models studied in this course) Digital lock

19. Chapter. 0 – p. 19 0.1 General Concepts Automata Theory --- Automata Theory --- Examples --- Examples --- transducer --- transducer --- Interpreter Interpreter (translating natural languages) (translating natural languages) Compiler Compiler (translating high-level languages into machine codes) (translating high-level languages into machine codes) Turing machine Turing machine (transforming strings of symbols) *** (transforming strings of symbols) *** (*** abstract models studied in this course) “ Interpreter ” 雙面翻譯 (a movie)

20. Chapter. 0 – p. 20 0.1 General Concepts Computability --- Computability --- Definition --- Definition --- Study of problem solving capabilities of computational models.

21. Chapter. 0 – p. 21 0.1 General Concepts Computability --- Computability --- Problem types based on resources --- Problem types based on resources --- Impossible problems Impossible problems Possible-with-unlimited-resources-but- impossible-with-limited-resources problems Possible-with-unlimited-resources-but- impossible-with-limited-resources problems Possible-with-limited-resources problems Possible-with-limited-resources problems

22. Chapter. 0 – p. 22 0.1 General Concepts Computability --- Computability --- Problem types based on time --- Problem types based on time --- Undecidable problems Undecidable problems Intractable problems Intractable problems Tractable problems Tractable problems

23. Chapter. 0 – p. 23 0.1 General Concepts Computability --- Computability --- Advantage --- Advantage --- Studies of computability help us not to waste time on “unsolvable problems” already investigated before.

24. Chapter. 0 – p. 24 0.1 General Concepts Computational complexity --- Computational complexity --- Definition --- Definition --- Study of “efficiency” of problem solving.

25. Chapter. 0 – p. 25 0.1 General Concepts Computational complexity --- Computational complexity --- Concepts (1/2) --- Concepts (1/2) --- To unify comparison, an abstract model is needed as the machine for executing problem solutions. To unify comparison, an abstract model is needed as the machine for executing problem solutions. Usually the most famous “Turing machine” (an automaton) is used. Usually the most famous “Turing machine” (an automaton) is used.

26. Chapter. 0 – p. 26 0.1 General Concepts Computational complexity --- Computational complexity --- Concepts (2/2) --- Concepts (2/2) --- Turing machine, though simple, has been proved to be able to simulate any problem solving steps (“algorithms”) designed by human beings! Turing machine, though simple, has been proved to be able to simulate any problem solving steps (“algorithms”) designed by human beings! Turing machine is the foundation for development of modern computation theory! Turing machine is the foundation for development of modern computation theory!

27. Chapter. 0 – p. 27 0.2 Problems Studied in Theory of Computation Essential problems studied --- Essential problems studied --- “What are the fundamental capabilities and limitations of computers?” “What are the fundamental capabilities and limitations of computers?” What can a computer do at all? --- What can a computer do at all? --- studied in the domain of Computability! What can a computer do efficiently? --- studied in the domain of Computational complexity! What can a computer do efficiently? --- studied in the domain of Computational complexity!

28. Chapter. 0 – p. 28 0.2 Problems Studied in Theory of Computation A basic concept --- A basic concept --- A Turing machine conceptually is no more than a pen and a paper with several reasoning rules! A Turing machine conceptually is no more than a pen and a paper with several reasoning rules! Turing machine

29. Chapter. 0 – p. 29 0.3 Fields Related to Scope of Theory of Computation FieldsRelated theory Compiling theoryFormal languages Switching circuit theoryAutomata theory Algorithm analysisComputational complexity Natural language processingFormal languages Syntactic pattern recognitionFormal languages Programming languagesFormal languages Artificial intelligenceFormal languages and automata theory Neural networksAutomata theory

30. Chapter. 0 – p. 30 0.4 Applications of Theory of Computation Applications --- Applications --- Text analysis --- Text analysis --- text search ( 文辭搜尋 ) text search ( 文辭搜尋 ) text editing ( 文句編輯 ) text editing ( 文句編輯 ) Compiler design ( 編譯器設計 ) Compiler design ( 編譯器設計 ) lexical analysis ( 辭彙分析 ) lexical analysis ( 辭彙分析 ) parser generation ( 剖析器產生 ) parser generation ( 剖析器產生 )

31. Chapter. 0 – p. 31 0.4 Applications of Theory of Computation Applications --- Applications --- Language design Language design programming language design ( 程式語言設計 ) programming language design ( 程式語言設計 ) document description language design ( 公文 描述語言設計 ) document description language design ( 公文 描述語言設計 ) e.g., HTML, XML, … e.g., HTML, XML, … picture language design ( 圖畫語言設計 ) picture language design ( 圖畫語言設計 ) e.g., SVG, VHML, … e.g., SVG, VHML, … special language design ( 特別語言設計 ) special language design ( 特別語言設計 )

32. Chapter. 0 – p. 32 0.4 Applications of Theory of Computation Applications --- Applications --- Digital system design Digital system design computer design ( 計算機設計 ) computer design ( 計算機設計 ) special digital system design ( 特殊數位系統 設計 ) special digital system design ( 特殊數位系統 設計 ) Protocol modeling and verification ( 網路 協定模式化與驗証 ) Protocol modeling and verification ( 網路 協定模式化與驗証 ) Expert system design( 專家系統設計 ) Expert system design( 專家系統設計 ) Cryptography( 密碼學 ) … Cryptography( 密碼學 ) …

33. Chapter. 0 – p. 33 0.4 Applications of Theory of Computation A specific application --- special digital system design A specific application --- special digital system design Mainly based on automata theory --- Mainly based on automata theory --- using finite automata or “sequential machines” (finite automata both with input and with output) Some specific examples of digital systems --- Some specific examples of digital systems --- Elevators, vending machines, traffic signal control systems … Elevators, vending machines, traffic signal control systems …

34. Chapter. 0 – p. 34 0.5 Brief History of Theory of Computation At beginning of 19th century, mathematicians believe: At beginning of 19th century, mathematicians believe: Axiomatization of mathematics ( 數學公理化 ) That is, That is, it is possible to decide the truth of any mathematical statement thru use of inference rules and axiom systems. it is possible to decide the truth of any mathematical statement thru use of inference rules and axiom systems.

35. Chapter. 0 – p. 35 0.5 Brief History of Theory of Computation In 1931, Kurt Gödel proposed the incompleteness theorem which says: In 1931, Kurt Gödel proposed the incompleteness theorem which says: the truth of certain mathematical statements cannot be proved by the use of mathematical axioms. the truth of certain mathematical statements cannot be proved by the use of mathematical axioms. That is, That is, some mathematical problems cannot be solved by algorithms. some mathematical problems cannot be solved by algorithms.

36. Chapter. 0 – p. 36 0.5 Brief History of Theory of Computation From then, the ideal of axiomatization of mathematics was broken. From then, the ideal of axiomatization of mathematics was broken. Then, mathematicians started to Then, mathematicians started to think the scope of algorithm capabilities, think the scope of algorithm capabilities, design theoretical computational machines, & design theoretical computational machines, & study their capability limitations. study their capability limitations.

37. Chapter. 0 – p. 37 0.5 Brief History of Theory of Computation Many theories were proposed, but the corresponding machines could not be manufactured at that time. Many theories were proposed, but the corresponding machines could not be manufactured at that time.

38. Chapter. 0 – p. 38 0.5 Brief History of Theory of Computation In 1936, Turing proposed the model of In 1936, Turing proposed the model of universal algorithm machine, universal algorithm machine, which later was called Turing machine Turing machine mentioned before.

39. Chapter. 0 – p. 39 0.5 Brief History of Theory of Computation Some similar ideas were proposed by other scholars, including Some similar ideas were proposed by other scholars, including Stephen C. Kleene, Stephen C. Kleene, Alonzo Church, Alonzo Church, Emil Post, etc., Emil Post, etc., in different forms of models. in different forms of models.

40. Chapter. 0 – p. 40 0.5 Brief History of Theory of Computation In 1943, neural physiologists In 1943, neural physiologists Warren S. McCulloch and Warren S. McCulloch and Walter Pitts Walter Pitts developed finite-state systems to simulate neural networks in biological systems. developed finite-state systems to simulate neural networks in biological systems. They are pioneers of automata theory. They are pioneers of automata theory.

41. Chapter. 0 – p. 41 0.5 Brief History of Theory of Computation In late 1940’s ， Von Neumann proposed the idea of In late 1940’s ， Von Neumann proposed the idea of stored program stored program for computer models. for computer models.

42. Chapter. 0 – p. 42 0.5 Brief History of Theory of Computation In 1951, a real computer following the idea of “stored program” was constructed: In 1951, a real computer following the idea of “stored program” was constructed: UNIVAC I, UNIVAC I, the world's first commercially available computer (manufactured by Eckert- Mauchly Computer Company). the world's first commercially available computer (manufactured by Eckert- Mauchly Computer Company).

43. Chapter. 0 – p. 43 0.5 Brief History of Theory of Computation In late 1950’s, linguist N. Chomsky proposed a mathematical model for grammars of natural languages. In late 1950’s, linguist N. Chomsky proposed a mathematical model for grammars of natural languages. In 1956, he proposed further the concept of context-free grammar, which may be used for defining computer languages. In 1956, he proposed further the concept of context-free grammar, which may be used for defining computer languages.

44. Chapter. 0 – p. 44 0.5 Brief History of Theory of Computation In 1956, Kleene proposed the concept of finite automaton for simulating neural networks proposed by McCulloch and Pitts. In 1956, Kleene proposed the concept of finite automaton for simulating neural networks proposed by McCulloch and Pitts. He also proposed regular expressions to describe strings of symbols, and proved them equivalent to finite automata. He also proposed regular expressions to describe strings of symbols, and proved them equivalent to finite automata.

45. Chapter. 0 – p. 45 0.5 Brief History of Theory of Computation In 1959 and 1960, John W. Backus & Peter Naur proposed sequentially a special expression for computer language grammars, called later Backus-Naur form (BNF). In 1959 and 1960, John W. Backus & Peter Naur proposed sequentially a special expression for computer language grammars, called later Backus-Naur form (BNF). It was used to describe computer language ALGOL-60, leading to development of compiling theory. It was used to describe computer language ALGOL-60, leading to development of compiling theory.

46. Chapter. 0 – p. 46 0.5 Brief History of Theory of Computation In 1969, Stephen A. Cook found that computation problems may be separated into tractable and intractable ones. In 1969, Stephen A. Cook found that computation problems may be separated into tractable and intractable ones. Intractable problems are also called NP- hard problems. Intractable problems are also called NP- hard problems. Such problems cannot be solved by computers except very small instances (with only small-sized inputs). Such problems cannot be solved by computers except very small instances (with only small-sized inputs).

47. Chapter. 0 – p. 47 0.6 Textbooks and Course Grading Textbook: Textbook: Hopcroft, Motwani, Ullman, Introduction to Automata Theory, Languages, and Computation (3rd Edition), Addison-Wesley, Boston, MA, USA, 2007 Hopcroft, Motwani, Ullman, Introduction to Automata Theory, Languages, and Computation (3rd Edition), Addison-Wesley, Boston, MA, USA, 2007 Reference textbook: Reference textbook: 蔡文祥, 正規語言與計算理論 (Formal Languages & The Theory of Computation)( 尚未 出版書籍；中文 ) ( 參考用 ) 蔡文祥, 正規語言與計算理論 (Formal Languages & The Theory of Computation)( 尚未 出版書籍；中文 ) ( 參考用 )

48. Chapter. 0 – p. 48 0.6 Textbooks and Course Grading Course grading: Course grading: Homeworks 33.33% Homeworks 33.33% mid-term exam 33.33% mid-term exam 33.33% final exam 33.33% final exam 33.33% Website for downloading course material and quizzes : Website for downloading course material and quizzes : http://www.cis.nctu.edu.tw/~whtsai/ http://www.cis.nctu.edu.tw/~whtsai/