1. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -1- Compiler Construction Principles & Implementation Techniques Dr. Ying JIN Associate Professor Oct. 2007

2. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -2- Outline 2.1 Overview 2.1.1 General Function of a Scanner 2.1.2 Some Issues about Scanning 2.2 Finite Automata 2.2.1 Definition and Implementation of DFA 2.2.2 Non-Determinate Finite Automata 2.2.3 Transforming NFA into DFA 2.2.4 Minimizing DFA 2.3 Regular Expressions 2.3.1 Definition of Regular Expressions 2.3.2 Regular Definition 2.3.4 From Regular Expression to DFA 2.4 Design and Implementation of a Scanner 2.4.1 Developing a Scanner from DFA 2.4.2 A Scanner Generator – Lex √ √

3. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -3- 2.3 Regular Expressions Definition of Regular Expressions Regular Definition From Regular Expression to DFA

4. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -4- Definition of Regular Expressions (RE) - Some Concepts - Formal Definition of RE - Example - Properties of RE - Extensions to RE - Limitations of RE - Using RE to define Lexical Structure

5. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -5- Some Concepts alphabet( 字母表 ) ： a non-empty finite set of symbols ， which is denoted as  ， one of its elements is called symbol. string( 符号串 ) ： finite sequence of symbols, we use or  to represent empty string( 空串 ); 空串集 { } is different from empty set  。 length of a string( 符号串长度 ) ： the number of symbols ina string, we use |  | to represent the length of the string  ; concatenate operator for strings( 符号串连接操作 ) ： if  and  are strings ， we use  as the concatenation of two strings, especially we have  =  =  ;

6. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -6- Some Operators on Set of Strings product of set of strings ( 符号串集的乘积 ) ： if A and B are two sets of strings, AB is called the product of two sets of strings, AB={  |  A ，   B} especially  A=A  =A ， where  represents empty set 。 power of set of strings( 符号串集合的方幂 ) ： if A is a set of strings, A i is called ith power of A, where i is a non-negative integer( 非负整数 ) 。 A 0 ={ } A 1 = A, A 2 = A A A K = AA......A (k) positive closure( 符号串集合的正闭包 ) ： A + =A 1  A 2  A 3...... star closure( 符号串集合的星闭包 ) ： A * =A 0  A 1  A 2  A 3......

7. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -7- Formal Definition For a given alphabet , a regular expression for  defines a set of strings of , If we use R  to represent a regular expression for , and L(R  ) to represent the set of strings that R  defines.

8. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -8- Formal Definition ■  is a regular expression ， L(  )={ } ■ is a regular expression ， L( )={ } ■ for any c , c is a regular expression, L(c)={c} ■ if A and B are regular expressions, following operators can be used – ( A ) ， L( (A) )= L(A) –choice among alternatives A | B ， L( A | B )=L(A)  L(B) –concatenation A B ， L( A B )= L(A)L(B) –repetation A* ， L( A*)= L(A)*

9. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -9- Example  ={ a,b }. RE 1.ab* 2. a(a|b)* Set of strings 1.{a,ab, abb, abbb, …… } 2. {a, aa, ab, aaa, aab, aba, abb, …… }

10. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -10- Comparing with DFA Equivalent in describing the set of strings; Can be conversed into each other; DFA is convenient for implementation; RE is convenient for defining and understanding; Both of them can be used to define the lexical structure of programming languages;

11. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -11- Properties A | B = B | A | 的可交换性 A | (B | C) =(A | B ) C| 的可结合性 A (B C) =(A B )C 连接的可结合性 A (B | C) =A B | A C 连接的可分配性 (A | B ) C =A C | B C 连接的可分配性 A** =A* 幂的等价性 A= A=A 是连接的恒等元素

12. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -12- Extensions Some extensions can be made to facilitate definition – A + –any symbol: “.” –range: [0-9] [a-z] [A-Z] –not in the range: ~(a|b|c) –optional: r?=( |r)

13. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -13- Limitations RE can not define such structure like –Pairing 配对， （） –Nesting 嵌套， RE can not describe those structures that include finite number of repetitions for example ： w c w, w is a string containing a and b; (a|b)* c (a|b)* can not be used, because it cannot guarantee that the strings on both sides of c are the same all the time;

14. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -14- Regular Definition

15. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -15- Definition It is inconvenient to define set of long strings with RE, so another formal notation is introduced, which is called “Formal Definition” ； The main idea is that naming some sub-expressions in RE; Example ： ( 1|2|…|9)(0|1|2|…9) * NZ_digit= 1|2|…|9 digit = NZ_digit | 0 NZ_digit digit *

16. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -16- Defining Lexical Structure of ToyL letter = a|…|z|A|…|Z digit = 0|…|9 NZ-digit = 1|…|9 Reserved words: Reserved = var| if | then | else| while| read| write| intIdentifiers: letter digit* Constant: integer: int = NZ-digit digit* | 0 Other symbols: syms = +|-| *|/ | > | < | = | : | ; | ( | ) | { | } Lexical structure: lex = Reserved | identifier |int | syms

17. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -17- From RE to NFA

18. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -18- Rules ■  is a regular expression ， L(  )={ } ■ is a regular expression ， L( )={ } ■ for any c , c is a regular expression, L(c)={c} S0S0 S S0S0 S c

19. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -19- Rules ■ ( A ), L( (A) )= L(A), no change; ■ A B ， L( A B )= L(A)L(B) NFA(A) NFA(B) 

20. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -20- Rules ■ ( A ), L( (A) )= L(A), no change; ■ A | B ， L( A | B )=L(A)  L(B) NFA(A) NFA(B)    

21. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -21- Rules ■ A* ， L( A*)= L(A)* NFA(A)   

22. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -22- Attention The rules introduced above are effective for those NFAs that have one start state and one terminal state; Any NFA can be extended to meet this requirement; …… NFA ……    

23. College of Computer Science & Technology Compiler Construction Principles & Implementation Techniques -23- Example (a | b)* a b b (a | b) a b  a b b b a  