1. Tutorial 1 Scanner & Parser
CPSC Compiler
Tutorial 1
Scanner & Parser

2. Scanner and Parser Scanner Parser Input token Grammar
Parser Tree (Syntax Tree)

3. Token – example 1 Jeremy sees the cute monkey.
Jeremy – noun
sees – verb
the – determiner
cute – adjective
monkey – noun

4. Token – example 2 int x = ( 3 + 21 ) * 6;
int – type (reserve/key word)
x – variable
= – assign (reserve/key word)
( – left-prentices (reserve/key word)
3 – digit
+ – plus (reserve/key word)
21 – digit
) – right-prentices (reserve/key word)
* – multiple (reserve/key word)
6 – digit
; – semi-colon/end (reserve/key word)

5. Jeremy sees the cute monkeys sleeps
Parsing
Top-Down Parsing
Bottom-up Parsing
Ambiguous
Jeremy sees the cute monkeys sleeps
int x = ( ) * 6;

6. Context Free Grammar (CFG)

7. Simple Debug Miss the end int x = 3 // “;” missing – add it in
Extra ending
int x = 3&; // “&” is extra – remove it
Output the possible part
x = 3 + // ??? – what happen here?
(Type, second argument, semi-colon, etc.)
Note: Some of them is impossible to debug.
(For example: misspell, missing argument)

8. Practice Parse the following: The big dog crush the small kid.
double x = y + 3 / 2; // Syntax
Write the grammar for the following:
sentence
Noun phrase
Verb phrase
noun I
verb
sit
Prep. phrase
prep on
you

9. Practice
Parse and write the grammar
( ) * 4 – ( )

10. Lex Lex will unify the string which fit the patterns
Good for search through a program or a document
You can specify in C or C++ for what action should take when an input string been found.

11. Lex - Structure Declarations/Definitions %% Rules/Production
- Lex expression
- white space
- C statement (optional)
Additional Code/Subroutines

12. Lex – statement Example%%
[+++]+.* ;
Remove all of comment in Aldor code.
- after lex expression, the C statement is empty. (So, takes no action)

13. Lex – Basic operators * - zero or more occurrences . - “ANY” character
.* - matches any sequence
| - separator
+ - one or more occurrences. (a+ :== aa*)
? - zero or one of something. (b? :== (b+null)
[ ] - choice, so [12345]  (1|2|3|4|5)
(Note: [*+] represent a choice between star and
plus. They lost their specialty.
[a-zA-Z]  a to z and A to Z, all the letters.
\ - \* matches *, and \. Match period or decimal point.

14. Lex – Additional Code %% main() { yylex(); [any C statements] }
- When you compile the Lex file, __.lex or __.l, it will generates lex.yy.c file, which define yylex().
- Type “man lex” in UNIX system for more information.