1. Lecture 10 YACC – Yet Another Compiler Compiler Introduction to YACC and Bison Topics Yacc/Bison IntroductionReadings: February 13, 2006 CSCE 531 Compiler Construction

2. – 2 – CSCE 531 Spring 2006 Overview Last Time Test 1 Post Mortem Lexical Analysis Hash table Token / Lexeme / Token Code Review of LR Parsing Today’s Lecture YACC introductionReferences: Homework: write a YACC specification for the language “core”

3. – 3 – CSCE 531 Spring 2006 YACC Generated LALR(1) Parsers % flex lang.l// lex.yy.c % bison lang.y // lang.c % gcc lex.yy.c lang.c –o parse % parse input lang.y lang.lFLEX lex.yy.c yylex() lang.c yyparse() BISON Input source program Executable Program

4. – 4 – CSCE 531 Spring 2006 YACC Format and Usage Yet Another Compiler Compiler Stephen Johnson 1976 Takes grammar specification and generates the Action and GOTO tables Takes grammar specification and generates the Action and GOTO tables Bison = new and improved YACC YACC Format YACC Format Definitions section % productions / semantic actions section %routines

5. – 5 – CSCE 531 Spring 2006 Simple0.y in web/Examples/SimpleYacc %token DIGIT % line : expr '\n' ; expr : expr '+' term | term | term ; term : term '*' factor | factor | factor ; factor : '(' expr ')' | DIGIT | DIGIT ;% Grammar  Bison Specification expr  expr '+' term | term term  term '*' factor | factor factor  '(' expr ')' | DIGIT We first augment with line  expr ‘\n’

6. – 6 – CSCE 531 Spring 2006 Bison Specification File Example deneb> more simple1.y %token DIGIT % expr : expr '+' expr | expr '*' expr | expr '*' expr | '(' expr ')' | '(' expr ')' | DIGIT | DIGIT ;%

7. – 7 – CSCE 531 Spring 2006 Bison Reporting Conflicts deneb> bison simple1.y simple1.y contains 4 shift/reduce conflicts. bison -v simple1.y simple1.y contains 4 shift/reduce conflicts. So what are they? The “.output file” contains a description of the state machine and actions.

8. – 8 – CSCE 531 Spring 2006.output deneb> more simple1.output State 8 contains 2 shift/reduce conflicts. State 9 contains 2 shift/reduce conflicts. Grammar Number, Line, Rule Number, Line, Rule 1 5 expr -> expr '+' expr 1 5 expr -> expr '+' expr 2 6 expr -> expr '*' expr 2 6 expr -> expr '*' expr 3 7 expr -> '(' expr ')' 3 7 expr -> '(' expr ')' 4 8 expr -> DIGIT 4 8 expr -> DIGIT

9. – 9 – CSCE 531 Spring 2006 Terminals, with rules where they appear $ (-1) '(' (40) 3 ')' (41) 3 '*' (42) 2 '+' (43) 1 error (256) DIGIT (257) 4 Nonterminals, with rules where they appear expr (8) on left: 1 2 3 4, on right: 1 2 3 on left: 1 2 3 4, on right: 1 2 3

10. – 10 – CSCE 531 Spring 2006 state 0 DIGIT shift, and go to state 1 DIGIT shift, and go to state 1 '(' shift, and go to state 2 '(' shift, and go to state 2 expr go to state 3 expr go to state 3 state 1 expr -> DIGIT. (rule 4) expr -> DIGIT. (rule 4) $default reduce using rule 4 (expr) $default reduce using rule 4 (expr)

11. – 11 – CSCE 531 Spring 2006 state 2 expr -> '('. expr ')' (rule 3) expr -> '('. expr ')' (rule 3) DIGIT shift, and go to state 1 DIGIT shift, and go to state 1 '(' shift, and go to state 2 '(' shift, and go to state 2 expr go to state expr go to state

12. – 12 – CSCE 531 Spring 2006 state 3 expr -> expr. '+' expr (rule 1) expr -> expr. '+' expr (rule 1) expr -> expr. '*' expr (rule 2) expr -> expr. '*' expr (rule 2) $ go to state 10 $ go to state 10 '+' shift, and go to state 5 '+' shift, and go to state 5 '*' shift, and go to state 6 '*' shift, and go to state 6 … for states 4 through 7

13. – 13 – CSCE 531 Spring 2006 Conflicts state 8 expr -> expr. '+' expr (rule 1) expr -> expr. '+' expr (rule 1) expr -> expr '+' expr. (rule 1) expr -> expr '+' expr. (rule 1) expr -> expr. '*' expr (rule 2) expr -> expr. '*' expr (rule 2) '+' shift, and go to state 5 '+' shift, and go to state 5 '*' shift, and go to state 6 '*' shift, and go to state 6 '+' [reduce using rule 1 (expr)] '+' [reduce using rule 1 (expr)] '*' [reduce using rule 1 (expr)] '*' [reduce using rule 1 (expr)] $default reduce using rule 1 (expr) $default reduce using rule 1 (expr)

14. – 14 – CSCE 531 Spring 2006 state 10 $ go to state 11 $ go to state 11 state 11 $default accept $default accept

15. – 15 – CSCE 531 Spring 2006 Main and yyerror

16. – 16 – CSCE 531 Spring 2006 bison simple0.y bison simple0.ydeneb> deneb> ls -lrt … - rw-r--r-- 1 matthews faculty 28499 Jun 30 12:04 simple0.tab.c deneb> wc simple0.tab.c 1084 4111 28499 simple0.tab.c 1084 4111 28499 simple0.tab.c

17. – 17 – CSCE 531 Spring 2006 gcc simple0.tab.c -ly Undefined first referenced symbol in file symbol in file yylex /var/tmp//ccW88jE5.o ld: fatal: Symbol referencing errors. No output written to a.out collect2: ld returned 1 exit status

18. – 18 – CSCE 531 Spring 2006 deneb> more simple1.y %token DIGIT % expr : expr '+' expr | expr '*' expr | expr '*' expr | '(' expr ')' | '(' expr ')' | DIGIT | DIGIT ;%

19. – 19 – CSCE 531 Spring 2006 deneb> bison simple1.y simple1.y contains 4 shift/reduce conflicts. bison -v simple1.y simple1.y contains 4 shift/reduce conflicts. deneb> ls -lrt … - rw-r--r-- 1 matthews faculty 2311 Jun 30 12:10 simple1.output

20. – 20 – CSCE 531 Spring 2006.output deneb> more simple1.output State 8 contains 2 shift/reduce conflicts. State 9 contains 2 shift/reduce conflicts. Grammar Number, Line, Rule Number, Line, Rule 1 5 expr -> expr '+' expr 1 5 expr -> expr '+' expr 2 6 expr -> expr '*' expr 2 6 expr -> expr '*' expr 3 7 expr -> '(' expr ')' 3 7 expr -> '(' expr ')' 4 8 expr -> DIGIT 4 8 expr -> DIGIT

21. – 21 – CSCE 531 Spring 2006 Terminals, with rules where they appear $ (-1) '(' (40) 3 ')' (41) 3 '*' (42) 2 '+' (43) 1 error (256) DIGIT (257) 4 Nonterminals, with rules where they appear expr (8) on left: 1 2 3 4, on right: 1 2 3 on left: 1 2 3 4, on right: 1 2 3

22. – 22 – CSCE 531 Spring 2006 state 0 DIGIT shift, and go to state 1 DIGIT shift, and go to state 1 '(' shift, and go to state 2 '(' shift, and go to state 2 expr go to state 3 expr go to state 3 state 1 expr -> DIGIT. (rule 4) expr -> DIGIT. (rule 4) $default reduce using rule 4 (expr) $default reduce using rule 4 (expr)4

23. – 23 – CSCE 531 Spring 2006 state 2 expr -> '('. expr ')' (rule 3) expr -> '('. expr ')' (rule 3) DIGIT shift, and go to state 1 DIGIT shift, and go to state 1 '(' shift, and go to state 2 '(' shift, and go to state 2 expr go to state expr go to state

24. – 24 – CSCE 531 Spring 2006 state 3 expr -> expr. '+' expr (rule 1) expr -> expr. '+' expr (rule 1) expr -> expr. '*' expr (rule 2) expr -> expr. '*' expr (rule 2) $ go to state 10 $ go to state 10 '+' shift, and go to state 5 '+' shift, and go to state 5 '*' shift, and go to state 6 '*' shift, and go to state 6 … for states 4 through 7

25. – 25 – CSCE 531 Spring 2006 Conflicts state 8 expr -> expr. '+' expr (rule 1) expr -> expr. '+' expr (rule 1) expr -> expr '+' expr. (rule 1) expr -> expr '+' expr. (rule 1) expr -> expr. '*' expr (rule 2) expr -> expr. '*' expr (rule 2) '+' shift, and go to state 5 '+' shift, and go to state 5 '*' shift, and go to state 6 '*' shift, and go to state 6 '+' [reduce using rule 1 (expr)] '+' [reduce using rule 1 (expr)] '*' [reduce using rule 1 (expr)] '*' [reduce using rule 1 (expr)] $default reduce using rule 1 (expr) $default reduce using rule 1 (expr)

26. – 26 – CSCE 531 Spring 2006 state 9 expr -> expr. '+' expr (rule 1) expr -> expr. '+' expr (rule 1) expr -> expr. '*' expr (rule 2) expr -> expr. '*' expr (rule 2) expr -> expr '*' expr. (rule 2) expr -> expr '*' expr. (rule 2) '+' shift, and go to state 5 '+' shift, and go to state 5 '*' shift, and go to state 6 '*' shift, and go to state 6 '+' [reduce using rule 2 (expr)] '+' [reduce using rule 2 (expr)] '*' [reduce using rule 2 (expr)] '*' [reduce using rule 2 (expr)] $default reduce using rule 2 (expr) $default reduce using rule 2 (expr)

27. – 27 – CSCE 531 Spring 2006 state 10 $ go to state 11 $ go to state 11 state 11 $default accept $default accept

28. – 28 – CSCE 531 Spring 2006 YACC Generated LALR(1) Parsers % flex lang.l// lex.yy.c % bison lang.y // lang.c % gcc lex.yy.c lang.c –o parse % parse input lang.y lang.lFLEX lex.yy.c yylex() lang.c yyparse() BISON Input source program Executable Program

29. – 29 – CSCE 531 Spring 2006 man yacc NAME yacc - yet another compiler-compiler yacc - yet another compiler-compilerSYNOPSIS yacc [-dltVv] [-b file_prefix] [ -Q [y | n] ] [-P parser] [-p sym_prefix] file yacc [-dltVv] [-b file_prefix] [ -Q [y | n] ] [-P parser] [-p sym_prefix] fileDESCRIPTION The yacc command converts a context-free grammar into a set of tables for a simple automaton that executes an LALR(1) parsing algorithm. The grammar may be ambiguous. Specified precedence rules are used to break ambiguities. The yacc command converts a context-free grammar into a set of tables for a simple automaton that executes an LALR(1) parsing algorithm. The grammar may be ambiguous. Specified precedence rules are used to break ambiguities. The output file, y.tab.c, must be compiled by the C compiler to produce a function yyparse(). This program must be loaded with the lexical analyzer program, yylex(), as well as main() and yyerror(), an error handling routine. These routines must be supplied by the user. The output file, y.tab.c, must be compiled by the C compiler to produce a function yyparse(). This program must be loaded with the lexical analyzer program, yylex(), as well as main() and yyerror(), an error handling routine. These routines must be supplied by the user.

30. – 30 – CSCE 531 Spring 2006 Bison Arguments: Bison –h deneb> bison -h GNU bison generates parsers for LALR(1) grammars. Usage: bison [OPTION]... FILE If a long option shows an argument as mandatory, then it is mandatory for the equivalent short option also. Similarly for optional arguments. Operation modes: -h, --help display this help and exit -h, --help display this help and exit -V, --version output version information and exit -V, --version output version information and exit -y, --yacc emulate POSIX yacc -y, --yacc emulate POSIX yacc

31. – 31 – CSCE 531 Spring 2006 Bison Arguments continued Output: -d, --defines also produce a header file -d, --defines also produce a header file -v, --verbose also produce an explanation of the automaton -v, --verbose also produce an explanation of the automaton -b, --file-prefix=PREFIX specify a PREFIX for output files -b, --file-prefix=PREFIX specify a PREFIX for output files -o, --output=FILE leave output to FILE -o, --output=FILE leave output to FILE -g, --graph also produce a VCG description of the automaton -g, --graph also produce a VCG description of the automaton Report bugs to. deneb>

32. – 32 – CSCE 531 Spring 2006 Bison Arguments continued Parser: -S, --skeleton=FILE specify the skeleton to use -S, --skeleton=FILE specify the skeleton to use -t, --debug instrument the parser for debugging -t, --debug instrument the parser for debugging --locations enable locations computation --locations enable locations computation -p, --name-prefix=PREFIX prepend PREFIX to the external symbols -p, --name-prefix=PREFIX prepend PREFIX to the external symbols -l, --no-lines don't generate `#line' directives -l, --no-lines don't generate `#line' directives -n, --no-parser generate the tables only -n, --no-parser generate the tables only -k, --token-table include a table of token names -k, --token-table include a table of token names

33. – 33 – CSCE 531 Spring 2006 Makefiles A makefile is a file that specifies how to build something usually a piece of software The makefile is a collection of targets with each target having a series of commands to build the target To build the system one merely types “make” Make maintains dependencies and recompiles only those files that need to be recompiled Make documents how to build a system and allows a novice to install a piece of software

34. – 34 – CSCE 531 Spring 2006 Makefiles The makefile is a collection of targets with each target having a series of commands to build the target The target specification has the form targ: files that targ depends on first unix command in a sequence to build target first unix command in a sequence to build target next command in sequence to build target next command in sequence to build target … … The Makefile specifies a forest of targets depending on other targets

35. – 35 – CSCE 531 Spring 2006 Example Makefile prog: prog.o routines.o prog.o: prog.c prog.h gcc –c prog.c routines.o: routines.c prog.h gcc –c routines.c

36. – 36 – CSCE 531 Spring 2006 Make Builtin Rules MacrosName=value $(Name)//invocation of macro; replace macro with its value $(Name)//invocation of macro; replace macro with its value Suffixes:.c,.f,.y,.l … Rules prog.o: prog.c $(CC) $(CFLAGS) –c prog.c Actually the real rules are generalizations of the above. Use “make –p” to see all the rules.

37. – 37 – CSCE 531 Spring 2006 /class/csce531-001/web/Examples/SimpleYacc/Makefile # Makefile for Simple Yacc and Lex examples CC=gccCFLAGS=CFLAGS=-DYYDEBUGLEX=flexYACC=bisonYACCFLAGS=YACCFLAGS=-t simple3: simple3.tab.h simple3.tab.o lex.yy.o $(CC) $(CFLAGS) simple3.tab.o lex.yy.o -ly -o simple3 $(CC) $(CFLAGS) simple3.tab.o lex.yy.o -ly -o simple3 simple3.tab.h: simple3.y bison -d $(YACCFLAGS) simple3.y bison -d $(YACCFLAGS) simple3.y simple3.tab.c: simple3.y bison $(YACCFLAGS) simple3.y bison $(YACCFLAGS) simple3.y lex.yy.c: simple3.l flex simple3.l flex simple3.l

38. – 38 – CSCE 531 Spring 2006 Lex Specification simple3.l %{ #include "simple0.tab.h" %} digit [0-9]+ % \n return('\n'); \+ return(PLUS); \* return(TIMES); \( return(LPAREN); \) return(RPAREN); {digit} return(DIGIT); %yywrap(){}

39. – 39 – CSCE 531 Spring 2006 Example Yacc Spec file simple3.y %token DIGIT TIMES PLUS LPAREN RPAREN % line : expr '\n' {printf("recognized an expr\n"); exit(0); exit(0); } ; expr : expr PLUS term | term | term ; term : term TIMES factor | factor | factor ; factor : LPAREN expr RPAREN | DIGIT | DIGIT ;%

40. – 40 – CSCE 531 Spring 2006 Simple4.l %{ #include "simple4.tab.h" %} digit [0-9]+ % \n return('\n'); \+ return(PLUS); \* return(TIMES); \( return(LPAREN); \) return(RPAREN); {digit} { yylval= atoi(yytext); return(INTEGER);} % yywrap(){ }

41. – 41 – CSCE 531 Spring 2006 Simple4.y %{ #include #include int yylineno = 1; %} %token INTEGER PLUS TIMES LPAREN RPAREN % line : expr '\n' {printf("%d\n", $1);} | line expr '\n' {printf("%d\n", $2);} | line expr '\n' {printf("%d\n", $2);} ; expr : expr PLUS term {$$ = $1 + $3;} | term | term ; term : term TIMES factor {$$ = $1 * $3;} | factor | factor ; factor : LPAREN expr RPAREN {$$ = $2;} | INTEGER {$$ = $1;} /* the default action */ | INTEGER {$$ = $1;} /* the default action */ ;%