1. Semantic Analysis (Generating An AST) CS 471 September 26, 2007

2. CS 471 – Fall 2007 1 Semantic Analysis Source code Lexical Analysis Parsing Semantic Analysis Valid programs: decorated AST lexical errors syntax errors semantic errors tokens AST

3. CS 471 – Fall 2007 2 Goals of a Semantic Analyzer Compiler must do more than recognize whether a sentence belongs to the language… Find all possible remaining errors that would make program invalid –undefined variables, types –type errors that can be caught statically Figure out useful information for later phases –types of all expressions –data layout

4. CS 471 – Fall 2007 3 Semantic Actions Can do useful things with the parsed phrases –Each terminal and nonterminal may be associated with type, e.g. exp: INT type is int –For rule: A  B C D Type must match A Value can be built with BCD

5. CS 471 – Fall 2007 4 Semantic Actions Semantic action executed when grammar production is reduced Recursive-descent parser: semantic code interspersed with control flow Yacc: fragments of C code attached to a grammar production

6. CS 471 – Fall 2007 5 Interpreter Could develop an interpreter that executes the program as part of the semantic actions! Example Grammar: E  id E  E + E E  E – E E  E * E E  -E

7. CS 471 – Fall 2007 6 Unions in Yacc %union allows us to declare a union datatype used to package the types/attributes of symbols %union { int pos; int ival; string sval; struct { int intval; enum Types valtype; } constantval; A_exp exp; } Exported as YYSTYPE

8. CS 471 – Fall 2007 7 Types in Yacc Using the values of union structs, tell Yacc the types Terminals %token ID STRING %token INT %token COMMA SEMI LBRACE RBRACE … And Nonterminals (use %type) %type expression program LHS of productiontype

9. CS 471 – Fall 2007 8 Symbols in Yacc The symbol $n (n > 0) refers to the attribute of nth symbol on the RHS The symbol $$ refers the attribute of the LHS The symbol $n (n  0) refers to contextual information Note: actions in middle contribute as a symbol! expr: expr1 PLUS expr2 $$ $1 $3

10. CS 471 – Fall 2007 9 Interpreter in Yacc %{ declarations of yylex and yyerror %} %union {int num; string id} % token INT % token ID % type exp % start exp %left PLUS MINUS %left TIMES %left UMINUS % [please fill in solution] E  id E  E + E E  E – E E  E * E E  -E Recall expr : expr1 PLUS expr2 $$ $1 $3

11. CS 471 – Fall 2007 10 Internally: A Semantic Stack Implemented using a stack parallel to the state stack StackInputAction 1 + 2 * 3 $shift INT: 1 + 2 * 3 $ reduce exp: 1 + 2 * 3 $shift exp: 1 +: 2 * 3 $shift exp: 1 +: INT: 2 * 3 $ reduce exp: 1 +: exp: 2 3 $shift exp: 1 +: exp: 2 *: $ shift exp: 1 +: exp: 2 *: INT: 3$ reduce exp: 1 +: exp: 2 *: exp: 3 $ reduce exp: 1 +: exp: 6 $ reduce exp: 7 $ accept

12. CS 471 – Fall 2007 11 Inlined TypeChecker and CodeGen You can even type check and generate code: expr : expr PLUS expr { if ($1.type == $3.type && ($1.type == IntType || $1.type == RealType)) $$.type = $1.type else error(“+ applied on wrong type!”); GenerateAdd($1, $3, $$); }

13. CS 471 – Fall 2007 12 Problems Difficult to read Difficult to maintain Compiler must analyze program in order parsed Instead … we split up tasks

14. CS 471 – Fall 2007 13 Compiler ‘main program’ void Compile() { TokenStream l = Lexer(input); AST tree = Parser(l); if (TypeCheck(tree)) IR ir = genIntermediateCode(tree); emitCode(ir); }

15. CS 471 – Fall 2007 14 Thread of control Input Stream Lexer Parser characters tokens AST compile parse getToken readStream AST

16. CS 471 – Fall 2007 15 Producing the Parse Tree Separates issues of syntax (parsing) from issues of semantics (type checking, translation to machine code) One leaf for every token One internal node for every reduction during parsing Concrete parse tree represents concrete syntax But … parse tree has problems Punctuation tokens redundant Structure of the tree conveys this info Enter the Abstract Syntax Tree

17. CS 471 – Fall 2007 16 AST Abstract Syntax Tree is a tree representation of the program. Used for –semantic analysis (type checking) –some optimization (e.g. constant folding) –intermediate code generation (sometimes intermediate code = AST with somewhat different set of nodes) Compiler phases = recursive tree traversals

18. CS 471 – Fall 2007 17 Do We Need An AST? Old-style compilers: semantic actions generate code during parsing Problems: hard to maintain limits language features not modular! expr ::= expr PLUS expr {: emitCode(add); :} input parser code stack

19. CS 471 – Fall 2007 18 Interesting Detour Old compilers didn’t create ASTs … not enough memory to store entire program Can also see reasons for C requiring forward declarations - avoids an extra compilation pass

20. CS 471 – Fall 2007 19 Positions In one pass compiler – errors reported using position of the lexer as approximation (global var) Abstract syntax data structures must have pos fields Line number Char number Line number is unambiguous Char number is a matter of style

21. CS 471 – Fall 2007 20 Abstract Syntax for Tiger /* absyn.h */ typedef struct A_var_ * A_var; struct A_var_ { enum {A_simpleVar,A_fieldVar,A_subscriptVar}kind; A_pos pos; union {S_symbol simple; struct {A_var var; S_symbol sym;} field; struct {A_var var; A_exp exp;} subscript; } u; };

22. CS 471 – Fall 2007 21 More Syntax (Constructors…p.98) A_var A_SimpleVar(A_pos pos, S_symbol sym); … A_exp A_WhileExp(A_pos pos, A_exp test, A_exp body); … A_expList A_ExpList(A_exp head, A_expList tail);

23. CS 471 – Fall 2007 22 Tiger Program (a := 5; a+1) translates to: A_SeqExp(2, A_ExpList(A_AssignExp(4, A_SimpleVar(2, S_Symbol(“a”)), A_IntExp(7,5)), A_ExpList((A_OpExp(11,A_plusOp, A_VarExp(A_SimpleVar(10, S_Symbol(“a”))),A_IntExp(12,1))), NULL))) AssignExp choose column of “:=“ for pos OpExp choose column of “+” for pos

24. CS 471 – Fall 2007 23 Some Odd Tiger Features Tiger allows mutually recursive declarations: let var a + 5 function f() : int = g(a) function g(i: int) = f() in f() end Thus: FunctionDec constructor takes a list of functions

25. CS 471 – Fall 2007 24 Correlation to Yacc (and your project) (Demo) Checklist 1.Detailed look at the Tiger AST (absyn.h) 2.Edit tiger.grm 3.The Tiger Language Manual PA3 and PA4 make heavy use of it Follow the structure to generate your yacc file