Semantic Analysis (Generating An AST) CS 471 September 26, 2007

CS 471 – Fall Semantic Analysis Source code Lexical Analysis Parsing Semantic Analysis Valid programs: decorated AST lexical errors syntax errors semantic errors tokens AST

CS 471 – Fall Goals of a Semantic Analyzer Compiler must do more than recognize whether a sentence belongs to the language… Find all possible remaining errors that would make program invalid –undefined variables, types –type errors that can be caught statically Figure out useful information for later phases –types of all expressions –data layout

CS 471 – Fall Semantic Actions Can do useful things with the parsed phrases –Each terminal and nonterminal may be associated with type, e.g. exp: INT type is int –For rule: A  B C D Type must match A Value can be built with BCD

CS 471 – Fall Semantic Actions Semantic action executed when grammar production is reduced Recursive-descent parser: semantic code interspersed with control flow Yacc: fragments of C code attached to a grammar production

CS 471 – Fall Interpreter Could develop an interpreter that executes the program as part of the semantic actions! Example Grammar: E  id E  E + E E  E – E E  E * E E  -E

CS 471 – Fall Unions in Yacc %union allows us to declare a union datatype used to package the types/attributes of symbols %union { int pos; int ival; string sval; struct { int intval; enum Types valtype; } constantval; A_exp exp; } Exported as YYSTYPE

CS 471 – Fall Types in Yacc Using the values of union structs, tell Yacc the types Terminals %token ID STRING %token INT %token COMMA SEMI LBRACE RBRACE … And Nonterminals (use %type) %type expression program LHS of productiontype

CS 471 – Fall Symbols in Yacc The symbol $n (n > 0) refers to the attribute of nth symbol on the RHS The symbol $$ refers the attribute of the LHS The symbol $n (n  0) refers to contextual information Note: actions in middle contribute as a symbol! expr: expr1 PLUS expr2 $$ $1 $3

CS 471 – Fall Interpreter in Yacc %{ declarations of yylex and yyerror %} %union {int num; string id} % token INT % token ID % type exp % start exp %left PLUS MINUS %left TIMES %left UMINUS % [please fill in solution] E  id E  E + E E  E – E E  E * E E  -E Recall expr : expr1 PLUS expr2 $$ $1 $3

CS 471 – Fall Internally: A Semantic Stack Implemented using a stack parallel to the state stack StackInputAction * 3 $shift INT: * 3 $ reduce exp: * 3 $shift exp: 1 +: 2 * 3 $shift exp: 1 +: INT: 2 * 3 $ reduce exp: 1 +: exp: 2 3 $shift exp: 1 +: exp: 2 *: $ shift exp: 1 +: exp: 2 *: INT: 3$ reduce exp: 1 +: exp: 2 *: exp: 3 $ reduce exp: 1 +: exp: 6 $ reduce exp: 7 $ accept

CS 471 – Fall Inlined TypeChecker and CodeGen You can even type check and generate code: expr : expr PLUS expr { if ($1.type == $3.type && ($1.type == IntType || $1.type == RealType)) $$.type = $1.type else error(“+ applied on wrong type!”); GenerateAdd($1, $3, $$); }

CS 471 – Fall Problems Difficult to read Difficult to maintain Compiler must analyze program in order parsed Instead … we split up tasks

CS 471 – Fall Compiler ‘main program’ void Compile() { TokenStream l = Lexer(input); AST tree = Parser(l); if (TypeCheck(tree)) IR ir = genIntermediateCode(tree); emitCode(ir); }

CS 471 – Fall Thread of control Input Stream Lexer Parser characters tokens AST compile parse getToken readStream AST

CS 471 – Fall Producing the Parse Tree Separates issues of syntax (parsing) from issues of semantics (type checking, translation to machine code) One leaf for every token One internal node for every reduction during parsing Concrete parse tree represents concrete syntax But … parse tree has problems Punctuation tokens redundant Structure of the tree conveys this info Enter the Abstract Syntax Tree

CS 471 – Fall AST Abstract Syntax Tree is a tree representation of the program. Used for –semantic analysis (type checking) –some optimization (e.g. constant folding) –intermediate code generation (sometimes intermediate code = AST with somewhat different set of nodes) Compiler phases = recursive tree traversals

CS 471 – Fall Do We Need An AST? Old-style compilers: semantic actions generate code during parsing Problems: hard to maintain limits language features not modular! expr ::= expr PLUS expr {: emitCode(add); :} input parser code stack

CS 471 – Fall Interesting Detour Old compilers didn’t create ASTs … not enough memory to store entire program Can also see reasons for C requiring forward declarations - avoids an extra compilation pass

CS 471 – Fall Positions In one pass compiler – errors reported using position of the lexer as approximation (global var) Abstract syntax data structures must have pos fields Line number Char number Line number is unambiguous Char number is a matter of style

CS 471 – Fall Abstract Syntax for Tiger /* absyn.h */ typedef struct A_var_ * A_var; struct A_var_ { enum {A_simpleVar,A_fieldVar,A_subscriptVar}kind; A_pos pos; union {S_symbol simple; struct {A_var var; S_symbol sym;} field; struct {A_var var; A_exp exp;} subscript; } u; };

CS 471 – Fall More Syntax (Constructors…p.98) A_var A_SimpleVar(A_pos pos, S_symbol sym); … A_exp A_WhileExp(A_pos pos, A_exp test, A_exp body); … A_expList A_ExpList(A_exp head, A_expList tail);

CS 471 – Fall Tiger Program (a := 5; a+1) translates to: A_SeqExp(2, A_ExpList(A_AssignExp(4, A_SimpleVar(2, S_Symbol(“a”)), A_IntExp(7,5)), A_ExpList((A_OpExp(11,A_plusOp, A_VarExp(A_SimpleVar(10, S_Symbol(“a”))),A_IntExp(12,1))), NULL))) AssignExp choose column of “:=“ for pos OpExp choose column of “+” for pos

CS 471 – Fall Some Odd Tiger Features Tiger allows mutually recursive declarations: let var a + 5 function f() : int = g(a) function g(i: int) = f() in f() end Thus: FunctionDec constructor takes a list of functions

CS 471 – Fall Correlation to Yacc (and your project) (Demo) Checklist 1.Detailed look at the Tiger AST (absyn.h) 2.Edit tiger.grm 3.The Tiger Language Manual PA3 and PA4 make heavy use of it Follow the structure to generate your yacc file