1. 4.1 4. Phase 2 : Syntax Analysis Part II The unit directory. What you must do. Example run. syner.cxx The lookahead convention. Error detection and recovery. Symbol table lookup. Parsing declarations. Parsing statements. Parsing expressions.

2. 4.2 The Unit Directory The unit directory for phase 2 is : /usr/users/staff/aosc/cm049icp/phase2 Among other things, it contains the following : – synprog.cxx : the test bed program for phase 2. – syner.template : A template file for your phase 2 program. – syner.h : header file for phase2.  AST : Abstract syntax tree data structure.  SymTab : Symbol table data structure.  syntax : Array of syntax error messages.  statics : Array of static semantic error messages.  type : Array of type error messages. – makefile : The makefile for phase 2. – syner : An executable for my phase 2 program.

3. 4.3 The Unit Directory II – tests/test*.c-- : Testing programs for the demo. – printers.cxx : Printing subprograms. void printAST(AST *ast, // AST int &line, // Line number int indent) // Indentation void printST(SymTab *st) // Symbol Table – utilities.cxx : General utilities void error(int number, // Error no. LexToken lexToken) // Token bool lookup(LexToken lexToken, // Token SymTab *st, // Sym. Tab. SymTab *&match) // Entry

4. 4.4 synprog.cxx The test bed program is as follows : #include “.../phase2/syner.h” void main() { SymTab *st = NULL ; AST *ast = NULL ; int line = 1 ; int indent = 0 ; int label = 0 ; synAnal(st, ast, label) ; printAST(ast, line, indent) ; printST(st) ; } printAST : Pretty prints the AST. printST : Pretty prints the symbol table. They ensure that your program’s output format is the same as mine. You must write this subprogram

5. 4.5 What You Must Do Your implementation of synAnal must be in a file called syner.cxx in your directory. Take a copy of makefile and syner.template. Print out a copy of syner.h. Print out a copy of utilities.cxx. Print out a copy of printers.cxx. Useful commands : testphase2, demophase2. – Shell scripts for running the phase 2 demo. – They rely on your synAnal using the printing subprograms from printers.cxx. – You must #include printers.cxx and utilities.cxx into your syner.cxx file.

6. 4.6 Example Run Assuming we have the following valid C-- program in a file called prog.c-- : int i = 10 ; { while (i > 0) { cout << i ; } ; } Make and run syner : jaguar> make syner jaguar> syner < prog.c-- 1 -- { 2 -- while (i > 0) /* Start label : 0 */ 3 -- { 4 -- cout << i ; 5 -- } /* End label : 1 */ ; 6 -- } Name Type Constant Initial Value i int False 10 jaguar> Output format may change. Make sure printers.cxx is #include d in your syner.cxx.

7. 4.7 Example Run II Assuming we have the following invalid C-- program in a file called prog.c-- : bool b = true ; { cout >> b ; } Run syner : jaguar> syner < prog.c-- Type error 218. Attempt to input to a non-integer variable. IDENTIFIER : b jaguar>

8. 4.8 Top Level Structure For syner.cxx Contents of syner.template : #include #include “.../lib/cstring.h” #include “.../phase2/syner.h” #include “.../phase1/lexer.h” #include “.../phase2/printers.cxx #include “.../phase2/utilities.cxx void synDec(SymTab *&st, LexToken &lexToken) { cout << “synDec\n” ; } void synDeclarations(SymTab *&st, LexToken &lexToken) { cout << “synDeclarations\n” ; }

9. 4.9 Top Level Structure For syner.cxx II // Forward declaration. void synExpression(SymTab *st, Expression *&expr, LexToken &lexToken, DataType &type) ; void synFactor(SymTab *st, Factor *&fact, LexToken &lexToken) { cout << “synFactor\n” ; } void synTerm(SymTab *st, Term *&term, LexToken &lexToken, DataType &type) { cout << “synTerm\n” ; } void synBasicExp(SymTab *st, BasicExp *&bexp, LexToken &lexToken) DataType &type) { cout << “synBasicExp\n” ; }

10. 4.10 Top Level Structure For syner.cxx III void synExpression(SymTab *st, Expression *&expr, LexToken &lexToken, DataType &type) { cout << “synExpression\n” } //Forward declaration. void synStatements(SymTab *st, AST *&ast, int &label, LexToken &lexToken) ; void synIfSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << “synIfSt\n” ; } void synWhileSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << “synWhileSt\n” ; }

11. 4.11 Top Level Structure For syner.cxx IV void synCinSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << “synCinSt\n” ; } void synCoutSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << “synCoutSt\n” ; } void synAssignSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << synAssignSt\n” ; }

12. 4.12 Top Level Structure For syner.cxx V void synStatement(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << “synStatement\n” ; } void synStatements(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { cout << “synStatements\n” ; }

13. 4.13 Top Level Structure For syner.cxx VI void synAnal(SymTab *&st, AST *&ast, int &label) { LexToken lexToken ; // Current token skipWhiteComments() ; lexAnal(lexToken) ; st = NULL ; ast = NULL ; synDeclarations(st, lexToken) ; synStatements(st, ast, label, lexToken) ; } Note the strange way pointers are passed as reference parameters : label holds the number of the next M68K label to be used. All labels are of the form L0, L1,.. etc.

14. 4.14 Lookahead Conventions C-- has an LL(1) grammar. – Sometimes we must read one token beyond the end of a syntactic construct.  When parsing an if statement to determine whether or not there’s an else part. Conventions : – 1. All the syntax analysis subprograms assume that they will be passed their first token as a parameter. – 2. All the syntax analysis subprograms will read one token beyond the end of the syntactic construct they are attempting to parse and will pass that token back to their caller. – Exception : synStatements does not follow convention 2. – Otherwise it would read past the end of the input file. – Luckily, synStatements does not need to lookahead.

15. 4.15 Error Detection & Recovery Detecting errors is one thing that we’ll award marks for. – RTFC for syner.cxx to see what errors you have to detect. Syntax errors. 27 = A ; Type errors. bool b ; { cin >> b ; } Static semantic errors. string s ; As usual, upon detection of an error simply call error with an error number and the offending lexical token. error(101, lexToken) ; error prints an error message and calls exit.

16. 4.16 Symbol Table Lookup One of the most common static semantic errors is the use of an undeclared variable. The following subprogram is in utilities.cxx : bool lookup(LexToken lexToken, SymTab *st, SymTab *&match) lookup looks inspects the symbol table to see if the identifier held in lexToken has already been declared. – If not already declared returns false and match is set to NULL. – If already declared returns true and sets match to a copy of the Entry for the identifier.

17. 4.17 Symbol Table Lookup II lookup is also useful for checking that identifiers have the correct type. Note that lookup is a value returning subprogram which may also assign a value to a reference parameter. – i.e. it is a side effecting ‘function’. This is disgusting programming practice. – It’s also standard programming practice in this instance. – Don’t do this kind of thing yourself unless you really want a zero mark.

18. 4.18 Code For synDeclarations void synDeclarations(SymTab *&st, LexToken &lexToken) { while (lexToken.tag != LBRACE) { synDec(st, lexToken) ; if (lexToken.tag != TERMINATOR) error(3, lexToken) ; else lexAnal(lexToken) ; } error is in utilities.cxx. –Prints out the error message corresponding to the number.  0..99 : Syntax errors.  100..199 : Static semantic errors.  200..299 : Type errors. – Prints out the offending token (using writeToken ). –Calls exit to terminate the program.

19. 4.19 Top Level Code For synDec void synDec(SymTab *&st, LexToken &lexToken) { SymTab *newEntry ; // For this declaration SymTab *dummy ; // For lookup LexToken idToken ; // Var/const name Create new Symtab entry and initialise it (set type to VOIDDATA ). if (lexToken.tag == CONST) { newEntry->constFlag = true ; Lex another token into lexToken. } lexToken is the type. Set type field. If not valid type raise an error. Lex the identifier token. Copy it to idToken for better error reporting. if (lexToken.tag == IDENT) newEntry->ident = lexToken.ident ; else error(1, idToken) ; if (lookup(lexToken, st, dummy)) error(101, idToken) ;

20. 4.20 Top Level Code For synDec II Lex next token. if (lexToken.tag == ASSIGN) { newEntry->initialise = new Factor ; Lex next token. if (lexToken.tag == BOOLLIT) { if (newEntry->type != BOOLDATA) error(217, idToken) ; newEntry->initialise->literal = true ; newEntry->initialise->type = BOOLDATA ; newEntry->initialise->litBool = lexToken.boolLit ; } else if (lexToken.tag == STRINGLIT) { As above but for a string. } else if (lexToken.tag == INTLIT) { As above but for an int. } else Not a literal so call error. Lex next token. }

21. 4.21 Top Level Code For synDec III if ((newEntry->constFlag) && (newEntry->Initialise == NULL)) error(103,idToken) ; if ((newEntry->type == STRINGDATA) && (!newEntry->constFlag)) error(104,idToken) ; Add newEntry to the front of st. } // synDec Note that this will build the symbol table in reverse order of the order of the declarations. – This is no problem. – When parsing some languages (e.g. C, C++) it’s actually an advantage.

22. 4.22 Top Level Code For synStatements void synStatements(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { AST *newast = NULL ; // Next statement AST *temp1 = NULL ; // For reversing AST *temp2 = NULL ; // the statement AST *temp3 = NULL ; // list if (lexToken.tag != LBRACE) error(4, lexToken) ; Lex the next token. while (lexToken.tag != RBRACE) { newast = new AST ; synStatement(st, newast, label, lexToken) ; if (lexToken.tag != TERMINATOR) error(8, lexToken) ;

23. 4.23 Top Level Code For synStatements II if (lexToken.tag != TERMINATOR) error(8, lexToken) ; newast->next = ast ; ast = newast ; Lex the next token. } temp1 = ast ; while (temp1 != NULL) { temp3 = temp1->next ; temp1->next = temp2 ; temp2 = temp1 ; temp1 = temp3 ; } ast = temp2 ; } // synStatements Statement list built in reverse order. Must reverse it back again after it has been parsed.

24. 4.24 synStatement synStatement will call the following subprograms : – synIfSt, synWhileSt, synCinSt, synCoutSt, synAssignSt. synStatement decides which to call via an if statement which examines lexToken.tag. – IF token : call synIfSt. WHILE token : call synWhileSt. CIN token : call synCinSt. COUT token : call synCoutSt, IDENT token : call synAssignSt. None of the above : call error. synIfSt and synWhileSt will call synStatements to syntax analyse their compound statements. – Recursive Descent parsing : a set of mutually recursive subprograms, one per production rule in the EBNF syntax.

25. 4.25 Parsing Input Statements By far the easiest of the statement parsing subprograms are synCinSt and synCoutSt : void synCinSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { ast->tag = CINST ; ast->cinst = new CinSt ; lexAnal(lexToken) ; if (lexToken.tag != INOP) error(9, lexToken) ; lexAnal(lexToken) ; if (lexToken.tag != IDENT) error(11, lexToken) ;

26. 4.26 Parsing Input Statements II if (!lookup(lexToken, st, ast->cinst->invar)) error(102, lexToken) ; if (ast->cinst->invar->type != INTDATA) error(216, lexToken) ; if (ast->cinst->invar->constFlag) error(105, lexToken) ; lexAnal(lexToken) ; } Most of the code is to check for syntax, static semantic and type errors.

27. 4.27 Parsing if-else Statements This is slightly more complex : – Need to use lookahead. – Must handle the label field. – Must parse the conditional expression. – Must parse the enclosed statements. void synIfSt(SymTab *st, AST *&ast, int &label, LexToken &lexToken) { DataType type = VOIDDATA ; Expression *expr = NULL ; AST *thenpart = NULL ; AST *elsepart = NULL ;

28. 4.28 Parsing if-else Statements II ast->tag = IFST ; ast->ifst = new IfSt ; ast->ifst->elselabel = label++ ; ast->ifst->endlabel = label++ ; lexAnal(lexToken) ; if (lexToken.tag != LPAREN) error(6, lexToken) ; lexAnal(lexToken) ; synExpression(st, expr, lexToken, type) ; if (type != BOOLDATA) error(202, lexToken) ; if (lexToken.tag != RPAREN) error(7, lexToken) ;

29. 4.29 Parsing if-else Statements III lexAnal(lexToken) ; synStatements(st, thenpart, label, lexToken) ; lexAnal(lexToken) ; if (lexToken.tag == ELSE) { lexAnal(lexToken) ; synStatements(st, elsepart, label, lexToken) ; lexAnal(lexToken) ; } ast->ifst->condition = expr ; ast->ifst->thenstats = thenpart ; ast->ifst->elsestats = elsepart ; } // synIfSt synStatements doesn’t lex ahead so must lex another token after each call to it.

30. 4.30 Expressions synIfSt must call synExpression to parse the conditional expression. – So must synWhileSt and synAssignSt. The next lecture covers how to write synExpression. For now, assume that all expressions are simply literal constants. if (true) while (false) x = 42; {... } ; Code to parse literals is on slide 20. Handling expressions can be a bit fiddly. – Not difficult, just fiddly.

31. 4.31 Summary Copy syner.template, makefile and syner (renamed dhsyner ) into your directory. Print out syner.h, utilities.cxx and printers.cxx. Rename syner.template to syner.cxx. Complete the stubs in syner.cxx in the following order : – synDeclarations, synDec, synStatements, synStatement, synCinSt, synCoutSt, synAssignSt, synWhileSt, synIfSt. For now, assume all expressions are simply literal constants. – synExpression uses code from slide 20.