Abstract Syntax Cheng-Chia Chen

2 Concrete v.s. Abstract Syntax Parsing results can generally be represented in two forms: –concrete syntax tree (CST or Parse Tree) –abstract syntax tree (AST or syntax tree) Concrete syntax trees (parse tree) is a record of the rules (and tokens) used to match some input text. Abstract syntax tree records the structure of the input and is insensitive to the grammar that produced it. AST is superior to CST since –CST reflects the structure of a sentence in a grammar AST reflects the structure of a sentence in the language. But a language can have a lot of grammars. –AST is generally more compact. –AST is usually more convenient for later analysis and processing.

Cheng-Chia Chen3 Example Given the grammar : Expr  Expr + Term Term  Term * Factor Factor  Number | (E) and the input ‘2 + 5 ‘ : We should have the following CST and AST:

Cheng-Chia Chen4 PlusExpr NumExpr :5 NumExpr :2 e1 e2 BinExpr op: plus NumExpr :5 NumExpr :2 e1 e2 Expr Factor Plus :+ Term Number:2 Term Factor Number :5 Expr The parse tree for ‘2+5’ Possible ASTs for ‘2+5’

Cheng-Chia Chen5 Abstract Syntax v.s. Concrete Syntax of Expressions Abstract Syntax E  E + E E  E – E E  E * E E  E / E E  id E  num Notes: 1.reflect the meaning of expressions; resulting AST more convenient for later processing 2.ambiguous; impractical to parsing Concrete Syntax (LALR(1)) E  E + F E  E – F F  F * T F  T / T T  id T  num T  (E) Notes: 1.quick parse possible 2.resulting CST is highly redundant Conclusion: generate AST (instead of CST) during concrete syntax parsing

Cheng-Chia Chen6 Building Syntax Tree for Expressions (using JavaCC) First devise the Concrete Syntax (LL(K)) in EBNF Form. Start  Exp Exp  Term ( “+” Term | “-” Tem)* Term  Factor (“*” Factor | “/” Factor)* Factor  | | “(“ Exp “)”

Cheng-Chia Chen7 Design the AST classes : Class Hierarchy for expressions Exp PlusExp MinusExp TimesExp DivideExp Identifier IntergerLiteral or simply a BinaryExp

Cheng-Chia Chen8 AST classes for expressions pubic abstract class Exp { pubic abstract int eval() } public class PlusExp extends Exp { private Exp e1,e2 ; public PlusExp(Exp a1, Exp a2) { e1=a1;e2=a2;} public int eval() { return e1.eval() + e2.eval(); } } …/* similar to PlusExp for class MinusExp, timesExp and DivideExp. Omitted here */ public class Identifier extends Exp { pubic String f0; public Identifier(String a0) { f0 = a0 ; } public int eval() { return symbolTable.lookup(f0); }}

Cheng-Chia Chen9 AST classes public class IntergerLiteral extends Exp { pubic String f0; public Identifier(String a0) { f0 = a0 ; } public int eval() { return Integer.parseInt(f0); } }

Cheng-Chia Chen10 Add actions to build syntax tree : Using the available AST classes Exp Start() : {Exp e;} { e = Exp() { return e;} } Exp Exp() : {Exp e1,e2;} { e1 = Term() ( “+” e2 = Term() { e1 = new PlusExp(e1,e2) ;} | “-” e2 = Term() { e1 = new MinusExp(e1,e2) ;} )* { return e1; } }

Cheng-Chia Chen11 Building AST Exp Term() : {Exp e1,e2;} { e1 = Factor() ( “*” e2 = Factor() { e1 = new TimesExp(e1,e2) ;} | “/” e2 = Factor() { e1 = new DivideExp(e1,e2) ;} )* { return e1; } } Exp Factor() : {Token t; Exp e;} { ( t = { e = new Identifier(t.image); } | t= { e = new IntegerLiteral(t.image); } | “(“ e = Exp() “)” ) { return e; } }

Cheng-Chia Chen12 ExpParser.jj options { STATIC = false; … } PARSER_BEGIN(ExpParser) public class ExpParser { … public static void main( String[] args) { ExpParser p = new ExpParser(System.in); Exp ast = p.start(); System.out.prinln( “result = “ + ast.eval() ); } Token : { } SKIP : { … } …

Cheng-Chia Chen13 Tools for automated generation of AST classes definitions from abstract syntax definition –sableCC version 3.X –JTB Disadvantage –The syntax tree and relevant classes generated may be less abstract than one could desire.

Cheng-Chia Chen14 ASTs for MiniJava at package syntaxtree;

Cheng-Chia Chen15 Abstract Syntax for MiniJava (I) Package syntaxtree; Program(MainClass m, ClassDecList c1) MainClass(Identifier i1, Identifier i2, Statement s) abstract class ClassDecl ClassDeclSimple(Identifier i, VarDeclList vl, methodDeclList m1) ClassDeclExtends(Identifier i, Identifier j, VarDecList vl, MethodDeclList ml) VarDecl(Type t, Identifier i) MethodDecl(Type t, Identifier I, FormalList fl, VariableDeclList vl, StatementList sl, Exp e) Formal(Type t, Identifier i)

Cheng-Chia Chen16 Abstract Syntax for MiniJava (II) abstract class type IntArrayType() BooleanType() IntegerType() IndentifierType(String s) abstract class Statement Block(StatementList sl) If(Exp e, Statement s1, Statement s2) While(Exp e, Statement s) Print(Exp e) Assign(Identifier i, Exp e) ArrayAssign(Identifier i, Exp e1, Exp e2)

Cheng-Chia Chen17 Abstract Syntax for MiniJava (III) abstract class Exp And(Exp e1, Exp e2) LessThan(Exp e1, Exp e2) Plus(Exp e1, Exp e2) Minus(Exp e1, Exp e2) Times(Exp e1, Exp e2) Not(Exp e) ArrayLookup(Exp e1, Exp e2) ArrayLength(Exp e) Call(Exp e, Identifier i, ExpList el) IntegerLiteral(int i) True() False() IdentifierExp(String s) This() NewArray(Exp e) NewObject(Identifier i) Identifier(Sting s) --list classes ClassDecList() ExpList() FormalList() MethodDeclList() StatementLIst() VarDeclList()

Cheng-Chia Chen18 Syntax Tree Nodes - Details package syntaxtree; import visitor.Visitor; import visitor.TypeVisitor; public class Program { public MainClass m; public ClassDeclList cl; public Program(MainClass am, ClassDeclList acl) { m=am; cl=acl; } public void accept(Visitor v) { v.visit(this); } public Type accept(TypeVisitor v) { return v.visit(this); }

Cheng-Chia Chen19 ClassDecl.java package syntaxtree; import visitor.Visitor; import visitor.TypeVisitor; public abstract class ClassDecl { public abstract void accept(Visitor v); public abstract Type accept(TypeVisitor v); }

Cheng-Chia Chen20 ClassDeclExtends.java package syntaxtree; import visitor.Visitor; import visitor.TypeVisitor; public class ClassDeclExtends extends ClassDecl { public Identifier i; public Identifier j; public VarDeclList vl; public MethodDeclList ml; public ClassDeclExtends(Identifier ai, Identifier aj, VarDeclList avl, MethodDeclList aml) { i=ai; j=aj; vl=avl; ml=aml; } public void accept(Visitor v) { v.visit(this); } public Type accept(TypeVisitor v) { return v.visit(this); }

Cheng-Chia Chen21 StatementList.java package syntaxtree; import java.util.Vector; public class StatementList { private Vector list; public StatementList() { list = new Vector(); } public void addElement(Statement n) { list.addElement(n); } public Statement elementAt(int i) { return (Statement)list.elementAt(i); } public int size() { return list.size(); }

Cheng-Chia Chen22 Package visitor/Visitor.java package visitor; import syntaxtree.*; public interface Visitor { public void visit(Program n); public void visit(MainClass n); public void visit(ClassDeclSimple n); public void visit(ClassDeclExtends n); public void visit(VarDecl n); public void visit(MethodDecl n); public void visit(Formal n); public void visit(IntArrayType n); public void visit(BooleanType n); public void visit(IntegerType n); public void visit(IdentifierType n); public void visit(Block n); public void visit(If n); public void visit(While n); public void visit(Print n); public void visit(Assign n); public void visit(ArrayAssign n); public void visit(And n); public void visit(LessThan n); public void visit(Plus n); public void visit(Minus n); public void visit(Times n); public void visit(ArrayLookup n); public void visit(ArrayLength n); public void visit(Call n); public void visit(IntegerLiteral n); public void visit(True n); public void visit(False n); public void visit(IdentifierExp n); public void visit(This n); public void visit(NewArray n); public void visit(NewObject n); public void visit(Not n); public void visit(Identifier n); }

Cheng-Chia Chen23 AST for statement : X = y.m(1,4+5) Statement s Assign (Identifier,Exp) Identifier(“x”)Call(Exp,Identifier,ExpList) IdentifierExp(“y”)Identifier(“m”) IntegerLiteral(1) ExpList:[, ] Plus(Exp,Exp) IntegerLiteral(4)(IntegerLiteral(5)

Cheng-Chia Chen24 MiniJava : Grammar(I) Program -> MainClass ClassDecl * Program(MainClass, ClassDeclList) Program Goal() : { MainClass m; ClassDeclList cl = new ClassDeclList(); ClassDecl c; } { m = MainClass() ( c = ClassDecl() {cl.addElement(c);})* {return new Program(m,cl) }

Cheng-Chia Chen25 MiniJava : Grammar(II) MainClass -> class id { public static void main ( String [] id ) { Statement } } MainClass( Identifier, Identifier, Statement) ClassDecl -> class id { VarDecl * MethodDecl * } -> class id extends id { VarDecl* MethodDecl * } ClassDeclSimple(…), ClassDecExtends(…) VarDecl -> Type id ; VarDecl(Type, Identifier) MethodDecl -> public Type id ( FormalList ) { VarDecl * Statement* return Exp ; } MethodDecl(Type,Identifier,FormalList,VarDeclList StatementList, Exp)

Cheng-Chia Chen26 MiniJava : Grammar(III) FormalList -> Formal ( “,” Formal)* Formal -> Type id Formal(Type, Identifier ) Type -> int [] IntArrayType() -> boolean -> int -> id IdentifierType()

Cheng-Chia Chen27 MiniJava : Grammar(IV) Statement -> { Statement * } -> if ( Exp ) Statement else Statement -> while ( Exp ) Statement -> System.out.println ( Exp ) ; -> id = Exp ; -> id [ Exp ] = Exp ; ExpList -> Exp (“,” Exp ) *

Cheng-Chia Chen28 MiniJava : Grammar(V) Exp -> Exp op Exp -> Exp [ Exp ] -> Exp. length -> Exp. Id ( ExpList ) -> INTEGER_LITERAL -> true -> false -> id -> this -> new int [ Exp ] -> new id ( ) -> ! Exp -> ( Exp )