Compiling, Assembling and Executing Java using Java Jan Bækgaard Pedersen & Alan Wagner University of British Columbia Vancouver, British Columbia Canada Compiler Construction using Java Compiler Construction using Java

Overview of the talk Experience: Using Java in a 4th year June 25, 2001 © Jan B. Pedersen, University of British Columbia Overview of the talk Experience: Using Java in a 4th year compiler course Suggestions: Other interesting projects within the Java language domain Conclusion

4 Pillars of Compiling Theory: Lexing, Parsing (Finite State June 25, 2001 © Jan B. Pedersen, University of British Columbia 4 Pillars of Compiling Theory: Lexing, Parsing (Finite State Machines), Types and Semantics “The Compiler”: Lexing, Parsing, Symbol table, Type checking, Code generation, etc. Intermediate Code Representation Runtime Systems / Code interpretation

Background 4 month long 4th year course in compiler June 25, 2001 © Jan B. Pedersen, University of British Columbia Background 4 month long 4th year course in compiler construction at University of British Columbia 46 students completed a 5 phase project: Lexing Parsing & Parse Tree Construction Symbol Table Construction Type Checking Code Generation

The Project Write a compiler for Mini Java, a subset of June 25, 2001 © Jan B. Pedersen, University of British Columbia The Project Write a compiler for Mini Java, a subset of Java without inheritance: Use Java, JFlex and CUP Multiple syntax tree passes Do extensive type checking Generate Mini Java Assembler Code

June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java - The Project A compiler for a subset of Java which includes: Data types: Integers, Booleans, Classes with public members (Fields, Constructors, Methods) Control Structures: If, While, Break and Continue Loop Constructors: For and While Operators: = > < ! ~ == <= >= != && ++ || -- * / & ^ % Other Keywords: This, New, Void, True, False, Null

Mini Java - The Project Tools and Implementation Details: June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java - The Project Tools and Implementation Details: Language Specification: Sun’s Website Implementation Language: Java Lexical Analyzer: JFlex Parser Generator: Java CUP

Mini Java in 5 “Easy” Steps June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java in 5 “Easy” Steps Phase 1: Lexical Analysis Determine keywords and operators. Write Flex file. Phase 2: Parsing and Parse Tree Construction Write CUP file. Construct tree using object hierarchy. Phase 3: Symbol Table Class, Field and Method symbols defined. Vars/Params defined; All symbols resolved.

Phase 1: Lexical Analysis June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 1: Lexical Analysis Java Language Specification: for_statement : for (for_init_opt ; expression_opt ; for_update_opt) statement Java Language Specification: for_statement : for (for_init_opt ; expression_opt ; for_update_opt) statement Write Flex Specification "for” { return token(sym.FOR); } "(" { return token(sym.LPAREN); } ")" { return token(sym.RPAREN); } ";" { return token(sym.SEMICOLON); } Collect Keywords and Symbols for ( ; )

Mini Java in 5 “Easy” Steps June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java in 5 “Easy” Steps Phase 1: Lexical Analysis Determine keywords and operators. Write Flex file. Phase 2: Parsing and Parse Tree Construction Write CUP file. Construct tree using object hierarchy. Phase 3: Symbol Table Class, Field and Method symbols defined. Vars/Params defined; All symbols resolved.

Phase 2: Parsing and Parse Tree Construction June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 2: Parsing and Parse Tree Construction Java Language Specification: for_statement : for (for_init_opt ; expression_opt ; for_update_opt) statement for_statement ::= FOR:f LPAREN for_init_opt:i SEMICOLON expression_opt:e SEMICOLON for_update_opt:u RPAREN statement:s {: RESULT = new ForStat(f, i, e, u, s); :} ; Build AST for_statement ::= FOR:f LPAREN for_init_opt:i SEMICOLON expression_opt:e SEMICOLON for_update_opt:u RPAREN statement:s ; Write CUP specification

Mini Java in 5 “Easy” Steps June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java in 5 “Easy” Steps Phase 1: Lexical Analysis Determine keywords and operators. Write Flex file. Phase 2: Parsing and Parse Tree Construction Write CUP file. Construct tree using object hierarchy. Phase 3: Symbol Table Class, Field and Method symbols defined. Vars/Params defined; All symbols resolved.

Insert Classes, Methods & Fields June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 3: Symbol Table class A { int myField; void foo(int arg) { int local; myField = arg; local = arg + 1; } Insert Classes, Methods & Fields into Symbol Table Symbol Table Name “Type” A myField foo CLASS FIELD METHOD

Insert parameters and local variables June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 3: Symbol Table class A { int myField; void foo(int arg) { int local; myField = arg; local = arg + 1; } Symbol Table Name “Type” CLASS FIELD METHOD PARAM LOCAL A myField foo arg local A myField foo CLASS FIELD METHOD Insert parameters and local variables

Resolve all uses of names June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 3: Symbol Table class A { int myField; void foo(int arg) { int local; myField = arg; local = arg + 1; } Symbol Table Name “Type” A myField foo arg local CLASS FIELD METHOD PARAM LOCAL  Resolve all uses of names

Mini Java in 5 “Easy” Steps June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java in 5 “Easy” Steps Phase 4: Type Checking Type definition traversal of parse tree. Type checking traversal of parse tree. Phase 5: Code Generation Intermidiate code. JVM instruction set with minor modifications.

Define Types For All Symbols June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 4: Type Checking Define Types For All Symbols class A { int myField; void foo(int arg) { int local; myField = arg; local = arg + 1; } Symbol Table A Int Void Name “Type” Type A myField foo arg local CLASS FIELD METHOD PARAM LOCAL

 Phase 4: Type Checking class A { int myField; void foo(int arg) { June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 4: Type Checking class A { int myField; void foo(int arg) { int local; myField = arg; local = arg + 1; } Symbol Table Name “Type” Type A myField foo arg local CLASS FIELD METHOD PARAM LOCAL A Int Void  Int Int Check Types

Mini Java in 5 “Easy” Steps June 25, 2001 © Jan B. Pedersen, University of British Columbia Mini Java in 5 “Easy” Steps Phase 4: Type Checking Type definition traversal of parse tree. Type checking traversal of parse tree. Phase 5: Code Generation Intermediate code. JVM instruction set with minor modifications. An assembler was provided that created class files that Sun’s JVM can execute.

Phase 5: Code Generation June 25, 2001 © Jan B. Pedersen, University of British Columbia Phase 5: Code Generation METHOD 1 50 4 "f" "(II)I" aload_0 swap putfield FIELDREF "A" "mya" "LA;" iload 1 iload 2 iconst_2 imul iadd istore 3 iload 3 ireturn iconst_0 ENDCODE Write Assembler File CLASS 1 "A" "java/lang/Object" METHOD 1 50 1 "<init>" "()V" aload_0 invokenonvirtual METHODREF "java/lang/Object" "<init>" "()V" return ENDCODE FIELD 1 "aa" "I" FIELD 1 "bb" "I" FIELD 1 "cc" "Z" FIELD 1 "mya" "LA;" No constant pool

Project Result For each phase the previous reference implementation June 25, 2001 © Jan B. Pedersen, University of British Columbia Project Result For each phase the previous reference implementation was provided as a basis for the next phase.

Example - Tree Sorting June 25, 2001 © Jan B. Pedersen, University of British Columbia Example - Tree Sorting class Tree { int value; Tree left, right; Tree(int value) { this.value = value; left = right = null; } Tree insert(int value) { if (this.value < value) { if (right == null) right = new Tree(value); else right = right.insert(value); } else if (this.value > value) { if (left == null) left = new Tree(value); left = left.insert(value); return this; void traverse(Io io) { if (left != null) left.traverse(io); io.out(value); if (right != null) right.traverse(io); } class Main { int main() { int no,temp; Tree t = null; Io io = new Io(); no = io.in(); for (int i=0;i<no;i++) { temp = io.in(); if (t == null) t = new Tree(temp); else t = t.insert(temp); t.traverse(io);

Implementation Language June 25, 2001 © Jan B. Pedersen, University of British Columbia Implementation Language Why Java? Widely accessible Runs on almost any architecture Portable binary: One classfile format Well documented JavaDoc Easy to learn

Compilers using OOP Certain differences from purely June 25, 2001 © Jan B. Pedersen, University of British Columbia Compilers using OOP Certain differences from purely imperative languages Parse tree construction Parse tree nodes are objects Node hierarchy vs. general record/struct/union Code local to tree nodes Parse tree traversal General function in super class Reimplemented by subclasses when needed

Example: Symbol Table Reimplemented June 25, 2001 © Jan B. Pedersen, University of British Columbia Example: Symbol Table public void defineAndResolveSymbols(SymbolTable st,int kind) { for (int c = 0; c < nchildren; c++) { if (children[c] != null) { children[c].defineAndResolveSymbols(st, kind); } Reimplemented By all nodes that define and/or resolve symbols from the symbol table. Genereralized method in the super class. Vars, Methods, Compilation, Names, Blocks

The Source Language Why Java? Tightly coupled with Java byte code June 25, 2001 © Jan B. Pedersen, University of British Columbia The Source Language Why Java? Tightly coupled with Java byte code Byte code based on Java source being compiled “Imperative” subset easily extracted Possibilities for many other projects Assembler / Disassembler Runtime system Debugger

One Step Further Suggestions for other Java related projects June 25, 2001 © Jan B. Pedersen, University of British Columbia One Step Further Suggestions for other Java related projects 1 Semester: Compiling MiniJava 2 Semesters/Follow up course: Assembler / Disassembler. Our assembler: 1,000 lines of C code Javap (disassembler): 1,000 lines of C++ code Executing Java / JVM JADE: 3,300 lines of Java code (4-6 weeks) JVM: (Est.) 5,000+ lines (8-12 weeks)

Conclusion The Java language family is flexible to work with June 25, 2001 © Jan B. Pedersen, University of British Columbia Conclusion The Java language family is flexible to work with Many interesting projects of different sizes and levels of difficulty exist Easy to incorporate with existing (professional) tools from the JDK family

For More Information… Project description and solutions for June 25, 2001 © Jan B. Pedersen, University of British Columbia For More Information… Project description and solutions for Mini Java compiler JADE debugger/executor for Mini Java assembler language (Java Assembly Debugger & Executor) soon to be available: matt@cs.ubc.ca