StringTemplate Terence Parr University of San Francisco

Outline What is StringTemplate What is StringTemplate Operations Operations Semantics Semantics Example Java/XML Generation Example Java/XML Generation ANTLR 3.0 template ANTLR 3.0 template

What is StringTemplate? Template engine designed with Tom Burns (CEO, jGuru.com) while building commercial sites over many years in response to JSP Template engine designed with Tom Burns (CEO, jGuru.com) while building commercial sites over many years in response to JSP small: 170k uncompressed binary (w/o test rig) small: 170k uncompressed binary (w/o test rig) Evolved from simple “document with holes” into a functional language capable of generating large class of languages Evolved from simple “document with holes” into a functional language capable of generating large class of languages Well suited to generative programming as well as dynamic page generation; being used in new ANTLR parser generator for code generation Well suited to generative programming as well as dynamic page generation; being used in new ANTLR parser generator for code generation Distinguishing characteristic: strictly enforces separation of model and view Distinguishing characteristic: strictly enforces separation of model and view

Argument for StringTemplate Car aerodynamics mostly trumps style today Car aerodynamics mostly trumps style today Similarly, the nature of generating text should dominate design decisions: use an output grammar Similarly, the nature of generating text should dominate design decisions: use an output grammar Don’t have output grammars, we have programs with print statements; template engines arose to encourage separation of logic/display. Don’t have output grammars, we have programs with print statements; template engines arose to encourage separation of logic/display. Enforcing strict separation also leads to similar grammar-like mechanism Enforcing strict separation also leads to similar grammar-like mechanism Conclusion: if you’re generating text, you should be using something akin to StringTemplate Conclusion: if you’re generating text, you should be using something akin to StringTemplate

Canonical Operations Attribute reference: Attribute reference: Template references (possibly recursive): Template references (possibly recursive): Apply template to multi-valued attribute: or ;}> Apply template to multi-valued attribute: or ;}> Conditional include: extends Name: Guest Conditional include: extends Name: Guest

Semantics Side-effect free expressions Side-effect free expressions No “state” No “state” No defined order of execution No defined order of execution Lazy evaluation (crucial!) Lazy evaluation (crucial!) Dynamically scoped; values inherited Dynamically scoped; values inherited Template inheritance: group to subgroup Template inheritance: group to subgroup Recursive template instantiation Recursive template instantiation

Template Groups Set of mutually-referential templates with formal arguments Set of mutually-referential templates with formal arguments group javaTemplates; method(type,name,args,body) ::= << public ( ) { } >> assign(lhs,expr) ::= “ = ;” if(expr,stat) ::= “if ( ) ” call(name,args) ::= “ ( );” …

Example: Dump Java Class Expected output: Expected output: class Dump { public int i; public java.lang.String name; public int[] data; public void main(class java.lang.String[] arg1); public void foo(int arg1, float[] arg2); public class java.lang.String bar(); }

Dump Java Class Templates group Java; class(name,fields,methods) ::= << class { > } >> field() ::= "public ;" method() ::= << public ( arg }; separator=", ">); >> type(t) ::= << [] >>

Dump Java Class Code public class Dump { public int i; public String name; public int[] data; public static void main(String[] args) throws IOException { StringTemplateGroup group = new StringTemplateGroup(new FileReader("Java.stg"), AngleBracketTemplateLexer.class); Class c = Dump.class; Field[] fields = c.getFields(); Method[] methods = c.getDeclaredMethods(); StringTemplate classST = group.getInstanceOf("class"); classST.setAttribute("name", c.getName()); classST.setAttribute("fields", fields); classST.setAttribute("methods", methods); System.out.println(classST); } public void foo(int x, float[] y) {;} public String bar() {return "";} }

Dump XML Instead group XML; class(name,fields,methods) ::= << $name$ $fields:field()$ $methods:method()$ >> field() ::= << $type(t=it.type)$ $it.name$ >> … Dump int i …

Sample ANTLR 3.0 Template parser(name, tokens, rules, DFAs) ::= << class extends Parser { <tokens: {public static final int = ;} > public (TokenStream input) { super(input); } } >>

ANTLR 3.0 Error Templates Internationalization: use a template group for each locale; load appropriate templates Internationalization: use a template group for each locale; load appropriate templates RULE_REDEFINITION(file,line,col,arg) ::= " rule redefinition” UNDEFINED_RULE_REF(file,line,col,arg) ::= " reference to undefined rule: ” loc() ::= " : : : " inherits file, line, col attributes from enclosing template

Language Translation MVC (parser + “unparser”) MVC (parser + “unparser”) model -- input stream and/or ASTs model -- input stream and/or ASTs view -- templates view -- templates controller -- parser controller -- parser Controller extracts data from model, provides to view without worrying about details of output structure Controller extracts data from model, provides to view without worrying about details of output structure Maps abstract input concept to output concept like assignment to assignment. Maps abstract input concept to output concept like assignment to assignment. Abstract concepts represented by one or more rules in parser grammar and one or more templates in template file Abstract concepts represented by one or more rules in parser grammar and one or more templates in template file

Translation grammar+ST variable returns [StringTemplate code=null] {StringTemplate t=null,d=null;} : t=type d=declarator SEMI { if ( currentFunctionName==null ) { code = template("globalVariable"); } else { code = template("variable"); } code.setAttribute("type", t); code.setAttribute("name", d); } ; declarator returns [StringTemplate code=null] : id:ID {code=text(id.getText());} ;

Translation Templates Java variable(type,name)::=" ;” globalVariable ::= variable Java variable(type,name)::=" ;” globalVariable ::= variable Python variable(type,name) ::= " ” globalVariable ::= variable Python variable(type,name) ::= " ” globalVariable ::= variable Bytecodes variable(type,name) ::= ".var is " globalVariable(type,name) ::= ".field " Bytecodes variable(type,name) ::= ".var is " globalVariable(type,name) ::= ".field "

Summary The nature of text generation and the enforcement of model-view separation dominate tool design- decisions: The nature of text generation and the enforcement of model-view separation dominate tool design- decisions: tools should resemble output grammars tools should resemble output grammars StringTemplate is a simple template engine that evolved while building dynamic sites. It is proving exceptionally well suited to code generation tasks including ANTLR 3.0 StringTemplate is a simple template engine that evolved while building dynamic sites. It is proving exceptionally well suited to code generation tasks including ANTLR 3.0 Open-source Java, BSD license (also a C# port; python coming) Open-source Java, BSD license (also a C# port; python coming) Links Links