1. CS 33601 Describing Syntax CS 3360 Spring 2012 Sec 3.1-3.4 Adapted from Addison Wesley’s lecture notes (Copyright © 2004 Pearson Addison Wesley)

2. 2 CS 3360 Outline Introduction Formal description of syntax Backus-Naur Form (BNF) Attribute grammars (probably next time )

3. 3 CS 3360 Introduction Who must use language definitions?  Implementers  Programmers (the users of the language) Syntax - the form or structure of the expressions, statements, and program units Semantics - the meaning of the expressions, statements, and program units

4. 4 CS 3360 Introduction (cont.) Example  Syntax of Java while statement while ( )  Semantics?

5. 5 CS 3360 Describing Syntax – Vocabulary A sentence is a string of characters over some alphabet A language is a set of sentences A lexeme is the lowest level syntactic unit of a language (e.g., *, sum, while) A token is a category of lexemes (e.g., identifier)

6. 6 CS 3360 Example index = 2 * count + 17; LexemesTokens indexidentifier =equal_sign 2int_literal *mult_op countidentifier +plus_op 17int_literal ;semicolon

7. 7 CS 3360 Describing Syntax Formal approaches to describing syntax:  Recognizers (once you have code) Can tell whether a given string is in a language or not Used in compilers, and called a parser  Generators (in order to build code) Generate the sentences of a language Used to describe the syntax of a language

8. 8 CS 3360 Formal Methods of Describing Syntax Context-Free Grammars (CFG – see automata course)  Developed by Noam Chomsky in the mid- 1950’s  Language generators, meant to describe the syntax of natural languages  Define a class of languages called context- free languages

9. 9 CS 3360 Formal Methods of Describing Syntax Backus-Naur Form  Invented by John Backus to describe Algol 58  Extended by Peter Naur to describe Algol 60  BNF is equivalent to context-free grammars  A metalanguage is a language used to describe another language.  In BNF, abstractions are used to represent classes of syntactic structures--they act like syntactic variables (also called nonterminal symbols)

10. 10 CS 3360 Backus-Naur Form  while ( ) This is a rule (also called a production rule); it describes the structure of a while statement

11. 11 CS 3360 Backus-Naur Form A rule has a left-hand side (LHS) and a right- hand side (RHS), and consists of terminal and non-terminal symbols A grammar is a finite non-empty set of rules An abstraction (or non-terminal symbol) can have more than one RHS  | { }

12. 12 CS 3360 Backus-Naur Form Syntactic lists are described using recursion  ident | ident, Example sentences: ident ident, ident ident, ident, ident

13. 13 CS 3360 Example A grammar for small language:   | ;  =  a | b | c | d  + | -  | 5 Sample program a = b + 5

14. 14 CS 3360 Exercise Define a grammar to generate all sentences of the form: subject verb object. where subject is “ i ” or “ we ”, and verb is “ love ” or “ like ”, and object is “ exercises ” or “ programming ”.

15. 15 CS 3360 Exercise Define the syntax of Java Boolean expressions consisting of:  Constants: false and true  Operators: !, &&, and ||

16. 16 CS 3360 Derivation A derivation is a repeated application of rules, starting with the start symbol and ending with a sentence (all terminal symbols) Example:  ident | ident, => ident, => ident, ident, => ident, ident, ident

17. 17 CS 3360 More Example a = b + 5 => => = => a = => a = + => a = b + => a = b + 5   | ;  =  a | b | c | d  + | -  | 5

18. 18 CS 3360 Derivation Every string of symbols in the derivation is a sentential form A sentence is a sentential form that has only terminal symbols A leftmost derivation is one in which the leftmost nonterminal in each sentential form is the one that is expanded A derivation may be neither leftmost nor rightmost

19. 19 CS 3360 Exercise Derive a = b + 5 by using a rightmost derivation.   | ;  =  a | b | c | d  + | -  | 5

20. 20 CS 3360 Parse Tree A hierarchical representation of a derivation 5 a = b +

21. 21 CS 3360 Ambiguity of Grammars A grammar is ambiguous if and only if it generates a sentential form that has two or more distinct parse trees.

22. 22 CS 3360 An Ambiguous Expression Grammar  | 5  / | - 5 5 5-/ 5 5 5-/

23. 23 CS 3360 An Unambiguous Expression Grammar If we use the parse tree to indicate precedence levels of the operators, we cannot have ambiguity  - |  / 5 | 5 5 5 5/ -

24. 24 CS 3360 Exercise Prove or disprove the ambiguity of the following grammar -> -> if then | if then else

25. 25 CS 3360 Operator Precedence Derivation: => - => 5 - => 5 - / 5 => 5 - 5 / 5  - |  / 5 | 5

26. 26 CS 3360 Operator Associativity Can we describe operator associativity correctly? A = A + B + C (A + B) + C or A + (B + C)? Does it matter?

27. 27 CS 3360 Operator Associativity Operator associativity can also be indicated by a grammar -> + | 5 (ambiguous) -> + 5 | 5 (unambiguous) 5 5 5 + +

28. 28 CS 3360 Left vs. Right Recursion A rule is left recursive if its LHS also appears at the beginning (left end) of its RHS. A rule is right recursive if its LHS also appears at the right end of its RHS. -> ** | -> c Example: c ** c ** c interpreted as c ** (c ** c)

29. 29 CS 3360 Exercise Define a BNF grammar for expressions consisting of +, *, and ** (exponential). The operator ** has precedence over *, and * has precedence over +. Both + and * are left associative while ** is right associative. Using the above grammar, draw a parse tree for the sentence: 7 + 6 + 5 * 4 * 3 ** 2 ** 1 Exercise to do in groups at the end of lecture

30. 30 CS 3360 Extended BNF (EBNF) Extended BNF (just abbreviations):  Optional parts are placed in brackets ([ ]) -> ident ( [ ] )  Put alternative parts of RHSs in parentheses and separate them with vertical bars -> (+ | -) const  Put repetitions (0 or more) in braces ({ }) -> letter {letter | digit}

31. 31 CS 3360 Example BNF:  + | - |  * | / | EBNF:  {(+ | -) }  {(* | /) }

32. 32 CS 3360 Exercise / Homework Write BNF rules for the following EBNF rules: 1. -> “(” [ ] “)” 2. -> (+ | -) const 3. -> letter {letter | digit} Due on Tuesday at the start of the session!

33. 33 CS 3360 Outline  Introduction  Describing syntax formally  Backus-Naur Form (BNF) Attribute grammars

34. 34 CS 3360 Attribute Grammars CFGs cannot describe all of the syntax of programming languages Additions to CFGs to carry some semantic info along through parse trees Primary value of attribute grammars:  Static semantics specification  Compiler design (static semantics checking)

35. 35 CS 3360 Basic Idea Add attributes, attribute computation functions, and predicates to CFGs Attributes  Associated with grammar symbols  Can have values assigned to them Attribute computation functions  Associated with grammar rules  Specify how to compute attribute values  Are often called semantic functions Predicate functions  Associated with grammar rules  State some of the syntax and static semantic rules of the language

36. 36 CS 3360 Example BNF -> meth end -> -> … AG 1. Syntax rule: -> meth [1] end [2] Predicate: [1].string == [2].string 2. Syntax rule: -> Semantic rule:.string.string

37. 37 CS 3360 Attribute Grammars Defined An attribute grammar is a CFG with the following additions:  A set of attributes A(X) for each grammar symbol X A(X) consists of two disjoint sets S(X) and I(X) S(X): synthesized attributes I(X): inherited attributes  Each rule has a set of functions that define certain attributes of the non-terminals in the rule  Each rule has a (possibly empty) set of predicates to check for attribute consistency

38. 38 CS 3360 Attribute Functions Let X 0  X 1... X n be a rule Functions of the form S(X 0 ) = f(A(X 1 ),..., A(X n )) define synthesized attributes Functions of the form I(X j ) = f(A(X 0 ),..., A(X n )), for 1 <= j <= n, define inherited attributes.  Often of the form: I(X j ) = f(A(X 0 ),..., A(X j-1 )) Initially, there are intrinsic attributes on the leaves.  Intrinsic attributes are synthesized attributes whose value are determined outside the parse tree.

39. 39 CS 3360 Example - Type Checking Rules BNF -> = -> | + -> A | B | C Rule  A variable is either int or float.  If the two operands of + has the same type, the type of expression is that of the operands; otherwise, it is float.  The type of the left side of assignment must match the type of the right side. Attributes  actual_type: synthesized for and  expected_type: inherited for  string: intrinsic for

40. 40 CS 3360 Example – Attribute Grammar 1. Syntax rule: -> = Semantic rule:.expected_type.actual_type 2. Syntax rule: -> [1] + [2] Semantic rule:.actual_type <- ( [1].actual_type == int && [2].actual_type == int) ? int : float Predicate:.actual_type ==.expected_type 3. Syntax rule: -> Semantic rule:.actual_type.actual_type Predicate:.actual_type ==.expected_type 4. Syntax rule: -> A | B | C Semantic rule:.actual_type.string)

41. 41 CS 3360 [2] A = A + B Example – Parse Tree A = A + B [1]

42. 42 CS 3360 [2] A = A + B Example – Flow of Attributes A = A + B [1] expected_type actual_type.expected_type.actual_type.actual_type [1].actual_type == int && [2].actual_type == int) ? int : float

43. 43 CS 3360 [2] A = A + B Example – Calculating Attributes A = A + B [1] expected_type actual_type float float int float int.expected_type.actual_type.actual_type [1].actual_type == int && [2].actual_type == int) ? int : float

44. 44 CS 3360 [2] A = A + B Example – Calculating Attributes A = A + B [1] expected_type actual_type int int float int float int float.expected_type.actual_type.actual_type [1].actual_type == int && [2].actual_type == int) ? int : float

45. 45 CS 3360 Attribute Grammars How are attribute values computed?  If all attributes were inherited, the tree could be decorated in top-down order.  If all attributes were synthesized, the tree could be decorated in bottom-up order.  In many cases, both kinds of attributes are used, and it is some combination of top-down and bottom-up that must be used.

46. 46 CS 3360 Group Exercise: homework due Tuesday February, 7 at the start of class BNF -> ? : -> | + -> id Rule  id's type can be bool, int, or float.  Operands of + must be numeric and of the same type.  The type of + is the type of its operands.  The first operand of ?: must be of bool and the second and third must be of the same type.  The type of ?: is the type of its second and third operands. Given the above BNF and rule: 1. Define an attribute grammar 2. Draw a decorated parse tree for “ id ? id : id + id ” assuming that the first id is of type bool and the rest are of type int.