1. 1 Note As usual, these notes are based on the Sebesta text. The tree diagrams in these slides are from the lecture slides provided in the instructor resources for the text, and were made by David Garrett.

2. 2 Context-free (CF) grammar A CF grammar is formally presented as a 4-tuple G=(T,NT,P,S), where: –T is a set of terminal symbols (the alphabet) –NT is a set of non-terminal symbols –P is a set of productions (or rules), where P  NT  (T  NT)* –S  NT

3. 3 Example 1 L 1 = { 0, 00, 1, 11 } G 1 = ( {0,1}, {S}, { S  0, S  00, S  1, S  11 }, S )

4. 4 Example 2 L 2 = { the dog chased the dog, the dog chased a dog, a dog chased the dog, a dog chased a dog, the dog chased the cat, … } G 2 = ( { a, the, dog, cat, chased }, { S, NP, VP, Det, N, V }, { S  NP VP, NP  Det N, Det  a | the, N  dog | cat, VP  V | VP NP, V  chased }, S ) Notes: S = Sentence, NP = Noun Phrase, N = Noun VP = Verb Phrase, V = Verb, Det = Determiner

5. 5 Examples of lexemes and tokens LexemesTokens fooidentifier i sumidentifier -3integer_literal 10integer_literal 1 ;statement_separator =assignment_operator

6. 6 BNF Fundamentals Sample rules [p. 128] → = → if then → if then else non-terminals/tokens surrounded by lexemes are not surrounded by keywords in language are in bold → separates LHS from RHS | expresses alternative expansions for LHS → if then | if then else = is in this example a lexeme

7. 7 BNF Rules A rule has a left-hand side (LHS) and a right- hand side (RHS), and consists of terminal and nonterminal symbols A grammar is often given simply as a set of rules (terminal and non-terminal sets are implicit in rules, as is start symbol)

8. 8 Describing Lists There are many situations in which a programming language allows a list of items (e.g. parameter list, argument list). Such a list can typically be as short as empty or consisting of one item. Such lists are typically not bounded. How is their structure described?

9. 9 Describing lists The are described using recursive rules. Here is a pair of rules describing a list of identifiers, whose minimum length is one:  ident | ident, Notice that ‘, ’ is part of the object language

10. 10 Example 3 L 3 = { 0, 1, 00, 11, 000, 111, 0000, 1111, … } G 3 = ( {0,1}, {S,ZeroList,OneList}, { S  ZeroList | OneList, ZeroList  0 | 0 ZeroList, OneList  1 | 1 OneList }, S )

11. 11 Derivation of sentences from a grammar A derivation is a repeated application of rules, starting with the start symbol and ending with a sentence (all terminal symbols).

12. 12 Example: derivation from G 2 Example: derivation of the dog chased a cat S  NP VP  Det N VP  the N VP  the dog VP  the dog V NP  the dog chased NP  the dog chased Det N  the dog chased a N  the dog chased a cat

13. 13 Example: derivations from G 3 Example: derivation of 0 0 0 0 S  ZeroList  0 ZeroList  0 0 ZeroList  0 0 0 ZeroList  0 0 0 0 Example: derivation of 1 1 1 S  OneList  1 OneList  1 1 OneList  1 1 1

14. 14 Observations about derivations Every string of symbols in the derivation is a sentential form. A sentence is a sentential form that has only terminal symbols. A leftmost derivation is one in which the leftmost nonterminal in each sentential form is the one that is expanded. A derivation can be leftmost, rightmost, or neither.

15. 15 An example grammar  | ;  =  a | b | c | d  + | -  | const

16. 16 A leftmost derivation => => = => a = => a = + => a = b + => a = b + const

17. 17 Parse tree A hierarchical representation of a derivation const a = b +

18. 18 Parse trees and compilation A compiler builds a parse tree for a program (or for different parts of a program). If the compiler cannot build a well-formed parse tree from a given input, it reports a compilation error. The parse tree serves as the basis for semantic interpretation/translation of the program.

19. 19 Extended BNF Optional parts are placed in brackets [ ] -> ident [( )] Alternative parts of RHSs are placed inside parentheses and separated via vertical bars → (+|-) const Repetitions (0 or more) are placed inside braces { } → letter {letter|digit}

20. 20 Comparison of BNF and EBNF sample grammar fragment expressed in BNF  + | - |  * | / | same grammar fragment expressed in EBNF  {(+ | -) }  {(* | /) }

21. 21 Ambiguity in grammars A grammar is ambiguous if and only if it generates a sentential form that has two or more distinct parse trees

22. 22 An ambiguous grammar for arithmetic expressions  | const  / | - const --//

23. 23 Disambiguating the grammar If we use the parse tree to indicate precedence levels of the operators, we can remove the ambiguity. The following rules give / a higher precedence than -  - |  / const| const const / -

24. 24 Links to BNF-style grammars for actual programming languages Below are some links to grammars for real programming languages. Look at how the grammars are expressed. –http://www.schemers.org/Documents/Standards/R5RS/http://www.schemers.org/Documents/Standards/R5RS/ –http://www.sics.se/isl/sicstuswww/site/documentation.htmlhttp://www.sics.se/isl/sicstuswww/site/documentation.html In the ones listed below, find the parts of the grammar that deal with operator precedence. –http://java.sun.com/docs/books/jls/index.htmlhttp://java.sun.com/docs/books/jls/index.html –http://www.lykkenborg.no/java/grammar/JLS3.htmlhttp://www.lykkenborg.no/java/grammar/JLS3.html –http://www.enseignement.polytechnique.fr/profs/informatique/Jean- Jacques.Levy/poly/mainB/node23.htmlhttp://www.enseignement.polytechnique.fr/profs/informatique/Jean- Jacques.Levy/poly/mainB/node23.html –http://www.lrz-muenchen.de/~bernhard/Pascal-EBNF.htmlhttp://www.lrz-muenchen.de/~bernhard/Pascal-EBNF.html

25. 25 Associativity of operators When multiple operators appear in an expression, we need to know how to interpret the expression. Some operators (e.g. +) are associative, meaning that the meaning of an expression with multiple instances of the operator is the same no matter how it is interpreted: (a+b)+c = a+(b+c) Some operators (e.g. -) are not associative: (a-b)-c  a-(b-c)e.g. try a=10, b=8, c=6 (10-8)-6 = -4 but 10-(8-6)=8

26. 26 Associativity of Operators Operator associativity can also be indicated by a grammar -> + | const (ambiguous) -> + const | const (unambiguous) const + +

27. 27 Links to BNF-style grammars for actual programming languages Below are some links to grammars for real programming languages. Look at how the grammars are expressed. –http://www.schemers.org/Documents/Standards/R5RS/http://www.schemers.org/Documents/Standards/R5RS/ –http://www.sics.se/isl/sicstuswww/site/documentation.htmlhttp://www.sics.se/isl/sicstuswww/site/documentation.html In the ones listed below, find the parts of the grammar that deal with operator associativity. –http://java.sun.com/docs/books/jls/index.htmlhttp://java.sun.com/docs/books/jls/index.html –http://www.lykkenborg.no/java/grammar/JLS3.htmlhttp://www.lykkenborg.no/java/grammar/JLS3.html –http://www.enseignement.polytechnique.fr/profs/informatique/Jean- Jacques.Levy/poly/mainB/node23.htmlhttp://www.enseignement.polytechnique.fr/profs/informatique/Jean- Jacques.Levy/poly/mainB/node23.html –http://www.lrz-muenchen.de/~bernhard/Pascal-EBNF.htmlhttp://www.lrz-muenchen.de/~bernhard/Pascal-EBNF.html