1. Syntax (2)

2. Clite Grammar (1) lexical level
Identifier  Letter { Letter | Digit }
Letter  a | b | ... | z | A | B | ... | Z
Digit  0 | 1 | ... | 9
Literal  Integer | Boolean | Float | Char
Integer  Digit { Digit }
Boolean  true | False
Float  Integer . Integer
Char  ‘ ASCII Char ‘

3. Issues Not Addressed by this Grammar
Comments
Whitespace
Distinguishing one token <= from two tokens < =
Distinguishing identifiers from keywords like if

4. Lexical Syntax (Lexer )
Input: a stream of characters from the
ASCII set, keyed by a programmer.
Output: a stream of tokens

5. Classes of Tokens Identifiers: Stack, x, i, push
Literals : , 'x', 3.25, true
Keywords : bool false true
char int float
if else while
main
Operators : = || && == != < <= >
>= + - * / !
Punctuation : ; , { } ( )

6. Whitespace Whitespace is any: No token may contain embedded whitespace
Tab
end-of-line character (or characters)
character sequence inside a comment
No token may contain embedded whitespace
(unless it is a character or string literal)
Example:
>= one token
> = two tokens

7. Whitespace Examples in Pascal
while a < b do
legal - spacing between tokens
whilea < b do
Whilea – valid identifier token
whilea < -invalid statement prefix

8. Keywords and Identifiers
Both an identifier and a keyword are lexically the same.
if is a keyword and it is also an identifier
In most languages keywords are reserved and cannot be used as identifiers
main in C and C++ is not a reserved word but it is special identifier

9. Integer and float values Range and storage
Not limited in Clite grammar
No idea about storage
both limits and storage space are semantic issues

10. Concrete Syntax (Parser)
Based on BNF/EBNF grammar
Input: tokens
Output: Concrete Syntax (parse) tree or
Abstract Syntax tree

11. Concrete Syntax of Clite
Metabraces {} imply left associativity
Metabrackets [] makes EquOp and RelOp non-associative
In C++, the expression:
if (a < x < b)
is not equivalent to
if (a < x && x < b)
But it is error-free!
Clite Differs from C/C++
Fewer operators
Equality and relational are non-associative

12. Clite Grammar (2) Expressions
Operator Associativity
Unary - ! none
* / left
+ - left
< <= > >= none
== != none
&& left
|| left
Expression  Conjunction { || Conjunction }
Conjunction  Equality { && Equality }
Equality  Relation [ EquOp Relation ]
EquOp  == | !=
Relation  Addition [ RelOp Addition ]
RelOp  < | <= | > | >=
Addition  Term { AddOp Term }
AddOp  + | -
Term  Factor { MulOp Factor }
MulOp  * | / | %
Factor  [ UnaryOp ] Primary
UnaryOp  - | !
Primary  Identifier | Literal | ( Expression ) |
Type ( Expression )

13. Clite Grammar (3) Statements
Program  int main ( ) { Declarations Statements }
Declarations  { Declaration }
Declaration  Type Identifier { , Identifier }
Type  int | bool | float | char
Statements  { Statement }
Statement  ; | Block | Assignment | IfStatement | WhileStatement
Block  { Statements }
Assignment  Identifier = Expression ;
IfStatement  if ( Expression ) Statement [ else Statement ]
WhileStatement  while ( Expression ) Statement

14. Abstract Syntax
Removes “syntactic redundancies ” and keeps essential elements of a language.
Pascal
while i < n do begin
i := i + 1;
end;
C/C++
while (i < n) {
i = i + 1; }
The only essential information
It is a loop
A terminating condition i < n
A body increments the current value of i.

15. Parse and Abstract Syntax trees
Parse tree is inefficient
The shape of the parse tree reveals the meaning of the program.
So we want a tree that removes its inefficiency and keeps its shape.
Remove separator/punctuation terminal symbols
Remove all trivial root nonterminals
Replace remaining nonterminals with leaf terminals

16. Example: z = x + 2*y;
Parse Tree
Abstract Syntax Tree

17. Partial Abstract Syntax of Clite
Assignment = Variable target; Expression source
Expression = Variable | Value | Binary | Unary
Variable = String id
Value = Integer Value
Binary = Operator op; Expression term1, term2
Unary = Operator op; Expression term
Operator = + | - | * | / | !

18. Example Abstract Syntax Tree for z = x+2*y
Assignment = Variable target; Expression source
Expression = Variable | Value | Binary | Unary
Variable = String id
Value = Integer Value
Binary = Operator op; Expression term1, term2
Unary = Operator op; Expression term
Operator = + | - | * | / | !
Assignment
Variable
Binary
Operator
Value z + * x y 2