Discussion #31/20 Discussion #3 Grammar Formalization & Parse-Tree Construction

Discussion #32/20 Topics Grammar Definitions Parse Trees Constructing Parse Trees

Discussion #33/20 Formal Definition of a Grammar A grammar G is a 4-tuple: G = (V N, V T, S,  ), where –V N, V T, sets of non-terminal and terminal symbols –S  V N, a start symbol –  = a finite set of relations from (V T  V N ) + to (V T  V N ) * –an element of , ( ,  ), is written as    and is called a production rule or a rewriting rule

Discussion #34/20 Examples of Grammars G 1 = (V N, V T, S,  ), where: V N = {S, B} V T = {a, b, c} S = S  = { S  aBSc, S  abc, Ba  aB, Bb  bb } G 2 = (V N, V T, S,  ), where: V N = {I, L, D} V T = {a, b, …, z, 0, 1, …, 9} S = I  = { I  L | ID | IL, L  a | b | … | z, D  0 | 1 | … | 9 } G 3 = (V N, V T, S,  ), where:  = { S  aA, V N = {S, A, B } A  aA | bB, V T = {a, b} B  bB |  } S = S

Discussion #35/20 Definition of a Context-Free Grammar A context-free grammar is a grammar with the following restriction: –The relation  is a finite set of relations from V N to (V T  V N ) + –i.e. the left hand side of a production is a single non- terminal –i.e. the right hand side of any production cannot be empty Context-free grammars generate context-free languages. With slight variations, essentially all programming languages are context-free languages.

Discussion #36/20 Examples of Grammars (again) Which are context-free grammars? G 1 = (V N, V T, S,  ), where: V N = {S, B} V T = {a, b, c} S = S  = { S  aBSc, S  abc, Ba  aB, Bb  bb } G 2 = (V N, V T, S,  ), where: V N = {I, L, D} V T = {a, b, …, z, 0, 1, …, 9} S = I  = { I  L | ID | IL, L  a | b | … | z, D  0 | 1 | … | 9 } G 3 = (V N, V T, S,  ), where:  = { S  aA, V N = {S, A, B } A  aA | bB, V T = {a, b} B  bB |  } S = S

Discussion #37/20 Backus-Naur Form (BNF) A traditional meta language to represent grammars for programming languages Every non-terminal is enclosed in Instead of the symbol  we use ::= Example I  L | ID | IL L  a | b | … | z D  0 | 1 | … | 9 BNF: ::= | | ::= a | b | … | z ::= 0 | 1 | … | 9

Discussion #38/20 Definition: Direct Derivative Let G = (V N, V T, S,  ) be a grammar and ,   (V N  V T ) *,  is said to be a direct derivative of , (written    ) if there are strings  1 and  2 (including possibly empty strings) such that  =  1 B  2,  =  1  2, B  V N and B   is a production of G.

Discussion #39/20 Example: Direct Derivatives G = (V N, V T, S,  ), where: V N = {I, L, D} V T = {a, b, …, z, 0, 1, …, 9} S = I  = { I  L | ID | IL L  a | b | … | z D  0 | 1 | … | 9 }  Rule Used 11 22 IL I  L  IbLb I  L  b Lbab L  a  b IDDI0D D  0 ID

Discussion #310/20 Definition: Derivation Let G = (V N, V T, S,  ) be a grammar A string  produces  (  reduces to  or  is the derivation of , written   +  ), if there are strings  0,  1, …,  n (n>0) such that  =  0   1,  1   2, …,  n-1   n,  n  .

Discussion #311/20 Example: Derivation Let G = (V N, V T, S,  ), where: V N = {I, L, D} V T = {a, b, …, z, 0, 1, …, 9} S = I  = { I  L | ID | IL L  a | b | … | z D  0 | 1 | … | 9 } I produces abc12 I  ID  IDD  ILDD  ILLDD  LLLDD  aLLDD  abLDD  abcDD  abc1D  abc12

Discussion #312/20 Definition: Language A sentential form is any derivative of the start symbol S. A language L generated by a grammar G is the set of all sentential forms whose symbols are all terminals; that is, L(G) = {  | S  +  and   V T * }

Discussion #313/20 Example: Language Let G = (V N, V T, S,  ), where: V N = {I, L, D} V T = {a, b, …, z, 0, 1, …, 9} S = I  = { I  L | ID | IL L  a | b | … | z D  0 | 1 | … | 9 } I produces abc12 L(G) = {abc12, x, m , a1b2c3, …} I  ID  IDD  ILDD  ILLDD  LLLDD  aLLDD  abLDD  abcDD  abc1D  abc12

Discussion #314/20 Syntax Analysis: Parsing The parse of a sentence is the construction of a derivation for that sentence The parsing of a sentence results in –acceptance or rejection –and, if acceptance, then also a parse tree We are looking for an algorithm to parse a sentence (i.e. to parse a program) and produce a parse tree.

Discussion #315/20 Parse Trees A parse tree is composed of –interior nodes representing syntactic categories (non-terminal symbols) –leaf nodes representing terminal symbols For each interior node N, the transition from N to its children represents the application of a production.

Discussion #316/20 Parse Tree Construction Top-down –Starts with the root (starting symbol) –Proceeds downward to leaves using productions Bottom-up –Starts from leaves –Proceeds upward to the root Although these seem like reasonable approaches to develop a parsing algorithm, we’ll see that neither works well  so we’ll need to find a better way.

Discussion #317/20 Example: Top-Down Parse for 4 * E V N = {E, D} V T = {0, 1, …, 9, +, , *, /, (, )} S = E  = { E  D | ( E ) | E + E | E – E | E * E | E / E, D  0 | 1 | … | 9 } EE* EE+ D DD 4 23 Problems: -How do we guess which rule applies? -Note that we produced the wrong parse tree (precedence is wrong)

Discussion #318/20 Ambiguous Grammar Two Different Parse Trees for 4*2+3  = { E  D | ( E ) | E + E | E – E | E * E | E / E, D  0 | 1 | … | 9 } E EE * EE+D DD4 23 E EE+ EE*D DD 42 3

Discussion #319/20 Example: Bottom-Up Parse 1.A  V | I | (A + A) | (A * A) 2.V  L | VL | VD 3.I  D | ID 4.D  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 5.L  x | y | z ( ( z * ( x + y ) ) ) ( ( V * ( V + V ) ) + I D) A ( A + A ) ( ( L * ( L + L ) ) + D D) ( ( A * ( A + A ) ) + I ) ( ( A * A ) + A ) Problem: I ?? D Problem: scanning the entire program repeatedly

Discussion #320/20 So, how do we develop a parsing algorithm? “Fix” the grammar –So that we can go top down, left to right, with no backup –LL(1) grammar: Left-to-right, Left-most non-terminal, one symbol look ahead “Fix” (How?) –Observe grammar properties: determine what’s needed to make them LL(1) –Transform grammars to make them LL(1) Note: works for many grammars, but not all