GRAMMARS & PARSING

Parser Construction Most of the work involved in constructing a parser is carried out automatically by a program, referred to as a compiler-compiler program, such as Yacc. To create a parser for a computer language, one constructs a description of the language, which is then employed as input to the compiler-compiler. This description of the language is in the form of a grammar for the language

Grammar 1 1. SENTENCE -> NOUNPHRASE VERB NOUNPHRASE 2. NOUNPHRASE -> the ADJECTIVE NOUN 3. NOUNPHRASE -> the NOUN 4. VERB -> pushed 5. VERB -> helped 6. ADJECTIVE -> pretty 7. ADJECTIVE -> poor 8. NOUN -> man 9. NOUN -> boy 10. NOUN -> cat

Grammar 1 is an example of a context-free grammar (the only kind we will deal with). The grammar consist of 10 productions e.g. production 3 is nounphrase -> the noun Here “nounphrase” is referred to as the lefthand side (lhs) and “the noun” is referred to as the righthand side (rhs)

The set of all lhs’s constitutes the set of nonterminals of the grammar. In this case they are: {SENTENCE, NOUNPHRASE,VERB, ADJECTIVE, NOUN} All the other symbols occurring in the grammar (i.e. in some rhs, but never as any lhs) are the terminals of the grammar. In this case {the,pushed,helped,pretty,poor,…}

The lhs of the first production is called the goal symbol, in this case “sentence”. A derivation of a string in the grammar is a list of strings starting with the goal symbol, in which each string, except the first, is obtained from the preceding one by applying a substitution of one of its symbols using one of the productions as a substitution rule

A string which has a derivation is said to be derivable. Derivable strings that consist entirely of terminal symbols are called sentences of the grammar. E.g. the man helped the poor boy is a sentence of Grammar 1. The set of all sentences of a grammar is called the language defined by the grammar

Grammar 1 (Cont.1) Derivation of the sentence: "the man helped the poor boy“ 1. SENTENCE (goal symbol) 2. ==> NOUNPHRASE VERB NOUNPHRASE (by Rule 1) 3. ==> the NOUN VERB NOUNPHRASE (Rule 3) 4. ==> the man VERB NOUNPHRASE (Rule 8) 5. ==> the man helped NOUNPHRASE 6. ==> the man helped the ADJECTIVE NOUN 7. ==> the man helped the poor NOUN 8. ==> the man helped the poor boy (this derivation shows that "the man helped the poor boy“ is a sentence in the language defined by the grammar.)

Grammar 1 (Cont.2) This derivation may also be represented diagrammatically by a syntax tree:

Typical format of a grammar for a programming language PROGRAM -> PROGRAM STATEMENT PROGRAM -> STATEMENT STATEMENT -> ASSIGNMENT-STATEMENT STATEMENT -> IF-STATEMENT STATEMENT -> DO-STATEMENT... ASSIGNMENT-STATEMENT -> IF-STATEMENT -> DO-STATEMENT ->......

Grammar 2 A simple grammar for arithmetic statements 1. E -> E + T 2. E -> T 3. T -> T * a 4. T -> a

Grammar 2 (Cont.1) Derivation of: a + a * a 1. E Goal Symbol 2. ==> E + T Rule 1 3. ==> E + T * a Rule 3 4. ==> E + a * a Rule 4 5. ==> T + a * a Rule 2 6. ==> a + a * a Rule 4

Grammar 2 (Cont.2) Derivation of: a + a * a written in reverse: 1. a + a * a Given sentential form 2. T + a * a Rule 4 in reverse 3. E + a * a Rule 2 in reverse 4. E + T * a Rule 4 5. E + T Rule 3 in reverse 6. E Rule 1

Syntax Analysis -1 One of the functions of syntax analysis (parsing) is to verify whether the source program is syntactically correct The parser obtains a string of tokens from the scanner and verifies that the string can be generated by the grammar for the source language. (If not, the source program is not syntactically correct.)

Syntax Analysis -2 There are many approaches to parsing, including both top-down and bottom-up approaches. The LR parsing method discussed uses a bottom- up approach (generating the derivation in reverse) It constructs a rightmost derivation, in which at any step a production can only be applied to the rightmost nonterminal in the string involved.

LR Parsers -1 An LR Parser uses an input buffer of the remaining source code to be read, a symbol stack, a state no. stack, and an action table to determine what action to take when the next symbol is read from the input buffer. The parser reads tokens (a i ) of the programming language grammar from the input buffer one at a time. Using the combination of the state no. on top of the state stack and the current input symbol, a i, in the input buffer, the parser consults the action table to determine whether it should perform a transition or reduce action (as defined on the next slide)

LR Parsers -2 When the action determined by the action table entry is make a transition to state s, the parser pushes the current input symbol a i onto the symbol stack and the new state no. s onto the state stack. s is now the new top of the state no. stack, and a i is the top of the symbol stack, and a i+1 is the new next input symbol. The remaining input to be processed is: a i+1... a n -|) where the symbol -| represents the end of the source file

LR Parsers -3 If the action determined by the action table entry is reduce using production A -> b, the parser performs the following sequence of actions: Pops r symbols from the symbol stack and r states from the state stack. Consults the action table to decide which state s r to make a transition to with respect to the nonterminal symbol A (the left hand side of the reduce production) as a result of the reduction, and the current state no. now at the top of the state-no stack.

LR Parsers -4 The nonterminal symbol A is pushed onto the symbol stack (replacing the symbols of the right hand side of the production that were previously on the symbol stack), and the new state s r after the reduction is pushed onto the state stack (replacing the states the were removed from the state stack). The remaining input remains unchanged (in contrast to transition actions). Some of the reduce actions cause associated code to be generated.

Example The following diagram graphically represents an action table generated for the grammar: E -> E + T | T T -> T * a | a It is called a parsing machine

Using the machine to parse a+a*a Step Number Stack Contents Remaining Input 1 Symbol: empty State: 0 a + a * a -| 2 Symbol: State: a * a -| 3 Symbol: a State: a * a -| 4 Symbol: T State: a * a -| 5 Symbol: E State: a * a -| 6 Symbol: E + State: * a -| 7 Symbol: E + a State: * a -| 8 Symbol: E + T State: a -| 9 Sym : E + T * St: Sym: E + T * a -| 10 State: | 11 Symbol: E + T State: 0 1 E -| 12 ACCEPT -|

Exercise Supply a rightmost top-down derivation for a + a * a.

Exercise (Cont.1) Now, look at the derivation given in the parse example in the last second slide. First, cross out all the state numbers and also the end marker, then rewrite that step of the parse in this form: e.g., Stack status: Remaining input Step 7 0 E T 6 + a -| is rewritten as: E + T + a and concatenated together becomes: E + T + a

Exercise (Cont.2) Do this to every step of the derivation, then cross out any duplicates of a given step. Now compare the result with the top-down derivation you obtained above. How are these two sets of results related to each other? Note that the parse provides a "bottom-up" derivation --- which contains the same steps as the top-down derivation but in the reverse order.