Parsing and Code Generation Set 24

Parser Construction Most of the work involved in constructing a parser is carried out automatically by a program, referred to as a compiler-compiler program, such as Yacc. To create a parser for a computer language, one constructs a description of the language, which is then employed as input to the compiler-compiler. This description of the language is in the form of a grammar for the language

Grammar 1 1. SENTENCE -> NOUNPHRASE VERB NOUNPHRASE 2. NOUNPHRASE -> the ADJECTIVE NOUN 3. NOUNPHRASE -> the NOUN 4. VERB -> pushed 5. VERB -> helped 6. ADJECTIVE -> pretty 7. ADJECTIVE -> poor 8. NOUN -> man 9. NOUN -> boy 10. NOUN -> cat

Grammar 1 is an example of a context-free grammar (the only kind we will deal with). The grammar consist of 10 productions e.g. production 3 is nounphrase -> the noun Here “nounphrase” is referred to as the lefthand side (lhs) and “the noun” is referred to as the righthand side (rhs)

The set of all lhs’s constitutes the set of nonterminals of the grammar. In this case they are: {SENTENCE, NOUNPHRASE,VERB, ADJECTIVE, NOUN} All the other symbols occurring in the grammar (i.e. in some rhs, but never as any lhs) are the terminals of the grammar. In this case {the,pushed,helped,pretty,poor,…}

The lhs of the first production is called the goal symbol, in this case “sentence”. A derivation of a string in the grammar is a list of strings starting with the goal symbol, in which each string, except the first, is obtained from the preceding one by applying a substitution of one of its symbols using one of the productions as a substitution rule

A string which has a derivation is said to be derivable. Derivable strings that consist entirely of terminal symbols are called sentences of the grammar. E.g. the man helped the poor boy is a sentence of Grammar 1. The set of all sentences of a grammar is called the language defined by the grammar

Grammar 1 (Cont.1) Derivation of the sentence: "the man helped the poor boy“ 1. SENTENCE (goal symbol) 2. ==> NOUNPHRASE VERB NOUNPHRASE (by Rule 1) 3. ==> the NOUN VERB NOUNPHRASE (Rule 3) 4. ==> the man VERB NOUNPHRASE (Rule 8) 5. ==> the man helped NOUNPHRASE 6. ==> the man helped the ADJECTIVE NOUN 7. ==> the man helped the poor NOUN 8. ==> the man helped the poor boy (this derivation shows that "the man helped the poor boy“ is a sentence in the language defined by the grammar.)

Grammar 1 (Cont.2) This derivation may also be represented diagrammatically by a syntax tree:

Typical format of a grammar for a programming language PROGRAM -> PROGRAM STATEMENT PROGRAM -> STATEMENT STATEMENT -> ASSIGNMENT-STATEMENT STATEMENT -> IF-STATEMENT STATEMENT -> DO-STATEMENT... ASSIGNMENT-STATEMENT -> IF-STATEMENT -> DO-STATEMENT ->......

Grammar 2 A simple grammar for arithmetic statements 1. E -> E + T 2. E -> T 3. T -> T * a 4. T -> a

Grammar 2 (Cont.1) Derivation of: a + a * a 1. E Goal Symbol 2. ==> E + T Rule 1 3. ==> E + T * a Rule 3 4. ==> E + a * a Rule 4 5. ==> T + a * a Rule 2 6. ==> a + a * a Rule 4

Grammar 2 (Cont.2) Derivation of: a + a * a written in reverse: 1. a + a * a Given sentential form 2. T + a * a Rule 4 in reverse 3. E + a * a Rule 2 in reverse 4. E + T * a Rule 4 5. E + T Rule 3 in reverse 6. E Rule 1

Syntax Analysis One of the functions of syntax analysis (parsing) is to verify whether the source program is syntactically correct The parser obtains a string of tokens from the scanner and verifies that the string can be generated by the grammar for the source language. (If not, the source program is not syntactically correct.)

There are many approaches to parser construction, including both top-down and bottom-up approaches. The LR parsing method discussed uses a bottom-up approach (generating the derivation in reverse) It constructs a rightmost derivation, in which at any step a production can only be applied to the rightmost nonterminal in the string involved. The method makes use of a parsing machine, which is shown on the next slide, and explained in the slides following

DEFINITIONS. The circles with numbers in them are called states. The labelled directed lines that go from from one state to another are said to represent transitions (as defined three slides on). Thus, e.g. the directed line, labelled with a “T” from state 2 to state 6, is said to represent a transition with respect to T from state 2 to state 6.

The other directed lines, which point from a state to some text involving a production, are said to represent reductions. Thus, e.g. the line pointing down from state 4 to the text: E → T if {*,+, } is said to represent the reduction E → T (substitution of lhs E for rhs T), to made if the next input symbol is a: *, or +, or the end of the source code, as represented by the symbol

LR Parsers -1 Print out the previous slide, and the example of it ’ s use 5 slides on, and follow this explanation along with the example. An LR Parser uses an input buffer of the remaining source code to be read, a symbol stack, a state no. stack, and a parsing machine (represented by a table) to determine what action to take when the next symbol is read from the input buffer of source code. The parser reads tokens (a i ) of the programming language grammar from the input buffer of source code one at a time. Using the combination of the state no. on top of the state stack and the next input symbol, a i, which is next to be read from the remaining source code, the parser consults the parsing machine to determine whether it should perform a transition or reduce action (as defined on the next slide)

LR Parsers -2 When the action determined by the parsing machine is make a transition to state s, the parser pushes the current input symbol a i onto the symbol stack and the new state no. s onto the state stack. s is now the new top of the state no. stack, and a i is the top of the symbol stack, and a i+1 is the new next input symbol. The remaining input to be processed is: a i+1... a n -|) where the symbol -| represents the end of the source file

LR Parsers -3 If the action determined by the action table entry is reduce using production A -> α, the parser performs the following sequence of actions, where r is the no. of symbols in α : Pops r symbols from the symbol stack and r states from the state stack. Consults the action table so as to make a transition with respect to the nonterminal symbol A (the left hand side of the reduce production), and the current state no. now at the top of the state-no stack.

LR Parsers -4 The remaining input remains unchanged (in contrast to transition actions). Some of the reduce actions cause associated code to be generated.

Example The following diagram graphically represents an action table generated for the grammar: E -> E + T | T T -> T * a | a It is called a parsing machine

Using the machine to parse a+a*a Step Number Stack Contents Remaining Input 1 Symbol: empty State: 0 a + a * a -| 2 Symbol: State: a * a -| 3 Symbol: a State: a * a -| 4 Symbol: T State: a * a -| 5 Symbol: E State: a * a -| 6 Symbol: E + State: * a -| 7 Symbol: E + a State: * a -| 8 Symbol: E + T State: a -| 9 Sym : E + T * St: Sym: E + T * a -| 10 State: | 11 Symbol: E + T State: 0 1 E -| 12 ACCEPT -|

Exercise 1. Supply a rightmost top-down derivation for a + a * a. 2. Supply a rightmost top-down derivation for a + a * a, this time without using Symbol Stack

Exercise (Cont.1) Now, look at the derivation given in the parse example in the last second slide. First, cross out all the state numbers and also the end marker, then rewrite that step of the parse in this form: e.g., Stack status: Remaining input Step 7 0 E T 6 + a -| is rewritten as: E + T + a and concatenated together becomes: E + T + a

Exercise (Cont.2) Do this to every step of the derivation, then cross out any duplicates of a given step. Now compare the result with the top-down derivation you obtained above. How are these two sets of results related to each other? Note that the parse provides a "bottom-up" derivation --- which contains the same steps as the top-down derivation but in the reverse order.