Programming Languages Translator Syntax and Semantics

Top–Down Parsing Bottom–Up Parsing A parse tree is created from root to leaves Tracing leftmost derivation Two types: Backtracking parser Predictive parser A parse tree is created from leaves to root Tracing rightmost derivation More powerful than top-down parsing

LL(1) Parsing LL(1) Use stack to simulate leftmost derivation Read input from (L) left to right Simulate (L) leftmost derivation 1 lookahead symbol Use stack to simulate leftmost derivation Part of sentential form produced in the leftmost derivation is stored in the stack. Top of stack is the leftmost nonterminal symbol in the fragment of sentential form.

Recursive-Descent Write one procedure for each set of productions with the same nonterminal in the LHS Each procedure recognizes a structure described by a nonterminal. A procedure calls other procedures if it need to recognize other structures. A procedure calls match procedure if it need to recognize a terminal.

Recursive Descent Parsing Top-down, left-to-right construction of the parse tree

Recursive-Descent: Example For this grammar: We cannot decide which rule to use for E, and If we choose E  E O F, it leads to infinitely recursive loops. Rewrite the grammar into EBNF procedure E { F; while (token=+ or token=-) { O; F; } } E ::= F {O F} O ::= + | - F ::= ( E ) | id E  E O F | F O  + | - F  ( E ) | id procedure F { switch token { case (: match(‘(‘); E; match(‘)’); case id: match(id); default: error; } procedure E { E; O; F; }

Problems in Recursive-Descent Difficult to convert grammars into EBNF Cannot decide which production to use at each point Cannot decide when to use -production A 

Bottom-up Parsing Use explicit stack to perform a parse Simulate rightmost derivation (R) from left (L) to right, thus called LR parsing More powerful than top-down parsing Left recursion does not cause problem Two actions Shift: take next input token into the stack Reduce: replace a string B on top of stack by a nonterminal A, given a production A  B

Bottom-up Parsing (cont.) Shift-Reduce Algorithms Reduce is the action of replacing the handle on the top of the parse stack with its corresponding LHS Shift is the action of moving the next token to the top of the parse stack

Example of Shift-reduce Parsing Grammar S’  S S  (S)S |  Parsing actions Stack Input Action $ ( ( ) ) $ shift $ ( ( ) ) $ shift $ ( ( ) ) $ reduce S   $ ( ( S ) ) $ shift $ ( ( S ) ) $ reduce S   $ ( ( S ) S ) $ reduce S  ( S ) S $ ( S ) $ shift $ ( S ) $ reduce S   $ ( S ) S $ reduce S  ( S ) S $ S $ accept Reverse of rightmost derivation from left to right 1  ( ( ) ) 2  ( ( ) ) 3  ( ( ) ) 4  ( ( S ) ) 5  ( ( S ) ) 6  ( ( S ) S ) 7  ( S ) 8  ( S ) 9  ( S ) S 10 S’  S

Example of LR(0) Parsing Stack Input Action $0 ( ( a ) ) $ shift $0(3 ( a ) ) $ shift $0(3(3 a ) ) $ shift $0(3(3a2 ) ) $ reduce $0(3(3A4 ) ) $ shift $0(3(3A4)5 ) $ reduce $0(3A4 ) $ shift $0(3A4)5 $ reduce $0A1 $ accept

Introduction(3) Bottom-Up Parser Example Bottom-Up Parsing Program Shift a OUTPUT: INPUT: a b b c d e $ Production S  aABe Bottom-Up Parsing Program A  Abc A  b B  d

Introduction(4) Bottom-Up Parser Example Bottom-Up Parsing Program Shift b Reduce from b to A OUTPUT: INPUT: a b b c d e $ Production S  aABe Bottom-Up Parsing Program A  Abc A b A  b B  d

Introduction(5) Bottom-Up Parser Example Bottom-Up Parsing Program Shift A OUTPUT: INPUT: a A b c d e $ Production S  aABe Bottom-Up Parsing Program A  Abc A A  b b B  d

Introduction(6) Bottom-Up Parser Example Bottom-Up Parsing Program Shift b OUTPUT: INPUT: a A b c d e $ Production S  aABe Bottom-Up Parsing Program A  Abc A A  b b B  d

Introduction(7) Bottom-Up Parser Example Bottom-Up Parsing Program Shift c Reduce from Abc to A OUTPUT: INPUT: a A b c d e $ Production c A b S  aABe Bottom-Up Parsing Program A  Abc A A  b b B  d

Introduction(8) Bottom-Up Parser Example Bottom-Up Parsing Program Shift A OUTPUT: INPUT: a A d e $ Production A S  aABe Bottom-Up Parsing Program b A  Abc A c A  b b B  d

Introduction(9) Bottom-Up Parser Example Bottom-Up Parsing Program Shift d Reduce from d to B OUTPUT: INPUT: a A d e $ Production A B d S  aABe Bottom-Up Parsing Program b A  Abc A c A  b b B  d

Introduction(10) Bottom-Up Parser Example Bottom-Up Parsing Program Shift B OUTPUT: INPUT: a A B e $ Production A B d S  aABe Bottom-Up Parsing Program b A  Abc A c A  b b B  d

Introduction(11) Bottom-Up Parser Example Bottom-Up Parsing Program Shift e Reduce from aABe to S OUTPUT: INPUT: a A B e $ a S e Production A B d S  aABe Bottom-Up Parsing Program b A  Abc A c A  b b B  d

Introduction(12) Bottom-Up Parser Example Bottom-Up Parsing Program Shift S Hit the target $ OUTPUT: INPUT: S $ a S e Production A B d S  aABe Bottom-Up Parsing Program b A  Abc A c A  b b B  d This parser is known as an LR Parser because it scans the input from Left to right, and it constructs a Rightmost derivation in reverse order.

Shift-Reduce Parsing Idea: build the parse tree bottom-up Lexer supplies a token, parser find production rule with matching right-hand side (i.e., run rules in reverse) If start symbol is reached, parsing is successful  7 8 <digit>  7 8 <num>  7 <digit> <num>  7 <num>  <digit> <num>  <num> 789 Production rules: Num  Digit | Digit Num Digit  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 reduce shift reduce shift reduce

Consider the context-free grammar S  a X X  b X | b Y Y  c Detail how an shift-reduce parser will parse the sentence abbc

Example <string exp> ::= <partial string> | <string exp> & <partial string> <partial string> ::= <partial string> - <nested string> | <nested string> <nested string> ::= <basic string> | < nested string> * <basic string> <basic string> ::= <basic string> <letter> |  <letter> ::= A|B|C|D|E|F|G|H|I|J|Y|Z & * -  

Bottom-up Parsing (cont.) LR parsers are table driven, where the table has two components, an ACTION table and a GOTO table The ACTION table specifies the action of the parser, given the parser state and the next token Rows are state names; columns are terminals The GOTO table specifies which state to put on top of the parse stack after a reduction action is done Rows are state names; columns are nonterminals

LR Parsing Table

LR(0) parsing Keep track of what is left to be done in the parsing process by using finite automata of items An item A  w . B y means: A  w B y might be used for the reduction in the future, at the time, we know we already construct w in the parsing process, if B is constructed next, we get the new item A  w B . Y

LR(0) items LR(0) item Initial Item Complete Item Closure Item of x production with a distinguished position in the RHS Initial Item Item with the distinguished position on the leftmost of the production Complete Item Item with the distinguished position on the rightmost of the production Closure Item of x Item x together with items which can be reached from x via -transition Kernel Item Original item, not including closure items

Finite automata of items Grammar: S’  S S  (S)S S   Items: S’  .S S’  S. S  .(S)S S  (.S)S S  (S.)S S  (S).S S  (S)S. S  . S S’  .S S’  S.   S  .(S)S S  . (   S S  (.S)S S  (S.)S   ) S S  (S).S S  (S)S.

DFA of LR(0) Items       S S S’  .S S  .(S)S S  . S’  S.

LR(0) Parsing Table A A’  A. A’  .A A  .(A) A  .a a A  a. ( a 1 A’  .A A  .(A) A  .a a A  a. 2 ( a A  (.A) A  .(A) A  .a A  (A.) 4 A 3 ) ( A  (A). 5

Bottom Up Technique It begins with terminal token, and scan for sub-expression whose operators have higher precedence and interprets it into terms of the rule of grammar until the root of the tree

The method A + B * C - D <. .> Then the sub-expression B * C is computed before other operations in the statement

The method So the bottom-up parser should recognize B * C (in terms of grammar) before considering the surrounding terms. First, we determine the precedence relations between operators in the grammar.

Operator Precedence We have Program = var Begin < for Which means program and var have equal precedence

Example We have But So which is first, is higher ; .> END

Example read ( value ); = < > = < > Start with higher operator or terminal one “value” as id

Example Search for non-terminal for id and so assign it as <N1> READ ( <N1> ) Next take read to another nonterminal <N2>

The method The operator precedence parser used a stack to save token that have been scanned.

Example <string exp> ::= <partial string> | <string exp> & <partial string> <partial string> ::= <partial string> - <nested string> | <nested string> <nested string> ::= <basic string> | < nested string> * <basic string> <basic string> ::= <basic string> <letter> |  <letter> ::= A|B|C|D|E|F|G|H|I|J|Y|Z Use one LL and shift reduce method to draw the parse tree for expression F * A & G - Y