Presentation is loading. Please wait.

Presentation is loading. Please wait.

Bottom-up Parser Table Construction David Walker COS 320.

Similar presentations


Presentation on theme: "Bottom-up Parser Table Construction David Walker COS 320."— Presentation transcript:

1 Bottom-up Parser Table Construction David Walker COS 320

2 Programming Language Parsing LL(k) (top-down) parsing –compute nullable, first, follow sets then parsing table –use synchronizing tokens (Follow(X)) for error recovery –derive ML program LR(k) parsing -- more powerful than LL(k) parsers –according to parser table: shift tokens from input on to stack reduce stack symbols using grammar rules accept or signal errors –Fisher-Burke error repair global technique (try every possible insertion/replacement/substitution in k-token window) –Now the magic: how to construct the parser table

3 finally the magic: how to construct an LR parser table At every point in the parse, the LR parser table tells us what to do next –shift, reduce, error or accept To do so, the LR parser keeps track of the parse “state” ==> a state in a finite automaton exp PLUS ( exp PLUS NUM PLUS ( NUM PLUS NUM ) PLUS NUM yet to read input: stack:

4 finally the magic: how to construct an LR parser table exp PLUS ( exp PLUS NUM PLUS ( NUM PLUS NUM ) PLUS NUM yet to read input: stack: 1 4 2 3 exp plus exp 5 minus exp finite automaton; terminals and non terminals label edges (

5 finally the magic: how to construct an LR parser table exp PLUS ( exp PLUS NUM PLUS ( NUM PLUS NUM ) PLUS NUM yet to read input: stack: 1 4 2 3 exp plus exp 5 minus exp finite automaton; terminals and non terminals label edges 1state-annotated stack: (

6 finally the magic: how to construct an LR parser table exp PLUS ( exp PLUS NUM PLUS ( NUM PLUS NUM ) PLUS NUM yet to read input: stack: 1 4 2 3 exp plus exp 5 minus exp finite automaton; terminals and non terminals label edges 1 exp 2state-annotated stack: (

7 finally the magic: how to construct an LR parser table exp PLUS ( exp PLUS NUM PLUS ( NUM PLUS NUM ) PLUS NUM yet to read input: stack: 1 4 2 3 exp plus exp 5 minus exp finite automaton; terminals and non terminals label edges 1 exp 2 PLUS 3state-annotated stack: (

8 finally the magic: how to construct an LR parser table exp PLUS ( exp PLUS NUM PLUS ( NUM PLUS NUM ) PLUS NUM yet to read input: stack: 1 4 2 3 exp plus exp 5 minus exp finite automaton; terminals and non terminals label edges 1 exp 2 PLUS 3 ( 1 exp 2 PLUS 3state-annotated stack: ( this state and input tell us what to do next

9 The Parse Table At every point in the parse, the LR parser table tells us what to do next according to the automaton state at the top of the stack –shift, reduce, error or accept statesTerminal seen next ID, NUM, :=... 1 2sn = shift & goto state n 3rk = reduce by rule k...a = accept n = error

10 The Parse Table Reducing by rule k is broken into two steps: –current stack is: A 8 B 3 C....... 7 RHS 12 –rewrite the stack according to X ::= RHS: A 8 B 3 C....... 7 X –figure out state on top of stack (ie: goto 13) A 8 B 3 C....... 7 X 13 statesTerminal seen next ID, NUM, :=...Non-terminals X,Y,Z... 1 2sn = shift & goto state ngn = goto state n 3rk = reduce by rule k...a = accept n = error

11 The Parse Table Reducing by rule k is broken into two steps: –current stack is: A 8 B 3 C....... 7 RHS 12 –rewrite the stack according to X ::= RHS: A 8 B 3 C....... 7 X –figure out state on top of stack (ie: goto 13) A 8 B 3 C....... 7 X 13 statesTerminal seen next ID, NUM, :=...Non-terminals X,Y,Z... 1 2sn = shift & goto state ngn = goto state n 3rk = reduce by rule k...a = accept n = error

12 LR(0) parsing each state in the automaton represents a collection of LR(0) items: –an item is a rule from the grammar combined with “@” to indicate where the parser currently is in the input eg: S’ ::= @ S $ indicates that the parser is just beginning to parse this rule and it expects to be able to parse S then $ next A whole automaton state looks like this: S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 state number collection of LR(0) items LR(1) states look very similar, it is just that the items contain some look-ahead info

13 LR(0) parsing To construct states, we begin with a particular LR(0) item and construct its closure –the closure adds more items to a set when the “@” appears to the left of a non-terminal –if the state includes X ::= s @ Y s’ and Y ::= t is a rule then the state also includes Y ::= @ t S’ ::= @ S $ 1 Grammar: 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

14 LR(0) parsing To construct states, we begin with a particular LR(0) item and construct its closure –the closure adds more items to a set when the “@” appears to the left of a non-terminal –if the state includes X ::= s @ Y s’ and Y ::= t is a rule then the state also includes Y ::= @ t S’ ::= @ S $ S ::= @ ( L ) 1 Grammar: 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

15 LR(0) parsing To construct states, we begin with a particular LR(0) item and construct its closure –the closure adds more items to a set when the “@” appears to the left of a non-terminal –if the state includes X ::= s @ Y s’ and Y ::= t is a rule then the state also includes Y ::= @ t S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 Grammar: Full Closure 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

16 LR(0) parsing To construct an LR(0) automaton: –start with start rule & compute initial state with closure –pick one of the items from the state and move “@” to the right one symbol (as if you have just parsed the symbol) this creates a new item...... and a new state when you compute the closure of the new item mark the edge between the two states with: –a terminal T, if you moved “@” over T –a non-terminal X, if you moved “@” over X –continue until there are no further ways to move “@” across items and generate new states or new edges in the automaton

17 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

18 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S’ ::= S @ $ S 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

19 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S’ ::= S @ $ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

20 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S’ ::= S @ $ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

21 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S’ ::= S @ $ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

22 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S’ ::= S @ $ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S S ( L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

23 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S S ( S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

24 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S S ( x S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

25 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S S ::= ( L ) @ S ( x ) S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

26 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S S ::= ( L ) @ S ( x, ) S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

27 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S S ::= ( L ) @ S ( x, ) S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

28 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ S ( x S, ) S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

29 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ S ( x ( S, ) S L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

30 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ S ( x ( x S, ) S L x 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

31 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 Grammar: S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 7 6 5 8 9 S ( x ( x S, ) S L Assigning numbers to states: x ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

32 computing parse table State i contains X ::= s @ $ ==> table[i,$] = a State i contains rule k: X ::= s @ ==> table[i,T] = rk for all terminals T Transition from i to j marked with T ==> table[i,T] = sj Transition from i to j marked with X ==> table[i,X] = gj statesTerminal seen next ID, NUM, :=...Non-terminals X,Y,Z... 1 2sn = shift & goto state ngn = goto state n 3rk = reduce by rule k...a = accept n = error

33 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L states()x,$SL 1 2 3 4... x ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

34 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L states()x,$SL 1s3 2 3 4... 2 x ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

35 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L states()x,$SL 1s3s2 2 3 4... 2 x ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

36 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L states()x,$SL 1s3s2g4 2 3 4... 2 x ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

37 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L states()x,$SL 1s3s2g4 2r2 3 4... 2 x ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

38 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L 2 x states()x,$SL 1s3s2g4 2r2 3s3s2 4... ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

39 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L 2 x states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4... ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

40 S’ ::= @ S $ S ::= @ ( L ) S ::= @ x 1 S ::= x @ S’ ::= S @ $ L ::= S @ S ::= ( @ L ) L ::= @ S L ::= @ L, S S ::= @ ( L ) S ::= @ x L ::= L, @ S S ::= @ ( L ) S ::= @ x S ::= ( L @ ) L ::= L @, S L ::= L, S @ S ::= ( L ) @ 2 4 3 76 5 8 9 S ( x ( x S, ) S L 2 x states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a... ( 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

41 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 1 ( x, x ) $ yet to read input: stack: 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

42 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

43 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 x 2 1.S’ ::= S $ 2.S ::= ( L ) 3.S ::= x 4.L ::= S 5.L ::= L, S

44 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 S 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

45 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 S 7 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

46 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

47 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 5 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

48 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 5, 8 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

49 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 5, 8 x 2 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

50 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 5, 8 S 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

51 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 5, 8 S 9 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

52 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S

53 states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 4a 5s6s8 6r1 7r3 8s3s2g9 9r4 ( x, x ) $ yet to read input: stack:1 ( 3 L 5 0. S’ ::= S $ S ::= ( L ) S ::= x L ::= S L ::= L, S etc......

54 LR(0) Even though we are doing LR(0) parsing we are using some look ahead (there is a column for each non-terminal) however, we only use the terminal to figure out which state to go to next, not to decide whether to shift or reduce states()x,$SL 1s3s2g4 2r2 3s3s2g7g5

55 LR(0) Even though we are doing LR(0) parsing we are using some look ahead (there is a column for each non-terminal) however, we only use the terminal to figure out which state to go to next, not to decide whether to shift or reduce states()x,$SL 1s3s2g4 2r2 3s3s2g7g5 statesno look-aheadSL 1shiftg4 2reduce 2 3shiftg7g5 ignore next automaton state

56 LR(0) Even though we are doing LR(0) parsing we are using some look ahead (there is a column for each non-terminal) however, we only use the terminal to figure out which state to go to next, not to decide whether to shift or reduce If the same row contains both shift and reduce, we will have a conflict ==> the grammar is not LR(0) Likewise if the same row contains reduce by two different rules statesno look-aheadSL 1shift, reduce 5g4 2reduce 2, reduce 7 3shiftg7g5

57 SLR SLR (simple LR) is a variant of LR(0) that reduces the number of conflicts in LR(0) tables by using a tiny bit of look ahead To determine when to reduce, 1 symbol of look ahead is used. Only put reduce by rule (X ::= RHS) in column T if T is in Follow(X) states()x,$SL 1s3s2g4 2r2s5r2 3r1 r5 g7g5 cuts down the number of rk slots & therefore cuts down conflicts

58 LR(1) & LALR LR(1) automata are identical to LR(0) except for the “items” that make up the states LR(0) items: X ::= s1 @ s2 LR(1) items X ::= s1 @ s2, T –Idea: sequence s1 is on stack; input stream is s2 T Find closure with respect to X ::= s1 @ Y s2, T by adding all items Y ::= s3, U when Y ::= s3 is a rule and U is in First(s2 T) Two states are different if they contain the same rules but the rules have different look-ahead symbols –Leads to many states –LALR(1) = LR(1) where states that are identical aside from look-ahead symbols have been merged –ML-Yacc & most parser generators use LALR READ: Appel 3.3 (and also all of the rest of chapter 3) look-ahead symbol added

59 Grammar Relationships Unambiguous GrammarsAmbiguous Grammars LR(0)SLR LALRLR(1) LL(0) LL(1)

60 summary LR parsing is more powerful than LL parsing, given the same look ahead to construct an LR parser, it is necessary to compute an LR parser table the LR parser table represents a finite automaton that walks over the parser stack ML-Yacc uses LALR, a compact variant of LR(1)


Download ppt "Bottom-up Parser Table Construction David Walker COS 320."

Similar presentations


Ads by Google