1. – 1 – CSCE 531 Spring 2006 Lecture 7 Predictive Parsing Topics Review Top Down Parsing First Follow LL (1) Table construction Readings: 4.4 Homework: Program 2 February 1, 2004 CSCE 531 Compiler Construction

2. – 2 – CSCE 531 Spring 2006 Overview Last Time Ambiguity in classic programming language grammars Expressions If-Then-Else Top-Down Parsing Modifying Grammars to facilitate Top-down parsing Today’s Lecture Regroup halfway to Test 1 First and Follow LL(1) propertyReferences:Homework:

3. – 3 – CSCE 531 Spring 2006 Removing the IF-ELSE Ambiguity Stmt  if Expr then Stmt | if Expr then Stmt else Stmt | other stmts Stmt  MatchedStmt | UnmatchedStmt MatchedStmt  if Expr then MatchedStmt else MatchedStmt | OthersStatements UnmatchedStmt  if Expr then MatchedStmt else | if Expr then MatchedStmt else UmatchedStmt

4. – 4 – CSCE 531 Spring 2006 Recursive Descent Parsers Recall the parser from Chapter 2 Recall the parser from Chapter 2 A recursive descent parser has a routine for each nonterminal. These routines can call each other. If one of these fails then it may backtrack to a point where there is an alternative choice. A recursive descent parser has a routine for each nonterminal. These routines can call each other. If one of these fails then it may backtrack to a point where there is an alternative choice. In certain cases the grammar is restricted enough where backtracking would never be required. In certain cases the grammar is restricted enough where backtracking would never be required. Such a parser is called a predictive parser. Such a parser is called a predictive parser. The parser from Chapter 2 is a predictive parser. The parser from Chapter 2 is a predictive parser.

5. – 5 – CSCE 531 Spring 2006 Transition Diagrams for Predictive Parsers To construct the transition diagram for a predictive parser:  Eliminate left recursion from the grammar  Left factor the grammar  For each nonterminal A do Create an initial state and final state. For each production A  X 1 X 2 … X n create a path from the initial state to the final state labeled X 1 X 2 … X n end

6. – 6 – CSCE 531 Spring 2006 Example E  T E’ E’  + T E’ | - T E’ | ε T  F T’ T’  * F T’ | / F T’ | ε F  id | num | ( E ) 1 E 23 TE’ 4 T 56 FT’ 1 E’ 23 + T 3 23 T - ε Etcetera Some of the rest in the text.

7. – 7 – CSCE 531 Spring 2006 Predictive Parsing using Transition Diagrams

8. – 8 – CSCE 531 Spring 2006 Table Driven Predictive Parsing x+(… WX Y S R $ Stack input output Predictive Parsing Program Parsing Table M

9. – 9 – CSCE 531 Spring 2006 Table Driven Predictive Parsing The stack is initialized to contain $S, the $ is the “bottom” marker. The stack is initialized to contain $S, the $ is the “bottom” marker. The input has a $ added to the end. The input has a $ added to the end. The parse table, M[X, a] contains what should be done when we see nonterminal X on the stack and current token “a” The parse table, M[X, a] contains what should be done when we see nonterminal X on the stack and current token “a” Parse Actions for Parse Actions for X = top of stack, and a = current token  If X = a = $ then halt and announce success.  If X = a != $ then pop X off the stack and advance the input pointer to the next token.  If X is nonterminal consult the table entry M[X, a], details on next slide.

10. – 10 – CSCE 531 Spring 2006 M[X, a] Actions  If X is nonterminal then consult M[X, a]. The entry will be either a production or an error entry. The entry will be either a production or an error entry. If M[X, a] = {X  UVW} the parser If M[X, a] = {X  UVW} the parser replaces X on the top of the stack with W, V, U with the U on the top As output print the name of the production used.

11. – 11 – CSCE 531 Spring 2006 Algorithm 4.3 Set ip to the first token in w$. Repeat Let X be the top of the stack and a be the current token if X is a terminal or $ then if X = a then pop X from the stack and advance the ip else error() else/* X is a nonterminal */ if M[X, a] = X  Y 1 Y 2 …Y k then begin pop X from the stack push Y k Y k-1 …Y 2 Y 1 onto the stack with Y 1 on top output the production X  Y 1 Y 2 …Y k end else error() Until X = $

12. – 12 – CSCE 531 Spring 2006 Parse Table for Expression Grammar id+-*/()$ E E  TE’ E’ E’  +TE’ E’  -TE’ E’  ε T T  FT’ T’ T’  ε T’  *FT’ T’  /FT’ T’  ε F F  id F  (E) Figure 4.15+

13. – 13 – CSCE 531 Spring 2006 Parse Trace of (z + q) * x + w * y StackInputOutput $E ( id + id ) * id + id * id $ $E’T E  T E’ $E’T’F ( id + id ) * id + id * id $ T  F T’ $E’T’)E( ( id + id ) * id + id * id $ F  ( E ) $E’T’)E id + id ) * id + id * id $ $E’T’)E’T E  T E’ $E’T’)E’T’F id + id ) * id + id * id $ T  F T’ $E’T’)E’T’id id + id ) * id + id * id $ F  id $E’T’)E’T’ + id ) * id + id * id $ $E’T’)E’ T’  ε

14. – 14 – CSCE 531 Spring 2006 First and Follow Functions We are going to develop two auxilliary functions for facilitating the computing of parse tables. FIRST(α) is the set of tokens that can start strings derivable from α, also if α  ε then we add ε to First(α). FOLLOW(N) is the set of tokens that can follow the nonterminal N in some sentential form, i.e., FOLLOW(N) = { t | S *  αNtβ }

15. – 15 – CSCE 531 Spring 2006 Algorithm to Compute First Input: Grammar symbol X Output: FIRST(X) Method  If X is a terminal, then FIRST(X) = {X}  If X  є is a production, then add є to FIRST(X).  For each production X  Y 1 Y 2 … Y k  If Y 1 Y 2 … Y i-1  є then add all tokens in FIRST(Y i ) to FIRST(X)  If Y 1 Y 2 … Y k  є then add є to FIRST(X)

16. – 16 – CSCE 531 Spring 2006 Example of First Calculation E  T E’ E’  + T E’ | - T E’ | є T  F T’ T’  * F T’ | / F T’ | є F  id | num | ( E ) FIRST(token) = {token} for tokens: + - * / ( ) id num FIRST(F) = { id, num, ( } FIRST(T’) = ? T’  є so … T’  *FT’ so … T’  /FT’ so … FIRST(T’) = {є … } FIRST(T) = FIRST(F) FIRST(E’) = ? FIRST(E) = ?

17. – 17 – CSCE 531 Spring 2006 Algorithm to Compute Follow (p 189) Input: nonterminal A Output: FOLLOW(A) Method  Add $ to FOLLOW(S), where $ is the end_of_input marker And S is the start state  If A  αBβ is a production, then every token in FIRST(β) is added to FOLLOW(B) (note not є)  If A  αB is a production or if A  αBβ is a production and β  є then every token in FOLLOW(A) is added to FOLLOW(B)

18. – 18 – CSCE 531 Spring 2006 Example of FOLLOW Calculation E  T E’ E’  + T E’ | - T E’ | є T  F T’ T’  * F T’ | / F T’ | є F  id | num | ( E )  Add $ to FOLLOW(E)  E  TE’ Add FIRST*(E’) to FOLLOW(T)  E’  + T E’ (similarly E’  +T E’) Add FIRST*(E’) to FOLLOW(T) E’  є, so FOLLOW(E’) is added to FOLLOW(T)  T  F T’ Add FIRST*(T’) to FOLLOW(F) T’  є, so FOLLOW(T’) is added to FOLLOW(F)  F  ( E ) Add FIRST( ‘)’ ) to FOLLOW(E) NFOLLOW(N) E { $ E’{ T { + - T’{ F{

19. – 19 – CSCE 531 Spring 2006 Construction of a Predictive Parse Table Algorithm 4.4 Input: Grammar G Output: Predictive Parsing Table M[N, a] Method  For each production A  α do  For each a in FIRST(α), add A  α to M[A, a]  If є is in FIRST(α), add A  α to M[A, b] for each token b in FOLLOW(A) If є is in FIRST(α) and $ is in FOLLOW(A) then add A  α to M[A, $]  Mark all other entries of M as “error”

20. – 20 – CSCE 531 Spring 2006 Predictive Parsing Example Example 4.18 in text table in Figure 4.15 (slide 11) Example 4.19 S  iEtSS’ | a S’  eS | є E  b Nonter- minals abeit$ S SaSaSaSa S  iEtSS’ S’ S’  eS S’  є E EbEbEbEb FIRST(S) = { i, a } FIRST(S’) = {є, e } FIRST(E) = { b } FOLLOW(S) = { $, e } FOLLOW(S’) = { $, e} FOLLOW(E) = { t

21. – 21 – CSCE 531 Spring 2006 LL(1) Grammars A grammar is called LL(1) if its parsing table has no multiply defined entries. LL(1) grammars Must not be ambiguous. Must not be ambiguous. Must not be left-recursive. Must not be left-recursive. G is LL(1) if and only if whenever A  α | β G is LL(1) if and only if whenever A  α | β  FIRST(α) ∩ FIRST(β) = Φ  At most one of α and β can derive є  If β *  є then FIRST(α) ∩ FOLLOW(A) = Φ

22. – 22 – CSCE 531 Spring 2006 Error Recovery in Predictive Parsing Panic Mode Error recovery If M[A, a] is an error, then throw away input tokens until one in a synchronizing set. If M[A, a] is an error, then throw away input tokens until one in a synchronizing set. Heuristics for the synchronizing sets for A Heuristics for the synchronizing sets for A  Add FOLLOW(A) to the synchronizing set for A  If ‘;’ is a separator or terminator of statements then keywords that can begin statements should not be in synchronizing set for the nonterminal “Expr” because a missing “;” would cause skipping keywords.  …

23. – 23 – CSCE 531 Spring 2006 Parse Table with Synch Entries Figure 4.18

24. – 24 – CSCE 531 Spring 2006 Trace with Error Recovery Figure 4.19

25. – 25 – CSCE 531 Spring 2006 Bottom up Parsing Idea – recognize right hand sides of productions so that we produce a rightmost derivation “Handle-pruning”

26. – 26 – CSCE 531 Spring 2006 Reductions in a Shift-Reduce Parser Figure 4.21 E  E + E | E * E | ( E ) | id Right-Sentential Form Handle Reducing Production id 1 + id 2 * id 3 id 1 E  id E + id 2 * id 3 id 2 E  id E + E * id 3 id 3 E  id How? E + E * E E * E E  E * E E + E E  E + E E

27. – 27 – CSCE 531 Spring 2006