ParsingParsing

2 Front-End: Parser  Checks the stream of words and their parts of speech for grammatical correctness scannerparser source code tokens IR errors

3 Front-End: Parser  Determines if the input is syntactically well formed scannerparser source code tokens IR errors

4 Front-End: Parser  Guides context-sensitive (“semantic”) analysis (type checking) scannerparser source code tokens IR errors

5 Front-End: Parser  Builds IR for source program scannerparser source code tokens IR errors

6 Syntactic Analysis  Natural language analogy: consider the sentence Hewrotetheprogram

7 Syntactic Analysis Hewrotetheprogram nounverbarticlenoun

8 Syntactic Analysis Hewrotetheprogram nounverbarticlenoun subjectpredicateobject

9 Syntactic Analysis  Natural language analogy Hewrotetheprogram nounverbarticlenoun subjectpredicateobject sentence

10 Syntactic Analysis  Programming language if( b <= 0 )a = b bool expr assignment if-statement

11 Syntactic Analysis syntax errors int* foo(int i, int j)) { for(k=0; i j; ) fi( i > j ) return j; }

Compiler Construction Sohail Aslam Lecture 11

13 Syntactic Analysis int* foo(int i, int j)) { for(k=0; i j; ) fi( i > j ) return j; } extra parenthesis Missing expression not a keyword

14 Semantic Analysis  Grammatically correct Hewrotethecomputer nounverbarticlenoun subjectpredicateobject sentence

15 Semantic Analysis  semantically (meaning) wrong! Hewrotethe computer nounverbarticlenoun subjectpredicateobject sentence

16 Semantic Analysis int* foo(int i, int j) { for(k=0; i < j; j++ ) if( i < j-2 ) sum = sum+i return sum; } undeclared var return type mismatch

17 Role of the Parser  Not all sequences of tokens are program.  Parser must distinguish between valid and invalid sequences of tokens.

18 Role of the Parser  Not all sequences of tokens are program.  Parser must distinguish between valid and invalid sequences of tokens.

19 Role of the Parser What we need  An expressive way to describe the syntax  An acceptor mechanism that determines if input token stream satisfies the syntax

20 Role of the Parser What we need  An expressive way to describe the syntax  An acceptor mechanism that determines if input token stream satisfies the syntax

21 Role of the Parser What we need  An expressive way to describe the syntax  An acceptor mechanism that determines if input token stream satisfies the syntax

22 Study of Parsing  Parsing is the process of discovering a derivation for some sentence

23 Study of Parsing  Mathematical model of syntax – a grammar G.  Algortihm for testing membership in L(G).

24 Study of Parsing  Mathematical model of syntax – a grammar G.  Algortihm for testing membership in L(G).

25 Context Free Grammars A CFG is a four tuple G=(S,N,T,P)  S is the start symbol  N is a set of non-terminals  T is a set of terminals  P is a set of productions

26 Why Not Regular Expressions? Reason: regular languages do not have enough power to express syntax of programming languages.

27 Limitations of Regular Languages  Finite automaton can’t remember number of times it has visited a particular state

28 Example of CFG  Context-free syntax is specified with a CFG

29 Example of CFG  Example SheepNoise → SheepNoise baa | baa  This CFG defines the set of noises sheep make

30 Example of CFG  We can use the SheepNoise grammar to create sentences  We use the productions as rewriting rules

31 Example of CFG SheepNoise → SheepNoise baa | baa RuleSentential Form -SheepNoise 2baa

32 Example of CFG SheepNoise → SheepNoise baa | baa RuleSentential Form - SheepNoise 1SheepNoise baa 2baa

33 Example of CFG And so on... RuleSentential Form - SheepNoise 1SheepNoise baa 1SheepNoise baa baa 2baa baa baa

34 Example of CFG  While it is cute, this example quickly runs out intellectual steam  To explore uses of CFGs, we need a more complex grammar

35 Example of CFG  While it is cute, this example quickly runs out intellectual steam  To explore uses of CFGs, we need a more complex grammar

36 More Useful Grammar 1expr → expr op expr 2 | num 3 | id 4op → + 5 | – 6 | * 7 | /

37 Backus-Naur Form (BNF)  Grammar rules in a similar form were first used in the description of the Algol60 Language.

38 Backus-Naur Form (BNF)  The notation was developed by John Backus and adapted by Peter Naur for the Algol60 report.  Thus the term Backus-Naur Form (BNF)

39 Backus-Naur Form (BNF)  The notation was developed by John Backus and adapted by Peter Naur for the Algol60 report.  Thus the term Backus-Naur Form (BNF)

40 Derivation:Derivation:  Let us use the expression grammar to derive the sentence x – 2 * y

41 Derivation: x – 2 * y RuleSentential Form - expr 1expr op expr 2 op expr 5 – expr 1 – expr op expr

42 Derivation: x – 2 * y RuleSentential Form 2 – op expr 6 –  expr 3 – 

43 DerivationDerivation  Such a process of rewrites is called a derivation.  Process or discovering a derivations is called parsing

44 DerivationDerivation  Such a process of rewrites is called a derivation.  Process or discovering a derivations is called parsing

45 DerivationDerivation We denote this derivation as: expr → * id – num * id

46 DerivationsDerivations  At each step, we choose a non-terminal to replace  Different choices can lead to different derivations.

47 DerivationsDerivations  At each step, we choose a non-terminal to replace  Different choices can lead to different derivations.

48 DerivationsDerivations  Two derivations are of interest 1.Leftmost derivation 2.Rightmost derivation

49 DerivationsDerivations  Leftmost derivation: replace leftmost non- terminal (NT) at each step  Rightmost derivation: replace rightmost NT at each step

50 DerivationsDerivations  Leftmost derivation: replace leftmost non- terminal (NT) at each step  Rightmost derivation: replace rightmost NT at each step

51 DerivationsDerivations  The example on the preceding slides was leftmost derivation  There is also a rightmost derivation

52 Rightmost Derivation RuleSentential Form - expr 1expr op expr 3expr op 6 expr  1 expr op expr 

53 Derivation: x – 2 * y RuleSentential Form 2 expr op  5 expr –  3 – 

54 DerivationsDerivations  In both cases we have expr → * id – num  id

55 DerivationsDerivations  The two derivations produce different parse trees.  The parse trees imply different evaluation orders!

56 DerivationsDerivations  The two derivations produce different parse trees.  The parse trees imply different evaluation orders!