Chapter 5: Languages and Grammar 1 Compiler Designs and Constructions ( Page ) Chapter 5: Languages and Grammar Objectives: Definition of Languages Types of Languages Dr. Mohsen Chitsaz

Chapter 5: Languages and Grammar2 Languages  Def: Set of words Set of Strings Elements of a language  Alphabet  Word (Token) (Vocabulary)  Grammar  Sentence  Semantic

Chapter 5: Languages and Grammar3  Natural Languages Example: They run a store  Formal Languages Example: If (Total > Max) Types of Languages

Chapter 5: Languages and Grammar4  Def: G={ , R, P, S} G={V t, V n, P, S} V t  V n =  Grammars

Chapter 5: Languages and Grammar5 Example SimpleDatatype = { Integer, Real, Char, Boolean } Datatype  SimpleDatatype SimpleDatatype  Integer SimpleDatatype  Real SimpleDatatype  Boolean SimpleDatatype  Char

Chapter 5: Languages and Grammar6 Example  V t : Terminal Symbols ( word, vocabulary, token } Integer, Char, Real, Boolean  V n : Datatype, SimpleDatatype  P: Datatype  SimpleDatatype SimpleDatatype  Integer SimpleDatatype  Real SimpleDatatype  Boolean SimpleDatatype  Char  S: Datatype

Chapter 5: Languages and Grammar7 Types of Grammars (Chomsky Hierarchy)      LHS RHS  Type Zero: Unrestricted ABC  a a  CaaF  Type One: Context Sensitive |  | <= |B| A -- > a A  Bc AB  aB Ca  abc

Chapter 5: Languages and Grammar8 Types of Grammars (Chomsky Hierarchy)  Type Two: Context Free |  | = 1   Vn  Type Three: Regular Grammar is a CFG with Right linear Right Linear A  w b A  w Left Linear A  B w A  w

Chapter 5: Languages and Grammar9 Context Free Grammar  Context Free Grammar G  Context Free Language of Grammar L(G)  Example:  var  id: ;  integer  real  char  Boolean

Chapter 5: Languages and Grammar10 Example  S -->oS1  S  o1  V n ={S}  V t ={0,1}  S={S}  P={S  oS1 S  o1}

Chapter 5: Languages and Grammar11 Language  Def: Language of a grammar: is a set of all sentences accepted by that grammar. L(G) = {w|S ->w}  Notation: Kleene Closure(*) Zero or more Positive Closure(+) One or more String Concatenation (.) Union (U)

Chapter 5: Languages and Grammar12 Language  Example: L(Digit) = {0,1,2,3,4,5,6,7,8,9} L(Alpha) = {a,b, …, z} L(Digit)+ = 2, 24 L(Alpha)* =a,, ab, abc L(Alpha) U (Digit) =a2, b6  Identifier: L(Alpha). ( L(Alpha) U L(Digit) )*

Chapter 5: Languages and Grammar13 Derivation  S  OS1  00S11   Each of the forms is called Sentential Form of this CFG L(G) = {On 1n; n >= 1}

Chapter 5: Languages and Grammar14 Example Write a grammar that produces a set of 0 ’ s & 1 ’ s in any order. It must start with a zero and end with a one

Chapter 5: Languages and Grammar15 Example  S  0A1  S  01  A  0A  A  1A  A  1  A  0  S  0A1  S  01  A  BA  B  0  B  1 S  A A  01 A  oB1 B  oB B  1B B  0 B  1

Chapter 5: Languages and Grammar16 Example  What is the language of this grammar? S  cS S  bD S  c D  cD D  bS D  b

Chapter 5: Languages and Grammar17 Example Write a grammar which produces odd numbers of * ’ s  L(G) = { *n; n>=1; n MOD 2 <>0}

Chapter 5: Languages and Grammar18 Example S  (S) S  () S  b  Write a grammar for the language L(G) = {b m C n d n e m f p (gh*) p | m>=2, n>=0, p>=1}  Write a grammar for the language L(G) = {b m C n | 1<= n <= m <= 2n}

Chapter 5: Languages and Grammar19 Tree Definition:  Node  Edge  Root  Children  Leaf(Terminal Node) Level Height (Depth) Internal Node External Node Preorder Traversal Inorder Traversal Postorder Traversal Implementation of Tree?

Chapter 5: Languages and Grammar20 Context Free Grammar  Derivation: How an input sentence can be recognized.  Parsing: Process of finding derivation  Parser: Automation of parsing. - Left Derivation (Leftmost derivation) - Right Derivation (Rightmost derivation)

Chapter 5: Languages and Grammar21 Example  Derive String of Left Derivation : Right Derivation: 1 S ----> 0AB 2 A ----> 1A 3 B ----> 0S 4 S ----> 0S 5 S ----> 1 6 A ----> 0

Chapter 5: Languages and Grammar22 Example  ---->  ----> +  ---->  ----> Id  Sentence id + id

Chapter 5: Languages and Grammar23  Derivation tree:  Leftmost Derivation:  Rightmost Derivation:

Chapter 5: Languages and Grammar24 Ambiguous Grammar Example  ---->  ----> Id  ----> +  ----> *  Sentence a+b*c Is this ambiguous?  Definition

Chapter 5: Languages and Grammar25 Finite State Machine (Automata) FSM (FSA)  Simplified model for digital system Memory Input Output  FSM is used as a language recognizer  Example: Real Number

Chapter 5: Languages and Grammar26 BNF (Bachus-Naus Form) + -

Chapter 5: Languages and Grammar27 FSM q1

Chapter 5: Languages and Grammar28 Grammar  > +  > -  > d  > d  > d  >.  > d  >

Chapter 5: Languages and Grammar29 Language Recognizer Head State = S Memory INPUT

Chapter 5: Languages and Grammar30 Language  Lex: Translate regular expression into lexical analyzer program  Grammar: Write a grammar to recognize the traffic lights

Chapter 5: Languages and Grammar31 Traffic Light  D ----> g D D ----> r S S ----> g D S ----> r S S ----> Language? gggrrggrrg …….

Chapter 5: Languages and Grammar32 Push Down Machine  CFG is accepted by a FSM controlling a Push-down stack Formal Definition:

Chapter 5: Languages and Grammar33 Deterministic Finite State Machine (Q, , ,q0, F)  (Every move is absolutely determined by the current state and next input)  Where:  Q: Finite Set of State  : Alphabet  q0: Starting State (q) F: Final States (Q)   : State Transition Function

Chapter 5: Languages and Grammar34 Deterministic Finite State Machine Example of a Real Number  Q = {S, q1, q2, q3, q4} q0 = {S} F = {q4}  = {+, -, d,. }

Chapter 5: Languages and Grammar35  = State Transition:  (S,+) = q1  (S,-) = q1  (S, d) = q2  (q1, d) = q2  (q2, d) = q2  (q2,. ) = q3  (q3, d) = q4  (q4, d) = q4

Chapter 5: Languages and Grammar36 Transition Table input token DFSM: Deterministic Finite State Machine No state with the same outgoing label. No state has more than one transition with the same label. +-.d S q1 q2 q1 q2 q3q2 q3 q4 Acc STATESTATE