LING 388 Language and Computers Lecture 6 9/18/03 Sandiway FONG

Administrivia Poll: 2 nd Call for votes Poll: 2 nd Call for votes  How many would want an extra (optional) introductory computer lab session?  Send to Charles

Administrivia Reminder Reminder  Next Tuesday  Computer Lab Class Meet in SBS 224Meet in SBS 224  Next Thursday  Computer Lab Class continues…  Homework 2 will be handed out

Review Last Lecture: Last Lecture:  Formal Grammars  production rules  X -> Y 1 …Y n n>=0 X  V N, Y i  (V N u V T ) 1<=i<=nX  V N, Y i  (V N u V T ) 1<=i<=n  rule application and sentential forms S =>* ,   (V N u V T )*S =>* ,   (V N u V T )* –=>* means “derives through zero or more rule applications”

Regular Grammars Regular Grammars are a class of formal grammars with a special restriction Regular Grammars are a class of formal grammars with a special restriction  aka Chomsky hierarchy type-3 grammars  Production rules are constrained to have format: X -> Y tX -> t (left recursive)X -> Y tX -> t (left recursive)  or X -> t YX -> t (right recursive)X -> t YX -> t (right recursive)  where X,Y  V N, t  V T i.e. all rules contain only a single non-terminal, and (possibly) a single terminal) on the right hand side  Note:  can’t have both left and right recursive rules at the same time

Regular Grammars (Remarkable) Property (Remarkable) Property  Regular grammars are formally equivalent to FSA (and regular expressions)

Regular Grammars Example: Example:  Grammar G ab  S -> aB  B -> aB  B -> bC  B -> b  C -> bC  C -> b  Note: G ab is right recursive  Starting non-terminal: S  V N = {S, B, C}(set of non-terminals)  V T = {a, b}(set of terminals)

Regular Grammars Question: Question:  What does grammar G ab generate? Answer: Answer:  Language: L(G ab ) = {a + b + } “one or more a s followed by one or more b s”

Regular Grammars Example of Derivation: Example of Derivation:  S  aB  aaB (using rule B -> aB)  … can iterate this to get aa…aB – i.e. any number of a s  From aaB, we can get:  aab(using rule B-> b) OR  aabC(using rule B-> bC)  aabbC(using rule C-> bC) … can iterate this to get aabb…bC – i.e. any number of b s… can iterate this to get aabb…bC – i.e. any number of b s  From aabbC, we can get: aabbbb(using rule C-> b)aabbbb(using rule C-> b)

Regular Grammars Consider a FSA equivalent to G ab that also describes L ab = {a + b + } Consider a FSA equivalent to G ab that also describes L ab = {a + b + }

Equivalent FSA SB C a a b b

Regular Grammars FSA Cases (right- recursive): Cases (right- recursive):  A -> a  A -> aB  A -> aA Note: Note:  A left-recursive grammar can be transformed to a weakly-equivalent right-recursive one a a A B A a A

Equivalent FSA Basically, we have that… Basically, we have that…  Each non-terminal corresponds to a state  Each terminal corresponds to a transition between two states  A recursive rule is implemented by a loop

Limitations Production Rule Limitation: Production Rule Limitation:  can’t have both left and right recursive rules at the same time  e.g. can’t have rules A -> aBA -> aB A -> BaA -> Ba in the same grammar  Question: Why?  Answer:  Would change the expressive power of the grammar formalism No longer preserve the equivalence with FSA and regular expressionsNo longer preserve the equivalence with FSA and regular expressions

Limitations Example: Example:  L = {a n b n | n>=0 } is not a regular language  However, the following grammar (with both left- recursive and right-recursive rules) will generate L :  A ->   A -> aB  B -> Ab  B -> b  Starting non-terminal: A

Limitations Sample Derivations : Sample Derivations :  A  aB  aAb  ab (using rule A -> )  or  aaBb aabb (using rule B -> b)aabb (using rule B -> b)  or aaAbbaaAbb aaaBbbaaaBbb aaabbb(using rule B -> b)aaabbb(using rule B -> b)

Definite Clauses Grammars (DCG) Introduction Introduction  DCGs are built-in to Prolog  Prolog was originally designed for natural language processing  Notation designed for a transparent encoding of grammar production rules in the database  Can handle general grammars  … not just regular grammars  Next Thursday’s Computer Lab

Definite Clause Grammars (DCG) Syntax : Syntax :  We have been writing production rules like  X -> Y t  In Prolog notation, the equivalent DCG rule looks something like  x --> y, [t]. Notes: Notes:  --> replaces ->  Terminal symbols are distinguished from non-terminal symbols by enclosing them in square brackets, e.g. [t]  All terminal and non-terminal symbols begin with a lowercase letter, i.e. are names  General case: can be structures  Commas separate symbols on the right hand side of the rule  A DCG rule ends with a period

Definite Clause Grammars (DCG) Example grammar: Example grammar:  Grammar G ab L(G ab ) = {a + b + }L(G ab ) = {a + b + }  Production Rules: S -> aBs --> [a],b.S -> aBs --> [a],b. B -> aBb --> [a],b.B -> aBb --> [a],b. B -> bCb --> [b],c.B -> bCb --> [b],c. B -> bb --> [b].B -> bb --> [b]. C -> bCc --> [b],c.C -> bCc --> [b],c. C -> bc --> [b].C -> bc --> [b].

Definite Clause Grammars (DCG) Querying the grammar: Querying the grammar:  Each DCG rule like x --> [a,b]. defines a clause for x/2  In x(X,Y)  X represents the input list to be parsed  Y represents the remainder of the input list after rule x has been parsed  In Prolog parlance, the pair X,Y is known as a difference list  Example queries:  ?- s([a,a,b,b,b],[]).  Yes.  ?- s([a,b,a],[]).  No.