1. November 2011Sentence Grammar1 CSA3202: HLT Sentence Grammar

2. November 2011Sentence Grammar2 Introduction This lecture has two aims: –Crash course in sentence-level grammar –Show how different linguistic phenomena can be captured by grammar rules. See also –Jurafsky and Martin Chapter 12 –Internet Grammar of English http://www.ucl.ac.uk/internet-grammar/

3. November 2011Sentence Grammar3 Part 1 English Grammar

4. November 2011Sentence Grammar4 Different Kinds of Linguistic Rules Morphological rules.. govern how words may be composed: re+invest+ing = reinvesting. Grammatical (syntax) rules.. govern how words and constituents combine to form grammatical sentences. Semantic rules.. govern how meanings may be combined.

5. November 2011Sentence Grammar5 Syntax: What? Part of speech info Syntactic structure Grammatical relations, e.g. –X is the subject –Y is the object –Z is the main verb of sentence S

6. Syntax: Why? The grammatical relations provide strong clues to the computation of meaning. For example, in a simple active sentence –The verb corresponds to a logical predicate –The subject to the first argument –The object to the second argument

7. Grammatical Relations and Logic Representation John kicked Fido kicked(John, Fido) subject main verb object

8. Ideal Language - Logic Mapping subject verb comp1 comp2,..., compN predicate(arg0, arg1,arg2,..., argN) sentence logic

9. November 2011Sentence Grammar9 Syntax: Why? You need knowledge of syntax in many applications: Full versus superficial analysis?

10. November 2011Sentence Grammar10 Syntax: Why? Knowledge of syntax important e.g. for –Spellchecking: POS helps detect error come over hear –Computing meaning for QA how many pretty girls and boys in the room? –Information extraction a record breaking win for Arsenal in 2009 –Translation I miss you/tu me manques (you miss me) What depth of syntactic analysis is needed?

11. Depth of Analysis Full –Syntax tree of entire sentence –Typically deep trees –More appropriate for fine-grained QA Superficial –Good for spotting NP chunks or Verb clusters –Relations e.g. coreference (Lawrence Gonzi, Larry) –Shallow trees –More appropriate for text classification November 2011Sentence Grammar11

12. November 2011Sentence Grammar12 Levels of Grammar Organisation There are roughly three levels –Word Classes: different parts of speech (POS). –Phrase Classes: sequences of words inheriting the characteristics of certain word classes. –Clause Classes: sequences of phrases containing at least one verb phrase. On the basis of these one may define: –Grammatical Relations: role played by constitutents e.g. subject; verb; object –Syntax-Semantics interface

13. November 2011Sentence Grammar13 Word Classes Closed classes. –determiners : the, a, an, four. –pronouns : it, he etc. –prepositions : by, on, with. –conjunctions : and, or, but. Open classes. –nouns refer to objects or concepts: cat, beauty, Coke. –adjectives describe or qualify nouns: fried chickens. –verbs describe what the noun does: John jumps. –adverbs describe how it is done: John runs quickly.

14. November 2011Sentence Grammar14 Word Class Characteristics Different word classes have characteristic subclasses and properties SubclassesProperties Nounproper; mass; count number; gender Verbtransitive; intransitive Number; gender; person, tense Adjectivedimension; age; colour Number, gender

15. November 2011Sentence Grammar15 Phrases Longer phrases may be used rather than a single word, but fulfilling the same role in a sentence. –Noun phrases refer to objects: four fried chickens. –Verb phrases state what the noun phrase does: kicks the dog. –Adjective phrases describe/qualify an object: sickly sweet. –Adverbial phrases describe how it is done: very carefully. –Prepositional phrases: add information to a verb phrase: on the table

16. November 2011Sentence Grammar16 Phrases can become Complex e.g. Noun Phrases Proper Name or Pronoun: Monday; it Specifiers, noun: the day Specifier, premodifier, noun: the first wet day Specifiers, qualifiers, noun, postmodifier: The first wet day that I enjoyed in June

17. November 2011Sentence Grammar17 But they all fit the same context Monday It The day The first wet day The first wet day that I enjoyed in June was sunny.

18. November 2011Sentence Grammar18 Clauses A clause is a combination of noun phrases and verb phrases Clauses can exist at the top level (main clause) or can be embedded (subordinate clause) –Top level clause is a sentence. E.g. The cat ate the mouse. –Embedded clause is subordinate e.g. John said that Sandy is sick. Unlike phrases, whole sentences can be used to say something complete, e.g. to state a fact or ask a question.

19. November 2011Sentence Grammar19 Different Kinds of Sentence Assertion: John ate the cat. Yes/No question: Did John eat the cat? Wh- question: What did John eat? Command: Eat the cat John!

20. November 2011Sentence Grammar20 Part II Context Free Grammar Rules

21. November 2011Sentence Grammar21 Formal Grammar A formal grammar consists of –Terminal Symbols (T) –Non Terminal Symbols (NT, disjoint from TS) –Start Symbol (a distinguished NT) –Rewrite rules of the form , where  and  are strings of symbols Different classes of grammar result from various restrictions on the form of rules

22. November 2011Sentence Grammar22 Classes of Grammar TypeGrammarsLanguagesMachines 0Phrase Structure Unrestricted Recursively Enumerable TM 1Context Sensitive LBA 2Context Free PDA 3Regular FSA of grammar typeincreasing strength

23. English is not a Regular Language Central embedding: –The bird sang –The bird the cat ate sang –the bird the cat the dog chased ate sang –the bird the cat the dog the man walked chased ate sang. These sentences are of the form a n b n This language cannot be regular

24. November 2011Sentence Grammar24 Restrictions on Rules For all rules  Type 0 (unrestricted): no restrictions Type 1 (context sensitive): |  |  |  | Type 2 (context free): –  is a single NT symbol Type 3 (regular) –Every rule is of the form A  aB or A  a where A,B  NT and a  T

25. November 2011Sentence Grammar25 Example Grammar Cabinet discusses police chief ’ s case French gunman kills four s  np vp np  n np  adj n np  n np vp  v np

26. November 2011Sentence Grammar26 Types of Grammar Symbol NT – symbols appearing on the left S  NT– symbol appearing only on the left from which every other symbol can be derived. T – symbols appearing only on the right To include words we also need a lexicon - special rule that rewrite T symbols e,g. n  [police] n  [gunman]

27. November 2011Sentence Grammar27 Grammar induces Phrase Structure French gunman kills four adj n v n np np vp s

28. November 2011Sentence Grammar28 Phrase Structure PS includes information about –Hierarchy (vertical) –Ordering (horizontal) PS constitutes a trace of the rule applications used to derive a sentence but NB. PS does not tell you the order in which the rules were actually used

29. November 2011Sentence Grammar29 Handling Sentence Types with Rules Declaratives John left. S → NP VP Imperatives Leave! S →VP Yes-No Questions Did John leave? S →Aux NP VP WH Questions When did John leave? S →Wh-word Aux NP VP

30. November 2011Sentence Grammar30 Handling Recursive Structures Flights to Miami Flights to Miami from Boston Flights to Miami from Boston in April Flights to Miami from Boston in April on Friday Flights to Miami from Boston in April on Friday under $300. Flights to Miami from Boston in April on Friday under $300 with lunch.

31. November 2011Sentence Grammar31 Recursive Rules NP → NP PP PP → Preposition NP NP → n

32. November 2011Sentence Grammar32 Subcategorisation: verb types Intransitive verb: no object John disappeared John disappeared the cat* Transitive verb: one object John opened the window John opened* Ditransitive verb: two objects John gave Mary the book John promised Mary that he would come John gave Mary*

33. November 2011Sentence Grammar33 Subcategorisation Rules Intransitive verb: no object VP → V Transitive verb: one object VP → V NP Ditransitive verb: two objects VP → V NP NP If you take account of the category of items following the verb, you end up with about 40 VP rules.

34. November 2011Sentence Grammar34 Subcategorisation Rules Intransitive verb: no object VP → V Transitive verb: one object VP → V X Ditransitive verb: two objects VP → V X Y X,Y  {NP, PP, S}

35. November 2011Sentence Grammar35 Which Language Class for NLP? Type 3 (Regular). Good for morphology. Cannot handle central embedding of sentences. The man that John saw eating died. Type 2 (Context Free). OK but problems handling certain phenomena e.g. agreement, subcategorisation. Type 1 (Context Sensitive). Computational properties not well understood. Too powerful. Type 0 (Turing). Too powerful.

36. November 2011Sentence Grammar36 Handling Agreement NP → Determiner N Include these days, this day Exclude this days, these day NP → NPSing NP → NPPlur NPPlur → DetSing NSing NPPlur → DetPlur NPlur Agreement also includes number, gender, case. Problem: proliferation of categories/rules.

37. Feb 2010 -- MRCLINT - Lecture 137 Generative Power of a Grammar G G GL L L undergeneration only but not all overgeneration all but not only all and only

38. November 2011Sentence Grammar38 Robustess versus Overgeneration In the design of a language processor it is necessary to keep in mind two incompatible features –maintain robustness –avoid overgeneration Robustness is crucial for analysis and understanding: the system must sometimes be lenient wrt possible input. For analysis, –robustness can be achieved by having an overgenerating (i.e. underspecified) grammar. –But this comes at the price of more ambiguity For generation, overgeneration means many spurious sentences will be generated

39. November 2011Sentence Grammar39 Undergeneration A grammar should generate all sentences in the language. There should not be sentences in the language that are not generated by the grammar. s  n vp vp  v n  [John] n  [gold] v  [found]

40. November 2011Sentence Grammar40 Precision versus Undergeneration Analysis –It is possible to achieve high precision by limiting the class of acceptable input to manageable cases. –But there is a risk that some input will be ruled out Generation

41. November 2011Sentence Grammar41 Appropriate Stuctures –A grammar should assign linguistically plausible structures. n v d a n John ate a juicy hamburger vp s n v d a n John ate a juicy hamburger np vp s np vs.

42. November 2011Sentence Grammar42 Ambiguity np  np pp pp  prep np (the man) (on the hill with a telescope by the sea) (the man on the hill) (with a telescope by the sea) (the man on the hill with a telescope)( by the sea) etc.

43. November 2011Sentence Grammar43 Criteria for Evaluating Grammars Does it undergenerate? Does this matter? Does it overgenerate? Does this matter? Does it assign appropriate/useful structures to sentences it generates? Is it simple to understand? How many rules are there? Does it contain generalisations or is it just a collection of special cases? How ambiguous is it? How is ambiguity dealt with?

44. November 2011Sentence Grammar44 Summary Regular grammars insufficien for sentence structure. CFG is good for handling constituent structure and embedding but there are shortcomings in handling certain phenomena: –agreement –subcategorisation –coordination Criteria for evaluating grammars are complex

46. Summary Context free grammar is good for modelling constituent structure for –Different sentence types (declarative, imperative, question etc) –Different phrase types –Verb subcategorisation But agreement (subject/verb) presents certain problems which can only be resolved within CFG by inventing more rules. November 2011Sentence Grammar46