1. CMSC 723: Intro to Computational Linguistics November 24, 2004 Lecture 12: Lexical Semantics Bonnie Dorr Christof Monz

2. Meaning So far, we have focused on the structure of language, not on what things mean We have seen that words have different meaning, depending on the context in which they are used Every day language tasks that require some semantic processing: Answering an essay question on an exam Deciding what to order at a restaurant by reading a menu Realizing you’ve been insulted

3. Meaning (continued) meaning representations are representations that link linguistic forms to knowledge of the world We are going to cover: What is the meaning of a word How can we represent the meaning What formalisms can be used Meaning representation languages

4. What Can Serve as a Meaning Representation? Anything that serves the core practical purposes of a program that is doing semantic processing What is a Meaning Representation Language? What is Semantic Analysis?

5. Requirements for Meaning Representation Verifiability Unambiguous Representation Canonical Form Inference Expressiveness

6. Verifiability System can match input representation against representations in knowledge base. If it finds a match, it can return Yes; Otherwise No. Does Maharani serve vegetarian food? Serves(Maharani,vegetarian food)

7. Unambiguous Representation Single linguistic input can have different meaning representations Each representation unambiguously characterizes one meaning. Example: small cars and motorcycles are allowed car(x) & small(x) & motorcycle(y) & small(y) & allowed(x) & allowed(y) car(x) & small(x) & motorcycle(y) & allowed(x) & allowed(y)

8. Ambiguity and Vagueness An expression is ambiguous if, in a given context, it can be disambiguated to have a specific meaning, from a number of discrete, possible meanings. E.g., bank (financial institution) vs bank (river bank) An expression is vague, if it refers to a range of a scalar variable, such that, even in a specific context, it’s hard to specify the range entirely. E.g., he’s tall, it’s warm, etc.

9. Representing Similar Concepts Distinct inputs could have the same meaning Does Maharani have vegetarian dishes? Do they have vegetarian food at Maharani? Are vegetarian dishes served at Maharani? Does Maharani serve vegetarian fare? Alternatives Four different semantic representations Store all possible meaning representations in KB

10. Canonical Form Solution: Inputs that mean same thing have same meaning representation Is this easy? No! Vegetarian dishes, vegetarian food, vegetarian fare Have, serve What to do?

11. How to Produce a Canonical Form Systematic Meaning Representations can be derived from thesaurus food ___ dish ___|____one overlapping meaning sense fare ___| We can systematically relate syntactic constructions [S [NP Maharani] serves [NP vegetarian dishes]] [S [NP vegetarian dishes] are served at [NP Maharani]]

12. Inference Consider a more complex request Can vegetarians eat at Maharani? Vs: Does Maharani serve vegetarian food? Why do these result in the same answer? Inference: Draw conclusions about truth of propositions not explicitly stored in KB serve(Maharani,VegetarianFood) => CanEat(Vegetarians,AtMaharani)

13. Non-Yes/No Questions Example: I'd like to find a restaurant where I can get vegetarian food. serve(x,VegetarianFood) Matching succeeds only if variable x can be replaced by known object in KB.

14. Meaning Structure of Language Human Languages Display a basic predicate-argument structure Make use of variables Make use of quantifiers Display a partially compositional semantics

15. Compositionality The compositionality principle is an important principle in formal semantics: The meaning of an expression is a strict function of the meanings of its parts It allows to build meaning representations incrementally Standard predicate logic does not adhere to this principle (donkey sentences)

16. Predicate-Argument Structure Represent concepts and relationships among them Some words act like arguments and some words act like predicates: Nouns as concepts or arguments: red(ball) Adj, Adv, Verbs as predicates: red(ball) Subcategorization (argument) frames specify number, position, and syntactic category of arguments Examples: NP give NP2 NP1 NP give NP1 to NP2 give(x,y,z)

17. Semantic (thematic) Roles Semantic Roles: Participants in an event Agent: George hit Bill. Bill was hit by George Patient: George hit Bill. Bill was hit by George Semantic (Selectional) Restrictions: Constrain the types of arguments verbs take George assassinated the senator *The spider assassinated the fly Verb subcategorization: Allows linking arguments in surface structure with their semantic roles Prepositions are like verbs Under(ItalianRestaurant,$15)

18. First Order Predicate Calculus (FOPC) FOPC provides sound computational basis for verifiability, inference, expressiveness Supports determination of truth Supports compositionality of meaning Supports question-answering (via variables) Supports inference

19. FOPC Syntax Terms Constants: Maharani Functions: LocationOf(Maharani) Variables: x in LocationOf(x) Predicates: Relations that hold among objects Serves(Maharani,VegetarianFood) Logical Connectives: Permit compositionality of meaning I only have $5 and I don’t have a lot of time Have(I,$5)  Have(I,LotofTime)

20. FOPC Semantics Sentences in FOPC can be assigned truth values True or False

21. Variables and Quantifiers Existential (  ): There exists A restaurant that serves Mexican food near UMD (  x) Restaurant(x) Serves(x,MexicalFood) Near(LocationOf(x),LocationOf(UMD)) Universal (  ): For all All vegetarian restaurants serve vegetarian food (  x) VegetarianRestaurant(x) -> Serves(x,VegetarianFood)

22. FOPC Examples John gave Mary a book Previously: Give(John,Mary,book) Better: (  x) Giving(x) Giver(John,x)  Givee(Mary,x) Given(book,x) Full Definition of Give: (  w,x,y,z) Giving(x)  Giver(w,x)  Givee(z,x)  Given(y,x)

23. Why use Variables? Multiple sentences containing “eat” I ate. I ate a turkey sandwich. I ate a turkey sandwich at my desk. I ate at my desk. I ate lunch. I ate a turkey sandwich for lunch I ate a turkey sandwich for lunch at my desk. Seven different Representations: Eating 1 (Speaker) Eating 2 (Speaker,TurkeySandwich) Eating 3 (Speaker,TurkeySandwich,Desk) Eating 4 (Speaker,Desk) Eating 5 (Speaker,Lunch) Eating 6 (Speaker,TurkeySandwich,Lunch) Eating 7 (Speaker,TurkeySandwich,Lunch,Desk)

24. Solution with Variables Eating(v,w,x,y) Examples revisited: (  w,x,y) Eating(Speaker,w,x,y) (  x,y) Eating(Speaker,TurkeySandwich,x,y) (  x) Eating(Speaker,TurkeySandwich,x,Desk) (  w,x) Eating(Speaker,w,x,Desk) (  w,y) Eating(Speaker,w,Lunch,y) (  y) Eating(Speaker,TurkeySandwich,Lunch,y) Eating(Speaker,TurkeySandwich,Lunch,Desk)

25. Representing Time Events are associated with points or intervals in time. We can impose an ordering on distinct events using notion of precedes. Temporal logic notation: (  w,x,t) Arrive(w,x,t) Constraints on variable t I arrived in New York (  t) Arrive(I,NewYork,t)  precedes(t,Now)

26. Interval Events Need t start and t end She was driving to New York until now (  t start,t end ) Drive(She,NewYork)  precedes(t start,Now)  Equals(t end,Now)

27. Relation Between Tenses and Time Relation between simple verb tenses and points in time is not straightforward Present tense used like future: We fly from Baltimore to Boston at 10 Complex tenses: Flight 1902 arrived late Flight 1902 had arrived late

28. Reference Point Reichenbach (1947) introduced notion of Reference point (R), separated out from Speech time (S) and Event time (E) Example: When Mary's flight departed, I ate lunch When Mary's flight departed, I had eaten lunch Departure event specifies reference point.

29. Reichenbach Applied to Tenses S S R,S S,R,ES,R S We refer to the S,R,E notation as a Basic Tense Structure (BTS)

30. Logical Inference The main motivation for using logic as a meaning representation is that it allows for sound and complete inference methods In propositional logic, a proposition P containing the propositional variable Q 1,…,Q n is valid, if P is true for all truth values of Q 1,…,Q n

31. Logical Inference Assume we have a number of sentences S 1,…,S n and their respective logical representations P 1,…,P n, and we want to determine whether some Q follows from them We check whether P 1 … P n -> Q is logically valid

32. Theorem Proving Considering all possible truth value instantiations is computationally infeasible: For n propositional variables, there are 2 n possible instantiations Finding computationally feasible ways to test for validity is the task of the research field of theorem proving (or automated reasoning)

33. Definitions What is the lexicon? A list of lexemes What is a lexeme? Word Orthography + Word Phonology + Word Sense What is the word sense? What is a dictionary? What is a computational lexicon?

34. Lexical Relations I: Homonomy What is homonomy? A bank holds investments in a custodial account Agriculture is burgeoning on the east bank Variants homophones: “read” vs. “red” homographs: “bass” vs. “bass”

35. Lexical Relations II: Polysemy What is polysemy? The bank is constructed from red brick I withdrew the money from the bank Distinguishing polysemy from homonymy is not straightforward

36. Word Sense Disambiguation For any given lexeme, can its senses be reliably distinguished? Assumes a fixed set of senses for each lexical item

37. Lexical Relations IV: Synonymy What is synonymy? How big is that plane? How large is that plane? Very hard to find true synonyms A big fat apple ?A large fat apple Influences on substitutability subtle shades of meaning differences polysemy register collocational constraints

38. Lexical Relations V: Hyponymy What is hyponymy? Not symmetric Example: car is a hyponym of vehicle and vehicle is a hypernym of car Test: That is a car implies That is a vehicle What is an ontology? Ex: CAR#1 is an object of type car What is a taxonomy? Ex: car is a kind of vehicle. CAR#1 is an object of type car What is an object hierarchy?

39. WordNet Most widely used hierarchically organized lexical database for English (Fellbaum, 1998) Demo: http://www.cogsci.princeton.edu/~wn/http://www.cogsci.princeton.edu/~wn/

40. Format of WordNet Entries

41. Distribution of Senses among WordNet Verbs

42. Lexical Relations in WordNet

43. Synsets in WordNet Example: {chump, fish, fool, gull, mark, patsy, fall guy, sucker, schlemiel, shlemiel, soft touch, mug} Definition: “a person who is gullible and easy to take advantage of”. Important: This exact synset makes up one sense for each of the entries listed in the synset. Theoretically, each synset can be viewed as a concept in a taxonomy Compare to: (  w,x,y,z) Giving(x) ^ Giver(w,x) ^ Givee(z,x) ^ Given(y,x). WN represents “give” as 45 senses, one of which is the synset {supply, provide, render, furnish}.

44. Hyponomy in WordNet

45. Automated Word Sense Disambiguation One of the main applications of WordNet is word-sense disambiguation. Supervised WSD: A training corpus is manually annotated with WordNet synsets. Foreach phrase-synset pair a list of words occurring in the context is stored. New phrases are classified according to the closet context vector

46. Automated Word Sense Disambiguation Unsupervised WSD: Given two phrases, consider all possible synsets. Select the two synsets that are closest in the WordNet hierarchy. Distance can be defined as: Number of edges (possibly weighted) Word overlap of the glosses

47. Selectional Preferences Verbs often exhibit type preferences for their arguments: Eat (OBJ: food) Think (SUBJ: intelligent entity) Analyzing a corpus with verb-argument pairs, it’s possible to derive the proper semantic types by looking at the hypernyms of the arguments

48. Readings J&M Chapter 17