Presentation is loading. Please wait.

Presentation is loading. Please wait.

Deep Text Understanding with WordNet Christiane Fellbaum Princeton University and Berlin-Brandenburg Academy of Sciences.

Similar presentations

Presentation on theme: "Deep Text Understanding with WordNet Christiane Fellbaum Princeton University and Berlin-Brandenburg Academy of Sciences."— Presentation transcript:

1 Deep Text Understanding with WordNet Christiane Fellbaum Princeton University and Berlin-Brandenburg Academy of Sciences

2 WordNet •What is WordNet and why is it interesting/useful? •A bit of history •WordNet for natural language processing/word sense disambiguation

3 What is WordNet? •A large lexical database, or “electronic dictionary,” developed and maintained at Princeton University •Includes most English nouns, verbs, adjectives, adverbs •Electronic format makes it amenable to automatic manipulation •Used in many Natural Language Processing applications (information retrieval, text mining, question answering, machine translation, AI/reasoning,...)‏ •Wordnets are built for many languages (including Danish!)‏

4 What’s special about WordNet? •Traditional paper dictionaries are organized alphabetically: words that are found together (on the same page) are not related by meaning •WordNet is organized by meaning: words in close proximity are semantically similar •Human users and computers can browse WordNet and find words that are meaningfully related to their queries (somewhat like in a hyperdimensional thesaurus)‏ •Meaning similiarity can be measured and quantified to support Natural Language Understanding

5 A bit of history Research in Artificial Intelligence (AI): How do humans store and access knowledge about concept? Hypothesis: concepts are interconnected via meaningful relations Knowledge about concepts is huge--must be stored in an efficient and economic fashion

6 A bit of history Knowledge about concepts is computed “on the fly” via access to general concepts E.g., we know that “canaries fly” because “birds fly” and “canaries are a kind of bird”

7 A simple picture animal (animate, breathes, has heart,...)‏ | bird (has feathers, flies,..)‏ | canary (yellow, sings nicely,..)‏

8 Knowledge is stored at the highest possible node and inherited by lower (more specific) concepts rather than being multiply stored Collins & Quillian (1969) measured reaction times to statements involving knowledge distributed across different “levels”

9 Do birds fly? --short RT Do canaries fly? --longer RT Do canaries have a heart? --even longer RT

10 Collins’ & Quillian’s results are subject to criticism (reaction time to statements like “do canaries move?” are influenced by prototypicality, word frequency, uneven semantic distance across levels)‏ But other evidence from psychological experiments confirms that humans organize knowledge about words and concept by means of meaningful relations Access to one concepts activates related concepts in an outward spreading (radial) fashion

11 A bit of history But the idea inspired WordNet (1986), which asked: Can most/all of the lexicon be represented as a semantic network where words are interlinked by meaning? If so, the result would be a semantic network (a graph)‏

12 WordNet If the (English) lexicon can be represented as a semantic network, which are the relations that connect the nodes?

13 Whence the relations? •Inspection of association norms stimulus: hand reponse: finger, arm •Classical ontology (Aristotle): IS-A (maple-tree), HAS-A (maple-leaves)‏ •Co-occurrence patterns in texts (meaningfully related words are used together)‏

14 Relations:Synonymy One concept is expressed by several different word forms: {beat, hit, strike} {car, motorcar, auto, automobile} { big, large} Synonymy = one:many mapping of meaning and form

15 Synonymy in WordNet WordNet groups (roughly) synonymous, denotationally equivalent, words into unordered sets of synonyms (“synsets”)‏ {hit, beat, strike} {big, large} {queue, line} Each synset expresses a distinct meaning/concept

16 Polysemy One word form expresses multiple meanings Polysemy = one:many mapping of form and meaning {table, tabular_array} {table, piece_of_furniture} {table, mesa} {table, postpone} Note: the most frequent word forms are the most polysemous!

17 Polysemy in WordNet A word form that appears in n synsets is n-fold polysemous {table, tabular_array} {table, piece_of_furniture} {table, mesa} {table, postpone} table is fourfold polysemous/has four senses

18 Some WordNet stats 117,659155,287total 3,6214,481adverb 18,15621,479adjective 13,76711,529verb 82,115117,798noun Synsets containing wf Word formsPart of speech

19 The “Net” part of WordNet Synsets arethe building block of the network Synsets are interconnected via relations Bi-directional arcs express semantic relations Result: large semantic network (graph)‏

20 Hypo-/hypernymy relates noun synsets Relates more/less general concepts Creates hierarchies, or “trees” {vehicle} / \ {car, automobile} {bicycle, bike} / \ \ {convertible} {SUV} {mountain bike} “A car is is a kind of vehicle” “The class of vehicles includes cars, bikes” Hierarchies can have up to 16 levels

21 Hyponymy Transitivity: A car is a kind of vehicle An SUV is a kind of car => An SUV is a kind of vehicle

22 Meronymy/holonymy (part-whole relation)‏ {car, automobile} | {engine} / \ {spark plug} {cylinder} “An engine has spark plugs” “Spark plus and cylinders are parts of an engine”

23 Meronymy/Holonymy Inheritance: A finger is part of a hand A hand is part of an arm An arm is part of a body =>a finger is part of a body

24 Structure of WordNet (Nouns)‏

25 WordNet Data Model bank fiddle violin violist fiddler string rec: 12345 - financial institute rec: 54321 - side of a river rec: 9876 - small string instrument rec: 65438 - musician playing violin rec:42654 - musician rec:25876 - string instrument rec:35576 - string of instrument rec:29551 - subatomic particle type-of part-of Vocabulary of a language ConceptsRelations 1 2 2 1 1 2


27 WordNet for Natural Language Processing Challenge: get a computer to “understand” language •Information retrieval •Text mining •Document sorting •Machine translation

28 Natural Language Processing •Stemming, parsing currently at >90% accuracy level •Word sense discrimination (lexical disambiguation) still a major hurdle for successful NLP •Which sense is intended by the writer (relative to a dictionary)? •Best systems: ~60% precision, ~60% recall (but human inter-annotator agreement isn’t perfect, either!)‏

29 Understanding text beyond the word level (joint work with Peter Clark and Jerry Hobbs)‏

30 Knowledge in text Human language users routinely derive knowledge from text that is NOT expressed on the surface Perhaps more knowledge is unexpressed than overtly expressed on the surface Grasser (1981) estimates explicit:implicit info = 1:8

31 An example Text: A soldier was killed in a gun battle Inferences: Soldiers were fighting one another The soldiers had guns with live ammunition Multiple shots were fired One soldier shot another soldier The shot soldier died as a result of the injuries caused by the shot The time interval between the fatal shot and the death was short

32 Humans use world knowledge to supplement word knowledge (How) can such knowledge be encoded and harnessed by automatic systems? Previous attempts (e.g., Cyc’s microtheories) --too few theories --uneven coverage of world knowledge

33 Recognizing Textual Entailment Task: Evaluate truth of hypothesis H given a text T (T) A soldier was killed in a gun battle (H) A soldier died Answer may be yes/no/probably/...

34 RTE Many automatic system attempt RTE via lexical, syntactic matching algorithms (“do the same words occur in T, H?” “do T, H have the same subject/object?”)‏ Not “deep” language understanding

35 Our RTE test suite 250 Text-Hypothesis pairs for 50% of them, H is entailed by T for the remaining 50%, H is not (necessarily) entailed Focus on semantic interpretation

36 RTE test suite Core of T statements came from newspaper texts H statements were hand-coded focus on general world knowledge

37 RTE test suite Manually analyzed pairs Distinguished, classified 19 types of knowledge among the T-H pairs some partial overlap

38 Exx: Types of knowledge (increasing order of difficulty)‏ Lexical: relation among irregular forms of a single lemma, Named Entities vs. proper nouns Lexical-semantic (paradigmatic): synonyms, hypernyms, meronyms, antonyms, metonymy, derivations Syntagmatic: selectional preferences, telic roles Propositional: cause-effect, preconditions World knowledge/core theories (e.g., ambush entails concealment)‏

39 Overall approach (bag of tricks)‏ •Initial text interpretation with language processing tools (Peter Clark et al.)‏ •Compute subsumption among text fragments •WordNet augmentations

40 Text interpretation First step: parsing (assign a structure to a sentence or phrase)‏ SAPIR parser (Harrison & Maxwell 1986)‏ SAPIR also produces a Logical Form (LF)‏

41 LFs LF structures are trees generated by rules parallel to grammar rules contain logic elements nouns, verbs, adj’s, prepositions represented as variables LFs are parsed and have part-of-speech tags LFs generate ground logical assertions

42 Example LF for "A soldier was killed in a gun battle." (DECL ((VAR X1 "a" "soldier") (VAR X2 "a" "battle" (NN "gun" "battle"))) (S (PAST) NIL "kill" ?X1 (PP "in" ?X2)))‏

43 Logical assertions logic for "A soldier was killed in a gun battle." object(kill,soldier) & in(kill,battle) & modifier(battle,gun)‏

44 Result: T, H in Logical Form

45 Matching sentences/fragments with subsumption A basic reasoning operation A person loves a person subsumes A man loves a woman Set 1 of clauses subsumes another Set 2 of clauses if each clause in S 1 subsumes some member of S 2. Similary, a set of clauses subsumes another set of clauses if the arguments of the first subsume or match the arguments of the second Argument (word) subsumption as in WordNet (X is a Y)‏ Matching = synonyms

46 Syntactic matching of predicates --both are the same --one is predicate “of” or modifier (my friend’s car, the car of my friend)‏ --predicates “subject” and “by” match (passives)‏

47 Lexical (word) matching Words related by derivational morphology (destroy, destruction) are considered matches in conjunction with syntactic matches

48 Recognize as equivalent: the bomb destroyed the shrine the destruction of the shrine by the bomb But not the destruction of the bomb by the shrine a person attacks with a bomb there is a bomb attack by a person

49 Benefits for text understanding/RTE (T) Moore is a prolific writer (H) Moore writes many books Moore is the Agent of write

50 Exploiting word and world knowledge encoded in WordNet

51 Use of WordNet glosses Glosses = definition of “concept” expressed by synset members {airplane, plane (an aircraft that has fixed wings and is powered by propellers or jets)} syntagmatic information, world knowledge

52 Translating glossed into First Order Logic Axioms { bridge, span (any structure that allows people or vehicles to cross an obstacle such as a river or canal...)} bridge N1 (x,y)‏ structure N1 (x) & allow V1 (x,e 1 ) & cross V1 (e 1,z,y) & obstacle N2 (y) & person/vehicle(z)‏ person N1 (z) --> person/vehicle(z)‏ vehicle N1 (z) --> vehicle/person(z)‏ river N2 (y) --> obstacle N2 (y)‏ canal N3 (y) --> obstacle N2 (y)

53 The nouns, verbs, adjectives, adverbs in the LF glosses were manually disambiguated Thus, each variable in the LFs was identified not just with a word form, but a form-meaning pair (sense) in WordNet LFs were generated for 110K glosses Particular emphasis on CoreWordNet

54 How well do our tricks perform?

55 An example that works Exploiting formally related words in WN: (T) …go through licensing procedures (H) …go through licensing processes Exploiting hyponymy (IS-A relation): (T) Beverley served at WEDCOR (H) Beverley worked at WEDCOR

56 More complex example that works (T) Britain puts curbs on immigrant labor from Bulgaria (H) Britain restricted workers from Bulgaria

57 Knowledge from WordNet Synset with gloss: {restrict, restrain, place_limits_on, (place restrictions on)} Synonymy: {put, place}, {curb, limit} Morphosemantic link: {labor} - {laborer} Hyponymy: {laborer} ISA {worker}

58 Example that doesn’t work (T) The Philharmonic orchestra draws large crowds (H) Large crowds were drawn to listen to the orchestra WordNet tells us that orchestra = collection of musicians musician = someone who plays musical instrument music = sound produced by musical instruments listen = hear = perceive sound But WN doesn’t tell us that playing results in sound production and that there is a listener

59 Examples that don’t work The most fundamental knowledge that humans take for granted trips up automatic systems Such knowledge is not explicitly taught to children But it must be “taught” to machines!

60 Core theories (Jerry Hobbs) •Attempt to encode fundamental knowledge •Space, time, causality,... •Essential for reasoning •Not encoded in WordNet glosses

61 Core theories •Manually encoded •Axiomatized

62 Core theories •Composite entities (things made of other things, stuff)‏ •Scalar notions (time, space,...)‏ •Change of state •Causality

63 Core theories Example of predications: change(e 1,e 2 )‏ changeFrom(e 1 )‏ changeTo(e 2 )‏

64 Core theories and WordNet map core theories to Core WN synsets encode meanings of synsets denoting events, event structure in terms of core theory predications

65 Examples let(x,e) not(cause(x,not(e)))‏ {go, become, get (“he went wild”)} go(x,e) changeTo(e)‏ free(x,y) cause(x,changeTo(free(y)))‏ (All words are linked to WN senses)‏

66 Example The captors freed the hostages The hostages were free free = let(x, go(y, free(y)))‏ not(cause(x, not(changeTo(free(y)))‏ cause(x, changeTo(free(y)))‏ free(x,y)

67 Preliminary evaluation (What) does each component contribute to RTE? For the 250 Text-Hypothesis pairs in our test suite:

68 7297when H or ¬H is not predicted (assumed not to be entailed)‏ 14WordNet glosses in LF 114WordNet relations 311syntactic transformations IncorrectCorrectwhen H or ¬H is predicted by:

69 Conclusion •Way to go! •Deliberately exclude statistical similarity measures (this hurts our results) •Symbolic approach: aim at deep level understanding

70 WordNet for Deeper Text Understanding •Axioms in Logical Form are useful for many other NL Understanding applications •E.g., automated question answering: translate Qs and As into logic representation •Logic representations enable reasoning (axioms can be fed into a reasoner/logic prover)‏

71 Thanks for your attention

Download ppt "Deep Text Understanding with WordNet Christiane Fellbaum Princeton University and Berlin-Brandenburg Academy of Sciences."

Similar presentations

Ads by Google