Download presentation

Presentation is loading. Please wait.

Published byDianna Filkins Modified over 3 years ago

1
Natural Logic for Textual Inference Bill MacCartney and Christopher D. Manning NLP Group Stanford University 29 June 2007

2
2 Inferences involving monotonicity Few states completely forbid casino gambling. OK Few western states completely forbid casino gambling. Few or no states completely forbid casino gambling. Few states completely forbid gambling. No Few states completely forbid casino gambling for kids. Few states or cities completely forbid casino gambling. Few states restrict gambling. What kind of textual inference system could predict this? Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

3
3 Textual inference: a spectrum of approaches robust, but shallow deep, but brittle natural logic lexical/ semantic overlap Jijkoun & de Rijke 2005 patterned relation extraction Romano et al. 2006 pred-arg structure matching Hickl et al. 2006 FOL & theorem proving Bos & Markert 2006 Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

4
4 What is natural logic? A logic whose vehicle of inference is natural language No formal notation: Just words & phrases: All men are mortal … Focus on a ubiquitous category of inference: monotonicity I.e., reasoning about the consequences of broadening or narrowing the concepts or constraints in a proposition Precise, yet sidesteps difficulties of translating to FOL: idioms, intensionality and propositional attitudes, modalities, indexicals,reciprocals, scope ambiguities, quantifiers such as most, reciprocals, anaphoric adjectives, temporal and causal relations, aspect, unselective quantifiers, adverbs of quantification, donkey sentences, generic determiners, … Aristotle, Lakoff, van Benthem, Sánchez Valencia 1991 Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

5
5Outline Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

6
6 The entailment relation: In natural logic, entailment is defined as an ordering relation over expressions of all semantic types (not just sentences) categorysemantic typeexample(s) common nouns etet penguin bird adjectives etet tiny small intransitive verbs etet hover fly transitive verbs eeteet kick strike temporal & locative modifiers (e t) (e t) this morning today in Beijing in China connectives tttttt and or quantifiers (e t) t (e t) (e t) t everyone someone all most some Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

7
7 Monotonicity of semantic functions Upward-monotone ( M) The default: “bigger” inputs yield “bigger” outputs Example: broken. Since chair furniture, broken chair broken furniture Heuristic: in a M context, broadening edits preserve truth Downward-monotone ( M) Negatives, restrictives, etc.: “bigger” inputs yield “smaller” outputs Example: doesn’t. While hover fly, doesn’t fly doesn’t hover Heuristic: in a M context, narrowing edits preserve truth Non-monotone (#M) Superlatives, some quantifiers ( most, exactly n ): neither M nor M Example: most. While penguin bird, most penguins # most birds Heuristic: in a #M context, no edits preserve truth In compositional semantics, meanings are seen as functions, and can have various monotonicity properties: Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

8
8 Downward monotonicity few athletes few sprinters restrictive quantifiers: no, few, at most n prohibit weapons prohibit guns negative & restrictive verbs: lack, fail, prohibit, deny without clothes without pants prepositions & adverbs: without, except, only drug ban heroin ban negative & restrictive nouns: ban, absence [of], refusal If stocks rise, we’ll get real paid If stocks soar, we’ll get real paid the antecedent of a conditional didn’t dance didn’t tango explicit negation: no, n’t Downward-monotone constructions are widespread! Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

9
9 Monotonicity of binary functions Some quantifiers are best viewed as binary functions Different arguments can have different monotonicities all All ducks fly All mallards fly All ducks move some Some mammals fly Some animals fly Some mammals move no No dogs fly No poodles fly No dogs hover not every Not every bird flies Not every animal flies Not every bird hovers Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

10
10 Composition of monotonicity Composition of functions composition of monotonicity Sánchez Valencia: a precise monotonicity calculus for CG Few forbid states completely casino gambling +++––– Few states completely forbid casino gambling o MM MM #M MM MM MM MM MM MM Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

11
11 The NatLog System linguistic pre-processing alignment entailment classification 1 2 3 textual inference problem prediction Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

12
12 Step 1: Linguistic Pre-processing Tokenize & parse input sentences (future: & NER & coref & …) Identify & project monotonicity operators Problem: PTB-style parse tree semantic structure! Few states completely forbid casino gambling JJ NNS RB VBD NN NN NP ADVP NP VP S +++––– Solution: specify projections in PTB trees using Tregex Few forbid states completely casino gambling few pattern: JJ < /^[Ff]ew$/ arg1: M on dominating NP __ >+(NP) (NP=proj !> NP) arg2: M on dominating S __ >+(/.*/) (S=proj !> S) Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

13
13 Step 2: Alignment Alignment = a sequence of atomic edits [cf. Harmeling 07] Atomic edits over token spans: DEL, INS, SUB, ADV Limitations: no easy way to represent movement no alignments to non-contiguous sets of tokens Benefits: well-defined sequence of intermediate forms can use adaptation of Levenshtein string-edit DP We haven’t (yet) invested much effort here Few states completely forbid casino gambling Few states have completely prohibited gambling ADV SUBADVINSDEL Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

14
14 Step 3: Entailment Classification Atomic edits atomic entailment problems Feature representation Basic features: edit type, monotonicity, “light edit” feature Lexical features for SUB edits: lemma sim, WN features Decision tree classifier Trained on small data set designed to exercise feature space Outputs an elementary entailment relation: = # | Composition of atomic entailment predictions Fairly intuitive: º , º #, º = =, etc. Composition yields global entailment prediction for problem Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

15
15 predict featurize Entailment model example typeINS monodown isLighttrue Few states completely forbid casino gambling. Few states have completely prohibited gambling. SUBINSDEL typeSUB monodown isLightfalse lemSim0.375 wnSyn1.0 wnAnto0.0 wnHypo0.0 typeDEL monoup isLightfalse compose = (equivalent) (forward) = (equivalent) (forward) Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

16
16 The FraCaS test suite FraCaS: mid-90s project in computational semantics 346 “textbook” examples of textual inference problems No delegate finished the report. Some delegate finished the report on time. Smith believed that ITEL had won the contract in 1992. ITEL won the contract in 1992. 9 sections: quantifiers, plurals, anaphora, ellipsis, … 3 possible answers: yes, no, unknown (not balanced!) 55% single-premise, 45% multi-premise (excluded) unk no Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

17
17 Results on FraCaS §Category#Acc. 1Quantifiers4484.09 2Plurals2441.67 3Anaphora650.00 4Ellipsis2528.00 5Adjectives1560.00 6Comparatives1668.75 7Temporal3661.11 8Verbs862.50 9Attitudes955.56 “Applicable”: 1, 5, 67576.00 All sections18359.56 yesunknototal yes6240—102 unk1545—60 no613221 total90912183 guess gold by section confusion matrix Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

18
18 The RTE3 test suite RTE: more “natural” textual inference problems Much longer premises: average 35 words (vs. 11) Binary classification: yes and no RTE problems not ideal for NatLog Many kinds of inference not addressed by NatLog Big edit distance propagation of errors from atomic model Maybe we can achieve high precision on a subset? Strategy: hybridize with broad-coverage RTE system As in Bos & Markert 2006 Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

19
19 A hybrid RTE system using NatLog NatLog pre-processing alignment classification {yes, no} Stanford pre-processing alignment classification [– , + ] threshold (balanced) {yes, no} xx threshold (optimized) {yes, no} Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

20
20 Results on RTE3 RTE3 Development Set (800 problems) System% yes precisio n recall accurac y Stanford50.2568.6666.9967.25 NatLog18.0076.3926.7058.00 Hybrid, balanced 50.0069.7567.7268.25 Hybrid, optimized 55.1369.1674.0369.63 RTE3 Test Set (800 problems) System% yes precisio n recall accurac y Stanford50.0061.7560.2460.50 NatLog23.8868.0631.7157.38 Hybrid, balanced 50.0064.5062.9363.25 Hybrid, optimized 54.1363.7467.3263.62 25 extra problems (significant, p < 0.01) Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

21
21Conclusion Natural logic enables precise reasoning about monotonicity, while sidestepping the difficulties of translating to FOL. The NatLog system successfully handles a broad range of such inferences, as demonstrated on the FraCaS test suite. Future work: Add proof search, to handle multiple-premise inference problems Consider using CCG parses to facilitate monotonicity projection Explore the use of more sophisticated alignment models Bring factive & implicative inferences into the NatLog framework :-) Thanks! Questions? Introduction Foundations of Natural Logic The NatLog System Experiments with FraCaS Experiments with RTE Conclusion

Similar presentations

OK

1 Chapter 7 Propositional and Predicate Logic. 2 Chapter 7 Contents (1) l What is Logic? l Logical Operators l Translating between English and Logic l.

1 Chapter 7 Propositional and Predicate Logic. 2 Chapter 7 Contents (1) l What is Logic? l Logical Operators l Translating between English and Logic l.

© 2018 SlidePlayer.com Inc.

All rights reserved.

To ensure the functioning of the site, we use **cookies**. We share information about your activities on the site with our partners and Google partners: social networks and companies engaged in advertising and web analytics. For more information, see the Privacy Policy and Google Privacy & Terms.
Your consent to our cookies if you continue to use this website.

Ads by Google

Ppt on single phase transformer Ppt on railway track maps Ppt on addition for class 2 Download free ppt on solar system Ppt on job evaluation answers Ppt on non biodegradable waste management Ppt on power line carrier communication system Ppt on inhabiting other planets heating Ppt on indian politics in present By appt only movie channels