1. Learning Textual Entailment from Examples
Fabio Massimo Zanzotto
Dipartimento Informatica Sistemistica e Comunicazione
University of Milano-Bicocca
Italy
Alessandro Moschitti, Marco Pennacchiotti, Maria Teresa Pazienza
Department of Computer Science, Systems and Production
University of Roma “Tor Vergata”
Italy

2. Motivating the approach (1)
“At the end of the year, all solid companies pay dividends.”
“At the end of the year, all solid insurance companies pay dividends.”
T1  H1 T1 H2
“At the end of the year, all solid companies pay dividends.”
“At the end of the year, all solid companies pay cash dividends.”
T1  H2
Similarity Models would ask:
sim(T1,H1) > sim(T1,H2) ?

3. Motivating the approach (2)
“At the end of the year, all solid companies pay dividends.”
“At the end of the year, all solid insurance companies pay dividends.”
T1  H1 S2 S1
< T1 H2
“At the end of the year, all solid companies pay dividends.”
“At the end of the year, all solid companies pay cash dividends.”
T1  H2 T3 H3
“All wild animals eat plants that have scientifically proven medicinal properties.”
“All wild mountain animals eat plants that have scientifically proven medicinal properties.”
T3  H3

4. Our Model
Building a textual entailment recogniser using annotated examples as training
Defining a similarity between pairs based on:
K((T’,H’),(T’’,H’’))=KI((T’,H’),(T’’,H’’))+KS((T’,H’),(T’’,H’’))
Intra-pair similarity
KI((T’,H’),(T’’,H’’))=s(T’,H’)s(T’’,H’’)
Cross-pair similarity
KS((T’,H’),(T’’,H’’)) KT(T’,T’’)+ KT(H’,H’’)

5. Outline Our Model: Experimental evaluation Conclusions
Challenges for the cross-pair similarity
Cross-pair similarity by examples
Formal definition
Intra-pair similarity
Cross-pair similarity
Experimental evaluation
Conclusions

6. Challenges for the cross-pair similarity (1)
Can we use syntactic tree similarity?

7. Challenges for the cross-pair similarity (2)
Can we use syntactic tree similarity? Not only!

8. Challenges for the cross-pair similarity (3)
Within the learning phase, each pair should be an example of rewrite rule (or inference rule)
The cross-pair similarity measure should consider:
the structural/syntactical similarity between, respectively, texts and hypotheses
the similarity among the intra-pair relations between constituents

9. Our Model: an example

10. Our Model: an example
Intra-pair operations

11. Our Model: an example
Intra-pair operations
 Finding anchors

12. Our Model: an example Intra-pair operations
Finding anchors
Naming anchors with placeholders

13. Our Model: an example Intra-pair operations Propagating placeholders
Finding anchors
Naming anchors with placeholders
Propagating placeholders

14. Our Model: an example Intra-pair operations Cross-pair operations
Finding anchors
Naming anchors with placeholders
Propagating placeholders
Cross-pair operations

15. Our Model: an example Intra-pair operations Cross-pair operations
Finding anchors
Naming anchors with placeholders
Propagating placeholders
Cross-pair operations
Matching placeholders across pairs

16. Our Model: an example Intra-pair operations Cross-pair operations
Finding anchors
Naming anchors with placeholders
Propagating placeholders
Cross-pair operations
Matching placeholders across pairs
Renaming placeholders

17. Our Model: an example Intra-pair operations Cross-pair operations
Finding anchors
Naming anchors with placeholders
Propagating placeholders
Cross-pair operations
Matching placeholders across pairs
Renaming placeholders
Calculating the similarity between syntactic trees with co-indexed leaves

18. Our Model: an example Intra-pair operations Cross-pair operations
Finding anchors
Naming anchors with placeholders
Propagating placeholders
Cross-pair operations
Matching placeholders across pairs
Renaming placeholders
Calculating the similarity between syntactic trees with co-indexed leaves

19. Our Model: an example
The initial example: sim(H1,H3) > sim(H2,H3)?

20. Similarity Models Defining a similarity between pairs based on:
K((T’,H’),(T’’,H’’))=KI((T’,H’),(T’’,H’’))+KS((T’,H’),(T’’,H’’))
Intra-pair similarity: Anchoring
KI((T’,H’),(T’’,H’’))=s(T’,H’)s(T’’,H’’)
Cross-pair similarity
KS((T’,H’),(T’’,H’’)) KT(T’,T’’)+ KT(H’,H’’)

21. Anchoring and Intra-pair Lexical Similarity
Anchoring and Intra-pair lexical similarity are strictly related:
Given a pair (T,H), WT and WH the sets of words of T and H, the set of anchors is
A WTWH
where (wt,wh )  A has the property:

22. Anchoring and Intra-pair Lexical Similarity
Lexical similarity between Words simw(wt,wh ) is defined using:
Token matching
Lemma matching
Edit distance between tokens
Verb entailment
Derivational related forms
Jiang&Conrath lexical distance
The intra pair lexical similarity is:
with idf
without idf

23. Cross-pair similarity
The cross pair similarity is based on the distance between syntatic trees with co-indexed leaves:
where
C is the set of all the correspondences between anchors of (T’,H’) and (T’’,H’’)
t(S, c) returns the parse tree of the hypothesis (text) S where placeholders of these latter are replaced by means of the substitution c
i is the identity substitution
KT(t1, t2) is a function that measures the similarity between
the two trees t1 and t2.

24. Refining Cross-pair Similarity
Controlling complexity
We reduced the size of the set of anchors using the notion of chunk
Focussing on information within a pair relevant for the entailment:
Text trees are pruned according to where anchors attach

25. Experimental Setting Corpora Resources
D1, T1 and D2, T2, are the development and the test sets of the RTE1 and RTE2
ALL=D1D2 T1 randomly split in 70%-30%
D2(50%)’ and D2(50%)’’ is a random split of D2
Resources
Charniak parser (Charniak, 2000) and the morpha lemmatiser (Minnen et al., 2001)
WordNet 2.0 (Miller, 1995) for verbs in entailmentand the derivationally related words
The wn::similarity package (Pedersen et al., 2004) to compute the Jiang&Conrath (J&C) distance (Jiang and Conrath, 1997)
A selected portion of the British National Corpus for the idf
SVM-light-TK2 (Moschitti, 2004) for the basic tree kernel function, KT, encodend in SVMlight (Joachims, 1999)

26. Experimental Results

27. Conclusions Textual entaliment can be learnt from examples
The Cross-pair similarity based on pairs of syntactic trees with co-indexed leaves seems to be promising
RTE corpora have some bias
Future directions:
Demonstrating that KS or approximating it with a Mercer Kernel
Including lexical prior knowledge in the model
Introducing different levels of analysis (i.e., semantic interpretation)

28. The end!
Thank you!
Questions?