Antonymy and Conceptual Vectors Didier Schwab, Mathieu Lafourcade, Violaine Prince Laboratoire d’informatique, de robotique Et de microélectronique de Montpellier CNRS - Université Montpellier II presented by Ch. Boitet (works with M. Lafourcade on conceptual vectors & UNL)

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Outline The main idea Background on conceptual vectors How we use CVs & why we need to distinguish CVs of antonyms Brief study of antonymies Representation of antonymies Measure for « antonymousness »

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors The main idea Work on meaning representation in NLP, using conceptual vectors (CV) applications = WSD & thematic indexing but V(existence) = V(non-existence) ! basic « concepts » activated the same Idea: use lexical functions to improve the adequacy For this, « transport » the lexical functions in the vector space

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Background on conceptual vectors Lexical Item = ideas = combination of concepts = Vector V Ideas space = vector space (generator space) Concept = idea = vector V c V c taken from a thesaurus hierarchy (Larousse) translation of Roget’s thesaurus, 873 leaf nodes the word ‘peace’ has non zero values for concept PEACE and other concepts

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Our conceptual vectors Thesaurus H : thesaurus hierarchy — K concepts Thesaurus Larousse = 873 concepts V(C i ) : a j = 1/ (2 ** D um (H, i, j)) 1/41 1/16 1/64 264

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Conceptual vectors Concept c4: ‘PEACE’ peace hierarchical relations conflict relations The world, manhood society

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Conceptual vectors Term “peace” c4:’PEACE’

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors finance profit exchange

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Angular or « thematic » distance D a (x,y) = angle(x,y) = acos(sim(x,y)) = acos(x.y /|x ||y |) 0 ≤ D(x,y) ≤  (positive components) If 0 then x and y are colinear : same idea. If  /2 : nothing in common. x y

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors The angular distance is a true distance D A (x, y) = acos(sim(x,y)) D A (x, y) = acos(x.y/|x||y|)) D A (x, x) = 0 D A (x, y) = D A (y, x) D A (x, y) + D A (y, z)  D A (x, z) D A (0, 0) = 0 and D A (x, 0) =  /2 by definition D A (  x,  y) = D A (x, y) with   0 D A (  x,  y) =  - D A (x, y) with  < 0 D A (x+x, x+y) = D A (x, x+y)  D A (x, y)

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Thematic Distance (examples) D a (anteater, anteater )= 0 (0°) D a (anteater, animal ) = 0,45 (26°) D a (anteater, train ) = 1,18 (68°) D a (anteater, mammal ) = 0,36 (21°) D a (anteater, quadruped ) = 0,42 (24°) D a (anteater, ant ) = 0,26 (15°) thematic distance ≠ ontological distance

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Vector Proximity Function V gives the vectors closest to a lexical item. V (life) = life, alive, birth… V (death) = death, to die, to kill…

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors How we build & use conceptual vectors Conceptual vectors give thematic representations of word senses of words (averaging CVs of word senses) of the content (« ideas ») of any textual segment New CVs for word senses are permanently learned from NL definitions coming from electronic dictionaries CVs of word senses are permanently recomputed for French, 3 years, words, CVs

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors SYGMART Morphosyntactic analysis Definitions Human usage dictionaries Conceptual vectors base New Vector Continuous building of the conceptual vectors database

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors We should distinguish CVs of different but related words… Non-existent : who or which does not exist cold : #ant# warm, hot Without a specific treatment, we get V(non-existence) = V(existence) V(cold) = V(hot) We want to obtain V(non-existence) ≠ V(existence) V(cold) ≠ V(hot)

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Applications: more precision Thematic analysis of texts Thematic analysis of definitions Resources: coherence & adequacy General coherence of the CV data base Conceptual Vector quality (adequacy) …in order to improve applications and resources

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Lexical functions may help! Lexical function (Mel’tchuk): WS  {WS 1 …WS n } synonymy (#Syn#), antonymy (#Anti#), intensification (#Magn#)… Examples : #Syn# (car) = {automobile} #Anti# (respect) = {disrespect; disdain} #Sing# (fleet) = {boat, ship; embarcation}

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Method: transport the LFs as functions on the CV space e.g. for antonymy, to getV(non-existence) ≠ V(existence) find vector function Anti such that: V(non-existence) = V(#Anti#(existence)) = Anti (V(existence)) similarly for other lexical functions we simply began by studying antinomy

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Brief study of antonymy Definition : Two lexical items are in antonymy relation if there is a symmetry between their semantic components relatively to an axis  Antonymy relations depend on the type of medium that supports symmetry  There are several types of antonymy  On the axis, there are fixed points:  Anti (V(car)) = V(car) because #Anti# (car) = 

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors 1- Complementary antonymy Values are boolean & symmetric (0  1) Examples : event/non-eventdead/alive existence/non-existence He is present  He is not absent He is absent  He is not present

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors 2- Scalar antonymy Values are scalar Symmetry is relative to a reference value Examples : cold/hot, small/tall This man is small  This man is not tall This man is tall  This man is not small This man is neither tall nor small reference value = « of medium height »

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors 3- Dual Antonymy (1) Conversive duals same semantics but inversion of roles Examples : sell/buy, husband/wife, father/son Jack is John’s son  John is Jack’s father Jack sells a car to John  John buys a car from Jack

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors 3- Dual Antonymy (2) Contrastive duals contrastive expressions accepted by usage Cultural : sun/moon, yin/yang Associative : question/answer Spatio-temporal : birth/death, start/finish

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Learning bootstrap based on a kernel composed of pre-computed vectors considered as adequate Learning must be coherent = preserve adequacy Adequacy = judgement that activations of concepts (coordinates) make sense for the meaning corresponding to a definition For coherence improvement, we use semantic relations between terms Coherence and adequacy of the base

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Based on the antonym vectors of concepts : one list for each kind of antonymy Anti c (EXISTENCE)= V (NON-EXISTENCE) Anti s (HOT) = V (COLD) Anti c (GAME) = V (GAME) Anti (X,C) builds the vector « opposite » of vector X in context C Antonymy function

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Construction of the antonym vector of X in context C The method is to focus on the salient notions in V(X) and V(C) If the notions can be opposed, then the antonym should have the inverse ideas in the same proportions The following formula was obtained after several experiments

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Anti R (V(X), V(C)) =  P i *Anti C (C i, V(C)) P i = V * max (V(X), V(C i )) Not symmetrical Stress more on vector X than on context C Consider an important idea of the vector to oppose even if it is not in the referent Construction of the antonym vector (2) i=1 N XiXi 1+C V (V(X))

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Results V (#Anti# (death, life & death)) = (LIFE 0,3), (birth 0,48), (alive 0,54)… V (#Anti# (life, life & death)) = (death 0,336), (killer 0,45), (murdered 0,53)… V (#Anti# (LIFE)) = (DEATH 0,034), (death 0,43), (killer 0,53)...

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Antonymy evaluation measure Assess « how much » two lexical items are antonymous Manti(A,B) = D A (A  B, Anti(A,C)  Anti(B,C)) A B Anti(B) Anti(A)

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Examples Manti (EXISTENCE, NON-EXISTENCE) = 0,03 Manti (existence, non-existence) = 0,44 Manti (EXISTENCE, CAR) = 1,45 Manti (existence, car) = 1,06 Manti (CAR, CAR) = 0,006 Manti (car, car) = 0,407

Schwab, Lafourcade, Prince, pres. by Ch. BoitetAntonymy and Conceptual Vectors Conclusion and perspectives Progress so far : Antonymy definition based on a notion of symmetry Implemented formula to compute an antonym vector Implemented measure to assess the level of antonymy between two items Perspectives : Use of the symbolic opposition found in dictionaries Search the opposite meaning of a word Study of the other semantic relations (hyperonymy/hyponymy, meronymy/holonymy…)