Lens effects in autonomous terminology and conceptual vector learning Mathieu Lafourcade LIRMM - France http://www.lirmm.fr/~lafourcade 1
Overwiew & Objectives lexical semantic representations conceptual vector model (cvm) autonomous learning by the system from a given « semantic space » (ontology) effects of swithing ontologies (general spec) global effects on the lexicon local effects on particular word ambiguity as noise towards self contained WSD annotations « I made a deposit at the bank » « I made a deposit at the bank<g:money> »
Conceptual vectors vector space An idea Concept combination — a vector Idea space = vector space A concept = an idea = a vector V with augmentation: V + neighboorhood Meaning space = vector space + {v}* 27
2D view of « meaning space » “product” “cat”
Conceptual vectors Thesaurus H : thesaurus hierarchy — K concepts Thesaurus Larousse = 873 concepts V(Ci) : <a1, …, ai, … , a873> aj = 1/ (2 ** Dum(H, i, j)) 1/16 1/16 1/4 1 1/4 1/4 1/64 1/64 4 2 6 93
Conceptual vectors Concept c4:peace conflict relations hierarchical relations The world, manhood society
Conceptual vectors Term “peace” c4:peace
exchange profit finance
Angular distance DA(x, y) = angle (x, y) 0 DA(x, y) if 0 then x & y colinear — same idea if /2 then nothing in common if then DA(x, -x) with -x — anti-idea of x x’ x y 36
Angular distance DA(x, y) = acos(sim(x,y)) DA(x, y) = acos(x.y/|x||y|)) DA(x, x) = 0 DA(x, y) = DA(y, x) DA(x, y) + DA(y, z) DA(x, z) DA(0, 0) = 0 and DA(x, 0) = /2 by definition DA(x, y) = DA(x, y) with 0 DA(x, y) = - DA(x, y) with < 0 DA(x+x, x+y) = DA(x, x+y) DA(x, y) 37
Thematic distance Examples DA(tit, tit) = 0 DA(tit, passerine) = 0.4 DA(tit, bird) = 0.7 DA(tit, train) = 1.14 DA(tit, insect) = 0.62 tit = insectivorous passerine bird … 43
Some vector operations Addition : Z = X Y zi = xi + yi vector Z is normalized Term to term mult : Z = X Y zi = (xi * yi)1/2 vector Z is not normalized Weak contextualization : Z = X (X Y) = (X,Y) “Z is X augmented by its mutual information with Y”
2D view of weak contextualization XY X(XY) X XY Y(XY) XY Y
Autonomous learning 1/2 set of known words K, set of unknow words U revise a word w of K OR (try to) learn a word w of U From the web : for w ask for a def D specific sites : dicts, synonyms list, etc. def analysis general sites : google, etc. corpus analysis for each word wd of D if not in K then add wd to U AND add VO to V* otherwise get the vector of wd AND add V(wd) to V* compute the new vector of w from def(D) and V* 98870 words for 400000 senses (vectors) learned in 3 years French « forever » looping process
Autonomous learning 2/2 insectivorous passerine bird … Weighted sum TXT Weighted sum (X,Y) V PH V N, GOV … V … V ADJ, … insectivorous passerine bird … V
Local expansion of vector space “product” “cat” cn c1 GS G finer mesh locally over the space
Point of G without link in S Folding and unfolding S Specialized ontology + a + b General ontology + b G + c Point of G without link in S vG … a b … c a … a+b … b … b+c … vS GGS(vG) = vGS 2000 873 GSG(vGS) = vG
Local lexical density given a point P count the number of points at distance d1, d2, dn P 0 ≤ d1 < d2 <…dn ≤ π/2
Lexical Distribution from Local density medium density curve top centered high density curve shifted on the left low density case left as an exercise
Macro level Local density variation G GS
Micro level Distance variation small angle = high similarity larger angle = less similarity G GS
Last words Switching of representation global and local test functions coarse grained to fine grained better semantic discrimation … and vice-versa conservation of resource global and local test functions for vector quality assessment decision taking about level of representation detectors when combined to lexical functions (antonymy, etc.) the basis for self adjustement toward a vector space of constant density wsd as a reduction of noise (in context or out of context) unification of ontologies self emergent structuration of terminology