Presentation is loading. Please wait.

Presentation is loading. Please wait.

RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria A Knowledge-Rich Approach to Measuring the Similarity between Bulgarian and Russian Words Preslav.

Published byAlfred Strickland Modified over 8 years ago

Similar presentations

Presentation on theme: "RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria A Knowledge-Rich Approach to Measuring the Similarity between Bulgarian and Russian Words Preslav."— Presentation transcript:

1 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria A Knowledge-Rich Approach to Measuring the Similarity between Bulgarian and Russian Words Preslav Nakov, Sofia University "St. Kliment Ohridski" Elena Paskaleva, Bulgarian Academy of Sciences Svetlin Nakov, Sofia University "St. Kliment Ohridski" Workshop “Multilingual Resources, Technologies and Evaluation for Central and Eastern European Languages”, RANLP 2009

2 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Introduction Objective Measure the extent to which a Bulgarian and a Russian word are perceived as similar by a person who is fluent in both languages Orthographic similarity Modified to account typical cross-lingual correspondences between Bulgarian and Russian, e.g. transformations of inflections Example Bulgarian афектирахме and Russian аффектировались are orthographically different but perceived as similar

3 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Orthographic Similarity Minimum Edit Distance Ratio (MEDR) MED(s 1, s 2 ) = the minimum number of INSERT / REPLACE / DELETE operations for transforming s 1 to s 2 (Levenshtein distance) MEDR is also known as normalized edit distance (NED) Longest Common Subsequence Ratio (LCSR) Maximal length subsequence common to both words, normalized by the longer word

4 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Modified Minimum Edit Distance Ratio (MMEDR) Our MMEDR similarity algorithm Reduces the Russian word to an intermediate Bulgarian-sounding form Applies a set of linguistically motivated transformation rules Compares orthographically the modified Russian word with the Bulgarian word Calculates weighted Levenshtein distance

5 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Linguistic Motivation behind the MMEDR Algorithm

6 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Linguistic Motivation Transliteration from Cyrillic to Cyrillic Full coincidence (equality) of letters Regular letter transitions Transformations of n-grams Lemmatization Transformation Weights

7 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transliteration What is transliteration? Transition of sounds and their letter correspondences in one language to letters in another language Russian → Bulgarian transliteration Full coincidence (equality) of letters E.g. a → a (азбука – азбука) Russian letters missing in Bulgarian E.g. ы → и, э → е (рыба – риба, поэт – поет) Removing a Russian letter E.g. пальто → палто Regular letter transitions E.g. муж → мъж, хлеб → хляб, сон → сън

8 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transformation of n-grams Regular sound-letter transitions from Russian to Bulgarian Transformations originating from spelling Double consonants, e.g. процесс → процес Voiceless to voiced consonants, e.g. бессмертный → безсмъртен Transformations of morphological origin Removing agglutinative morphemes (ся and сь), e.g. веселиться → веселить Transforming endings, e.g. стенной → стенен

9 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transformation of Russian Adjectives Russian Ending Bulgarian Ending Example -нный-ненвоенный → военен -ный-енвечный → вечен -нний-ненранний → ранен -ний-енвечерний → вечерен -ский-скивражеский → вражески -ый-истрелковый → стрелкови -нной-ненстенной → стенен -ной-енродной → роден -ой-иделовой → делови

10 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transformation of Russian Verbs Russian Ending Bulgarian Ending Examples -овать-амдекорировать → декорирам -ить-ябродить → бродя -ять-яблеять → блея -ать-амдавать → давам -уть-агаснуть → гасна -еть-еябелеть → белея

11 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Lemmatization Bulgarian and Russian are highly- inflectional languages Variety of endings express the different forms of the same word What is lemmatization? Replacement of inflected wordforms with their lemmata E.g. късният → късен (Bulgarian), равняющимся → равнять (Russian) Lemmatization can handle inflections

12 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transformation Weights We use weights for letter substitutions when measuring Levenshtein distance We account regular phonetic and spelling letter correspondences Some substitutions are unlikely E.g. о → у is more likely than о → щ Replacing letter with itself has cost 0 Regular letter substitution cost is 1 Consonants and vowels with similar sequences of distinctive phonetic features have less substitution cost (e.g. б → в)

13 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transformation Weights а w(а, е)=0.7; w(а, и)=0.8; w(а, о)=0.7; w(а, у)=0.6; w(а, ъ)=0.5; w(а, ю)=0.8; w(а, я)=0.5 бw(б, в)=0.8; w(б, п)=0.6 вw(в, ф)=0.6 гw(г, х)=0.5 дw(д, т)=0.6 е w(е, и)=0.6; w(е, о)=0.7; w(е, у)=0.8; w(е, ъ)=0.5; w(е, ю)=0.8; w(е, я)=0.5 жw(ж, з)=0.8; w(ж, ш)=0.6 зw(з, с)=0.5 и w(и, й)=0.6; w(и, о)=0.8; w(и, у)=0.8; w(и, ъ)=0.8; w(и, ю)=0.7; w(и, я)=0.7 йw(й, ю)=0.7; w(й, я)=0.7 кw(к, т)=0.8; w(к, х)=0.6 мw(м, н)=0.7 о w(о, у)=0.6; w(о, ъ)=0.8; w(о, ю)=0.7; w(о, я)=0.8 пw(п, ф)=0.8; w(п, х)=0.9 сw(с, ц)=0.6; w(с, ш)=0.9 т w(т, ф)=0.8; w(т, х)=0.9; w(т, ц)=0.9 у w(у, ъ)=0.5; w(у, ю)=0.6; w(у, я)=0.8 фw(ф, ц)=0.8 хw(х, ш)=0.9 цw(ц, ч)=0.8 чw(ч, ш)=0.9 ъw(ъ, ю)=0.8; w(ъ, я)=0.8 юw(ю, я)=0.8

14 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria The MMEDR Algorithm in Details

15 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria The MMEDR Algorithm MMEDR algorithm steps (order is important): 1. Lemmatize the Bulgarian word 2. Lemmatize the Russian word 3. Transform the Russian word’s ending 4. Transliterate the Russian word 5. Remove some double consonants in the Russian word 6. Calculate weighted Levenshtein distance 7. Normalize and calculate the MMEDR value

16 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Lemmatizing Bulgarian and Russian Words How to perform lemmatization? Use of large morphological dictionaries Wordforms are replaced with corresponding lemmata Lemmatization if optional step in MMEDR For each word it is either performed or not When multiple lemmata are found, all of them are considered Highest value of MMEDR is taken

17 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Transforming the Russian Endings The following endings are replaced in the Russian words: нный → нен; ный → ен; нний → нен; ний → ен; ий → и; ый → и; нной → нен; ной → ен; ой → и; ский → ски; ься → ь; овать → ам; ить → я; ять → я; ать → ам; уть → а; еть → ея

18 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Removing Double Consonants The following substitutions are performed in the Russian words: бб → б; жж → ж; кк → к; лл → л; мм → м; пп → п; рр → р; сс → с; тт → т; фф → ф Note that not all double consonants are replaced, e.g. дд is left дд E.g. наддавать → наддавам

19 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Calculating Weighted Levenshtein Distance Starting from classical Levenshtein distance (MED) we modify it to use weights for letter substitutions (MMED) We use the previously discussed linguistically motivated weights We calculate MMEDR as follows:

20 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Calculating the Final Result The final MMEDR value is calculated by maximum of all MMEDR values: with / without lemmatization of the Bulgarian word with / without lemmatization of the Russian word with / without transformation of the Russian word ending Lemmatization sometimes produces multiple lemmata, so all of them are considered

21 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria MMEDR Algorithm: Example Bulgarian word: афектирахме Russian word: аффектировались Traditional MEDR similarity MED(афектирахме, аффектировались) = 7 Apply normalization MEDR = 1–(7/15) = 8/15 ≈ 53% Even though these words "sound similar" to Bulgarian / Russian fluent speakers

22 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria MMEDR Algorithm: Example (2) Our improved MMEDR similarity: Lemmatization produces афектирам and аффектировать We replace the double Russian consonant -фф- by -ф- We obtain афектирам and афектировать We replace the Russian ending -овать by the Bulgarian ending -ам We obtain identical words: афектирам and афектирам Thus our MMEDR similarity is 100%

23 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Another MMEDR Example Bulgarian word избягам and the Russian word отбегать (both meaning ‘to run out’) MED(избягам,отбегать) = 5 MEDR = 1 – (5/8) = 3/8 = 37.5% MMEDR first transforms отбегать to отбегам MMED(избягам, отбегам) = 0.8 + 1 + 0.5 = 2.3 MMEDR = 1 – (2.3/7) = 47/70 ≈ 67%

24 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Experiments and Evaluation

25 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Experimental Setup Model the problem as information retrieval (IR) task: Retrieve all similar pairs of words from Bulgarian and Russian lists of words Measure similarity between 200 x 200 = 40,000 Bulgarian-Russian pairs of words 163 pairs annotated as similar by linguist 39,837 considered unrelated Rank the 40,000 pairs by MMEDR algorithm Evaluate the quality of the ranking with 11pt interpolated average precision

26 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Resources Textual resources The first 200 words from the Russian novel The Lord of the World (Властелин мира) by Alexander Belyayev The first 200 words form the Bulgarian translation of the novel Grammatical resources (for lemmatization) Grammatical dictionary of Bulgarian 1M wordforms and 70,000 lemmata Grammatical dictionary of Russian 1.5M wordforms and 100,000 lemmata

27 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Results MMEDR significantly outperforms traditional orthographic similarity measures: Algorithm 11-pt interpolated average precision LCSR69.06% MEDR72.30% MMEDR90.58%

28 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Results – Produced Ranking #Bulgarian wordRussian wordMMEDRSimilar?PrecisionRecall 1беляев 1.0000Yes100.00%0.68% 2на 1.0000Yes100.00%1.37% 3глава 1.0000Yes100.00%2.05% 4кандидат 1.0000Yes100.00%2.74% 5за 1.0000Yes100.00%3.42% 6наполеоннаполеоны1.0000Yes100.00%4.11% 7не 1.0000Yes100.00%4.79% 8минас1.0000No87.50%4.79% 9мимой1.0000Yes88.89%5.48% 10мимы1.0000Yes90.00%6.16%... 93четвъртиятчетвертым0.9375Yes94.57%59.59% 94оставятостается0.9286Yes94.62%60.27%... 39998сав0.0000No0.37%100% 39999сак0.0000No0.37%100% 40000боядисвалик0.0000No0.37%100%

29 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Conclusion We proposed orthographical similarity measure algorithm for Bulgarian / Russian Outperforms traditional orthographic similarity measures Accuracy is still far from 100% Evaluation performed with stop words included No publications on orthographic similarity for Bulgarian / Russian Can not compare the results with others

30 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Future Work Combine the ideas of MMEDR with machine learning techniques Automatically learning transformation rules for n-grams correspondences Perform evaluation with stop words excluded Evaluation for different pairs of languages

31 RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Questions? A Knowledge-Rich Approach to Measuring the Similarity between Bulgarian and Russian Words

Download ppt "RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria A Knowledge-Rich Approach to Measuring the Similarity between Bulgarian and Russian Words Preslav."

Similar presentations

RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Unsupervised Extraction of False Friends from Parallel Bi-Texts Using the Web as a Corpus Preslav.

RANLP 2009 – September 12-18, 2009, Borovets, Bulgaria Unsupervised Extraction of False Friends from Parallel Bi-Texts Using the Web as a Corpus Preslav.
RANLP 2007 – September 27-29, 2007, Borovets, Bulgaria Cognate or False Friend? Ask the Web! Svetlin Nakov, Sofia University St. Kliment Ohridski Preslav.

RANLP 2007 – September 27-29, 2007, Borovets, Bulgaria Cognate or False Friend? Ask the Web! Svetlin Nakov, Sofia University "St. Kliment Ohridski" Preslav.
PHONE MODELING AND COMBINING DISCRIMINATIVE TRAINING FOR MANDARIN-ENGLISH BILINGUAL SPEECH RECOGNITION Yanmin Qian, Jia Liu ICASSP2010 Pei-Ning Chen CSIE.

PHONE MODELING AND COMBINING DISCRIMINATIVE TRAINING FOR MANDARIN-ENGLISH BILINGUAL SPEECH RECOGNITION Yanmin Qian, Jia Liu ICASSP2010 Pei-Ning Chen CSIE.
1 Minimally Supervised Morphological Analysis by Multimodal Alignment David Yarowsky and Richard Wicentowski.

1 Minimally Supervised Morphological Analysis by Multimodal Alignment David Yarowsky and Richard Wicentowski.
Spelling Correction for Search Engine Queries Bruno Martins, Mario J. Silva In Proceedings of EsTAL-04, España for Natural Language Processing Presenter:

Spelling Correction for Search Engine Queries Bruno Martins, Mario J. Silva In Proceedings of EsTAL-04, España for Natural Language Processing Presenter:
CS 47051 Morphological Parsing CS 4705. 2 Parsing Taking a surface input and analyzing its components and underlying structure Morphological parsing:

CS Morphological Parsing CS Parsing Taking a surface input and analyzing its components and underlying structure Morphological parsing:
Baselines for Recognizing Textual Entailment Ling 541 Final Project Terrence Szymanski.

Baselines for Recognizing Textual Entailment Ling 541 Final Project Terrence Szymanski.
Automatic Identification of Cognates, False Friends, and Partial Cognates University of Ottawa, Canada University of Ottawa, Canada.

Automatic Identification of Cognates, False Friends, and Partial Cognates University of Ottawa, Canada University of Ottawa, Canada.
Normalizing Microtext Zhenzhen Xue, Dawei Yin and Brian D. Davison Lehigh University.

Normalizing Microtext Zhenzhen Xue, Dawei Yin and Brian D. Davison Lehigh University.
Computational Linguistic Techniques Applied to Drugname Matching Bonnie J. Dorr, University of Maryland Greg Kondrak, University of Alberta June 26, 2003.

Computational Linguistic Techniques Applied to Drugname Matching Bonnie J. Dorr, University of Maryland Greg Kondrak, University of Alberta June 26, 2003.
Video Shot Boundary Detection at RMIT University Timo Volkmer, Saied Tahaghoghi, and Hugh E. Williams School of Computer Science & IT, RMIT University.

Video Shot Boundary Detection at RMIT University Timo Volkmer, Saied Tahaghoghi, and Hugh E. Williams School of Computer Science & IT, RMIT University.
Crosslingual Ontology-Based Document Retrieval (Search) in an eLearning Environment RANLP, Borovets, 2007 Eelco Mossel University of Hamburg.

Crosslingual Ontology-Based Document Retrieval (Search) in an eLearning Environment RANLP, Borovets, 2007 Eelco Mossel University of Hamburg.
Evaluation.  Allan, Ballesteros, Croft, and/or Turtle Types of Evaluation Might evaluate several aspects Evaluation generally comparative –System A vs.

Evaluation.  Allan, Ballesteros, Croft, and/or Turtle Types of Evaluation Might evaluate several aspects Evaluation generally comparative –System A vs.
Linguisitics Levels of description. Speech and language Language as communication Speech vs. text –Speech primary –Text is derived –Text is not “written.

Linguisitics Levels of description. Speech and language Language as communication Speech vs. text –Speech primary –Text is derived –Text is not “written.
Evaluation.  Allan, Ballesteros, Croft, and/or Turtle Types of Evaluation Might evaluate several aspects Evaluation generally comparative –System A vs.

Evaluation.  Allan, Ballesteros, Croft, and/or Turtle Types of Evaluation Might evaluate several aspects Evaluation generally comparative –System A vs.
Evaluation of N-grams Conflation Approach in Text-based Information Retrieval Serge Kosinov University of Alberta, Computing Science Department, Edmonton,

Evaluation of N-grams Conflation Approach in Text-based Information Retrieval Serge Kosinov University of Alberta, Computing Science Department, Edmonton,
Chapter 5: Information Retrieval and Web Search

Chapter 5: Information Retrieval and Web Search
Semantic and phonetic automatic reconstruction of medical dictations STEFAN PETRIK, CHRISTINA DREXEL, LEO FESSLER, JEREMY JANCSARY, ALEXANDRA KLEIN,GERNOT.

Semantic and phonetic automatic reconstruction of medical dictations STEFAN PETRIK, CHRISTINA DREXEL, LEO FESSLER, JEREMY JANCSARY, ALEXANDRA KLEIN,GERNOT.
L. Padmasree Vamshi Ambati J. Anand Chandulal J. Anand Chandulal M. Sreenivasa Rao M. Sreenivasa Rao Signature Based Duplicate Detection in Digital Libraries.

L. Padmasree Vamshi Ambati J. Anand Chandulal J. Anand Chandulal M. Sreenivasa Rao M. Sreenivasa Rao Signature Based Duplicate Detection in Digital Libraries.
Multi-Style Language Model for Web Scale Information Retrieval Kuansan Wang, Xiaolong Li and Jianfeng Gao SIGIR 2010 Min-Hsuan Lai Department of Computer.

Multi-Style Language Model for Web Scale Information Retrieval Kuansan Wang, Xiaolong Li and Jianfeng Gao SIGIR 2010 Min-Hsuan Lai Department of Computer.

Similar presentations

Ads by Google