Presentation is loading. Please wait.

Presentation is loading. Please wait.

Vamshi Ambati 14 Sept 2007 Student Research Symposium

Published byDevi Widyawati Hartanto Modified over 5 years ago

Similar presentations

Presentation on theme: "Vamshi Ambati 14 Sept 2007 Student Research Symposium"— Presentation transcript:

1 Vamshi Ambati 14 Sept 2007 Student Research Symposium
Noun Phrase driven Bootstrapping of Word Alignment for Syntax Projection Vamshi Ambati 14 Sept 2007 Student Research Symposium

2 Agenda Rule Learning for MT Syntax Projection Task Word Alignment Task
Bootstrapping Word Alignment Experiment and Results

3 Machine Translation for Resource poor Languages
A major portion of the human languages are ‘resource-poor’ Less parallel corpus into Major languages Less monolingual corpus Less annotation tools Less grammarians Less bilingual speakers Machine Translation in such a scenario is extremely difficult

4 Machine Translation for Resource poor Languages
AVENUE [Alavie’03 et.al] –

5 Rule Learning for MT NP::NP [PP NP] -> [NP PP] ( (X1::Y2) (X2::Y1)
(X0 = X2) ((Y1 NUM) = (X2 NUM)) ((Y1 NUM) = (Y2 NUM)) ((Y1 PERS) = (Y2 PERS)) (Y0 = Y1) ) PP::PP [ADVP NP POSTP] -> [ADVP PREP NP] (X1::Y1) (X2::Y3) (X3::Y2) (X0 = X3) (Y0 = Y2)

6 How can such rules be learnt?
Given annotated data for TL we have creative ways to do this Nothing more valuable than annotated data But, these are “resource-poor” languages Can we look from the ‘Source side’ and transfer annotation ?

7 Syntax Projection Named Entity Projection [Rama] ate an apple
rAma ne ek apple khaya

8 Syntax Projection Named Entity Projection [Rama] ate an apple
[rAma] ne ek apple khaya

9 Syntax Projection Base NP Projection [Rama] ate [an apple]
rAma ne ek apple khaya

10 Syntax Projection Base NP Projection [Rama] ate [an apple]
[rAma] ne [ek apple] khaya

11 Syntax Projection Constituent Phrase Projection rAma ne ek apple khaya

12 Syntax Projection Constituent Phrase Projection

13 Rule Learning Goal English: Rama ate an apple
Hindi: raMa ne apple khaya S::S [NP NP VP] -> [NP VP NP] S::S [NP NP ‘khaya’] -> [NP ‘ate’ NP] S::S [‘rAma’ ‘ne’ NP ‘khaya’] -> [‘Rama’ ‘ate’ NP]

14 Word Alignment Task Training data Source language Target language
f : source sentence (Hindi) j = 1,2,...,J Target language e : target sentence (English) i = 1,2,...,I

15 Word Alignment Models IBM1 – lexical probabilities only
IBM2 – lexicon plus absolut position IBM3 – plus fertilities IBM4 – inverted relative position alignment IBM5 – non-deficient version of model 4 HMM – lexicon plus relative position [Brown et.al. 1993, Vogel et.al. 1996, Och et al 1999]

16 Our Approach Better Syntax Projection requires better Word Alignment
Our Hypothesis: Word Alignment can be improved using Syntax projection Project Base NPs to TL and obtain a clean NP table Perform a constrained alignment in Parallel Corpus using the NP table

17 Why Base NPs ? NPs are semantically and syntactically cohesive across languages NPs show minimal categorical divergence when compared to its colleagues NPs are building blocks of a sentence and their translation improves MT quality [Philipp Koehn, PhD thesis 2003]

18 Constrained Alignment
[PESA: Phrase Pair Extraction as Sentence Splitting, Vogel ’05 ]

19 Constrained Alignment
Ex: Rama ate [an apple] rAma ne [ek apple] khaya

20 Constrained Alignment
Ex: Rama ate [an apple] rAma ne khaya [ek apple]

21 NP based Bootstrapping: Algorithm
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table

22 Corpus There are quite a large number of Malayalees living here .
is found in the west coast of Great Nicobar called the Magapod Island . Plotemy calls them ' Nagadip ' , a Hindu name for naked island malayAlama logoM kI bahuwa badZI saMKyA hE . xvIpa ke paScimI wata para sWiwa mEgApOda xvIpa meM pAyA jAwA hE . plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna logoM ke lie prayukwa howA hE .

23 Source side Parsed There are quite a large number of Malayalees living here . is found in the west coast of Great Nicobar called the Magapod Island . Plotemy calls them ' Nagadip ' , a Hindu name for naked island (S1 (S (NP (EX There)) (VP (AUX are) (NP (NP (PDT quite) (DT a) (JJ large) (NN number)) (PP (IN of) (NP (NP (NNS Malayalees)) (VP (VBG living) (ADVP (RB here))))))) (. .))) (S1 (S (VP (AUX is) (VP (VBN found) (PP (IN in) (NP (NP (DT the) (JJ west) (NN coast)) (PP (IN of) (NP (NNP Great) (NNP Nicobar))) (VP (VBN called) (S (NP (DT the) (NNP Magapod) (NNP Island)))))))) (. .))) (S1 (S (NP (NNP Plotemy)) (VP (VBZ calls) (SBAR (S (NP (PRP them)) (VP (POS ') (NP (NP (NNP Nagadip) (POS ')) (, ,) (NP (NP (DT a) (NNP Hindu) (NN name)) (PP (IN for) (NP (JJ naked) (NN island))))))))) (. .)))

24 NP based Bootstrapping
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table

25 Aligned Corpus ;;Sentence id = 1
SL:There are quite a large number of Malayalees living here . TL:malayAlama logoM kI bahuwa badZI saMKyA hE . Alignment:((1,8),(2,7),(11,8),(3,1),(4,8),(5,5),(6,6),(7,3),(8,1),(9,1),(10,1)) ;;Sentence id = 2 SL:is found in the west coast of Great Nicobar called the Magapod Island . TL:xvIpa ke paScimI wata para sWiwa mEgApOda xvIpa meM pAyA jAwA hE . Alignment:((1,12),(2,10),(11,2),(12,7),(13,1),(14,13),(3,9),(4,2),(5,4),(6,4),(7,2),(8,7),(9,1),(10,11))

26 NP based Bootstrapping
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table

27 Extract Source NPs NP:1:There :. NP:1:quite a large number :.
NP:1:Malayalees : NP:2:the west coast : NP:2:Great Nicobar : NP:2:the Magapod Island : NP:6:Plotemy : NP:6:them : NP:6:Nagadip ' : NP:6:a Hindu name : NP:6:naked island :

28 NP based Bootstrapping
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table

29 Extract NP translation Pairs
NP:1:There :. NP:1:quite a large number :malayAlama logoM kI bahuwa badZI saMKyA hE . NP:1:Malayalees :malayAlama NP:2:the west coast :ke paScimI wata NP:2:Great Nicobar :xvIpa ke paScimI wata para sWiwa mEgApOda NP:2:the Magapod Island :xvIpa ke paScimI wata para sWiwa mEgApOda NP:6:Plotemy :plotemI NP:6:them :inheM NP:6:Nagadip ' :plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna logoM ke NP:6:a Hindu name :plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna logoM ke lie prayukwa howA hE . NP:6:naked island :plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna

30 NP based Bootstrapping
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table

31 Feature Extraction for NP Pairs
Features Source length in words Target length in words Absolute length difference Freq Source base np Freq of Target base np Freq of the S-T pair Source 2 Target probability Target 2 Source probability

32 Calculate Features of NP pairs
NP:1:There:.:1:1:0:2449:2318:258: : NP:1:quite a large number:malayAlama logoM kI bahuwa badZI saMKyA hE .:4:8:4:3:1:1: e-13:0 NP:1:Malayalees:malayAlama:1:1:0:3:2:1: : NP:2:the west coast:ke paScimI wata:3:3:0:15:2:1: e-06: e-11 NP:2:Great Nicobar:xvIpa ke paScimI wata para sWiwa mEgApOda:2:7:5:6:2:1: e-05:0 NP:2:the Magapod Island:xvIpa ke paScimI wata para sWiwa mEgApOda:3:7:4:1:2:1: e-06:0 NP:6:Plotemy:plotemI:1:1:0:1:1:1:1:1 NP:6:them:inheM:1:1:0:2153:27:16: :0 NP:6:Nagadip ':plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna logoM ke:2:15:13:1:1:1: e-05:0 NP:6:a Hindu name:plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna logoM ke lie prayukwa howA hE .:3:20:17:1:1:1: e-12:0 NP:6:naked island:plotemI inheM nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna:2:13:11:1:1:1: e-06:0

33 NP based Bootstrapping
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table

34 Prune based on manual thresholds
NP:1:There:. NP:1:Malayalees:malayAlama NP:2:the west coast:ke paScimI wata NP:6:Plotemy:plotemI NP:6:them:inheM

35 NP based Bootstrapping
Word Align ‘S’ and ‘T’ using IBM-4 Extracted NP on source side using the Parse Extracted translations by harvesting Viterbi Alignment Calculate features for NP pairs Prune based on thresholds Perform Constrained Word Alignment and Lexicon extraction using the NP table (Folding)

36 Constrained Alignment: NP Folding
There Malayalees NP are quite a large number of NP living here . the west coast is found in NP of Great Nicobar called the Magapod Island . them Plotemy NP calls NP ' Nagadip ' , a Hindu name for naked island . . malayAlama NP logoM kI bahuwa badZI saMKyA NP ke paScimI wata xvIpa NP para sWiwa mEgApOda xvIpa meM pAyA jAwA hE . them plotemI NP NP nAgAxvIpa kahawA hE jo ki hiMxI kA Sabxa hE Ora nagna logoM ke lie prayukwa howA hE .

37 Experiments English Hindi (Resource constrained) English Hindi

38 Word Alignment Experiments
Training: 5000 sentences Testing: 200 sentences Human Extracted NP table – 21,736

39 Word Alignment Results (5k corpus)
Experiments with 5k training corpus and 200 test sentences Experiment Prec Rec F- AER Model 4 40.20 30.96 34.98 65.12 Model 4+ NPs ( 1 ) 41.33 32.12 36.15 63.85 Model 4+ NPs ( 2 ) 41.23 32.26 36.02 63.88

40 NP Projection Results (5k)
Evaluation: NP_Table harvested from 5k test bed corpus Iteration Identified Accuracy (on gold std) After Pruning NPs (1) 21693 33% (7200) 4708 61% (2619) NPs (2) 21690 33.12% (7200) 4819 60.8% (2601)

41 Word Alignment Experiments
Training: 50K Eng-Hin Corpus Testing: 200 Eng-Hin aligned sentences Human Extracted NP table – 21,736

42 Word Alignment Results (55k corpus)
Experiments with 55k training corpus and 200 test sentences Experiment Prec Rec F- AER Model 4 48.16 45.91 46.75 53.76 Model 4+ NPs ( 1 ) 48.19 46.50 46.83 53.17 Model 4+ NPs ( 2 ) 48.14 46.49 46.82 53.18

43 NP Projection Results (55k)
Evaluation: NP_Table created by human alignment Iteration Identified Precision (on gold std) After Pruning NPs (1) 306366 38% (8906) 88352 58.2% (4236) NPs (2) 306294 37% (8124) 93165 59.16%

44 From here.. Improvements Machine Translation Reliable NP Projection
Hierarchical Word Alignment Machine Translation Rule Learning Refined Probabilistic translation Lexicon Clean Linguistically motivated Phrase table with probabilities

45 Questions ?

46 Thanks !

Download ppt "Vamshi Ambati 14 Sept 2007 Student Research Symposium"

Similar presentations

Statistical Machine Translation

Statistical Machine Translation
Feature Forest Models for Syntactic Parsing Yusuke Miyao University of Tokyo.

Feature Forest Models for Syntactic Parsing Yusuke Miyao University of Tokyo.
CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)

CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)
Statistical Machine Translation Part II: Word Alignments and EM Alexander Fraser Institute for Natural Language Processing University of Stuttgart 2011.11.11.

Statistical Machine Translation Part II: Word Alignments and EM Alexander Fraser Institute for Natural Language Processing University of Stuttgart
Learning Accurate, Compact, and Interpretable Tree Annotation Recent Advances in Parsing Technology WS 2011/2012 Saarland University in Saarbrücken Miloš.

Learning Accurate, Compact, and Interpretable Tree Annotation Recent Advances in Parsing Technology WS 2011/2012 Saarland University in Saarbrücken Miloš.
Learning and Inference for Hierarchically Split PCFGs Slav Petrov and Dan Klein.

Learning and Inference for Hierarchically Split PCFGs Slav Petrov and Dan Klein.
Statistical Machine Translation Part II – Word Alignments and EM Alex Fraser Institute for Natural Language Processing University of Stuttgart 2008.07.22.

Statistical Machine Translation Part II – Word Alignments and EM Alex Fraser Institute for Natural Language Processing University of Stuttgart
1 A Tree Sequence Alignment- based Tree-to-Tree Translation Model Authors: Min Zhang, Hongfei Jiang, Aiti Aw, et al. Reporter: 江欣倩 Professor: 陳嘉平.

1 A Tree Sequence Alignment- based Tree-to-Tree Translation Model Authors: Min Zhang, Hongfei Jiang, Aiti Aw, et al. Reporter: 江欣倩 Professor: 陳嘉平.
Machine Translation via Dependency Transfer Philip Resnik University of Maryland DoD MURI award in collaboration with JHU: Bootstrapping Out of the Multilingual.

Machine Translation via Dependency Transfer Philip Resnik University of Maryland DoD MURI award in collaboration with JHU: Bootstrapping Out of the Multilingual.
Learning Accurate, Compact, and Interpretable Tree Annotation Slav Petrov, Leon Barrett, Romain Thibaux, Dan Klein.

Learning Accurate, Compact, and Interpretable Tree Annotation Slav Petrov, Leon Barrett, Romain Thibaux, Dan Klein.
Statistical Phrase-Based Translation Authors: Koehn, Och, Marcu Presented by Albert Bertram Titles, charts, graphs, figures and tables were extracted from.

Statistical Phrase-Based Translation Authors: Koehn, Och, Marcu Presented by Albert Bertram Titles, charts, graphs, figures and tables were extracted from.
Machine Translation Prof. Alexandros Potamianos Dept. of Electrical & Computer Engineering Technical University of Crete, Greece May 2003.

Machine Translation Prof. Alexandros Potamianos Dept. of Electrical & Computer Engineering Technical University of Crete, Greece May 2003.
Semi-Automatic Learning of Transfer Rules for Machine Translation of Low-Density Languages Katharina Probst April 5, 2002.

Semi-Automatic Learning of Transfer Rules for Machine Translation of Low-Density Languages Katharina Probst April 5, 2002.
A Hierarchical Phrase-Based Model for Statistical Machine Translation Author: David Chiang Presented by Achim Ruopp Formulas/illustrations/numbers extracted.

A Hierarchical Phrase-Based Model for Statistical Machine Translation Author: David Chiang Presented by Achim Ruopp Formulas/illustrations/numbers extracted.
Probabilistic Parsing Ling 571 Fei Xia Week 5: 10/25-10/27/05.

Probabilistic Parsing Ling 571 Fei Xia Week 5: 10/25-10/27/05.
LEARNING WORD TRANSLATIONS Does syntactic context fare better than positional context? NCLT/CNGL Internal Workshop Ankit Kumar Srivastava 24 July 2008.

LEARNING WORD TRANSLATIONS Does syntactic context fare better than positional context? NCLT/CNGL Internal Workshop Ankit Kumar Srivastava 24 July 2008.
Does Syntactic Knowledge help English- Hindi SMT ? Avinesh. PVS. K. Taraka Rama, Karthik Gali.

Does Syntactic Knowledge help English- Hindi SMT ? Avinesh. PVS. K. Taraka Rama, Karthik Gali.
Czech-to-English Translation: MT Marathon 2009 Session Preview Jonathan Clark Greg Hanneman Language Technologies Institute Carnegie Mellon University.

Czech-to-English Translation: MT Marathon 2009 Session Preview Jonathan Clark Greg Hanneman Language Technologies Institute Carnegie Mellon University.
PFA Node Alignment Algorithm Consider the parse trees of a Chinese-English parallel pair of sentences.

PFA Node Alignment Algorithm Consider the parse trees of a Chinese-English parallel pair of sentences.
An Integrated Approach for Arabic-English Named Entity Translation Hany Hassan IBM Cairo Technology Development Center Jeffrey Sorensen IBM T.J. Watson.

An Integrated Approach for Arabic-English Named Entity Translation Hany Hassan IBM Cairo Technology Development Center Jeffrey Sorensen IBM T.J. Watson.

Similar presentations

Ads by Google