Presentation is loading. Please wait.

Presentation is loading. Please wait.

Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)

Similar presentations


Presentation on theme: "Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)"— Presentation transcript:

1 Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)

2 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT2 2 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali

3 Freedom of movement must be encouraged LM scores A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT3 3 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali LM scores TM scores ReoM scores

4 career paths …while ensuring that Freedom of movement must be encouraged LM scores A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT4 4 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali LM scores TM scores ReoM scores

5 LM scores A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT5 5 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Freedom of movement must be encouraged while ensuring that career paths … LM scores TM scores ReoM scores

6 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT6 6 Reordering Models Reordering Models E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores Many solutions have been proposed with different reo. classes, features, train modes etc. Tillman 04, Zens & Ney 06 AlOnaizan & Papineni 06 Galley & Manning 08 Green & al.10, Feng & al.10 …

7 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT7 7 Reordering Models Reordering Models E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores No matter what reordering model is used, permutation search space must be limited!  The power of all reordering models is bound to the reordering constraints in use Tillman04, Zens&Ney06 AlOnaizan & Papineni06 Galley & Manning08 Green &al.10, Feng &al.10 … Many solutions have been proposed with different reo. classes, features, train modes etc. Tillman 04, Zens & Ney 06 AlOnaizan & Papineni 06 Galley & Manning 08 Green & al.10, Feng & al.10 …

8 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT8 8 E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores

9 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT9 9 Reordering Constraints E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali #perm.=11!≈40,000,000 ReoM scores

10 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT10A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT10 E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Source-to-Source distortion #perm.=11!≈40,000,000 D(x,y)=|y-x-1| w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w Reordering Constraints

11 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT11A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT11 E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Source-to-Source distortion #perm.=11!≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w DL: distortion limit Reordering Constraints

12 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT12A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT12 The problem with DL… Arabic-English AR EN AR EN w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w

13 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT13A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT13 German-English DE EN DE EN w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w The problem with DL…

14 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT14A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT14 Source-to-Source distortion w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w Current solution: increase the DLimit #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000

15 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT15A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT15 Source-to-Source distortion w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w Current solution: increase the DLimit Generally leads to worse translations! #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 DL=7  #perm.≈7,000,000

16 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT16A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT16 Source-to-Source distortion #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 DL=7  #perm.≈7,000,000 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w Our solution:

17 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT17A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT17 Source-to-Source distortion #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 DL=7  #perm.≈7,000,000 DL=3 & modif(D)  #perm.≈20,000 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w w3w w4w w5w w6w w7w w8w w9w w Our solution: modify distortion for each test sentence Simplifies the task of reordering models!

18 18 Rest of the talk: How to modify the distortion matrix? What effect on translation quality? What effect on baseline runtimes? A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT

19 19A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT19 Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules Shallow syntax chunking: cheaper and easier than deep parsing constrains reorderings in a softer way Fuzzy (non-determinisic) reordering rules: generate N permutations for each matching sequence final reordering decision is taken during translation, guided by all SMT models (reoM, LM...) Few rules for language pair, to only capture long reordering

20 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT20A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT20 Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades

21 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT21A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT21 Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 PC 5 CC 1 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules

22 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT22A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT22 Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 PC 5 CC 1 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules

23 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT23A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT23 CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 PC 5 CC 1 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules Reordering selection Reordered source LM

24 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT24A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT24 CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules Reordering selection Reordered source LM Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1

25 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT25A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT25 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

26 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT26A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT26 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w

27 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT27A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT27 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

28 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT28A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT28 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

29 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT29A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT29 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

30 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT30A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT30 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

31 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT31A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT31 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

32 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT32A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT32 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w CC 1 w0w VC 2 w1w PC 3 w2w w3w NC 4 w4w w5w PC 5 w6w w7w Pct 6 w8w CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

33 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT33A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT33 Experiments Tasks: NIST-MT09 for Ar-En, WMT10 for De-En Systems based on Moses, include state-of-the-art hierarchical lexicalized reordering models [Tillmann 04; Koehn & al 05; Galley & Manning 08] Baseline Distortion Limits: 5 in Ar-En, 10 in De-En Evaluation by: - BLEU for lexical match & local order - KRS for global order

34 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT34A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT34 Arabic-English: Test set: eval09- NW *Distortion modified with 3-best reorderings per rule-matching sequence

35 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT35A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT35 Arabic-English: Test set: eval09- NW Distortion modified with 3-best reorderings per rule-matching sequence Translation Quality Translation Time +0.9 BLEU +0.6 KRS (signif.)

36 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT36A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT36 German-English: Test set: newstest10 *Distortion modified with 3-best reorderings per rule-matching sequence

37 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT37A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT37 German-English: Test set: newstest10 Distortion modified with 3-best reorderings per rule-matching sequence Translation Quality Translation Time +0.4 BLEU +0.7 KRS (signif.)

38 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT38A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT38 Conclusions Modified distortion allows for finer & linguistically motivated definition of search space We achieve better translation & faster decoding in language pairs where long reordering concentrates on few patterns Our method is complementary to reordering modeling For now, few reordering rules are needed to modify distortion We are currently working on a fully data-driven approach to replace the rules

39 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT39A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT39 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w2 3T w3w3 4H20123Y56 w4w4 5ATTENTION! w5w5 6N43201U34 w6w6 7K5432FOR23 w7w7 8S w8w w9w w

40 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT40A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT40 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w2 3T w3w3 4H20123Y56 w4w4 5ATTENTION! w5w5 6N43201U34 w6w6 7K5432FOR23 w7w7 8S w8w w9w w

41 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT41A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT41 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w w0w w1w w2w2 3T w3w3 4H20123Y56 w4w4 5ATTENTION! w5w5 6N43201U34 w6w6 7K5432FOR23 w7w7 8S w8w w9w w


Download ppt "Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)"

Similar presentations


Ads by Google