Presentation is loading. Please wait.

Presentation is loading. Please wait.

Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)

Similar presentations


Presentation on theme: "Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)"— Presentation transcript:

1 Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)

2 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT2 2 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali

3 Freedom of movement must be encouraged LM scores A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT3 3 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali LM scores TM scores ReoM scores

4 career paths …while ensuring that Freedom of movement must be encouraged LM scores A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT4 4 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali LM scores TM scores ReoM scores

5 LM scores A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT5 5 PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Freedom of movement must be encouraged while ensuring that career paths … LM scores TM scores ReoM scores

6 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT6 6 Reordering Models Reordering Models E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores Many solutions have been proposed with different reo. classes, features, train modes etc. Tillman 04, Zens & Ney 06 AlOnaizan & Papineni 06 Galley & Manning 08 Green & al.10, Feng & al.10 …

7 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT7 7 Reordering Models Reordering Models E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores No matter what reordering model is used, permutation search space must be limited!  The power of all reordering models is bound to the reordering constraints in use Tillman04, Zens&Ney06 AlOnaizan & Papineni06 Galley & Manning08 Green &al.10, Feng &al.10 … Many solutions have been proposed with different reo. classes, features, train modes etc. Tillman 04, Zens & Ney 06 AlOnaizan & Papineni 06 Galley & Manning 08 Green & al.10, Feng & al.10 …

8 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT8 8 E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores

9 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT9 9 Reordering Constraints E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali #perm.=11!≈40,000,000 ReoM scores

10 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT10A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT10 E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Source-to-Source distortion #perm.=11!≈40,000,000 D(x,y)=|y-x-1| w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432 Reordering Constraints

11 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT11A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT11 E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Source-to-Source distortion #perm.=11!≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432 DL: distortion limit Reordering Constraints

12 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT12A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT12 The problem with DL… Arabic-English AR EN AR EN w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432

13 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT13A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT13 German-English DE EN DE EN w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432 The problem with DL…

14 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT14A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT14 Source-to-Source distortion w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432 Current solution: increase the DLimit #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000

15 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT15A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT15 Source-to-Source distortion w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432 Current solution: increase the DLimit Generally leads to worse translations! #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 DL=7  #perm.≈7,000,000

16 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT16A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT16 Source-to-Source distortion #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 DL=7  #perm.≈7,000,000 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3201234567 w3w3 4320123456 w4w4 5432012345 w5w5 6543201234 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432 Our solution:

17 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT17A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT17 Source-to-Source distortion #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3  #perm.≈7,000 DL=7  #perm.≈7,000,000 DL=3 & modif(D)  #perm.≈20,000 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012340078 w2w2 3201230067 w3w3 4320123456 w4w4 5432012345 w5w5 6543201230 w6w6 7654320123 w7w7 8765432012 w8w8 9876543201 w9w9 982254320 w 10 111098765432 Our solution: modify distortion for each test sentence Simplifies the task of reordering models!

18 18 Rest of the talk: How to modify the distortion matrix? What effect on translation quality? What effect on baseline runtimes? A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT

19 19A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT19 Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules Shallow syntax chunking: cheaper and easier than deep parsing constrains reorderings in a softer way Fuzzy (non-determinisic) reordering rules: generate N permutations for each matching sequence final reordering decision is taken during translation, guided by all SMT models (reoM, LM...) Few rules for language pair, to only capture long reordering

20 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT20A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT20 Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades

21 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT21A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT21 Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 PC 5 CC 1 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules

22 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT22A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT22 Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 PC 5 CC 1 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules

23 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT23A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT23 CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 NC 4 PC 5 CC 1 PC 5 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules Reordering selection Reordered source LM 0.9 0.4 0.1 0.7

24 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT24A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT24 CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Pct 6 w- $Ark fy AltZAhrp E$rAt AlmslHyn mn AlktA}b. and took part in the march dozens of militants from the Brigades Chunk-based fuzzy reordering rules Chunk-based fuzzy reordering rules Reordering selection Reordered source LM 0.9 0.7 0.4 0.1 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1

25 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT25A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT25 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 01234567 VC 2 w1w1 20123456 PC 3 w2w2 32012345 w3w3 43201234 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 76543201 w7w7 87654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

26 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT26A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT26 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00034567 VC 2 w1w1 20123456 PC 3 w2w2 32012345 w3w3 43201234 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 76543201 w7w7 87654320 Pct 6 w8w8 98765432

27 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT27A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT27 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00034567 VC 2 w1w1 20123456 PC 3 w2w2 32012345 w3w3 42201234 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 76543201 w7w7 87654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

28 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT28A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT28 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00034567 VC 2 w1w1 20100456 PC 3 w2w2 32012345 w3w3 42201234 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 76543201 w7w7 87654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

29 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT29A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT29 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00000567 VC 2 w1w1 20100456 PC 3 w2w2 32012345 w3w3 42201234 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 76543201 w7w7 87654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

30 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT30A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT30 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00000567 VC 2 w1w1 20100456 PC 3 w2w2 32012345 w3w3 42201234 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 72543201 w7w7 82654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

31 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT31A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT31 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00000567 VC 2 w1w1 20100456 PC 3 w2w2 32012340 w3w3 42201230 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 72543201 w7w7 82654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

32 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT32A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT32 Modifying the distortion matrix CC 1 VC 2 PC 3 NC 4 PC 5 Pct 6 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 012345678 CC 1 w0w0 00000567 VC 2 w1w1 20100456 PC 3 w2w2 32012340 w3w3 42201230 NC 4 w4w4 54320123 w5w5 65432012 PC 5 w6w6 72543201 w7w7 82654320 Pct 6 w8w8 98765432 CC 1 VC 2 PC 3 NC 4 PC 5 VC 2 PC 3 Reorderings to encode in the distortion matrix NC 4 PC 5 CC 1 Pct 6

33 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT33A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT33 Experiments Tasks: NIST-MT09 for Ar-En, WMT10 for De-En Systems based on Moses, include state-of-the-art hierarchical lexicalized reordering models [Tillmann 04; Koehn & al 05; Galley & Manning 08] Baseline Distortion Limits: 5 in Ar-En, 10 in De-En Evaluation by: - BLEU for lexical match & local order - KRS for global order

34 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT34A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT34 Arabic-English: Test set: eval09- NW *Distortion modified with 3-best reorderings per rule-matching sequence

35 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT35A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT35 Arabic-English: Test set: eval09- NW Distortion modified with 3-best reorderings per rule-matching sequence Translation Quality Translation Time +0.9 BLEU +0.6 KRS (signif.)

36 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT36A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT36 German-English: Test set: newstest10 *Distortion modified with 3-best reorderings per rule-matching sequence

37 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT37A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT37 German-English: Test set: newstest10 Distortion modified with 3-best reorderings per rule-matching sequence Translation Quality Translation Time +0.4 BLEU +0.7 KRS (signif.)

38 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT38A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT38 Conclusions Modified distortion allows for finer & linguistically motivated definition of search space We achieve better translation & faster decoding in language pairs where long reordering concentrates on few patterns Our method is complementary to reordering modeling For now, few reordering rules are needed to modify distortion We are currently working on a fully data-driven approach to replace the rules

39 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT39A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT39 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3T01234567 w3w3 4H20123Y56 w4w4 5ATTENTION! w5w5 6N43201U34 w6w6 7K5432FOR23 w7w7 8S65432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432

40 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT40A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT40 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3T01234567 w3w3 4H20123Y56 w4w4 5ATTENTION! w5w5 6N43201U34 w6w6 7K5432FOR23 w7w7 8S65432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432

41 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT41A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT41 w0w0 w1w1 w2w2 w3w3 w4w4 w5w5 w6w6 w7w7 w8w8 w9w9 w 10 012345678910 w0w0 0123456789 w1w1 2012345678 w2w2 3T01234567 w3w3 4H20123Y56 w4w4 5ATTENTION! w5w5 6N43201U34 w6w6 7K5432FOR23 w7w7 8S65432012 w8w8 9876543201 w9w9 987654320 w 10 111098765432


Download ppt "Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)"

Similar presentations


Ads by Google