1. Imposing Constraints from the Source Tree on ITG Constraints for SMT Hirofumi Yamamoto, Hideo Okuma, Eiichiro Sumita National Institute of Information and Communications Technology ATR Spoken Language Communication Research Labs. Kindai University School of Science and Engineering Department of Information

2. Background In current SMT, erroneous word reordering is one of the most serious problems, especially for dis- similar language pair such as English-Chinese or English-Japanese. 1) To introduce linguistic syntax directly. Not robust to parsing error Tree-to-string String-to-tree Tree-to-tree

3. Background In current SMT, erroneous word reordering is one of the most serious problems, especially for not similar language pair such as English-Chinese or English-Japanese. 2) To assign probabilistic constraints for word reordering Weaker constraints than the first type To introduce syntax information to second type IBM distortion, Lexical reordering, ITG

4. ITG Constraints Translation source sentences are represented by binary tree. Translation target sentences can be generated by rotating branches of nodes of source tree. BADC dbca BADC acbd Above target word order cannot be generated from any source binary tree. Source binary tree instance is not considered.

5. Basic Idea of IST-ITG To use ITG constraints under the given source tree BADC BADC abcd, abdc, bacd, badc, cdad, cdba, dcab, dcba abcd, bacd, cabd, cbad, dabc, dbac, dcab, dcba In original ITG constraints, 22 combinations are allowed.

6. The Number of Word Order Combinations For binary source tree, word order combinations are allowed without constraints. Under the IST-ITG constraints, this number is reduced to. If Without constraints ITG constraints IST-ITG If Without constraints ITG constraints IST-ITG

7. Extension to Non-binary Tree Parsing results sometimes are not binary tree. For the nodes which have more than two branches, any word reorderings are allowed. BADC abcd, abdc, acbd, acdb, adbc, adcb, bcda, bdca, cbda, cdba, dbca, dcba

8. Extension to Non-binary Tree Parsing results sometimes are not binary tree. For the node which have more than two branches, any word reorderings are allowed. For non-binary tree, the number of combinations of IST-ITG can represented by. ( represents number of branches in -th node)

9. IST-ITG in Phrase-based SMT (1) × The unit of parsing tree is “word”, but the unit of phrase-based SMT is “phrase”. Units are different. Additional rules for phrase-based SMT 1) Word reordering that breaks a phrase is not allowed. 2) Phrase internal word reordering is not checked. ○ Word-to-word alignments are sometimes not one-to-one. But phrase-to-phrase alignments are always one-to-one

10. IST-ITG in Phrase-based SMT (2) EFG 23 A Ph BCD 1 4 5 1:NG 2:NG 3:OK 4:NG 5:OK (unacceptable)

11. IST-ITG in Phrase-based SMT (2) EFG 23 A Ph BCD 1 4 5 1:NG 2:NG 3:OK 4:NG 5:OK Ph

12. IST-ITG in Phrase-based SMT (2) EFG 23 A Ph BCD 1 4 5 1:NG 2:NG 3:OK 4:NG 5:OK Ph

13. IST-ITG in Phrase-based SMT (2) EFG 23 A Ph BCD 1 4 5 1:NG 2:NG 3:OK 4:NG 5:OK

14. IST-ITG in Phrase-based SMT (2) EFG 23 A Ph BCD 1 4 5 1:NG 2:NG 3:OK 4:NG 5:OK Ph

15. IST-ITG in Phrase-based SMT (2) EFG 23 A Ph BC D 1 4 5 1:NG 2:NG 3:OK 4:NG 5:OK

16. Decoding Algorithm with IST-ITG EFGA BCD 1000 001 1 2 0 2 0 0:Untranslated 1 ： Translated 2 ： Translating d e HI 00 0

17. Decoding Algorithm with IST-ITG EFGA BCD 1001 101 1 2 2 NG 0 HI 00 0 If phrases A and B are translated, Sub-tree that includes more than two “2”  NG d e a b

18. Decoding Algorithm with IST-ITG EFGA BCD 1000 001 1 2 0 2 0 HI 00 0 Consider minimum Translating sub-tree (sub-tree that includes both “0” and “1”.) d e

19. Decoding Algorithm with IST-ITG EFGA BCD 1110 001 1 1 0 2 1 HI 10 2 All of minimum Translating sub-tree are translated.  OK d e f g h

20. Decoding Algorithm with IST-ITG EFGA BCD 1010 001 1 2 0 2 2 HI 00 0 Translate sub-part of minimum Translating sub-tree.  OK d e g

21. English and Japanese Patent Corpus Experiments # of sent. Total Words # of entry E/J Train E/J Dev E/J Eval Experimental corpus size 1.8M 916 899 60M/64M 30K/32K 29K/32K 188K/118K 4,072/3,646 3,967/3,682 Single reference

22. Other Experimental Conditions LM training: SRI Language model toolkit (5-grams) Word alignment for TM training: GIZA++ Decoder: Moses compatible in-house decoder named CleopATRa Evaluation measures BLEU,NIST,WER,PER

23. English and Japanese Patent Translation Experimental Results IBM+Lex IBM+Lex+IST BLEUNISTWERPER 31.17 32.20 7.50 7.61 76.30 71.18 38.61 38.15 English-to-Japanese IST-ITG30.267.41 74.90 38.93 Monotone 24.91 6.95 79.97 40.02 No Constraint 26.83 7.19 81.10 39.52 IBM 28.34 7.29 78.35 39.25

24. English and Japanese Patent Translation Experimental Results IBM+Lex IBM+Lex+IST BLEUNISTWERPER 31.17 32.20 7.50 7.61 76.30 71.18 38.61 38.15 English-to-Japanese IST-ITG30.267.41 74.90 38.93 Monotone 24.91 6.95 79.97 40.02 No Constraint 26.83 7.19 81.10 39.52 IBM 28.34 7.29 78.35 39.25

25. English and Japanese Patent Translation Experimental Results IBM+Lex +IST-ITG BLEUNISTWERPER 29.93 29.77 7.54 7.50 77.27 72.80 39.12 39.73 Japanese-to-English

26. English and Japanese Patent Translation Experimental Results IBM+Lex +IST-ITG BLEUNISTWERPER 29.93 29.77 7.54 7.50 77.27 72.80 39.12 39.73 Japanese-to-English

27. Chinese-to-English Translation Experiments NIST MT08 English-to-Chinese track IBM+Lex +IST-ITG W-BleuC-BleuWERCER 21.0 23.2 35.2 37.0 75.0 69.7 74.1 67.9 Experimental Results Training data for TM Training data for LM Development data Evaluation data 6.2M 20.1M 1,664 1,859 1 reference 4 reference

28. Conclusion We proposed new word reordering constrains IST-ITG using source tree structure. It is extension of ITG constraints. We conducted three experiments of proposed method: E-J and J-E patent translation and NIST MT08 E-C track. In all experiments, improvements of BLEU and WER are confirmed. Especially, improvement for WER is very large, and effectiveness for global word reordering is confirmed.

29. Thank you!